agentleFS
Topaz1618Topaz1618/pawskills/skills/data-audit-reproducibility/SKILL.md

data-audit-reproducibility

Skill0 starsChanged 3 days agoMITView on GitHub

Raw data preservation, data source tracing, data processing audit, and historical execution reproducibility standard. Ensures humans can independently inspect the data and reproduce the results.

The file

Show its rules and text

We split this skill into the rules it gives an agent

16 rules. About 0 are common, 16 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈16

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every significant transformation is recorded.

    line 21
  2. Distinctive:

    Every output can be traced back to its input source.

    line 22
  3. Distinctive:

    When data processing is involved, the following process must be made explicit:

    line 28
  4. Distinctive:

    Only a statistical summary must not be stored.

    line 70
  5. Distinctive:

    Only Agent-summarized data must not be retained.

    line 71
  6. Distinctive:

    If complete preservation is impossible due to storage, privacy, or data volume limits, the limits, the truncation range, and an alternative …

    line 79
  7. Distinctive:

    Every piece of significant data should record at least:

    line 103
  8. Distinctive:

    If the source is unclear, it must be marked as unknown.

    line 117
  9. Distinctive:

    Every significant data transformation should record at least:

    line 121
  10. Distinctive:

    Must be able to reconcile:

    line 155
  11. Distinctive:

    When the actual business involves splitting, merging, or duplicate records, the counting criteria must be explicitly stated.

    line 159
  12. Distinctive:

    If modification is necessary:

    line 167
  13. Distinctive:

    When reproducing, prefer historical input data rather than re-collecting.

    line 194
  14. Distinctive:

    If external services, models, or third-party data have already changed, it must be clearly stated that the historical environment cannot be …

    line 196
  15. Distinctive:

    Must be able to answer:

    line 235
  16. Distinctive:

    When these cannot be answered, do not claim the data processing has complete audit capability.

    line 248

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Data Audit and Reproducibility Standard

I. Core Principle

Data must be independently inspectable by humans.

An Agent's analysis result cannot substitute for raw data.

Final output must be traceable to real inputs and the actual processing steps.

Must guarantee:

  1. Raw data has an actual storage location.
  2. Raw data has not been overwritten by processing results.
  3. Every significant transformation is recorded.
  4. Every output can be traced back to its input source.
  5. Historical executions can be identified and reproduced.
  6. Humans can inspect the data without relying on the Agent's summaries.

II. Data Lifecycle

When data processing is involved, the following process must be made explicit:

Raw data ↓ Data parsing ↓ Data normalization ↓ Data filtering ↓ Data cleaning ↓ Data transformation ↓ Data analysis ↓ Final output

The actual process may differ, but it must faithfully reflect the steps the program really executes.

Processing steps must not be fabricated in order to match the illustrative flow.

III. Raw Data Preservation

Raw data must be stored separately from processing results.

The scope includes:

  • Raw API responses.
  • Web collection results.
  • Raw JSON.
  • Raw CSV.
  • Raw HTML.
  • Raw database exports.
  • Raw files.
  • User-provided raw data.
  • Results returned by external tools.

Preservation requirements:

  1. Preserve the complete raw content as far as possible.
  2. Cleaned data must not overwrite raw data.
  3. Only a statistical summary must not be stored.
  4. Only Agent-summarized data must not be retained.
  5. Record the raw data storage location.
  6. Record the data generation time.
  7. Record the source.
  8. Record the file size.
  9. Record the data hash.
  10. Record whether the preservation is complete.

If complete preservation is impossible due to storage, privacy, or data volume limits, the limits, the truncation range, and an alternative storage location must be explicitly recorded.

Incomplete data must not be marked as complete data.

IV. Separation of Raw Data and Processed Data

The following directory organization is recommended; existing project directories may be used as they are:

data/ ├── raw/ │ └── raw inputs ├── intermediate/ │ └── intermediate processing results ├── processed/ │ └── cleaned and normalized results ├── output/ │ └── final output └── audit/ └── data processing records

Existing data must not be relocated repeatedly just to adopt this directory layout.

V. Data Source Records

Every piece of significant data should record at least:

  • Data name.
  • Data source.
  • Source file or interface.
  • Retrieval time.
  • Retrieval batch.
  • Generating module.
  • Raw file location.
  • File hash.
  • Data format.
  • Data version.
  • Data scale.

If the source is unclear, it must be marked as unknown; the source must not be guessed.

VI. Data Transformation Records

Every significant data transformation should record at least:

  • Execution ID.
  • Input file.
  • Input file hash.
  • Output file.
  • Output file hash.
  • Executing function.
  • Executing program version.
  • Parameters used.
  • Configuration used.
  • Transformation rules.
  • Input record count.
  • Output record count.
  • Excluded record count.
  • Exclusion reasons.
  • Execution time.
  • Execution status.

It must be possible to explain how the input data became the output data.

VII. Data Count Reconciliation

Significant processing stages should record count changes.

For example:

Raw records: 1000 Parsed successfully: 980 Parse failures: 20 Retained after filtering: 650 Excluded by filtering: 330 Final output: 650

Must be able to reconcile:

Raw count = successfully processed count + failure count

When the actual business involves splitting, merging, or duplicate records, the counting criteria must be explicitly stated.

Data must not be fabricated in order to satisfy count equality.

VIII. Data Modification and Integrity Protection

Raw data is in principle stored read-only.

If modification is necessary:

  • Retain the pre-modification version.
  • Generate a new version.
  • Record the reason for modification.
  • Record the modifying program.
  • Record the modification time.
  • Record the before and after hashes.

A hash is used to check whether a file has changed; it does not mean the data content itself is correct.

IX. Historical Execution Reproduction

Significant executions must preserve sufficient reproduction information:

  • run_id.
  • Input data version.
  • Input data hash.
  • Code version or commit ID.
  • Configuration file version.
  • Runtime parameters.
  • Dependency versions.
  • Execution environment.
  • Random seed (where applicable).
  • External service version or response snapshot (where applicable).
  • Output location.

When reproducing, prefer historical input data rather than re-collecting.

If external services, models, or third-party data have already changed, it must be clearly stated that the historical environment cannot be fully restored.

It must not be claimed that every execution can be reproduced one hundred percent.

X. Independent Human Audit

Humans must be able to directly inspect:

  1. Raw data.
  2. Data file structure.
  3. Real records.
  4. Processing parameters.
  5. Transformation rules.
  6. Filtering conditions.
  7. Excluded records.
  8. Final output.
  9. Execution logs.
  10. Data version and integrity information.

Humans must not be required to understand the data only through the Agent's explanation.

Provide direct access via CSV, JSON, database queries, or other means when necessary.

XI. Privacy and Security

Sensitive information must receive appropriate protection.

But the following must be distinguished:

  • Fields that need masking.
  • Fields that can be retained normally.
  • Fields that must not be persisted.
  • Fields that may only be retained for a short period.

Audit information for an entire event must not be deleted merely to protect a single sensitive field.

For content that cannot be saved, the necessary source, time, operation, and limitation notes must be retained.

XII. Acceptance Criteria

Must be able to answer:

  • Where is the raw data?
  • Where did the data come from?
  • Is the raw data complete?
  • Was the data modified?
  • What processing did each step undergo?
  • Which records were excluded?
  • Why were they excluded?
  • How was the final data generated?
  • Which execution produced this result?
  • Can it be re-executed using historical inputs?

When these cannot be answered, do not claim the data processing has complete audit capability.

XIII. Final Principle

Data is fact; the Agent's interpretation is opinion.

The fact itself must be preserved so that humans retain the ability to inspect and judge independently.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.