agentleFS
SergejDAGDASergejDAGDA/retrospective-audit-skill/skills/retrospective-audit/SKILL.md

retrospective-audit

Skill0 starsChanged 26 days agoMITView on GitHub

Audit an Agent Skill against real past Codex Desktop task history. Use when the user asks to backtest, retrospectively audit, evaluate, calibrate, improve, or compare a SKILL.md against how work was actually done across previous Codex tasks; when repeated manual corrections, repair loops, skipped steps, stale procedures, or process drift suggest the skill may no longer match real workflow. Work read-only until the user explicitly approves edits.

1 other project uses a version of this.

Adopted by

Most recently taken up by

  • Other versions1 project uses a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 1

    1. The01Geek/prflow★ 99same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

The file

Show its rules and text

We split this skill into the rules it gives an agent

14 rules. About 0 are common, 12 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈12

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Run a read-only retrospective audit first.

    line 10
  2. Very specific:

    Read the target SKILL.md completely.

    line 20
  3. Distinctive:

    Prefer full task history over titles, previews, summaries, or memory.

    line 22
  4. Distinctive:

    Stop for user agreement on the sample unless the user already gave a specific task list.

    line 28
  5. Distinctive:

    After approval, read the selected tasks fully, including paginated or older turns when available.

    line 30
  6. Distinctive:

    If task-history access is unavailable or incomplete, say so directly.

    line 37
  7. Distinctive:

    Use memory only to find candidate tasks, not to prove conclusions.

    line 43
  8. Distinctive:

    Treat manual actions outside Codex as invisible unless the history records them.

    line 45
  9. Distinctive:

    Do not call a deviation a skill defect by default.

    line 47
  10. Distinctive:

    Call a pattern durable only when it appears in at least three independent relevant tasks.

    line 53
  11. Distinctive:

    Keep this contract compact.

    line 72
  12. Very specific:

    Use the five-section report structure from references/report-template.md.

    line 92
  13. Distinctive:

    Stop and ask the user before:

    line 109
  14. Distinctive:

    If the user asks to proceed directly to edits, still summarize the evidence and patch plan first, then apply only the approved changes.

    line 118

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Retrospective Audit

Core rule

Run a read-only retrospective audit first. Do not edit the skill, project files, Git state, Codex tasks, or external systems unless the user gives a separate explicit approval after seeing the audit and patch plan.

This skill audits the skill process, not the project outcome. The goal is to find where written instructions diverge from real Codex task behavior and whether the divergence matters.

Required workflow

  1. Read the target SKILL.md completely.
  2. Discover relevant past Codex Desktop tasks using available task/thread tools. Prefer full task history over titles, previews, summaries, or memory.
  3. Present candidate tasks before deep reading:
    • title;
    • date, when available;
    • short relevance reason;
    • include/exclude recommendation.
  4. Stop for user agreement on the sample unless the user already gave a specific task list.
  5. After approval, read the selected tasks fully, including paginated or older turns when available.
  6. Decompose the target skill into a testable contract.
  7. Compare each selected task against that contract.
  8. Separate durable patterns from one-off exceptions.
  9. Produce recommendations and a patch plan only. Do not apply it.

If task-history access is unavailable or incomplete, say so directly. Do not infer missing evidence from project memory, summaries, titles, previews, or user expectations.

Evidence rules

  • Use memory only to find candidate tasks, not to prove conclusions.
  • Base conclusions on the actual task history whenever possible.
  • Treat manual actions outside Codex as invisible unless the history records them.
  • Do not call a deviation a skill defect by default. First evaluate whether it:
    • improved the result;
    • caused an error, rework, or repair loop;
    • was an intentional exception;
    • came from a task-specific constraint;
    • protects against a rare but expensive risk.
  • Call a pattern durable only when it appears in at least three independent relevant tasks.
  • Mark weaker signals as hypotheses or exceptions.

Skill contract decomposition

Extract these checkable elements from the target skill:

  • trigger conditions: when the skill should activate;
  • expected inputs: files, paths, approvals, task history, repo state, or user decisions it expects;
  • ordered workflow: required steps and sequencing;
  • checks and stop-lines: validations, approvals, read-only boundaries, and escalation points;
  • completion criteria: what counts as done;
  • outputs: expected files, reports, diffs, proposals, commits, or releases;
  • optional branches: conditions under which references, scripts, tools, or external systems are used.

Keep this contract compact. Quote only short fragments when needed.

Per-task comparison labels

For each selected task, mark contract items as:

  • followed;
  • skipped;
  • reordered;
  • repeated;
  • added outside the skill;
  • required manual correction;
  • caused rework or repair cycle;
  • impossible to determine from history.

Capture concise evidence for each non-trivial label: task title/date, relevant turn summary, and observed consequence.

Final report

Use the five-section report structure from references/report-template.md.

For every recommendation include:

  • concrete skill rule or fragment;
  • tasks supporting the finding;
  • observed evidence;
  • expected effect;
  • confidence: high, medium, or low;
  • proposed wording for the change.

End with a compact patch plan. The plan may name files and sections to change, but must not be applied during the audit turn.

Stop-lines

Stop and ask the user before:

  • reading a very large number of tasks when a smaller representative sample is available;
  • using external systems beyond Codex task history and local files;
  • changing any files;
  • committing, pushing, publishing, installing, or uninstalling anything;
  • treating fewer than three examples as a durable pattern.

If the user asks to proceed directly to edits, still summarize the evidence and patch plan first, then apply only the approved changes.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.