agentleFS
zirui-songzirui-song/claude-skills/skills/spec-curve-sweep/SKILL.md

spec-curve-sweep

Skill17 starsChanged 3 months agoMITView on GitHub

Use to map where a headline result lives and dies across the full space of defensible specifications — enumerates researcher degrees of freedom, runs a 200-500 spec grid in the background with checkpointing and resume, then produces a specification curve, a variance-decomposition table, a note on where the result flips, and a compiled online appendix. Trigger phrases include "specification curve", "overnight robustness sweep", "run every defensible spec", "how fragile is this result".

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read the primary estimation do-file and list every knob actually available:

    line 13
  2. Distinctive:

    Run detached; report the job ID.

    line 40

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Spec Curve Sweep

The heavy sibling of robustness-battery. Use when the user wants the whole space mapped, not five variants.

1. Enumerate researcher degrees of freedom

Read the primary estimation do-file and list every knob actually available:

  • Clustering level (firm, industry, industry × year, two-way)
  • Fixed-effect sets
  • Control vectors (none / baseline / extended)
  • Sample filters (rating bands, domestic-only, pre-COVID, balanced panel)
  • Winsorization thresholds (none / 1% / 5%)
  • Alternative outcome and treatment measures (fixed-pre vs. contemporaneous, levels vs. excess)
  • Event-window lengths

2. Propose the grid before running it

Target 200–500 specifications. Present the cross-product and the exclusions: name every combination ruled out as indefensible and say why (e.g., FE set that mechanically absorbs the treatment, sample cut that leaves < 30 clusters). The user approves the grid before it runs.

3. Build the runner

  • A parameterized driver do-file taking the knobs as arguments.
  • A Python runner that executes the grid with parallelism, checkpoints each completed spec to disk immediately, and is resumable — a killed run restarts from the checkpoint, not from spec 1.
  • One row per spec: all knob settings, coef, se, t, p, n, n_clusters, status.
  • Failed specs are recorded with their Stata error code, never dropped.

4. Launch in the background and monitor

Run detached; report the job ID. Check progress periodically rather than blocking.

5. Deliverables

  1. Specification curve figure — specs sorted by coefficient, significance shading, the headline spec marked.
  2. Variance-decomposition table — which knob explains the most variation in the estimate.
  3. A short note naming the specifications where the result flips sign or loses significance.
  4. Compiled online-appendix .tex containing the full grid.
  5. A PR with everything, plus a summary of what a hostile referee would seize on.

Honesty rule

Report fragility straight. Do not bury, footnote, or aggregate away the variants that hurt the result — those are the ones the user needs before a referee finds them. If the result only survives in a minority of defensible specs, that is the headline of the report.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.