spec-curve-sweep
Skill17 starsChanged 3 months agoMITView on GitHub
Use to map where a headline result lives and dies across the full space of defensible specifications — enumerates researcher degrees of freedom, runs a 200-500 spec grid in the background with checkpointing and resume, then produces a specification curve, a variance-decomposition table, a note on where the result flips, and a compiled online appendix. Trigger phrases include "specification curve", "overnight robustness sweep", "run every defensible spec", "how fragile is this result".
In this repository14 other files it gives its agents.
- codex-bridgeSkill
- codex-review-planSkill
- overleafSkill
- plan-debateSkill
- power-firstSkill
- referee-panelSkill
- refresh-pipelineSkill
- review-codex-planSkill
- robustness-batterySkill
- stata-preflightSkill
- sync-overleafSkill
- tableSkill
- verify-claimsSkill
- verify-numbers-pipelineSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 13
Read the primary estimation do-file and list every knob actually available:
- Distinctive: line 40
Run detached; report the job ID.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Spec Curve Sweep
The heavy sibling of robustness-battery. Use when the user wants the whole space mapped,
not five variants.
1. Enumerate researcher degrees of freedom
Read the primary estimation do-file and list every knob actually available:
- Clustering level (firm, industry, industry × year, two-way)
- Fixed-effect sets
- Control vectors (none / baseline / extended)
- Sample filters (rating bands, domestic-only, pre-COVID, balanced panel)
- Winsorization thresholds (none / 1% / 5%)
- Alternative outcome and treatment measures (fixed-pre vs. contemporaneous, levels vs. excess)
- Event-window lengths
2. Propose the grid before running it
Target 200–500 specifications. Present the cross-product and the exclusions: name every combination ruled out as indefensible and say why (e.g., FE set that mechanically absorbs the treatment, sample cut that leaves < 30 clusters). The user approves the grid before it runs.
3. Build the runner
- A parameterized driver do-file taking the knobs as arguments.
- A Python runner that executes the grid with parallelism, checkpoints each completed spec to disk immediately, and is resumable — a killed run restarts from the checkpoint, not from spec 1.
- One row per spec: all knob settings,
coef,se,t,p,n,n_clusters,status. - Failed specs are recorded with their Stata error code, never dropped.
4. Launch in the background and monitor
Run detached; report the job ID. Check progress periodically rather than blocking.
5. Deliverables
- Specification curve figure — specs sorted by coefficient, significance shading, the headline spec marked.
- Variance-decomposition table — which knob explains the most variation in the estimate.
- A short note naming the specifications where the result flips sign or loses significance.
- Compiled online-appendix
.texcontaining the full grid. - A PR with everything, plus a summary of what a hostile referee would seize on.
Honesty rule
Report fragility straight. Do not bury, footnote, or aggregate away the variants that hurt the result — those are the ones the user needs before a referee finds them. If the result only survives in a minority of defensible specs, that is the headline of the report.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

