paper-replicate
Skill0 starsChanged 3 months agoMITView on GitHub
Plan, request-authorize, implement, test, compare, and finalize a quantitative-paper portfolio backtest against a checksum-bound, metric-bearing replication target from extraction/reported.json. Use only when the finalized extraction declares at least one replication target with reported metric IDs; route theorem-only or formula-only papers to paper-theory-reconstruct. A direct user request to replicate authorizes uninterrupted execution without a second approval prompt. Supports exact, approximate, and illustrative metric-bearing backtests with hash-bound lineage, causality and accounting audits, metric-ID-native comparison, and immutable runs.
In this repository11 other files it gives its agents.
- paper-extractSkill
- paper-searchSkill
- paper-theory-reconstructSkill
- quant-paperSkill
- strategy-translateSkill
- paper-extractSkill
- paper-replicateSkill
- paper-searchSkill
- paper-theory-reconstructSkill
- quant-paperSkill
- strategy-translateSkill
The file
We split this skill into the rules it gives an agent
12 rules. About 0 are common, 9 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈9
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 8
Create one auditable run. Never mix plans, data, code, or results across run IDs.
- Distinctive: line 32
Run the deterministic eligibility preflight before creating a run:
- Very specific: line 39
If the finalized extraction has no metric-bearing target, stop and use paper-theory-reconstruct.
- Very specific: line 42
Write plan.md and v3 config.json.
- Very specific: line 56
When invoked by $quant-paper, pass --workflow-request <absolute-workflow.json> instead.
- Distinctive: line 70
Run the standardized gate:
- Distinctive: line 79
Only after the audit passes, run:
- Distinctive: line 108
Never tune parameters, dates, universes, definitions, or variants after seeing results.
- Distinctive: line 110
Never call proxy or survivorship-biased data exact.
- Distinctive: line 111
Never hide costs, financing, missing-data treatment, or definition mismatches.
- Distinctive: line 112
Never regenerate approval, audit results, comparisons, or finalized artifacts in place.
- Distinctive: line 113
Treat surprising agreement as an audit trigger.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Paper replicate
Create one auditable run. Never mix plans, data, code, or results across run IDs.
Resolve paths and contracts
Resolve the absolute workspace root and this skill's directory. Read references/plan_template.md, references/run_schema.md, and references/config.schema.json before Phase A. Read references/benchmark_template.md before Phase C. Every machine-readable artifact is checked against its bundled current JSON Schema as well as phase-specific lineage and semantic invariants.
papers/<paper-id>/replications/<run-id>/
├── plan.md
├── config.json
├── approval.json
├── audit_report.json
├── data/{prices.csv,manifest.json}
├── code/
│ ├── signal.py
│ ├── portfolio.py
│ ├── backtest.py
│ └── tests/
├── outputs/
└── run_manifest.jsonPhase A: plan and bind request authorization
- Run the deterministic eligibility preflight before creating a run:
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/check_eligibility.py" \
<paper-id> --root "$QPA_WORKSPACE_ROOT/papers"If the finalized extraction has no metric-bearing target, stop and use paper-theory-reconstruct; never synthesize a target or metric.
- Select one declared target ID and classify the run as
exact_replication,approximate_replication, orillustrative_reconstruction. - Write
plan.mdand v3config.json. Declare data vintage and bias policy; event timing and calendar; portfolio limits; costs and financing; metric-ID calculators, definition compatibility, and tolerances; audit tolerances and required paper-specific tests. - Validate the plan:
QPA_REPLICATE_SKILL_DIR="/absolute/path/to/paper-replicate"
QPA_WORKSPACE_ROOT="/absolute/path/to/workspace"
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/validate_run.py" \
<paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers" --phase plan \
--show-approval-material- Treat a direct user instruction to replicate a paper or declared target as authorization to execute the plan created for that request. Do not surface the hashes, halt, or ask for a second approval. A request only to review, diagnose, extract, or draft a plan is not execution authorization.
- Immediately record the request authorization with both
--expected-*-sha256values and--confirm-replication-request. The resultingapproval.jsonpreserves hash-bound lineage for the plan, config, extraction, source manifest, target, and classification without adding a conversational gate. Keep--confirm-user-approvedfor workflows where the user explicitly approves a separately presented plan.
When invoked by $quant-paper, pass --workflow-request <absolute-workflow.json> instead. The recorder verifies that the immutable workflow goal authorizes replication and binds approved_by to the workflow ID and request hash.
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/record_approval.py" \
<paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers" \
--expected-plan-sha256 <printed-plan-sha256> \
--expected-config-sha256 <printed-config-sha256> \
--confirm-replication-requestPhase B: implement, audit, and run
- Fetch or place approved data and write a v2 data manifest.
fetch_data.pysupports only approvedyfinanceconfigs, which cannot be exact replications. - Implement small
signal.py,portfolio.py, andbacktest.pymodules plus paper-specificunittesttests named inconfig.audit.required_custom_tests. - Run the standardized gate:
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/run_audit_tests.py" \
<paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers"It requires backtest.run() to return both decision_weights and lagged applied weights, then enforces the exact configured shift, price/weight/gross-return reconciliation, future perturbation causality, exposure limits, turnover and cost accounting, missing-data policy, timestamp alignment, and required custom tests. Any subsequent code, data, plan, config, or extraction change invalidates the report.
- Only after the audit passes, run:
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/run_backtest.py" <paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers"
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/plot_nav.py" <paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers"The harness computes costs and metrics. Replicated values are keyed by exact metric ID; outputs cannot be overwritten.
Phase C: compare, judge, and freeze
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/compare_metrics.py" <paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers"Author outputs/benchmark.md from the comparison, input comparability, diagnostics, ambiguities, and audit evidence. Use exactly one verdict: replicated, partially replicated, not replicated, or inconclusive. Numerical closeness alone is insufficient.
Finalize only after plots and benchmark are complete:
python3 "$QPA_REPLICATE_SKILL_DIR/scripts/finalize_run.py" <paper-id> <run-id> --root "$QPA_WORKSPACE_ROOT/papers"The resulting manifest freezes every replication input and output. Use a new run ID for any change.
Rules
- Require a finalized extraction with at least one metric-bearing replication target.
- Route formula, theorem, phase-boundary, and figure reconstructions without reported metrics to
paper-theory-reconstruct. - Never tune parameters, dates, universes, definitions, or variants after seeing results.
- Interpret the original direct replication request as authorization only for the bounded replication workflow; it does not authorize later scope expansion, external publication, or live trading.
- Never call proxy or survivorship-biased data exact.
- Never hide costs, financing, missing-data treatment, or definition mismatches.
- Never regenerate approval, audit results, comparisons, or finalized artifacts in place.
- Treat surprising agreement as an audit trigger.
- Author only the current
config.schema.jsoncontract; reject noncurrent run configurations and create a new reviewed plan and run ID.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

