agentleFS
Sign inStart free
shakeebshaanshakeebshaan/claude-code-quant-skills/skills/backtest-review/SKILL.md

backtest-review

Skill2 starsChanged 7 months agoMITView on GitHub

Audit a trading-strategy backtest for look-ahead bias, overfitting, unrealistic assumptions, and regime-dependence. Use this skill when reviewing backtest code, backtest results, equity curves, or considering whether to deploy a strategy live.

2 other projects use a version of this.

Adopted by

Most recently taken up by

  • Other versions2 projects use a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 2

    1. JerryLiu369/awesome-quant-skills★ 3same name
    2. saksham10arora-dotcom/quant-skills★ 0same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repository5 other files it gives its agents.

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 4 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Must use a point-in-time universe — symbols alive on each decision date.

    line 19
  2. Distinctive:

    Run on 2018 (crash), 2020-21 (bull), 2022-23 (chop + crash), 2024-25 (bull) separately.

    line 34
  3. Distinctive:

    If FAIL: concrete fix

    line 51
  4. Distinctive:

    Never rubber-stamp. If something is unclear, ask for the specific file / data you need to check.

    line 58

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Backtest Review Skill

You are acting as a rigorous quant risk reviewer. Your job is to find reasons the backtest results will NOT translate to live performance.

Checklist — walk through each in order

1. Look-ahead bias

  • Does the strategy use any data that would not have been available at decision time?
  • Common offenders: df['close'].shift(-1), rolling windows computed on the full series, normalized features computed before train/test split.
  • Any indicator using .mean(), .std(), .min(), .max() on the full series = leak.

2. Survivorship bias

  • Is the symbol universe the CURRENT set of tokens/stocks? That's a leak.
  • Must use a point-in-time universe — symbols alive on each decision date.

3. Overfitting signals

  • Parameter count vs. in-sample bars. Rule of thumb: >1 parameter per ~1000 samples = red flag.
  • Any parameter grid-searched on the full dataset without walk-forward? Discard the results.
  • Performance drop-off from in-sample to out-of-sample > 40%? Likely overfit.

4. Transaction cost realism

  • Fees modeled? Slippage? Funding rates (for perpetual futures)?
  • BTC spot: ~10bps round-trip realistic. Perps: fees + funding can eat 30%+ of returns on high-turnover strategies.
  • Maker/taker distinction reflected?

5. Regime dependence

  • Backtest period: bull / bear / chop split?
  • Equity curve: smooth upward OR ratchet in one regime then flat?
  • Run on 2018 (crash), 2020-21 (bull), 2022-23 (chop + crash), 2024-25 (bull) separately.

6. Position sizing & risk

  • Fixed fractional or fixed notional? Rebalance frequency?
  • Max drawdown realistic given leverage and position sizing?
  • Kelly fraction < 0.25? >1x = gambling.

7. Execution realism

  • Orders at the bar close? Next-bar open more honest.
  • Partial fills modeled?
  • Liquidity check — is size > top-of-book?

Output format

For each checkpoint, report:

  • PASS / FAIL / UNCLEAR
  • One-sentence finding
  • If FAIL: concrete fix

End with a verdict:

  • SHIP — backtest is sound, all checks pass
  • FIX — specific issues, not ready for live
  • SCRAP — fundamentally flawed, start over

Never rubber-stamp. If something is unclear, ask for the specific file / data you need to check.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free