engine-sanity
Skill82 starsChanged 3 months agoMITView on GitHub
The mandatory pre-flight contract checks (Stage S0) that must pass before trusting NexusTrade's certification engine for any strategy work. Use at the start of a bakeoff or certification campaign to prove the walk-forward engine runs end-to-end, windows don't leak, fold winners persist, and the engine doesn't fabricate values for dead/not-yet-listed names. Each check has a STOP-and-report failure mode; these runs do NOT count toward any certification minimum.
In this repository23 other files it gives its agents.
- skills/alt-data-indicators/AGENTS.mdAGENTS.md
- skills/breadth-audit/AGENTS.mdAGENTS.md
- skills/bug-protocol/AGENTS.mdAGENTS.md
- skills/deploy-gate/AGENTS.mdAGENTS.md
- skills/engine-sanity/AGENTS.mdAGENTS.md
- skills/lockbox-holdout/AGENTS.mdAGENTS.md
- skills/options-structure-rules/AGENTS.mdAGENTS.md
- skills/portfolio-certification/AGENTS.mdAGENTS.md
- skills/run-episode/AGENTS.mdAGENTS.md
- skills/strategy-bakeoff/AGENTS.mdAGENTS.md
- skills/sweep-reoptimization/AGENTS.mdAGENTS.md
- skills/walk-forward-oos/AGENTS.mdAGENTS.md
- alt-data-indicatorsSkill
- breadth-auditSkill
- bug-protocolSkill
- deploy-gateSkill
- lockbox-holdoutSkill
- options-structure-rulesSkill
- portfolio-certificationSkill
- run-episodeSkill
- strategy-bakeoffSkill
- sweep-reoptimizationSkill
- walk-forward-oosSkill
The file
Files
Engine Sanity Checks (Stage S0)
MANDATORY; blocks everything. Four contract checks that prove the certification engine is trustworthy before any design work. Each has a STOP-and-report failure mode — a failure here means the engine itself is broken (bug-protocol), and no downstream number can be trusted. These S0 runs (and S1 baseline scoring) explicitly DO NOT COUNT toward the certification-body minimum — they are contract checks and benchmarks, not candidate certifications.
The four checks
- Walk-forward sweep smoke.
create_portfolioa trivial book (e.g.SPY > SMA50). Runrun_walk_forward_studywithmode:"validation",fold_count:2,engine_kind:"sweep", onegene_intent× 3 values,certification:false. It must start and complete with per-fold OOS + aggregate. Proves: the certification engine actually runs end-to-end. - Window fidelity (one fold). Backtest the materialized fold winner on its validation and OOS
windows; the fold's
validationStatistics/oosStatisticsmust match standalone backtests within rounding. Confirm disjointness: train∩validation, train∩OOS, validation∩OOS all empty, and OOS starts after validation ends. Proves: the walk-forward windowing isn't leaking. - Persistence.
get_walk_forward_study_resultsreturns a realselectedChatPortfolioIdfor every completed fold. Proves: fold winners materialize as real objects you can inspect/deploy. - Dead-name handling. Tiny 2022-window backtest on the full universe including a name not yet listed then (e.g. SNDK). Pass: the name is absent, or appears in a not-yet-listed reject list with no fabricated price/indicator values. Fail: the engine errors, or returns any non-null reading for a name that didn't trade. Proves: the engine doesn't fabricate data on not-yet-listed names.
On failure
Any failed S0 check → STOP, file the bug-protocol hand-off doc, and do not proceed to design work. A broken engine invalidates everything downstream; there is no point certifying strategies on it.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

