scenarios
Skill81 starsChanged 3 days agoMITView on GitHub
End-to-end scenario tests in tests/scenarios/ — creating a scenario from a user's diagnostics download, running them, updating outputs.json snapshots, and reading a scenario diff. Use when a scenario test fails, when reproducing a user-reported optimization result, or when adding a regression scenario.
3 other projects use a version of this.
Adopted by
Most recently taken up by
- jantichy/claudetheir own version, first seen Oct 27, 2026
- Autonoma-AI/autonomatheir own version, first seen Oct 9, 2026
- envisioning/skillstheir own version, first seen Sep 22, 2026
Other versions3 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 3
- Autonoma-AI/autonoma★ 242same name
- jantichy/claude★ 52same name
- envisioning/skills★ 2same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository18 other files it gives its agents.
- AGENTS.mdAGENTS.md
- agent-setupSkill
- config-flowSkill
- coverageSkill
- documentationSkill
- elementsSkill
- explain-planSkill
- integrationSkill
- lintSkill
- manifestSkill
- modelSkill
- performanceSkill
- pythonSkill
- release-notesSkill
- testingSkill
- translationsSkill
- update-docsSkill
- update-versionSkill
The file
We split this skill into the rules it gives an agent
11 rules. 0 are common, 2 are distinctive, and 9 are very specific to this project.
0
Common
Most of the 3 edited versions we read keep these.
2
Distinctive
This author's own choices. Worth a look.
9
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 28
No registration is needed, but a folder whose name does not start with scenario, or that lacks outputs.json, is silently skipped.
- Distinctive: line 124
All datetime operations during the test see this frozen time.
- Very specific: line 160
Ask the user for Settings → Devices & Services → HAEO → Download Diagnostics
- Very specific: line 161
Create tests/scenarios/scenario_<name>/ and save the download as scenario_<name>/scenario.json
- Very specific: line 162
Run the scenario suite once — a session fixture splits the unified file into the four files, taking outputs.json from the outputs the …
- Very specific: line 163
Run it again: the split happens after collection, so the new scenario is only picked up on the second run, which compares HAEO's results …
- Very specific: line 164
If the results should replace the recorded outputs, rerun with --snapshot-update.
- Very specific: line 172
Add config.json, environment.json, and inputs.json
- Very specific: line 173
Add an outputs.json containing {} so discovery picks the folder up
- Very specific: line 174
Run uv run pytest -m scenario -k scenario_<name> --snapshot-update to fill outputs.json
- Distinctive: line 193
Look for a newly over-constraining constraint or a policy compilation change.
“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Scenario testing
Scenario tests are end-to-end integration tests with realistic configurations and time-frozen states.
Structure
tests/scenarios/
├── test_scenarios.py # Parametrized runner, one test per scenario folder
├── conftest.py # Fixtures, including the scenario.json splitter
├── syrupy_json_extension.py # Reads and writes outputs.json as the snapshot
├── visualization.py # Topology and dispatch plan SVGs
└── scenario*/
├── config.json # Hub and element configuration
├── environment.json # Captured timestamps; optimization_start_time freezes time
├── inputs.json # HA sensor states to inject
└── outputs.json # Expected sensor outputs (snapshot)Discovery
_discover_scenarios() in test_scenarios.py globs scenario*/ and keeps only folders that hold all four JSON files.
No registration is needed, but a folder whose name does not start with scenario, or that lacks outputs.json, is silently skipped.
Scenario files
config.json
The hub config entry's data with its element subentries under participants, keyed by element name, plus the entry's version and minor_version.
The test sets the entry up at that version, so Home Assistant migrates an older config to the current schema exactly as it would a user's.
A config without minor_version is taken as current and is not migrated.
A config in the current schema looks like this, with each element's fields in the same sections as its stored subentry:
{
"version": 1,
"minor_version": 4,
"common": {
"name": "Test Hub",
"horizon": {
"type": "preset",
"value": "3_days"
}
},
"advanced": {},
"participants": {
"Battery": {
"name": "Battery",
"element_type": "battery",
"connection": {
"type": "connection_target",
"value": "Inverter"
},
"storage": {
"capacity": {
"type": "entity",
"value": [
"sensor.battery_capacity"
]
}
}
}
}
}Copy an existing scenario's config.json rather than writing one from scratch, and keep the version and minor_version it carries.
A config from an older export, such as one with common.horizon_preset, must keep its older minor_version so it migrates.
environment.json
Captured runtime context. The optimization_start_time is used as the test's freeze timestamp, and the timezone and currency are applied to Home Assistant.
Diagnostics from before HAEO recorded the currency have none, so add the user's currency when creating a scenario from one:
{
"ha_version": "2024.1.0",
"haeo_version": "0.1.0",
"timezone": "UTC",
"currency": "AUD",
"diagnostic_request_time": "2024-01-15T12:00:00+00:00",
"diagnostic_target_time": null,
"optimization_start_time": "2024-01-15T12:00:00+00:00",
"optimization_end_time": "2024-01-15T12:00:00+00:00",
"horizon_start": "2024-01-15T12:00:00+00:00"
}inputs.json
Home Assistant sensor states to inject before optimization. Array of state objects with keys:
entity_id: Entity ID stringstate: State value as stringattributes: Object withunit_of_measurement,forecast, etc.
[
{
"entity_id": "sensor.solar_power",
"state": "3500",
"attributes": {
"unit_of_measurement": "W"
}
}
]outputs.json
Snapshot of expected sensor outputs after optimization. This file is generated and updated by the test runner.
Time freezing
The test runner extracts the timestamp from environment.json and uses freezegun for deterministic results.
All datetime operations during the test see this frozen time.
Running scenarios
Scenarios carry the scenario marker.
The test gate excludes them and the scenario gate runs them, both in CI and in uv run check:
# Run all scenarios
uv run check scenario
uv run pytest -m scenario
# Run specific scenario
uv run pytest -m scenario -k scenario1
# Update snapshots
uv run pytest -m scenario --snapshot-updateBuild the frontend card first
Scenario tests render topology SVGs by shelling out to the card's scenario export script.
Without a build, every scenario fails with Card export script not found, which reads like an optimizer regression and is not one:
npm --prefix frontend/haeo-forecast-card ci
npm --prefix frontend/haeo-forecast-card run buildCI does this before its scenario job, and the scenario gate of uv run check does it automatically.
Running scenarios by hand means doing it yourself once.
Creating new scenarios
The usual path is a user's diagnostics download, which becomes a regression test directly:
- Ask the user for Settings → Devices & Services → HAEO → Download Diagnostics
- Create
tests/scenarios/scenario_<name>/and save the download asscenario_<name>/scenario.json - Run the scenario suite once — a session fixture splits the unified file into the four files, taking
outputs.jsonfrom the outputs the diagnostics recorded, and deletes the original - Run it again: the split happens after collection, so the new scenario is only picked up on the second run, which compares HAEO's results with the recorded outputs
- If the results should replace the recorded outputs, rerun with
--snapshot-update; reviewoutputs.json, runuv run check --fixso prettier formats the split files, and commit
Name the folder scenario_<behavior> for the behavior under test rather than the next number in sequence.
The scenario prefix is required for discovery.
To hand-author a scenario instead:
- Create
tests/scenarios/scenario_<name>/ - Add
config.json,environment.json, andinputs.json - Add an
outputs.jsoncontaining{}so discovery picks the folder up - Run
uv run pytest -m scenario -k scenario_<name> --snapshot-updateto filloutputs.json - Review and commit the generated outputs
Snapshot format
Each scenario has its own outputs.json file capturing the sensor states after optimization.
Snapshots include state values, attributes, forecasts, and metadata.
Reading a scenario diff
A changed outputs.json means optimizer behavior changed.
Decide first whether that is intended — --snapshot-update is not a way to make a test pass.
- Values shift slightly everywhere: usually a solver tie-break or blend weight change. Confirm the objective value is unchanged before accepting.
- One element's sensors change: narrow to that adapter's
outputs()or its model element. - Shadow prices change but power does not: a constraint was added, removed, or reformulated. The primal solution is the same but the duals moved.
- Optimization status becomes failed: the model is infeasible. Look for a newly over-constraining constraint or a policy compilation change.
Debugging scenarios
Visualizations are regenerated into scenario*/visualizations/ on each run and are committed.
Open them to see the network topology and the dispatch plan.
To optimize a scenario without Home Assistant in the loop:
uv run diag --file tests/scenarios/scenario1/ # rerun the optimization
uv run diag --file tests/scenarios/scenario1/ --compare # stored vs recomputed, side by side
uv run diag --file tests/scenarios/scenario1/ -o # stored outputs onlyTo browse the same data in a live Home Assistant instance, use uv run sim scenario1.
See local simulation.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

