Scenario design

A scenario evaluates behavior with typed inputs, measured checks, and saved evidence. Dispatch captures its output and pins images and source; use research to ground native measurement APIs.

Author the unit

import antioch

logger = antioch.Logger("task")


@antioch.scenario(tags=["smoke"], cases=[antioch.case({"seed": 2}, id="seed-2")])
def evaluate(run: antioch.ScenarioRun, seed: int = 1) -> None:
    """Evaluate the requested behavior."""
    # Build, step, and measure the task here.

The first parameter is run: antioch.ScenarioRun. Remaining parameters need scalar annotations (bool, int, float, str, or Literal) and defaults; antioch.param(default, ge=..., le=..., description=...) adds bounds and documentation. Use Literal for a closed set of choices. The parameter names, types, defaults, bounds, and descriptions appear during collection so callers can inspect the contract before running it.

The runner starts Kit before the body. Declare startup with config=antioch.SimulationConfig(...), not a startup call inside the scenario; config=None skips runner-owned Kit but still uses remote compute under CLI dispatch. Keep simulator imports inside functions; discovery runs locally without Isaac.

Keep module imports free of simulation, network, or file-writing effects. antioch scenario collect --json validates definitions, not the body; scenario_paths in the manifest limits discovery; without it, discovery scans the project. antioch.scenarios() reads definitions already registered in the current process, whereas client collection discovers and expands project source.

Inputs and execution policy

antioch.case accepts parameter overrides, a Cartesian grid, or correlated combinations; use helper objects, not bare dictionaries, in cases=. IDs can format resolved parameters; omitted IDs are derived. Expansion is capped at 2,000 cases per scenario. Use cases for independent outcomes and comparisons, or an internal loop for one evaluation/dataset lifecycle; suite selectors group cases in YAML.

cases = [
    antioch.case({"speed": 0.2}, id="slow", tags=["smoke"]),
    antioch.case(grid={"speed": [0.2, 0.4], "seed": [1, 2]}, id="speed-{speed}-seed-{seed}"),
    antioch.case(combinations=[{"speed": 0.2, "load_kg": 20.0}, {"speed": 0.4, "load_kg": 5.0}]),
]

A grid crosses independent values; combinations preserve intentional pairs. Shared overrides can accompany either expansion, but a parameter cannot be set both in the shared overrides and the expansion. Case tags combine with scenario tags for selection. Keep IDs stable enough to compare the same case across revisions.

Decorator keywords: name, description, tags, cases, config, blueprint, capture, service, profile, restart_services, and kit_cache.

kit_cache names a folder, such as a workflow task's volume, that each run loads Isaac Sim's Kit cache from before Isaac Sim starts and saves it to when it passes, so its first frames skip compiling what an earlier run did. Plain code calls antioch.load_kit_cache(path) before antioch.start_simulation() and antioch.save_kit_cache(path) once it has rendered.

service selects an eligible active container; it does not activate a profile. Omit it only when the active graph has one eligible service. profile includes helper services in the dispatched revision; restart_services restarts their processes and waits for fresh health before each managed run. Direct calls apply neither policy. For development with those helpers, or to target a session explicitly, start the required graph with antioch session new --profile perception. A submitted bundle reaches every mapped service, but a service outside restart_services keeps its existing process. If its files differ from those that process started with, the run reports earlier service code. Restart that helper between runs or add it to this policy. See manifest services and restart behavior.

Measured verdicts

run.add_result("max_error_m", max_error_m)
run.add_result("error_limit_m", error_limit_m)
run.check("position", max_error_m <= error_limit_m, detail=f"{max_error_m:.4f} m")
run.add_artifact("outputs/measurements.json", description="Measured task trajectory")

Use one named check per task criterion, with measured detail. Checks return the supplied boolean and do not stop execution; reusing a criterion replaces its verdict, so accumulate failures or worst-case values for conditions that must hold throughout motion. Geometric clearance is not measured contact, and a commanded grasp is not a measured lift; label proxies as proxies.

On normal completion, any failed check gives FAILED; all passing checks or no checks give PASSED, so no checks proves only that execution completed. run.fail and assertion failures stop the body with FAILED; run.skip reports an unmet precondition; run.set_outcome overrides checks, but unexpected exceptions still produce ERRORED.

Keep thresholds and summaries in JSON results and do not write the reserved checks key. run.add_results({...}) stores several results together; values must be finite, JSON-compatible values, so convert arrays and tensors before storing a small summary. Keep large arrays, tables, images, and files in artifacts instead. Artifact names are unique within a run; use name= when files share a basename. output and telemetry are reserved artifact names. An artifact uploads when add_artifact is called; keep many small files in one archive to avoid an upload for each.

run.params, case_id, tags, scenario_run_id, suite_run_id, suite, and suite_position describe the current evaluation. wall_s and sim_s distinguish elapsed wall time from simulation time; sim_s can be absent without a simulator, and a loop that steps Newton or MuJoCo-Warp without Kit sets it with run.set_sim_time(seconds) (telemetry). Long loops should call run.raise_if_cancelled() so a requested cancellation can stop useful work promptly; it reads a flag refreshed in the background and never waits on the network, so calling it every step is fine. In-session recording explains why a direct call cannot rely on the platform killing its parent process.

Telemetry and read-back

A module-scope antioch.Logger writes into whichever recording is active when it is called, a scenario's or a script's antioch.Telemetry, beside native rr.log calls. Use scalar for metrics, image for raw RGB/RGBA review frames, and value for Rerun archetypes; image calls take a path and then pixels: logger.image("camera/front", rgb). Read telemetry for image encoding, timelines, geometry, layouts, and diagnosing RRDs.

After execution, inspect the checks, results, and relevant artifacts through run history. Smooth video alone does not establish physical correctness.

For imported scenario calls or notebook recording inside a session, read in-session recording. Native simulator APIs belong to Isaac Sim, Isaac Lab, and research.

Choose the next reference