NVIDIAKnown publisherNVIDIA/omniperf/.agents/skills/benchmark-isaaclab/SKILL.md

benchmark-isaaclab

Skill34 starsChanged 7 months agoApache-2.0View on GitHub

Run Isaac Lab benchmark scripts and interpret their outputs. Covers RL training throughput, non-RL environment step FPS, camera/load/startup benchmarks, batch suites, parameter gotchas, output files, and JSON result structure. Use when the user asks to run or compare Isaac Lab benchmarks. NOT for RL convergence or policy-quality validation, profiling capture (use profiling), trace analysis (use nsys-analyze), or applying performance fixes (use perf-tuning).

In this repository12 other files it gives its agents.
- benchmark-isaacsimSkill
- diagnose-perfSkill
- install-isaaclabSkill
- install-isaacsimSkill
- install-profilersSkill
- kit-app-streaming-debugSkill
- nsys-analyzeSkill
- nvtx-pythonSkill
- perf-tuningSkill
- profiling-apiSkill
- profilingSkill
- tracy-memorySkill

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.

≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.

Common
Distinctive
Very specific

Very specific:
Do not patch Isaac Sim shutdown by default: If Tracy shutdown hangs after outputs are complete, use the scoped last-resort guidance in the …
line 20
Very specific:
All in scripts/benchmarks/.
line 24

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.

Files

SKILL.md

Isaac Lab Benchmarking

Parameter references may be outdated. Always verify with ./isaaclab.sh -p <script> --help. For profiling details (Tracy, Nsight), see the profiling skill. For installation, see the install-isaaclab skill.

Setup

See the install-isaaclab skill for installation (clone, conda env, Isaac Sim linking).

Before Running Any Benchmark

Use a WARM run for headline FPS/frametime — see the COLD/WARM/TRACY method in the profiling skill
Set CPU governor to performance — see perf-tuning skill
Do not patch Isaac Sim shutdown by default. If Tracy shutdown hangs after outputs are complete, use the scoped last-resort guidance in the profiling skill

Benchmark Scripts

All in scripts/benchmarks/. Run via ./isaaclab.sh -p scripts/benchmarks/<script>.py.

Script	What it measures	Key params
`benchmark_non_rl.py`	Environment step FPS (most common)	`--task`, `--num_envs`, `--num_frames`
`benchmark_rlgames.py`	RL-Games training throughput	`--task`, `--num_envs`, `--max_iterations`
`benchmark_rsl_rl.py`	RSL-RL training throughput	`--task`, `--num_envs`, `--max_iterations`
`benchmark_cameras.py`	Camera system FPS + autotune	`--num_tiled_cameras`, `--num_standard_cameras`, `--height`, `--width`, `--autotune`
`benchmark_load_robot.py`	Robot loading time	`--num_envs`, `--robot {anymal_d,h1,g1}`
`benchmark_startup.py`	App startup time profiling	`--task` (required), `--num_envs`, `--top_n`
`benchmark_lazy_export.py`	Lazy export/import speed	`--iterations`, `--tasks` (stdout only, no JSON backend)
`benchmark_view_comparison.py`	XformPrimView vs PhysX	`--num_envs`, `--num_iterations`, `--profile` (stdout/cProfile, no JSON backend)
`benchmark_xform_prim_view.py`	XformPrimView performance	`--num_envs`, `--num_iterations`, `--profile` (stdout/cProfile, no JSON backend)

Note: benchmark_lazy_export.py, benchmark_view_comparison.py, and benchmark_xform_prim_view.py do NOT support --benchmark_backend or --output_path. They output results to stdout. Use --profile (where available) to save cProfile .prof files.

Common params: --device, --enable_cameras, --benchmark_backend, --output_path, --distributed

Note: --headless is deprecated. Omit --viz for headless mode, or use --viz none.

Passing Kit args (for profiling, output control, etc.):

./isaaclab.sh -p scripts/benchmarks/benchmark_non_rl.py \
    --task=Isaac-Ant-Direct-v0 --viz none --num_envs=4096 \
    --kit_args "--/app/profilerBackend=tracy --/log/file=/tmp/kit.log"

Batch suites

bash scripts/benchmarks/run_training_benchmarks.sh    # RSL-RL, 500 iters
bash scripts/benchmarks/run_non_rl_benchmarks.sh      # non-RL, various env counts
bash scripts/benchmarks/run_physx_benchmarks.sh        # PhysX assets

Critical Gotcha: Parameter Format

Isaac Lab uses UNDERSCORES (standard argparse). Isaac Sim uses HYPHENS.

Isaac Lab:  --num_envs 4096  --num_frames 100  --enable_cameras
Isaac Sim:  --num-cameras 8  --num-gpus 1      --num-frames 600

Mixing them up is a common source of silent misconfiguration.

Common Tasks and Env Counts

Camera tasks (add --enable_cameras):

Isaac-Cartpole-RGB-Camera-Direct-v0: 512-4096 envs

Classic physics (4096 / 8192 / 16384 envs):

Isaac-Ant-Direct-v0, Isaac-Cartpole-Direct-v0, Isaac-Humanoid-Direct-v0

Locomotion (4096 envs):

Isaac-Velocity-Rough-Anymal-C-v0, Isaac-Velocity-Rough-H1-v0, Isaac-Velocity-Rough-G1-v0

Manipulation (128-8192 envs):

Isaac-Reach-Franka-v0, Isaac-Factory-GearMesh-Direct-v0

Output Files

benchmark_<type>_<task>_<timestamp>.json — main results file
kit.log — execution log (if --/log/file= is set via --kit_args)
*.tracy / *.nsys-rep — profiling traces (only with profiling args)

JSON structure

Array of phase objects, each with phase_name, measurements (list of {name, data, type, unit}), and metadata (list of {name, data, type}). Phases: benchmark_info, startup, runtime, hardware_info, version_info. RL benchmarks add a train phase.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

It worked
Partly
It didn't

What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free