report-performance-analysis
Skill7 starsChanged 53 days agoMITView on GitHub
Use this skill whenever the user asks whether a Roboclaws live-agent report run is faster, wants to compare Agent SDK/Codex/Claude cleanup reports, asks about model-call work, residual latency, speedup validity, or report performance under provider/network variance. The skill extracts sanitized JSON metrics first and answers only from generated metric/comparison artifacts, not from report.html scraping or raw prompts.
In this repository8 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- eval-evolutionSkill
- eval-harnessSkill
- household-worldSkill
- raw-fpv-visual-labelerSkill
- scene-gaussian-map-alignmentSkill
- visual-result-showcaseSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 8
Use this skill to analyze Roboclaws live-agent report performance honestly under unstable provider and network conditions.
- Distinctive: line 15
Run fixed scripts before answering:
- Distinctive: line 26
Treat single-run output as diagnostic unless an explicit baseline or manifest is supplied.
- Distinctive: line 32
Call out apples-to-oranges baselines, missing telemetry, uncalibrated normalized model time, and high model/SDK residual latency.
- Distinctive: line 80
Use a compact answer:
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Report Performance Analysis
Use this skill to analyze Roboclaws live-agent report performance honestly under unstable provider and network conditions.
Workflow
- Read
references/metric-contract.md. - Identify the run directory or comparison manifest.
- Run fixed scripts before answering:
- One run:
../../scripts/reports/extract_live_report_metrics.py <run_dir> - Two runs or manifest:
../../scripts/reports/compare_live_report_metrics.py ... - Calibration data:
../../scripts/reports/calibrate_model_latency.py ...
- One run:
- Answer from generated JSON packets. Do not scrape
report.html, raw prompt text, model text, full tool payloads, private evaluator truth, or compact continuation state.
Interpretation Rules
- Lead with quality and effect before wall time.
- Treat single-run output as diagnostic unless an explicit baseline or manifest is supplied.
- Missing token, duration, image, or calibration fields are unavailable, not zero.
- A faster candidate is not a speed win when quality regressed unless the comparison packet records an explicit waiver.
- Call out apples-to-oranges baselines, missing telemetry, uncalibrated normalized model time, and high model/SDK residual latency.
Common Commands
Extract one run:
.venv/bin/python scripts/reports/extract_live_report_metrics.py \
output/path/to/seed-7 \
--output output/report-performance-analysis/metrics.json \
--write-model-call-metricsCompare two runs:
.venv/bin/python scripts/reports/compare_live_report_metrics.py \
--baseline-run-dir output/baseline/seed-7 \
--candidate-run-dir output/candidate/seed-7 \
--output output/report-performance-analysis/comparison.jsonCompare with explicit normalized timing:
.venv/bin/python scripts/reports/calibrate_model_latency.py \
--output output/report-performance-analysis/calibration.json \
output/baseline/seed-7/model_call_metrics.jsonl \
output/candidate/seed-7/model_call_metrics.jsonl
.venv/bin/python scripts/reports/compare_live_report_metrics.py \
--baseline-run-dir output/baseline/seed-7 \
--candidate-run-dir output/candidate/seed-7 \
--calibration output/report-performance-analysis/calibration.json \
--output output/report-performance-analysis/normalized-comparison.jsonCompare a manifest:
.venv/bin/python scripts/reports/compare_live_report_metrics.py \
--manifest skills/report-performance-analysis/templates/comparison-manifest.json \
--output output/report-performance-analysis/comparison.jsonAnswer Shape
Use a compact answer:
Verdict: <accepted speed win | diagnostic only | rejected | blocked>
Quality: <same/better/regressed and why>
Work: <model calls, tool calls, uncached/output/reasoning tokens>
Timing: <wall time, model API time, estimated model work availability, residual>
Limitations: <missing telemetry/calibration/apples-to-oranges/live gate>
Evidence: <metric JSON path or comparison JSON path>Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

