agentleFS
MiaoDXMiaoDX/roboclaws/skills/report-performance-analysis/SKILL.md

report-performance-analysis

Skill7 starsChanged 53 days agoMITView on GitHub

Use this skill whenever the user asks whether a Roboclaws live-agent report run is faster, wants to compare Agent SDK/Codex/Claude cleanup reports, asks about model-call work, residual latency, speedup validity, or report performance under provider/network variance. The skill extracts sanitized JSON metrics first and answers only from generated metric/comparison artifacts, not from report.html scraping or raw prompts.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 4 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use this skill to analyze Roboclaws live-agent report performance honestly under unstable provider and network conditions.

    line 8
  2. Distinctive:

    Run fixed scripts before answering:

    line 15
  3. Distinctive:

    Treat single-run output as diagnostic unless an explicit baseline or manifest is supplied.

    line 26
  4. Distinctive:

    Call out apples-to-oranges baselines, missing telemetry, uncalibrated normalized model time, and high model/SDK residual latency.

    line 32
  5. Distinctive:

    Use a compact answer:

    line 80

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Report Performance Analysis

Use this skill to analyze Roboclaws live-agent report performance honestly under unstable provider and network conditions.

Workflow

  1. Read references/metric-contract.md.
  2. Identify the run directory or comparison manifest.
  3. Run fixed scripts before answering:
    • One run: ../../scripts/reports/extract_live_report_metrics.py <run_dir>
    • Two runs or manifest: ../../scripts/reports/compare_live_report_metrics.py ...
    • Calibration data: ../../scripts/reports/calibrate_model_latency.py ...
  4. Answer from generated JSON packets. Do not scrape report.html, raw prompt text, model text, full tool payloads, private evaluator truth, or compact continuation state.

Interpretation Rules

  • Lead with quality and effect before wall time.
  • Treat single-run output as diagnostic unless an explicit baseline or manifest is supplied.
  • Missing token, duration, image, or calibration fields are unavailable, not zero.
  • A faster candidate is not a speed win when quality regressed unless the comparison packet records an explicit waiver.
  • Call out apples-to-oranges baselines, missing telemetry, uncalibrated normalized model time, and high model/SDK residual latency.

Common Commands

Extract one run:

.venv/bin/python scripts/reports/extract_live_report_metrics.py \
  output/path/to/seed-7 \
  --output output/report-performance-analysis/metrics.json \
  --write-model-call-metrics

Compare two runs:

.venv/bin/python scripts/reports/compare_live_report_metrics.py \
  --baseline-run-dir output/baseline/seed-7 \
  --candidate-run-dir output/candidate/seed-7 \
  --output output/report-performance-analysis/comparison.json

Compare with explicit normalized timing:

.venv/bin/python scripts/reports/calibrate_model_latency.py \
  --output output/report-performance-analysis/calibration.json \
  output/baseline/seed-7/model_call_metrics.jsonl \
  output/candidate/seed-7/model_call_metrics.jsonl

.venv/bin/python scripts/reports/compare_live_report_metrics.py \
  --baseline-run-dir output/baseline/seed-7 \
  --candidate-run-dir output/candidate/seed-7 \
  --calibration output/report-performance-analysis/calibration.json \
  --output output/report-performance-analysis/normalized-comparison.json

Compare a manifest:

.venv/bin/python scripts/reports/compare_live_report_metrics.py \
  --manifest skills/report-performance-analysis/templates/comparison-manifest.json \
  --output output/report-performance-analysis/comparison.json

Answer Shape

Use a compact answer:

Verdict: <accepted speed win | diagnostic only | rejected | blocked>
Quality: <same/better/regressed and why>
Work: <model calls, tool calls, uncached/output/reasoning tokens>
Timing: <wall time, model API time, estimated model work availability, residual>
Limitations: <missing telemetry/calibration/apples-to-oranges/live gate>
Evidence: <metric JSON path or comparison JSON path>

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.