agentleFS
JaxsonmanJaxsonman/Crucible/skills/run/SKILL.md

run

Skill0 starsChanged 36 days agoMITView on GitHub

Run a Crucible benchmark for a task against a matrix of model configurations, in the background, then summarise the results. Spends usage, so it confirms first.

45 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use doctor, plan and 4 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • MicrosoftSkill

    doctor

    Diagnose an Amplifier CLI environment through the CLI doctor agent. USE WHEN a user requests CLI runtime, settings or missing-capability inspection. DO NOT USE for project bugs or other hosts.

    Also in 3 projects that use this

  • BlockSkill

    plan

    Draft a TDD, PRD, or lightweight plan in the conversation without committing it.

    Also in 4 projects that use this

  • VercelSkill

    setup

    Set up Vercel CLI and project configuration. Use when the user says "set up Vercel", "configure Vercel", "link to Vercel", or "vercel init".

    Also in 5 projects that use this

  • AnthropicSkill

    status

    Case status summary by audience — client-facing (plain language), internal (for the professor), or court-ready (formal caption format per local rules). Same facts, different framing and depth. Use when a student needs to update the client, brief the professor, or prepare a court status report.

    Also in 3 projects that use this

  • Hugging FaceSkill

    implement

    Make tests pass. Invoke after /write-tests produces failing tests.

    Also in 4 projects that use this

  • Microsoft AzureSkill

    verify

    End-to-end app testing pipeline for browser-based web apps on cloud browsers via Azure Playwright Workspaces: verify the live app, author tests for working flows, heal failing tests, then re-run with video and produce one consolidated report. Three phases (app verification, test authoring, test healing) plus a scoped final sanity run; each phase emits a sub-report. Use when asked to "verify my app", "make sure my app works and has good test coverage", "do a full QA pass", "add tests where I'm missing coverage", "fix failing tests and add new ones", "smoke test before deploy", "QA my deployment", or any combined verify-and-test ask. Requires Azure CLI (`az login`) and a Playwright Workspace resource ID — bootstraps configs on first invocation. Browser-based web apps with Playwright only — not for unit/API tests, native mobile/desktop, load/perf, or full accessibility audits.

    Also in 4 projects that use this

The file

We split this skill into the rules it gives an agent

4 rules. 0 are common, 1 is distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 41 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Preview: Run the same command plus --dry-run (it costs nothing).

    line 19
  2. Very specific:

    When the task has a check and --judge is on, say that the judge is advisory: the check decides success and the report shows judge …

    line 23
  3. Distinctive:

    Confirm: Ask the user whether to start.

    line 27
  4. Very specific:

    Run in the background: Run the real command (without --dry-run) with the Bash tool's run_in_background: true, from the repo root.

    line 29

“Common” is measured: kept by at least 6 in 10 of the 41 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Run a benchmark

Arguments: $ARGUMENTS. The first word is the task id (a directory under .crucible/tasks/); the rest are crucible run flags. The CLI is the only thing that runs attempts; never run attempts yourself.

  1. Resolve the command. Build crucible run .crucible/tasks/<id> <flags>. If the user gave neither --matrix nor --configs, use --matrix default. Pass --repeat, --judge-config and --max-budget-usd through only if the user gave them. Never raise --max-budget-usd on your own. Do not add --judge yet; step 2 decides.
  2. Preview. Run the same command plus --dry-run (it costs nothing). Its check: line says whether the task has a check. Add --judge only when it says check: none or the user asked for judging, and if you add it, run the dry run again with it. Show the planned attempts and their count. Each attempt is a real claude run, and --judge adds a judge run per attempt. When the task has a check and --judge is on, say that the judge is advisory: the check decides success and the report shows judge disagreements beside it. State the per-attempt budget cap from the dry run's max budget per attempt line (the flag, else the task's max_budget_usd, else 5.0) so the user sees the worst case.
  3. Confirm. Ask the user whether to start. Do not run anything paid until they say yes. If they decline or change flags, redo step 2.
  4. Run in the background. Run the real command (without --dry-run) with the Bash tool's run_in_background: true, from the repo root. Tell the user it started and that they can keep working; do not wait or poll in a loop.
  5. When it finishes (you are notified), read its output. If it exited non-zero, show the error and stop. Otherwise run crucible report <id> and summarise: which configurations passed, their time and cost, and the recommendation if there is one. Say plainly when the evidence is thin (few attempts, unknown results).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.