agentleFS
alexnodelandalexnodeland/hoist/.claude/skills/results/SKILL.md

results

Skill2 starsChanged 33 days agoMITView on GitHub

Write up an experiment's results in Hoist, updating together the dated results page in docs/results/, the index (docs/results/README.md), the claims ledger (claims.md), the paper (paper/hoist.md), the RFC-001 amendment, the roadmap and every page that quotes a changed number. Use after a replay, live run or analysis produces new numbers, when pre-registering a run, or when a number anywhere must change.

1 other project uses a version of this.

Adopted by

Most recently taken up by

  • Other versions1 project uses a version of this.

    Word for word · 0

    None found.

    Edited · 0

    None found.

    Wrote their own version · 1

    1. revnu-app/revnu-plugin★ 1same name

    How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.

  • In this repository5 other files it gives its agents.

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Every metric gets a definition, and a proxy says it is a proxy (AGENTS.md, "Honest reports").

    line 20
  2. Very specific:

    docs/results/claims.md: when a headline number is new or changed, its row (claim, number with interval, kind of evidence, what recomputes …

    line 25
  3. Distinctive:

    Keep the kind of evidence with every number: *live* (agents run with the speculator), *replay* (against τ²-bench's environment), *record* …

    line 35
  4. Very specific:

    Write as the pages do: calm, exact, short sentences, no hype (brand/messaging.md).

    line 37

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Results, claims and the pages that quote them

A number in Hoist lives in one place, the results page that recomputes it, and is quoted, with its scope, everywhere else. A round is written up in one commit, so that no page quotes a number another page has changed. Read the latest round first (for example git show --stat 5313707, the frontier round) and follow its shape.

Before the run: pre-register a live claim

A live claim for a new model family comes with a plan fixed in advance (RFC-001 §3.7): docs/results/<topic>-YYYY-MM-DD-plan.md, committed before the first episode, with the tasks, arms, trials, the comparison and the rule that decides whether a saving is established. docs/results/frontier-2026-09-27-plan.md is the model. Live runs spend credit: run them only when the owner asks (see CLAUDE.md, "Never").

The round's page: docs/results/<topic>-YYYY-MM-DD.md

  • A title that states the finding, an opening paragraph on the question, then ## Findings: bullets that each give a number with its scope (the model, the benchmark, live, replay or record, and the 95% interval).
  • ## Setup, then sections per model, domain or analysis; ## Deviations from the plan when there was one; ## Caveats (or "What these runs cannot show"); ## Published; and ## Reproduce, with the commands that recompute every number on the page.
  • Beside it: the rows (<topic>-YYYY-MM-DD.json), the episode archive (-episodes.tar.gz or .tar.xz), and any answer bundle (answers-YYYY-MM-DD-<topic>.jsonl.gz with its .md), so it replays without a key.
  • Publish only benchmark episodes, whose customers are synthetic, and leave out account data such as a subscription's usage windows. A real user's sessions are shared only through hoist redact (docs/privacy.md).
  • Every metric gets a definition, and a proxy says it is a proxy (AGENTS.md, "Honest reports").

In the same commit

  1. docs/results/README.md: a row under the round's date, | [Title](page.md) ([rows](...), [episodes](...)) | What it found |, in the order found.
  2. docs/results/claims.md: when a headline number is new or changed, its row (claim, number with interval, kind of evidence, what recomputes it), and the "What the evidence does not show" bullets it affects.
  3. paper/hoist.md: the abstract and contributions if a headline moved, the table and section in §4, and §6's limitations. Then rebuild the LaTeX manuscript in paper/latex/ (make, which needs pandoc, rsvg-convert and latexmk). If those tools are missing, say that the PDF was not rebuilt.
  4. docs/rfc/001-habit-compiler.md: a new ### 3.N Amendment K: <what the round found> (YYYY-MM-DD) after the last amendment, with a Details line linking the plan, the page and the paper's sections, bullets of what was run and found, and "What this changes": the RFC sections it moves, and Next.
  5. docs/roadmap.md: the evidence list, and what is still open.
  6. Every page that quotes a changed number: README.md ("Measured, with its scope"), brand/messaging.md (the key messages and the 150-word description), and the site: website/index.md, website/guide/why.md, website/guide/concepts/*.md. Find them with rg -n '<old number>'.
  7. The site's research pages: a page website/research/<topic>.md that includes the results page (<!--@include: ../../docs/results/<page>.md-->, with title and description front matter), its entry in the sidebar in website/.vitepress/config.mts, and a link in website/research/index.md. Then run make site.

Numbers

  • Copy numbers as the page computes them. 27.9% is not "almost 30%", and a replayed number never stands for a live one.
  • Keep the kind of evidence with every number: live (agents run with the speculator), replay (against τ²-bench's environment), record (from published trajectories alone, where detours are lower bounds).
  • An interval that includes no change means the saving is not established. Say so, as the frontier page says of airline.
  • Write as the pages do: calm, exact, short sentences, no hype (brand/messaging.md).

Check

make ci, and make site for the research pages. Then rg -n the old and new numbers across the repository, to find any page still quoting the old one.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.