publish-results
Skill8 starsChanged 11 days agoMITView on GitHub
Required after any benchmark run that changes a public number, before any release PR merges, and for any edit to the README benchmark section, docs/compare.md or the website. It carries one result to every place a reader sees it, the README, docs, comparison, website, FAQ data and report, so no page is left stale. CI enforces it through tests/claims.test.mts.
In this repository3 other files it gives its agents.
- .claude/CLAUDE.mdCLAUDE.md
- growthSkill
- ship-changeSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 1 is distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 21
Add every new page that shows a number to surfaces.
- Very specific: line 22
When a claim stops being true, add its old wording to banned.
- Very specific: line 24
Run node --test tests/claims.test.mts.
- Very specific: line 43
Every number comes from committed rows in bench/results/.
- Distinctive: line 45
Keep the "where it does not win" section.
- Very specific: line 56
After the merge, wait for the pages workflow, then fetch the live site and confirm it shows the new numbers: curl -s …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Publish results everywhere
A result is not delivered until every reader sees the same numbers. One stale page makes the whole project look careless. This skill lists every place a result lives. tests/claims.test.mts fails CI when a page drifts from the rows or carries a stale claim, so a forgotten page cannot merge.
When to run it
- A benchmark run changes a number you publish: a cost, a pass count, a ratio, a competitor's result.
- A release PR is about to merge.
- A change alters what the docs promise: a default, a review rule, a model choice.
1. Update the source of truth
bench/results/headline.json holds every public number, the rows it comes from, the pages that must show it, and the stale phrases that must not appear.
- Point each entry at the committed rows and the arm, and set
shownto the value the pages print. - Add every new page that shows a number to
surfaces. - When a claim stops being true, add its old wording to
banned.
Run node --test tests/claims.test.mts. It recomputes each number from the rows, so a typo or a stale total fails here.
2. Update every surface
Change all of them in the same PR. Never one now and the rest later.
| Surface | What to update |
README.md | The table cell under the banner, the ## Benchmarks tables, "Where AutoMode does not win", the ## How it compares paragraph and any review or model claim |
docs/compare.md | The feature table's cost row, "Measured on the same sessions", "Where the others are ahead" and the switch section |
site/app/(home)/page.tsx | RESULTS (the chart widths come from it), the stat cards, #fit, the FAQ list, which also feeds the FAQ schema that search engines and AI assistants read |
site/content/docs/benchmarks/results.mdx, site/content/docs/compare.mdx | The tables, "Where AutoMode does not win" and the byline dates |
site/content/docs/guides/*.mdx, faq.mdx, concepts/prompt-cache.mdx | Any number, and any claim about reviews or models. The calculator's per-step costs live in site/components/calculator.tsx |
bench/experiments/<report>.md | The results section, the fixes and the limits |
marketing/posts/*.md | Every post that is not sent. tests/marketing.test.mts fails on a number that left the README or headline.json |
CHANGELOG.md | A line under [Unreleased] when the shipped behavior changed |
3. Keep it honest
- Every number comes from committed rows in
bench/results/. Never round in AutoMode's favor, and never invent a competitor result. Use our own run, or the project's published result, cited as theirs. - Lead with the most common usage: mixed sessions, and people who run Opus.
- Keep the "where it does not win" section. A claim without its limit is marketing, not evidence.
- Meta descriptions stay between 100 and 165 characters (
tests/site.test.mts).
4. Check, then verify live
node --test tests/claims.test.mts tests/site.test.mts
git ls-files '*.md' '*.html' | xargs grep -nE "<an old number or claim>"- Build the site (
npm run buildinsite/), servesite/out/under/AutoMode/, and look at it at desktop and phone widths. The site is dark only. - After the merge, wait for the
pagesworkflow, then fetch the live site and confirm it shows the new numbers:curl -s https://mralexivy.github.io/AutoMode/ | grep '<a new number>'.
Then follow ship-change for the issue, branch and PR.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

