verify-scoring-change
Skill735 starsChanged 35 days agoMITView on GitHub
Use when reviewing, verifying, or triaging a PR or proposed fix that changes a scorer, a metric, an answer-extraction or verdict-parsing path, or code that consumes a grader model's output. Also use when asked whether a scoring fix is safe to merge or whether reported numbers will move. Do NOT use for reviewing a whole evaluation against every standard (eval-quality-workflow) or for general PR review (ci-maintenance-workflow).
- Reads credentials
- Deletes or force-pushes
- Installs packages
In this repository18 other files it gives its agents.
- AGENTS.mdAGENTS.md
- build-repo-contextSkill
- check-trajectories-workflowSkill
- ci-maintenance-workflowSkill
- code-quality-fix-allSkill
- code-quality-review-allSkill
- create-evalSkill
- ensure-test-coverageSkill
- eval-quality-workflowSkill
- eval-report-workflowSkill
- eval-validity-reviewSkill
- generate-asset-actionsSkill
- investigate-datasetSkill
- prepare-releaseSkill
- prepare-submission-workflowSkill
- read-eval-logsSkill
- security-audit-evalSkill
- write-an-adrSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
39 rules. About 0 are common, 22 are distinctive, and 17 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈22
Distinctive · estimate
This author's own choices. Worth a look.
17
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 19
Every claim about how inspect_ai behaves must be checked against the installed pin before it goes in the verdict.
- Distinctive: line 33
Create a worktree on the PR head.
- Very specific: line 40
Check whether the eval has extras: grep -n "^<eval>" pyproject.toml under [project.optional-dependencies].
- Very specific: line 41
Run uv sync --group dev, adding --extra <name> for each match.
- Very specific: line 42
If the scorer imports a model provider module (for example inspect_ai.model._providers.openai), add --group test, which is where the …
- Distinctive: line 46
Create the artefact directory in the main checkout, outside the worktree.
- Distinctive: line 51
Write NOTES.md there as you go, and VERDICT.md at the end.
- Distinctive: line 52
Do not git add this directory.
- Very specific: line 55
Read the four non-secret settings from the main checkout's .env, and nothing else from it.
- Very specific: line 60
If INSPECT_LOG_DIR is unset, prior logs are in the main checkout's logs/.
- Distinctive: line 62
Read the PR body and any linked issue, then set them aside.
- Distinctive: line 101
Run the same probe on both trees from one worktree:
- Very specific: line 105
Keep any eval() log directory alive while you read the log.
- Very specific: line 113
Use a task's local_path override when it has one.
- Very specific: line 115
Use get_model("mockllm/openai/gpt-4o") when a proxied model name matters.
- Very specific: line 116
Read the @task signature before passing keyword arguments.
- Distinctive: line 118
If reproduction needs model API access you do not have, mark this UNVERIFIED.
- Distinctive: line 122
Use the same inputs. Record both outputs side by side.
- Distinctive: line 123
If the PR fixes a different bug from the one it describes, say which.
- Distinctive: line 135
Run one case in each direction yourself.
- Distinctive: line 139
When the fix is on the ground-truth side of a comparison, the dataset decides reachability.
- Distinctive: line 141
Before trusting any head comparison, re-score prior logs on the reverse-applied tree and check the result equals the stored scores.
- Very specific: line 145
If any live path moves a number: N in eval.yaml is bumped, and the README changelog and changelog.d/ fragment name which metrics move and …
- Very specific: line 146
If no live path moves a number, N is unchanged.
- Distinctive: line 149
Check for open PRs on the same eval:
- Very specific: line 153
Use git merge-tree --write-tree <this> <that> for a conflict dry run.
- Distinctive: line 162
When a parser gains tolerance for malformed input, probe the shapes it now accepts:
- Very specific: line 211
Read pytest's own summary line, and confirm one was printed.
- Distinctive: line 213
Read what each new test patches.
- Distinctive: line 217
When the revert breaks collection:
- Distinctive: line 219
If a new test module fails to import because the PR added a symbol, append a one-line placeholder for that symbol to the reverted file and …
- Distinctive: line 220
If the PR added a whole module, write a placeholder module that aliases the new names to the old scorer's behaviour and report both counts.
- Distinctive: line 227
Run on the PR head:
- Very specific: line 253
If the model under test was instructed to write the file being read, a missing or unparseable file at that path is the model's output …
- Distinctive: line 255
Read the eval's prompts before deciding whose failure it is.
- Very specific: line 296
Do not post to GitHub. The maintainer decides what goes on the thread.
- Distinctive: line 297
Do not fix the PR unless asked.
- Very specific: line 301
Write one bullet per item, each tagged with one of the four kinds, and write none under a kind that had nothing.
- Distinctive: line 304
If a framework behaviour you checked surprised you, or the tree corrected your first reading, it goes here.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Verify a Scoring Change
A scoring change moves reported numbers. The review has to establish three things: that the bug exists, that the fix removes it, and what else moves. The output is a verdict a maintainer can act on without repeating your work.
The seven questions to answer are the Scoring-Change Gate. The rules behind them are in BEST_PRACTICES.md. Read these four sections first:
- "Route sample-level failures to the right outcome"
- "Handle grader failures as instrument failures, not model failures"
- "Say what a metric does when its denominator is empty"
- "Dict-valued scores and metric registration"
The receipts rule
Every claim about how inspect_ai behaves must be checked against the installed pin before it goes in the verdict.
- Find the source with
uv run python -c "import inspect_ai, pathlib; print(pathlib.Path(inspect_ai.__file__).parent)". - Cite
file:line, or run a probe and paste its output. - A claim without a receipt is left out.
The same rule applies to the eval. Before judging, read its README changelog, its eval.yaml version, and git log --oneline -- src/inspect_evals/<eval>/<scorer file>. Whether a branch is reachable, whether a zero was a deliberate decision, and who produces the artefact being scored are answered by that history rather than by the diff.
Behaviours that have been cited wrongly from memory
references/framework-receipts.md lists the inspect_ai behaviours that reviewers have got wrong from memory, each with a file:line on the pin it was read on. Open it when a verdict is about to rely on how the framework treats empty inputs, nan, reducers, grader parse misses, redacted reasoning, sandbox exceptions, or mockllm. Line numbers drift between releases, so re-check any row you quote against the current pin.
Setup
Create a worktree on the PR head.
git fetch origin pull/<N>/head:pr-<N> && git worktree add .claude/worktrees/pr-<N> pr-<N>- Record the head SHA with
git rev-parse pr-<N>.
Sync the worktree so the code under review is what runs.
- Check whether the eval has extras:
grep -n "^<eval>" pyproject.tomlunder[project.optional-dependencies]. Extra names are not one-to-one with directories (bfclandbfcl_v4are separate extras), so read the matches. - Run
uv sync --group dev, adding--extra <name>for each match. A missing extra makes the scorer fail at construction and voids every probe run afterwards. - If the scorer imports a model provider module (for example
inspect_ai.model._providers.openai), add--group test, which is where the provider packages live. - An eval under
packages/has no root extra and its own lockfile. Run its tests through its tox environment (uv run --group dev tox -e <eval> -- <pytest args>; tox appends the arguments topytest, so a probe cannot go this route) and its probes withuv run --project packages/<eval> python probe.pyfrom the worktree root. For apackages/eval, receipts cite the isolated lock'sinspect_aipin. - Confirm
uv run python -c "import inspect_evals; print(inspect_evals.__file__)"prints a path inside the worktree. If it does not, nothing you run is testing the PR.
- Check whether the eval has extras:
Create the artefact directory in the main checkout, outside the worktree.
- Path:
agent_artefacts/<eval>_<version>/scoring_review_pr<N>/, using the version the PR head declares. - A PR that bumps more than one eval gets one directory per eval, with VERDICT.md in the first and a one-line pointer in the others.
- A PR to a shared helper that touches many evals and bumps none uses
agent_artefacts/utils_<helper>/scoring_review_pr<N>/, with no version component. - Write NOTES.md there as you go, and VERDICT.md at the end.
- Do not
git addthis directory. It is ignored onmain; a local checkout may lag. - The worktree directory is removed when the review ends. The verdict reaches the maintainer through the batch runner or, when asked, as a PR comment.
- Path:
Read the four non-secret settings from the main checkout's
.env, and nothing else from it.- The file holds live API keys. The only command to run against it is
grep -E '^(RUN_SLOW_TESTS|RUN_DATASET_DOWNLOAD_TESTS|INSPECT_LOG_DIR|INSPECT_EVAL_MODEL)=' .env; nothing else opens it. - pytest-dotenv reads
.envfrom a nested worktree.RUN_SLOW_TESTS=yesorRUN_DATASET_DOWNLOAD_TESTS=yesrun Docker and download tests that CI skips by default. Set them empty on the pytest command line (RUN_SLOW_TESTS= RUN_DATASET_DOWNLOAD_TESTS= ...) when you want CI's selection, and expect local counts to differ from the PR body's when Docker is present. INSPECT_EVAL_MODEL, if set, is loaded too. A scorer test that callsget_model()with no active model then fails building that provider instead of on its assertion. Setting it empty on the command line (INSPECT_EVAL_MODEL= ...) stops the provider build, butget_model()then raisesModel name '' should be in the format of <api_name>/<model_name>, where CI, with the variable absent, raisesNo model specified. pytest-dotenv does not overwrite a variable that is already set, so the absent state cannot be reproduced from the command line; a test that asserts on the message needsmonkeypatch.delenv.- If
INSPECT_LOG_DIRis unset, prior logs are in the main checkout'slogs/.
- The file holds live API keys. The only command to run against it is
- Read the PR body and any linked issue, then set them aside. Treat each claim in the description as an item to check.
Shell traps that have cost runs
- To run a specific uv version, for example the one the
uv==dev dependency pins, useuvx uv@<version> <command>. Write it out in full each time; zsh does not word-split a variable holding it. - A bare
=====separator is executed as a command. Useecho "---". - An unquoted
--include=*.pyfails on glob expansion. - Backticks inside a double-quoted grep pattern are command substitution.
- The pipeline status array is
pipestatus, notPIPESTATUS. $BASE:tests/x.pyapplies the:tmodifier. Write"${BASE}:tests/x.py".- A variable holding several paths reaches a command as one argument unless written
${=FILES}. import inspect_evals.<eval>.<module> as mbinds the package's re-exported function when__init__.pyexports a same-named symbol. Useimportlib.import_module.
Timeouts
Wrap every pytest run in a timeout so a hang is recorded instead of waited out. The command depends on the OS; check once with command -v timeout gtimeout.
- Linux:
timeout 600 <command>. - macOS with Homebrew coreutils (
brew install coreutils):gtimeout 600 <command>. - macOS without coreutils:
perl -e 'alarm shift; exec @ARGV or die "exec failed: $!\n"' 600 <command>. Theor diemakes a failedexecvisible, since a missing command inside a pipeline otherwise reads as exit 0.
The commands below are written with timeout 600; substitute the form for your platform. A run the timeout killed exits 124 under timeout or gtimeout and 142 under the perl form, and prints no pytest summary line.
The seven questions
1. Does the bug reproduce on main?
Reproduce the claimed failure in a run of the shipped task wiring, or in a test that exercises it. A stub of the code under review does not count as reproduction.
How to reproduce:
- For a metric:
SampleScoreobjects shaped like what the scorer emits, fed to the metric directly, or the task wiring withmockllmand a canned grader reply. - For a grader path: feed a reply with no parseable verdict and a raised API error, and record each score.
- For an extraction fix: run the scorer on the output the PR says was misparsed.
- For a type-guard fix: register a spy metric through
Task(metrics=[...])that records the type of every leaf it receives. The implicit epoch reducer converts leaves tofloatbefore list-registered metrics run, so a hand-builtintleaf is not a faithful input. - For a refactor: there is no bug to reproduce. Run a parity probe that computes old and new side by side on the reverse-applied tree, where both are importable.
- For a test-only PR: the source diff is empty. Run the merge-base test file against the head test file, across framework versions if the change is version-dependent.
- For a PR that ports an upstream scoring rule: replay upstream's published prediction files through the scorer objects of both trees, and fetch upstream's scorer to compare rules.
Run the same probe on both trees from one worktree:
- Reverse-apply the PR's source diff as in question 6, run the probe, restore.
- The reverse-apply and
generate_readmesboth change the worktree while probes import from it. Run each as one atomic command (apply, probe, restore) and nothing else in between. - Keep any
eval()log directory alive while you read the log. Samples load lazily and a removed temporary directory raisesFileNotFoundError.
Probe shapes that avoid downloads and keep the code under review in the path being probed:
- Patch the module's
hf_datasetto return aMemoryDatasetof hand-built samples around the task constructor. - Patch
datasets.load_datasetandinspect_ai.dataset._sources.hf.inspect_cache_dirinstead, when the framework's shuffle-before-map behaviour matters. - Patch the eval's download-or-ensure-data function to return a fixture directory.
- Pass a
mockllmModelas the task's grader parameter when it accepts one, withmax_samples=1so list outputs align with sample order. - Use a task's
local_pathoverride when it has one. - Rebind a sandbox-reading scorer module's
sandboxname to a fake with an asyncread_file. - Use
get_model("mockllm/openai/gpt-4o")when a proxied model name matters. - Read the
@tasksignature before passing keyword arguments.
If reproduction needs model API access you do not have, mark this UNVERIFIED.
2. Does it stop reproducing on the PR head?
- Use the same inputs. Record both outputs side by side.
- If the PR fixes a different bug from the one it describes, say which.
3. Do reported numbers move, and is that declared?
Decide reachability first:
- A leading
if not scoresguard is unreachable from the framework, so changing it moves nothing. - A denominator that empties inside the metric is reachable in a healthy run and does move numbers.
- A grader path on the default task configuration is live.
Measure how often and in which direction:
- Run one case in each direction yourself. A "can only decrease" claim has been wrong before.
- Sweep a list of completion shapes through the old and new comparison functions. A probe on stored samples alone has found only the upward moves, and the shapes the PR body lists have so far been the set where old and new rules coincide.
- For a metric change, run a seeded random-population sweep through the registered metric object on both trees.
- When a
returnbecomes araise, or a rule's reach depends on the shape of dataset records or schemas, run the new branch over the dataset and count how often it fires. A raise that hits 5% of a healthy run is a different change from the one described. - When the fix is on the ground-truth side of a comparison, the dataset decides reachability.
- For an extraction or parsing change the population is model completions. Look in
INSPECT_LOG_DIR(from the step 4 grep, elselogs/in the main checkout) for prior.evallogs of the eval. Readlog.eval.modelfrom each log header and keep only the eval's runs made with a non-mock model, by model and date; concurrent reviewers writemockllmruns to the same directory. Re-score those completions on both trees. - Before trusting any head comparison, re-score prior logs on the reverse-applied tree and check the result equals the stored scores. That proves the revert reproduces shipped behaviour.
Check the declaration:
- If any live path moves a number:
Nineval.yamlis bumped, and the README changelog andchangelog.d/fragment name which metrics move and in which direction. - If no live path moves a number,
Nis unchanged. Xis bumped only when the task interface changed, whether or not numbers move. See TASK_VERSIONING.md for both axes.
Check for open PRs on the same eval:
- Two PRs that both bump
4-Ato5-Amerge without a textual conflict and ship a duplicate version. gh pr list --search <path>is full-text search and most hits are false positives. Confirm each withgh pr view N --json files.- Use
git merge-tree --write-tree <this> <that>for a conflict dry run. It writes objects into the repository's object database and touches no worktree. gh pr view --json mergeablereturnsUNKNOWNuntil GitHub has computed it.
4. Does every exclusion sentinel mean one thing?
- List every path that yields
nan,Score.unscored(), or an omitted key, and write down what each means. - "The grader failed" and "the grader was not consulted because the agent complied" are different facts. If both produce
nanin the same key, a metric that excludesnandrops the second group from its denominator with no record of having done so. - Apply the coherence rule. Model-attributed reasons (
invalid_response_format,refusal,no_response) belong on scored samples in the denominator. Instrument reasons (grader_failed,scoring_failed) belong on exclusions.
When a parser gains tolerance for malformed input, probe the shapes it now accepts:
- an unclosed tag or fence followed by bare text
- an opening fence with no closing fence (a truncated completion)
- a fence inside a string literal or docstring
- a closing tag or fence followed by prose
- a nested block
- a case variant
Tolerance that lets a refusal count as the harmful action is a new false-positive channel. It appeared in #2294 (a refusal after an unclosed tag), #2131 (a refusal typed inside reasoning) and #2326 (a truncated fence).
5. Is anything swallowed or fallen into?
Grep the changed files and their callees for:
except Exceptionfollowed by a constant returnelse 0.0orif not <list>: return 0.0in a function that divides== CORRECTon a grader result with no unscored branchmetadata={"reason":(the key that does not migrate toScore.reason, which exists from inspect_ai 0.3.261)- a bare
score.value["key"]in a list-registered metric
Then classify each hit:
- A zero that a test asserts on purpose is a decision and may stand.
- A zero that nothing asserts is a defect.
- A pre-existing defect is a blocker if the PR makes it reachable (for example, a new filter that can empty the list a pre-existing
else 0.0guards). Otherwise it is a follow-up.
6. Do tests pin both sides?
Revert the PR's own source changes against its merge base, then run the PR's tests and count the failures. That count is the number of behaviours pinned.
BASE=$(git merge-base HEAD origin/main)
test -z "$(git status --porcelain -- src/)" &&
git diff "$BASE" HEAD -- src/ | git apply -R &&
timeout 600 uv run pytest tests/<eval> -v --tb=line --color=no --continue-on-collection-errors
git checkout HEAD -- src/ && git clean -fd -- src/ # restore the PR head whatever happened aboveWhy this shape:
- The merge base rather than
main, so unrelated newer changes to the eval are not pulled in. - Reverse-apply rather than checking out a directory, so files the PR added are removed and changes to shared helpers such as
src/inspect_evals/utils/are included. - The
&&chain, so pytest runs only ifsrc/was clean and the reverse-apply succeeded.git applyis atomic: a leftover probe edit or a placeholder line makes it apply nothing and exit 1, and an unguarded pytest would then run against the PR head and report the head's results as the reverted tree's. The restore line runs either way. timeout 600is the Linux form. See Timeouts above for macOS.- Evals under
packages/use the tox command from question 7.
Reading the result:
- Read pytest's own summary line, and confirm one was printed. A run the timeout killed has no summary line, and pytest's output is colour-coded by default so a grep for
FAILEDmisses without--color=no. - Record the summary count, and also list each new test's result on the reverted tree. A fixture that happens to give the same value under both codes pins nothing.
- Read what each new test patches. A test that monkeypatches the function under review pins nothing and passes on both trees.
- Also list any pre-existing test that passes on the reverted tree and fails on the PR head. Authors often run only the file they edited.
- For a refactor the expected count is zero. Run each deleted test against the head from a copy of the merge-base file: an assertion failure is a behaviour change to report, a collection error means the test only named a moved symbol, and a pass means coverage was dropped for no reason. The parity probe from question 1 must pass.
When the revert breaks collection:
- If a new test module fails to import because the PR added a symbol, append a one-line placeholder for that symbol to the reverted file and rerun.
- If the PR added a whole module, write a placeholder module that aliases the new names to the old scorer's behaviour and report both counts.
- The revert covers
src/only, so a test the PR deleted stays deleted. Diff the eval's test files against the merge base for removed test functions, and run any removed test against the head from a copy of the merge-base file.
Then confirm a correct zero still reports 0.0. For a grader path the three cases are: parse miss, raised API error, genuine negative verdict.
7. Are the checks green?
Run on the PR head:
uv run ruff checkuv run ruff format --checkuv run mypy <changed files>uv run --group dev python tools/check_changelog.pyuv run --group dev python tools/generate_readmes.py --create-missing-readmes, thenuv run --group dev pre-commit run mdformat --all-filesanduv run --group dev pre-commit run markdownlint-fix --all-files. CI (checks.yml) runs these three in this order and fails on any resulting change anywhere in the tree.timeout 600 uv run pytest tests/<eval> --tb=line --color=no, oruv run --group dev tox -e <eval>for an eval underpackages/, whose tests are excluded from the root suite. This is the head run that question 6 compares against; run it once.
Reading the results:
- After the three README commands, the pass criterion is CI's:
git status --porcelainshows nothing anywhere in the tree. Runninggenerate_readmeswithout the two formatters leaves dozens of READMEs reformatted; that is the missing formatting step, not generator drift. A branch behindmaincan still show changes in other evals' READMEs from listing-derived content; record those as drift from the base and confirm they disappear after mergingorigin/mainbefore recording the check as green. Restore withgit checkout -- .oncegit statuslists only files these commands wrote. - A pytest run that collects no tests is recorded as not run. A fired timeout is recorded as a check that did not complete.
- Fork PRs get no CI until a maintainer approves it, so a local ruff or mypy is often the first anyone has run.
gh pr checksreports a cancelled review workflow asfail. Read the run's conclusion before recording a failed check.
Two mypy traps recur:
- A
Metric-typed value is a union and cannot be called. Narrow withcast(MetricProtocol, m). math.isnan(score.value)on theValueunion. Narrow withisinstance(score.value, float)first.
Common mistakes
Applying the instrument rule before finding out who writes the artefact.
- The words "sandbox", "file", and "JSON" describe a mechanism. They do not say who produced the content.
- If the model under test was instructed to write the file being read, a missing or unparseable file at that path is the model's output failing. That is a verdict (
Score(0.0), in the denominator) rather than an instrument failure. - The same applies to a judge that is the model under test grading itself.
- Read the eval's prompts before deciding whose failure it is. Attribution comes before the instrument rule.
Citing a framework behaviour from memory. Sound reasoning on a stale premise produces a confident wrong finding. See the receipts rule.
Missing a version collision between sibling PRs. See question 3.
Trusting the PR body's vocabulary claim. Grep for the constants in the code.
The verdict
## Verdict: MERGE-READY | NEEDS-WORK | UNVERIFIED
One sentence on what the PR does and whether it does it.
**Checks run:** one line per command with its result.
### Findings
1. **[REQUIRED|ADVISORY] <file>:<line>**: what is wrong, the fix, the receipt.
### Gate
| Q | Result | Receipt |
| --- | --- | --- |
| 1 Reproduces on main | yes/no/unverified | ... |
**Outcome-changing or mechanical:** one line.
**Sequencing:** sibling PRs, version collisions, dependencies. Omit if none.
### Learnings
- **skill:** a step in this skill that was missing, wrong, or unclear, and what you did instead.
- **gotcha:** an environment or tooling trap that cost time (a missing extra, a hang, a tool that failed).
- **framework:** an `inspect_ai` behaviour you checked on the pin that belongs in the receipts list, with its `file:line`.
- **eval:** something about this eval a future reviewer needs (who the judge is, where the fallback came from, which extra it needs).
**Reviewer:** model id, PR head SHA, git SHA of this file, inspect_ai version, date.Rules for the verdict:
- UNVERIFIED is the verdict whenever question 1 or 2 could not be run.
- End VERDICT.md with the same content as a fenced JSON block, so a batch runner can collect verdicts and learnings without parsing prose. Keys:
verdict,summary,checks_run[],findings[{severity,file,line,summary,fix,receipt}],gate[{question,result,receipt}],outcome_changing,sequencing,learnings[{kind,note,receipt}]withkindone ofskill,gotcha,frameworkandeval, andreviewer{model,pr_head,skill_sha,inspect_ai,date}. - Do not post to GitHub. The maintainer decides what goes on the thread. If asked to post, end with
*Posted by [Claude Code](https://claude.com/claude-code) on <maintainer>'s behalf.*and ping no one. - Do not fix the PR unless asked.
The Learnings section is how the skill improves:
- Write one bullet per item, each tagged with one of the four kinds, and write
noneunder a kind that had nothing. - Include the things that went wrong on the way to the verdict, not only the verdict.
- A maintainer reads the learnings from every run in a batch and folds the accepted ones into this file. A note here reaches the next reviewer; a note that stays in NOTES.md does not.
- If a framework behaviour you checked surprised you, or the tree corrected your first reading, it goes here. The maintainer decides what is folded into this file.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

