claude-jev
AGENTS.md27 starsChanged 48 days agoMITView on GitHub
A Claude Code plugin. Five hooks send small judgments to TypeSafe's Jev, a System One model that returns typed answers instead of text. README.md explains what each hook decides and why. Read it before changing behavior. VISION.md is the acceptance policy for what this plugin becomes and refuses to become. Check a non-trivial change against its align/resist pair before building it, and cite it when resisting. - Hooks fail open. A failure never blocks or alters a session. Command hooks…
- Reads credentials
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 33,315 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository4 other files it gives its agents.
- entire-searchSkill
- entireSkill
- releaseSkill
- verify-claude-jevSkill
The file
We split this AGENTS.md into the rules it gives an agent
6 rules. About 0 are common, 2 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 64
Every other tunable is a module-level constant, or a userConfig field in .claude-plugin/plugin.json when the user sets it from /claude-jev …
- Distinctive: line 94
Keep that, because Claude Code runs them from arbitrary directories.
- Very specific: line 103
Write rules in this file as bullets, one instruction each.
- Very specific: line 113
Run each scripts/test_*.py with python3 (scripts/test_rules_scope.py, scripts/test_stats_afk.py, and any other), then npm run test:afk and …
- Distinctive: line 173
Every hook must exit 0 on a malformed or empty event.
- Very specific: line 176
Run a hand-fed hook event with CLAUDE_CONFIG_DIR="$(mktemp -d)".
≈ “Common” is an estimate: at least 1 in 5 of the 42,283 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
claude-jev
A Claude Code plugin. Five hooks send small judgments to TypeSafe's Jev, a
System One model that returns typed answers instead of text. README.md
explains what each hook decides and why. Read it before changing behavior.
Vision
VISION.md is the acceptance policy for what this plugin becomes and refuses to
become. Check a non-trivial change against its align/resist pair before
building it, and cite it when resisting.
Layout
| Path | What lives there |
scripts/jev.py | API client and CLI. Every other script imports it. |
scripts/prompt_router.py | Measurement reference for the routing eval; production runs src/prompt-router.ts under node type stripping |
scripts/rules.py | PostToolUse on edits, PreToolUse/PostToolUse on Bash to record shell writes, and Stop — rule enforcement |
hooks/register.ts | Experimental function-hooks module: session.compact -> node --experimental-strip-types src/compactor.ts rows, and the /claude-jev settings pane. A bridge, not a second implementation. |
plugin.json | Agent Plugins 1.0.0 identity. Claude Code still loads .claude-plugin/plugin.json and hooks/. |
src/ | The TypeScript hook implementations the plugin runs, executed directly as source under node type stripping (needs node >= 22.18) |
src/compact/strategy.ts | Shared compaction strategy: checks, thresholds, selectBlocks. src/compactor.ts is the Claude rows bridge. |
adapters/afk/ | The AFK host adapter: a TypeScript implementation of the same hooks with its own manifest, hooks.json, and README. Its known-gaps list is the contract — do not claim parity that table does not state. |
adapters/pi/ | The Pi adapter. package.json pi.extensions points at adapters/pi/jev.ts (session_before_compact and /jev). Selection imports src/compact/strategy.ts. Block shaping, the <read-files> index, the 14k+2k split, and Pi logs stay in the adapter. |
scripts/comparators.py | ast-grep lookups the rule hook adds to a judgment |
scripts/observed.py | Scores what a past turn actually did |
scripts/stats.py | The Stats row in /claude-jev, or python3 scripts/stats.py — scores live decisions from the three logs under ~/.claude: jev-router-log.jsonl (router, subagent, rules), jev-compact-log.jsonl, jev-calls.jsonl (every API call, written by jev.ask) |
scripts/check_no_comments.py | Hard ban: fails if scripts/, eval/, or hooks/ source has a # / // / /* */ comment |
eval/ | Offline measurement. See eval/README.md. |
.claude/skills/ | Maintainer skills for this repository, not shipped. release cuts a version: changelog section to bump, tag, push, GitHub release. |
hooks/hooks.json | Hook registration. New hook means an entry here. modules names the function-hooks module; older Claude Code ignores the key. |
Invariants
- Hooks fail open. A failure never blocks or alters a session. Command
hooks exit 0 and print nothing on any error, missing key, or timeout; keep
the
except Exception: returnat the top of every hookmain. Therowsbridge answers{"fallback": ...}on stdout and exits 0, andhooks/register.tsthen callsnext(e)so the built-in summary runs. - The one external program is ast-grep, pinned by version and sha256 in
scripts/comparators.py, fetched to~/.claude/jev-binby a detached process outside the hook's budget, and never required: every comparator answers""without it, and the judgment proceeds as before. - Standard library only, per host.
scripts/and the Python files ineval/are Python 3 standard library only: no third-party imports, andurllib.requestis the HTTP client. - The hooks in
src/, the shared modules inadapters/afk/src/shared/, andeval/compact.tsare TypeScript with zero runtime dependencies. They use Node's standard modules only;typescript,oxlint, and@types/nodeare devDependencies. The shipped plugin runs them as source under node type stripping, with no build step. hooks/register.tsholds no judgment. It runsnode --experimental-strip-types src/compactor.ts rows, falls through tonext(e)on failure, and draws the/claude-jevpane. Its debug-log line is thesummarystring the compactor sends, and the pane's key source, provider, and last call come frompython3 scripts/jev.py status.- One key variable per provider: each entry in
PROVIDERS(scripts/jev.py) names its URL, key prefix, and variable (TYPESAFE_API_KEY,OPENROUTER_API_KEY). A new provider is a new entry there; do not add any other variable. TheprovideruserConfigfield pins one; onautothe variables are read inPROVIDERSorder and the key's prefix picks. Every other tunable is a module-level constant, or auserConfigfield in.claude-plugin/plugin.jsonwhen the user sets it from/claude-jevor/config. Claude Code hands those fields to hooks asCLAUDE_PLUGIN_OPTION_<FIELD>, its own variables, read only throughjev.plugin_option();hooks/register.tspasses the saved key to the compactor under the same name.CLAUDE_CONFIG_DIRis Claude Code's own variable, not a plugin tunable:jev.config_dir()honors it and every path under the user's config directory goes through that helper, never through a literal~/.claude. - No code comments. Line, block, and JSDoc comments are banned in
scripts/,eval/, andhooks/source (.py,.ts,.js). Shebangs and LICENSE text stay. Python module/function docstrings are documentation strings, not comments, and stay where Conventions require them. Markdown underdocs/,README.md, andAGENTS.mdis prose, not code comments — leave it. Rationale for thresholds and constants (ACT,MIN_CONFIDENCE,KEEP_THRESHOLD, and the rest) lives in eval results, docs, or PR evidence — not inline comments. Changing a threshold without eval evidence is still a guess. Enforce withpython3 scripts/check_no_comments.py(exit 1 on any hit). scripts/observed.pyis the shared scorer.eval/replay.pyandscripts/stats.pyboth call it. Editing it moves every accuracy number inREADME.md.- Probes pay for live calls.
jev.askis assigned in exactly one place, thecached_askwiring ineval/rules_eval.py'scmd_run, which wraps the real client with the cache. Eval and hook verification never stubs, mocks, or monkeypatches the ask path — a stub proves wiring, not behavior — and cost is bounded by shrinking the sample, never by faking the client. Enforce withpython3 scripts/check_no_stubs.py(exit 1 on any.ask =assignment outside the sanctioned wiring). - Hook scripts import siblings through
sys.path.insert(0, dirname(__file__)). Keep that, because Claude Code runs them from arbitrary directories.
Conventions
- Each script's module docstring states the hook it serves, the fail-open contract, and the env var. Keep that shape when you add one.
- Question definitions live in
scripts/jev.py(intent_bundle,subagent_bundle) or next to the hook that asks them. Put the meaning in theinstructionsandcriteriatext — Jev never sees the key names. - Write rules in this file as bullets, one instruction each.
rules.pyparses instruction files bullet by bullet, and a rule buried inside a prose paragraph classifies poorly. - The user-facing entry point is the
/claude-jevpane inhooks/register.ts. There is noskills/orcommands/directory; do not add one. A new setting or report is a row in that pane. .agents/skills/is canonical for shared agent skills; symlink them into toolskills/dirs, never copy.
Verify
Run each scripts/test_*.py with python3 (scripts/test_rules_scope.py,
scripts/test_stats_afk.py, and any other), then npm run test:afk and
npm run test:pi. CI (.github/workflows/check.yml) runs those tests.
The local / script contract for this repo is:
python3 -m compileall -q scripts eval
python3 scripts/check_no_comments.py
python3 scripts/check_no_stubs.py
python3 scripts/check_versions.py
for f in scripts/test_*.py; do python3 "$f" || exit 1; done
npm run test:afk
npm run test:pi
echo '{"prompt":"hi","transcript_path":""}' | CLAUDE_CONFIG_DIR="$(mktemp -d)" python3 scripts/prompt_router.py; echo "exit=$?"compileall plus stdin hook smoke (exit 0, fail open) is what a default
cloud agent or Grok Bot run can claim. Ruff is a maintainer check, required
for a release and for CI, and not part of that python3-only claim:
ruff format --check .
ruff check .ruff.toml configures it. It is not a dependency file: no hook imports
Ruff.
The AFK adapter has its own contract, and claiming it requires running it:
cd adapters/afk && npm ci && npm run build
echo '{"session_id":"test","cwd":"'$(pwd)'"}' | node dist/session-start.js; echo "exit=$?"npm ci && npm run build is what a doc-only run can claim for
adapters/afk/; the second line is its no-key hook smoke, and the keyed
hooks are not part of any default claim. Live Claude Code session verification is out-of-band: it needs a
machine with claude and TYPESAFE_API_KEY, and is not claimed as proved
by those local checks alone.
For a scripted launch → doctor → drive → evidence → cleanup loop (isolated
HOME, feature map under .cursor/skills/verify-claude-jev/features/), use
the project-local Cursor skill:
.cursor/skills/verify-claude-jev/bin/control-jev launch
.cursor/skills/verify-claude-jev/bin/control-jev doctor
# then drive one feature from features/; evidence under
# .cursor/skills/verify-claude-jev/artifacts/<RUN_ID>/
.cursor/skills/verify-claude-jev/bin/control-jev cleanupSee .cursor/skills/verify-claude-jev/SKILL.md. Keep the map honest with
/maintain-verification-skill when hooks or skills change.
python3 scripts/comparators.py which prints the ast-grep the rule hook
would run, or (none); fetch downloads the pinned one. The hook must exit
0 within its budget either way — check with the binary renamed away.
Every hook must exit 0 on a malformed or empty event. Feed the script you changed a matching JSON event on stdin and check the exit code.
- Run a hand-fed hook event with
CLAUDE_CONFIG_DIR="$(mktemp -d)". Hooks log under the config directory, and a test event in the real one lands in the/claude-jevstats with no transcript behind it.
The rows bridge answers bad input with {"fallback": ...} and exit 0:
echo '' | node --experimental-strip-types src/compactor.ts rowsOut-of-band (not claimed on Grok Bot / default cloud agents): exercise the function-hooks module end to end on a machine with Claude Code and a real key — load the plugin from disk, compact, and read the debug log:
export CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1
claude -p "..." --session-id "$ID" --plugin-dir "$PWD"
claude -p "/compact" --resume "$ID" --plugin-dir "$PWD" -d
grep 'jev-compact\|core never ran' ~/.claude/debug/"$ID".txt"a hook's N messages stand ... core never ran" means the summary was replaced; "built-in summary runs" means the bridge fell through.
To measure a routing or threshold change, run the relevant eval and compare
against the table in README.md:
python3 eval/replay.py run --variant v7_no_unclear --sample 250
python3 eval/rules_eval.py run --sample 250 --seed 0
python3 eval/rules_eval.py report --sweep
node --experimental-strip-types eval/compact.ts compact --synth 60 --seed 0 --workers 8 --max-inflight 12The compaction command is the only way to measure the TypeScript compactor. It prints re-fetch coverage and planted-constraint survival in one gate and exits 2 below either floor, so a wording that keeps paths but drops what the user said, or the reverse, cannot pass on one number. It also fails on any failed backend chunk: the compactor is fail-open, so a chunk that never answers leaves its blocks unscored, and unscored rows are kept, which raises coverage.
--seed 0 --sample 250 selects the same real edits as the numbers in
README.md; keep it when comparing. A rule-question wording change misses
the answer cache and re-costs the run.
run --outmoves the answer file, not the cache, so a re-score times the cache and its latency row is meaningless. SetJEV_RULES_CACHEto a fresh path to re-ask every question; the live 247-edit pass is ~320 calls and ~40s.
The rules eval judges each record at its sha, and reads
eval/global_CLAUDE.md in place of ~/.claude/CLAUDE.md. Change a case's
sha only when a rule-file change in that repo is the thing being measured.
Second decision models: name them backend:model (bare id = System One) in
--decision-model / --judge-model; unknown backends raise, and caches key
on model so providers are interchangeable. The decisions endpoint 503s above
~24 requests in flight, so pair --workers 8 --max-inflight 12. When the
TypeSafe account is out of credits, pin
CLAUDE_PLUGIN_OPTION_PROVIDER=openrouter: provider order does not fall
through.
These call api.typesafe.ai with real past prompts and cost money. Ask before
running a full sweep.
Data and numbers
eval/observed/andeval/authored/are gitignored. They hold extracted transcripts and repo-specific cases, and they do not survive a clone. Do not commit them or write code that assumes they exist.- Numbers in
README.mdandeval/README.mdcome from eval runs. Change one only with a run behind it, and say which run. - A behavior change gets a bullet under
## [Unreleased]inCHANGELOG.md./release(.claude/skills/release) bumpsversionin rootplugin.json,.claude-plugin/plugin.json, and the plugin entry in.claude-plugin/marketplace.jsontogether, and refuses an empty section. The AFK adapter's version inadapters/afk/.claude-plugin/plugin.jsonandadapters/afk/package.jsonis bumped by hand. CI runsscripts/check_versions.py, which catches a half-done bump: it fails when those manifests disagree or one is missing, and it never writes or bumps a version.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

