agentleFS
0x70670x7067/claude-jev/AGENTS.md

claude-jev

AGENTS.md27 starsChanged 48 days agoMITView on GitHub

A Claude Code plugin. Five hooks send small judgments to TypeSafe's Jev, a System One model that returns typed answers instead of text. README.md explains what each hook decides and why. Read it before changing behavior. VISION.md is the acceptance policy for what this plugin becomes and refuses to become. Check a non-trivial change against its align/resist pair before building it, and cite it when resisting. - Hooks fail open. A failure never blocks or alters a session. Command hooks…

  • Reads credentials
  • Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.

    Of the AGENTS.md files we read, the 33,315 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.

  • In this repository4 other files it gives its agents.

The file

We split this AGENTS.md into the rules it gives an agent

6 rules. About 0 are common, 2 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Every other tunable is a module-level constant, or a userConfig field in .claude-plugin/plugin.json when the user sets it from /claude-jev …

    line 64
  2. Distinctive:

    Keep that, because Claude Code runs them from arbitrary directories.

    line 94
  3. Very specific:

    Write rules in this file as bullets, one instruction each.

    line 103
  4. Very specific:

    Run each scripts/test_*.py with python3 (scripts/test_rules_scope.py, scripts/test_stats_afk.py, and any other), then npm run test:afk and …

    line 113
  5. Distinctive:

    Every hook must exit 0 on a malformed or empty event.

    line 173
  6. Very specific:

    Run a hand-fed hook event with CLAUDE_CONFIG_DIR="$(mktemp -d)".

    line 176

≈ “Common” is an estimate: at least 1 in 5 of the 42,283 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
AGENTS.md

claude-jev

A Claude Code plugin. Five hooks send small judgments to TypeSafe's Jev, a System One model that returns typed answers instead of text. README.md explains what each hook decides and why. Read it before changing behavior.

Vision

VISION.md is the acceptance policy for what this plugin becomes and refuses to become. Check a non-trivial change against its align/resist pair before building it, and cite it when resisting.

Layout

PathWhat lives there
scripts/jev.pyAPI client and CLI. Every other script imports it.
scripts/prompt_router.pyMeasurement reference for the routing eval; production runs src/prompt-router.ts under node type stripping
scripts/rules.pyPostToolUse on edits, PreToolUse/PostToolUse on Bash to record shell writes, and Stop — rule enforcement
hooks/register.tsExperimental function-hooks module: session.compact -> node --experimental-strip-types src/compactor.ts rows, and the /claude-jev settings pane. A bridge, not a second implementation.
plugin.jsonAgent Plugins 1.0.0 identity. Claude Code still loads .claude-plugin/plugin.json and hooks/.
src/The TypeScript hook implementations the plugin runs, executed directly as source under node type stripping (needs node >= 22.18)
src/compact/strategy.tsShared compaction strategy: checks, thresholds, selectBlocks. src/compactor.ts is the Claude rows bridge.
adapters/afk/The AFK host adapter: a TypeScript implementation of the same hooks with its own manifest, hooks.json, and README. Its known-gaps list is the contract — do not claim parity that table does not state.
adapters/pi/The Pi adapter. package.json pi.extensions points at adapters/pi/jev.ts (session_before_compact and /jev). Selection imports src/compact/strategy.ts. Block shaping, the <read-files> index, the 14k+2k split, and Pi logs stay in the adapter.
scripts/comparators.pyast-grep lookups the rule hook adds to a judgment
scripts/observed.pyScores what a past turn actually did
scripts/stats.pyThe Stats row in /claude-jev, or python3 scripts/stats.py — scores live decisions from the three logs under ~/.claude: jev-router-log.jsonl (router, subagent, rules), jev-compact-log.jsonl, jev-calls.jsonl (every API call, written by jev.ask)
scripts/check_no_comments.pyHard ban: fails if scripts/, eval/, or hooks/ source has a # / // / /* */ comment
eval/Offline measurement. See eval/README.md.
.claude/skills/Maintainer skills for this repository, not shipped. release cuts a version: changelog section to bump, tag, push, GitHub release.
hooks/hooks.jsonHook registration. New hook means an entry here. modules names the function-hooks module; older Claude Code ignores the key.

Invariants

  • Hooks fail open. A failure never blocks or alters a session. Command hooks exit 0 and print nothing on any error, missing key, or timeout; keep the except Exception: return at the top of every hook main. The rows bridge answers {"fallback": ...} on stdout and exits 0, and hooks/register.ts then calls next(e) so the built-in summary runs.
  • The one external program is ast-grep, pinned by version and sha256 in scripts/comparators.py, fetched to ~/.claude/jev-bin by a detached process outside the hook's budget, and never required: every comparator answers "" without it, and the judgment proceeds as before.
  • Standard library only, per host. scripts/ and the Python files in eval/ are Python 3 standard library only: no third-party imports, and urllib.request is the HTTP client.
  • The hooks in src/, the shared modules in adapters/afk/src/shared/, and eval/compact.ts are TypeScript with zero runtime dependencies. They use Node's standard modules only; typescript, oxlint, and @types/node are devDependencies. The shipped plugin runs them as source under node type stripping, with no build step.
  • hooks/register.ts holds no judgment. It runs node --experimental-strip-types src/compactor.ts rows, falls through to next(e) on failure, and draws the /claude-jev pane. Its debug-log line is the summary string the compactor sends, and the pane's key source, provider, and last call come from python3 scripts/jev.py status.
  • One key variable per provider: each entry in PROVIDERS (scripts/jev.py) names its URL, key prefix, and variable (TYPESAFE_API_KEY, OPENROUTER_API_KEY). A new provider is a new entry there; do not add any other variable. The provider userConfig field pins one; on auto the variables are read in PROVIDERS order and the key's prefix picks. Every other tunable is a module-level constant, or a userConfig field in .claude-plugin/plugin.json when the user sets it from /claude-jev or /config. Claude Code hands those fields to hooks as CLAUDE_PLUGIN_OPTION_<FIELD>, its own variables, read only through jev.plugin_option(); hooks/register.ts passes the saved key to the compactor under the same name. CLAUDE_CONFIG_DIR is Claude Code's own variable, not a plugin tunable: jev.config_dir() honors it and every path under the user's config directory goes through that helper, never through a literal ~/.claude.
  • No code comments. Line, block, and JSDoc comments are banned in scripts/, eval/, and hooks/ source (.py, .ts, .js). Shebangs and LICENSE text stay. Python module/function docstrings are documentation strings, not comments, and stay where Conventions require them. Markdown under docs/, README.md, and AGENTS.md is prose, not code comments — leave it. Rationale for thresholds and constants (ACT, MIN_CONFIDENCE, KEEP_THRESHOLD, and the rest) lives in eval results, docs, or PR evidence — not inline comments. Changing a threshold without eval evidence is still a guess. Enforce with python3 scripts/check_no_comments.py (exit 1 on any hit).
  • scripts/observed.py is the shared scorer. eval/replay.py and scripts/stats.py both call it. Editing it moves every accuracy number in README.md.
  • Probes pay for live calls. jev.ask is assigned in exactly one place, the cached_ask wiring in eval/rules_eval.py's cmd_run, which wraps the real client with the cache. Eval and hook verification never stubs, mocks, or monkeypatches the ask path — a stub proves wiring, not behavior — and cost is bounded by shrinking the sample, never by faking the client. Enforce with python3 scripts/check_no_stubs.py (exit 1 on any .ask = assignment outside the sanctioned wiring).
  • Hook scripts import siblings through sys.path.insert(0, dirname(__file__)). Keep that, because Claude Code runs them from arbitrary directories.

Conventions

  • Each script's module docstring states the hook it serves, the fail-open contract, and the env var. Keep that shape when you add one.
  • Question definitions live in scripts/jev.py (intent_bundle, subagent_bundle) or next to the hook that asks them. Put the meaning in the instructions and criteria text — Jev never sees the key names.
  • Write rules in this file as bullets, one instruction each. rules.py parses instruction files bullet by bullet, and a rule buried inside a prose paragraph classifies poorly.
  • The user-facing entry point is the /claude-jev pane in hooks/register.ts. There is no skills/ or commands/ directory; do not add one. A new setting or report is a row in that pane.
  • .agents/skills/ is canonical for shared agent skills; symlink them into tool skills/ dirs, never copy.

Verify

Run each scripts/test_*.py with python3 (scripts/test_rules_scope.py, scripts/test_stats_afk.py, and any other), then npm run test:afk and npm run test:pi. CI (.github/workflows/check.yml) runs those tests. The local / script contract for this repo is:

python3 -m compileall -q scripts eval
python3 scripts/check_no_comments.py
python3 scripts/check_no_stubs.py
python3 scripts/check_versions.py
for f in scripts/test_*.py; do python3 "$f" || exit 1; done
npm run test:afk
npm run test:pi
echo '{"prompt":"hi","transcript_path":""}' | CLAUDE_CONFIG_DIR="$(mktemp -d)" python3 scripts/prompt_router.py; echo "exit=$?"

compileall plus stdin hook smoke (exit 0, fail open) is what a default cloud agent or Grok Bot run can claim. Ruff is a maintainer check, required for a release and for CI, and not part of that python3-only claim:

ruff format --check .
ruff check .

ruff.toml configures it. It is not a dependency file: no hook imports Ruff.

The AFK adapter has its own contract, and claiming it requires running it:

cd adapters/afk && npm ci && npm run build
echo '{"session_id":"test","cwd":"'$(pwd)'"}' | node dist/session-start.js; echo "exit=$?"

npm ci && npm run build is what a doc-only run can claim for adapters/afk/; the second line is its no-key hook smoke, and the keyed hooks are not part of any default claim. Live Claude Code session verification is out-of-band: it needs a machine with claude and TYPESAFE_API_KEY, and is not claimed as proved by those local checks alone.

For a scripted launch → doctor → drive → evidence → cleanup loop (isolated HOME, feature map under .cursor/skills/verify-claude-jev/features/), use the project-local Cursor skill:

.cursor/skills/verify-claude-jev/bin/control-jev launch
.cursor/skills/verify-claude-jev/bin/control-jev doctor
# then drive one feature from features/; evidence under
# .cursor/skills/verify-claude-jev/artifacts/<RUN_ID>/
.cursor/skills/verify-claude-jev/bin/control-jev cleanup

See .cursor/skills/verify-claude-jev/SKILL.md. Keep the map honest with /maintain-verification-skill when hooks or skills change.

python3 scripts/comparators.py which prints the ast-grep the rule hook would run, or (none); fetch downloads the pinned one. The hook must exit 0 within its budget either way — check with the binary renamed away.

Every hook must exit 0 on a malformed or empty event. Feed the script you changed a matching JSON event on stdin and check the exit code.

  • Run a hand-fed hook event with CLAUDE_CONFIG_DIR="$(mktemp -d)". Hooks log under the config directory, and a test event in the real one lands in the /claude-jev stats with no transcript behind it.

The rows bridge answers bad input with {"fallback": ...} and exit 0:

echo '' | node --experimental-strip-types src/compactor.ts rows

Out-of-band (not claimed on Grok Bot / default cloud agents): exercise the function-hooks module end to end on a machine with Claude Code and a real key — load the plugin from disk, compact, and read the debug log:

export CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1
claude -p "..." --session-id "$ID" --plugin-dir "$PWD"
claude -p "/compact" --resume "$ID" --plugin-dir "$PWD" -d
grep 'jev-compact\|core never ran' ~/.claude/debug/"$ID".txt

"a hook's N messages stand ... core never ran" means the summary was replaced; "built-in summary runs" means the bridge fell through.

To measure a routing or threshold change, run the relevant eval and compare against the table in README.md:

python3 eval/replay.py run --variant v7_no_unclear --sample 250
python3 eval/rules_eval.py run --sample 250 --seed 0
python3 eval/rules_eval.py report --sweep
node --experimental-strip-types eval/compact.ts compact --synth 60 --seed 0 --workers 8 --max-inflight 12

The compaction command is the only way to measure the TypeScript compactor. It prints re-fetch coverage and planted-constraint survival in one gate and exits 2 below either floor, so a wording that keeps paths but drops what the user said, or the reverse, cannot pass on one number. It also fails on any failed backend chunk: the compactor is fail-open, so a chunk that never answers leaves its blocks unscored, and unscored rows are kept, which raises coverage.

--seed 0 --sample 250 selects the same real edits as the numbers in README.md; keep it when comparing. A rule-question wording change misses the answer cache and re-costs the run.

  • run --out moves the answer file, not the cache, so a re-score times the cache and its latency row is meaningless. Set JEV_RULES_CACHE to a fresh path to re-ask every question; the live 247-edit pass is ~320 calls and ~40s.

The rules eval judges each record at its sha, and reads eval/global_CLAUDE.md in place of ~/.claude/CLAUDE.md. Change a case's sha only when a rule-file change in that repo is the thing being measured.

Second decision models: name them backend:model (bare id = System One) in --decision-model / --judge-model; unknown backends raise, and caches key on model so providers are interchangeable. The decisions endpoint 503s above ~24 requests in flight, so pair --workers 8 --max-inflight 12. When the TypeSafe account is out of credits, pin CLAUDE_PLUGIN_OPTION_PROVIDER=openrouter: provider order does not fall through.

These call api.typesafe.ai with real past prompts and cost money. Ask before running a full sweep.

Data and numbers

  • eval/observed/ and eval/authored/ are gitignored. They hold extracted transcripts and repo-specific cases, and they do not survive a clone. Do not commit them or write code that assumes they exist.
  • Numbers in README.md and eval/README.md come from eval runs. Change one only with a run behind it, and say which run.
  • A behavior change gets a bullet under ## [Unreleased] in CHANGELOG.md. /release (.claude/skills/release) bumps version in root plugin.json, .claude-plugin/plugin.json, and the plugin entry in .claude-plugin/marketplace.json together, and refuses an empty section. The AFK adapter's version in adapters/afk/.claude-plugin/plugin.json and adapters/afk/package.json is bumped by hand. CI runs scripts/check_versions.py, which catches a half-done bump: it fails when those manifests disagree or one is missing, and it never writes or bumps a version.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.