agentleFS
EtanHeyEtanHey/golems/skills/golem-powers/false-green-gate/SKILL.md

false-green-gate

Skill0 starsChanged 16 days agoApache-2.0View on GitHub

'done/green/fixed' needs a live probe. Triggers: completion claim, pr-loop completion, deploy/render/dashboard 'done'.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

Files
SKILL.md

Skill: False-Green Kill-Gate (gen-18 Track 2 — the gen-17 north star)

Fleet law: canon #4 owns "done = user-visible". This skill is the mechanical same-turn live-outcome probe gate.

Scope

The gate stays in scope by default, including no-tool completion claims. It exempts only a turn with at least one assistant tool call where every call is on the closed conversational allowlist: voice_ask, voice_speak, brain_search, brain_recall, brain_expand, or AskUserQuestion, including the exact VoiceLayer and BrainLayer MCP names for those tools. Bash, edits, delegation, Codex shell/exec, other MCP tools, and cmux tools keep the turn in scope. Seat names and claimed repo-diff metadata do not determine scope. A coach using Bash is gated; a voice-only coaching drill is exempt. Probes must be same-turn.

What It Is

A deterministic detector over an agent transcript: a completion claim is FALSE-GREEN unless the same turn carries the live-outcome probe its claim domain requires. This is the MECHANICAL gate that /deploy-verify and /never-fabricate describe in prose — "manual gates drift, automated gates don't." The pinned RED/GREEN transcript fixtures ARE the replayable gate (R-003/R-014 pattern, consumed in the T6 deterministic-CU smoke-spec shape).

The historical evals/fixtures/ corpus is pinned for Jev replay. This gate's suites replay it unchanged, with no synthetic activity. New scope contracts live in evals/scope/fixtures/, outside the Jev preflight tree.

The Rule — required same-turn probe by claim domain

Claim domainRequired live probe (same turn)Violation if missing
render / audio / mp3ffprobe proving size>0 AND duration>0 on the claimed pathFALSE_GREEN_FFPROBE
…cloned / 2-voice render--reference resolves to a registered clone (<GOLEMS_EXPERT_VOICE>-c4/ben-c1; placeholder examplechannel), no system-TTS fallbackFALSE_GREEN_VOICE
dashboard / UI / siteHTTP 200 on the served URLFALSE_GREEN_DASHBOARD_200
AND a signed-in click-through (200 alone is not enough)FALSE_GREEN_CLICK_THROUGH
deploy / merged / installedbuild-stamp that POST-DATES the mergeFALSE_GREEN_STAMP
AND a live round-trip (pgrep/launchctl/health 200)FALSE_GREEN_LIVE_PROBE
build / contractthe operational entrypoint exercised (wrapper dry-run / contract test)FALSE_GREEN_LIVE_PROBE
generic "done"at least one live round-tripFALSE_GREEN_LIVE_PROBE
"rigorously verified manually" + no probe— (ranked last, false-green-permitting)MANUAL_SUBSTITUTE

"Same turn" = the events since the last human message — the probe tool calls and their results the agent ran before the claim. Negated/in-progress statements ("not done yet") are not claims.

How /pr-loop Consumes It

/pr-loop already states the Deploy Truth Gate (merged ≠ serving) and Post-Merge Verification in prose. This gate makes it mechanical: before a worker/lead emits TASK_DONE or a "done/deployed/green/ render-complete" message, run the gate on the turn — /false-green-gate (bun skills/golem-powers/false-green-gate/scripts/false-green-gate-cli.mjs <transcript|->, exit 3 = FLAG). A FLAG means the claim is not yet earned: run the missing probe, then claim. Compose with /deploy-verify (the four-step deploy sequence) and /never-fabricate (read the output before reporting).

Run It

bun test skills/golem-powers/false-green-gate/evals/false-green-gate.test.mjs   # replay (CI-safe)
python3 skills/golem-powers/false-green-gate/evals/run_suite.py                 # hook stdout-schema suite
bun skills/golem-powers/false-green-gate/scripts/false-green-gate-cli.mjs <transcript.jsonl|->

Programmatic: import { detectFalseGreen } from "./src/false-green-gate.mjs" → { verdict, claim, domains, violations }.

Stop Hook Output

scripts/false-green-gate-hook.mjs never blocks (GO-5 E2):

  • allow: {}
  • advisory: {} plus a log line; with GOLEMS_STOP_ADVISORY_RENDER=1, {"systemMessage":"FALSE-GREEN-GATE advisory: ..."}

Advisories are logged, not rendered (2026-10-19 Stop-advisory eval: false-green 3/25 useful, fleet-wrap 0/25, 0/47 follow-ups responded). A flag prints {} and appends one golems.stop-advisory.v1 line (ts, gate, session_id, transcript_path, codes, message, cwd) to ~/.claude/hooks/stop-advisories.jsonl (STOP_ADVISORY_LOG_PATH overrides). Set GOLEMS_STOP_ADVISORY_RENDER=1 to render the systemMessage instead. A log-write failure still prints {}.

Stated Limits (honesty rule)

  • Evidence is marker-anchored over the same-turn blob; a probe described but not actually run can read as present. The fixtures pin the known specimens; new evasion shapes are added as RED fixtures (R-003 model).
  • "Same turn" is bounded by the last human message; a probe run two turns earlier does not count (by design — the false-green class is precisely a stale or absent same-turn probe).
  • TODO(P2): Evidence-in-dashboard decision-answering claims belong in qa-verdict/render-done gates, not this generic false-green gate.

Provenance

RED specimens: brainlayer/915ace6b#5 (stale BrainBar stamp), narrationlayer/e3a91210#1 ('give it a play', no mp3), codex/019edebd#3 (REAPER build-green contract-broken), R-004 dashboard family, R-035 manual- substitute. GREEN references: narrationlayer/019ee201#4 (ffprobe pass), voicelayer/019ed6c2#5 (daemon round-trip), dashboard 200+click-through, clone-voice-resolved, contract-exercised, no-claim (N/A).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.