bestax
CLAUDE.md14 starsChanged 8 days agoMITView on GitHub
Everything in .github/workflows/ is human-authored by design. Two enforcement layers apply here and they are not the same rule — do not collapse them: - Automated sessions (the AI fix loop, ai-triage, ai-scan) are denied write access mechanically: Edit(.github/**) and friends sit in every write agent's --disallowedTools, and the fix loop refuses any PR that touches .github/**. This is not advisory, and editing this file does not relax it. Those sessions ingest untrusted issue and PR text, so for them…
- Reads credentials
- Installs packages
In this repository19 other files it gives its agents.
- AGENTS.mdAGENTS.md
- bulma-ui/AGENTS.mdAGENTS.md
- bestax-mcp/CLAUDE.mdCLAUDE.md
- bestax-migrate/CLAUDE.mdCLAUDE.md
- bulma-ui/CLAUDE.mdCLAUDE.md
- CLAUDE.mdCLAUDE.md
- create-bestax/CLAUDE.mdCLAUDE.md
- docs/CLAUDE.mdCLAUDE.md
- eslint-plugin/CLAUDE.mdCLAUDE.md
- skills/CLAUDE.mdCLAUDE.md
- templates/CLAUDE.mdCLAUDE.md
- bulma-ui/llms.txtllms.txt
- bestax-custom-componentSkill
- bestax-formSkill
- bestax-iconsSkill
- bestax-layout-scaffoldSkill
- bestax-migrateSkill
- bestax-optimizeSkill
- bestax-themingSkill
The file
Show its rules and text
We split this CLAUDE.md into the rules it gives an agent
20 rules. About 0 are common, 10 are distinctive, and 10 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈10
Distinctive · estimate
This author's own choices. Worth a look.
10
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 11
An interactive session working alongside a maintainer: should not hide behind that.
- Distinctive: line 47
When unsure which row a change belongs in, grep the workflows for the thing you are changing.
- Distinctive: line 62
Every rule below serves one threat model, stated here once so a review has a place to stop and an author has something to cite instead of …
- Distinctive: line 115
Every AI workflow here is built to preserve these.
- Distinctive: line 186
When bumping, bump every occurrence in a single commit and verify there is exactly one:
- Very specific: line 220
harden-runnerhas an assertion tied to its pin: Every block-mode job asserts the effective policy out of /home/agent/agent.json (rule 10). - Very specific: line 225
The pin does not fully determine which agent binary runs: Before installing, the pre-step calls isTLSEnabled(owner), a 3-second probe of a …
- Distinctive: line 245
Read the third column for which credential — since #455 and #457 neither session can reach repository write any more, and "the session …
- Very specific: line 258
Never add Bash(gh api:*) to a session that ingests untrusted issue/PR text — it is a general-purpose write primitive wearing a read-shaped …
- Very specific: line 260
Never add Edit, Write, MultiEdit, or Task to ai-scan.yml.
- Very specific: line 263
Prefer removing the need for the boundary over hardening it.
- Distinctive: line 294
Every repository variable that steers this automation is tabulated in the ai-development docs guide, including its unset default.
- Very specific: line 332
Use plain pull_request, never pull_request_target, plus an explicit head-repo guard (#312).
- Distinctive: line 368
Do not close this by checking out the PR head — running PR-authored code in a job holding a credential is the thing that pin prevents, and …
- Distinctive: line 591
Do not maintain any of this by hand.
- Distinctive: line 619
Keep this inventory correct when you add a job, and qualify per job rather than per workflow.
- Very specific: line 635
Verify rather than assume, on any run: the assertion step passes, and harden-runner's post-step prints the effective EgressPolicy:.
- Very specific: line 677
When the pre-step selects the closed-source agent-ebpf instead (rule 1), none of it can be checked from here.
- Very specific: line 781
Run 33221210633's log shows auto-close-duplicates at block connecting to api.github.com and github.com and nothing else.
- Very specific: line 892
Never cite a line number in prose.
≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
.github — CI and AI automation
Everything in .github/workflows/ is human-authored by design. Two enforcement layers apply
here and they are not the same rule — do not collapse them:
- Automated sessions (the AI fix loop,
ai-triage,ai-scan) are denied write access mechanically:Edit(.github/**)and friends sit in every write agent's--disallowedTools, and the fix loop refuses any PR that touches.github/**. This is not advisory, and editing this file does not relax it. Those sessions ingest untrusted issue and PR text, so for them the prohibition is absolute. - An interactive session working alongside a maintainer should not hide behind that. State the rule, state the security reason below that it exists for, make sure the maintainer has actually weighed that reason — then act on their answer. Refusing to apply a change the maintainer has decided on does not make the repository safer; it just turns one decision into several round-trips. The maintainer was always the one deciding.
The reason to raise, so nobody has to rediscover it: a change in this directory can grant
capability with no permissions diff for a reviewer to notice. The permissions: block looks
identical before and after an allowlist widens. That is why these changes get argued rather than
waved through — not because agents are untrusted, but because the usual review signal is absent.
Scale the pushback to the blast radius:
| Change | What to do |
--allowedTools / --disallowedTools on a session holding a credential | Argue it properly, every time. Name the credential in that job, explain why each entry cannot write, and get an explicit decision. Never as a side effect of another task. See rule 2. |
Workflow logic, triggers, permissions:, action SHAs, verdict/budget paths, anything a workflow reads as a gate or trigger | Name the invariant at stake (I1, I2, or the rule below), confirm it holds, then act. |
Genuinely inert config — a cosmetic label, a docs-build cadence, a semver-major dependabot.yml ignore | Just make the change and say what it does. Do not gate it behind a debate. |
The qualifiers in that last row are load-bearing. The dividing line is cosmetic vs. load-bearing, not the file the change lives in:
- Labels are not uniformly inert.
needs-security-reviewis a refusal gate —claude-repro,claude-fix,@claudeand@bestaxbotall decline a flagged item until a maintainer clears it.ai-loop,ai-loop-pausedanddeep-reviewsteer automation the same way. Deleting or renaming one of those disables a control with no workflow diff at all, which is the exact hazard this section exists to name. A newdocumentationlabel is inert; a label a workflow reads is middle row.review-convergedis written byreview-converged.ymlfor the owner to read. That workflow reads it back only to decide whether to add or remove it, and no automation gates on it. - Schedules are not uniformly inert. A
schedule:onai-scanor the stale sweep decides when a security control runs. A docs-build cadence does not. dependabot.ymlignores are not uniformly inert. Asemver-majorignore only declines a breaking upgrade, but one covering patch or minor suppresses CVE fixes for that dependency. Scope every ignore you add, and never widen an existing one'supdate-typesas a drive-by.
When unsure which row a change belongs in, grep the workflows for the thing you are changing. If anything reads it, it is middle row.
Origin: an interactive session hit the old blanket "propose a diff, do not apply one" line while
adding a dependabot.yml ignore entry — a change with no security surface at all — and stalled
on it across three separate asks. The blanket rule was cheap to state but it spent the
maintainer's attention on the one case that did not need it, which is attention not available for
the allowlist case that does.
This file is the security contract for these workflows. The rules below are not style preferences — each one is load-bearing, and most were written after a review round or a red-team found the failure it prevents. Where a rule has a documented origin, it is cited.
The trust boundary
Every rule below serves one threat model, stated here once so a review has a place to stop and an author has something to cite instead of shipping another defence for the same step.
What these jobs are exposed to. The deep review and the fix loop check out branch code from
an open PR and run its package manager and its tests; @claude and @bestaxbot join that class
when a review event triggers them, and check out the default branch when a comment or an issue
does. Everything under a PR checkout is attacker-influenced: the sources, the lockfile,
package-manager hooks such as .pnpmfile.cjs, .gitattributes, and any file a later step reads
back from the working tree. The scan, triage and repro sessions check out the default branch
and install nothing, so their exposure is the issue and PR text they ingest, which is
attacker-authored on every one of these jobs. A model token shares the job with whatever it
reads. Where the PAT sits differs and is the thing to check: triage hands a finished payload to
a publish job that holds the PAT and runs no model; repro's publisher holds no PAT at all and
posts with GITHUB_TOKEN (I2); the fix loop and @bestaxbot hand the PAT to the session
itself. That exposure is the job: reviewing and fixing code means running it, and no rule here
pretends otherwise.
The accepted defences. These are the answer, and a review does not re-litigate whether they
are sufficient: actions pinned to one SHA repo-wide (rule 1); tool allowlists held as a
confinement boundary (rule 2); explicit opt-in for anything that spends usage (rule 3); a
deterministic sanitizer or renderer on the publication paths that have one — repro's draft and
triage's comment (rule 5, I2); plain
pull_request with a head-repo guard, never pull_request_target (rule 7); sender exclusions
on comment triggers (rule 8); trust decisions re-verified against the live API rather than read
from the event payload (the trusted-labeler gates); an enforced egress policy
(rule 10, and read I1 for exactly what that leg adds); and a human merge — the loop never
merges. A finding that one of these could be stronger is advisory unless it shows the control
absent or false.
Where it stops. A step boundary inside one job is not a trust boundary. A file a step writes for a later step of the same job is trusted to the degree the job is: the checkout already influences every step that runs after it, so hardening that hand-off against the checkout is worth doing when it is cheap, and is not a blocking defect when it is not. Origin: #647, where the step that writes the review delta for a later step drew round after round of correct, ever-narrower findings while every rule in this file held, because nothing said when enough was enough.
What is blocking on that axis, however the change is dressed:
- anything that widens who can trigger a job — a new trigger, a looser
if:, a sender exclusion dropped, a label gate bypassed; - anything that widens what a credentialed job can reach — a tool, an endpoint, a scope, a
PAT where a
GITHUB_TOKENwas, a checkout of PR code where the default branch was; - anything that widens what leaves the job — a new comment or artifact path, model or issue text reaching a publisher without the sanitizer, a posting identity that re-triggers.
That list ranks exposure, and only exposure. An exposure finding naming none of those goes on the record as advisory and does not hold the merge. A functional defect — broken shell or jq, a swallowed error, a silent no-op, a dead condition, a race — is blocking on its own merits under the normal review criteria, and nothing in this section lowers it.
The two invariants
Every AI workflow here is built to preserve these. If a change breaks one, the change is wrong, regardless of how convenient it is.
I1 — in the publishing pipeline, the model-auth token never shares a job with code execution.
CLAUDE_CODE_OAUTH_TOKENpays for a session; anything that can run code in the same job can read it out of the environment.claude-repro.ymlsplits drafting from publishing for exactly this reason: the drafting job has noBash, noTask, no network tool, so it cannot fetch or execute. Note the direction of the guarantee — it raises the cost of exfiltration, it is not a proof. The credential-shape check inCollect draft(literal, base64, and hex forms) is a backstop for the same reason, and is documented as one.Read the scope precisely, because the unqualified version is false. This is a property
claude-reprois built to preserve, not a repo-wide fact.claude-implement,claude-pr-loop(fix/verify),claude-review,claudeandbestaxbot-replyall check out a branch and run repository code in the same job as that token, and they must — fixing and reviewing code is the job. What holds them is a different, weaker set: the tool allowlist, the protected-path deny rules, the trusted-labeler gates, and — as of #578, for all five of them — an enforced egress policy on a measured allowlist. Read that last leg for exactly what rule 10 says it delivers: it narrows where a session that does not escalate can send data, andapi.github.comis necessarily on every one of those allowlists, so it does not bound what the session can do through a tool. Nor does it hold an injected session that runs code: that code can reach past the list ("What the agent allows beyond the list" under rule 10). Saying I1 covers those jobs would be the exact overstatement the checklist at the bottom of this file ends on. It covers the pipeline that was designed around it.harden-runner's egress block enforces again as of #487 —
blockused to degrade silently toaudit, and now enforces and is asserted (rule 10). Do not promote it to a third leg of I1, which an earlier draft of this file did whileclaude-repro.ymlsaid the opposite: on the exfil path I1 exists to close —Writeplus a later publishing job — the drafting job's egress policy bounds nothing, because the publishing job has no policy of its own. Be exact about what it does add: it narrows where a session that does not escalate can send data, not what it can do with the hosts it is allowed.api.github.comis necessarily allow-listed in every one of these jobs, so egress-block cannot stop a write issued through a tool. Never trade away a tool restriction on the grounds that "egress is blocked anyway" — the two controls cover different things, which is the whole reason both exist.I2 — no untrusted or model-authored free text reaches a re-trigger-capable identity. Comments posted with
GITHUB_TOKENdo not emit workflow events. Comments posted with a PAT do. Two shapes address this, and only one of them is airtight — be precise about which you are relying on.claude-repro.ymlchanges the IDENTITY: the drafted reproduction is sanitized deterministically and posted viaGITHUB_TOKEN, whose comments emit no events at all. That is structural — no text can re-trigger anything, whatever it says.ai-triage.ymlkeeps the PAT identity (#361 wanted it) and narrows the TEXT instead. Since #457 the session posts nothing; a deterministic renderer owns every structural string (the markers,Duplicate of #N,Fixes #N, the auto-close notice) and every issue reference is a validated integer. But the published body still carries two bounded model-authored fields, a candidate's title and a one-line reason, escaped and defanged against the KNOWN re-trigger vectors — mentions, markers, autolinks, machine sentinels. It cannot defang a trigger token nobody has invented yet: a literal/retestin a title survives into the comment today. So for that residual class rule 8's sender exclusions are still the layer, not a second one, and a future comment-triggered workflow that forgets them reopens it.Publishing the two fields is a deliberate, revisitable trade — a prose-free comment (bare
#Nreferences, or a renderer-owned enum in place of the reason) would make this structural too, at the cost of the signal a human reads the comment for. Do not describe the triage path as closing I2 by construction; describe it as narrowing the surface and keeping rule 8.
Hard requirements
1. Pin every third-party action to a full commit SHA — the same SHA everywhere
SECURITY.md advertises SHA pinning as an active control, so this is a promise to users, not
housekeeping. One SHA per action across all workflow files, and the trailing comment must name
the version that SHA actually is.
Current pin: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 (this is v7.0.1).
When bumping, bump every occurrence in a single commit and verify there is exactly one:
grep -rho "actions/checkout@[a-f0-9]\{40\}" .github/ | sort | uniq -c # expect one lineThat one is scoped to the action you are bumping. Nothing scopes the rule that way, so also run the repo-wide form, which names any action carrying more than one SHA:
grep -rhoE "[A-Za-z0-9_.-]+/[A-Za-z0-9_./-]+@[a-f0-9]{40}" .github/ \
| sort -u | cut -d@ -f1 | uniq -d # expect NO outputThe per-action grep only ever sees the action you thought to type, so a second action left on a stale SHA by an earlier bump stays invisible to it however many times you run it.
Note both greps cover .github/, not .github/workflows/: .github/actions/verified-commit/
pins actions/github-script too, and the narrower path cannot see it — it would print exactly
the "one line" this rule tells you to expect while the composite action sat on a stale SHA.
Those greps are the whole verification. No CI gate checks workflow content — format:check
does lint .github markdown and .mjs, and ci.yml runs .github/scripts/pin-react.mjs, so
the directory is not untouched by CI, but nothing verifies a pin, a policy or a permission.
Three consequences worth holding on to:
- Do not move a security-relevant step into a local composite action. Not for the pin's
sake — both greps recurse over
.github/, so a pin under.github/actions/still matches, and an earlier draft of this bullet claimed the opposite. The reason is that reading the job is the whole review signal here, andharden-runneris the case that matters: itsegress-policyandallowed-endpointsare argued per job against the credential in that job (rule 2), and rule 10's inventory is checked by reading each one. Behind a wrapper, a job's egress posture is no longer visible where the job is, and a shared wrapper collapses those separate allowlists into one — a widening with nopermissions:diff, which is the hazard at the top of this file. harden-runnerhas an assertion tied to its pin. Every block-mode job asserts the effective policy out of/home/agent/agent.json(rule 10). That path is an internal detail of the action with no compatibility guarantee, so a bump that moves or renames it fails every block job at once. That is the intended direction, but check it when bumping rather than being surprised by it.- The pin does not fully determine which agent binary runs. Before installing, the pre-step
calls
isTLSEnabled(owner), a 3-second probe of a StepSecurity endpoint that fails open — on any error or timeout it logsUnable to check TLS_STATUS. Defaulting to TLS enabled.and returns true, selectingagent-ebpfinstead ofagent. Only the latter is source-available, so the paths and status strings rule 10 asserts can only be verified against that one. Today this org's endpoint answers 403 (TLS_NOT_ENABLED), so the verifiable binary is what runs; a StepSecurity outage would swap it with no diff and no pin change. If every block job goes red at once with no workflow change, check this before anything else.
Origin: #361 shipped two new workflows pinned to 9c091bb… (v7.0.0, twelve commits behind)
while nineteen other usages were on v7.0.1 — and both were commented # v7, so the drift was
invisible to a reader. Copilot caught it; nothing in CI would have.
2. Never widen --allowedTools on a session that holds a credential
Treat every allowlist in this directory as security-critical. It is a confinement boundary,
not a convenience list, and widening it grants capability with no permissions diff for a
reviewer to notice — the permissions: block looks identical before and after.
Two sessions where the allowlist is the only thing between untrusted text and a credential. Read the third column for which credential — since #455 and #457 neither session can reach repository write any more, and "the session can't write to the repo" is not the same claim as "the allowlist stopped mattering":
| Workflow | Credential in the session's job | What the allowlist is holding back |
ai-triage.yml | CLAUDE_CODE_OAUTH_TOKEN; job GITHUB_TOKEN is read-scoped (contents/issues/pull-requests) since #457 | Repository write: nothing any more. This session held AI_LOOP_PAT — full repo write, unscoped by the job's permissions: block — until #457 split publishing out: the session emits a structured payload, scripts/render-triage-comment.mjs renders the comment, and the PAT lives only in a publish job that runs no model. The model credential: everything. The session still runs Bash (GET-only gh) and Task beside CLAUDE_CODE_OAUTH_TOKEN, so by I1 the allowlist is what stands between an injected session and reading that token out of the environment. #457 closed the concrete instance of that — Read is not workspace-confined, and the prefix match accepted gh issue comment N --body "$CLAUDE_CODE_OAUTH_TOKEN". Narrow, not retired. |
ai-scan.yml | CLAUDE_CODE_OAUTH_TOKEN; job GITHUB_TOKEN is read-scoped (contents/issues/pull-requests) since #455 | Repository write: nothing any more. The budget marker and the needs-security-review label moved to separate gate and label jobs that run no repository code, and only the coarse verdict enum crosses between them, so widening the allowlist can no longer grant issue/PR write. The model credential: everything. This job still runs Bash beside CLAUDE_CODE_OAUTH_TOKEN, so by I1 the allowlist is what stands between an injected session and reading that token out of the environment. Egress-block IS enforced here as of #487 (rule 10), and the two are complementary rather than redundant: while the allowlist holds, it narrows where data can go, though api.github.com is necessarily allow-listed, so it cannot stop a write issued through a tool. It is no backstop once the allowlist is defeated: a shell on the runner can reach past the list ("What the agent allows beyond the list"). Narrow, not retired. |
Concrete rules:
- Adding any entry to those two allowlists is a security change. Say so in the PR description and explain why the entry cannot write.
- Never add
Bash(gh api:*)to a session that ingests untrusted issue/PR text — it is a general-purpose write primitive wearing a read-shaped name. - Never add
Edit,Write,MultiEdit, orTasktoai-scan.yml. --disallowedToolsis defense in depth, and its deny rules do take precedence over the allows — but do not lean on it as the primary control. Narrow the allowlist.- Prefer removing the need for the boundary over hardening it. #455 is the worked example:
ai-scan.ymlwas one job whose write scopes the model session merely happened not to use, and it became three (gate/scan/label) so the session's own job grants are read-only. The allowlist did not change; what changed is that it is no longer the only thing standing behind it. When a session's job holds a write scope for the benefit of some other step, that is the shape to look for. - #457 is the same move against the harder case, and worth reading second:
ai-triage's session did not merely sit near a write credential, it held one — the Claude action installs itsgithub_tokenas the session'sGH_TOKEN, so passingAI_LOOP_PAThanded an untrusted-text-ingesting session full repo write. Splitting the job was the only way to get it a genuinely read-only credential without minting a new secret, becauseGITHUB_TOKENis job-scoped and the budget marker needsissues: writesomewhere. The lesson to carry: when a session's credential is passed to the action rather than merely present in the job, narrowing the allowlist cannot reach it at all — only moving the work can.
3. Opt in explicitly for anything that spends model usage
Gate on == 'on', never on != 'off'. Unset, empty, a typo, and every spelling of "stop" must
all mean off — no misspelling should be able to start a control that spends usage on every
incoming item.
if: vars.AI_LOOP_ENABLED == 'true' &&
(vars.AI_SCAN_MODE == 'on' || vars.AI_SCAN_MODE == 'y')- GitHub Actions
==on strings is case-insensitive, soONandYwork too. Do not add case variants to the expression. - The one deliberate exception is
AI_TRIAGE_MODE, whose label path is!= 'off'so that human-applied labels still work with the variable unset. It is an exception because a label requires a trusted human first. - Every repository variable that steers this automation is tabulated in the ai-development docs
guide, including its unset default. Add new ones there in the same PR. Origin: that table
documented
AI_LOOP_ENABLEDbackwards until Copilot caught it — the gates are== 'true', so unset means disabled.
4. Fail closed on verdicts, fail open on budgets
Both directions are deliberate and the split is the point:
- A verdict (is this item malicious?) fails closed — a crashed, missing, unparsable, or forged sentinel flags rather than passes.
- A budget or counter fails open — if the daily-limit read errors, the item is simply not scanned. A counter problem must not be able to wedge every incoming issue.
The cost of the second is real and should stay written down: downstream, an unscanned item is indistinguishable from a clean one.
5. Secret masking scrubs logs, not comment bodies
Anything a session writes into a comment is published verbatim. Never build a comment body out of model output without a deterministic sanitizer between them, and never assume masking will save you — it applies to the job log only.
6. Scope comment edits by marker, never --edit-last
gh ... --edit-last picks the most recent comment by that author, which is not necessarily
yours. It could overwrite a historical <!-- ai-triage:dedupe --> marker that
auto-close-duplicates.mjs still reads — silently destroying an auto-close candidate. Select by
the marker your workflow owns.
More generally: when probing for a machine comment, match on marker + (bestaxbot OR a
Bot-type author). Never probe one specific login. bestaxbot is a machine User account, not a
Bot-type app, so a type == 'Bot' test alone misses it and a login test alone breaks the next
time the identity changes. A probe that grants trust rather than finding a comment pins the
identity instead, and isDeepReviewAuthor in scripts/review-converged.mjs says why.
7. Fork PRs never run with secrets
Use plain pull_request, never pull_request_target, plus an explicit head-repo guard (#312).
8. Sender exclusions belong on every comment-triggered workflow
contains() matches a raw substring, so an @mention reproduced anywhere in a body — including
inside a quoted block or a code fence — is enough to re-trigger a write-capable session. Require
sender.type == 'User' and sender.login != 'bestaxbot', and forbid the session from
writing the trigger string at all.
9. Logic worth testing does not belong in YAML
Shell embedded in a workflow step cannot be unit-tested without extracting it first. Put
non-trivial parsing in scripts/*.mjs with a test sibling — root pnpm test runs every
scripts/*.test.mjs.
Three extractions exist. The two from #454: the scan-verdict parser
(scripts/parse-scan-verdict.mjs, called by ai-scan.yml) and the publish sanitizer
(scripts/sanitize-repro-draft.mjs, called by claude-repro.yml) — their test siblings pin
the fail-closed matrix and the byte behavior of the shell they replaced, so edit script and
tests together. Then #457's triage renderer/publisher
(scripts/render-triage-comment.mjs, called twice by ai-triage.yml — once per mode), which
is the largest and the one to read first: it validates a model-authored payload, renders the
comment from renderer-owned constants, and upserts it by marker. Its test sibling runs the
real auto-close-duplicates.mjs consumer over rendered output, so the two cannot drift.
Smaller instances of the same shape remain inline (the exec-file sentinel
checks in ai-triage.yml, claude-review.yml, and claude-repro.yml's author job); when
one of those next needs an edit, extract it and reuse parse-scan-verdict.mjs's exported
helpers rather than growing the YAML.
Extraction has a bootstrap gap: a new script flags its own introducing PR. A workflow runs
the YAML from the PR merge ref, but the jobs here deliberately check out the default branch
(never PR head code). A script the PR adds therefore does not exist in the workspace its own
run reads: node exits MODULE_NOT_FOUND, the caller takes its fallback, and a fail-closed
verdict path labels the item on infrastructure grounds without the verdict ever being read.
Origin: #570 applied needs-security-review to itself this way, which then gated
claude-repro, claude-fix, @claude and @bestaxbot on that PR.
Do not close this by checking out the PR head — running PR-authored code in a job holding a credential is the thing that pin prevents, and trading it away for a cosmetic label is a far worse deal than the flag. Treat the gap as expected cost instead: say in the PR description that the flag is self-inflicted, and clear the label by hand once the script is on the default branch. The same applies to moving or renaming a script a workflow already calls — the rename lands in the YAML a run before it lands in the tree that run checks out.
10. harden-runner on new jobs starts at block, and every block job asserts it
New jobs ship with egress-policy: block. An existing live job may be introduced at audit
first, because flipping straight to block risks breaking it if the action's runtime egress needs
an un-allow-listed host — but that is a temporary state that owes a follow-up issue, not a
resting place.
Block jobs also set disable-sudo-and-containers: true, unless a step needs sudo or Docker. A
job that leaves it off says which step needs it, in the comment above its harden-runner. What
the setting and its check do and do not prove is under "What disable-sudo-and-containers and
its check prove", below.
Every block-mode job must carry the assertion step (placement rules below the snippet):
- name: Assert egress policy is enforced
run: |
set -uo pipefail
[ -e /home/agent/agent.json ] || { echo "::error::harden-runner installed no agent"; exit 1; }
jq -e '.egress_policy == "block"' /home/agent/agent.json || exit 1
# In a job that sets disable-sudo-and-containers, this line and the two in the loop:
jq -e '.disable_sudo_and_containers == true' /home/agent/agent.json || exit 1
for _ in $(seq 1 30); do
if grep -q '^Initialized' /home/agent/agent.status 2>/dev/null; then
if sudo -n true 2>/dev/null; then echo "::error::sudo still works"; exit 1; fi
if [ -w /var/run/docker.sock ]; then echo "::error::docker.sock is writable"; exit 1; fi
exit 0
fi
sleep 1
done
exit 1Copy the full version from a block job that matches the new one on
disable-sudo-and-containers, which scripts/harden-runner-sudo.test.mjs holds together. The
shapes above are each load-bearing:
- The
-echeck comes first because harden-runner has deliberate paths that install nothing and still exit 0 (a StepSecurity outage, theskip-harden-runnerrepo property, a container or slim runner). Without it a vendor outage fails every block job at once, with a barejq: could not open file. It still fails, because a job holding a credential must not run unprotected, but it says why. Do not write down how many block jobs that is: an earlier version of this line did, and it was wrong before theai-scan/ai-triagejob splits. Derive it, and grep for the key rather than the string —grep -rn "^ *egress-policy: block$". A plaingrep egress-policy: blockcounts one more:deploy-worker.ymlnames the policy in a comment, which is the same way anode -ein a comment misclassifies a job below. - The status check polls rather than testing once. The pre-step waits only for the file to exist and gives up after ~9s, while the agent resolves every allow-listed host before writing its status, so a cold resolver or a long list can leave it absent or empty at this point.
^Initialized, not-qx.writeStatusappends without a trailing newline, so a second status would concatenate onto the same line and an exact-line match would stop matching.- The
disable-sudo-and-containerslines are explained under "Whatdisable-sudo-and-containersand its check prove", below.
Placement. The default is immediately after harden-runner, and most jobs use it. Two things override it, both derived from one rule:
A fail-closed check must not be able to consume, skip, or precede the thing that keeps the fail-closed path reachable.
- After any gate an
always()-guarded fail-closed step depends on, and before any step that spends a metered budget.ai-scan'sgateis the live example, and it needs both: putting the assertion first would abort beforerunis set and skip the labeler; letting the same step read and charge the budget would spend a slot on every run that failed to prove enforcement, andAI_SCAN_DAILY_LIMITsuch failures in a UTC day (a StepSecurity outage reaches every incoming item at once) setrun=false, which skips that same labeler. So: decide → assert → charge. That job also recordsenforcement=failedbefore exiting, becauselabeldeliberately fails OPEN on ordinary gate trouble and must fail CLOSED on this one — see its comment. - After the write, when the job is the fail-closed path.
ai-scan'slabelandai-triage'scleanupexist to guarantee a label is applied and removed respectively; a check that can fail ahead of that call defeats the guarantee outright, which is also why their harden-runner steps arecontinue-on-error. Assert afterwards underalways(): the write has happened, and a job that ran unprotected still goes red. These two are the only assertions in the repo placed after their job's work — if you add a third, say why here.
Both exceptions were previously "resolved" by simply leaving the assertion off those jobs. Do not go back to that: an unasserted block job is indistinguishable from an enforcing one, which is the whole failure #487 was.
A related trap, one step earlier: harden-runner is step 0, so if it fails the job can die before
emitting anything. In ai-scan that skipped the fail-closed labeler, because its guard reads
needs.gate.outputs.run. The fix is continue-on-error: true on that harden-runner so the gate
step still emits its outputs, with the assertion immediately after to fail the job and set
enforcement=failed. Non-blocking there is not "unprotected and ignored": the job still goes red,
it just gets to record why first. Whenever a downstream always() job keys off an upstream job's
outputs rather than only its result, check what happens if step 0 dies.
That fix covers RUNTIME failure only, and the distinction is not pedantic. Action resolution
and download happen during job initialization, before any step is added to the job. A failure
there fails the job with no step ever running, so continue-on-error — a step property — never
applies and the assertion never executes. Do not describe the flag as covering "download" or
"bootstrap" failures; an earlier revision of this section did, and it was wrong.
Residual, stated rather than papered over. If
harden-runnercannot be prepared at all, the item goes unscanned and unflagged, and no guard can fix it: every job that would carry the flag pins the same action, solabelfails initialization too. This is vendor-availability shaped rather than attacker-reachable — the pin is a SHA served from GitHub's own CDN — but it is a genuine hole in the fail-closed path and it should be named when the posture is described.
Worth tracing the whole matrix when touching any of this, because these paths look alike and only some should flag:
What fails in ai-scan | label runs? | Correct outcome |
| harden-runner at runtime (gate) | yes, via enforcement=failed | flag — nothing was enforcing |
| harden-runner preparation | no — label cannot initialize either | residual above; nothing reaches it |
| the assertion (gate or scan) | yes | flag — enforcement was not proven |
the scan job | yes (result is failure, not skipped) | flag — empty verdict hits the enum default |
| the gate's budget read | no | no flag — counter trouble fails open (rule 4) |
| budget spent, or bot author | no | no flag — nothing was scanned and nothing is wrong |
Both lines are required; either alone is a false pass. agent.json holds the policy the
pre-step decided, serialized after every policy decision, so it catches a downgrade — which is
what #487 was. agent.status is written only once the firewall rules are actually installed,
and it is the line that catches the other failure: agent.json is written before the agent
starts, so if the agent fails to come up, the pre-step logs timed out, prints the agent log and
still exits 0, leaving block in the config and no firewall.
Note the actor there, because it is easy to state wrongly: the pre-step never reverts
anything — it contains no revert call at all (grep -c RevertChanges src/setup.ts → 0). Every
revert is the agent's; the pre-step only makes one visible by printing agent.log. When
debugging a red assertion, Reverted changes is in the agent's output, not harden-runner's.
What the pair does NOT prove is that the policy stays armed. In step-security/agent, the
sequence is writeStatus("Initialized") in agent.go and then a serve loop whose
error branch calls RevertChanges. So any runtime error after
initialization tears the firewall down while agent.status still reads Initialized and
agent.json still reads block — both assertion lines pass, nothing is enforcing.
Separately, refreshDNSEntries re-resolves every 30s and on failure only logs
failed to insert new ipaddress in firewall, so an allow-listed host whose IPs rotate can
quietly stop being reachable.
Two further false-pass modes are known and not covered, both latent for this repo but worth knowing before anyone leans harder on the check:
- A private repo or fork without an active StepSecurity subscription. The agent re-decides
the policy in-process (
if config.Private && !isActive→ audit) and writesInitializedoutside every policy branch, whileagent.json— written by the pre-step — still readsblock. Both lines pass, audit rules are installed. bestax is public, so this does not bite here; a private mirror would inherit a green check asserting the opposite. denied-endpoints, new in v2.21.0. Set besideallowed-endpointsit is dropped with acore.info; set alone on a non-enterprise org the tier gate drops it too, leaving block with an empty allow list and an empty deny list behind acore.warning. Both files still read as armed. We do not use the input; do not adopt it without re-reading this rule.
Take the assertion for exactly what it is: proof that enforcement was armed at that step, not
a guarantee for the life of a 30-60 minute session. There is no step-level check for the latter —
if a session behaves oddly, read the StepSecurity run report for Reverted changes rather than
trusting the green step.
Do not "fix" this by adding a liveness check, and this note exists because review has now
proposed it twice. Asserting systemctl is-active agent would only cover the fraction of a second
between the agent initializing and this step running; the revert that matters happens minutes
later, mid-session, where no step-level check reaches. In exchange it would hard-code an internal
unit name and bind every block job to the TLS-path binary selection described in rule 1 — so a
StepSecurity outage would turn all of them red at once. The gap is real and is stated here
deliberately rather than papered over with a check that does not close it.
That second failure is not theoretical. The PR that introduced this assertion shipped with only
the jq line and passed green on a job where the firewall had reverted. Which leads to the
sharpest operational rule here:
An unresolvable host in
allowed-endpointsdisables the entire policy. harden-runner's agent aborts and reverts when it cannot resolve an allow-listed domain. A dead entry is not inert —statsig.anthropic.comhad gone NXDOMAIN, and its presence silently turned the firewall off. Before adding a host, resolve it; when a run reportsReverted changesortimed out, suspect the allowlist before anything else.
Where the state actually stands, since "which jobs enforce" was mis-stated repeatedly during
issue #487 and the distinction is load-bearing. Scope: this covers the AI/automation workflows
and the jobs adjacent to them, per job, not every job in the directory — the ordinary build
jobs (ci.yml, deploy.yml, test-deploy.yml, visual-regression.yml, story-screenshots.yml,
scorecard.yml, dependency-review.yml) carry no harden-runner and are out of scope here.
- Enforcing and asserted — every
ai-scanjob (gate,scan,label), everyai-triagejob (gate,triage,publish,cleanup),claude-repro(authoronly),deploy-worker(deploy),supply-chain(consumer-sbomandsign-sbom),security-txt-expiry(check),create-vite-drift(check),auto-close-duplicates(auto-close),claude(claude),claude-implement(implement),bestaxbot-reply(respond),claude-review(review),claude-pr-loop(fixandverify),mcp-registry(publish),codemod-registry(validateandpublish),review-converged(sync),skills-publish(relay,generateandpublish),templates-publish(generateandpublish). The command below lists them, andgrep -rn "disable-sudo-and-containers: true" .github/workflows/lists the jobs that setdisable-sudo-and-containers. - Audit, deliberately, pending a measured allowlist — none, as of #578. That issue closed the
group by measuring every member instead of guessing for them. They all now use the same
allowlist, and the gap between that and what the measurement produced is the part worth
keeping: the runs surfaced a strict subset of the application hosts the Claude-session jobs
above already used, and
nodejs.orgappeared in no run at all. It is on the list anyway, and that unmeasured host is why "measured" is not the same as "complete". None ofai-scan/scan,ai-triage/triageorclaude-repro/authorrunsactions/setup-node, so their list never needed it; all six of #578's jobs do run it, and setup-node falls back tonodejs.org/distwhen the toolcache misses and theactions/node-versionslookup fails. The audit runs could not surface that, because the toolcache hit every time and an unexercised path leaves no endpoint in any log. Review caught it, not measurement. Keep the group here rather than deleting it: audit remains the correct starting point for an existing live job whose egress has never been observed, and the recipe under this rule is what keeps it a short state rather than an open-ended one. What it is not is a resting place, which is the whole reason #578 existed. - No harden-runner at all, and these are two different groups — do not merge them into one
"API-only" line, which understates the second:
- Genuinely API-only — no checkout, no repository code:
on-slop,auto-label-claude-prs,close-stale-bestaxbot-prs,stale;claude-repro'sprepareandcleanup;claude-pr-loop'ssweep,gateandhalt;supply-chain'sattach-sbom(it moves release artifacts and runs nothing). - They check out and EXECUTE repository code —
claude-repro'spublish(scripts/sanitize-repro-draft.mjs, over attacker-influenced text while holdingissues: write),claude-pr-loop'shandoff(scripts/handoff-message.mjs), andsupply-chain'ssbom(installs the monorepo, runs SBOM generators) andverify-provenance(installs published packages, runs verification scripts). Calling any of these API-only — as this list did twice — understates the unmonitored execution and egress surface in the one inventory meant to state it precisely.auto-close-duplicateswas the fifth here until it was given an enforced policy; it is now in the first group, and it is the only one that has moved.
- Genuinely API-only — no checkout, no repository code:
Do not maintain any of this by hand; it has now been wrong three times, and each time the error moved a code-executing job into the harmless-looking group. Derive it:
# every job with a harden-runner, and its policy
grep -rl "harden-runner@" .github/workflows/ | sort
# candidate code-executing jobs: a checkout is the tell, since nothing here
# runs repo code without one
grep -rln "actions/checkout@" .github/workflows/ | sortThen read the jobs — and read them, rather than grepping for node, because a node -e inside a
comment is enough to misclassify a job (claude-pr-loop's gate has exactly that and runs no
code at all).
The authoritative check is that no block job lacks the assertion; parse the YAML rather than
eyeballing it, because the two are in different jobs and sometimes several steps apart.
on-slop and auto-label-claude-prs are the two to look at first if this group is ever
worked: both run on pull_request_target — the highest-privilege trigger class here — with
write scopes. They hold no model token, which is why they are not in the group above, but they
are not "API-only" in the harmless sense either.
claude-repro deserves the emphasis: only its author job is hardened. publish — the job
that runs the sanitizer over attacker-influenced text and holds issues: write — has no egress
policy at all. Anyone citing "claude-repro enforces egress" is overstating it by every job but author.
Keep this inventory correct when you add a job, and qualify per job rather than per workflow. Its
first version omitted bestaxbot-reply and named claude-repro unqualified — a table that
quietly misses a model-token job, or implies coverage a workflow only partly has, is worse than
no table, for the same reason a comment that overstates its mechanism is.
Citing egress-block as a control is now legitimate for the first group only, and only for
what it actually does: it narrows where a step that does not escalate can send data, without
bounding it ("What the agent allows beyond the list" says what reaches past a job's list, root
included), and it does nothing about what a session does with an allow-listed host (see I1).
Widening an allowlist remains a security change under rule 2, and sign-sbom's list is still
assembled by reading the actions rather than from a measured run.
So are mcp-registry's publish list, codemod-registry's validate and publish lists,
skills-publish's relay, generate and publish lists and templates-publish's generate
and publish lists, which no run had exercised when they were written.
The codemod-registry, skills-publish and templates-publish comments give the reason for each host.
Verify rather than assume, on any run: the assertion step passes, and harden-runner's post-step
prints the effective EgressPolicy:.
What disable-sudo-and-containers and its check prove
Each job's comment points here. The setting closes the ways a step gets root past the list ("Root, and containers" under "What the agent allows beyond the list", below), and none of the paths that need no root. Citations are to harden-runner at the pin and to step-security/agent at the version the pin installs.
- The
agent.jsonline shows that harden-runner took the input: the pre-step writes the field from it (src/setup.ts), and the agent reads it from that file (config.go). Without the line, a dropped input would surface only as "sudo still works". Like theegress_policyline, it shows what the pre-step decided, not what the agent did. - The sudo and
docker.socktests wait forInitialized, because the agent closes both before it writes that status (disableSudoAndContainersinsudo.go). They show both were closed when the step ran, not that they stay closed: a later agent revert, which the flag's own Docker purge might set off, puts sudo back along with the firewall (RevertChangesinagent.go), and as with the policy above, no step-level check sees it. - They cover sudo and
docker.sock, not every way to reach a container.containerd.sockis not tested: on the hosted runners checked so far, the runner user cannot write it whether or not the agent closed it, so a test there would tell us nothing. - They depend on the runner image as well as the pin: on how the image grants the runner
sudo, and on the modes it gives the sockets. So a runner image change, not only a pin bump, can
turn every job that sets the flag red at once. In
ai-scan, a failure ingateorscanflags the item, as their other assertion failures do. Insupply-chain'ssign-sbomit leaves the SBOMs unsigned:attach-sbomstill attaches them with a warning, and the release still ships. - The agent also purges Docker in the background (
uninstallDockerinsudo.go), alongside the job's first steps. It runs whenever the agent starts with the flag, including on runs that stop early and run no repository code, and it holds the dpkg lock while it runs. A step in such a job has no sudo for apt in any case. - The post step skips the
agent.servicejournal. harden-runner prints it withsudo journalctlonly when neither sudo setting is on (src/cleanup.ts), so a job that sets the flag does not print it.agent.logstill prints.
What the agent allows beyond the list
A block job's allowed-endpoints names hosts the job adds to the firewall. Others are added
beside them: the agent's, below, and the Actions cache host, which the pre-step may append. The
list is not a bound on the job's egress. Everything in this section is about the open-source
agent, step-security/agent at the version the pin installs, read from its source.
When the pre-step selects the closed-source agent-ebpf instead (rule 1), none of it can be
checked from here.
The agent adds these to every block job's list, each on port 443 (addImplicitEndpoints):
- The meta list. GitHub's
actionsdomains as StepSecurity's/github/metaserves them at startup, minus any ending ingithubusercontent.com, which is why a job that needsobjects.githubusercontent.comstill lists it. It has includedgithub.com,*.github.com(which coversapi.github.com),ghcr.io,*.githubapp.com, and the exact entriesproductionresultssa0.blob.core.windows.netup toproductionresultssa<N>.blob.core.windows.net, N being wherever the list stops at the time. The agent resolves each of those at startup, as it does any exact entry (below), and admits the address it gets on 443 when the firewall goes up, before the job looks anything up. They are shared storage front-end addresses, so other storage accounts served from them get through too ("Addresses, not names", below). And each one is a further entry that reverts the firewall if it fails to resolve at startup. The list is StepSecurity's, not ours, and can change with no diff here: a run'sPost Harden runnerlog shows what that run fetched (fetched GitHub meta domains), andcurl -s https://agent.api.stepsecurity.io/v1/github/metawhat it would fetch now. - The built-in entries, present even when that fetch fails:
codeload.github.com,*.actions.githubusercontent.com,actions-results-receiver-production.githubapp.comandproductionresultssa*.blob.core.windows.net. They carry the Actions cache and the runner control-plane hosts "Measuring an allowlist" names, excepthosted-compute-*.githubapp.com, which no built-in entry matches: the meta list's*.githubapp.comadmits it when the fetch succeeds. - StepSecurity's telemetry hosts,
agent.api.stepsecurity.ioandprod.app-api.stepsecurity.io, unless the job setsdisable-telemetry: true(grep -rn disable-telemetry .github/workflows/lists any that do).
A wildcard entry admits a host when the job looks it up. Every other entry, the job's own
included, is resolved at startup, and one that does not resolve reverts the firewall (rule 10),
so the meta list is the other place to look when every block job fails its assertion at once. A
failed meta fetch narrows the policy rather than opening it, which is why a job still lists a
GitHub host it needs. StepSecurity's global blocklist is applied to every entry, the job's own
included, so it can drop a host a job lists, also with no diff here
(removing globally blocklisted in the agent log).
These are the known paths past a job's list. They are what reading the source has turned up so far, not a complete account, so add to them rather than treating them as one:
- GitHub, under any account. The meta list's
github.com,*.github.comandghcr.iotake writes from any account whose token a step holds, the job's own or one the step brought with it: a gist, a push to a repository, a package. It needs no root, so neither sudo setting below closes it. For a job whose own list names none of those hosts, the meta list is what opens it. - Any storage account whose name starts
productionresultssa. The built-in wildcard matches on prefix and suffix alone (matchWildcardDomain), and Azure account names run to 24 characters, so an account anyone can create matches it and takes uploads. That is a way out of every block job here, and it is older than the meta list. - Root, and containers. In a job that leaves
disable-sudo-and-containersoff, any step cansudo, and root can change the firewall rules themselves.disable-sudotakes away less than its name says: it empties the runner's sudoers file (sudo.go) and leaves the Docker socket open to the runner user, which is in thedockergroup, so a step can still get root in a container (docker run --privileged --net=host) and rewrite the rules from there. A container on a user-defined Docker network needs nosudoto get out: its traffic arrives from that network's bridge, and the agent'sDOCKER-USERrules match onlydocker0(firewall.go).disable-sudo-and-containersalso closes the Docker and containerd sockets, and it is the only setting that shuts both.grep -rn disable-sudo .github/workflows/lists any job that sets either. - Public DNS over HTTPS. The firewall (
addBlockRulesin the agent'sfirewall.go) accepts TCP 443 to8.8.8.8,8.8.4.4and1.1.1.1from the agent's own user, which is root, so a step that can get root, throughsudoor a host-network container, can carry data out in DNS queries and leave the rules as they are. - Addresses, not names. The rules admit IP addresses, so a host served from the same address as an admitted one, such as another storage account behind the same front end, is reachable too.
- Connections already open.
ESTABLISHED,RELATEDtraffic is accepted, so a connection open before the firewall went up stays usable. - Azure platform and private addresses.
168.63.129.16,169.254.169.254and the private ranges are accepted with no lookup (addBlockRules). - The Actions cache host. harden-runner's pre-step appends the cache's storage host to the
job's list when it can look the host up (
Adding cacheHostin its log).
A comment says a host is not listed; it never says a job cannot reach a host. Where a step can get root, no host is out of reach, and the paths that need no root still carry data out. A comment may give the reason a job leaves a host off its list and point here. It does not restate what the agent adds.
Measuring an allowlist
An audit-mode run's endpoints are in the run log, not only the StepSecurity web UI — the
Post Harden runner step dumps the agent log, which records every DNS resolution and connection.
So this is a command rather than a transcription exercise, and it is the answer to "where does
the list come from" wherever this rule says a list must be measured:
gh run view <run-id> --log \
| grep -oE 'domain: [a-z0-9.-]+\., pid' \
| sed 's/domain: //; s/\., pid//' | sort -uTwo things about reading its output, both learned assembling the #578 lists:
Runner infrastructure appears in that log and need not be allow-listed.
results-receiver.actions.githubusercontent.com,productionresultssa<N>.blob.core.windows.net,run-actions-<N>-azure-*.actions.githubusercontent.comandhosted-compute-*.githubapp.comare the runner talking to its own control plane. The agent's built-in entries admit the first three without a listing ("What the agent allows beyond the list"). The meta list's*.githubapp.comadmits the last when the meta fetch succeeds, and one run passed with it not admitted at all (run 33286967625, below), which suggests the runner can do without it but does not prove it ("One run is not a measurement"). Leave them off a list. The blob host has no one name a list could carry, since N varies between runs. An exact entry for one of the others is not free, though: the agent resolves every exact entry at startup, and one that does not resolve then reverts the firewall (rule 10), so an entry the job never needed can still turn its policy off. Run 33221210633's log showsauto-close-duplicatesatblockconnecting toapi.github.comandgithub.comand nothing else. That shows the job did not use these hosts, not that leaving them off works for a job that does. Run 33286967625, below, is the evidence for leaving them out.What leaving them out costs was read from a run, not assumed. Run 33286967625 ran
claude-reviewatblockon harden-runner v2.21.0, whose agent had the built-in entries and no meta list. Itssetup-nodestep restored the pnpm cache throughresults-receiverandproductionresultssa6beforepnpm install --frozen-lockfilecompleted. Neither was on the job's list, and the pre-step had added a different blob host. So the built-in entries carry the cache on their own. The same run's agent log readsdomain not allowedfor ahosted-compute-*.githubapp.comhost, and the job passed, which suggests the runner can do without that host and does not prove it. After a pin bump, check the built-in entries are still inaddImplicitEndpointsbefore leaning on that run. Worth stating plainly because the first revision of this bullet predicted a cache miss and called it an accepted cost. That was a guess, it was wrong, and the run is what showed it. If you catch yourself writing down what a policy will cost, go and read a run instead.- One run is not a measurement. Vendor telemetry samples, so a host can be absent from a
run and present in the next —
telemetry.vercel.com(turbo) showed up exactly once across the six #578 runs. Prefer denying telemetry at the source (CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC,TURBO_TELEMETRY_DISABLED) over allow-listing a host you would rather not talk to; that is the standing call here, and it also keeps the next audit report clean for whoever reads it.
Then resolve every host before it goes in the list, per the rule above.
When the agent drops a job's first connections
On some hosted runners the agent drops a step's connection to a host the job allows. The agent
admitted one address for the host and the step's own DNS lookup returned another, so the agent log
reads ip address dropped and the step gets connection refused. It shows at the start of a job,
and a later re-resolution (refreshDNSEntries, above) can admit the address. The upstream report is
step-security/harden-runner#708, and #1258 has this repo's runs.
The cost here was a misread. The checks that re-verify the triggering user's live role read a failed
call as role none, so a dropped run logged that the user lacked access and skipped its work as if
they were untrusted. Each check now calls lookup_role. Its waits add up to 95 seconds and each
call is capped at 10 seconds, so at worst it takes 155 seconds before it fails the step with an
::error:: that says the API call failed. scripts/role-lookup.test.mjs computes that worst case
from the function, and a step that calls it needs a timeout longer than that.
- An answer from the API,
readandnoneincluded, is handled as before. - A 404, which the endpoint returns for a renamed or deleted account, reads as
none. - A rate limit (403 or 429) is retried like a dropped connection.
- Any other 4xx fails the step at once, since a retry won't change it.
scripts/role-lookup.test.mjs runs each check against a stub gh, holds the copies of
lookup_role equal, and checks each step's timeout against the worst case. The copies stay inline:
each one runs before any checkout, and rule 1 keeps security steps in the workflow.
In the deep review the steps after the role check read the API the same way. The dedupe step's read
of the reviews is the job's first call on an ai-loop opened event, since the role check calls
nothing there. The focus step reads the comments and looks up each steer author's role. A failure
there stops the run rather than picking a review mode. A missing steer, or an author misread as
untrusted, can pick the wrong one, such as a verify pass where a full review was asked for.
What it does not cover:
- A later step's host can be dropped the same way: a checkout of
github.com, the npm registry, the model API. The retries cover only the GitHub API reads above. - On those runners a job can also sit in the runner's post-job cleanup, every step finished, until
its
timeout-minutesends it. No step timeout reaches that. Where the job's own timeout is long, the role lookup, the deep review's dedupe and focus steps, and the checkout carry short step timeouts, so that a step which hangs fails on its own. The job timeouts are unchanged.
Review checklist for a workflow change
- Does any allowlist grow? If yes, name the credential in that job and justify each entry.
- Does a model session gain the ability to execute, fetch, or reach the network?
- Does model or issue text reach a comment body? Through what sanitizer?
- Is the posting identity
GITHUB_TOKEN(inert) or a PAT (re-triggering)? - New repository variable? Gate is
== 'on'-shaped, and the docs table lists its unset default. - Action SHAs match the repo-wide pin, and the version comment is truthful — run both greps in rule 1, including the repo-wide one that covers every action rather than the one you thought to type.
- New
blockjob carries the effective-policy assertion (rule 10), placed per that rule's placement section; security-relevant steps stay in the workflow rather than moving behind a local composite action (rule 1 — pins under.github/actions/are fine and one exists, so this is about per-job review, not pin location). - New
blockjob setsdisable-sudo-and-containersand checks it in that assertion, or says above its harden-runner which step needs sudo or Docker (rule 10;scripts/harden-runner-sudo.test.mjschecks both). - New verdict path fails closed; new counter path fails open.
- Security comments claim exactly what the mechanism delivers — no more.
That last item is not padding. Three separate review rounds on #361 flagged comments that overstated their mechanism: a flag described as blocking "every AI entry point" when third-party reviewers never see it, a session called "read-only" when its token was write-scoped, and an encoding check described as a proof when it is a backstop. A comment that overstates its control is worse than no comment, because the next reader stops checking.
How to be exact: claim less
Exact means claiming less, not explaining more. The way to make a security comment true is to delete the clause that overstates it, not to add the qualifiers that would make it true.
- A security comment names the control and, when the control is a third party's, cites where
it lives, in one line:
# matched against the run actor by check-human-actor in anthropics/claude-code-action at the pinned SHA. Do not restate the third party's internals; the restatement rots the moment the pin moves, and nobody re-reads it then. - A count lives in a command, never in a sentence. Write the grep that produces it (rule 10's
egress-policy: blockgrep is the model). A hand-maintained number is wrong within months, and this file has been caught doing exactly that. - Evidence (dates, tallies, measurements) goes on the issue and is linked. A comment says what the mechanism is; the issue says how we know. A run id may stand in a workflow comment as the receipt for the flip it justified, and nowhere in a guide.
- One home per fact. A mechanism is explained once, where it is configured; every other
mention is a pointer (
# same list as fix; rationale there). Two copies are two things to keep equal, and a reviewer will find the day they differ. - Claim the case in front of you. "Every", "only", "always" and "exactly one" are one counterexample from false; "the review arm" survives, "every arm" does not.
- Never cite a line number in prose. Lines move on the next edit; a step
id, a job name, a rule number, or a heading does not. - A PR body says what changed and why. State an invariant as the command that checks it
(
grep -cthe pin), not as today's count of what it found.
The fragile-prose conformance check enforces the count, run-id, and line-reference rules;
mark a deliberate exception with bestax:count-ok on the same line and say why.
Where the rest is documented
- Design and operation of the AI loop, and the repository-variable table — the
ai-development guide in
docs/docs/guides/getting-started/. - User-facing security posture —
SECURITY.md. - Release and publish pipeline —
ci.yml, plusVERSIONING.mdfor the semantic-release contract. Note thatmainis protected by a repository ruleset whose only automation bypass is the release GitHub App, and its token is minted after install and build so repo-owned build code can never reach it. bestax-mcp's MCP Registry listing is published from each release bymcp-registry.yml, which is kept out ofci.ymlon purpose: npm's trusted publisher namesci.yml, and the header ofmcp-registry.ymlsays why the third-party publisher binary must not run under it. - Codemod Registry.
codemod-registry.ymlpublishes bestax-migrate's wrapper package by manual dispatch. It is kept out ofci.ymlfor the reason its header gives: npm's trusted publisher namesci.yml, and the third-partycodemodCLI must not run under it. - The bestax plugin.
skills-publish.ymlpushes the treescripts/gen-skills-repo.mjsgenerates to allxsmith/bestax-skills, with a deploy key held in theskills-publishenvironment, which admitsmainonly. The generator runs in one job and the key sits in another, which runs no repository code. The workflow's header gives the triggers, that split, why a release reaches it through a dispatch, and why the environment has no required reviewer. - The templates mirror.
templates-publish.ymlpushes the treescripts/gen-templates-repo.mjsgenerates to allxsmith/bestax-templates, which StackBlitz opens the ported templates from, in the same two-job shape asskills-publish.yml: a deploy key held in thetemplates-publishenvironment, which admitsmainonly, and a key job that runs no repository code. It has no release trigger and no relay. Its header points toskills-publish.ymlwherever the reasoning is the same.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

