Daily Pipeline

Runs the full public-scan workflow once per day without the user driving it. Mode via $ARGUMENTS: default = autonomous (public actions included); --dry-run = no irreversible public actions; --status-only = Phase 1 only.

Non-negotiable guardrails

1. One scan/day — if a scan already ran today (state file), do status-only.
2. Quality gate on filing — always publish the write-up post; file a courtesy issue / open a PR ONLY when curation found ≥1 real, exploitability-shaped, high-confidence item. Else post-only (clean-scan format).
3. Strict-norm detection — real SECURITY.md / commercial backing / security team → post-only or one-vuln-per-issue, never a grouped issue (dstack #3908 lesson).
4. De-branded issue text — finding-first, code-path note, one footer link max.
5. Never rescan — dedup candidates against docs/scans/ slugs.
6. Kill switch — if .daily-paused exists at repo root, abort.
7. Manual-disclosure backlog is a blocking warning. Count the pending_private_disclosure* entries in reports/.daily_state.json. Any entry pending more than 7 days gets a loud banner at the top of the run (repo, severity, days waiting). Any High/Critical entry pending more than 14 days blocks a new scan entirely — run status-only and tell the user to clear the queue. On 2026-08-21 six drafted reports were found unsent, the oldest 22 days, including an unauthenticated-admin finding; nothing surfaced them because every phase only looked forward at the next scan. If the report cannot be delivered by ANY channel, apply guardrail 10 (unreachable) rather than losing a scan a day.
8. State shape. Run history lives in runs_recent; the runs key is a vestigial empty list — do not write to it. Ad-hoc keys join one of four families: pending_private_disclosure*, withheld_finding_*, excluded_repos/excluded_note, *_note. Drafted disclosure emails live in reports/disclosures/ (gitignored); when one is sent, swap its pending entry for a sent record with date and channel. A post published with its finding withheld while the private send is still pending is pending_private_disclosure_* (the family guardrail 7 counts), even when the channel is user-only and unreachable by the pipeline — never withheld_finding_*, which is for reports ALREADY delivered privately. On 2026-09-16 a 31-day undelivered High (liaohch3/claude-tap) was found mis-filed under withheld_finding_* and had evaded the banner for ~20 runs; there is no Gmail-Sent evidence source for a Twitter DM, so ask the user whether they've already reached out rather than assuming.
9. Channel viability is a SELECTION criterion, adaptive to queue depth. Before committing to a candidate, resolve how a finding would reach the maintainer: gh api repos/{o}/{r}/private-vulnerability-reporting --jq .enabled, plus an email in SECURITY.md (root/.github//docs//docs site), README, FUNDING.yml, profile or org profile (commit noreply aliases do not count). Queue depth 0–2 → any channel is fine. Depth 3+ → require an autonomous channel (PVR on, or a finding that can honestly go in a public issue); skip email-only candidates and say why. At any depth, never pick a candidate with NO channel (PVR off AND no email anywhere AND SECURITY.md forbids public issues) — that combination is the only permanent deadlock the series has produced. Measured 2026-09-18: 14 disclosures went autonomous, 12 needed a human send, and every pipeline stoppage came from the human-gated half (6-report backlog: 25 days, 4 scans lost; claude-tap: 33 days, 3 scans lost). Adaptive on purpose — PVR-on repos skew mature and commercially backed, so a permanent preference would bias the series and drop the small projects that most need the review.
10. unreachable is the only way to clear an undeliverable report. Close a pending_private_disclosure* without delivery ONLY with all four proofs recorded in the entry: PVR {"enabled":false} checked on two separate days; no email in any of the locations above; a public issue forbidden by their own SECURITY.md or self-disclosing; and any remaining contact on a platform the operator has said they do not use (ask, never assume). Then re-key to unreachable_disclosure_<slug> so guardrail 7 stops counting it, and add one line to the public post: "Reported channel unavailable — the maintainer's stated private channel is disabled and no other private contact could be found. Detail remains withheld." Keep the detail withheld — a High with no fix and no maintainer aware serves attackers first. unreachable records a failed delivery; it is not a licence to disclose. If a channel opens later, re-key back and send.
11. A run records itself before it can fail. Open the run record (a runs_recent entry with outcome: "in-progress") the moment Phase 2 picks a target and close it in Phase 6; Phase 0 runs scanner/run_orphan_check.py, which finds records left in-progress by an earlier day and reports/<slug>/ output with no post and no record, and pauses the pipeline (.daily-paused) until the user has looked. From 2026-09-25 to 2026-10-03 seven runs ended in curation before Phase 6 wrote any state, so guardrail 1 never fired and every morning spent a fresh scan.

State

reports/.daily_state.json (gitignored): {"last_run":"YYYY-MM-DD","last_slug":"...","runs":[...]}. If last_run == today, rate-limit to status-only. The run record is opened as soon as Phase 2 picks a target and closed in Phase 6 (same runs_recent entry, real outcome; halted: <reason> on any early stop).

Phase 0 — Preconditions

Abort if .daily-paused. Read state. If already ran today → status-only. Then run .venv/Scripts/python.exe scanner/run_orphan_check.py --write-pause (no flag in --status-only): exit 2 means an earlier run died unfinished — STOP (status-only: report it and carry on).

Phase 1 — Status sweep (always)

gh search prs/issues --author=elfrost (exclude elfrost/ai-patchlab). For each open disclosure updated since last_run, read latest comments/reviews and act:

• maintainer asks for a PR → fork → branch → single-purpose fix → PR referencing the issue (commit author 5491654+elfrost@users.noreply.github.com).
• merged/closed-as-fixed → update docs/scans/<slug>.md timeline + docs/index.md ✅ badge + memory project_first_resolved_disclosure.md.
• rejection → keep honest record (quote response in post, note badge, do not delete). Update outcomes ONLY after upstream merge/close, never on open. Stop here if --status-only.

Phase 2 — Candidate discovery + pre-check

gh search repos --language=python --stars=1500..6000 --sort=updated filtered to AI/agent/LLM/RAG/inference. Drop already-scanned slugs. Responsiveness check (recent closed issues + merged PRs from ≥2 contributors). Strict-norm detection → publication mode. Pick ONE; record why. Then open the run record before Phase 3.

Phase 3 — Scan

.venv/Scripts/python.exe scanner/run_scan.py --from-git-url "<url>" --reports-dir reports/<slug> --min-severity medium (--ignore-file for obvious sample/demo subtrees). Then read reports/<slug>/coverage.json — the authoritative record of what the scan examined, written every run from the scanners' own meta findings and derived before --ignore-file suppression. If complete is false, the finding count is not a clean bill of health: carry the rows verbatim into the post's Scan coverage section and state in one sentence what was not examined. This replaces transcribing the Phase 1 preflight by hand; the preflight still runs, because catching a broken Semgrep before burning a scan beats reporting it afterwards.

Phase 4 — Curate

Group by rule family; auto-flag tests/sample/examples/demos/fixtures as candidate-FP. Inspect top 5 real candidates in-repo via gh api .../contents/<path>. Write per-finding verdicts, then append them to reports/<slug>/verdicts.json as source: "curation" rows — one per rule family with its count, beside the deterministic rows the scanner already wrote. reason_code must come from the closed vocabulary in scanner/verdicts.py:CURATION_REASON_CODES; reuse or extend the module, never invent a code inline. rule is REQUIRED (name the rule family); verdict is one of false-positive/by-design/not-applicable/hardening/real or omitted, and your sentence goes in detail; confirmed-real is first-party only, upstream version drift is dependency-currency. Then validate — .venv/Scripts/python.exe scanner/run_verdict_check.py --check reports/<slug>/verdicts.json must exit 0 — and copy the validated file to corpus/verdicts/<slug>.json, which is committed (reports/ is gitignored). Evaluate the quality gate boolean.

Phase 5 — Publish (gated)

Always: write docs/scans/<slug>.md (including the Scan coverage block copied from reports/<slug>/coverage.json, never hand-written) + prepend a row to docs/index.md + a bullet to docs/scan-log.md (bump both scan counts). If gate TRUE and not strict-norm: file focused de-branded courtesy issue (+ PR for clean one-line fixes). If strict-norm: post-only or one-issue-per-critical. If gate FALSE: post-only. Open docs: PR on elfrost/ai-patchlab, merge, verify publication via gh run list --workflow=pages-build-deployment --limit 1 = success AND the post returns 200 serving the new text. Do NOT gate on pages/builds/latest — on this Actions-published site it reports errored even when the deploy succeeded (2026-08-26).

Phase 6 — Record

Close the run record in reports/.daily_state.json (same runs_recent entry, real outcome); update memory on resolutions/lessons; print a 3-line summary.

Rules

Never rescan a covered repo. Never file when the gate is false. Never skip the strict-norm check before a grouped issue. Always use the noreply commit email. Keep honest records of rejections. In --dry-run, zero irreversible public actions.