improvements-pipeline
Skill17 starsChanged 2 days agoApache-2.0View on GitHub
Maintain the improvements/ upstream-findings pipeline for stellar-raven-codemode. Use when filing or updating service-improvement findings, resolving intake targets, maintaining improvements/intake.json, interpreting improvements lint failures, running recurrence probes, regenerating improvements/INDEX.md, reviewing upstream issues or PRs opened from findings, or handling eval-round and drift-refresh improvement maintenance.
- Reads credentials
In this repository9 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- audit-reviewabilitySkill
- cloudflare-observability-reviewSkill
- golden-truthSkill
- live-drift-resolutionSkill
- retrieval-system-auditSkill
- run-evalsSkill
- truth-maintenanceSkill
The file
We split this skill into the rules it gives an agent
12 rules. About 0 are common, 9 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈9
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 37
Dedupe: Search IDs, titles, and recurrences.
- Distinctive: line 53
Create a new collection only for a verified finding with an identified owner that no existing collection can hold.
- Distinctive: line 60
Calibrate content or code drift: before filing:
- Distinctive: line 63
Run any snippet you call broken against the current dependency and name the failing stage.
- Distinctive: line 64
Search adjacent and repeated prose for the same claim, so the smallest fix leaves no contradiction.
- Very specific: line 95
When the consumer-side workaround is a golden change, land it through golden-truth.
- Very specific: line 111
Search analytics read with the maintenance key are good prevalence evidence.
- Distinctive: line 116
Use this loop on a drift refresh, or when asked whether earlier findings were resolved.
- Distinctive: line 129
Read every recorded URL back from GitHub before you treat it as evidence.
- Distinctive: line 131
Check who wrote a comment you cite.
- Distinctive: line 149
Stay silent on untouched open issues: No reminder, status, backlink-only, or "still reproducible" comments.
- Very specific: line 171
Record a future check: only as a dated .agents/TODO.md entry that names the finding IDs, refs, and exact re-check, written so a stranger …
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Improvements pipeline
improvements/ records upstream service, data, content, and spec gaps for their owners. Own-repo
fixes go to .agents/TODO.md. improvements/README.md is the
charter; read it first. After any edit here, run:
npm run improvements:index # INDEX.md is generated; never hand-edit it
npm run improvements:lint # the gate; add `-- --live` when intake repos or GitHub refs changedStatus is an evidence bar
| Status | Requires |
proposed | A plausible finding, usually from a transcript or review; not yet proven live |
verified | Your own live re-execution against the upstream surface, with the command and response recorded |
reported-upstream | The durable filed ref in evidence |
declined-upstream | The trigger still reproduces and the owner declined; a decline ref and a non-empty disposition. Do not re-file. Revisit only on drift or materially new evidence |
fixed-upstream | A live re-check that shows the fix. A queue of deletion candidates, not an archive |
- GitHub state is not truth. A closed issue or merged PR is a reason to re-run the original trigger; only the live result moves a status.
- A residual after a fix becomes a successor finding. Do not stretch the old one.
- An owner's decline is evidence about placement and scope, not evidence that the fact is false.
- Resolution deletes the active file and adds a receipt to
improvements/resolved.json. IDs are unique across the active tree and that ledger and are never reused: take the next number above the maximum of both.
File a finding
- Dedupe. Search IDs, titles, and recurrences. Update an existing finding before adding one.
Classify the failing surface, then choose the collection and prefix:
Collection Prefix Collection Prefix lumenloopll-skillssk-stellar-light-scoutsls-workers-ai-providerwai-stellar-docssd-canonical-sourcecs-For a web surface:
docs-content(the page is wrong, stale, or omits what it undertakes to explain),docs-search(ranking fails over suitable content),site-content,site-search, orcanonical-source(the defect belongs to the spec, implementation, or product that owns the fact). Acanonical-sourcefinding with no service collection goes inimprovements/canonical-source/with a per-findingrepooverride. Search absence alone is not a content defect: name the canonical source, show why the surface undertakes to expose it, and name the smallest correction. Skills findings target the source repos; a re-pin is not a fix. Create a new collection only for a verified finding with an identified owner that no existing collection can hold.- Write it. Standard frontmatter,
discovered: YYYY-MM-DD, a non-emptyevidencelist, andFinding,Evidence, andRecommendationsections. Give evidence a stranger can reproduce, quantify prevalence ("4 of 11 completed events"), and address the recommendation to the service owner. A GitHub issue or PR URL in evidence requires areported-upstream,declined-upstream, orfixed-upstreamstatus. - Calibrate content or code drift before filing:
- Separate the version where you saw the drift from the version that introduced it, and cite the live successor repo, not an archived one.
- Run any snippet you call broken against the current dependency and name the failing stage.
- Search adjacent and repeated prose for the same claim, so the smallest fix leaves no contradiction.
- Resolve the owner through
improvements/intake.json: a per-finding override wins, then the service repo, then an explicitunclearormixedrule. Leaveunclearwith its reason when no public owner can receive the finding. Never file to this repo to close the loop. File through the script:
npm run improvements:file -- --file improvements/<collection>/<finding>.md --dry-run npm run improvements:file -- --file improvements/<collection>/<finding>.mdThe script renders and scrubs the body, resolves the repo, posts, appends the URL, sets
reported-upstream, and regenerates the index. It refusesunclearormixedtargets without--repoand refuses to re-file a filed record. Do not hand-file a shortened body: the body must keep the automated-content notice, the five sections (Finding,Evidence,Recommendation,Source Record,Resolution Handoff), and both links. The script adds theravenlabel where the target repo has it; a missing label is not a reason to hand-file.- Read the dry run as the maintainer. The title and first paragraph state the affected surface
and defect without eval IDs, internal workflow terms, or clipped transcript text. The smallest
correction comes before optional context. Follow
references/upstream-writing-style.md.
More rules:
- An immutable snapshot link is valid only when the committed blob matches the finding being rendered. Commit first, or omit the snapshot.
- A finding fixed before anyone filed it needs no ceremonial issue. Add dated live evidence and any existing upstream ref that explains the fix.
- A finding that backs a golden partial cap (ADR-0008) does not wait in
verified: re-check it live, dedupe, and file it in the same round, or record why no public owner can receive it. - When the consumer-side workaround is a golden change, land it through
golden-truth. Cite the case file in the finding's evidence and point the case'struth.verified.rootCauseat the finding.
Docs search findings
A content gap goes to the docs repo. A search gap splits by owner:
- Upstream ranking. The search backend ranks or chunks a page badly. Raven has no write path to it. File upstream with the query, the expected page, and the observed hits.
- Raven scoping. The adapter's path scopes, candidate window, or query-term guard hide a page
the backend returned. Fix it in the spec or adapter only as a general mechanism with a measured
win on a free, read-only, retrieval-level A/B against the golden docs pages.
docs/stellar-docs.mddefines the scoping contract. Record the change, the A/B, and the live re-check in the finding's evidence.
Search analytics read with the maintenance key are good prevalence evidence; cite the query and
window. Never print or commit MINTLIFY_ADMIN_API_KEY. It is never a Worker or sandbox binding.
Follow up on filed findings
Use this loop on a drift refresh, or when asked whether earlier findings were resolved.
- Enumerate refs:
rg -n "github.com/.+/(issues|pull)/" improvements. Takereported-upstreamandverifiedfindings first. Include inboundupstream-improvement-readyissues on this repo; they are signals to verify, not proof. Keep one state table in the round ledger:
finding trigger upstream ref ref state PR checks or blocker live re-check repo action next check - Read each ref with
ghor the GitHub tools: state, close reason, linked PRs, latest maintainer comment, review decision, checks. For a PR this repo opened, also note any author action it needs.- Read every recorded URL back from GitHub before you treat it as evidence. Remove dangling refs and never describe an unposted draft as a comment.
- Check who wrote a comment you cite. A line that says "maintainer accepted" must not link one of our own comments.
- For byte-exact text use
gh api ... --jq .body. The GitHub MCPissue_readtool HTML-escapes bodies and strips autolinks and HTML comments.
- Re-run the original trigger:
npm run improvements:probesfor a finding with aprobe, the smallest liveexecuteor service call for an eval-origin finding, or the refreshed inventory for a drift-origin finding. Classify and act:
Outcome Action fixed Set fixed-upstream, add dated evidence and the resolving refstill reproduces Keep the status; add a dated local recurrence closed but unfixed Keep reported-upstream; record why closure did not resolve itsuperseded Link the successor; do not stretch the old finding inconclusive No status change; add a dated .agents/TODO.mdentry that names the next checkinbound handoff Reply with the live result, update the finding only at the evidence bar, close the notification. An evidence-only PR never sets fixed-upstreamwithout a live check- Stay silent on untouched open issues. No reminder, status, backlink-only, or "still reproducible" comments. Comment only for maintainer activity that needs a reply, a claimed fix that needs a verification result, materially new evidence or a correction, or work on our own PR.
Drain
fixed-upstream. A reviewer other than the author re-derives each deletion candidate: reads the finding and the upstream ref, confirms the fix is deployed and not merely merged, re-executes the trigger, scans adjacent behavior for residuals, and reconciles everyrg -n "<finding-id>" .reference (goldens, register, research, intake). Then:npm run improvements:resolve -- --file improvements/<collection>/<finding>.md \ --live-recheck "<dated command/URL/result>" \ --review-evidence "<distinct reviewer and evidence>" \ --references-reviewed --upstream-commented --dry-runDrop
--dry-runonly after the printed resolution comment is posted on every upstream ref. Use--upstream-comment-nafor a finding that was never filed,--upstream-ref-gone <url>for a ref that GitHub now answers with 404, and--resolving-reffor each resolving issue or PR. The resolver appends the receipt, removes the intake override, deletes the file, and regenerates the index. A superseded or partly fixed record stays until its open ref carries a successor link. A regression after deletion gets a new ID that cites the receipt.- Record a future check only as a dated
.agents/TODO.mdentry that names the finding IDs, refs, and exact re-check, written so a stranger can act on it. Nothing schedules these entries; the next round reads the queue. When an upstream rate limit blocks an authorized filing, retry in the current session.
Probes and cadence
In an eval round, file or update findings before the round closes. On a drift refresh, run the probes, refresh statuses from live evidence, and run the live lint.
Optional frontmatter: probe.type: http-text, probe.url, and probe.expect.status, contains,
or excludes. npm run improvements:probes re-checks findings that are neither fixed nor declined.
A hit becomes a dated recurrences entry. A failure or inconclusive result is a review signal, not
a status change. Use --include-declined only on a drift round or with new evidence, and
--service <s> or --exclude-service <s> when the task has a collection boundary.
Intake
improvements/intake.json maps services and per-finding overrides to owner/repo. When you add or
rename a repo, keep the string canonical and run npm run improvements:lint -- --live, which
checks every intake repo and every recorded GitHub URL.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

