sova-ingest-review
Skill3 starsChanged 2 days agoApache-2.0View on GitHub
Ingest PR review feedback into agent memory for continuous learning. Provide PR number.
In this repository44 other files it gives its agents.
- AGENTS.mdAGENTS.md
- templates/AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- templates/CLAUDE.mdCLAUDE.md
- dashboard-designSkill
- database-patternsSkill
- design-systemSkill
- design-tasteSkill
- issue-templateSkill
- sova-address-prSkill
- sova-after-mergeSkill
- sova-agent-readinessSkill
- sova-audit-instructionsSkill
- sova-debugSkill
- sova-develop-explainSkill
- sova-develop-fullSkill
- sova-developSkill
- sova-extract-knowledgeSkill
- sova-find-taskSkill
- sova-health-auditSkill
- sova-integrate-prSkill
- sova-issueSkill
- sova-new-featureSkill
- sova-optimize-knowledgeSkill
- sova-prSkill
- sova-rearrange-commitsSkill
- sova-researchSkill
- sova-review-fullSkill
- sova-review-prSkill
- sova-reviewSkill
- sova-simplifySkill
- sova-specSkill
- sova-standupSkill
- sova-statusSkill
- sova-testSkill
- sova-update-docsSkill
- testing-patternsSkill
- visual-auditSkill
- dashboard-designSkill
- database-patternsSkill
- design-tasteSkill
- issue-templateSkill
- testing-patternsSkill
- visual-auditSkill
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 2 are distinctive, and 6 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
6
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 81
Read the existing memory: .claude/agent-memory/common-mistakes.md if present, and the cookbook (the relevant domain files of …
- Very specific: line 83
Before writing anything, check whether each lesson is already recorded in a higher tier: .claude/rules/*.md, AGENTS.md, and …
- Very specific: line 92
Update the cookbook in the PRIMARY checkout (resolve PRIMARY_ROOT from git rev-parse --git-common-dir.
- Very specific: line 98
If it already exists, increment its count rather than adding a duplicate.
- Very specific: line 100
If a finding is high-impact (severity >= 8) and .claude/agent-memory/MEMORY.md exists, also add it there
- Very specific: line 107
Extract broader knowledge: Run the sova-extract-knowledge skill for session-wide lessons
- Distinctive: line 111
Only record actionable, specific lessons, not generic advice
- Distinctive: line 113
NEVER use emojis in any output
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Ingest PR Review Feedback
Parse structured review findings from TaskRun records and update agent memory.
Instructions
- Get the PR number from
the arguments provided when this skill is invoked. If empty, ask the user. Validate it is a positive integer before using it anywhere below: reject non-numeric input and ask the user to clarify rather than passing raw argument text into generated code.<PR_NUMBER>in every snippet in this command means this validated integer, never the rawthe arguments provided when this skill is invokedtext. - Query the database for reviewer TaskRun records linked to this PR. Substitute
the validated integer for
<PR_NUMBER>: it becomes part of a Python expression evaluated viapython3 -c, so unvalidated text here is a code injection risk, not just a bad query:# Find the reviewer run's handoff data python3 -c " import asyncio, json, os os.environ.setdefault('SOVA_DATABASE_URL', 'sqlite+aiosqlite:///.claude/sova.db') from sova.db.session import init_db, get_session from sova.db.models import TaskRun from sqlalchemy import select async def main(): await init_db(run_migrations=False) async with await get_session() as session: stmt = select(TaskRun).where( TaskRun.pr_number == <PR_NUMBER>, TaskRun.role.in_(['reviewer', 'command:review-pr']), TaskRun.status == 'done', ).order_by(TaskRun.id.desc()).limit(1) run = (await session.execute(stmt)).scalar_one_or_none() if not run or not run.handoff_json: print('NO_FINDINGS') return print(json.dumps(run.handoff_json, indent=2)) asyncio.run(main()) " - If the output is
NO_FINDINGS, do not stop here: the SOVA reviewer may simply not have run a TaskRun for this PR while external reviewers (CodeRabbit, human reviewers) still left comments. Continue to step 4 (there is nothing to parse from this source, so skip it) and step 5. Only report "No reviewer findings found for PR #N (no SOVA reviewer TaskRun and no external review comments)" and stop if step 5 also finds nothing. - Parse the
pending_findingsarray from the handoff JSON (skip this step if step 2 returnedNO_FINDINGS). Each finding has:file: file pathline: line numberseverity: 1-10 scorecategory: type of issue (bug, style, performance, etc.)description: what the issue issuggestion: how to fix it
Also fetch external review comments (CodeRabbit, human reviewers). Review bodies and inline (line-anchored) comments come from two different sources:
gh pr viewonly returns top-level review bodies, and inline comments must be fetched separately via the PR review-comments API:gh pr view <PR_NUMBER> --json reviews,comments --jq '(.reviews[] | {author: .author.login, state: .state, body: .body}), (.comments[] | {author: .author.login, body: .body})' gh api --paginate repos/<OWNER>/<REPO>/pulls/<PR_NUMBER>/comments --jq '.[] | {author: .user.login, path: .path, line: .line, body: .body}'--paginatematters here: a CodeRabbit review on a sizable PR routinely exceeds the API's default 30-per-page limit, and without it everything past the first page is silently dropped.Normalize each external comment into the same
{file, line, severity, category, description}shape step 4's findings use, so steps 6-9 have a defined input on this path too (an external comment has noseverity/categoryfield of its own):file/line: the inline comment'spath/linefields above; a top-level review body with no inline anchor has neither.severity/category: derive from a stated reviewer label where one exists (CodeRabbit's_Critical_/_Major_/_Minor_or its_Potential issue_marker map to roughly 8/5/4 and categorybug), otherwise default to severity 4, categorystylefor unlabelled human prose. Do not use LLM summarization to invent a severity the comment never stated (the Rules section below forbids LLM summarization for parsing structured data; this default is how an unstructured comment still gets a defined, documented value rather than an improvised one).description: the comment body itself.
- Classify findings into memory categories:
- Severity >= 7: likely a "common_mistake", so check
.claude/agent-memory/common-mistakes.md(or the cookbook's mistakes section) for existing entries - Severity 1-6 with "style" or "naming" category: "style preference"
- Severity 4-6 with "bug" category: "review pattern" worth codifying
- Repeated patterns across findings: "review pattern" worth codifying
- Test-related findings: "test coverage gap"
- Severity >= 7: likely a "common_mistake", so check
- Read the existing memory:
.claude/agent-memory/common-mistakes.mdif present, and the cookbook (the relevant domain files of.claude/agent-memory/cookbook/when that directory exists, otherwise.claude/agent-memory/cookbook.md). - Before writing anything, check whether each lesson is already recorded in
a higher tier:
.claude/rules/*.md,AGENTS.md, anddocs/*-guidelines.md.- Already a rule, and the PR followed it: skip it; nothing to learn.
- Already a rule, but the PR still broke it: the rule is not being enforced. Record it in the mistakes section as an enforcement gap naming the rule and the command that should have caught it (for example "review did not check X"), instead of restating the rule.
- Not recorded anywhere: continue to step 9.
- Update the cookbook in the PRIMARY checkout (resolve
PRIMARY_ROOTfromgit rev-parse --git-common-dir; a worktree's copy of.claude/agent-memory/may be gitignored and is deleted with the worktree):$PRIMARY_ROOT/.claude/agent-memory/cookbook/<domain>.mdwhen that directory exists (domains listed in itsREADME.md), otherwise$PRIMARY_ROOT/.claude/agent-memory/cookbook.md:- Append new findings to the matching domain (no duplicates), one bullet each:
- **<rule as a short imperative>** (PR #123; confirmed: 1): <rule and why>.File by topic, never under a per-PR heading: per-PR sections cannot be deduplicated or promoted. Cite the PR inline. - A recurring mistake goes in
$PRIMARY_ROOT/.claude/agent-memory/common-mistakes.mdas- **<mistake>** (occurrences: N): <how to catch it in a diff>.If it already exists, increment its count rather than adding a duplicate; thesova-reviewskill checks every diff against this list. - If a finding is high-impact (severity >= 8) and
.claude/agent-memory/MEMORY.mdexists, also add it there
- Append new findings to the matching domain (no duplicates), one bullet each:
- Report what was learned and which files were updated.
Cross-References
- Nothing records review learnings automatically: the
sova-address-prskill, thesova-integrate-prskill and thesova-after-mergeskill do not write the cookbook, so run this command when a PR's review taught something worth keeping - Extract broader knowledge: Run the
sova-extract-knowledgeskill for session-wide lessons
Rules
- Only record actionable, specific lessons, not generic advice
- Parse structured data from DB records; do NOT use LLM summarization
- NEVER use emojis in any output
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

