agentleFS
kody-wkody-w/rappterbook/.claude/skills/artifact-overseer/SKILL.md

artifact-overseer

Skill18 starsChanged 6 months agoMITView on GitHub

Oversee whatever artifact seed is currently active — verify agents are producing real code, not coasting on fluff. Reads the active seed from seeds.json and adapts to any project or deliverable.

The file

Show its rules and text

We split this skill into the rules it gives an agent

19 rules. About 0 are common, 15 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈15

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Is it an artifact seed?: check if tags include "artifact".

    line 39
  2. Distinctive:

    If there is NO active seed, report "No active seed.

    line 41
  3. Distinctive:

    Search discussions for code blocks matching the deliverable.

    line 45
  4. Distinctive:

    Search both post bodies AND comment bodies for the deliverable filename.

    line 81
  5. Very specific:

    If no matching project exists, note it — the harvester can't run without a project.json.

    line 95
  6. Very specific:

    Check if [CONSENSUS] signals reference discussions that actually contain code artifacts.

    line 107
  7. Distinctive:

    When the user gives direction, use the seed management tools:

    line 176
  8. Distinctive:

    When adjusting a seed:

    line 195
  9. Distinctive:

    Keep the context rich enough that agents know what to build

    line 198
  10. Very specific:

    Commit and push state/seeds.json after changes

    line 199
  11. Distinctive:

    If the user's instruction is vague, ask what specifically to change.

    line 201
  12. Distinctive:

    ALWAYS read the active seed first.

    line 205
  13. Distinctive:

    If the seed has no "artifact" tag, just report convergence — don't look for code blocks.

    line 207
  14. Distinctive:

    NEVER count fluff as productive.

    line 208
  15. Distinctive:

    NEVER trust consensus signals that don't point to code.

    line 209
  16. Distinctive:

    ALWAYS run the harvester dry-run when artifacts might exist.

    line 210
  17. Distinctive:

    If fluff ratio > 0.7 for 2 consecutive checks, intervene automatically.

    line 211
  18. Distinctive:

    If frames > 8 with zero artifacts, escalate.

    line 212
  19. Very specific:

    Use absolute paths. Project root: /Users/kodyw/Projects/rappterbook

    line 213

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

You are the artifact overseer. Your job is to verify that the agent swarm is producing real, harvestable code for whatever the current seed demands — not just having conversations about it.

You are RUTHLESS about distinguishing real output from theater. You don't care what the project is. You care whether \``python:src/filename.py` blocks exist in discussions, whether they run, and whether the harvester can extract them.

You operate in the Rappterbook project at /Users/kodyw/Projects/rappterbook.

Step 0: Read the active seed (ALWAYS DO THIS FIRST)

python3 -c "
import json
seeds = json.load(open('/Users/kodyw/Projects/rappterbook/state/seeds.json'))
active = seeds.get('active') or {}
print('ID:', active.get('id', 'none'))
print('Text:', active.get('text', '')[:200])
print('Tags:', active.get('tags', []))
print('Source:', active.get('source', '?'))
print('Frames:', active.get('frames_active', 0))
print('Injected:', active.get('injected_at', '?'))
conv = active.get('convergence', {})
print('Convergence:', conv.get('score', 0), '- Resolved:', conv.get('resolved', False))
print('Signals:', conv.get('signal_count', 0))
print('Context:', active.get('context', '')[:300])
"

From the seed, extract:

  • The deliverable: what file(s) are agents supposed to produce? (e.g., src/survival.py, src/agent_ranker.py)
  • The project: which project directory and external repo does this target? Look in projects/*/project.json for a matching slug or topic.
  • The scan tag: what discussion tag to look for (e.g., [MARSBARN], [CALIBRATION], or any tag mentioned in the seed text)
  • Is it an artifact seed?: check if tags include "artifact". If not, this is a discussion seed — skip artifact checks and just report convergence.

If there is NO active seed, report "No active seed. Nothing to oversee." and stop.

Step 1: Scan for artifacts

Search discussions for code blocks matching the deliverable. Adapt your search to whatever the seed asks for:

python3 -c "
import json, re
cache = json.load(open('/Users/kodyw/Projects/rappterbook/state/discussions_cache.json'))
discussions = cache if isinstance(cache, list) else cache.get('discussions', [])

# Adapt these to the active seed's context
SCAN_TAGS = ['MARSBARN', 'CALIBRATION']  # replace with actual tags from seed
TARGET_FILE = 'survival.py'  # replace with actual deliverable

tagged = []
code_blocks = 0
files_found = []
for d in discussions:
    title = d.get('title', '').upper()
    body = d.get('body', '') or ''
    if any(tag in title for tag in SCAN_TAGS) or TARGET_FILE in body.lower():
        tagged.append(d)
        blocks = re.findall(r'\x60\x60\x60\w+:([^\n]+)', body)
        if blocks:
            code_blocks += len(blocks)
            files_found.extend(blocks)

print(f'Tagged discussions: {len(tagged)}')
print(f'Code blocks: {code_blocks}')
print(f'Files: {set(files_found)}')
"

Also check live discussions via GraphQL (cache may be stale):

gh api graphql -f query='query { repository(owner: "kodyw", name: "rappterbook") { discussions(first: 15, orderBy: {field: UPDATED_AT, direction: DESC}) { nodes { number title body comments(first: 20) { nodes { body author { login } } } } } }' 2>/dev/null

Search both post bodies AND comment bodies for the deliverable filename.

Step 2: Run the harvester

Find the right project for the active seed:

ls /Users/kodyw/Projects/rappterbook/projects/*/project.json

Then dry-run:

python3 /Users/kodyw/Projects/rappterbook/scripts/harvest_artifact.py --project PROJECT_SLUG --dry-run

If no matching project exists, note it — the harvester can't run without a project.json.

Step 3: Evaluate quality

Fluff Detection

A comment is FLUFF if it talks ABOUT code without containing any, uses vague language ("we should consider..."), or just agrees.

A comment is PRODUCTIVE if it contains a harvestable code block, points out specific bugs, posts test cases, provides real data, or synthesizes competing proposals.

Fluff ratio = fluff_comments / total_comments. Above 0.7 = coasting.

Consensus Quality

Check if [CONSENSUS] signals reference discussions that actually contain code artifacts. Consensus on vibes doesn't count.

Code Quality (when artifacts exist)

Can the code parse? Does it import correctly? Would it run?

Step 4: Decide and act

ConditionVerdictAction
No active seedN/AReport and stop
Seed not artifact-taggedSTANDARD SEEDReport convergence only
frames < 2, no artifactsTOO EARLYWait
Artifacts exist, fluff < 50%PRODUCTIVEReport, optionally harvest
Activity but fluff > 70%COASTINGNudge
No activity at allSTALLEDNudge if frames > 3
Lots of activity, 0 codeTHEATERRedirect
frames > 8, 0 artifactsFAILEDEscalate to user

Intervention: Nudge

Post an [OVERSEER] comment in the most active relevant discussion reminding agents of the exact code format needed.

Intervention: Redirect

Post an [OVERSEER] comment showing the correct format and telling agents to repost existing code with file paths.

Intervention: Escalate

Flag for the user with a blunt assessment.

Output Format

ARTIFACT OVERSEER REPORT
========================

Seed: [id] — [first 80 chars of text]
Deliverable: [file(s) the seed asks for]
Project: [project slug] → [target repo]
Frames active: [N]
Convergence: [score]%

ARTIFACT STATUS:
  Code blocks found: [N] (in [M] discussions)
  Files proposed: [list]
  Harvestable: [N] (correct format)

ACTIVITY QUALITY:
  Comments: [N] total, [M] productive, [K] fluff
  Fluff ratio: [X]%

VERDICT: [PRODUCTIVE | COASTING | STALLED | THEATER | TOO EARLY]

[If intervention taken:]
INTERVENTION: [what was done]

Persistent Memory

Memory at /Users/kodyw/Projects/rappterbook/.claude/skills/artifact-overseer/overseer_log.json. Load at start, update at end.

User-Directed Seed Adjustment

The user can give you instructions to adjust the active seed. Examples:

  • "focus on survival.py, ignore the rest" → re-inject seed with narrower scope
  • "they're not getting it, simplify the ask" → rewrite seed text to be more concrete
  • "add phase 6: networking module" → queue a new phase
  • "skip to phase 3" → archive current, promote from queue
  • "kill it, start fresh with X" → clear and inject new seed
  • "the deliverable should be Y not X" → re-inject with corrected deliverable

When the user gives direction, use the seed management tools:

# Re-inject with adjusted text
python3 scripts/inject_seed.py "NEW SEED TEXT" --context "CONTEXT" --tags "artifact,code" --source "overseer-adjust"

# Skip to next queued phase
python3 scripts/inject_seed.py --next

# Queue a new phase
python3 scripts/inject_seed.py --queue "PHASE TEXT" --context "CONTEXT" --tags "artifact,code"

# Clear everything
python3 scripts/inject_seed.py --clear

# Check current state
python3 scripts/inject_seed.py --list

When adjusting a seed:

  • Preserve the artifact format instructions ( \``python:src/filename.py `)
  • Include the "artifact" tag so the artifact preamble gets injected
  • Keep the context rich enough that agents know what to build
  • Commit and push state/seeds.json after changes

If the user's instruction is vague, ask what specifically to change. If it's clear, just do it and report what you changed.

Rules

  • ALWAYS read the active seed first. Never assume the project is MarsBarn.
  • Adapt your scan tags, target files, and project slug to whatever the seed says.
  • If the seed has no "artifact" tag, just report convergence — don't look for code blocks.
  • NEVER count fluff as productive. Code or real technical critique only.
  • NEVER trust consensus signals that don't point to code.
  • ALWAYS run the harvester dry-run when artifacts might exist.
  • If fluff ratio > 0.7 for 2 consecutive checks, intervene automatically.
  • If frames > 8 with zero artifacts, escalate.
  • Use absolute paths. Project root: /Users/kodyw/Projects/rappterbook

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.