agentleFS
prime-radiant-incprime-radiant-inc/greenfield/skills/source-completeness/SKILL.md

source-completeness

Skill284 starsChanged 6 months agoApache-2.0View on GitHub

Validates that all user-facing surfaces discovered in source code are captured in the behavioral specs. Catches coverage gaps where the analysis pipeline missed features, tools, env vars, CLI flags, or other behavioral interfaces. Run AFTER Layer 3 specs are written, BEFORE sanitization.

  • Reads credentials

The file

Show its rules and text

We split this skill into the rules it gives an agent

13 rules. About 0 are common, 12 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈12

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Run AFTER Layer 3 deep documentation is complete, ideally as part of Gate 1 or as a pre-Gate-1 completeness check.

    line 23
  2. Distinctive:

    Spec extraction: every registered name in the tool/command catalog section of the spec.

    line 51
  3. Distinctive:

    Diff: every registered name in source must appear in the spec.

    line 53
  4. Distinctive:

    Spec extraction: all env var names from the environment contract spec.

    line 70
  5. Very specific:

    Spec extraction: every --flag name in the CLI contract spec.

    line 92
  6. Distinctive:

    Diff: every flag must appear. Hidden flags should be documented as hidden.

    line 94
  7. Distinctive:

    Spec extraction: every subcommand from the CLI interface spec.

    line 102
  8. Distinctive:

    Diff: every registered subcommand must appear.

    line 104
  9. Distinctive:

    Spec extraction: every event type documented in the extensibility or eventing spec.

    line 112
  10. Distinctive:

    Diff: every dispatched event type must be documented.

    line 114
  11. Distinctive:

    Source extraction: Search for settings property accesses and config key strings.

    line 118
  12. Distinctive:

    Source extraction: Search for error class definitions, error type enums, and error message constants.

    line 126
  13. Distinctive:

    When public documentation is available (docs mode was active), cross-reference the source findings against official docs:

    line 182

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Source-to-Spec Completeness Validation

Why This Exists

The analysis pipeline (Layers 1-3) reads source code and produces behavioral specs. But it can miss things:

  • Registered commands or tools defined in source but not in the tool catalog
  • Environment variables read by the code but not documented
  • CLI flags defined in the argument parser but not cataloged
  • Subcommands registered dynamically or conditionally
  • Config keys accessed in settings but not listed in the config spec
  • Events dispatched but not inventoried
  • Error categories defined but not documented

The fidelity validation (Layer 7) only checks raw→clean. This skill checks source→raw — it goes back to the actual source code and verifies that every discoverable user-facing surface was captured in the specs.

When to Run

Run AFTER Layer 3 deep documentation is complete, ideally as part of Gate 1 or as a pre-Gate-1 completeness check. If gaps are found, dispatch deep-dive agents to cover the missing areas before proceeding.

Also run AFTER Layer 7 fidelity validation to catch anything the fidelity check couldn't find (because it was never in the raw specs).

The Checks

Search patterns depend on target shape. For a single-file bundle, grep the bundle directly. For a source tree, grep recursively across the source root. Patterns also depend on the target language's idioms for registering each surface — the examples below give common patterns for several languages; adjust to the actual target.

Check 1: Tool / Command Registration Completeness

Applies to targets that expose a registered command, tool, or handler surface (CLIs with subcommands, plugin hosts, RPC servers with method registration).

Source extraction: grep for the registration pattern used by the target. Examples:

# Named-object pattern (common in JS/TS/Go handler registration)
grep -roE 'name:\s*"[A-Z][a-zA-Z]+"' <source-root> | sort -u

# Decorator-based registration (Python)
grep -roE '@(command|tool|handler|register)\(' <source-root>

# Macro-based registration (Rust)
grep -roE '#\[(command|tool|handler)' <source-root>

# Assigned-constant pattern (JS/TS)
grep -roE 'var [a-zA-Z0-9_]+="[A-Z][a-zA-Z]+"' <source-root> | grep -v 'Exception\|Error\|Element'

Spec extraction: every registered name in the tool/command catalog section of the spec.

Diff: every registered name in source must appear in the spec. Missing names are P0-CRITICAL gaps.

Check 2: Environment Variable Completeness

Source extraction: patterns by language:

# Node/TypeScript
grep -roE 'process\.env\.[A-Z_][A-Z0-9_]*' <source-root> | sed 's/.*process\.env\.//' | sort -u
# Python
grep -roE 'os\.environ(\.get)?\(?"[A-Z_][A-Z0-9_]*"' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u
# Go
grep -roE 'os\.Getenv\("[A-Z_][A-Z0-9_]*"\)' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u
# Rust
grep -roE 'env::var\("[A-Z_][A-Z0-9_]*"\)' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u

Spec extraction: all env var names from the environment contract spec.

Diff: categorize missing env vars:

  • User-facing (DATABASE_*, AWS_*, GCP_*, LOG_*, and similar) → P0: must document
  • Internal-only (test vars, build vars) → P2: note but don't require
  • Standard (HOME, PATH, SHELL, TERM) → ignore

Check 3: CLI Flag Completeness

Source extraction:

# Quoted flag strings — works across most CLI frameworks
grep -roE '"--[a-z][a-z0-9-]+"' <source-root> | sort -u

# Framework-specific definitions:
#   Rust/clap:    .arg("--flag-name")
#   Python/argparse: add_argument("--flag-name")
#   Go/flag:      flag.StringVar(&v, "--flag-name", ...)
grep -roE '\.arg\("--[a-z-]+"|add_argument\("--[a-z-]+"|flag\.\w+Var\([^,]+,\s*"--[a-z-]+"' <source-root>

Spec extraction: every --flag name in the CLI contract spec.

Diff: every flag must appear. Hidden flags should be documented as hidden.

Check 4: Subcommand Completeness

Applies to CLIs or REPLs with a subcommand surface.

Source extraction: target-dependent. For subcommand CLIs, grep for the subcommand registration or dispatch table (clap subcommands, Cobra, argparse subparsers). For REPL-style interfaces, grep for the command-prefix pattern (e.g., /help, :quit).

Spec extraction: every subcommand from the CLI interface spec.

Diff: every registered subcommand must appear.

Check 5: Event Dispatch Completeness

Applies to targets that dispatch named events — common shapes include observer patterns, signals, message buses, and webhook emission.

Source extraction: grep for event dispatch calls and event-type string constants. Common shapes: emit("event-name", ...), dispatch(EventType.X), pubsub.publish("...", ...), signal/slot registrations, raise EventName(...).

Spec extraction: every event type documented in the extensibility or eventing spec.

Diff: every dispatched event type must be documented.

Check 6: Config Key Completeness

Source extraction: Search for settings property accesses and config key strings.

Spec extraction: Extract all config keys from the configuration spec.

Diff: User-configurable keys must appear. Internal state keys are P2.

Check 7: Error Category Completeness

Source extraction: Search for error class definitions, error type enums, and error message constants.

Spec extraction: Extract all documented error categories.

Diff: User-visible errors must be documented.

Report Format

# Source-to-Spec Completeness Report

## Summary
| Check | Source Count | Spec Count | Missing | Extra |
|-------|-------------|------------|---------|-------|
| Registered commands/tools | N | N | N | N |
| Env Vars | N | N | N | N |
| CLI Flags | N | N | N | N |
| Subcommands | N | N | N | N |
| Events dispatched | N | N | N | N |
| Config Keys | N | N | N | N |
| Error Categories | N | N | N | N |

## P0-CRITICAL Gaps (must fix)
[List all missing user-facing surfaces]

## P1 Gaps (should fix)
[List missing non-critical surfaces]

## P2 Gaps (advisory)
[List internal-only surfaces]

## Verdict: PASS | FAIL
PASS requires zero P0 gaps.

Agent Dispatch Pattern

Dispatch as greenfield:analyzer since it needs to read both source code and specs. Can be parallelized:

  • One agent per check (7 agents), or
  • Batch checks 1-3 (commands/tools, env vars, CLI flags) and checks 4-7 (subcommands, events, config keys, errors) into 2 agents

Relationship to Other Validations

Source Code → [Layer 1-3 analysis] → Raw Specs → [Layer 5 sanitization] → Output Specs
                                          ↑                                        ↑
                                    source-completeness                     fidelity-validation
                                    (source → raw)                      (raw → clean)
                                          ↑                                        ↑
                                    "Did we capture it?"                   "Did we preserve it?"

Together these two validations form a complete chain: source-completeness ensures nothing was missed during analysis, and fidelity-validation ensures nothing was lost during sanitization.

Integration with Public Documentation

When public documentation is available (docs mode was active), cross-reference the source findings against official docs:

  • Features documented in official docs but missing from specs → P0 gap
  • Features in source but not in official docs → likely internal, lower priority
  • Features in official docs but not in source → version mismatch or deprecated

This cross-reference is especially valuable because official documentation represents the intended user-facing surface, while source extraction finds the actual surface. Discrepancies in either direction are informative.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.