source-completeness
Skill284 starsChanged 6 months agoApache-2.0View on GitHub
Validates that all user-facing surfaces discovered in source code are captured in the behavioral specs. Catches coverage gaps where the analysis pipeline missed features, tools, env vars, CLI flags, or other behavioral interfaces. Run AFTER Layer 3 specs are written, BEFORE sanitization.
- Reads credentials
In this repository21 other files it gives its agents.
- analysis-pipelineSkill
- autonomous-discoverySkill
- behavioral-spec-writingSkill
- binary-analysisSkill
- community-intelligenceSkill
- container-executionSkill
- contract-detectionSkill
- doc-researchSkill
- ecosystem-analysisSkill
- fidelity-validationSkill
- git-archaeologySkill
- incremental-analysisSkill
- multi-source-synthesisSkill
- provenance-methodologySkill
- runtime-observationSkill
- second-pass-reviewSkill
- source-analysisSkill
- spec-sanitizationSkill
- test-suite-analysisSkill
- validation-methodologySkill
- visual-explorationSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
13 rules. About 0 are common, 12 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈12
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 23
Run AFTER Layer 3 deep documentation is complete, ideally as part of Gate 1 or as a pre-Gate-1 completeness check.
- Distinctive: line 51
Spec extraction: every registered name in the tool/command catalog section of the spec.
- Distinctive: line 53
Diff: every registered name in source must appear in the spec.
- Distinctive: line 70
Spec extraction: all env var names from the environment contract spec.
- Very specific: line 92
Spec extraction: every --flag name in the CLI contract spec.
- Distinctive: line 94
Diff: every flag must appear. Hidden flags should be documented as hidden.
- Distinctive: line 102
Spec extraction: every subcommand from the CLI interface spec.
- Distinctive: line 104
Diff: every registered subcommand must appear.
- Distinctive: line 112
Spec extraction: every event type documented in the extensibility or eventing spec.
- Distinctive: line 114
Diff: every dispatched event type must be documented.
- Distinctive: line 118
Source extraction: Search for settings property accesses and config key strings.
- Distinctive: line 126
Source extraction: Search for error class definitions, error type enums, and error message constants.
- Distinctive: line 182
When public documentation is available (docs mode was active), cross-reference the source findings against official docs:
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Source-to-Spec Completeness Validation
Why This Exists
The analysis pipeline (Layers 1-3) reads source code and produces behavioral specs. But it can miss things:
- Registered commands or tools defined in source but not in the tool catalog
- Environment variables read by the code but not documented
- CLI flags defined in the argument parser but not cataloged
- Subcommands registered dynamically or conditionally
- Config keys accessed in settings but not listed in the config spec
- Events dispatched but not inventoried
- Error categories defined but not documented
The fidelity validation (Layer 7) only checks raw→clean. This skill checks source→raw — it goes back to the actual source code and verifies that every discoverable user-facing surface was captured in the specs.
When to Run
Run AFTER Layer 3 deep documentation is complete, ideally as part of Gate 1 or as a pre-Gate-1 completeness check. If gaps are found, dispatch deep-dive agents to cover the missing areas before proceeding.
Also run AFTER Layer 7 fidelity validation to catch anything the fidelity check couldn't find (because it was never in the raw specs).
The Checks
Search patterns depend on target shape. For a single-file bundle, grep the bundle directly. For a source tree, grep recursively across the source root. Patterns also depend on the target language's idioms for registering each surface — the examples below give common patterns for several languages; adjust to the actual target.
Check 1: Tool / Command Registration Completeness
Applies to targets that expose a registered command, tool, or handler surface (CLIs with subcommands, plugin hosts, RPC servers with method registration).
Source extraction: grep for the registration pattern used by the target. Examples:
# Named-object pattern (common in JS/TS/Go handler registration)
grep -roE 'name:\s*"[A-Z][a-zA-Z]+"' <source-root> | sort -u
# Decorator-based registration (Python)
grep -roE '@(command|tool|handler|register)\(' <source-root>
# Macro-based registration (Rust)
grep -roE '#\[(command|tool|handler)' <source-root>
# Assigned-constant pattern (JS/TS)
grep -roE 'var [a-zA-Z0-9_]+="[A-Z][a-zA-Z]+"' <source-root> | grep -v 'Exception\|Error\|Element'Spec extraction: every registered name in the tool/command catalog section of the spec.
Diff: every registered name in source must appear in the spec. Missing names are P0-CRITICAL gaps.
Check 2: Environment Variable Completeness
Source extraction: patterns by language:
# Node/TypeScript
grep -roE 'process\.env\.[A-Z_][A-Z0-9_]*' <source-root> | sed 's/.*process\.env\.//' | sort -u
# Python
grep -roE 'os\.environ(\.get)?\(?"[A-Z_][A-Z0-9_]*"' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u
# Go
grep -roE 'os\.Getenv\("[A-Z_][A-Z0-9_]*"\)' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -u
# Rust
grep -roE 'env::var\("[A-Z_][A-Z0-9_]*"\)' <source-root> | grep -oE '[A-Z_][A-Z0-9_]+' | sort -uSpec extraction: all env var names from the environment contract spec.
Diff: categorize missing env vars:
- User-facing (
DATABASE_*,AWS_*,GCP_*,LOG_*, and similar) → P0: must document - Internal-only (test vars, build vars) → P2: note but don't require
- Standard (
HOME,PATH,SHELL,TERM) → ignore
Check 3: CLI Flag Completeness
Source extraction:
# Quoted flag strings — works across most CLI frameworks
grep -roE '"--[a-z][a-z0-9-]+"' <source-root> | sort -u
# Framework-specific definitions:
# Rust/clap: .arg("--flag-name")
# Python/argparse: add_argument("--flag-name")
# Go/flag: flag.StringVar(&v, "--flag-name", ...)
grep -roE '\.arg\("--[a-z-]+"|add_argument\("--[a-z-]+"|flag\.\w+Var\([^,]+,\s*"--[a-z-]+"' <source-root>Spec extraction: every --flag name in the CLI contract spec.
Diff: every flag must appear. Hidden flags should be documented as hidden.
Check 4: Subcommand Completeness
Applies to CLIs or REPLs with a subcommand surface.
Source extraction: target-dependent. For subcommand CLIs, grep for the subcommand registration or dispatch table (clap subcommands, Cobra, argparse subparsers). For REPL-style interfaces, grep for the command-prefix pattern (e.g., /help, :quit).
Spec extraction: every subcommand from the CLI interface spec.
Diff: every registered subcommand must appear.
Check 5: Event Dispatch Completeness
Applies to targets that dispatch named events — common shapes include observer patterns, signals, message buses, and webhook emission.
Source extraction: grep for event dispatch calls and event-type string constants. Common shapes: emit("event-name", ...), dispatch(EventType.X), pubsub.publish("...", ...), signal/slot registrations, raise EventName(...).
Spec extraction: every event type documented in the extensibility or eventing spec.
Diff: every dispatched event type must be documented.
Check 6: Config Key Completeness
Source extraction: Search for settings property accesses and config key strings.
Spec extraction: Extract all config keys from the configuration spec.
Diff: User-configurable keys must appear. Internal state keys are P2.
Check 7: Error Category Completeness
Source extraction: Search for error class definitions, error type enums, and error message constants.
Spec extraction: Extract all documented error categories.
Diff: User-visible errors must be documented.
Report Format
# Source-to-Spec Completeness Report
## Summary
| Check | Source Count | Spec Count | Missing | Extra |
|-------|-------------|------------|---------|-------|
| Registered commands/tools | N | N | N | N |
| Env Vars | N | N | N | N |
| CLI Flags | N | N | N | N |
| Subcommands | N | N | N | N |
| Events dispatched | N | N | N | N |
| Config Keys | N | N | N | N |
| Error Categories | N | N | N | N |
## P0-CRITICAL Gaps (must fix)
[List all missing user-facing surfaces]
## P1 Gaps (should fix)
[List missing non-critical surfaces]
## P2 Gaps (advisory)
[List internal-only surfaces]
## Verdict: PASS | FAIL
PASS requires zero P0 gaps.Agent Dispatch Pattern
Dispatch as greenfield:analyzer since it needs to read both source code and specs. Can be parallelized:
- One agent per check (7 agents), or
- Batch checks 1-3 (commands/tools, env vars, CLI flags) and checks 4-7 (subcommands, events, config keys, errors) into 2 agents
Relationship to Other Validations
Source Code → [Layer 1-3 analysis] → Raw Specs → [Layer 5 sanitization] → Output Specs
↑ ↑
source-completeness fidelity-validation
(source → raw) (raw → clean)
↑ ↑
"Did we capture it?" "Did we preserve it?"Together these two validations form a complete chain: source-completeness ensures nothing was missed during analysis, and fidelity-validation ensures nothing was lost during sanitization.
Integration with Public Documentation
When public documentation is available (docs mode was active), cross-reference the source findings against official docs:
- Features documented in official docs but missing from specs → P0 gap
- Features in source but not in official docs → likely internal, lower priority
- Features in official docs but not in source → version mismatch or deprecated
This cross-reference is especially valuable because official documentation represents the intended user-facing surface, while source extraction finds the actual surface. Discrepancies in either direction are informative.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

