Evaluate SDLC Layers

Systematically evaluate the SDLC Layer Separation Architecture implementation and support iterative improvement. Treats the implementation as first-pass until validated.

Arguments

• --dry-run — Run all checks, produce report only. Do not apply fixes.
• --fix — After evaluation, apply safe fixes for broken references, missing metadata, or obvious gaps. Report what was changed.
• (no args) — Evaluate and produce report; offer to fix or delegate fixes.

Evaluation Checklist

Run each check and record PASS / FAIL / SKIP with evidence.

1. Cross-Reference Validation

For each linked path in plugins/development-harness/docs/sdlc-layers/ and related docs:

• sam-definition.md — exists at plugins/development-harness/skills/work-backlog-item/references/sam-definition.md
• plugins/development-harness/CLAUDE.md — exists
• stateless-agent-methodology/research/arl/PROVENANCE.md — exists (sibling repo or configured path)
• Layer 0 docs → TASK_FILE_FORMAT.md — exists at plugins/development-harness/docs/TASK_FILE_FORMAT.md
• Layer 1 → role-resolution-protocol.md — exists in development-harness
• Layer-0 redirect stubs (artifact-conventions.md, task-file-format.md, sam-pipeline.md, arl-touchpoints.md) contain redirect pointers to canonical locations. Validate each redirect target exists.

Evidence: List each path checked and result (exists / 404 / wrong content).

2. Doc Completeness

• Layer 0 content files (6): README, rt-ica-gate, verification-protocol, evidence-discipline, orchestrator-discipline, context-fit-complexity
• Layer 0 redirect stubs (4): sam-pipeline, arl-touchpoints, artifact-conventions, task-file-format — each must contain a redirect pointing to its canonical skill reference location
• Layer 1: All 5 docs present (README, layer-1-overview, linting-discovery-protocol, workflow-pattern-taxonomy, harness-role-mapping)
• ARL: arl-meta-layer.md, arl-human-probing-design.md

Evidence: Glob or Read results for each expected file.

3. Knowledge-Explorer Layer Filter

• uv run research/knowledge-explorer.py list --layer 0 — returns entries with layer: "0"
• uv run research/knowledge-explorer.py list --layer 1 — returns entries with layer: "1"
• Entries without layer metadata are excluded when --layer is used (expected)

Evidence: Paste command output for each.

4. Research Entry Layer Metadata

• evaluation-testing/harness-engineering-openai.md — has layer: "0"
• developer-tools/copier-astral.md — has layer: "1"
• research/README.md — has "Layer Mapping" section

Evidence: Grep for layer: in frontmatter of each.

5. Integration Points

• work-backlog-item SKILL, groom subcommand (references/workflows/groom/start.md) — documents ARL human-probing integration; references arl-human-probing-design
• role-resolution-protocol.md — has "Layer 0 gates apply before role resolution"
• plugins/development-harness/CLAUDE.md — references layer model

Evidence: Grep or Read for key phrases.

6. Consistency with Plan

• Plan deliverables (from attached plan) — compare File and Directory Changes table to actual files
• Dependency order — Layer 0 → Layer 1 → Research → SAM/ARL → ARL probing → work-backlog-item

Evidence: List any plan items not yet implemented or diverged.

Output Format

Produce a structured report:

## SDLC Layer Evaluation Report
Date: {YYYY-MM-DD}

### Summary
- Cross-Reference: {PASS|FAIL|PARTIAL} — {brief}
- Doc Completeness: {PASS|FAIL|PARTIAL}
- Knowledge-Explorer: {PASS|FAIL|PARTIAL}
- Research Metadata: {PASS|FAIL|PARTIAL}
- Integration Points: {PASS|FAIL|PARTIAL}
- Plan Consistency: {PASS|FAIL|PARTIAL}

### Findings
1. [Category] {finding} — {suggested fix}
2. ...

### Recommended Actions
- [ ] {action 1}
- [ ] {action 2}

Iteration

After evaluation:

1. If --fix: Apply safe fixes (broken paths, missing frontmatter fields, obvious typos). Report each change.
2. If no --fix: Present findings; offer to create backlog items or apply fixes.
3. Re-run: After fixes, re-run evaluation to confirm improvements.

Experiments

Flow experiments and learnings live in sam-flow-experiments. Clone via SSH: git clone git@github.com:Jamie-BitFlight/sam-flow-experiments.git. When iterating, consider running experiments against concept fixtures to validate changes.

References

• SDLC Layers
• verify-done — evidence discipline
• work-backlog-item — orchestration pattern (groom subcommand: references/workflows/groom/start.md)