Verification Loop

Contents

• Design philosophy
• When to use
• Project type detection
• Check 1 — Build verification
• Check 2 — Type verification
• Check 3 — Lint verification
• Check 4 — Test verification
• Check 5 — Security scan
• Check 6 — Diff review
• Integration with implement-phase
• Standalone invocation
• Configuration
• Best practices
• Troubleshooting
• Appendix: quick reference
• Related skills

Reference files (open one when a step points to it):

• references/go.md: Verification commands — Go
• references/nodejs-typescript.md: Verification commands — Node.js / TypeScript
• references/python.md: Verification commands — Python
• references/rust.md: Verification commands — Rust

A systematic 6-check verification framework that validates implementation quality across multiple dimensions: build, types, lint, tests, security, and diff review. Catches issues early, ensures code compiles, type-checks, passes linting, runs tests, has no security issues, and contains no unintended changes.

Terminology: this skill uses "Checks" (Check 1–6). Don't confuse with "Phases" (plan phases) or "Steps" (implement-phase steps).

Design philosophy

Defense in depth — each check catches a different category of issue:

Fail fast — checks are ordered by detection speed. Build errors appear in seconds; security scans take longer. Fast checks first provide rapid feedback on common issues.

Project-agnostic — detect project type automatically, apply the matching tooling. Node.js, Python, Go, Rust, and mixed-language projects all work.

Idempotent — running the loop multiple times produces the same result. Each check passes or fails deterministically based on codebase state.

When to use

Automatically invoked by implement-phase as Step 2 (Exit Condition Verification), before integration testing, after implementation subagents complete.

Manually invoked for:

• Validating changes before committing
• Checking code quality after refactoring
• Pre-merge verification
• Reproducing CI/CD failures locally

Don't use for:

• Reading or analyzing code (no changes made)
• Running exploratory tests (use the test runner directly)
• Checking a single file (use individual tools)

Project type detection

Detect project type first, load the matching reference file, then run the 6 checks with those commands.

Detection logic:

detect_project_type() {
  if [ -f "package.json" ]; then
    if [ -f "tsconfig.json" ]; then echo "typescript"; else echo "nodejs"; fi
  elif [ -f "pyproject.toml" ] || [ -f "setup.py" ]; then echo "python"
  elif [ -f "go.mod" ]; then echo "go"
  elif [ -f "Cargo.toml" ]; then echo "rust"
  else echo "unknown"; fi
}

After detection, only load the matching reference file — don't load all four, that's wasted context.

Check 1 — Build verification

Purpose: ensure the codebase compiles, bundles, and produces valid artifacts.

PASS: build command exits 0, no compilation errors, all artifacts generated, no missing-dependency errors.

FAIL: non-zero exit code, compilation/bundling errors, missing or incomplete artifacts, unresolved deps.

Commands → look up the matching language reference file (references/<lang>.md).

Failure handling: parse the error output → identify root cause (missing import? syntax error? type mismatch?) → spawn fix subagent with error context → re-run build. Max 3 retries.

Output shape:

CHECK_1_BUILD_VERIFICATION:
  STATUS: PASS | FAIL
  COMMAND: [command run]
  EXIT_CODE: 0 | [non-zero]
  DURATION: 12.3s
  ARTIFACTS: [list]
  ERRORS: [] | [list of errors]

Check 2 — Type verification

Purpose: ensure type safety across the codebase with no type errors.

PASS: type checker exits 0, no type errors, all assertions valid, no missing type definitions.

FAIL: type errors, missing type definitions for dependencies, invalid type assertions, unreachable code (in strict mode).

Commands → language reference.

Failure handling: parse errors → categorize (missing types → add annotations; type mismatch → fix types; missing definitions → install @types/* or stubs) → spawn fix subagent → re-run. Max 3 retries.

Output shape:

CHECK_2_TYPE_VERIFICATION:
  STATUS: PASS | FAIL
  COMMAND: [command run]
  EXIT_CODE: 0 | [non-zero]
  DURATION: 8.2s
  FILES_CHECKED: [count]
  ERRORS: [] | [ { file, line, error } ]

Check 3 — Lint verification

Purpose: ensure code follows style guidelines and catches potential bugs.

PASS: linter exits 0, no errors (warnings may be acceptable by config), all auto-fixable issues resolved, formatting matches project standards.

FAIL: lint errors, unfixable formatting issues, security-related lint rules violated, complexity thresholds exceeded.

Commands → language reference. Always run auto-fix first, then re-run the check without it and assert clean — this resolves mechanical issues before flagging real ones.

Failure handling: auto-fix → parse remaining errors → spawn fix subagent for non-auto-fixable issues → re-run. Max 3 retries.

Output shape:

CHECK_3_LINT_VERIFICATION:
  STATUS: PASS | FAIL
  COMMANDS: [
    { cmd: [lint-fix], exit_code: 0 },
    { cmd: [lint-check], exit_code: 0 }
  ]
  AUTO_FIXED: [count]
  REMAINING_ERRORS: 0 | [ { rule, file, line } ]
  WARNINGS: [count]

Check 4 — Test verification

Purpose: ensure all tests pass and new code has appropriate coverage.

PASS: every test project reported, all tests pass (exit 0) in every stability run, no unjustified skips, coverage thresholds met (if configured).

FAIL: any test failure, a test project that produced no result (NOT RUN), a test whose result differs between runs (FLAKY), coverage below threshold, test timeout, test infrastructure errors.

Commands → language reference.

Account for every test project. List the test projects first (for .NET from the .sln or dotnet sln list; for workspaces and monorepos from the workspace config) and compare them with the projects that printed a result. Report each missing project as NOT RUN, then run it on its own and report its counts. A run is green only when every test project reported: dotnet test <sln> skips a test project when a project it references fails to run and prints nothing for it, so summing the summary lines reports a green-looking subset.

Check stability. Run the full suite three times by default, and five times when the change touches performance, allocation, timing or concurrency tests. Report every test whose result differs between runs as FLAKY, with its fail/pass counts. When one full run takes longer than about ten minutes, run the full suite once and repeat only the test projects the change touched, plus any that failed or were flaky.

Failure handling: identify failing tests → categorize (test bug → fix the test; implementation bug → fix the code; environmental → fix test setup) → spawn fix subagent → re-run failing tests first (faster feedback), then full suite after fix. Max 3 retries.

Output shape:

CHECK_4_TEST_VERIFICATION:
  STATUS: PASS | FAIL
  COMMAND: [command run]
  EXIT_CODE: 0 | [non-zero]
  DURATION: 45.2s
  TESTS_RUN: [count]
  TESTS_PASSED: [count]
  TESTS_FAILED: [count]
  TESTS_SKIPPED: [count]
  TEST_PROJECTS: [reported]/[expected]
  NOT_RUN: [] | [ { project, counts when run alone } ]
  STABILITY_RUNS: [count]
  FLAKY: [] | [ { test, failed, passed } ]
  COVERAGE: [percent]
  FAILURES: [] | [ { test, file, error } ]

Check 5 — Security scan

Purpose: detect secrets, debug code, and security vulnerabilities before code reaches production.

PASS: no hardcoded secrets, no console.log/print in production code, no debug flags enabled, no known vulnerable deps (if scanned).

FAIL: secrets detected (API keys, passwords, tokens), debug code in production paths, debug flags left enabled, critical vulnerabilities in deps.

5a. Secrets detection

Use whichever tool is installed; pick the first available:

# git-secrets
git secrets --scan

# gitleaks
gitleaks detect --source .

# trufflehog
trufflehog filesystem .

# Manual patterns (fallback)
grep -r "API_KEY\|SECRET\|PASSWORD\|TOKEN" src/ --include="*.ts" --include="*.js"
grep -r "sk-\|pk_\|api_key\|secret_key" src/

Common secret patterns:

# AWS
AKIA[0-9A-Z]{16}
aws_secret_access_key

# API Keys (generic)
[aA][pP][iI]_?[kK][eE][yY].*=.*['"][a-zA-Z0-9]{20,}['"]

# Private Keys
-----BEGIN (RSA|DSA|EC|OPENSSH) PRIVATE KEY-----

# Tokens
(ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9_]{36,}     # GitHub
xox[baprs]-[0-9a-zA-Z-]+                    # Slack

5b. Console / debug code detection

Language-specific — see the matching reference file. Allowed exceptions: logger framework calls (logger.info, logger.debug), error handling in catch blocks (if configured), test files, development-only files.

5c. Debug flag detection

# Environment checks
grep -rn "NODE_ENV.*development\|DEBUG.*true\|SKIP_AUTH" src/

# Disabled security
grep -rn "verify.*false\|secure.*false\|https.*false" src/

# TODO/FIXME with security implications
grep -rn "TODO.*security\|FIXME.*auth\|HACK.*bypass" src/

5d. Dependency vulnerability scan

Per-language commands in the matching reference (npm audit, pip-audit, govulncheck, cargo audit).

Failure handling

• Secrets detected → CRITICAL. Must remove before proceeding.
• Console logs → WARNING. Auto-remove or justify with logger.
• Debug flags → WARNING. Must disable for production.
• Vulnerabilities → varies by severity (CRITICAL blocks; others warn).

Output shape:

CHECK_5_SECURITY_SCAN:
  STATUS: PASS | FAIL
  SCANS_RUN: [
    { scan: "secrets", status: "PASS", findings: 0 },
    { scan: "console_logs", status: "WARN", findings: 3 },
    { scan: "debug_flags", status: "PASS", findings: 0 },
    { scan: "dependencies", status: "PASS", findings: 0 }
  ]
  CRITICAL_FINDINGS: 0 | [count]
  WARNINGS: [count]
  DETAILS: [] | [ { type, file, line } ]

Check 6 — Diff review

Purpose: verify only intended changes are present; no unintended modifications.

PASS: all changes relate to the current phase/task, no unrelated file modifications, file count within expected range, no accidental deletions or additions, no formatting-only changes to unrelated files.

FAIL: changes outside scope, unexpected file additions/deletions, modifications to protected files, large diffs in files that should have minimal changes.

6a. Changed-file inventory

git diff --name-only HEAD~1          # list
git diff --name-status HEAD~1        # with status (A/M/D)
git diff --name-only main...HEAD     # compared to base branch
git diff --stat HEAD~1               # with line counts

6b. Scope verification

# Expected patterns (input to verification)
EXPECTED_PATTERNS=(
  "src/auth/*"
  "tests/auth/*"
  "src/types/auth.ts"
)

CHANGED_FILES=$(git diff --name-only HEAD~1)

for file in $CHANGED_FILES; do
  matches_expected=false
  for pattern in "${EXPECTED_PATTERNS[@]}"; do
    if [[ $file == $pattern ]]; then matches_expected=true; break; fi
  done
  if ! $matches_expected; then echo "UNEXPECTED: $file"; fi
done

6c. File count verification

COUNT=$(git diff --name-only HEAD~1 | wc -l)
if [ $COUNT -lt $MIN_FILES ] || [ $COUNT -gt $MAX_FILES ]; then
  echo "WARNING: Changed $COUNT files, expected $MIN_FILES-$MAX_FILES"
fi

6d. Protected files check

PROTECTED_FILES=(
  "package-lock.json"
  ".env*"
  "*.lock"
  "docker-compose.yml"
  "Dockerfile"
  ".github/workflows/*"
)

for file in $CHANGED_FILES; do
  for protected in "${PROTECTED_FILES[@]}"; do
    if [[ $file == $protected ]]; then echo "PROTECTED FILE CHANGED: $file"; fi
  done
done

6e. Diff size analysis

git diff --stat HEAD~1

git diff --numstat HEAD~1 | while read added deleted file; do
  total=$((added + deleted))
  if [ $total -gt 500 ]; then echo "LARGE DIFF: $file (+$added -$deleted)"; fi
done

Failure handling

1. Unexpected files → review. Valid → update expected scope. Invalid → revert unintended changes.
2. Protected files → require explicit approval.
3. Large diffs → review for scope creep.
4. Missing expected files → investigate incomplete implementation.

Output shape:

CHECK_6_DIFF_REVIEW:
  STATUS: PASS | FAIL
  FILES_CHANGED: [count]
  EXPECTED_RANGE: [min-max]
  SCOPE_VIOLATIONS: 0 | [list]
  PROTECTED_FILES_CHANGED: 0 | [list]
  LARGE_DIFFS: 0 | [list]
  SUMMARY: [ added, modified, deleted counts ]
  DETAILS: [ { file, status, lines } ]

Integration with implement-phase

As Step 2 of implement-phase, this skill runs all 6 checks and returns a structured result.

Input context:

VERIFICATION_INPUT:
  phase_number: 2
  phase_name: "Authentication Service"
  plan_path: docs/plans/auth-implementation.md
  expected_file_patterns: ["src/auth/*", "tests/auth/*"]
  expected_file_range: [5, 15]
  protected_files: ["package-lock.json", ".env"]
  skip_checks: []          # optional

Output format:

VERIFICATION_LOOP_STATUS: PASS | FAIL
CHECKS_RUN: 6
CHECKS_PASSED: 6 | [count]
CHECKS_FAILED: 0 | [count]
FAILED_CHECKS: [] | [ { check, name, error, details } ]
REPORT:
  check_1_build:    { status, duration }
  check_2_types:    { status, duration }
  check_3_lint:     { status, auto_fixed, duration }
  check_4_tests:    { status, tests_run, tests_passed, coverage, duration }
  check_5_security: { status, secrets_found, console_logs, duration }
  check_6_diff:     { status, files_changed, scope_violations, duration }
TOTAL_DURATION: [seconds]

When PASS, the caller (implement-phase) proceeds immediately to Step 3 (Integration Testing) — no user pause. See implement-phase/SKILL.md for the continuous-execution contract.

Retry logic (handled by implement-phase):

1. Invoke verification-loop
2. If any check FAILS:
   a. Identify failure type
   b. Spawn fix subagent with failure context
   c. Re-run verification-loop (or just failed checks)
   d. Repeat until PASS (max 3 retries per check)
3. If all checks PASS:
   a. Proceed to Step 3 (Integration Testing)

Standalone invocation

/verification-loop

# With specific checks
/verification-loop checks:build,types,tests

# Skip checks
/verification-loop skip:security,diff

# Specific project path
/verification-loop path:./packages/auth

Configuration

Auto-detects project type and uses appropriate defaults. Override via project config:

// package.json or .verification.json
{
  "verification": {
    "checks": {
      "build":   { "command": "<command>", "timeout": "<ms>" },
      "types":   { "command": "<command>", "timeout": "<ms>" },
      "lint":    { "command": "<command>", "autoFix": true },
      "tests":   { "command": "<command>", "coverage": { "threshold": 80 } },
      "security":{ "scanSecrets": true, "scanConsoleLogs": true, "excludePatterns": [] },
      "diff":    { "protectedFiles": [], "maxFileCount": 50 }
    }
  }
}

Environment variables:

VERIFICATION_SKIP_CHECKS=<comma-separated>
VERIFICATION_BUILD_TIMEOUT=<ms>
VERIFICATION_TEST_TIMEOUT=<ms>
VERIFICATION_SECRETS_SCAN=<bool>
VERIFICATION_CONSOLE_LOG_SCAN=<bool>

Best practices

Check authors:

1. Order by speed — fast checks first for rapid feedback.
2. Keep checks independent — each runs in isolation.
3. Clear error messages — include file, line, fix suggestion.
4. Support auto-fix where possible.
5. Cache when possible to avoid redundant work.

Verification consumers:

1. Run locally before commit to catch issues before CI.
2. Trust the verification — if it passes, proceed confidently.
3. Fix root causes — don't skip checks, fix the underlying issues.
4. Review security warnings — even non-blocking findings need attention.
5. Keep scope tight — large diffs often indicate scope creep.

CI/CD integration:

1. Run full verification, don't skip checks.
2. Cache dependencies to speed build/install checks.
3. Parallelize where possible (independent checks can run concurrently).
4. Fail fast on first failure to save CI time.
5. Report clearly — surface which check failed and why.

Troubleshooting

Debug mode:

/verification-loop --verbose

# Or
VERIFICATION_DEBUG=true /verification-loop

Appendix: quick reference

Check summary

Minimum viable verification (pre-commit hook)

# Fast path: build + types + lint
npm run build && npx tsc --noEmit && npm run lint

Full verification (pre-merge)

# All 6 checks
npm run build && \
npx tsc --noEmit && \
npm run lint && \
npm test && \
npm run security:scan && \
./scripts/verify-diff.sh

Related skills

• implement-phase — parent skill that invokes verification-loop in Step 2
• devflow:code-review — follows verification-loop in the implement-phase pipeline
• devflow:security-review — deep security analysis (beyond Check 5)