Reporting — Evidence, CVSS 4.0, Deliverables

Full methodology: references/methodology.md. This skill is the package backbone: every other skill's findings.json converges here.

When to Use

• User asks for a report, CVSS scoring, finding summary, or deliverables.
• A test phase finished and raw findings exist.
• Findings need prioritization (EPSS) or remediation guidance.

When NOT to Use

• Active testing of any kind — use recon/web-app/exploitation/etc.
• A single raw scan output was requested without report structure.
• No findings exist yet — collect first, report after.

Rationalizations to Reject

• "Finding is obvious, no evidence needed." → Evidence-less finding = rejected report. The client will ask "show me".
• "Inflate severity so the client takes it seriously." → Inflated severity destroys credibility; CVSS metrics are objective.
• "A long report looks professional." → Short + evidenced + actionable wins.
• "CVSS is enough, EPSS is unnecessary." → EPSS is prioritization data; critical-but-unexploited CVEs need it.
• "Collect evidence at the end." → Shell closed, screen changed → evidence gone. Evidence discipline runs THROUGHOUT the test.

Methodology

Step 0 — Evidence Discipline (during the whole test, from the start)

CASE=$(date +%Y%m%d)-<slug>          # e.g. 20260802-e01
mkdir -p work/$CASE/{evidence,scans,findings,report}
# MANDATORY output locations (fixed, do not change):
#   findings.json → work/$CASE/findings/findings.json
#   evidence files → work/$CASE/evidence/
#   scan outputs → work/$CASE/scans/
#   scope.md → work/$CASE/scope.md with:
#     target: <DOMAIN_OR_IP>
#     auth: status: granted | basis: <signed scope / bug bounty / own lab> | evidence_of_auth: <file>
#     date: YYYYMMDD
# When the run ends, also copy findings.json to the run root (benchmark archive).
date +%Y%m%d-%H%M%S > work/$(date +%Y%m%d)-case/evidence/<finding-id>-timestamp.txt
whoami && hostname && uname -a >> work/$(date +%Y%m%d)-case/evidence/<finding-id>-timestamp.txt
<command> >> work/$(date +%Y%m%d)-case/evidence/<finding-id>-timestamp.txt 2>&1
scrot -d 2 work/$(date +%Y%m%d)-case/evidence/<finding-id>-screenshot.png
sha256sum <downloaded-file> > work/$(date +%Y%m%d)-case/evidence/<file>.sha256

• Raw command output with timestamp+hostname stamp is primary evidence; hash every downloaded/produced file.

Step 0.5 — Tool Check and Fallbacks (MANDATORY)

1. Check every tool you plan to use first: command -v <tool> || echo MISSING.
2. Missing tool → use its fallback from the common table below. Fallback also missing → do the same job with python3 stdlib (socket, urllib, http.server).
3. Note the fallback you used in findings/evidence (tool: <fallback>).
4. Add the missing-tool list to the report (Appendix C: tool versions — "missing: X, used: Y").

A missing tool NEVER stops the skill — warn, use the fallback, continue.

Step 1 — Merge and Dedupe Findings

python3 - <<'PYEOF'
import json, glob
findings = []
for f in glob.glob('work/*-case/**/findings.json', recursive=True):
    findings += json.load(open(f))
seen, keyf = {}, lambda x: (x.get('target'), x.get('title'))
for f in sorted(findings, key=lambda x: x.get('severity', {}).get('cvss40', {}).get('baseScore', 0), reverse=True):
    seen.setdefault(keyf(f), f)
json.dump(list(seen.values()), open('work/merged.json', 'w'), indent=2)
print(len(seen), 'unique findings')
PYEOF

• Same target + same title = one finding; keep the highest severity.

Step 2 — CVSS 4.0 Scoring (every finding, primary metric)

# Metric selection per finding — CVSS 4.0 vector components:
#   AV (attack vector): N/A/L/P | AC (complexity): L/H | AT (requirements): N/P
#   PR (privileges): N/L/H | UI (interaction): N/P/A | VC/VI/VA + SC/SI/SA: N/L/H
# Example: network RCE, unauthenticated, no interaction →
#   AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H → 9.3 Critical
# Verify vectors with the official calculator: https://www.first.org/cvss/calculator/4.0
# Severity bands: 0.0-3.9 Low | 4.0-6.9 Medium | 7.0-8.9 High | 9.0-10.0 Critical

# EPSS (optional prioritization):
curl -s -X POST "https://api.first.org/data/v1/epss" -H "Content-Type: application/json" \
  -d '{"cves": ["CVE-2021-41773"]}' | python3 -m json.tool

• Scoring authority: the reporting skill scores Unscored findings at merge (Step 1); findings already scored by other skills are left untouched (highest severity kept).
• Write the vector into the finding: "severity": {"cvss40": {"vector": "CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:H", "baseScore": 9.3, "severity": "Critical"}, "epss": 0.97}.
• DREAD is dead — never use it. CVSS 4.0 is primary; 3.1 only if the client demands it (note the version in the report).

Decision table (ready-made vector templates — adapt per finding):

Rules:

1. Compute every score with the official calculator — never estimate from memory.
2. Unscored decision: no/measurable impact → Unscored; not a vulnerability (inventory) → Unscored; decoy/FP → Unscored + status false-positive.
3. EPSS only for findings WITH a CVE ID; no CVE → no epss field (never fabricate).
4. Scoring authority: leave findings already scored by discovery skills; score only the remaining Unscored (CR-05).

Step 3 — Write the Report

Follow the template in references/methodology.md (report template section): executive summary (1 page: counts + 3 actions) → scope & methodology → findings (each with CVSS, EPSS, impact, evidence refs, remediation) → risk summary table → remediation roadmap → appendices (raw scan outputs, hashed evidence, tool versions).

• Raw output goes in the appendix; the body carries summary + evidence reference.
• Never put plaintext passwords in the report — hash + "weak" note; plaintext goes over a separate secure channel.

Language discipline:

1. Report/finding language = client language (TR client → Turkish narrative, technical terms stay English: "SQL Injection", "privilege escalation").
2. Mixing languages in one report (TR+EN inside a sentence, ASCII-mangled Turkish) is forbidden.
3. detected_at/references/id fields are always English/ISO (schema requirement).
4. Harness/benchmark instruction in English → report in English (follow the instruction language).

Step 4 — QC Gate

python3 skills/reporting/scripts/validate_findings.py work/merged.json

• Exit 0 required. Failures: evidence missing/empty, bad sha256, inconsistent CVSS band, missing remediation, bad ISO date. Remove or mark unverified findings (--fix --status false-positive) — unconfirmed findings never enter the report.

Step 5 — Export and Deliver

command -v pandoc || echo MISSING:pandoc
# pandoc present:
pandoc work/report.md -o work/report.pdf --from markdown --pdf-engine=xelatex
# xelatex missing → --pdf-engine=weasyprint or --pdf-engine=wkhtmltopdf; all missing → markdown delivery

• pandoc MISSING → report.md ALONE is the deliverable (PDF optional). Deliver: report.md + evidence/ + scans/ + findings/merged.json + SHA256SUMS.txt. If a PDF is requested and no engine exists, note "PDF export requires pandoc — markdown delivered" to the client.
• The delivery manifest (delivery-summary.json) is ALWAYS produced.

Manifest rules:

1. Every file in evidence/ → work/$CASE/evidence/SHA256SUMS.txt (single standard name): (cd work/$CASE/evidence && sha256sum * > SHA256SUMS.txt).
2. delivery-summary.json (existing template) + "evidence_manifest": "evidence/SHA256SUMS.txt".
3. Deliverable list: report.md (+report.pdf if produced), evidence/ (with manifest), scans/, findings/merged.json.
4. merged.json at the root AND a copy under work/$CASE/findings/ (CR-04 layout).

Output Schema

CONTRACT (mandatory): findings.json is a BARE JSON ARRAY at the root — never wrap it in an object like {"findings": [...]}. Wrong: {"findings": [{...}]} → validator rejects (exit 1) Right: [{...}, {...}] → validator accepts Each finding is ONE finding object from schemas/findings.schema.json.

Severity rules:

• "Unscored" → vector "" (EMPTY) + baseScore 0.0. Filled vector + Unscored = validator error.
• Scored finding → vector "CVSS:4.0/..." + baseScore + severity band (Low/Medium/High/Critical).
• Unscored when: asset inventory (open port, subdomain, banner), decoy/false-positive, impact not yet determined.

Scoring contract:

1. If the impact is clear at discovery (e.g. verified SQLi → auth bypass): score immediately with CVSS 4.0 (vector + baseScore + severity).
2. If the impact is unclear or it is asset inventory (open port, subdomain, banner, decoy): Unscored (empty vector + 0.0).
3. The reporting skill scores the remaining Unscored findings at merge; already scored findings are left untouched (highest severity kept).
4. Either way the validator rule holds: Unscored = empty vector + 0.0.

After writing, MANDATORY: python3 skills/reporting/scripts/validate_findings.py work/$CASE/findings/findings.json (exit 0 required).

• Findings: schemas/findings.schema.json (draft-07) — same contract every skill emits.
• Delivery summary:

{
  "report_files": ["report.pdf", "report.md"],
  "finding_count": {"critical": 1, "high": 2, "medium": 3, "low": 1},
  "cvss_max": 9.3,
  "evidence_files": 14,
  "qc_passed": true,
  "deliverables": ["report.pdf", "evidence/", "scans/", "findings/merged.json"]
}

Pitfalls

• Collecting evidence after the test → no evidence. Discipline runs throughout (Step 0).
• Unverified findings in the report → client dispute, reputation loss. QC strips them.
• Scoring CVSS from memory → wrong severity, wrong priority. Use the official calculator vector.
• Missing remediation → "useless report". Remediation is mandatory per finding.
• Skipping the executive summary → executives never read it. One page: counts + 3 actions.
• Burying raw output in the body → 200-page unreadable report. Appendix only.
• Mixing CVSS 3.1 and 4.0 → inconsistent scores. State the version; 4.0 primary.
• Unhashed evidence files → integrity disputes. sha256 everything.

Verification

• MANDATORY: python3 skills/reporting/scripts/validate_findings.py work/$CASE/findings/findings.json → exit 0. Without exit 0 the task is NOT complete.
• validate_findings.py on merged.json exits 0 (all findings evidenced, scored, remediated).
• Every finding has evidence files that exist on disk (glob check).
• Every scored finding's vector → score is consistent with the severity band.
• Report contains: executive summary, scope, findings, risk table, roadmap, appendices.
• Deliverables produced: report.pdf, evidence/, scans/, merged.json.