agentleFS
SCStelzSCStelz/security-investigator/.github/skills/mitre-coverage-report/SKILL.md

mitre-coverage-report

Skill252 starsChanged 11 days agoMITView on GitHub

MITRE ATT&CK Coverage Report — YAML-driven PowerShell pipeline gathers analytic rule MITRE tags, custom detection techniques, SOC Optimization recommendations, and alert/incident operational data via az rest/az monitor/Graph API, writes a deterministic scratchpad, LLM renders the report. Covers tactic-level coverage matrix, technique-level drill-down with rule mapping, coverage gap identification, SOC Optimization threat scenario alignment, untagged rule remediation, ICS/OT technique tracking, and MITRE Coverage Score (5 weighted dimensions). Inline chat and markdown file output.

The file

Show its rules and text

We split this skill into the rules it gives an agent

22 rules. About 0 are common, 15 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈15

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    [Query File Reference](#query-file-reference): - All 5 YAML files

    line 77
  2. Distinctive:

    Read the scratchpad file (path printed by PS1 at completion)

    line 115
  3. Distinctive:

    Read SKILL-report.md for rendering templates

    line 116
  4. Distinctive:

    ALWAYS ask the user for output mode: if not specified: inline chat summary, markdown file report, or both (default: both)

    line 152
  5. Very specific:

    ALWAYS ask the user for timeframe: if not specified: the -Days parameter controls the alert/incident KQL lookback (Phase 3).

    line 153
  6. Distinctive:

    Read scratchpad + SKILL-report.md: before rendering — the scratchpad is the sole data source

    line 156
  7. Very specific:

    Read config.json for sentinel_workspace_id, subscription_id, Azure MCP parameters

    line 185
  8. Very specific:

    Verify prerequisites: az login session active, correct subscription set

    line 187
  9. Very specific:

    Run Invoke-MitreScan.ps1 — it handles all 3 phases automatically:

    line 191
  10. Very specific:

    Verify scratchpad has all 3 phase sections (PHASE_1 through PHASE_3)

    line 230
  11. Distinctive:

    Check SCORE section has all 5 dimensions

    line 231
  12. Distinctive:

    If Phase 3 shows FAILED for M4/M5 (token expiry), note this in the report — the Operational dimension defaults to 0

    line 232
  13. Very specific:

    All queries are defined as YAML files in queries/phase1-3/.

    line 249
  14. Distinctive:

    When rendering SOC Optimization threat scenarios, order by coverage gap (recommended minus active) descending, but assign badges based on …

    line 319
  15. Distinctive:

    When Phase 3 data is available, identify paper tiger rules — rules with MITRE tags that have NEVER produced an alert in the lookback period.

    line 334
  16. Very specific:

    When a SOC Optimization threat scenario has State == CompletedByUser, the user has manually marked it as reviewed.

    line 360
  17. Distinctive:

    Operational = 0: when Phase 3 KQL queries fail (token expiry).

    line 403
  18. Distinctive:

    ICS/OT techniques (T0xxx): use a separate numbering scheme from ATT&CK for ICS.

    line 442
  19. Distinctive:

    When rendering recommendations (§6), use these cloud/identity-relevant technique priorities:

    line 447
  20. Distinctive:

    After the report is generated, the user may request an SVG dashboard visualization.

    line 488
  21. Distinctive:

    Only use this path when the user explicitly wants a custom layout, different widgets, or styling the deterministic renderer doesn't support.

    line 513
  22. Distinctive:

    Use the rendered report + scratchpad data to build visualization widgets

    line 516

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

MITRE ATT&CK Coverage Report — Instructions

Purpose

This skill generates a comprehensive MITRE ATT&CK Coverage Report analyzing detection coverage across the ATT&CK Enterprise framework. It inventories all analytic rules and custom detections, maps them to MITRE tactics and techniques, identifies coverage gaps, and provides prioritized recommendations for improving detection posture.

Entity Type: Sentinel workspace (from config.json)

ScopeData SourcesUse Case
Workspace-wide (default)Analytic Rules (REST), Custom Detections (Graph), SOC Optimization (REST), SecurityAlert/SecurityIncident (KQL)Full MITRE coverage analysis
Operational correlationSecurityAlert, SecurityIncidentWhich MITRE-tagged rules actually produce alerts and incidents

What this report covers: Tactic-level coverage matrix with per-tactic technique counts and percentages, technique-level drill-down with rule-to-technique mapping, coverage gap identification against the full ATT&CK Enterprise framework, SOC Optimization threat scenario alignment (AiTM, ransomware, BEC, etc.), untagged rule remediation with AI-suggested MITRE tags, ICS/OT technique tracking, operational MITRE correlation (which rules actually fire), and a composite MITRE Coverage Score.

Complementary to: This skill pairs with the sentinel-ingestion-report skill — ingestion report covers data volume, tier optimization, and cost; MITRE coverage report covers detection posture against the ATT&CK framework. Run both for a complete workspace assessment.


Architecture

 ┌──────────────────────────────────────────────────────────────────┐
 │  YAML query files        PowerShell script         LLM render   │
 │  queries/phase1-3/  ──→  Invoke-MitreScan.ps1  ──→  Phase 4    │
 │  (6 .yaml files)         (~1030 lines)             (SKILL-      │
 │                          • az rest (Sentinel API)   report.md)  │
 │                          • Invoke-MgGraphRequest                │
 │                          • az monitor (KQL)                     │
 │                          • mitre-attck-enterprise.json          │
 │                          • m365-platform-coverage.json (CTID)   │
 │                          ↓                                      │
 │                     temp/mitre_scratch_<ts>.md                  │
 │                     (~35 KB, 18+ sections)                     │
 └──────────────────────────────────────────────────────────────────┘

Execution model:

  • Phases 1-3 (data gathering): Fully automated by Invoke-MitreScan.ps1. Phase 1 uses az rest (Sentinel REST API) and optionally Invoke-MgGraphRequest (Graph API). Phase 2 uses az rest (SOC Optimization API). Phase 3 uses az monitor log-analytics query (KQL).
  • Phase 4 (rendering): LLM reads the scratchpad + SKILL-report.md and renders the report. This is the only phase requiring LLM involvement.

Static reference: mitre-attck-enterprise.json contains ATT&CK Enterprise v16.1 with 14 tactics, 216 techniques, and 475 sub-techniques. The PS1 loads this at startup to compute coverage gaps against the full framework. This file is version-controlled and should be updated when MITRE publishes new ATT&CK releases.

Platform coverage reference: m365-platform-coverage.json is a compact CTID (Center for Threat-Informed Defense) mapping of M365 Defender product capabilities to ATT&CK techniques. Contains detect/protect/respond coverage for 81 detect techniques across 38 capabilities (7 SecurityAlert product groups). Used for the 3-tier platform coverage classification:

  • Tier 1 (Alert-Proven): SecurityAlert from M6 query has MITRE technique attribution — highest confidence
  • Tier 2 (Deployed Capability): Product is active (has alerts) and CTID claims detect coverage for the technique — medium confidence
  • Tier 3 (Catalog Capability): CTID maps coverage but no alert evidence for the product in this workspace — lowest confidence

To rebuild from upstream: download the CTID M365 mapping JSON, transform with PowerShell (group by parent technique, map capabilities to SecurityAlert ProductName). See temp/ctid_raw.json for the raw source.


Companion Files — When to Load

FilePurposeWhen to Load
SKILL.md (this file)Architecture, workflow, rendering rules, score methodology, domain referenceAlways — primary entry point
SKILL-report.mdReport templates (§1-§6), section-to-scratchpad mapping, formatting rulesPhase 4 rendering only
Invoke-MitreScan.ps1PowerShell data-gathering pipeline (Phases 1-3)Execution only — no need to read unless debugging
mitre-attck-enterprise.jsonATT&CK Enterprise v16.1 static referenceReferenced by PS1 at runtime — no manual loading
m365-platform-coverage.jsonCTID M365 platform coverage reference (detect/protect/respond)Referenced by PS1 at runtime — no manual loading

📑 TABLE OF CONTENTS

  1. Quick Start - 3-step execution pattern
  2. Critical Workflow Rules - Prerequisites and prohibitions
  3. Execution Workflow - Phases 0-4
  4. Query File Reference - All 5 YAML files
  5. Output Modes - Inline chat vs. Markdown file
  6. Deterministic Rendering Rules - Rules A-D (mandatory for Phase 4)
  7. MITRE Coverage Score - 5-dimension scoring methodology
  8. Domain Reference - ATT&CK interpretation, tactic priorities, Sentinel-specific mappings
  9. SVG Dashboard Generation - Visual dashboard from completed report

Quick Start (TL;DR)

3-step execution pattern:

Step 1:  Run Invoke-MitreScan.ps1 (Phases 1-3 — data gathering)
Step 2:  Read scratchpad + SKILL-report.md (Phase 4 prep)
Step 3:  Render report incrementally (§1 via create_file, then §2–§6 appended via replace_string_in_file)

Step 1: Run Data Gathering

# From workspace root — run all phases (default: 30 days alert/incident lookback):
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1"

# Specify a custom alert/incident lookback:
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1" -Days 7

# Run a specific phase (for re-runs / debugging):
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1" -Phase 1

Output: Scratchpad file at temp/mitre_scratch_<timestamp>.md (~28 KB, 12 sections).

Timing: Full run takes ~60-90 seconds (varying with REST API response times and KQL auth state).

Step 2: Load Rendering Context

  1. Read the scratchpad file (path printed by PS1 at completion)
  2. Read SKILL-report.md for rendering templates

Step 3: Render Report (Incremental Writes)

Render the report across multiple tool calls — one section per call — to avoid single-call output token limits that truncate large reports:

  1. create_file → header + disclaimer + §1 (Executive Summary, Score, Inventory, Top 3 Recs)
  2. replace_string_in_file → append §2 (Tactic Coverage Matrix)
  3. replace_string_in_file → append §3 (Technique Deep Dive — largest section)
  4. replace_string_in_file → append §4 (Coverage Gap Analysis)
  5. replace_string_in_file → append §5 (Operational MITRE Correlation)
  6. replace_string_in_file → append §6 + Appendix

Apply SKILL-report.md templates to scratchpad data, following Rules A–D. See SKILL-report.md for full section templates and the anchor pattern for each append.

🔴 Verbatim table sections — use the deterministic slicer, never hand-copy. Several report tables (§3 TechniqueTables, §5.1 CombinedTacticCoverage, §5.2 AlertFiring, §5.3 ActiveVsTagged, §5.4 IncidentsByTactic, §5.5 DataReadiness, §5.6 ConnectorHealth) are pre-rendered by the PS1 under ## PRERENDERED in the scratchpad. Copy them with the read-only helper instead of transcribing by hand:

python .github/skills/mitre-coverage-report/slice_scratch.py --scratch temp/mitre_scratch_<ts>.md --list
python .github/skills/mitre-coverage-report/slice_scratch.py --scratch temp/mitre_scratch_<ts>.md --section AlertFiring

The slicer prefers the ## PRERENDERED copy when a section name also exists as a raw data block, strips pipeline scaffolding (<!-- … --> comments, SectionTitle: markers) wherever it appears, preserves #### sub-headings, and collapses blank runs — so the output drops straight into the report as a valid markdown table. Do NOT paste the raw Key | Value | … data blocks (the ones with a <!-- header --> comment and no |---| separator row) — they render as plain text, not tables, and pasting the whole scratchpad tail into one section corrupts the report.

⛔ Do NOT render §1–§6 in a single create_file call. The output will truncate silently. The scratchpad is ~60 KB; the rendered report exceeds the single-call output budget.

🔴 ALL 6 APPENDS ARE MANDATORY. Do NOT stop after §5 — §6 (Recommendations) and the Appendix (Score Methodology, Limitations) are critical and must be appended. After the 6th append, run grep_search for ## 6. Recommendations and ## Appendix on the report file to verify both exist. If either is missing, append the missing content immediately.


⚠️ CRITICAL WORKFLOW RULES - READ FIRST ⚠️

Before starting ANY MITRE coverage report:

  1. Run Invoke-MitreScan.ps1 — this single script handles ALL data gathering (Phases 1-3). The LLM does NOT run queries, transcribe output, or write scratchpad sections
  2. Read config.json for workspace ID, tenant, subscription, and Azure MCP parameters
  3. ALWAYS ask the user for output mode if not specified: inline chat summary, markdown file report, or both (default: both)
  4. ALWAYS ask the user for timeframe if not specified: the -Days parameter controls the alert/incident KQL lookback (Phase 3). Default: 30 days. Phases 1-2 (REST API) are not time-bounded
  5. ALWAYS use create_file for markdown reports (never use terminal commands)
  6. ALWAYS sanitize PII from saved reports — use generic placeholders for real rule names, workspace names, and tenant GUIDs in committed files
  7. Read scratchpad + SKILL-report.md before rendering — the scratchpad is the sole data source
  8. Custom Detections may be SKIPPED — the Graph API requires CustomDetection.Read.All which needs admin consent. If skipped, the report notes this and shows AR-only analysis. Do NOT treat SKIPPED as an error — it's a graceful degradation

Prerequisites

DependencyRequired BySetup
Azure CLI (az)All phases (REST + KQL)Install: aka.ms/installazurecli. Authenticate: az login --tenant <tenant_id> then az account set --subscription <subscription_id>
Azure RBACPhase 1-2 (REST API)Microsoft Sentinel Reader on the workspace (analytic rule inventory + SOC Optimization)
KQL authPhase 3 (az monitor)az login with https://api.loganalytics.io/.default scope (CA policy may enforce re-auth)
Microsoft.Graph PowerShellPhase 1 M2 (Custom Detections)Install-Module Microsoft.Graph.Authentication -Scope CurrentUser. Required scope: CustomDetection.Read.All. PS1 skips gracefully if unavailable
PowerShell 7.0+Script execution#Requires -Version 7.0

🔴 PROHIBITED

  • ❌ Running REST/KQL queries via MCP tools during data gathering — PS1 handles all queries
  • ❌ Writing or modifying scratchpad sections manually — PS1 is the sole writer
  • ❌ Fabricating technique counts, rule names, or coverage percentages
  • ❌ Inventing ATT&CK technique IDs or names not in the reference JSON
  • ❌ Overriding MITRE Coverage Score dimensions — the PS1 computes these deterministically
  • ❌ Rendering the report without first reading the scratchpad file
  • ❌ Reporting "100% coverage" for any tactic unless the data actually shows every technique covered

Execution Workflow

Phase 0: Initialization

  1. Read config.json for sentinel_workspace_id, subscription_id, Azure MCP parameters
  2. Confirm output mode and timeframe with user (pass -Days to PS1; default 30)
  3. Verify prerequisites: az login session active, correct subscription set

Phases 1-3: Data Gathering (automated by PS1)

Run Invoke-MitreScan.ps1 — it handles all 3 phases automatically:

PhaseQueriesDescriptionExecution Type
1M1, M2Rule inventory — Analytic rules with MITRE tactics/techniques (REST), Custom Detection rules with mitreTechniques (Graph, graceful skip)REST + Graph
2M3SOC Optimization — Coverage recommendations with threat scenario context, MITRE tagging suggestions for untagged rulesREST
3M4, M5, M6, M7, M8Operational correlation — SecurityAlert firing counts per rule with MITRE cross-reference, SecurityIncident volume by tactic, platform-native alert MITRE coverage, table ingestion volume for data readiness validation, data connector health from SentinelHealthKQL

Post-processing (automated by PS1):

TaskPhaseDescription
Tactic coverage matrix1For each ATT&CK tactic, count enabled rules and covered techniques against the framework reference
Technique drill-down3Map every framework technique to its covering rules AND pre-compute tier/product annotations from CTID cross-reference
Untagged rule identification1Find rules with no MITRE tactics AND no techniques
ICS technique extraction1Separate T0xxx (ICS/OT) technique mappings
Threat scenario parsing2Extract active/recommended detection counts and per-tactic breakdowns from SOC Optimization
AI MITRE tagging suggestions2Extract suggested tactics/techniques for untagged rules. Cross-reference against Phase 1 actual rule tags to verify if suggestions were applied (emits VerifyStatus: Applied/Partial/NotApplied/NotFound per rule, plus summary counts AR_TagsApplied/AR_TagsPartial/AR_TagsNotApplied/AR_TagsNotFound)
Alert-to-MITRE correlation3Cross-reference firing alerts with Phase 1 MITRE tags
Active tactic coverage3Compute which tactics have rules that actually fire alerts
Platform alert MITRE extraction3Extract MITRE techniques attributed by platform-native product alerts (M6)
Product presence detection3Derive active M365 Defender products from SecurityAlert ProductName
CTID tier classification3Cross-reference active products with CTID mapping to classify techniques as Tier 1/2/3
Combined tactic coverage3Merge custom rule and platform Tier 1/2 coverage per tactic
Data readiness cross-reference3Extract KQL table dependencies from rule queries, cross-reference with M7 ingestion volumes, classify rules as Ready/Partial/NoData
Connector health enrichment3Cross-reference M8 SentinelHealth connector status with Data Readiness — flag "Ready" rules whose feeding connector is degraded or failing
Table tier classification3Cross-reference M9 table tier metadata with rule KQL table dependencies — flag rules targeting Basic/Data Lake tier tables as "TierBlocked" (phantom coverage: rule structurally cannot fire regardless of data volume)
Coverage Score computationAllWeighted composite score from 5 dimensions

Scratchpad output: PS1 writes all results to temp/mitre_scratch_<timestamp>.md (~28 KB, ~12 named sections). See SKILL-report.md for the Section-to-Scratchpad Mapping.

Phase 4: Render Output (LLM)

🔴 MANDATORY — Load scratchpad + report template before rendering:

  1. Read the scratchpad file (path printed by PS1). This single file contains ALL data from Phases 1-3.
  2. Read SKILL-report.md for the complete rendering templates and formatting rules.

Pre-render validation:

  1. Verify scratchpad has all 3 phase sections (PHASE_1 through PHASE_3)
  2. Check SCORE section has all 5 dimensions
  3. If Phase 3 shows FAILED for M4/M5 (token expiry), note this in the report — the Operational dimension defaults to 0

Render — Section-by-Section:

SectionData Source (scratchpad keys)Required
§1 Executive SummaryAll phases + SCORE✅ Coverage Score, Workspace at a Glance, Top 3
§2 Tactic CoveragePHASE_1.TacticCoverage✅ 14-tactic matrix with coverage %
§3 Technique Deep DivePHASE_3.TechniqueDetail (enriched with Tier/TierProducts)✅ Per-tactic technique tables with pre-computed tier badges
§4 Coverage Gap AnalysisPHASE_1.TacticCoverage + PHASE_3.TechniqueDetail + PHASE_2.ThreatScenarios✅ Gaps, priorities, threat scenario alignment
§5 Operational MITRE CorrelationPHASE_3.AlertFiring + IncidentsByTactic + ActiveTacticCoverage + PlatformAlertCoverage + PlatformTechniquesByTier + PlatformTacticCoverage + DataReadiness + DataReadiness_Summary + MissingTables + TierBlockedTables + ConnectorHealth + ConnectorHealth_Summary✅ Which rules fire, platform coverage, combined tactic view, data readiness, tier-blocked phantom coverage, connector health
§6 RecommendationsAll phases✅ Untagged rule remediation, Content Hub suggestions, coverage priorities

Query File Reference

All queries are defined as YAML files in queries/phase1-3/.

YAML Format

id: mitre-m1                                   # Unique identifier
name: Analytic Rule MITRE Extraction            # Human-readable name
description: Fetch rules with tactics/techniques # What it does
phase: 1                                        # Which phase (1-3)
type: rest                                      # rest | graph | kql
url: https://management.azure.com/...           # REST API URL with placeholders
jmespath: value[].{...}                         # JMESPath projection (REST)

Complete Query Inventory

PhaseFileIDTypeDescription
1M1-AnalyticRuleMitre.yamlmitre-m1restScheduled + NRT analytic rules with MITRE tactics, techniques, severity, query text
1M2-CustomDetectionMitre.yamlmitre-m2graphCustom Detection rules with mitreTechniques (graceful skip if auth unavailable)
2M3-SocOptCoverage.yamlmitre-m3restSOC Optimization coverage recommendations with threat scenarios and MITRE tagging suggestions
3M4-AlertFiringByMitre.yamlmitre-m4kqlSecurityAlert firing counts per rule with severity breakdown (30d lookback)
3M5-IncidentsByTactic.yamlmitre-m5kqlSecurityIncident volume by tactic with classification breakdown
3M6-PlatformAlertCoverage.yamlmitre-m6kqlPlatform-native SecurityAlert detections with MITRE technique attribution (excludes custom rules)
3M7-TableIngestionVolume.yamlmitre-m7kql7-day average daily ingestion volume per table from Usage table for data readiness validation
3M8-ConnectorHealth.yamlmitre-m8kqlSentinelHealth data connector fetch status — latest state, success/failure counts, health % per connector (supplements M7 with early-warning connector failure detection)
3M9-TableTierClassification.yamlmitre-m9cliLog Analytics table tier metadata (Analytics/Basic/Data Lake) via az monitor log-analytics workspace table list — identifies tables that analytics rules cannot query

Output Modes

Mode 1: Inline Chat Summary (default for quick requests)

Compact executive summary rendered directly in chat with MITRE Coverage Score and top coverage gaps.

Mode 2: Markdown File Report

Full detailed report saved to reports/sentinel/mitre_coverage_report_<YYYYMMDD_HHMMSS>.md.

Mode 3: Both (default when user says "report" or "generate report")

Inline chat executive summary + full markdown file.

Ask user if not specified:

"How would you like the MITRE coverage report? I can provide:

  1. Inline chat summary — MITRE Score + top gaps in chat
  2. Markdown file — detailed report saved to reports/sentinel/
  3. Both (recommended) — summary in chat + full report file"

Deterministic Rendering Rules

These rules eliminate LLM interpretation variance. Apply them EXACTLY during Phase 4 rendering.

Rule A: Coverage Level Classification

Assign emoji badges to each tactic row in the coverage matrix based on the percentage of techniques covered:

Coverage %BadgeLevel
0%🔴No coverage
1-15%🟠Critical gap
16-30%🟡Partial
31-50%🔵Moderate
51-75%🟢Good
>75%✅Strong

⛔ PROHIBITED: Assigning badges based on "importance" or "this tactic is more relevant." The badge MUST match the percentage threshold table above.

Rule B: Threat Scenario Priority

When rendering SOC Optimization threat scenarios, order by coverage gap (recommended minus active) descending, but assign badges based on completion rate (proportional to scenario size):

Completion RatePriorityBadge
<15%🔴 HighVery early stage — most recommendations unaddressed
15–35%🟠 MediumWork in progress — significant room for improvement
35–60%🟡 LowApproaching healthy coverage for typical environments
≥60%✅ MetStrong coverage — well above realistic implementation targets

Why rate-based? Recommendation counts reflect the full Content Hub template catalogue including templates for vendor products not deployed in the environment (e.g., all firewall vendors). A 609-rule scenario will be permanently 🔴 under absolute-gap thresholds even at 80% coverage. Rate-based badges give proportional, meaningful progress signals.

CompletedBySystem note: CompletedBySystem is a SOC Optimization state, not a rate indicator. Some CompletedBySystem entries have low rates (recommended >> active). Always use the completion rate for badge assignment. The State column is displayed for context but does NOT override the rate-based badge.

Rule C: "Paper Tiger" Detection

When Phase 3 data is available, identify paper tiger rules — rules with MITRE tags that have NEVER produced an alert in the lookback period. These rules are tagged but non-operational, and their coverage is theoretical, not proven.

ConditionClassificationDisplay
Rule tagged with MITRE + 0 alerts in lookback⚠️ Paper tigerNote in technique drill-down
Rule tagged with MITRE + ≥1 alert✅ Operationally validatedNormal display
Phase 3 data unavailable (FAILED/SKIPPED)—Skip paper-tiger analysis, note data gap

⛔ PROHIBITED: Reporting coverage percentages as "validated" when Phase 3 data is missing. If M4/M5 failed, state: "Coverage percentages reflect rule tagging only — operational validation unavailable (Phase 3 KQL queries failed)."

Rule D: Recommendation Ranking

Rank recommendations by impact using this priority order:

PriorityCategoryCriteria
1🔴 Low-rate threat scenariosSOC Optimization scenarios with <15% completion rate. Exclude CompletedByUser scenarios with ≥50% completion rate (Rule E — Reviewed & Addressed). Only include ⚠️ Premature CompletedByUser (<50% rate)
2🔴 Zero-coverage detectable tacticsTactics with 0% coverage AND ✅ Detectable classification (see tactic table). Exclude ⬜ Inherent blind spot tactics (Reconnaissance, Resource Development) — report these as acknowledged limitations, not actionable gaps
3🟠 Untagged rule remediationRules with AI-suggested MITRE tags from SOC Optimization
4🟠 Paper tiger rulesMITRE-tagged rules that never fire (if Phase 3 available)
5🟡 Low-coverage tacticsTactics with 1-15% coverage
6🟡 Content Hub suggestionsTemplate-based rules available for uncovered techniques
7⬜ Inherent blind spot tacticsZero-coverage tactics classified as ⬜ Inherent blind spot. Acknowledge the limitation; suggest compensating controls (threat intel feeds, brand monitoring) only if relevant to the organization

Rule E: CompletedByUser Completion-Rate Gate

When a SOC Optimization threat scenario has State == CompletedByUser, the user has manually marked it as reviewed. However, marking a scenario "complete" after enabling 2/500 recommendations is fundamentally different from enabling 28/46. Use the completion rate (ActiveDetections / RecommendedDetections × 100) to determine rendering treatment:

CompletedByUser + Completion RateTreatmentRationale
≥ 50%🟢 Reviewed & Addressed — render in a separate muted "Reviewed Scenarios" summary below the active gaps table. Exclude from §6 recommendations and Coverage Priority MatrixUser has genuinely triaged the scenario; remaining gap is likely non-applicable templates or platform-only coverage
< 50%⚠️ Premature Completion — render in the main active gaps table with full gap badge + ⚠️ flag in the State column. Include in §6 recommendationsGap is too large relative to recommendations to be a deliberate triage decision

Threshold: 50% is the default. This balances trust in the user's judgment against protection from rubber-stamped completions.

Scratchpad column: CompletionRate is pre-computed by the PS1 and included in the ThreatScenarios row. The LLM reads this value directly — do not recompute it.

Interaction with Rule B (rate-based badges): Rule B still applies for badge assignment on all scenarios. Rule E only controls where CompletedByUser scenarios are rendered (active table vs reviewed summary) and whether they appear in §6 recommendations.

CompletedBySystem scenarios are not affected — they continue to use rate-based badges (Rule B) without the completion-rate gate, since the system assessment is independent of user action.


MITRE Coverage Score

The MITRE Coverage Score is a composite metric (0-100) computed by the PS1 from 5 weighted dimensions. Each dimension scores 0-100 independently, then the weighted sum produces the final score.

Dimensions

#DimensionWeightFormulaWhat It Measures
1Breadth25%(Σ per-technique readiness credit / total ATT&CK techniques) × 100 blended 60/40 with combined platform coverageReadiness-weighted technique coverage. Each technique gets fractional credit based on the best rule covering it: Fired=1.0, Ready=0.75, Partial=0.50, NoData=0.25, TierBlocked=0.0. AR and CD rules follow the same readiness constraints. One firing rule gives full credit even if other rules covering the same technique are NoData
2Balance10%(tactics with ≥1 rule / 14 tactics) × 100Whether coverage spans all kill chain phases or clusters in a few
3Operational30%(MITRE-tagged rules that fired alerts / total MITRE-tagged enabled rules) × 100Whether tagged rules actually produce detections (not paper tigers). Highest weight: directly rewards purple teaming and operationally validated detections
4Tagging15%(rules with MITRE tags / total rules) × 100Completeness of MITRE classification across the rule inventory
5SOC Alignment20%(completed SOC recommendations / total SOC coverage recommendations) × 100Alignment with Microsoft's threat-scenario-driven coverage model

Score Interpretation

Score RangeAssessmentTypical Profile
80-100🟢 StrongBroad coverage, balanced tactics, operationally validated, well-tagged, SOC-aligned
60-79🔵 GoodSolid coverage with some gaps; may have clustering or unvalidated rules
40-59🟡 ModerateSignificant gaps in breadth or operational validation; improvement opportunities
20-39🟠 DevelopingLimited coverage across the framework; many uncovered tactics
0-19🔴 CriticalMinimal detection coverage; urgent investment needed

Score Context Notes

  • Operational = 0 when Phase 3 KQL queries fail (token expiry). Report this: "Operational score 0 reflects data unavailability, not necessarily poor operational coverage."
  • SOC Alignment = 50 (default) when no SOC Optimization recommendations exist. This is a neutral baseline, not a penalty.
  • Breadth score is naturally low because the ATT&CK framework contains 216+ techniques, many of which are endpoint-specific or pre-compromise with limited Sentinel visibility. Do NOT present this as a crisis — contextualize it: "Prioritize coverage by threat scenario relevance rather than pursuing raw percentage."
  • Custom Detections SKIPPED affects Breadth and Tagging dimensions (rules not counted). Note the impact in the report.
  • Platform Coverage is reported as a supplementary metric alongside the MITRE Score (not folded into the 5 dimensions). The scratchpad includes Platform_Tier1/2/3, Platform_ActiveProducts, and RuleBasedPlusPlatform_Coverage. Render this in §1 and §5 per SKILL-report.md templates. The CTID tier classification requires m365-platform-coverage.json — if the file is missing, platform tiers default to empty and the report notes the limitation.

Domain Reference

ATT&CK Enterprise Tactic Kill Chain Order

The 14 ATT&CK Enterprise tactics in kill chain order (PS1 uses this ordering for all output):

#Tactic (Sentinel API name)Display NameCloud/Identity RelevanceDetectability
1ReconnaissanceReconnaissance🟡 Low — mostly pre-compromise; limited Sentinel visibility⬜ Inherent blind spot
2ResourceDevelopmentResource Development🟡 Low — attacker infrastructure; limited Sentinel visibility⬜ Inherent blind spot
3InitialAccessInitial Access🔴 High — phishing, valid accounts, external services✅ Detectable
4ExecutionExecution🟠 Medium — scripting, cloud admin commands✅ Detectable
5PersistencePersistence🔴 High — account manipulation, app registrations, inbox rules✅ Detectable
6PrivilegeEscalationPrivilege Escalation🔴 High — tenant policy modification, valid accounts✅ Detectable
7DefenseEvasionDefense Evasion🟠 Medium — many techniques are endpoint-focused✅ Detectable
8CredentialAccessCredential Access🔴 High — brute force, token theft, AiTM✅ Detectable
9DiscoveryDiscovery🟡 Medium — account/cloud service discovery✅ Detectable
10LateralMovementLateral Movement🟠 Medium — remote services, internal spearphishing✅ Detectable
11CollectionCollection🟡 Medium — email collection, data from cloud storage✅ Detectable
12CommandAndControlCommand and Control🟠 Medium — application layer protocol, web service✅ Detectable
13ExfiltrationExfiltration🟠 Medium — exfiltration over C2 channel, cloud account✅ Detectable
14ImpactImpact🟠 Medium — resource hijacking (crypto mining), account removal✅ Detectable

Detectability classification:

  • ✅ Detectable: Techniques in this tactic generate observable events in Sentinel data sources (sign-in logs, audit logs, endpoint telemetry, email events, etc.). KQL detection rules can be written and deployed.
  • ⬜ Inherent blind spot: Techniques in this tactic describe attacker activity that occurs outside the monitored environment (e.g., attacker creating fake accounts on external services, acquiring infrastructure). CTID mappings for these tactics are typically protect/respond capabilities (Conditional Access blocking, PAM restrictions), not detect. No KQL detection rules exist or can realistically be created. Do not recommend deploying rules for inherent blind spot tactics — acknowledge the limitation and recommend compensating controls (e.g., brand monitoring services, threat intelligence feeds) if relevant.

Sentinel-Specific MITRE Mapping Notes

  • Sentinel uses PascalCase for tactic names in the REST API: InitialAccess, CommandAndControl, CredentialAccess. The ATT&CK STIX data uses kebab-case (initial-access). The reference JSON maps between these.
  • Sub-techniques (T1xxx.xxx) are tracked by Sentinel but the REST API properties.techniques field may contain both parent techniques (T1078) and sub-techniques (T1078.004). The PS1 counts at the parent technique level for coverage matrix purposes.
  • ICS/OT techniques (T0xxx) use a separate numbering scheme from ATT&CK for ICS. These are extracted and reported separately since they don't map to the Enterprise framework.
  • Custom Detection mitreTechniques uses the same technique ID format but may specify sub-techniques that analytic rules don't. The PS1 aggregates both sources.

Tactic-Specific Detection Guidance

When rendering recommendations (§6), use these cloud/identity-relevant technique priorities:

TacticKey Sentinel-Detectable TechniquesPriority
InitialAccessT1078 (Valid Accounts), T1566 (Phishing), T1133 (External Remote Services)🔴 Must-have
PersistenceT1098 (Account Manipulation), T1136 (Create Account), T1078 (Valid Accounts)🔴 Must-have
CredentialAccessT1110 (Brute Force), T1528 (Steal App Access Token), T1621 (MFA Request Gen)🔴 Must-have
PrivilegeEscalationT1484 (Domain/Tenant Policy Mod), T1078 (Valid Accounts), T1098 (Account Manipulation)🔴 Must-have
DefenseEvasionT1078 (Valid Accounts), T1484 (Domain/Tenant Policy Mod), T1562 (Impair Defenses)🟠 Important
ExfiltrationT1567 (Exfil Over Web Service), T1537 (Transfer to Cloud Account)🟠 Important
CollectionT1114 (Email Collection), T1213 (Data from Info Repos)🟠 Important

SOC Optimization Threat Scenario Reference

SOC Optimization recommendations map to named threat scenarios. When rendering §4, interpret these:

ScenarioKey Attack PatternPriority Tactics
AiTM (Adversary in the Middle)Session token theft, AiTM phishingInitialAccess, CredentialAccess
BEC (Financial Fraud)Email account takeover for wire fraudInitialAccess, CredentialAccess, Persistence
BEC (Mass Credential Harvest)Large-scale phishing campaignsInitialAccess, CredentialAccess, DefenseEvasion
Human Operated RansomwarePost-compromise hands-on keyboardLateralMovement, CredentialAccess, DefenseEvasion, Impact
Credential ExploitationCredential stuffing, password sprayInitialAccess, CredentialAccess, Discovery
IaaS Resource TheftCloud compute hijacking (crypto mining)CredentialAccess, Persistence, Impact
Network InfiltrationTraditional network-based attacksDiscovery, LateralMovement, C2
X-Cloud AttacksCross-cloud lateral movementCredentialAccess, PrivilegeEscalation, Persistence
ERP (SAP)SAP financial process manipulationInitialAccess, DefenseEvasion

SOC Optimization Recommendation States

StateMeaningReport Treatment
ActiveRecommendation is open and actionableShow as gap — count toward coverage deficit
InProgressUser has started addressing the recommendationShow as in-progress — partial credit
CompletedBySystemMicrosoft's automated assessment found coverage adequateUse rate-based badge (may still show 🔴/🟠/🟡 if completion rate is low). State displayed in table for context
CompletedUser manually marked as completeShow as met — ✅

SVG Dashboard Generation

After the report is generated, the user may request an SVG dashboard visualization.

Trigger: "generate SVG dashboard", "visualize this report", "SVG from the MITRE report"

✅ DEFAULT: run the deterministic renderer (render_dashboard.py)

Do this first — do NOT hand-author the SVG. render_dashboard.py produces the manifest-driven 5-row dashboard non-interactively, parsing every value from the scratchpad + report + svg-widgets.yaml (no hardcoded run data). It is faster, deterministic, and produces a known-good layout. Run it:

python .github/skills/mitre-coverage-report/render_dashboard.py \
  --scratch temp/mitre_scratch_<ts>.md \
  --manifest .github/skills/mitre-coverage-report/svg-widgets.yaml \
  --report reports/sentinel/mitre_coverage_report_<label>_<ts>.md \
  --out reports/sentinel/mitre_coverage_report_<label>_<ts>_dashboard.svg

It reads the donut center, score dimensions, tactic bars, threat-scenario table, and KPI values from the scratchpad, and the Top-3 recommendation cards from the report's ### 🎯 Top 3 Recommendations table (falling back to top threat-scenario gaps if absent). Output is self-contained SVG with explicit fill on every <text>.

ActionStatus
Running render_dashboard.py when the user asks to visualize/generate a dashboard✅ REQUIRED (default path)
Hand-authoring the SVG via the svg-dashboard skill instead of running the script❌ PROHIBITED unless the user explicitly asks for a bespoke/custom layout the renderer can't produce

Fallback — bespoke/interactive dashboards (svg-dashboard skill)

Only use this path when the user explicitly wants a custom layout, different widgets, or styling the deterministic renderer doesn't support. Edit svg-widgets.yaml first if the change is layout/field-level — the renderer reads it at generation time, so many "customizations" don't require hand-authoring.

  1. Load the svg-dashboard skill
  2. Use the rendered report + scratchpad data to build visualization widgets
  3. Recommended widget types for MITRE coverage:
    • Score card — MITRE Coverage Score with 5 dimension breakdown
    • Bar chart — Per-tactic coverage percentages (14 bars)
    • Donut chart — Rule inventory breakdown (AR enabled/disabled, CD enabled/disabled, untagged)
    • Table — Top 5 coverage gaps (tactic + gap %)
    • KPI cards — Total techniques covered, SOC scenarios met, untagged rules

Troubleshooting

IssueSolution
Phase 3 KQL queries fail (token expired)Re-authenticate: az login --tenant <tenant_id> --scope https://api.loganalytics.io/.default
Custom Detections SKIPPEDNormal if Graph API admin consent not granted. Report proceeds with AR-only analysis
SOC Optimization returns 0 recsWorkspace may not have SOC Optimization enabled, or all recommendations are already completed
Breadth score seems low (10-20%)This is typical — 216+ techniques means even well-covered workspaces have low percentages. Focus on threat-scenario-aligned priorities, not raw percentage
ICS techniques appear in outputNormal if Defender for IoT rules are deployed. They're reported separately from Enterprise ATT&CK
az rest returns 403Check RBAC: user needs Microsoft Sentinel Reader on the workspace

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.