agentleFS
bordenetbordenet/superpowers-plus/skills/engineering/evidence-adjudicator/skill.md

evidence-adjudicator

Skill8 starsChanged 6 months agoMITView on GitHub

Synthesizes evidence from all investigator branches into a root cause verdict. Builds reasoning trees, detects contradictions, weighs evidence strength over agent count, and produces a ranked diagnosis. Dispatched by debug-conductor.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 5 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    All completed branch evidence (supporting + disconfirming per branch)

    line 41
  2. Distinctive:

    Do two hypotheses together explain more than either alone?

    line 83
  3. Very specific:

    Before accepting the verdict, apply adversarial-search thinking to the leading hypothesis:

    line 97
  4. Distinctive:

    If this pass produces a credible alternative, demote the leading hypothesis confidence by 0.1 and add the alternative to the reasoning tree.

    line 102
  5. Distinctive:

    All evidence processed and reasoning tree complete

    line 141
  6. Distinctive:

    No hypothesis >0.5 confidence → investigation may need new evidence sources

    line 147

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
skill.md

Evidence Adjudicator (Root Cause Synthesizer)

Role: Synthesize all investigator evidence into a root cause verdict. Weigh evidence strength, not investigator count. Dispatched by: debug-conductor — invoked after investigators complete. Evidence type: RootCauseVerdict (see skills/_shared/evidence-schema.md)

When to Use

Dispatched by debug-conductor after all investigator branches complete. Synthesizes branch evidence into a ranked root cause verdict with reasoning tree.

Adjudication Protocol

Step 1: Collect All Branch Evidence

Receive from conductor:

  • All completed branch evidence (supporting + disconfirming per branch)
  • Branch verdicts and confidence scores
  • Killed/merged branch records (what was tried and abandoned)

Step 2: Build Reasoning Tree

For each hypothesis (branch):

  1. List supporting evidence with confidence scores
  2. List disconfirming evidence with confidence scores
  3. Calculate net evidence strength: Σ(supporting × confidence) - Σ(disconfirming × confidence)
  4. Arrange into a tree structure:
Root: "What caused the incident?"
├── H1: "Event ordering bug" (net: +2.4)
│   ├── [+] Deployment correlation (0.85)
│   ├── [+] Out-of-order events in logs (0.90)
│   ├── [+] Reproduction succeeded 3/3 (0.95)
│   └── [-] Network latency increase (0.30) — addressed: within normal variance
├── H2: "SIP timeout misconfiguration" (net: +0.3)
│   ├── [+] Timeout matches symptom duration (0.70)
│   └── [-] Timeout is effect, not cause (0.85) — strong disconfirmation
└── H3: "Network degradation" (net: -0.5)
    ├── [+] 5ms latency increase (0.20) — weak signal
    └── [-] Latency within normal range (0.70) — strong disconfirmation

Step 3: Identify Critical Divergence Points

Where do investigators disagree?

  1. Find pairs of branches with contradictory evidence about the same fact
  2. At each divergence: which evidence is stronger? (source reliability, specificity, reproducibility)
  3. Prefer evidence from reproduction over correlation (experiment > observation)
  4. Prefer specific evidence over general ("this config change at this time" > "something changed")

Step 4: Check for Compound Root Causes

Many real incidents have multiple contributing factors:

  1. Does removing any single hypothesis leave unexplained evidence?
  2. Do two hypotheses together explain more than either alone?
  3. Example: "Ambiguous tool description" alone → 5% failure. + "High context load" → 20% failure. Compound cause.

Step 5: Validate Disconfirming Evidence

For the winning hypothesis:

  1. Was every piece of disconfirming evidence addressed?
  2. "Addressed" means: explained why it doesn't invalidate the hypothesis
  3. Unaddressed disconfirming evidence → lower confidence
  4. Branches that never produced disconfirming evidence are suspect — may have only looked for confirmation

Step 5b: Adversarial Disconfirmation Pass

Before accepting the verdict, apply adversarial-search thinking to the leading hypothesis:

  1. Invert the hypothesis — "If this is NOT the root cause, what else explains the evidence?"
  2. Challenge the strongest supporting evidence — is there an alternative interpretation?
  3. Look for confirmation bias — did all branches converge too quickly?
  4. If this pass produces a credible alternative, demote the leading hypothesis confidence by 0.1 and add the alternative to the reasoning tree.

Step 6: Produce Verdict

{
  "rootCause": "Call router v2.3.1 async event processing delivers events out of order under load",
  "confidence": 0.88,
  "supportingEvidence": [
    { "source": "deployment-history", "finding": "v2.3.1 deployed 2h before incident", "timestamp": "2026-03-29T10:15:00Z", "confidence": 0.85, "type": "supporting" },
    { "source": "call-router-logs", "finding": "Events arriving out of order", "timestamp": "2026-03-29T10:20:00Z", "confidence": 0.90, "type": "supporting" },
    { "source": "reproduction", "finding": "Reproduced 3/3 with async + load", "timestamp": "2026-03-29T11:00:00Z", "confidence": 0.95, "type": "supporting" }
  ],
  "disconfirmingEvidence": [
    { "source": "network-metrics", "finding": "5ms latency increase — within normal variance, does not explain 2600ms increase", "timestamp": "2026-03-29T10:25:00Z", "confidence": 0.30, "type": "disconfirming" }
  ],
  "alternativeCauses": [
    { "cause": "SIP timeout misconfiguration", "confidence": 0.15, "reason": "Timeout is effect, not cause" },
    { "cause": "Network degradation", "confidence": 0.05, "reason": "Metrics within normal range" }
  ],
  "divergencePoints": ["Was the delay caused by network or application-layer processing?"],
  "gaps": ["Have not tested under exact production load (500 concurrent vs 50 in staging)"]
}

Confidence Calibration

ScoreMeaningRequirements
0.9–1.0Near-certainReproduction confirmed + all disconfirming addressed + no gaps
0.7–0.9StrongReproduction or strong correlation + most disconfirming addressed
0.5–0.7ModerateCorrelation evidence + some gaps remain
0.3–0.5WeakPlausible hypothesis but significant gaps
<0.3InsufficientSpeculation; investigation needs more evidence

Stop Conditions

  • Single root cause identified with ≥0.8 confidence
  • Ranked hypothesis list produced (even if top < 0.8)
  • All evidence processed and reasoning tree complete
  • Token budget exhausted

Escalation Conditions

  • Two hypotheses both >0.6 confidence with contradictory evidence → need human judgment
  • No hypothesis >0.5 confidence → investigation may need new evidence sources
  • Compound root cause detected → flag complexity for operator

Common Patterns This Adjudicator Detects

PatternEvidence Shape
Single clear causeOne hypothesis >0.8, all others <0.3
Compound causeTwo hypotheses each 0.4–0.6, together explain >0.8
Wrong consensusMultiple investigators agree but reproduction fails → all wrong
Minority correctOne low-confidence branch has stronger evidence than high-confidence majority
Insufficient evidenceAll hypotheses <0.5 → need more data, not more investigation

Failure Modes

ModeSymptomRecovery
Confirmation biasFavoring first evidence seenWeight by strength, not order
Missing contradictionOverlooking conflicting evidenceExplicit contradiction detection pass
Over-counting agentsSame finding from N agents treated as N evidenceDeduplicate by root fact

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.