trail-debug
Skill40 starsChanged 34 days agoApache-2.0View on GitHub
Explain why an Agentmetry detection fired, or investigate a suspicious or missing event in the trail. Use when the user asks why a rule fired, why something was flagged, whether a finding is a false positive, or to trace an event through the trail.
- Reads credentials
In this repository4 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- dogfood-weekSkill
- outreach-nextSkill
- ship-safeSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 17
If the chain does not verify, stop and say so.
- Distinctive: line 59
If the finding matches one of these, name the issue.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Trail debug
Answer the question from the trail. Do not fix the rule.
1. Verify before reading
cd apps/orchestrator
.venv/Scripts/python.exe -m agentmetry.cli verify --trail data/agentmetry-trail.jsonlIf the chain does not verify, stop and say so. Every conclusion below is drawn from the file, so a file that fails its own check makes them worthless.
The em-dash below is quoted from real CLI output, not house copy.
A FAILED — no checkpoint in this anchor log covers this trail is a
different verdict from tampering. It means the configured anchor log is about
some other file, which is a misconfiguration rather than evidence of rewriting.
2. Find the detection and its evidence
A detection event carries detection.event_ids, which are the exact events that
caused it. Pull those, not the surrounding minute.
grep '"rule_id":"credential-exfil"' data/agentmetry-trail.jsonl | tail -1 | python -m json.toolThen look up each id in detection.event_ids. The rule fired on a sequence;
the individual calls are usually ordinary, and that is the whole design.
3. Explain it
Say, concretely:
- Which rule, and the plain-English sequence it matched
- The specific events, with their commands and timestamps
- Which trait each event contributed. Traits are the vocabulary rules match on: a credential read, an egress, an untrusted-input read
- Whether the label is right
4. If it is a false positive
Say so plainly. The five known false-positive sources are already filed:
| Issue | Shape |
| #44 | .env.example read as credential access |
| #49 | credential path named inside an MCP argument |
| #50 | pipe into an interpreter running its own script |
| #51 | GitHub-raw fetch plus any later python -c, unlinked |
| #55 | observable techniques collapsing into T1059 |
If the finding matches one of these, name the issue. If it is a new shape, open an issue with the session's events as the reproduction. That is the highest value output this skill produces: a corpus case nobody invented.
The rule that matters
Do not edit rules.py, traits.py, engine.py or mitre.py. They are
frozen until 2026-09-05 because they are hashed into the ruleset fingerprint,
and changing one restarts a four-week beta gate currently at 2 of 4.
Fixing a false positive the day you find it feels obviously right and is exactly how the last clock got reset: three detection improvements landed in week one, each an improvement, and collectively they made the week meaningless.
File it. Land the batch on 2026-09-05, #55 first, so the fingerprint moves once.
You may disposition a detection as a false positive through the CLI or dashboard without touching the freeze. That is triage, not a rule change, and an untriaged critical is itself what turns a week red.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.