probe
Skill0 starsChanged 2 months agoApache-2.0View on GitHub
Run the probe-driven validation rounds for the agentic ontology — Type 4 (adversarial — capability-negation / cross-product-fabrication / synonym-swap-with-negation, ≥2 of 3 PASS) and Type 6 (implicit-ADR confirmation — maintainer writes 5 ADRs they know are followed; ≥3 of 5 must surface in implicit-adrs.md top-10). Both rounds are MAINTAINER-AUTHORED; this skill is the protocol mechanics, not the answers.
17 other projects use a version of this.
Adopted by
Most recently taken up by
- pantheon-org/tekhnetheir own version, first seen Nov 15, 2026
- ntholm86/principles-of-earned-autonomy-skills-suitetheir own version, first seen Nov 11, 2026
- agentmemoir/skillstheir own version, first seen Nov 1, 2026
- lgy1027/specflow-agent-packtheir own version, first seen Nov 1, 2026
- simota/agent-skillstheir own version, first seen Oct 23, 2026
- Igor-lx/claudeHarnesstheir own version, first seen Oct 15, 2026
Other versions17 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 17
- vasanthsreeram/Alvarmethod★ 155same name
- simota/agent-skills★ 99same name
- clonable-eden/cekernel★ 17same name
- pantheon-org/tekhne★ 10same name
- gilbertwuu/learning-journey★ 9same name
- JhCircle/Defense-ASu-skills★ 4same name
- LarryStanley/pi-delegate★ 4same name
- lgy1027/specflow-agent-pack★ 4same name
- ntholm86/principles-of-earned-autonomy-skills-suite★ 4same name
- agentmemoir/skills★ 3same name
7 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository27 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- alignSkill
- code-walkSkill
- conceptsSkill
- contributeSkill
- coverageSkill
- doc-gap-checkSkill
- enrichSkill
- enumerateSkill
- find-implicit-adrsSkill
- implementSkill
- ingest-docsSkill
- log-issueSkill
- navigateSkill
- next-batchSkill
- orientSkill
- panelSkill
- probe-defineSkill
- probe-runSkill
- reflect-featureSkill
- retrieveSkill
- reviewSkill
- scanSkill
- statusSkill
- streamsSkill
- test-coverageSkill
- triageSkill
The file
We split this skill into the rules it gives an agent
7 rules. 0 are common, 0 are distinctive, and 7 are very specific to this project.
0
Common
Most of the 13 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 37
Read adrs/drafts/research/agentic-code-ontology/PROBES.md (Type 4 protocol)
- Very specific: line 45
Use AskUserQuestion to capture each.
- Very specific: line 58
Search concepts.yaml for an entity / operation matching the probe's wording.
- Very specific: line 79
Read existing lineage/{repo}/implicit-adrs.md to surface its top-10 candidates
- Very specific: line 83
Use AskUserQuestion to capture (or invite the maintainer to type into a temp file).
- Very specific: line 90
Read implicit-adrs.md's candidates list.
- Very specific: line 106
Probe history persists: Every round appends to lineage/{repo}/probe-rounds.yaml with date, probes, verdicts, aggregate.
“Common” is measured: kept by at least 6 in 10 of the 13 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Probe-driven validation (DOC-164 slice 8+)
The agentic ontology's MVP acceptance is probe-driven, not coverage-%-driven (per adrs/drafts/research/agentic-code-ontology/PROBES.md). Two rounds the maintainer authors and the skill executes:
- Type 4 — Adversarial probes (per refresh). Three maintainer-authored "capability negation" probes (capabilities that don't exist — synonym-swap-with-negation, cross-product-fabrication, or capability-negation). The ontology should answer "no" / return empty. ≥2 of 3 must PASS for the refresh to be accepted.
- Type 6 — Implicit-ADR confirmation (per refresh, but lower frequency in practice — quarterly). Maintainer privately writes 5 ADRs they know the codebase embodies. The skill compares against
lineage/{repo}/implicit-adrs.md's top-10 candidates. ≥3 of 5 must appear (semantically equivalent) for the implicit-ADR feature to be supported. <3 PASS = block ontology MVP acceptance.
This skill is maintainer-driven — the implementer cannot author probes (that defeats the adversarial-round design per LSN-013-class case-law: implementer-authored probes only test what the implementer expected the ontology to handle).
Argument forms
| Form | Behaviour |
/probe --adversarial [<repo>] | Capture 3 adversarial probes the maintainer types in. Run each against the ontology (concepts.yaml + understanding/*.md). Report ≥2/3 PASS or FAIL. |
/probe --implicit-adrs [<repo>] | Capture 5 maintainer-written implicit ADRs. Compare semantically against implicit-adrs.md top-10 candidates. Report ≥3/5 PASS or FAIL. |
/probe --status [<repo>] | Read-only. Print the last probe round's results + recommended next round date (per quarter or per-refresh). |
Prerequisites
lineage/{repo}/concepts.yamlexists (for adversarial probes — concept catalog is what gets queried).lineage/{repo}/implicit-adrs.mdexists (for Type 6).- Maintainer is in the session and ready to type the probes (this is interactive; the skill cannot synthesise probes).
Protocol
Round A — /probe --adversarial
Step 1 — Orient
- Read
adrs/drafts/research/agentic-code-ontology/PROBES.md(Type 4 protocol) - Note the 3 probe construction patterns the maintainer can choose from:
- Capability-negation — take a real capability, flip a key attribute (e.g. real: "dark mode that toggles in toolbar"; flipped: "dark mode that ties to user profile-server-stored")
- Cross-product-fabrication — combine two real concepts into a capability the codebase doesn't have (e.g. "OIDC-driven scheduled jobs that emit Prometheus metrics")
- Synonym-swap-with-negation — replace a real concept's name with a related-but-different concept (e.g. "server-side i18n" — when only client-side exists)
Step 2 — Maintainer authors 3 probes
Use AskUserQuestion to capture each. Per-probe questions:
- "Adversarial probe 1: name a capability the codebase does NOT have (a fabrication, negation, or synonym-swap of a real one). One sentence."
- "Adversarial probe 2: same but a different fabrication pattern."
- "Adversarial probe 3: third probe."
Step 3 — Verify each probe is genuinely fabricated
For each probe, Grep the substrate's nodes.jsonl and search the source repo for any literal/semantic match. If a probe accidentally describes a real capability, surface and ask the maintainer to revise.
Step 4 — Query the ontology for each probe
For each probe:
- Search
concepts.yamlfor an entity / operation matching the probe's wording. - Sample 3-5 sidecars whose concepts most overlap with the probe's keywords; check if any sidecar's
understanding/conceptswould falsely confirm the fabricated capability.
Verdict per probe:
- PASS — no concept entry matches; no sidecar's
understandingwould falsely confirm. The ontology correctly returns "no such capability." - PASS+ — empty result + an explanatory
null-cause field ("no node matches: combinations of axes X and Y are not represented") - FAIL — confident match to a real-but-different concept (false-positive hallucination)
- CRITICAL FAIL — match to a fabricated node not present in
nodes.jsonl
Step 5 — Report
- Per-probe verdict + 1-line rationale
- Aggregate:
<N>/3 PASS - Acceptance criterion: ≥2/3 PASS for the refresh to be accepted; <2 = the maintainer iterates the file-analyser / concept-merger prompts to fix the hallucination class
- Append the round to
lineage/{repo}/probe-rounds.yaml(creates if missing) for historical tracking
Round B — /probe --implicit-adrs
Step 1 — Orient
- Read
adrs/drafts/research/agentic-code-ontology/PROBES.mdType 6 - Read existing
lineage/{repo}/implicit-adrs.mdto surface its top-10 candidates
Step 2 — Maintainer writes 5 implicit ADRs
Use AskUserQuestion to capture (or invite the maintainer to type into a temp file). Each ADR: one sentence + 2-3 example file:line citations.
"Write 5 implicit ADRs you know the ODD codebase embodies — patterns followed across the codebase that no
adrs/file documents yet. One sentence each + 2-3 file:line citations from memory or quick check. The skill compares against the auto-generated implicit-adrs.md top-10."
Step 3 — Compare against implicit-adrs.md top-10
For each maintainer-written ADR:
- Read
implicit-adrs.md's candidates list. For each candidate, check semantic equivalence against the maintainer's ADR (similar decision_statement; matching file:line citations). - Verdict per maintainer ADR:
- PASS — appears in the top-10 (semantically equivalent, even if phrased differently)
- PARTIAL — appears outside top-10 but in the candidates list (the ontology saw it, just didn't rank high)
- FAIL — does not appear at all (extractor gap; the file-analyser prompts need refinement)
Step 4 — Report
- Per-ADR verdict + 1-line rationale
- Aggregate:
<N>/5 PASS - Acceptance criterion: ≥3/5 PASS for the implicit-ADR feature to be supported. <3 = block ontology MVP acceptance per ADR's PROBES section.
- Append to
lineage/{repo}/probe-rounds.yaml
Rules
- Maintainer authors; implementer cannot. The implementer (Claude in this session) must NOT autocomplete or suggest probes.
AskUserQuestioncaptures verbatim from the maintainer; the implementer's only role is to verify the probe is genuinely fabricated (Round A) or compare against the ontology (Round B). - Probe history persists. Every round appends to
lineage/{repo}/probe-rounds.yamlwith date, probes, verdicts, aggregate. This builds the regression record perPROBES.md's "caught bugs become permanent regression probes." - Don't grade silently. Verdicts ALWAYS surface to the maintainer; the skill never auto-accepts an ontology refresh based on a probe round.
Cross-references
- Probe spec:
adrs/drafts/research/agentic-code-ontology/PROBES.md - Substrate-level probes (existence-of-capability):
lineage/PROBES.md - Concept catalog:
lineage/{repo}/concepts.yaml(queried by Type 4) - Implicit-ADRs catalog:
lineage/{repo}/implicit-adrs.md(compared against by Type 6) - ADR:
adrs/drafts/agentic-code-ontology.mdrev 2 — Validation section + slice 8
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

