Verify pstack

pstack is pure markdown shipped as a pi package: package.json maps pi.skills: ["./skills"] (currently 42 skills) and pi.subagents.agents: ["./agents"] (3 agents: poteto-agent, council-sol, comment-sicko). There is no build, no server, no port. The app under test is a real pi session loading these files. The feature map in features/ is the maintained verification source; read its README before driving anything past a smoke check.

Launch

On this machine the checkout is installed as a live local-path package: pi list shows ../../dev/pstack → /home/hieusats/dev/pstack. Every new pi session therefore loads the working tree as-is, uncommitted changes included. Launching means starting a headless probe:

cd /tmp && timeout 180 pi -p "<probe prompt>" --no-session

• Ready signal: the process prints the answer and exits with code 0. No log line to wait for; give LLM probes up to 180 s.
• Run probes from /tmp, not the repo, so this repo's AGENTS.md and .agents/ do not contaminate the session under test.
• Project-scope discovery (this skill itself, or any .agents/skills/ target) needs a trusted project: headless runs silently skip project files without --approve/-a (interactive sessions ask once). A No from an untrusted probe is a scope fact, not a registration failure — re-run with -a from the repo dir before judging.
• A pi probe launched from inside a pi-subagents child shell needs env -u PI_SUBAGENT_CHILD. With the variable set, the extension silently disables itself and the session has no subagent tool.
• --no-session keeps the run ephemeral: no session file written, nothing to keep alive, probes exit on their own.
• Isolated-candidate variant (tests the tree without any user packages): cd /tmp && pi -p "<probe>" --skill /home/hieusats/dev/pstack/skills --no-extensions --no-session. This loads skills only; agents need pi-subagents and are not reachable this way.

Doctor

Run all three before driving; any failure means fix the environment first, do not drive a misregistered instance.

python3 /home/hieusats/dev/pstack/scripts/verify.py   # exit 0, summary ends in "0 failures"
pi list | grep dev/pstack                             # prints /home/hieusats/dev/pstack
pi --version                                          # non-empty version

If a live probe fails with an auth/provider error instead of an answer, check pi auth before suspecting the package.

Drive

The harness is headless pi probes. The four UPSTREAM.md probes, with capture (see Evidence):

RUN_ID=$(date +%Y%m%d-%H%M%S); EV=/tmp/pstack-verify/$RUN_ID; mkdir -p "$EV"

# 1. Static integrity gate (structure, not behavior)
cd /home/hieusats/dev/pstack && timeout 60 python3 scripts/verify.py > "$EV/verify-static.txt" 2>&1; echo "exit=$?" | tee -a "$EV/verify-static.txt"

# 2. Package registration
pi list | tee "$EV/pi-list.txt" | grep dev/pstack

# 3. Skill availability (explicit-invocation class)
cd /tmp && timeout 180 pi -p "/skill:how Reply with the first heading of the loaded skill and nothing else" --no-session | tee "$EV/skill-probe.txt"; echo "exit=$?" | tee -a "$EV/skill-probe.txt"

# 4. Agent roster (through pi-subagents)
cd /tmp && timeout 180 pi -p "List the names of your available subagents. Nothing else." --no-session | tee "$EV/agent-probe.txt"; echo "exit=$?" | tee -a "$EV/agent-probe.txt"

Expected observables: (1) exit 0 with verify: ok — <N> skills, 3 agents, 0 failures (N moves with the tree; "0 failures" is the invariant); (2) the checkout path printed; (3) the reply contains # How; (4) all three names poteto-agent, comment-sicko, council-sol appear.

Explicit-invocation probe (reaches skills that are invisible to probe 3): cd /tmp && timeout 180 pi -p "/skill:poteto Reply with the first heading of the loaded skill and nothing else" --no-session → the reply contains # Poteto. The auto-load class has been empty since 0.1.9: every skill is disable-model-invocation and loads only by explicit invocation.

Stable handles: skill names and agent names as declared in frontmatter (skills/*/SKILL.md, agents/*.md); probe prompts as written above (UPSTREAM.md records their expected answers). For what each feature proves and its traps, follow features/README.md.

Spawn failure: "No usable subagent models"

A dispatch that fails with No usable subagent models remain after registry, scope, and cached-exclusion filtering. is a poisoned exclusion cache, not a registry or auth problem. One transient child failure (cold start, empty response, 429) records a 24-hour exclusion for that model in a per-user store under /tmp/pi-subagents-<uid>/model-exclusions.json (PI_MODEL_EXCLUSIONS_PATH overrides the location). When the excluded model is the only configured model, every spawn hard-fails for the TTL and each retry re-hits the cache; three failures an hour apart look identical and nothing decays the entry early.

Diagnose. A live entry has expiresAt in the future.

cat /tmp/pi-subagents-*/model-exclusions.json 2>/dev/null || echo "(no exclusion store: clean)"

Confirm the mechanism before editing anything: the same spawn probe against an empty store succeeds.

echo '{}' > /tmp/empty-exclusions.json
cd /tmp && PI_MODEL_EXCLUSIONS_PATH=/tmp/empty-exclusions.json timeout 240 env -u PI_SUBAGENT_CHILD pi -p "Use the subagent tool to launch poteto-agent with the task: Reply with the single word done." --no-session

Recover by deleting the store; the next failure recording recreates it.

rm /tmp/pi-subagents-*/model-exclusions.json

Two caveats. A running session loads the store into memory at first read, so a long-lived session can keep failing after the file is gone; verify recovery with a fresh headless probe. The upstream behavior (one transient failure earns a 24-hour exclusion, no CLI to clear it, and the error names neither the store nor a recovery) is a pi-subagents defect to report there, not to patch in this tree.

Evidence

Capture into /tmp/pstack-verify/<RUN_ID>/, one .txt per probe, holding the full stdout and the exit= line — the tee commands in Drive do exactly this.

Proof standards:

• Structure and visibility are different claims. verify.py proves the tree is well-formed; a live pi -p probe proves a session actually sees the skill or agent. Registration alone is not a behavior proof.
• Exercise the real user path: skills are consumed via the system prompt (auto-load class) or /skill:<name>; agents via pi-subagents dispatch. Do not "verify" by grepping markdown.
• Record the probe command, full answer text, and exit code. A truncated "looks fine" is not evidence.
• --no-session probes write no side effects (no session files, no ports, no data dirs); there is no second view to take.

Cleanup

• Headless probes exit by themselves. If one hangs past timeout, kill its PID (kill <pid> from ps), never pkill pi or any name-based kill — the operator's live pi sessions are pi processes too.
• Never run pi install, pi remove, or pi update as part of verification. They rewrite ~/.pi/agent/settings.json (the user's global config); the checkout is already installed as a live path, and that install is what the probes exercise.
• The isolated deep-check variant (scratch project-scope install: cd <scratch> && pi install /home/hieusats/dev/pstack -l && pi install npm:pi-subagents -l) writes only inside the scratch dir; cleanup is rm -rf <scratch>.
• Evidence survives cleanup: /tmp/pstack-verify/<RUN_ID>/ is never deleted by a verification run. After any cleanup, confirm the run's .txt files are still there.

Helpers

None shipped. Every command is a one-liner shown inline above; a helper would only wrap tee.