agentleFS
Omni-CraftOmni-Craft/OmniCraft/.claude/skills/pi-native-e2e-dev/SKILL.md

pi-native-e2e-dev

Skill1 starsChanged 4 months agoApache-2.0View on GitHub

Spin up a live local OmniCraft server + runner and exercise the native Pi TUI harness (pi-native) end-to-end — launch the real `pi` CLI via `omnicraft pi`, drive turns through the web/bridge, smoke-test, and bug-bash. Load when developing, testing, or debugging the pi-native harness (omnicraft/inner/pi_native_executor.py, pi_native_harness.py, omnicraft/pi_native.py, pi_native_bridge.py, pi_native_credentials.py) or its bridge / extension / auth / model behavior.

  • Deletes or force-pushes
  • Installs packages
  • Sends data out

1 other project uses a version of this, 1 of them edited.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 1 is distinctive, and 6 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 6

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Type-driven smoke: instead of the POST, type a prompt directly in the attached TUI and confirm it answers + mirrors to …/items.

    line 144
  2. Very specific:

    If a queued message never reaches Pi, watch whether inbox/*.json drains.

    line 163
  3. Very specific:

    It's a TUI, not omni run: Use omnicraft pi. There is no omni run <bundle> path for pi-native.

    line 183
  4. Very specific:

    config.yaml's server: defaults to a remote server: Always pass --server "$SERVER" (or --server "" to auto-spawn local).

    line 186
  5. Very specific:

    No live LLM without auth: If the Prereq-5 probe prints None and pi isn't logged in, turns won't get a real answer.

    line 190
  6. Distinctive:

    Never print/echo provider keys or gateway tokens: Use the boolean/id probes above.

    line 196
  7. Very specific:

    If you can't reach a ready Pi TUI (missing pi, no tmux/node, no auth, headless limits), say so — don't claim a turn passed.

    line 256

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Pi native harness: end-to-end dev & testing (local server/runner)

The pi-native harness wraps the real Pi coding-agent TUI (@earendil-works/pi-coding-agent, the pi CLI). Unlike the SDK harnesses (cursor / copilot / antigravity), it does not run in-process: omnicraft pi ensures a host daemon, the daemon spawns a runner that launches pi inside a runner-owned tmux terminal, and your TTY attaches to it. OmniCraft's web-UI turns are forwarded into that live pi process through a file-inbox bridge + a packaged JS extension (pi.sendUserMessage). This skill is the proven recipe for running it for real against a live local server + runner — not just the unit tests.

Like the other harnesses, the runner imports from your current checkout, so testing here exercises exactly the code you're on. (CWD/venv selects the code, not PYTHONPATH.)

What actually runs where

your TTY ── (attach / pexpect) ──► omnicraft pi (CLI, local)
                                        │ ensures
                                        ▼
                                  host daemon ──► local OmniCraft server (AP)
                                        │ spawns                      ▲
                                        ▼                             │ HTTP
                                  runner ── launches ──► pi (TUI, in tmux)
                                                              │ loads
                                                              ▼
                                                 omnicraft pi-native extension (JS)

Two ways a turn reaches Pi — test both:

  1. Type in the TUI (your attached terminal). Exercises Pi natively; the extension mirrors the transcript back to the server (POST …/events).
  2. Web / API message. Server → runner → PiNativeExecutor.run_turn → enqueue_user_message() writes inbox/<ordinal>_msg_*.json → the resident extension polls the inbox → pi.sendUserMessage(...). This is the harness-specific path most worth covering.

Prerequisites (check these first)

  1. You're on the branch you want to test, and running from that checkout (.venv/bin/omnicraft / .venv/bin/python from this repo).
  2. The pi CLI is on PATH — the harness can't launch without it:
    which pi && pi --version
    # install if missing:  npm install -g @earendil-works/pi-coding-agent
    # or point at an explicit binary:  export OMNICRAFT_PI_PATH=/path/to/pi
    .venv/bin/python -c "from omnicraft.onboarding.harness_readiness import harness_is_configured; print('pi-native ready:', harness_is_configured('pi-native'))"
  3. tmux is on PATH. The native wrapper attaches your TTY to the runner-owned Pi tmux pane (_preflight_local_tools hard-fails without it).
  4. node is on PATH. The extension is JS executed inside Pi (also required by the e2e extension tests). node --version.
  5. Auth is resolvable (booleans/ids only — never print keys). Native Pi normally logs in from its own ~/.pi/agent. OmniCraft bridges the provider you set with omnicraft setup instead, writing a managed per-session models.json and passing --provider omnicraft --model <resolved>. Verify what it will use:
    .venv/bin/python -c "from omnicraft.pi_native_credentials import resolve_pi_native_provider as r; p=r(); print('provider:', getattr(p,'provider_id',None), '| api:', getattr(p,'api',None), '| model:', getattr(p,'model',None))"
    None → no omnicraft provider configured; Pi falls back to its own /login (run omnicraft setup, or log into pi directly). A Databricks default resolves to the AI-Gateway anthropic-messages surface with a refreshed bearer token.
  6. Network egress to the model backend. A turn that hangs/fails to connect on a locked-down host is usually egress, not a harness bug.

Step 1 — start a local server (real server + runner)

cd /path/to/omnicraft
.venv/bin/omni server start          # detached managed server on a free loopback port
.venv/bin/omni server status         # prints the URL, e.g. http://127.0.0.1:6767
SERVER=http://127.0.0.1:6767         # use the printed URL below
curl -s "$SERVER/health"             # {"status":"ok"}

(omnicraft pi --server "" also auto-spawns a persistent local server and uses it — handy for a one-shot manual run, but a known $SERVER URL is better for scripted API observation below.)

Step 2 — launch the native Pi terminal against the local server

omnicraft pi attaches an interactive TUI, so run it where you can hold it open. Two patterns:

A. Background terminal (recommended for scripted drives). Launch it in one terminal and drive/observe from another:

.venv/bin/omnicraft pi --server "$SERVER" 2>&1   # attaches the Pi TUI; leave it running

It prints Web UI: <url> and a resume hint to stderr — grab the conversation id (the …/c/<conv_…> segment). Capture it for the API calls below:

CONV=conv_xxxxxxxx   # from the "Web UI:" line / resume hint

B. PTY driver (fully automated). Drive it under pexpect exactly like the claude-native-e2e-test skill's cuj_driver.py (a proven, generalizable base): spawn omnicraft pi --server <url> in a PTY with cwd=<checkout>, capture the conv id from the printed URL, send keystrokes / poll the API, then tear down the whole process tree (see Teardown — pexpect Ctrl-C only detaches tmux).

Pass-through Pi CLI args go after the command (persisted as terminal_launch_args), e.g. omnicraft pi --server "$SERVER" -- --model <id>; omnicraft still injects --provider omnicraft --model <resolved> when a provider is configured (see pi_native_credentials.py).

Step 3 — drive a turn (and smoke-test)

Via the web/bridge path (exercises PiNativeExecutor). Post a user message to the running session; the runner routes it through the harness → bridge inbox → extension → pi.sendUserMessage:

curl -s -X POST "$SERVER/v1/sessions/$CONV/events" \
  -H 'content-type: application/json' \
  -d '{"type":"message","data":{"role":"user","content":[{"type":"input_text","text":"Reply with exactly the single word: PONG"}]}}'

Then observe the mirrored transcript (the extension forwards Pi's output back via POST …/events):

sleep 20
curl -s "$SERVER/v1/sessions/$CONV/items" | python -m json.tool | tail -40

A healthy run shows your user message and a non-empty assistant reply (PONG) mirrored into the session — proving the full stack: server → runner → harness → inbox → extension → Pi → transcript forwarder. You'll also see Pi render the message in the attached TUI.

  • Type-driven smoke: instead of the POST, type a prompt directly in the attached TUI and confirm it answers + mirrors to …/items.
  • Specific model: see Step 2 pass-through note; confirm the resolved model in the Prereq-5 probe.

Inspect the bridge (debugging)

Everything the harness writes for a session lives under a hashed bridge dir:

.venv/bin/python -c "from omnicraft.pi_native import pi_bridge_dir_for_session as d; print(d('$CONV'))"
# ~/.omnicraft/pi-native/<sha256(conv)[:32]>/
#   inbox/                 <- *.json user_message / interrupt payloads (poller drains + deletes)
#   sessions/              <- pi --session-dir state
#   config.json            <- sessionId, serverUrl, inboxDir, authHeaders (extension config)
#   omnicraft_pi_native_extension.js
ls -la "$(.venv/bin/python -c "from omnicraft.pi_native import pi_bridge_dir_for_session as d; print(d('$CONV'))")/inbox"

If a queued message never reaches Pi, watch whether inbox/*.json drains. The managed Pi config dir (PI_CODING_AGENT_DIR) holds the generated models.json that wires Pi's provider/model. Key env vars: HARNESS_PI_NATIVE_BRIDGE_DIR, HARNESS_PI_NATIVE_REQUEST_SESSION_ID, OMNICRAFT_PI_NATIVE_CONFIG, OMNICRAFT_PI_PATH (legacy HARNESS_PI_PATH), PI_CODING_AGENT_DIR.

Targeted scenarios

GoalHow
Web→Pi deliveryPOST a message (Step 3); confirm a fresh inbox/*.json appears then drains and the reply mirrors to …/items
Native tools (shell/edit/read)prompt Pi to create→read→edit a file and run a shell command; confirm it touches disk
Resumestop the TUI, omnicraft pi --server "$SERVER" --resume "$CONV" — reattaches; --resume (no value) opens the pi-native picker
Interruptmid-turn, enqueue an interrupt (pi_native_bridge.enqueue_interrupt(bridge_dir)) or use the UI stop; confirm Pi's abort() fires and the next turn isn't poisoned (see test_pi_native_interrupt_replay_e2e.py)
Policy / guardrailadd a guardrail that denies a keyword; native Pi tool calls are gated by the extension POSTing …/policies/evaluate (not the turn-scoped evaluator) — confirm a DENY blocks
Model routingflip the configured provider/model; re-check the Prereq-5 probe and that the answer still lands
Concurrency / leaksdrive several sessions; then sweep for orphaned pi / runner / tmux (see Cleanup)

Gotchas (these cost real time)

  1. It's a TUI, not omni run. Use omnicraft pi. There is no omni run <bundle> path for pi-native; the executor only enqueues into the bridge — Pi must be alive (attached) for a turn to be processed.
  2. config.yaml's server: defaults to a remote server. Always pass --server "$SERVER" (or --server "" to auto-spawn local). If a local server rejects pi-native, it's running stale code — restart it from your checkout (allowlist: omnicraft/spec/_omnicraft_compat.py).
  3. No live LLM without auth. If the Prereq-5 probe prints None and pi isn't logged in, turns won't get a real answer. Configure a provider via omnicraft setup or pi /login.
  4. tmux must be reachable from the CLI process. Direct tmux attach needs the runner-owned socket visible locally; a missing socket/tmux fails the attach.
  5. Turns take ~20–90s — wrap scripted waits/timeout generously.
  6. Never print/echo provider keys or gateway tokens. Use the boolean/id probes above.

Code & tests

  • Executor (bridge enqueue): omnicraft/inner/pi_native_executor.py
  • Harness wrap (harness: pi-native): omnicraft/inner/pi_native_harness.py
  • CLI launch / daemon-runner / tmux attach: omnicraft/pi_native.py (run_pi_native); CLI command pi(...) in omnicraft/cli.py
  • Bridge (inbox, extension/config writers): omnicraft/pi_native_bridge.py
  • Auth/model → Pi models.json: omnicraft/pi_native_credentials.py
  • Extension (JS, polls inbox, posts events/policies): omnicraft/resources/pi_native/omnicraft_pi_native_extension.js
  • Readiness gate: omnicraft/onboarding/harness_readiness.py
.venv/bin/python -m pytest \
  tests/test_pi_native_bridge.py \
  tests/test_pi_native_credentials.py \
  tests/test_pi_native_extension.py \
  tests/test_pi_native_interrupt_replay_e2e.py -q   # interrupt e2e needs `node`
# JS unit tests: node omnicraft/resources/pi_native/omnicraft_pi_native_extension.test.js

Bug-bash (fan out)

Stress the harness with several scenario probes against the same $SERVER: the web→inbox→extension delivery path (lost messages / inbox that won't drain), interrupt replay semantics, native-tool policy gating, transcript-forwarder fidelity (does every assistant block reach …/items?), resume/reattach, and orphaned pi/runner/tmux after teardown. Cross-check the API — a start failure can leave the TUI empty while the session records an error.

Watch-outs from the code (verify live — not a live-bug-bash log)

  • Empty inbox = no turn. PiNativeExecutor yields TurnComplete once the message is queued, not once Pi answers; the actual answer is async via the extension. Judge success by …/items, not the POST returning queued: true.
  • Native Pi tool calls bypass the turn-scoped evaluator. They're gated only by the extension's POST …/policies/evaluate; if the extension's config.json lacks serverUrl/authHeaders, gating silently no-ops.
  • History on a rebuilt session depends on Pi's own --session-dir state under the bridge dir, not on OmniCraft re-injecting transcript.

Teardown — non-negotiable

A pexpect Ctrl-C detaches from tmux; the runner, the tmux server, and pi keep running. Tear down the process tree from the child PID (ps --ppid … → SIGTERM/SIGKILL) and separately tmux -S <sock> kill-server (the tmux server reparents to init). Then verify nothing lingers:

.venv/bin/omni server stop                 # stop the managed server + local daemon
pgrep -af "(^|/)pi( |$)|harnesses\._runner|runner\._entry|tmux"   # confirm no orphans
# remove a session's bridge dir if you want a clean slate:
# rm -rf "$(.venv/bin/python -c "from omnicraft.pi_native import pi_bridge_dir_for_session as d; print(d('$CONV'))")"

Honesty

If you can't reach a ready Pi TUI (missing pi, no tmux/node, no auth, headless limits), say so — don't claim a turn passed. The strongest evidence is the round trip observed over the API: your user message and a non-empty assistant reply mirrored into GET /v1/sessions/$CONV/items.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.