copilot-sdk-e2e-dev
Skill1 starsChanged 4 months agoApache-2.0View on GitHub
Spin up a live local OmniCraft server and exercise the GitHub Copilot SDK harness end-to-end — build copilot agents, run real turns, smoke-test, and bug-bash. Load when developing, testing, or debugging the copilot harness (omnicraft/inner/copilot_executor.py, copilot_harness.py, omnicraft/onboarding/copilot_auth.py) or its auth / model / tool-bridge behavior.
- Reads credentials
- Deletes or force-pushes
1 other project uses a version of this, 1 of them edited.
Adopted by
Most recently taken up by
- omnigent-ai/omnigentedited, first seen Oct 21, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 1
- omnigent-ai/omnigent★ 11k92% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository9 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .github/copilot-instructions.mdCopilot instructions
- antigravity-native-e2e-devSkill
- antigravity-sdk-e2e-devSkill
- cli-setup-verifySkill
- cursor-sdk-e2e-devSkill
- fucho-e2e-devSkill
- harness-integration-guideSkill
- pi-native-e2e-devSkill
The file
We split this skill into the rules it gives an agent
7 rules. About 0 are common, 2 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 32
Verify (booleans only — never print the token):
- Very specific: line 36
If both are False, run omni setup and register a Copilot token, or export GH_TOKEN=$(gh auth token) (when gh is logged into an account with …
- Very specific: line 51
Use the URL below as $SERVER.
- Very specific: line 91
Shell / file tools: add --tools coding.
- Very specific: line 92
Specific model: add --model gpt-5-mini (or claude-haiku-4.5, auto).
- Very specific: line 120
2. Full orchestration (dispatch → collect → synthesize): Use the fucho-e2e-dev driver (in the internal agent-framework clone) — it boots a …
- Distinctive: line 136
All three CUJs (smoke / fanout / review-pr) pass on a copilot brain (verified live: fanout dispatched 8 sub-agents, 8/8 OK + a synthesis).
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Copilot SDK harness: end-to-end dev & testing
The copilot harness drives the GitHub Copilot SDK (github-copilot-sdk,
imported as copilot) — a persistent CopilotClient + CopilotSession per
OmniCraft conversation — and bridges OmniCraft's sys_* tools into Copilot as SDK
Tools. The Python SDK bundles the Copilot CLI binary it drives as a backing
server, so there is no separate @github/copilot install. This skill is the
proven recipe for running it for real against a live local server — not just
the unit tests.
The harness runs as a local runner from your current checkout, so
omni run <bundle> --server <url>exercises exactly the code you're on.
Prerequisites (check these first)
- You're on the branch you want to test. The copilot harness is an
optional extra — install it (without disturbing other extras) with
uv sync --frozen --extra dev --extra copilot. NB: a bareuv run --frozen --extra devre-syncs the venv and prunes the copilot SDK; for live testing call.venv/bin/omni/.venv/bin/pythondirectly and avoiduv runmid-session. - The SDK is installed:
.venv/bin/python -c "import copilot; print(copilot.__file__)". - A GitHub token with Copilot access is configured. Copilot needs a
fine-grained PAT with the "Copilot Requests" permission, or an OAuth token
from the GitHub CLI / Copilot CLI app (classic
ghp_PATs are rejected). Verify (booleans only — never print the token):
If both are.venv/bin/python -c "from omnicraft.onboarding.copilot_auth import copilot_github_token_configured; import os; print('config:', copilot_github_token_configured(), 'env:', bool(os.environ.get('GH_TOKEN') or os.environ.get('COPILOT_GITHUB_TOKEN')))"False, runomni setupand register a Copilot token, orexport GH_TOKEN=$(gh auth token)(whenghis logged into an account with Copilot). Check the account's entitlement withgh api /copilot_internal/user(look forchat_enabled/cli_enabled). - Network egress to GitHub's Copilot backend. A turn that hangs or fails to connect on a locked-down host is usually egress, not a harness bug.
Step 1 — start a local server
cd /path/to/omnicraft
.venv/bin/omni server --port 7788 --no-open # foreground; or `omni server start` for detached
curl -s http://127.0.0.1:7788/health # {"status":"ok"}Use the URL below as $SERVER.
Step 2 — build a copilot agent bundle
A spec with spec_version must be a directory containing config.yaml —
not a single .yaml file. Minimal copilot agent:
mkdir -p /tmp/copilot-dev
cat > /tmp/copilot-dev/config.yaml <<'YAML'
spec_version: 1
name: copilot-dev
description: Copilot SDK dev/test agent.
executor:
type: omnicraft
config:
harness: copilot
# model: gpt-5-mini # optional; omit for Copilot auto-select
prompt: |
You are a terse test agent. Answer in as few words as possible.
YAMLFor sub-agents, tools, guardrails/policies, copy the field shapes from
examples/fucho/config.yaml and examples/lilo/config.yaml. (Declare policies
under guardrails.policies: — a top-level policies: key is silently dropped on
the spec_version + config.yaml path.)
Step 3 — run a turn (and smoke-test)
SERVER=http://127.0.0.1:7788
timeout 280 .venv/bin/omni run /tmp/copilot-dev \
-p "Reply with exactly the single word: PONG" \
--server "$SERVER" 2>&1A healthy run prints connection lines then the reply (PONG). If that works,
the full stack is good: token, egress, bundled CLI, harness.
- Shell / file tools: add
--tools coding. - Specific model: add
--model gpt-5-mini(orclaude-haiku-4.5,auto).
Targeted scenarios
| Goal | How |
| Native tools (shell/edit/read) | --tools coding, prompt to create→read→edit a file; confirm it actually touches disk |
Bridged sys_* / sub-agent dispatch | declare a sub-agent (harness copilot so auth is satisfied), prompt the parent to delegate — exercises the SDK Tool async-handler bridge into _tool_executor |
| Model routing | run the same bundle with several --model values; an unknown id fails loud, a databricks-* id is dropped to auto with a warning |
| LLM-phase policy | add a guardrail that denies a keyword; confirm PHASE_LLM_REQUEST/PHASE_LLM_RESPONSE blocks it |
| Concurrency / leaks | fire several omni run … & at once; then pgrep -af "copilot/bin/copilot" to check for orphaned bundled-CLI subprocesses |
Running fucho (or any orchestrator) on a copilot brain
The copilot harness can serve as an async orchestrator brain (fucho / lilo),
not just a standalone agent — it dispatches to sub-agents via the bridged
sys_* tools and synthesizes their results. Two ways to exercise it:
1. Committed regression guard (brain smoke).
tests/e2e/test_fucho_copilot_e2e.py boots a local server from your checkout and
runs examples/fucho with --harness copilot --model auto, asserting the brain
boots and replies. It is skipped unless a Copilot token is configured (so CI
without one skips it). Run it with:
.venv/bin/python -m pytest -o addopts="" tests/e2e/test_fucho_copilot_e2e.py -v2. Full orchestration (dispatch → collect → synthesize). Use the
fucho-e2e-dev driver (in the internal agent-framework clone) — it boots a
local server, polls the AP API, auto-answers elicitations, and asserts the
fan-out. Drive the brain on copilot with --brain-harness copilot, and always
pass a Copilot-catalog --brain-model (auto, claude-haiku-4.5,
gpt-5-mini): the driver's default --brain-model is a Claude id that Copilot
(no Databricks gateway) can't route. From the agent-framework clone:
.venv/bin/python .claude/skills/fucho-e2e-dev/fucho_driver.py \
--local --code-dir <this-worktree> \
--cuj smoke --brain-harness copilot --brain-model auto # brain only
# --cuj fanout … and --cuj review-pr --repo omnicraft-ai/omnicraft --pr <n> …
# exercise real sub-agent dispatch (claude_code + codex) under a copilot brain.All three CUJs (smoke / fanout / review-pr) pass on a copilot brain (verified
live: fanout dispatched 8 sub-agents, 8/8 OK + a synthesis). Note omni run -p
exits after the dispatch turn (the brain parks until woken), so a sub-agent's
final answer lands server-side — read it over the AP API
(GET /v1/sessions/{id}/items, child sessions), not just stdout.
Gotchas (these cost real time)
config.yaml'sserver:defaults to a remote server. Omitting--serversends your turn to that remote deploy — which may be stale and reject the copilot harness withexecutor.config.harness: must be one of […]. Always pass--server http://127.0.0.1:<port>. (If a local server rejectscopilot, it's running stale code — restart it from your checkout.)- A spec with
spec_versionmust be a directory +config.yaml, never a single.yamlfile. - Copilot needs a GitHub token (fine-grained PAT w/ Copilot Requests, or a
gh/Copilot-CLI OAuth token). Resolution precedence: spec
executor.auth(api_key) > storedcopilot:config block (omni setup) > ambientCOPILOT_GITHUB_TOKEN/GH_TOKEN/GITHUB_TOKEN. Classicghp_rejected. - No Databricks gateway. Copilot talks only to GitHub's backend, so a
databricks-*model is silently resolved to Copilot's auto-select — it will not route through the AI Gateway like claude-sdk/codex/pi. - Use a model id from the account's catalog. free_limited offers
auto,claude-haiku-4.5,gpt-5-mini. Run.venv/bin/python+client.list_models()to discover the live set; an unknown id fails loud (server-side failed session). - Turns take 30–90s — always wrap in
timeout 280. - Never print/echo the GitHub token in logs or commands.
Code & tests
- Executor (SDK bridge):
omnicraft/inner/copilot_executor.py - Wrap (HARNESS_COPILOT_ env → executor):*
omnicraft/inner/copilot_harness.py - Auth / token resolution:
omnicraft/onboarding/copilot_auth.py - Spawn env:
_build_copilot_spawn_envinomnicraft/runtime/workflow.py
uv run --frozen --extra dev python -m pytest \
tests/inner/test_copilot_executor.py \
tests/inner/test_copilot_harness.py \
tests/runtime/test_copilot_spawn_env.py \
tests/onboarding/test_copilot_auth.py -qBug-bash (fan out)
To stress the harness, run several scenario probes in parallel — each builds a
bundle and runs real turns against the same $SERVER, then reports what broke.
Highest-value targets: the Tool async-handler bridge (hangs / lost tool
results / errors reported as success), model routing, policy enforcement,
streamed-output rendering, and orphaned bundled-CLI processes after teardown.
Cross-check the AP API (GET /v1/sessions/{id}/items) — a start failure can exit
0 with empty stdout while the server records a failed session.
Known sharp edges (found via live bug-bash — "as of this writing")
- Native tools bypass
on:[tool_call]policies and aren't recorded. Copilot's built-increate/view/edit/bashrun inside the SDK, so anon:[tool_call]DENY guardrail (e.g.blast_radius) never sees them, and they leave nofunction_callitem in the transcript (only streamed narration). Bridgedsys_*tools ARE gated and recorded. Gate Copilot's built-ins at the LLM phase (PHASE_LLM_REQUEST/RESPONSE, which fire) or via the OS-env sandbox — noton:[tool_call]. (Same shape as the cursor harness.) - Copilot fails loud (unlike cursor's swallowed start failures). Bad token, empty/invalid model, and unknown model ids all exit non-zero with a clear error AND a server-side failed session + error item — verified, not swallowed.
omni run -pagainst an async orchestrator exits after the dispatch turn, so a delegated sub-agent's final answer is persisted server-side but may not reach stdout in one-shot mode. Read the session over the AP API to see it.- Non-graceful exit can orphan the bundled CLI. Graceful teardown reaps it
(
client.stop()); after aSIGKILL/hard-exit, sweeppgrep -af "copilot/bin/copilot".
Cleanup
.venv/bin/omni server stop # or kill the foreground `omni server`
rm -rf /tmp/copilot-dev # remove scratch bundles
pgrep -af "copilot/bin/copilot" # confirm no orphaned bundled-CLI subprocesses lingerDoes it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

