dogfooding-shipit
Skill9 starsChanged 20 days agoApache-2.0View on GitHub
Running ShipIt inside ShipIt: the manual `dev` Compose preview, the inner orchestrator's RUNTIME_MODE=local limitations, which credentials to set (and which to leave unset), repo seeding, and driving the inner instance over its HTTP API. Load when working on the dogfood loop, debugging the inner ShipIt UI, or touching local-mode behavior.
- Reads credentials
- Deletes or force-pushes
In this repository15 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- add-endpointSkill
- android-buildSkill
- client-architectureSkill
- deployment-architectureSkill
- design-languageSkill
- docs-navigatorSkill
- frontend-design:frontend-designSkill
- git-architectureSkill
- prompt-architectureSkill
- server-architectureSkill
- session-containersSkill
- session-lifecycleSkill
- session-processesSkill
- testing-and-qualitySkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 22
No inner file watcher
- Very specific: line 76
Never test onboarding by deleting credentials from inner Settings or wiping .inner-shipit/ — the first is destructive to the developer's …
- Very specific: line 110
Delete a seeded role in the inner UI and the next dev boot puts it back.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Dogfooding ShipIt in ShipIt
Opening the ShipIt repo in production ShipIt surfaces the dev Compose service as a manual preview — a whole second orchestrator, plus a Dockerfile.dogfood build, so it starts on demand rather than every boot. It shares the agent container's /workspace/node_modules (populated by agent.install at session boot) and runs Vite's dev server on the exposed port 3000, proxying /api, /ws, and /preview to the inner orchestrator on internal port 4000. It does not run its own npm install or a production vite build — the Compose file explains why that would be redundant and unsafe.
Start it with shipit service start dev. A first start may take minutes; a start that times out is still running — re-check with shipit service list. Start it whenever your change can be seen in the inner UI, and in it run the path the user takes, in the state the user is in (CLAUDE.md → Debugging the UI). A few minutes of start time buys a check unit tests cannot make, so "that needs the dogfood instance" is never a reason to ship a UI change unverified, to hand the decision back to the user, or to hand the user a check you can run on the machine you were given. If the container is paused, start fails with cannot start a paused container, try unpause instead; shipit service restart dev recovers it.
Leave it running when you are done — never shipit service stop dev. Stopping it by hand buys nothing: archiving the session stops the service anyway. And the user often opens the inner UI after you report, so a stopped service only makes them wait through another start. Say that it is running and point at it with a [the inner UI](shipit-preview://dev) chat link — the url from shipit service list is the container IP, which is yours to curl and not an address their browser can reach.
Local mode is a real exception to "ShipIt always runs in Docker"
The inner orchestrator runs with RUNTIME_MODE=local, which skips Docker entirely: no inner containers, no inner Compose, and inner agents spawn in-process via claude-adapter / codex-adapter rather than in a session worker container.
That makes several things degrade, by design — do not treat them as bugs:
- No inner terminal
- No inner file watcher
- No inner preview
Full design: docs/118-shipit-ui-local.
Credentials — set them once, outside
The dev service's credentials are user-supplied secrets, set once in the outer Project Settings → Secrets (docs/184-remove-platform-secret-forwarding). Platform secret forwarding was deliberately removed, so nothing is inherited.
GITHUB_TOKEN plus any service credential you want to exercise is the set. The dev service's x-shipit-secrets block in docker-compose.yml declares every storageEnv name the model catalogue knows (read it there for the current list), so a key set once out there appears in every dogfood session without a visit to inner Settings (docs/131 req 11).
Declaring a name costs nothing; supplying a value is the opt-in. An unset secret resolves to nothing and seeds nothing. That per-name choice is the only granularity there is — there is no second switch.
A supplied key becomes a credential ROUTE, not just a variable
This matters and is not what the environment alone gives you. A bare variable is read by listConfiguredCredentials (service-routing.ts), so its models are already eligible and it does get a synthetic env:<NAME> route id. What it does not have is a row — so inner Settings → Model providers shows nothing, and it can be neither ordered, nor persistently benched, nor failed over to (stringSelectionFor reaches it only when nothing is stored). scripts/seed-inner-credentials.ts closes that: at boot it POSTs each supplied variable to /api/credential-routes, so the inner instance holds a real credential.
The stored credential reaches a local turn through applyLocalMcp → localMcpSpawnEnv → selectAgentEnvForPush, which applies SHIPIT_CREDENTIAL_* to process.env around each spawn, read live from the store. No orchestrator restart is needed — a credential seeded (or added by hand) after boot works on the next turn. Verified with a real inner turn.
⚠ Two billing hazards, and the first one is the one people miss
1. Any metered key can become what background work spends on. Session naming and PR descriptions resolve an unpinned model with firstEligibleNonTurnSelection (non-turn-model.ts) — the first eligible model in catalogue order, over whatever credentials the install holds. So supplying only a DeepSeek key makes that the background-work model, and it bills. No CLI is involved, and this applies to every metered key mode, not just the vendor-native ones. Check Settings → Model providers → Background work, and pin it if you care.
2. Three names bypass a connected account. ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN and OPENAI_API_KEY are read by the vendor CLIs directly. Two things protect a spawn, and neither is universal:
- A redirected spawn is safe:
applyServiceRoutingdeletes every Anthropic credential variable and sets exactly one, so a GLM or DeepSeek turn can never pick up an ambient Anthropic key. - An unshaped spawn — the harness on its own vendor — is safe only when a scoped
HOMEresolves, whichresolveLocalAgentHomedoes only for anaccountroute. With no route selected it deliberately keeps the process-global home,scrubEnvAuthForScopedHomeis a no-op, and the ambient variable is what the CLI uses.
Non-turn work has a further wrinkle: resolveLocalAgentHome reads the runner's resident route, while runNonTurnSpawn selects its own target — so the home can belong to a different credential than the work was routed to.
The seed prints a ⚠ line for each hazard that applies. local-agent-credentials.ts maintains the unscoped fallback home (planning#284) and is not on the per-turn path.
Testing onboarding — the onboarding service, not a wipe
dev is a configured install and cannot be un-configured. Every key supplied in the outer Project Settings → Secrets is injected into it, adoptEnvCredentials turns each one into a stored credential at boot (docs/252 req 20), and resolveHarnessOnboarding then stamps harnessOnboardingCompletedAt on the first read — permanently, since nothing clears it. DOGFOOD_SEED_CREDENTIALS=0 is not enough: it stops the seeder POSTing, and adoption still makes rows out of the variables.
So there is a second manual service:
shipit service start onboarding # port 3001; `dev` can stay up on 3000Two differences from dev, and nothing else. It declares GITHUB_TOKEN and no service credential — a name in x-shipit-secrets is what injects the value, so the list is the mechanism; GitHub is supplied because the subject is the services onboarding and re-pasting a token to reach it is friction rather than coverage. And it has its own SHIPIT_STATE_DIR at .inner-shipit/onboarding: a separate SQLite db, credential store and agent home. It comes up on "Add a service, and the chat starts working", both harnesses reading no model it can run yet, composer disabled.
Never add a catalogue storageEnv to that block. One line added in good faith turns the fresh instance into a second configured one — the variable is adopted into a stored credential at boot and the install stamps itself onboarded — and the symptom is an absence: the panel under test simply never appears. scripts/seed-inner-credentials.test.ts asserts exact membership so this fails the build instead.
It seeds nothing (DOGFOOD_SEED=0), so the repo list starts empty too. If you would rather have a repo waiting, swap that for DOGFOOD_SEED_CREDENTIALS: "0", which keeps the credential half off whatever is declared.
Reset it to a fresh install — and note that deleting the directory is the whole reset, because everything an install remembers hangs off it:
shipit service stop onboarding && rm -rf .inner-shipit/onboarding && shipit service start onboardingNever test onboarding by deleting credentials from inner Settings or wiping .inner-shipit/ — the first is destructive to the developer's real setup and the second throws away the configured instance they are about to go back to.
Seeding
At dev-service boot, scripts/seed-inner.ts runs in the background and seeds four things in order, all prefixed [seed] in the service logs (docs/131-dogfood-seed-sessions):
- Credentials (
scripts/seed-inner-credentials.ts) — every supplied service key becomes a credential route, labelled… (dogfood secret)in inner Settings → Model providers. - Roles (
scripts/seed-inner-roles.ts) — a few agent roles, so the role surfaces are not empty:deep-dive,quick-look,second-opinion, andneeds-a-credential. Second because it reads what the install can run — a role's harness, model and level are resolved out ofsettings.agents, which step 1 has just widened. - A sample transcript (
scripts/seed-inner-transcript.ts) — one session, "Sample transcript (seeded)", whose conversation is fixture data. See below. - Repos (
scripts/seed-inner-sessions.js) — adds and trusts the repos inscripts/dogfood-seed.json, so the inner UI comes up with a repo ready to work in instead of an empty slate. Last, because a cold clone takes minutes.
Behavior for all four: skips what is already present, exits 0 on any failure (never blocks boot), honors DOGFOOD_SEED=0, and has a switch of its own — DOGFOOD_SEED_CREDENTIALS=0, DOGFOOD_SEED_ROLES=0, DOGFOOD_SEED_TRANSCRIPT=0. A step that throws is logged and the remaining steps still run.
Already-present means left completely alone: a credential or role you edited in the inner UI survives a restart, and rotating the outer secret does not propagate — delete the inner credential to re-seed it. The name is a role's identity, so a re-pointed deep-dive is never reconciled back.
The sample transcript — a conversation to look at, without talking to an agent
Any change to how a conversation renders needs a conversation with the awkward shapes in it. This one is a committed list of turns, each labelled with the shape it covers: a turn that ends in an agent reply, a turn with no agent response text at all, a failed tool call inside a turn that still answers, an error row, a notice, a card that still needs the user, the same card after the user acted, a turn that has both hidden work and an action card, a reply that is a file rather than prose, a long reply, and an ordinary newest turn as the control.
npx tsx scripts/seed-inner-transcript.ts --list # what each turn covers
npx tsx scripts/seed-inner-transcript.ts --force # rewrite it after editing the fixtureIt writes $SHIPIT_STATE_DIR/.shipit.db directly, so you can run it from your own container — /workspace/.inner-shipit is bind-mounted — and the inner UI picks it up on the next history load, no restart. Add a turn by adding an entry, then --force.
Unlike the other steps it keys on the session id and leaves an existing session completely alone, including turns you typed into it yourself. --force is the only thing that overwrites, and it overwrites the whole transcript.
What the seeded roles are for
Nothing is hardcoded: a role's (harness, service, billing mode, model, level) is resolved against what this install can actually run, so the set differs per install and a role is never stranded by someone else's secrets. reviewer is never written — it exists on every install and resolves its params per run (docs/264-agent-roles req 2).
needs-a-credential is deliberately unavailable, so the "shown, disabled, with its reason" state is visible too: the seeder derives it by taking the first catalogue entry its harness can speak to that this install holds no credential for, so it reads Provider disconnected in Settings → Roles and is greyed out in the composer's role menu. An install that holds every credential simply gets no such role. Connecting that service turns it into an ordinary working role — that is not a bug.
Delete a seeded role in the inner UI and the next dev boot puts it back. To try the empty-install state instead, set DOGFOOD_SEED_ROLES: "0" on the dev service.
Driving the inner instance over HTTP
You do not have to click through the inner UI. The dev service publishes 3000 and Vite proxies /api to the inner orchestrator.
First resolve the host, against the outer orchestrator:
curl -s http://${SHIPIT_HOST}:${SHIPIT_PORT}/api/sessions/${SHIPIT_SESSION_ID}/servicesThen these calls cover the whole loop:
| Call | Purpose |
GET /api/sessions/all | What sessions exist |
POST /api/sessions/headless {repoUrl, initialPrompt} | Start a new session |
POST /api/sessions/:id/agent/dispatch {text} | Send a turn — wakes a session nobody has open |
GET /api/sessions/:id/status | Is it still running? |
GET /api/sessions/:id/history | Read the conversation back |
Key references
docs/118-shipit-ui-local— local-mode design and degraded behaviorsdocs/131-dogfood-seed-sessions— seedingdocs/264-agent-roles— what a role is, and why the reviewer's params resolve rather than pindocs/184-remove-platform-secret-forwarding— why credentials are user-suppliedsrc/server/shipit-docs/preview.md(shipped to containers as/shipit-docs/preview.md) — resolving service hosts from inside a container
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

