Dev Pipeline Skill

You orchestrate runoff (MCP server v3.0). The host calls MCP tools; this skill tells you which tool when.

First action on every session: runoff_show_config — providers, pipeline steps, routing, memory backend, dreamify state.

Setup (human): npm run setup:mcp -- --host cursor · docs/guides/mcp-host-setup.md

Decision tree

What does the user want?
│
├─ Run / change code in a repo
│   ├─ Full DAG (implement → review → retry)     → runoff_run_pipeline
│   ├─ One step only                             → runoff_run_step
│   ├─ Parallel providers + judge                → runoff_run_pipeline (mode: race)
│   │       then pause                           → runoff_race_apply | runoff_race_abort
│   └─ Resume after approval / checkpoint        → runoff_run_pipeline (sessionId + approvalDecision)
│
├─ Understand or edit configuration
│   ├─ Current config summary                    → runoff_show_config
│   └─ Agent graph (JSON / Mermaid / HTML)       → runoff_show_agent_graph
│
├─ Debug a run
│   ├─ List active/paused runs                   → runoff_query_runs
│   ├─ List / inspect traces                     → runoff_query_traces
│   ├─ Score a trace (human eval)                → runoff_score_trace
│   └─ A/B experiments / eval report             → runoff_query_experiments
│
├─ Evolve the harness — extracted to the standalone `agent-evolution` project
│   (host-agnostic `propose → evaluate → accept` loop; no longer part of runoff)
│
├─ Memory & learning
│   ├─ Backend status (+ optional remote probe)    → runoff_memory_status (probe=true)
│   ├─ Search patterns / lessons                 → runoff_query_memory
│   ├─ Offline evolution (batch)                 → runoff_dream_run
│   ├─ Tune retrieval hyperparams                → runoff_dreamify_tune
│   └─ Export memory for external ingest         → runoff_dream_export
│
└─ Unsure                                       → runoff_show_config, then ask user goal

Tier 1 — Onboarding (first run)

1. runoff_show_config — confirm providers (mock vs CLI vs API).
2. runoff_run_pipeline with a small prompt and workDir pointing at the target repo.
3. If mock: no API keys needed. For real CLI agents see docs/guides/coding-agent-backends.md.

Example configs: examples/configs/ · scaffold: npm run pipeline:init -- --work-dir <dir> --profile feature

Tier 2 — Operator (debug & observability)

Enable orchestration.dream.promoteGlobalKnowledge: true to promote approved-run trace insights into lesson memory (Dream rule B7).

Hot-path remote pattern search requires orchestration.memoryHybridRetrieve: true (default off).

Memory architecture: docs/architecture/memory-layers.md

Harness maturity boundary: harness-evolution control-plane artifacts (datasets, verifiers, promotion bundles) live in the standalone agent-evolution project; its audit material never mutates user repositories — user-repo edits still go through pipeline/workspace paths and explicit accept/apply decisions.

MCP response contract

Non-JSON exceptions: runoff_show_agent_graph with format=mermaid|html|editor|canvas returns raw text/HTML (not JSON).

Hosts must parse JSON body fields (status, error, reason, cleanupErrors) — isError alone is not sufficient for pipeline/step/race semantics.

Pipeline results may include observation, and completed steps may include stepResults.<step>.observation. These deterministic, schema-versioned work-memory summaries carry evidence, coverage gaps, next-action hints, and references back to artifacts, checkpoints, and traces. Use them for next-turn reasoning, then inspect artifacts/traces for complete audit material.

Host reading order after runoff_run_pipeline: parse JSON → inspect observation.status / observation.nextHint → inspect relevant stepResults.<step>.observation → open referenced artifacts or query traces when full evidence is needed. For needs_clarification, answer scopePreflight.clarificationQuestions and resume with the same sessionId plus explicit scopePreflight overrides. For awaiting_judge, follow observation.nextHint and then call runoff_race_apply or runoff_race_abort; for awaiting_approval, resume with approvalDecision.

Host resume explanation

• Parse JSON body from content[0].text — do not rely on isError alone.
• Prefer PipelineResult.resumeReusePlan → observation.resumeReusePlan → runoff_query_runs format=full → run.resumePlanner.
• Explain rerun steps first; include downstreamOf when present; collapse skipped to counts unless the user asks for audit.
• When no planner is present, do not claim "no reruns" or "all reused" — say planner evidence is unavailable.

Full decision matrix, bilingual templates, and anti-patterns: docs/guides/host-resume-ux.md.

Tier 3 — Config author (graph & governance)

Orchestration modes (orchestration.mode): dag (default) · workflow · llm-driven.

Governance order: Policy → Guardrails → Approval. See docs/architecture/governance-config.md.

All MCP tools (16)

Execution modes

Pipeline (default)

Sequential / parallel stages per pipeline.config.json. Review step can trigger retry (retry.maxRounds).

runoff_run_pipeline(prompt: "...", workDir: "/abs/path/to/repo")

Race mode

Multiple providers in parallel; pipeline pauses for judge; finalize with race tools.

runoff_run_pipeline(prompt: "...", mode: "race")
# after judge selects winner:
runoff_race_apply(sessionId: "...")
# or:
runoff_race_abort(sessionId: "...")

Smart routing & cache

• Routing: complexity → provider (routing rules in config).
• Cache: identical prompt + provider → cached response (TTL ~30min, LRU 64). Stats in runoff_show_config.

CLI helpers (not MCP)

Do not

• Call runoff_run_step for steps marked builtin in config (host-native agents).
• Assume remote memory (Mem0/Zep) is read on the hot pipeline path — hooks use local sync; use runoff_query_memory for hybrid.
• Skip runoff_race_apply / runoff_race_abort after race mode — workspaces stay open until finalized.

Docs hub

• docs/README.md
• docs/guides/getting-started-30min.md
• ROADMAP.md · open items issues/OPEN-BACKLOG.md