pipeline-orchestrator
Skill0 starsChanged 7 months agoMITView on GitHub
Staged execution pipeline with plan→prd→exec→verify→fix loop, dual reasoning (think/act), parallel sub-agent spawning, and resumable state. Inspired by OMC and Chat2Graph. Use for any complex multi-step task that benefits from structured decomposition, verification, and automatic retry.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- zhanguoliang715/agent-pipeline-skillstheir own version, first seen Nov 8, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- zhanguoliang715/agent-pipeline-skills★ 1same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
The file
Show its rules and text
We split this skill into the rules it gives an agent
33 rules. About 0 are common, 16 are distinctive, and 17 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈16
Distinctive · estimate
This author's own choices. Worth a look.
17
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 66
Before creating the plan, reason through:
- Very specific: line 75
Write state/pipeline-{id}/plan.json with this structure:
- Distinctive: line 163
Every acceptance criterion must be binary: pass or fail.
- Distinctive: line 166
If you can't write a verification command, write a clear checklist item instead.
- Distinctive: line 177
Before spawning agents, reason through:
- Distinctive: line 179
Do we have all prerequisite outputs from prior groups?
- Distinctive: line 182
Should any tasks be merged to avoid conflicts?
- Very specific: line 245
Update progress.json as each sub-agent completes:
- Very specific: line 246
When spawning: set status: "running", record startedAt and agentSessionId
- Very specific: line 247
When complete: set status: "completed", record completedAt, result, filesModified, selfAssessment
- Very specific: line 248
When failed: set status: "failed", record error details in result
- Distinctive: line 268
Purpose: Check every sub-task against its acceptance criteria.
- Distinctive: line 271
Before verifying, reason through:
- Distinctive: line 274
Do the automated verification commands still make sense given what was actually built?
- Distinctive: line 283
Cross-task integration: If multiple tasks modified related files, check for conflicts or inconsistencies.
- Distinctive: line 346
Before fixing, reason through:
- Very specific: line 391
Update meta.json with attempt count after each loop.
- Distinctive: line 399
Every stage uses Think → Act.
- Very specific: line 435
All state lives in ~/.openclaw/workspace/state/pipeline-{id}/.
- Distinctive: line 465
When the agent starts and a pipeline command is given (or on "Resume pipeline X"):
- Very specific: line 466
Read meta.json to find current stage and status.
- Very specific: line 467
If status is "executing" — check progress.json for incomplete tasks.
- Very specific: line 468
If status is "verifying" — re-run verification from scratch.
- Very specific: line 469
If status is "fixing" — check attempt count, continue fix loop.
- Very specific: line 470
If status is "completed" or "cancelled" — inform user, no action needed.
- Very specific: line 511
Create state directory: mkdir -p ~/.openclaw/workspace/state/pipeline-{id}
- Very specific: line 512
Write meta.json with status "planning"
- Very specific: line 520
If no ID given, find most recent pipeline in state/ directory
- Very specific: line 521
Read meta.json and progress.json
- Very specific: line 541
Update meta.json: set status to "cancelled"
- Distinctive: line 542
Do NOT delete state files (useful for post-mortem)
- Distinctive: line 569
VERIFY: Run test commands, check file existence, validate output format.
- Distinctive: line 571
FIX: If t4 validation logic fails edge case → re-spawn only t4 with failure context.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Pipeline Orchestrator
A structured execution engine for complex, multi-step tasks. Decomposes work into a dependency graph, writes acceptance criteria, executes with parallel sub-agents, verifies results, and auto-retries failures — all with resumable state.
When to Use This Skill
Trigger phrases:
- "Run pipeline: ..."
- "Pipeline status"
- "Resume pipeline ..."
- "Cancel pipeline ..."
Good fit when:
- Task has 3+ distinct sub-tasks
- Sub-tasks have dependencies (some must finish before others start)
- Quality matters — you want verification, not just execution
- Task is complex enough that a single-shot attempt risks missing pieces
- You want parallel execution to save time
Skip this when:
- Simple single-step task (just do it)
- Pure conversation / Q&A
- Task is exploratory with no clear deliverables
Architecture Overview
┌─────────────────────────────────────────────────────┐
│ PIPELINE │
│ │
│ ┌──────┐ ┌──────┐ ┌──────┐ ┌────────┐ │
│ │ PLAN │──▶│ PRD │──▶│ EXEC │──▶│ VERIFY │ │
│ └──────┘ └──────┘ └──────┘ └────────┘ │
│ │ │
│ pass │ fail │
│ ▼ ▼ │
│ DONE ┌─────┐ │
│ │ FIX │──┐ │
│ └─────┘ │ │
│ ▲ │ │
│ └──────┘ │
│ (max 3 loops) │
│ │
│ State: ~/.openclaw/workspace/state/pipeline-{id}/ │
└─────────────────────────────────────────────────────┘Each stage uses the Dual Reasoning pattern: Think first (analyze, plan, assess risk), then Act (execute the plan, no improvisation).
Pipeline Stages
Stage 1: PLAN
Purpose: Decompose the task into a directed acyclic graph (DAG) of sub-tasks.
Think Phase: Before creating the plan, reason through:
- What are ALL the discrete pieces of work?
- What depends on what? (draw the DAG mentally)
- Which tasks can run in parallel?
- What's the complexity of each? (simple: <5 min, medium: 5-20 min, complex: 20+ min)
- What could go wrong at each step?
- Are there any implicit dependencies the user didn't mention?
Act Phase:
Write state/pipeline-{id}/plan.json with this structure:
{
"id": "pipeline-abc123",
"task": "Original task description from user",
"created": "2026-04-04T04:00:00Z",
"tasks": [
{
"id": "t1",
"name": "Short descriptive name",
"description": "What needs to be done",
"dependencies": [],
"complexity": "simple|medium|complex",
"parallelGroup": 1,
"contextFiles": ["path/to/relevant/file.js"]
},
{
"id": "t2",
"name": "Another task",
"description": "Depends on t1 output",
"dependencies": ["t1"],
"complexity": "medium",
"parallelGroup": 2,
"contextFiles": []
}
],
"parallelGroups": {
"1": ["t1", "t3"],
"2": ["t2", "t4"],
"3": ["t5"]
},
"estimatedDuration": "~15 minutes",
"risks": ["Risk 1", "Risk 2"]
}Key rules:
parallelGroupnumbers define execution waves. Group 1 runs first, then group 2 after group 1 completes, etc.- Tasks in the same
parallelGroupMUST have no dependencies on each other. dependencieslists task IDs that must complete before this task starts.contextFiles— only include files the sub-agent actually needs. Less is more.
Advisory model: Sonnet (text generation, no code execution needed).
Stage 2: PRD (Product Requirements Document)
Purpose: Define explicit, testable acceptance criteria for every sub-task.
Think Phase: For each task in the plan, reason through:
- What does "done" look like, specifically?
- How will we verify this programmatically vs. manually?
- What edge cases should the acceptance criteria cover?
- Are the criteria tight enough to catch a bad implementation?
Act Phase:
Write state/pipeline-{id}/prd.json:
{
"pipelineId": "pipeline-abc123",
"requirements": [
{
"taskId": "t1",
"taskName": "Short descriptive name",
"acceptanceCriteria": [
"File X exists at path Y",
"Function Z returns correct output for inputs [a, b, c]",
"No lint errors in modified files"
],
"verificationMethod": "automated|manual|hybrid",
"verificationCommands": [
"test -f path/to/file.js && echo PASS || echo FAIL",
"node -e \"require('./module').test()\" 2>&1"
],
"verificationChecklist": [
"Output matches expected format",
"No regressions in existing functionality"
]
}
]
}Key rules:
- Every acceptance criterion must be binary: pass or fail. No "looks good enough."
verificationCommands— shell commands that return PASS/FAIL or exit 0/1. Used in VERIFY stage.verificationChecklist— for things that need agent judgment (manual review).- If you can't write a verification command, write a clear checklist item instead.
Advisory model: Sonnet (text generation, analytical).
Stage 3: EXEC (Execution)
Purpose: Execute sub-tasks using parallel sub-agents where possible.
Think Phase: Before spawning agents, reason through:
- Which parallel group are we executing?
- Do we have all prerequisite outputs from prior groups?
- What's the minimal context each sub-agent needs?
- Are there any shared resources that could conflict (same file edited by two agents)?
- Should any tasks be merged to avoid conflicts?
Act Phase:
Parallel execution via sessions_spawn:
For each parallel group (in order), spawn sub-agents for all tasks in that group. Maximum 4 concurrent sub-agents.
Each sub-agent prompt MUST include:
- Task description (from plan.json)
- Acceptance criteria (from prd.json)
- Relevant context (read contextFiles and include content, not just paths)
- Output expectations (what files to create/modify, what to report back)
- Constraints (don't modify files outside scope, don't install new dependencies without noting it)
Sub-agent prompt template:
You are executing a sub-task in a pipeline.
## Your Task
{task.description}
## Acceptance Criteria (you MUST meet ALL of these)
{for each criterion in prd.requirements[taskId].acceptanceCriteria}
- [ ] {criterion}
## Context
{content of contextFiles}
## Output
When done, report:
1. What you did (brief)
2. Files created/modified (list)
3. Self-assessment against each acceptance criterion (pass/fail per criterion)
4. Any issues or concerns
Do NOT modify files outside your task scope.
Do NOT install packages without documenting them.
Stay focused. Complete the task. Report results.Progress tracking:
Initialize and update state/pipeline-{id}/progress.json:
{
"pipelineId": "pipeline-abc123",
"currentGroup": 1,
"tasks": {
"t1": {
"status": "pending|running|completed|failed",
"startedAt": null,
"completedAt": null,
"agentSessionId": null,
"result": null,
"filesModified": [],
"selfAssessment": {}
}
}
}Update progress.json as each sub-agent completes:
- When spawning: set
status: "running", recordstartedAtandagentSessionId - When complete: set
status: "completed", recordcompletedAt,result,filesModified,selfAssessment - When failed: set
status: "failed", record error details inresult
Execution flow:
for each parallelGroup in order:
spawn sub-agents for all tasks in group (max 4 concurrent)
yield and wait for all to complete
update progress.json
if any failed: continue (will be caught in VERIFY)
proceed to next groupIf a parallel group has >4 tasks: batch them in sets of 4. Wait for a batch to complete before spawning the next.
Advisory model: Opus (actual implementation work — sub-agents need the best model).
Stage 4: VERIFY
Purpose: Check every sub-task against its acceptance criteria.
Think Phase: Before verifying, reason through:
- Which tasks reported success vs. failure in self-assessment?
- Are there any suspicious "pass" claims that need deeper checking?
- Do the automated verification commands still make sense given what was actually built?
- Are there cross-task integration concerns (task A and B both pass individually but break together)?
Act Phase:
For each task, run verification:
- Automated checks: Execute each
verificationCommandfrom prd.json viaexec. Record stdout/stderr and exit code. - Checklist review: For each
verificationChecklistitem, read the relevant files and assess pass/fail. - Cross-task integration: If multiple tasks modified related files, check for conflicts or inconsistencies.
Write state/pipeline-{id}/verify.json:
{
"pipelineId": "pipeline-abc123",
"verifiedAt": "2026-04-04T04:15:00Z",
"attempt": 1,
"results": [
{
"taskId": "t1",
"verdict": "pass|fail|partial",
"criteria": [
{
"criterion": "File X exists at path Y",
"result": "pass",
"evidence": "File exists, 142 lines"
},
{
"criterion": "Function Z returns correct output",
"result": "fail",
"evidence": "Returns undefined instead of expected array",
"failureDetails": "Function exists but doesn't handle empty input case"
}
],
"automatedResults": [
{
"command": "test -f path/to/file.js",
"exitCode": 0,
"output": "",
"result": "pass"
}
]
}
],
"summary": {
"total": 5,
"passed": 3,
"failed": 1,
"partial": 1,
"allPassed": false
}
}Verdict logic:
pass— ALL acceptance criteria metfail— one or more criteria clearly not metpartial— criteria ambiguous or partially met (treat as fail for retry purposes)
If all passed: Pipeline complete. Update meta.json status to "completed". Report to user.
If any failed/partial: Proceed to FIX stage.
Advisory model: Sonnet (comparison/checking, not implementation).
Stage 5: FIX (Bounded Retry Loop)
Purpose: Re-execute only failed sub-tasks with additional context about what went wrong.
Think Phase: Before fixing, reason through:
- What exactly failed and why?
- Is the failure in the implementation, or in the acceptance criteria (bad spec)?
- Does the fix require changes to upstream tasks?
- Have we tried this fix before? (check prior attempts in verify.json)
- Is this fixable by an agent, or does it need human intervention?
Act Phase:
For each failed/partial task, spawn a fix sub-agent with an enhanced prompt:
You are FIXING a failed sub-task (attempt {N} of 3).
## Original Task
{task.description}
## What Was Done Previously
{previous result from progress.json}
## What Failed
{failure details from verify.json}
## Acceptance Criteria (you MUST meet ALL)
{criteria list}
## Fix Instructions
Focus specifically on the failure points. Do not redo work that already passes.
The following criteria FAILED:
{list of failed criteria with evidence}
Fix these issues. Report your changes.Retry logic:
for attempt in 1..3:
fix failed tasks
run VERIFY again
if all pass: done
if still failing after attempt 3:
mark remaining failures as "needs-human"
report to user with detailsUpdate meta.json with attempt count after each loop.
Advisory model: Opus (needs deep understanding to fix failures).
Dual Reasoning Pattern
Every stage uses Think → Act. This is implemented as a prompting discipline, not separate model calls.
Think prompt prefix (prepend to every stage):
THINK PHASE — Do not take any action yet.
Analyze the following before proceeding:
1. SITUATION: What is the current state? What do we know?
2. RISKS: What could go wrong in this stage?
3. DEPENDENCIES: What does this stage depend on? What depends on it?
4. EDGE CASES: What non-obvious scenarios should we handle?
5. PRIOR FAILURES: Have previous attempts failed? Why? (check verify.json)
6. PLAN: What specific steps will we take, in what order?
Output your reasoning in the format above, then proceed to ACT.Act prompt suffix (append after think output):
ACT PHASE — Execute the plan from your THINK phase.
Rules:
- Follow your plan. No improvisation.
- If you discover something unexpected, note it but stay on plan.
- Report results in structured format.
- If blocked, explain why rather than guessing.See references/dual-reasoning.md for full pattern details.
State Management
All state lives in ~/.openclaw/workspace/state/pipeline-{id}/.
Files:
| File | Created at | Updated at | Purpose |
meta.json | PLAN start | Every stage | Pipeline metadata, current stage, status |
plan.json | PLAN end | Never | Task decomposition & DAG |
prd.json | PRD end | Never | Acceptance criteria |
progress.json | EXEC start | During EXEC, FIX | Execution status per task |
verify.json | VERIFY end | Each VERIFY run | Verification results |
meta.json schema:
{
"id": "pipeline-abc123",
"task": "Original task description",
"status": "planning|prd|executing|verifying|fixing|completed|failed|cancelled|needs-human",
"currentStage": "PLAN|PRD|EXEC|VERIFY|FIX",
"created": "ISO timestamp",
"updated": "ISO timestamp",
"completedAt": null,
"fixAttempts": 0,
"maxFixAttempts": 3,
"taskCount": 5,
"passedCount": 0,
"failedCount": 0
}Resumability: When the agent starts and a pipeline command is given (or on "Resume pipeline X"):
- Read
meta.jsonto find current stage and status. - If status is
"executing"— checkprogress.jsonfor incomplete tasks. Re-spawn only those. - If status is
"verifying"— re-run verification from scratch. - If status is
"fixing"— check attempt count, continue fix loop. - If status is
"completed"or"cancelled"— inform user, no action needed.
See references/state-schema.md for full JSON schemas.
Pipeline ID Generation
Generate a short, unique ID for each pipeline:
date +%s | md5sum | head -c 8Or in the agent: use the first 8 characters of a hash of the task description + timestamp. The ID must be filesystem-safe (alphanumeric + hyphens only).
Smart Model Routing (Advisory)
These are recommendations. The orchestrating agent should request the appropriate model when spawning sub-agents, if the platform supports it.
| Stage | Recommended Model | Rationale |
| PLAN | Sonnet | Text decomposition, no code needed |
| PRD | Sonnet | Analytical writing, criteria definition |
| EXEC | Opus | Actual implementation, code quality matters |
| VERIFY | Sonnet | Comparison against criteria, lightweight |
| FIX | Opus | Debugging requires deep understanding |
In OpenClaw, model can be suggested via sub-agent instructions ("Use careful, thorough reasoning for this implementation task").
Invocation Reference
Start a new pipeline
User says: "Run pipeline: Build a REST API with auth, CRUD for users, and deploy script"
Agent action:
- Generate pipeline ID
- Create state directory:
mkdir -p ~/.openclaw/workspace/state/pipeline-{id} - Write
meta.jsonwith status"planning" - Execute PLAN stage
- Continue through stages automatically
Check status
User says: "Pipeline status" or "Pipeline status abc123"
Agent action:
- If no ID given, find most recent pipeline in
state/directory - Read
meta.jsonandprogress.json - Report: stage, status per task, pass/fail counts, elapsed time
Quick check via script:
bash ~/.openclaw/workspace/skills/pipeline-orchestrator/scripts/pipeline-status.sh [pipeline-id]Resume a pipeline
User says: "Resume pipeline abc123"
Agent action:
- Read
state/pipeline-abc123/meta.json - Determine last incomplete stage
- Resume from that stage (see Resumability section above)
Cancel a pipeline
User says: "Cancel pipeline abc123"
Agent action:
- Update
meta.json: set status to"cancelled" - Do NOT delete state files (useful for post-mortem)
- Confirm cancellation to user
Execution Walkthrough (Example)
Task: "Build a CLI tool that converts CSV to JSON with validation"
PLAN output:
t1: Parse CLI arguments (simple, group 1)
t2: CSV reader module (medium, group 1)
t3: JSON writer module (simple, group 1)
t4: Validation logic (medium, group 2, depends: t2)
t5: Integration + main entry point (medium, group 3, depends: t1,t2,t3,t4)
t6: Tests (medium, group 4, depends: t5)Parallel groups:
- Group 1: t1, t2, t3 (spawn 3 sub-agents simultaneously)
- Group 2: t4 (depends on t2 — wait for group 1)
- Group 3: t5 (depends on all prior)
- Group 4: t6 (depends on t5)
EXEC: 3 sub-agents run in parallel for group 1, then sequential for groups 2-4.
VERIFY: Run test commands, check file existence, validate output format.
FIX: If t4 validation logic fails edge case → re-spawn only t4 with failure context. Re-verify. Up to 3 attempts.
Error Handling
| Scenario | Behavior |
| Sub-agent crashes / no response | Mark task as failed, include in FIX stage |
| State file corrupted | Re-create from available state, log warning |
| All 3 fix attempts exhausted | Set status needs-human, report details to user |
| User cancels mid-execution | Set status cancelled, sub-agents may still complete (orphaned work is ok) |
| Circular dependency detected in plan | Abort PLAN stage, report error, ask user to clarify |
| >20 sub-tasks in plan | Warn user about complexity, suggest breaking into multiple pipelines |
File References
- Stage details:
references/pipeline-stages.md - Dual reasoning pattern:
references/dual-reasoning.md - State file schemas:
references/state-schema.md - Status script:
scripts/pipeline-status.sh
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

