agentleFS
Kastalien-ResearchKastalien-Research/thoughtbox/.agents/skills/experiment/SKILL.md

experiment

Skill64 starsChanged 4 months agoMITView on GitHub

Run parallel experiments across git worktrees. Define competing approaches to the same problem, dispatch sub-agents to implement each in isolation, then compare results. Use when you have 2-5 alternatives and want to test all of them before committing to one.

27 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use cost, decide and 8 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • maxvaegaSkill

    cost

    Estimate what Claude Cowork / Claude Code conversations would cost if run on the pay-per-token Claude API instead of the subscription. Reports token counts and cache-aware API cost in USD and EUR, for the current conversation or for the whole history (cost per conversation). Trigger when the user asks things like "how much would this conversation cost on the API", "API cost of my Cowork conversations", "token usage / token count", "cost per conversation", "/cost", or wants to compare subscription vs API pricing.

    Also in 3 projects that use this

  • dirienSkill

    decide

    Three-agent adversarial debate protocol for strategic decisions. Two advocates steelman opposing positions while a moderator identifies risks and synthesizes a binding consensus. Use when choosing between alternatives, evaluating tradeoffs, or making high-stakes decisions.

    Also in 3 projects that use this

  • javatarzSkill

    wiki

    Access and manage the local GitHub Wiki clone. Use this skill when you need to read wiki content (architecture, design docs, guides), update wiki documentation, check wiki status, or push wiki changes. The wiki contains high-level system design and architectural documentation.

    Also in 3 projects that use this

  • HeshamFSSkill

    loop

    Run a command or check on a recurring interval

    Also in 3 projects that use this

  • ibahgatSkill

    autopilot

    Run autonomous multi-stage execution loops until acceptance criteria pass or blockers are reached.

    Also in 3 projects that use this

  • AnthropicSkill

    debug

    Structured debugging session — reproduce, isolate, diagnose, and fix. Trigger with an error message or stack trace, "this works in staging but not prod", "something broke after the deploy", or when behavior diverges from expected and the cause isn't obvious.

    Also in 4 projects that use this

The file

Show its rules and text

We split this skill into the rules it gives an agent

17 rules. 0 are common, 15 are distinctive, and 2 are very specific to this project.

  • 0

    Common

    Most of the 25 edited versions we read keep these.

  • 15

    Distinctive

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  • could be automatic: a setting could enforce it
  1. Distinctive:

    Run a parallel experiment: $ARGUMENTS

    line 8
  2. Distinctive:

    Test multiple competing approaches to the same problem simultaneously.

    line 12
  3. Distinctive:

    Conflict check: If two tracks modify the same files in incompatible ways, flag this to the user before proceeding.

    line 40
  4. Distinctive:

    Verify all worktrees were created:

    line 55
  5. Distinctive:

    Write tests for your changes

    line 72
  6. Distinctive:

    Run tests and report results

    line 73
  7. Distinctive:

    Commit your work to the worktree's branch

    line 74
  8. Distinctive:

    After all sub-agents complete, produce a comparison:

    line 138
  9. Distinctive:

    When the user picks a winner:

    line 168
  10. Very specific:

    Worktrees are siblings: Always create worktrees as siblings to the repo (../), never inside it.

    line 186
  11. Distinctive:

    Independent tracks only: If tracks would conflict on the same files, split differently or run sequentially.

    line 188
  12. Very specific:

    Clean branch names: Always use exp/ prefix for experiment branches.

    line 190
  13. Distinctive:

    Do NOT create more than 5 tracks — if there are more alternatives, group them

    line 194
  14. Distinctive:

    Do NOT run experiments for trivial choices (if one option is obviously better, just do it)

    line 195
  15. Distinctive:

    Do NOT let sub-agents work outside their assigned worktree

    line 196
  16. Distinctive:

    Do NOT merge without user approval

    line 197
  17. Distinctive:

    Do NOT leave worktrees around after the experiment concludes — remind the user to clean up

    line 198

“Common” is measured: kept by at least 6 in 10 of the 25 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Run a parallel experiment: $ARGUMENTS

Purpose

Test multiple competing approaches to the same problem simultaneously. Each approach gets its own git worktree and sub-agent. Results are compared at the end so the best approach can be merged.

This is NOT for parallelizing parts of one plan (use /workflows-work for that). This is for when you genuinely don't know which approach is best and want to try all of them.

Process

Phase 1: Define Tracks (orchestrator does this)

Parse the input (a gap analysis, a list of alternatives, a brainstorm, or a user description) and produce a track list:

TRACK DEFINITION
================
Base branch: <current branch or specified base>

Track A: <name>
  Branch: exp/<slug-a>
  Approach: <1-2 sentence description>
  Files touched: <predicted file list>
  Effort: low | medium | high

Track B: <name>
  Branch: exp/<slug-b>
  ...

[up to 5 tracks]

Conflict check: If two tracks modify the same files in incompatible ways, flag this to the user before proceeding. Parallel experiments must be independent.

Present the track list to the user and wait for confirmation before proceeding.

Phase 2: Create Worktrees

For each track, create a worktree branching from the base:

# From the repo root
git worktree add ../<repo-name>-exp-<slug> -b exp/<slug>

Worktree naming convention: ../<repo-name>-exp-<slug> (sibling to the main repo directory).

Verify all worktrees were created:

git worktree list

Phase 3: Dispatch Sub-Agents (parallel)

Dispatch one sub-agent per track using the Agent tool. All independent tracks run in parallel.

Each sub-agent receives:

  1. Working directory: The worktree path (CRITICAL — sub-agent must cd here)
  2. Track definition: What to implement and why
  3. Source context: Relevant file contents from the base branch (read before dispatching)
  4. Acceptance criteria: What "done" looks like for this track
  5. Instructions:
    • Work ONLY in your assigned worktree directory
    • Write tests for your changes
    • Run tests and report results
    • Commit your work to the worktree's branch
    • Return the structured summary (see format below)

Sub-Agent Prompt Template

You are implementing Track [X]: [name]

WORKING DIRECTORY: [worktree path]
BRANCH: exp/[slug]

## What to implement
[approach description]

## Files to modify
[file list with current contents provided inline]

## Acceptance criteria
[specific, testable criteria]

## Rules
1. Work ONLY in [worktree path] — never touch the main repo
2. Write tests first, then implement
3. Run tests: [test command]
4. Commit when tests pass:
   git add [files]
   git commit -m "exp([slug]): [description]"
5. Return the structured summary below

## Return Format
[see Sub-Agent Summary Format]

Sub-Agent Summary Format

Each sub-agent MUST return:

## Experiment Track: [name]

### Result
- Branch: exp/[slug]
- Status: COMPLETE | PARTIAL | FAILED
- Tests: N passing, N failing

### What was built
[2-3 sentences describing the implementation]

### What works well
[Strengths of this approach]

### What doesn't work well
[Weaknesses, rough edges, things that felt wrong]

### Demo readiness
[How well does this serve the original use case?]
Score: 1-5 (1=unusable, 5=demo-ready)

### Files changed
[list with +/- line counts]

Phase 4: Compare Results

After all sub-agents complete, produce a comparison:

EXPERIMENT RESULTS
==================
Question: [original problem being solved]
Base branch: [base]
Tracks tested: [N]

| Criterion        | Track A          | Track B          | Track C          |
|------------------|------------------|------------------|------------------|
| Tests passing    | Y/N              | Y/N              | Y/N              |
| Demo readiness   | [score]/5        | [score]/5        | [score]/5        |
| Code changes     | +N/-N lines      | +N/-N lines      | +N/-N lines      |
| Complexity added  | low/med/high     | low/med/high     | low/med/high     |
| Strengths        | [key strength]   | [key strength]   | [key strength]   |
| Weaknesses       | [key weakness]   | [key weakness]   | [key weakness]   |

RECOMMENDATION: Track [X] because [reason]

NEXT STEPS:
- To adopt Track X: git merge exp/<slug> into <base branch>
- To clean up: git worktree remove ../<repo>-exp-<slug> (for each)
- To inspect: cd ../<repo>-exp-<slug> && git log --oneline

Present the comparison and recommendation. Do NOT merge or clean up worktrees — let the user decide.

Phase 5: Cleanup (only on user request)

When the user picks a winner:

# Merge the winner
git merge exp/<winning-slug>

# Remove all experiment worktrees
git worktree remove ../<repo>-exp-<slug-a>
git worktree remove ../<repo>-exp-<slug-b>
git worktree remove ../<repo>-exp-<slug-c>

# Delete experiment branches
git branch -d exp/<slug-a> exp/<slug-b> exp/<slug-c>

Operational Rules

  1. Orchestrator never implements: You dispatch sub-agents. You read source files to provide context. You do NOT write code yourself.
  2. Worktrees are siblings: Always create worktrees as siblings to the repo (../), never inside it.
  3. Confirm before creating: Present the track list and get user approval before creating worktrees.
  4. Independent tracks only: If tracks would conflict on the same files, split differently or run sequentially.
  5. No premature merging: Present results and let the user choose. The user may want to inspect, combine, or discard.
  6. Clean branch names: Always use exp/ prefix for experiment branches. Never timestamps or UUIDs.

Anti-Patterns

  • Do NOT create more than 5 tracks — if there are more alternatives, group them
  • Do NOT run experiments for trivial choices (if one option is obviously better, just do it)
  • Do NOT let sub-agents work outside their assigned worktree
  • Do NOT merge without user approval
  • Do NOT leave worktrees around after the experiment concludes — remind the user to clean up

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.