/build-from-screenshot — Autonomous Screenshot/URL-to-Working-App Pipeline

You are the master orchestrator for converting screenshots or a live URL into a fully working, tested application. You receive one or more image file paths OR a URL and guide the entire process through 12 phases, using specialized skills and agents. The output target (React, Vue, Svelte, React Native) is determined during intake and drives agent selection in the build phase.

Key enforcement rules:

• TDD is mandatory — Phase 3 (TDD) MUST complete before Phase 4 (Build). No exceptions.
• Visual QA uses pixel diff — Phase 5 uses scripts/visual-diff.js for programmatic comparison, not manual eyeballing.
• E2E tests are generated — Phase 6 generates and runs Playwright E2E tests appropriate to the app type.
• App-type aware — Chrome extensions, PWAs, and web apps each get tailored test strategies.
• Token inference requires confirmation — Phase 2 extracts tokens via AI vision and MUST get user confirmation before locking.
• Renderer-driven — Phase 4 dispatches the converter agent named by the resolved renderer manifest (renderer-registry.js resolve <renderer>), not a hardcoded outputTarget table.

Input

The user provides: $ARGUMENTS (a URL or one or more image file paths)

Detect input type:

• If $ARGUMENTS starts with http:// or https:// → URL capture mode
  • The pipeline will navigate to the URL and capture screenshots automatically
  • Store the original URL for the build report
• Otherwise → File paths mode
  • Treat $ARGUMENTS as one or more image file paths (space-separated or glob)
  • Verify each file exists before proceeding

URL capture mode steps:

1. Use Chrome DevTools MCP to navigate to the URL: mcp__chrome-devtools__navigate_page
2. Resize to standard viewports (desktop 1440px, tablet 768px, mobile 375px): mcp__chrome-devtools__resize_page
3. Take screenshots at each viewport: mcp__chrome-devtools__take_screenshot
4. Save screenshots to .claude/visual-qa/screenshots/source/

Configuration

Load .claude/pipeline.config.json at the start. This provides:

• Visual diff thresholds and iteration limits
• TDD enforcement settings
• E2E strategy per app type
• Quality gate thresholds
• Lighthouse score minimums
• Token inference confidence threshold

Progress Tracking

Track the run in .claude/plans/pipeline-state.json, in the shape /build-from-figma defines (.claude/skills/build-from-figma/SKILL.md → Progress Tracking), with "pipeline": "screenshot" and the URL or file list as source. Rewrite it at every phase boundary and after each component and page result; the SessionStart hook reads it to report where an interrupted run stopped. Phases the renderer excludes are recorded as skipped.

Phases: 0 token-sync, 1 intake (screenshot-intake), 2 token-inference (canva-token-inference, needs user confirmation), 3 tdd-scaffold, 4 component-build, 4.5 storybook, 5 visual-diff (against the source screenshots), 5.5 dark-mode, 6 e2e-tests, 7 cross-browser, 8 quality-gate, 8.5 responsive, 9 report.

Phase 0: Token Drift Check (Conditional)

Identical to /build-from-figma. Only runs when tokenSync.autoCheck is true AND a lockfile exists.

./scripts/sync-tokens.sh --json

Phase 1: Intake

Invoke the screenshot-intake skill.

Input: The URL or file paths from $ARGUMENTS, plus captured screenshots (if URL mode) Output: .claude/plans/build-spec.json with "source": "screenshot" and "outputTarget": "react"|"vue"|"svelte"|"react-native"

This phase:

1. If URL mode: screenshots are already captured during input processing
2. If file mode: validates and catalogs the provided image files
3. Analyzes screenshots with Claude vision for structure, components, text, layout
4. Scans the local project for framework, existing components, UI libraries
5. Asks the user 3-5 targeted questions (scope, output target, component confirmation, reuse, business logic, integration)
6. Writes the build spec including outputTarget based on the user-chosen framework

Resume check: If .claude/plans/build-spec.json already exists with "source": "screenshot", ask the user if they want to reuse it or regenerate.

Phase 2: Token Inference

Invoke the canva-token-inference skill.

Input: build-spec.json with screenshot paths (source: "screenshot") Output: src/styles/design-tokens.lock.json, framework-appropriate config, src/styles/tokens.css

This phase:

1. Analyzes source screenshots with Claude vision to extract colors, typography, spacing, effects
2. Assigns confidence scores (high/medium/low) to each token
3. Presents tokens to user for confirmation — this is MANDATORY, do not skip
4. After user confirms/corrects, writes the lockfile
5. Generates framework-appropriate config (Tailwind for React/Vue/Svelte, StyleSheet tokens for React Native) and CSS custom properties from lockfile
6. Validates completeness

Resume check: If src/styles/design-tokens.lock.json already exists with "source": "screenshot", ask the user if they want to reuse it or re-infer.

Phase 3: TDD Scaffold

Invoke the tdd-from-figma skill. (This skill reads build-spec.json and works identically regardless of source.)

Input: build-spec.json + design-tokens.lock.json Output: Test files appropriate to the output target

Identical to /build-from-figma Phase 3.

Phases 4-9: Component Lanes, then Parallel Phases

After Phase 3 confirms RED, resolve the renderer manifest (node scripts/renderer-registry.js resolve <renderer> --json) and run Phases 4-9 exactly as /build-from-figma describes (.claude/skills/build-from-figma/SKILL.md → Phases 4-9): the component-lanes workflow for Phases 4-5 when the Workflow tool is available, then parallel-orchestration for Phases 4.5-9. The screenshot differences:

• Workflow args: "source": "screenshot"; converter is manifest.converter (figma-react-converter, the generic React builder, for React renderers; react-native-converter for expo); testCommand is manifest.commands.test. The page references are the source captures in .claude/visual-qa/screenshots/source/, keyed by the width they were captured at. Set skipVisual when the manifest excludes visual-diff (expo).
• parallel-orchestration: drop every phase in manifest.phases.exclude (expo excludes visual-diff, dark-mode, cross-browser and responsive), and also cross-browser for appType: "chrome-extension". Context: pipeline source "screenshot", test files src/components/**/*.test.*, reference screenshots .claude/visual-qa/screenshots/source/. e2e-tests follows component-build when visual-diff is excluded.

Without the Workflow tool, invoke parallel-orchestration with all of ["component-build", "storybook", "visual-diff", "dark-mode", "e2e-tests", "cross-browser", "quality-gate", "responsive", "report"] minus those exclusions. If orchestration.enabled is false, run the phases sequentially as documented below.

Phase 4: Component Build (Renderer-Driven)

Read renderer from build-spec.json, resolve its manifest, and dispatch the converter it names:

node scripts/renderer-registry.js resolve <renderer> --json

Converter selection:

• If manifest.language === "react", prefer the source-appropriate React converter. For the screenshot pipeline that is figma-react-converter (the generic React builder) — it builds React components from reference screenshots regardless of source. The shipped React manifests (nextjs, vite) already set converter to figma-react-converter.
• Otherwise dispatch manifest.converter directly (e.g. react-native-converter for the expo renderer, and the future vue-converter / svelte-converter).

The component file extension, directory, and page-routing convention come from manifest.component (ext, dir, pageRouting); the test command comes from manifest.commands.test.

Input: build-spec.json, resolved renderer manifest, lockfile, existing test files, source screenshots Output: Component and page files for the renderer's framework

This phase:

1. Reads build-spec.json — verifies source is "screenshot" and reads renderer
2. Resolves the manifest and dispatches the converter (see selection rule above)
3. References lockfile for all token values (no approximating)
4. Uses source screenshots for layout/structure decisions
5. Generates components (at manifest.component.dir with manifest.component.ext) that satisfy the test files from Phase 3
6. Runs manifest.commands.test after each component batch to confirm GREEN

Critical rule: If tests fail, fix the component — never modify the test files.

Phase 4.5: Storybook Generation (Non-Blocking)

Identical to /build-from-figma Phase 4.5. Skipped when the renderer has no browser story (e.g. expo).

./scripts/generate-stories.sh

Phase 5: Visual Verification (Automated Pixel Diff)

Same process as /build-from-figma Phase 5, but reference screenshots come from the source captures (URL screenshots or provided image files).

Reference screenshots: Stored in .claude/visual-qa/screenshots/source/

For each page:

1. Start: pnpm dev (background) — skip if appType is chrome-extension or the renderer excludes `visual-diff` (e.g. expo)
2. Wait for server ready

3. Reference screenshots already exist in .claude/visual-qa/screenshots/source/

4. FOR iteration IN 1..maxVisualIterations:
   a. Chrome DevTools MCP: navigate → resize → take_screenshot
      → Save to .claude/visual-qa/screenshots/chromium/

   b. Run pixel diff:
      → node scripts/visual-diff.js --batch \
          .claude/visual-qa/screenshots/chromium \
          .claude/visual-qa/screenshots/source \
          --output-dir .claude/visual-qa/diffs --json

   c. Parse JSON results:
      → IF all mismatchPct <= threshold: PASS → break
      → IF any FAIL and iteration < max:
        - Read diff images + region analysis
        - Fix component code targeting specific regions
        - Run test command (ensure tests still pass)
        - Continue to next iteration

5. Stop dev server

Note: When the resolved renderer excludes visual-diff (e.g. expo, which has no browser rendering), this phase is skipped. Record it as skipped in pipeline-state.json.

Phases 5.5 through 9

Identical to /build-from-figma. All shared phases work the same regardless of design source:

• Phase 5.5: Dark Mode verification (check-dark-mode.sh) — skipped when the renderer excludes dark-mode
• Phase 6: E2E test generation (e2e-test-generator skill)
• Phase 7: Cross-browser screenshots (Firefox, WebKit) — skipped when the renderer excludes cross-browser
• Phase 8: Quality gate (coverage, types, build, tokens, Lighthouse)
• Phase 8.5: Responsive screenshots (check-responsive.sh) — skipped when the renderer excludes responsive
• Phase 9: Build report (.claude/visual-qa/build-report.md)

The build report should note:

• Source: Screenshot (URL) or Source: Screenshot (files) depending on input type
• Source URL: <url> if URL mode was used
• Output Target: react|vue|svelte|react-native
• Include the token inference confidence summary

Error Recovery

• URL unreachable: Check the URL is valid and the site is accessible. Offer to retry or accept manually captured screenshots instead.
• Screenshot capture fails: Ask user to manually capture screenshots and provide file paths.
• Image files not found: List missing files. Ask user to verify paths.
• Token inference low confidence: Present all tokens with detailed confidence breakdown. Offer to accept user-provided brand guidelines as override.
• Converter agent missing: If the converter named by the resolved manifest does not exist yet, inform the user. For react renderers, fall back to figma-react-converter (the generic React builder).
• Dev server will not start: Check for port conflicts, missing dependencies. Run pnpm install if needed.
• Tests will not pass after 3 attempts: Mark component as needing manual intervention, continue with remaining.
• Build fails: Check TypeScript errors first, then dependency issues. Report blockers.
• Session interrupted: On resume, read .claude/plans/pipeline-state.json. Skip completed phases whose artifacts still exist and resume from the first incomplete one.

Completion

When all phases complete, present:

1. The build report summary
2. Source type (URL or files) and output target framework
3. Count of pages/components built and verified
4. Token inference accuracy (how many tokens were confirmed vs corrected)
5. Any items needing manual review
6. Next steps (e.g., "run pnpm dev to see the app")