Browser Testing Methodology

1. When to Write a Browser Test vs Unit Test

Browser test (Playwright):

• Cross-component flows (sidebar click updates chat panel)
• SSE streaming verification (message send → streaming indicator → response rendered)
• Real API calls through the full stack (client → Hono → Agent SDK)
• CSS/layout regressions visible only in a real browser
• Browser-specific behavior (keyboard shortcuts, focus management)

Unit test (Vitest):

• Individual component rendering and props
• Hook logic and state transitions
• Service functions and data transformations
• Schema validation (Zod)
• Pure utility functions

Rule of thumb: If the behavior spans multiple FSD layers or requires a real server, it's a browser test.

2. Page Object Model Patterns

POMs live in apps/e2e/pages/ and are injected as Playwright fixtures via fixtures/index.ts.

Each POM encapsulates locators and interaction methods for one page or component:

// apps/e2e/pages/FeaturePage.ts
import type { Page, Locator } from '@playwright/test';

export class FeaturePage {
  readonly page: Page;
  readonly primaryAction: Locator;

  constructor(page: Page) {
    this.page = page;
    this.primaryAction = page.getByRole('button', { name: /action/i });
  }

  async doAction() {
    await this.primaryAction.click();
  }
}

Register in fixtures:

// apps/e2e/fixtures/index.ts
import { test as base } from '@playwright/test';
import { FeaturePage } from '../pages/FeaturePage';

export const test = base.extend<{ featurePage: FeaturePage }>({
  featurePage: async ({ page }, use) => {
    await use(new FeaturePage(page));
  },
});

Test files import from fixtures, never from @playwright/test directly:

import { test, expect } from '../../fixtures';

3. Selector Strategy

Priority order:

1. getByRole() — Best: semantic, resilient to UI changes

   page.getByRole('button', { name: /send/i });
   page.getByRole('textbox', { name: /message/i });
   page.getByRole('tab', { name: /settings/i });

2. data-testid — Good: stable contract between test and implementation

   page.locator('[data-testid="chat-panel"]');
   page.locator('[data-testid="message-item"][data-role="assistant"]');

3. CSS class — Last resort: fragile, breaks on styling changes. Avoid unless no other option.

4. Wait Strategy

Never use page.waitForTimeout() or setTimeout. These are flaky and slow.

Instead:

• Element visibility: locator.waitFor({ state: 'visible' })
• Element disappearance: locator.waitFor({ state: 'hidden' })
• Streaming responses: send through the page object, then wait for the turn to end

  await chatPage.sendAndLand('Hello'); // returns once the reply has started
  await chatPage.waitForTurnToEnd(); // returns once the turn has ended

  waitForTurnToEnd() passes at once on an idle chat, so never call it straight after a bare sendMessage. A turn parked on an approval or a question has not ended: answer it first.
• Navigation: await expect(page).toHaveURL(/session=/)
• API calls: await page.waitForResponse(resp => resp.url().includes('/api/sessions'))

5. Test Tagging

• @smoke — Critical path tests, no SDK dependency, fast (<5s)
• @integration — SDK-dependent tests, require ANTHROPIC_API_KEY, slower (10-60s)

Add tags to test.describe() titles:

test.describe('Feature — Description @smoke', () => { ... });

Run by tag: npx playwright test --grep @smoke

6. Debugging Methodology

1. Reproduce: Run the failing test with PWDEBUG=1 or --trace on
2. Snapshot: Use Playwright MCP browser_snapshot to capture the current accessibility tree
   • Several sessions using Playwright MCP at once collide on its default on-disk profile. Run it with --isolated; for a third-party site that needs you signed in, use the agent browser (contributing/browser-verification.md § The agent browser).
3. Compare: Check if expected elements still exist with expected attributes
4. Classify: Is it a TEST bug or CODE bug?
   • TEST bug: Selector changed, timing issue, new UI pattern → update POM/spec
   • CODE bug: Feature regression, broken logic → fix source code
5. Fix: If test bug, update the POM/spec. If code bug, fix the source and ask the user first

7. Manifest Management

apps/e2e/manifest.json is automatically updated by the custom reporter (apps/e2e/reporters/manifest-reporter.ts) after each run. It is gitignored (DOR-726) — every run rewrites its run-stats fields, so it regenerates from nothing rather than staying tracked. apps/e2e/manifest-curated.json is the tracked half: just relatedCode/explorationNotes, which the reporter merges back into manifest.json on every write, so curating a test survives a manifest.json that starts from scratch. Edit the curated file by hand when you set either field for a test; don't hand-edit manifest.json, the next run overwrites it.

• Test entries keyed by spec filename (e.g., send-message)
• specFile/feature refresh from every run that saw the file, so a spec that moves directories is never left pointing at a path that no longer exists
• Run history capped at 100 entries
• /browsertest report reads this file for health dashboards
• /browsertest:maintain uses relatedCode + lastRun for stale detection

description is derived, not curated — never hand-edit it. It is a join of the file's own test titles, kept in sync automatically: never touched by a filtered run (-g, --shard, .only), because that view is partial and would collapse the description to whatever titles the filter happened to match. A full, unfiltered run only refreshes it when E2E_REFRESH_MANIFEST=1 is set (the e2e package script sets it; CI's sharded runs never qualify) — so renaming a test's title is enough, but only a run that both opts in and sees the whole file will pick the rename up. Curated, hand-written context — selectors, timing quirks, gotchas specific to a test — belongs in explorationNotes, which the reporter never touches.

8. DorkOS-Specific Patterns

SSE stream testing: DorkOS uses Server-Sent Events for real-time updates. Test by sending with chatPage.sendAndLand() (the reply has started) and waiting with chatPage.waitForTurnToEnd() (the turn has ended). chatPage.turnRunning is the underlying locator: the chat panel's data-turn-status="streaming" or data-turn-lifecycle="blocked". chatPage.turnComplete is the finished turn's summary in the live lane.

Session URL state: Sessions are tracked via ?session= URL parameter. After creating a new session, verify the URL updates: await expect(page).toHaveURL(/session=/).

Multi-panel layout: The app has a sidebar + main panel layout. Some interactions affect both panels (e.g., clicking a session in sidebar updates chat panel). Test these cross-panel flows with browser tests, not unit tests.

Settings dialog: Opens as a modal overlay. Use Escape key or click-outside to close. The dialog has data-testid="settings-dialog".

Feature-Sliced Design alignment: Test directories mirror FSD features: tests/chat/, tests/session-list/, tests/settings/, tests/pulse/, etc.

9. Learning Methodology

This section guides the exploration phase of test creation — the "learn by doing" loop that discovers selectors and timing through real interaction rather than guessing.

What to Observe During Navigation

At each snapshot, look for:

• Element hierarchy: parent containers, list structures, nested components
• State transitions: loading spinners, skeleton screens, empty states → populated states
• Conditional rendering: elements that appear only after an action (modals, toasts, dropdowns)
• Keyboard accessibility: focus order, aria-expanded, aria-selected attributes
• Dynamic IDs: elements with generated IDs or keys that change between runs

Evaluating Selectors at Each Snapshot

For every element you plan to interact with or assert against:

1. First choice — getByRole(): Check the element's role and accessible name in the snapshot. Prefer { name: /pattern/i } for resilience.
2. Second choice — data-testid: If the role is generic (div, span) or the name is dynamic, check for a testid attribute.
3. Third choice — getByText(): Only for static, stable text content. Never for timestamps, counts, or user-generated content.
4. Avoid: CSS classes, XPath, nth-child — these break on any styling or layout change.

When to Stop Exploring

• You have observed every state transition the test description requires
• You have confirmed selectors for all elements you will interact with or assert
• You have identified the timing characteristics (immediate render vs async load) of each assertion target
• Stop here — do not explore adjacent features or "nice to have" flows

Handling DorkOS Dynamic Content

• SSE streams: Messages arrive via Server-Sent Events. chatPage.turnRunning matches while a turn is running. Always call chatPage.waitForTurnToEnd() before asserting final message content.
• Optimistic updates: Some UI updates appear before server confirmation. Re-locate elements after mutations rather than holding stale references.
• Session side effects: Creating or switching sessions triggers URL changes, sidebar re-renders, and SSE reconnections. Allow these to settle before proceeding.

Building on Previous Knowledge

Before writing a new test:

1. Read GOTCHAS.md — avoid repeating known mistakes.
2. Check explorationNotes in manifest.json for related tests — reuse timing strategies and selector patterns.
3. Review existing POMs — extend them rather than creating parallel locator definitions for the same elements.

10. Mock Browser Tests (TestModeRuntime)

For browser tests that don't need real Claude API responses, use the chromium-mock Playwright project. This runs against a server started with DORKOS_TEST_RUNTIME=true, which registers TestModeRuntime instead of ClaudeCodeRuntime.

Before each test, configure the scenario via the control API:

const MOCK_PORT = process.env.DORKOS_MOCK_PORT || '4243';
await request.post(`http://localhost:${MOCK_PORT}/api/test/scenario`, {
  data: { name: 'simple-text' }, // or 'tool-call', 'todo-write', 'error'
});

Reset between tests with POST /api/test/reset to return to the default scenario.

Mock tests import from @playwright/test directly (not fixtures) and use ChatPage from pages/ChatPage.ts. They live in tests/chat-mock.spec.ts and are matched by testMatch: ['**/chat-mock.spec.ts'] in the chromium-mock project config.

The mock Vite client runs on port 4248 (proxying to the mock server on 4243), keeping it clear of the cockpit leg's own pair (4245 API / 4244 Vite, DORKOS_COCKPIT_PORT / DORKOS_COCKPIT_VITE_PORT). Every leg keeps its data in a throwaway DORK_HOME under /tmp, keyed by its port and deleted before every boot, so no run reads or writes the dev data directory or a real ~/.dork (DOR-1223). Output worth keeping from a local run (reports, traces, screenshots) goes to /Volumes/ext-ssd-1/dork-os/artifacts/<YYYY-MM-DD>-<PR or ticket>/ when that drive is mounted; see contributing/browser-verification.md#where-local-output-goes.