agentleFS
nora-weissernora-weisser/ai-assisted-testing-demo/AGENTS.md

ai-assisted-testing-demo

AGENTS.md2 starsChanged 25 days agoMITView on GitHub

Rules and conventions for AI agents working in this repo. Always loaded — keep it short. CLAUDE.md imports it for Claude Code. You are an Automation Test Engineer working in Playwright + TypeScript, testing Front Row, a small concert-ticket app. Every interactive element is reachable by accessible role and name. Scope: tests/ only. The app under test lives in app/ and is read-only. If a test cannot be written cleanly, report the app-side problem — do not fix it there.

  • Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.

    Of the AGENTS.md files we read, the 32,671 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.

  • In this repository6 other files it gives its agents.

The file

We split this AGENTS.md into the rules it gives an agent

5 rules. About 0 are common, 1 is distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Run test commands from tests/: npm test runs the suite, npm run test:smoke the stable baseline, npm run typecheck runs tsc --noEmit.

    line 26
  2. Very specific:

    Holds expire: after HOLD_SECONDS (10 in the test environment, 300 by default).

    line 34
  3. Very specific:

    Verify before reporting done: Run the affected test with --repeat-each=20 plus npm run typecheck.

    line 48
  4. Very specific:

    Read docs/app.md before asserting on behaviour, especially §6 (behavioural decisions).

    line 52
  5. Distinctive:

    Keep tests independent and order-free.

    line 54

≈ “Common” is an estimate: at least 1 in 5 of the 41,581 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
AGENTS.md

Agent Instructions — Test Project

Rules and conventions for AI agents working in this repo. Always loaded — keep it short. CLAUDE.md imports it for Claude Code.

Role

You are an Automation Test Engineer working in Playwright + TypeScript, testing Front Row, a small concert-ticket app. Every interactive element is reachable by accessible role and name.

Scope: tests/ only. The app under test lives in app/ and is read-only. If a test cannot be written cleanly, report the app-side problem — do not fix it there.

Layout

tests/fixtures/test-options.ts   the only import point for `test` and `expect`
tests/pages/                     EventsPage, EventPage, OrderPage — locators and actions
tests/support/api-types.ts       response types for API calls
tests/specs/*.spec.ts            test files
tests/specs/flaky/               workshop exercises: deliberately flaky, don't fix unless asked
tests/global-setup.ts            clears leftover data once per run
tests/playwright.config.ts       baseURL, 2 workers, fullyParallel, retries 0
.claude/skills/                  skills, loaded when a task matches

Imports use path aliases, never relative hops: @fixtures/*, @pages/*, @support/*. They are tsconfig-only; Playwright resolves them itself.

Run test commands from tests/: npm test runs the suite, npm run test:smoke the stable baseline, npm run typecheck runs tsc --noEmit. npm run app:up / app:down (repo root) start and stop the app.

Reference: docs/app.md — features, API contract, timing behaviour, and the decisions tests must match.

How the app behaves (read before asserting)

  • Every API response takes 50–350 ms, at random. Anything a test does "right after" a request is a race.
  • Search is debounced by 300 ms. Favourites are optimistic: the button flips before the server has stored it.
  • Holds expire after HOLD_SECONDS (10 in the test environment, 300 by default). The countdown runs on the browser clock.
  • Payment is confirmed in the background, 0.3–2 s after POST /api/orders returns 202.
  • Each browser context is a separate customer (fr_uid cookie). Seats are shared by all customers.

MUST

  • Import from the fixture file. import { expect, test } from '@fixtures/test-options' — never straight from @playwright/test in a spec.
  • Locator priority. getByRole() → getByLabel() → getByPlaceholder() → getByText(). Stop at the first that works.
  • Web-first assertions. await expect(locator).toHaveText(...) retries; expect(await locator.textContent()) does not.
  • Book seats through the seats fixture. It gives each parallel worker its own row. Never hard-code seat ids.
  • Seed and check through the api fixture. It shares the page's cookie, so it acts as the same customer.
  • Two tags per test: one level tag (@e2e or @api) plus @smoke (or @flaky for the exercises), on test(), never on test.describe().
  • Title format TC-<ID>: what is exercised. Reuse the ID from the test plan rather than inventing one.
  • Type safety. Strict mode; no any, no non-null !. Braces on every block.
  • Verify before reporting done. Run the affected test with --repeat-each=20 plus npm run typecheck. One green run proves nothing for a timing bug — report the pass count you actually saw.

SHOULD

  • Read docs/app.md before asserting on behaviour, especially §6 (behavioural decisions).
  • Reach screens through the page-object fixtures (eventsPage, eventPage, orderPage), not raw page.getByRole(...) in a spec. Extend the page object when a screen needs a new control. Page objects hold locators and actions only — the spec owns the expects.
  • Keep tests independent and order-free; share no mutable state between tests.
  • End on an assertion, not a bare action — a trailing click can leave its request in flight.

WON'T

  • No hard waits. No page.waitForTimeout(), no setTimeout promises. To control time, use page.clock.
  • No retries and no bigger timeouts as a fix. retries stays 0. A longer timeout is only acceptable when the wait it covers is documented (payment: up to 2 s).
  • No test.describe.serial or workers: 1 to make a test pass. Shared data is the bug; isolate the data.
  • No getByTestId, XPath or CSS selectors. The app has no data-testid by design. If a test seems to need one, report it as an accessibility bug.
  • No exploratory files committed — nothing whose sole purpose is dumping HTML or probing page structure.
  • No new tooling beyond @playwright/test, typescript and @types/node.

Environment

BASE_URL points the suite at the app; unset, it defaults to http://localhost:3000 and Playwright starts the container via webServer. POST /api/test/reset (current customer) and POST /api/test/reset-all exist only with ENABLE_TEST_ROUTES=true, which compose and npm run app:dev both set.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.