agentleFS
CaduS0uzaCaduS0uza/qa-supreme/skills/api-contract-testing/SKILL.md

api-contract-testing

Skill1 starsChanged 2 months agoMITView on GitHub

Test HTTP/GraphQL APIs and the contracts between services — status codes, schema shape, authz, idempotency, pagination, error bodies — with Supertest, httpx, RestAssured, or Pact. Use when asked to test an endpoint, a backend route, a webhook, or an integration between services.

6 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

6 rules. 0 are common, 5 are distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 5 edited versions we read keep these.

  • 5

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Write one test per row. An endpoint without rows 3, 4 and 5 is untested, whatever the coverage report says.

    line 13
  2. Distinctive:

    Add pagination (first page, last page, beyond-last, bad cursor), rate limiting, and content negotiation when they exist.

    line 26
  3. Distinctive:

    Use when two services deploy independently:

    line 49
  4. Very specific:

    Never assert only the status code.

    line 65
  5. Distinctive:

    Never share auth state across tests without proving the scoping test still isolates users.

    line 66
  6. Distinctive:

    Every test creates its own data and cleans up, or runs inside a rolled-back transaction.

    line 68

“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

API & Contract Testing

The integration level is where most real bugs live and where tests are cheapest per bug caught. Spend here before spending on E2E.

The seven questions every endpoint must answer

Write one test per row. An endpoint without rows 3, 4 and 5 is untested, whatever the coverage report says.

#QuestionTypical assertion
1Happy path shape200 + schema match (not just a status code)
2Validation400 with a field-level error body, not a 500
3Authenticationno token / expired token / token for a deleted user -> 401
4Authorizationvalid token of user B against user A's resource -> 404 or 403, never 200
5Tenant scopinglist endpoints return only the caller's tenant rows
6Idempotency & retriessame request twice -> one side effect (charges, emails, rows)
7Not-found & conflictunknown id -> 404; duplicate create -> 409, not 500

Add pagination (first page, last page, beyond-last, bad cursor), rate limiting, and content negotiation when they exist.

Schema over field-picking

Assert the whole response shape, not two fields — a silently added password_hash in the payload is a leak your expect(res.body.id) will never catch.

// zod / ajv / pydantic — pick the project's validator
expect(() => UserResponse.parse(res.body)).not.toThrow()
expect(Object.keys(res.body).sort()).toEqual(['createdAt', 'email', 'id', 'name'])

Real data path

Hit a real database (containerized, migrated, seeded per test) — mocking your own repository turns an integration test into a slow unit test. Mock only third parties, and mock them with a recorded contract (MSW, WireMock, VCR), never an ad-hoc stub that returns a shape the vendor never sends.

Consumer-driven contracts (Pact)

Use when two services deploy independently:

  1. Consumer test declares the interaction and generates a pact file.
  2. Provider verifies the pact in its own CI against a real provider state.
  3. Broker gates deploy: provider cannot ship while a consumer pact is unverified.

Without step 3 the contract is decoration. Say so explicitly if the broker is missing.

Webhooks

  • Signature verification: valid, tampered body, replayed timestamp, wrong secret.
  • Idempotency: same event id twice -> one effect. This is a P0 in every payment system.
  • Out-of-order delivery: updated before created.

Hard rules

  • Never assert only the status code. 200 with a wrong body is the most common silent bug.
  • Never share auth state across tests without proving the scoping test still isolates users.
  • Time-sensitive endpoints: control the clock server-side (test seam), do not sleep.
  • Every test creates its own data and cleans up, or runs inside a rolled-back transaction.

Done means

Runner output pasted, the seven questions answered per endpoint (or explicitly marked N/A), and every authz test proven to fail when the guard is removed (mutation-proof).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.