agentleFS
AgentShekelAgentShekel/agentic-workflow/skills/testing-methodology/SKILL.md

testing-methodology

Skill4 starsChanged 30 days agoMITView on GitHub

[METHODOLOGY] Testing strategy (when to write which tests, test quality standards, test pyramid). Preloaded by dev-qa-engineer agent.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

18 rules. About 0 are common, 17 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈17

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Purpose: Verify basic project setup works.

    line 29
  2. Distinctive:

    Setting up smoke tests?: Read smoke-tests.md — CI integration, example templates.

    line 39
  3. Distinctive:

    Purpose: Test business logic in isolation.

    line 44
  4. Distinctive:

    Writing unit tests?: Read unit-tests.md — patterns, mocking, examples.

    line 56
  5. Distinctive:

    Purpose: Test API endpoints, database, and external services.

    line 61
  6. Very specific:

    All API endpoints (POST/PUT/DELETE especially)

    line 64
  7. Distinctive:

    Rule: Every API endpoint and every DB write operation must have a corresponding integration test.

    line 70
  8. Distinctive:

    Writing integration tests?: Read integration-tests.md — API testing, DB setup, fixtures.

    line 72
  9. Distinctive:

    Purpose: Test critical user journeys end-to-end.

    line 77
  10. Distinctive:

    Written: After deploy to dev, before manual testing (if proposed/requested)

    line 84
  11. Distinctive:

    Writing E2E tests?: Read e2e-tests.md — Playwright/Cypress setup, page objects, CI.

    line 86
  12. Distinctive:

    No code written (research/docs)

    line 105
  13. Distinctive:

    Clean state: - Always start with known database state

    line 146
  14. Distinctive:

    If mocking 3+ dependencies -> use integration or E2E test instead.

    line 200
  15. Distinctive:

    Rule: If mocking more than testing -> wrong test type.

    line 216
  16. Distinctive:

    How: Use framework mocking (jest.mock, unittest.mock)

    line 225
  17. Distinctive:

    Why: Test real interactions, avoid external costs/delays

    line 230
  18. Distinctive:

    Why: Test complete real-world scenario

    line 234

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Test Master

Test Pyramid:

        /\
       /E2E\        <- Few (3-5 critical flows)
      /------\
     /Integr.\      <- Some (all endpoints + DB)
    /----------\
   /   Unit     \   <- Many (all business logic)
  /--------------\
 /    Smoke      \  <- Minimal (1-2 basic tests)
/------------------\

When to Use Each Test Type

Smoke Tests

Purpose: Verify basic project setup works.

Use for:

  • Testing framework is configured
  • Environment variables accessible
  • Basic imports work
  • Infrastructure is functional

Written: During infrastructure setup (once per project)

Setting up smoke tests? Read smoke-tests.md — CI integration, example templates.


Unit Tests

Purpose: Test business logic in isolation.

Use for:

  • Functions with calculations, validations, transformations
  • Decision-making logic (if/else, switch)
  • Data processing and formatting
  • Error handling logic

Written: By code-developer during each task (immediately after code)

Skip for: Simple getters/setters, one-line changes, trivial updates

Writing unit tests? Read unit-tests.md — patterns, mocking, examples.


Integration Tests

Purpose: Test API endpoints, database, and external services.

Use for:

  • All API endpoints (POST/PUT/DELETE especially)
  • Database operations (create/update/delete)
  • External service integrations (payments, email, webhooks)

Written: As separate task at end of feature (if defined in Tech Spec)

Rule: Every API endpoint and every DB write operation must have a corresponding integration test. Missing integration tests for these is a quality gap.

Writing integration tests? Read integration-tests.md — API testing, DB setup, fixtures.


E2E Tests

Purpose: Test critical user journeys end-to-end.

Use for:

  • Top 3-5 most critical user flows
  • Large features (>5 tasks)
  • Critical business processes (auth, payment, core features)

Written: After deploy to dev, before manual testing (if proposed/requested)

Writing E2E tests? Read e2e-tests.md — Playwright/Cypress setup, page objects, CI.


Decision Framework

Should I write unit tests for this?

YES if:

  • Function has business logic
  • Function makes decisions
  • Function transforms data
  • Function handles errors
  • Task specifies testing

NO if:

  • Simple getter/setter
  • One-line text change
  • Trivial config update
  • No code written (research/docs)

Should I write integration tests?

YES if:

  • Tech Spec specifies integration tests
  • Feature has API endpoints
  • Feature interacts with database
  • Feature calls external services

NO if:

  • Tech Spec says "None"
  • Feature is purely client-side
  • Already covered by E2E tests

Should I write E2E tests?

YES if:

  • Feature has >5 tasks
  • Feature touches critical flows
  • Feature has breaking changes
  • User explicitly requests
  • Tech Spec specifies E2E tests

NO if:

  • Small feature (<3 tasks)
  • Non-critical functionality
  • Well covered by unit + integration tests
  • Time/cost constraints

Key Testing Principles

  1. Write tests immediately - In the same session as the code, before moving on
  2. Test behavior, not implementation - Focus on what, not how
  3. Keep tests fast - Unit: milliseconds, Integration: seconds, E2E: minutes
  4. Isolate tests - Mock external dependencies in unit tests
  5. One concern per test - Each test validates one thing
  6. Clear test names - Describe what's tested and expected outcome
  7. Independent tests - Each test runs with its own setup, no shared state
  8. Clean state - Always start with known database state
  9. Tests must verify real behavior - Assert on actual results, not mock calls
  10. Every test earns its place - Each test catches a specific failure no other test catches (see below)

Redundant Testing Anti-pattern

Tests that duplicate coverage waste time and create maintenance burden.

Signs of redundant testing:

  • Same behavior verified by both unit test and integration test with no added value
  • E2E test that only checks what unit tests already cover
  • Multiple test files testing the same function with same scenarios
  • "Tests for completeness" that exist without protecting against real regressions

Rule: Each test must justify its existence — it catches a specific failure that no other test catches. If removing the test reduces zero confidence, it belongs nowhere.


Test Quality Requirements

What Makes a BAD Test

Tests nothing:

expect(true).toBe(true)

Tests only that mock was called:

expect(api.call).toHaveBeenCalled()  // Without checking result

No assertions:

render(<Component />)  // Just renders, checks nothing

What Makes a GOOD Test

Tests actual result:

expect(calculateTotal(100, 0.2)).toBe(80)

Tests real state change:

cart.add({ id: 1 })
expect(cart.items).toHaveLength(1)

Rule: Excessive Mocking = Wrong Test Type

If mocking 3+ dependencies -> use integration or E2E test instead.


When to Prioritize E2E Over Unit Tests

For some project types, E2E tests are MORE valuable than unit tests:

Project TypePrimary TestsWhy
API/BackendUnit + IntegrationLogic in functions
CLI ToolsUnit + IntegrationTestable in isolation
UI AppsE2E + IntegrationLogic in UI interaction
Browser ExtensionsE2E (real browser)APIs can't be mocked reliably
Mobile AppsE2EPlatform APIs need real env

Rule: If mocking more than testing -> wrong test type.


Mocking Strategy

Unit Tests

  • Mock: Database, API calls, file system, time
  • Why: Fast, isolated, deterministic
  • How: Use framework mocking (jest.mock, unittest.mock)

Integration Tests

  • Real: Database (test DB), file system
  • Mock: External services (payments, email)
  • Why: Test real interactions, avoid external costs/delays

E2E Tests

  • Real: Everything (use test/sandbox mode for external services)
  • Why: Test complete real-world scenario

Test Quality Review

When reviewing existing tests, read test-quality-review.md — categories of bad tests, severity levels, decision criteria.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.