testing-methodology
Skill4 starsChanged 30 days agoMITView on GitHub
[METHODOLOGY] Testing strategy (when to write which tests, test quality standards, test pyramid). Preloaded by dev-qa-engineer agent.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- carmentacollective/carmentatheir own version, first seen Jan 22, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- carmentacollective/carmenta★ 2same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository48 other files it gives its agents.
- skills/dev-methodology/references/agents.mdAGENTS.md
- skills/skill-authoring/references/agents.mdAGENTS.md
- acceptance-protocolSkill
- agency-intakeSkill
- ai-visibility-methodologySkill
- banner-design-guideSkill
- benchmark-researchSkill
- brand-methodologySkill
- code-reviewingSkill
- code-writingSkill
- codex-bridgeSkill
- deploy-pipelineSkill
- design-assets-guideSkill
- design-system-methodologySkill
- design-task-decompositionSkill
- dev-methodologySkill
- docs-pipelineSkill
- documentation-writingSkill
- engagement-contractSkill
- engagement-protocolSkill
- feature-researchSkill
- human-voiceSkill
- infrastructure-setupSkill
- marketing-task-decompositionSkill
- post-deploy-qaSkill
- pre-deploy-qaSkill
- presentation-designSkill
- project-planningSkill
- prompt-engineeringSkill
- security-auditingSkill
- semantic-drift-methodologySkill
- seo-auditingSkill
- skill-authoringSkill
- skill-test-designSkill
- skill-testingSkill
- system-optimization-protocolSkill
- task-decompositionSkill
- tech-spec-planningSkill
- ui-styling-guideSkill
- ui-ux-methodologySkill
- user-spec-planningSkill
- validation-pipelineSkill
- yandex-analytics-methodologySkill
- yandex-direct-guideSkill
- yandex-metrika-guideSkill
- yandex-search-guideSkill
- yandex-webmaster-guideSkill
- yandex-wordstat-guideSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
18 rules. About 0 are common, 17 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈17
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 29
Purpose: Verify basic project setup works.
- Distinctive: line 39
Setting up smoke tests?: Read smoke-tests.md — CI integration, example templates.
- Distinctive: line 44
Purpose: Test business logic in isolation.
- Distinctive: line 56
Writing unit tests?: Read unit-tests.md — patterns, mocking, examples.
- Distinctive: line 61
Purpose: Test API endpoints, database, and external services.
- Very specific: line 64
All API endpoints (POST/PUT/DELETE especially)
- Distinctive: line 70
Rule: Every API endpoint and every DB write operation must have a corresponding integration test.
- Distinctive: line 72
Writing integration tests?: Read integration-tests.md — API testing, DB setup, fixtures.
- Distinctive: line 77
Purpose: Test critical user journeys end-to-end.
- Distinctive: line 84
Written: After deploy to dev, before manual testing (if proposed/requested)
- Distinctive: line 86
Writing E2E tests?: Read e2e-tests.md — Playwright/Cypress setup, page objects, CI.
- Distinctive: line 105
No code written (research/docs)
- Distinctive: line 146
Clean state: - Always start with known database state
- Distinctive: line 200
If mocking 3+ dependencies -> use integration or E2E test instead.
- Distinctive: line 216
Rule: If mocking more than testing -> wrong test type.
- Distinctive: line 225
How: Use framework mocking (jest.mock, unittest.mock)
- Distinctive: line 230
Why: Test real interactions, avoid external costs/delays
- Distinctive: line 234
Why: Test complete real-world scenario
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Test Master
Test Pyramid:
/\
/E2E\ <- Few (3-5 critical flows)
/------\
/Integr.\ <- Some (all endpoints + DB)
/----------\
/ Unit \ <- Many (all business logic)
/--------------\
/ Smoke \ <- Minimal (1-2 basic tests)
/------------------\When to Use Each Test Type
Smoke Tests
Purpose: Verify basic project setup works.
Use for:
- Testing framework is configured
- Environment variables accessible
- Basic imports work
- Infrastructure is functional
Written: During infrastructure setup (once per project)
Setting up smoke tests? Read smoke-tests.md — CI integration, example templates.
Unit Tests
Purpose: Test business logic in isolation.
Use for:
- Functions with calculations, validations, transformations
- Decision-making logic (if/else, switch)
- Data processing and formatting
- Error handling logic
Written: By code-developer during each task (immediately after code)
Skip for: Simple getters/setters, one-line changes, trivial updates
Writing unit tests? Read unit-tests.md — patterns, mocking, examples.
Integration Tests
Purpose: Test API endpoints, database, and external services.
Use for:
- All API endpoints (POST/PUT/DELETE especially)
- Database operations (create/update/delete)
- External service integrations (payments, email, webhooks)
Written: As separate task at end of feature (if defined in Tech Spec)
Rule: Every API endpoint and every DB write operation must have a corresponding integration test. Missing integration tests for these is a quality gap.
Writing integration tests? Read integration-tests.md — API testing, DB setup, fixtures.
E2E Tests
Purpose: Test critical user journeys end-to-end.
Use for:
- Top 3-5 most critical user flows
- Large features (>5 tasks)
- Critical business processes (auth, payment, core features)
Written: After deploy to dev, before manual testing (if proposed/requested)
Writing E2E tests? Read e2e-tests.md — Playwright/Cypress setup, page objects, CI.
Decision Framework
Should I write unit tests for this?
YES if:
- Function has business logic
- Function makes decisions
- Function transforms data
- Function handles errors
- Task specifies testing
NO if:
- Simple getter/setter
- One-line text change
- Trivial config update
- No code written (research/docs)
Should I write integration tests?
YES if:
- Tech Spec specifies integration tests
- Feature has API endpoints
- Feature interacts with database
- Feature calls external services
NO if:
- Tech Spec says "None"
- Feature is purely client-side
- Already covered by E2E tests
Should I write E2E tests?
YES if:
- Feature has >5 tasks
- Feature touches critical flows
- Feature has breaking changes
- User explicitly requests
- Tech Spec specifies E2E tests
NO if:
- Small feature (<3 tasks)
- Non-critical functionality
- Well covered by unit + integration tests
- Time/cost constraints
Key Testing Principles
- Write tests immediately - In the same session as the code, before moving on
- Test behavior, not implementation - Focus on what, not how
- Keep tests fast - Unit: milliseconds, Integration: seconds, E2E: minutes
- Isolate tests - Mock external dependencies in unit tests
- One concern per test - Each test validates one thing
- Clear test names - Describe what's tested and expected outcome
- Independent tests - Each test runs with its own setup, no shared state
- Clean state - Always start with known database state
- Tests must verify real behavior - Assert on actual results, not mock calls
- Every test earns its place - Each test catches a specific failure no other test catches (see below)
Redundant Testing Anti-pattern
Tests that duplicate coverage waste time and create maintenance burden.
Signs of redundant testing:
- Same behavior verified by both unit test and integration test with no added value
- E2E test that only checks what unit tests already cover
- Multiple test files testing the same function with same scenarios
- "Tests for completeness" that exist without protecting against real regressions
Rule: Each test must justify its existence — it catches a specific failure that no other test catches. If removing the test reduces zero confidence, it belongs nowhere.
Test Quality Requirements
What Makes a BAD Test
Tests nothing:
expect(true).toBe(true)Tests only that mock was called:
expect(api.call).toHaveBeenCalled() // Without checking resultNo assertions:
render(<Component />) // Just renders, checks nothingWhat Makes a GOOD Test
Tests actual result:
expect(calculateTotal(100, 0.2)).toBe(80)Tests real state change:
cart.add({ id: 1 })
expect(cart.items).toHaveLength(1)Rule: Excessive Mocking = Wrong Test Type
If mocking 3+ dependencies -> use integration or E2E test instead.
When to Prioritize E2E Over Unit Tests
For some project types, E2E tests are MORE valuable than unit tests:
| Project Type | Primary Tests | Why |
| API/Backend | Unit + Integration | Logic in functions |
| CLI Tools | Unit + Integration | Testable in isolation |
| UI Apps | E2E + Integration | Logic in UI interaction |
| Browser Extensions | E2E (real browser) | APIs can't be mocked reliably |
| Mobile Apps | E2E | Platform APIs need real env |
Rule: If mocking more than testing -> wrong test type.
Mocking Strategy
Unit Tests
- Mock: Database, API calls, file system, time
- Why: Fast, isolated, deterministic
- How: Use framework mocking (jest.mock, unittest.mock)
Integration Tests
- Real: Database (test DB), file system
- Mock: External services (payments, email)
- Why: Test real interactions, avoid external costs/delays
E2E Tests
- Real: Everything (use test/sandbox mode for external services)
- Why: Test complete real-world scenario
Test Quality Review
When reviewing existing tests, read test-quality-review.md — categories of bad tests, severity levels, decision criteria.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

