preflight-test-engineer
Skill2 starsChanged 16 days agoApache-2.0View on GitHub
Pre-flight codebase analysis, automated /tests/ suite scaffolding, and hermetic multi-stage verification for Python and TypeScript/React projects. Use when asked to test a codebase before running, scaffold a test suite into /tests/, verify test coverage, or execute pre-flight sanity checks.
In this repository19 other files it gives its agents.
- .agents/AGENTS.mdAGENTS.md
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- cross-platform-dev-scriptsSkill
- lessons-learntSkill
- multi-agent-docsSkill
- public-repo-release-reviewSkill
- readme-designerSkill
- saas-app-builderSkill
- third-party-skill-verifierSkill
- skill-creatorSkill
- cross-platform-dev-scriptsSkill
- lessons-learntSkill
- multi-agent-docsSkill
- preflight-test-engineerSkill
- public-repo-release-reviewSkill
- readme-designerSkill
- saas-app-builderSkill
- third-party-skill-verifierSkill
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 1 is distinctive, and 7 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 55
Run the deterministic analyzer to inspect the project stack, existing test runners, and unmapped modules:
- Very specific: line 68
Check for pytest.ini, pyproject.toml, vitest.config.ts, jest.config.js.
- Very specific: line 121
Add @pytest.mark.asyncio for async endpoints.
- Very specific: line 122
Use mocker.patch() to intercept network calls.
- Very specific: line 123
Add @given(st.text()) with Hypothesis for property-based fuzzing.
- Very specific: line 125
Use userEvent.setup() and screen.getByRole() for RTL tests.
- Very specific: line 126
Add toHaveNoViolations() with jest-axe for accessibility contracts.
- Very specific: line 129
Verify empty inputs ("", [], {}, None/null), extreme numeric values, malformed JSON, and network timeouts.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
🚀 Pre-Flight Test Engineer
The Pre-Flight Test Engineer skill deterministically inspects a target codebase, identifies missing test coverage across functions, classes, API routes, and React components, scaffolds an industrial-grade /tests/ directory layout, and executes a 4-stage pre-flight verification sequence before application execution.
🏗️ Progressive Architecture & Directory Layout
skills/preflight-test-engineer/
├── SKILL.md # Core 5-phase testing workflow
├── attestation.json # Multi-agent verification attestation
├── evals/test-cases.json # Trigger evaluation prompts
├── references/
│ ├── python-testing.md # Pytest, asyncio, fixtures, hypothesis patterns
│ ├── typescript-react-testing.md # Vitest, RTL, MSW, accessibility patterns
│ └── test-pyramid-and-mocking.md # Hermetic fixtures & boundary case checklist
└── scripts/
├── analyze_codebase.py # AST & stack inspector (Python + TS/JS/React)
├── scaffold_tests.py # Deterministic /tests/ scaffolding engine
└── run_preflight.py # 4-stage pre-flight validation runner⚡ 5-Phase Pre-Flight Execution Workflow
[Phase 1: Scan] ──> Run analyze_codebase.py to detect stack & test gaps
│
[Phase 2: Strategy] ──> Map functions/components to Test Pyramid layers
│
[Phase 3: Scaffold] ──> Run scaffold_tests.py to generate /tests/ harness
│
[Phase 4: Fortify] ──> Add boundary, negative, and property-based assertions
│
[Phase 5: Pre-Flight] ──> Run run_preflight.py (Syntax -> Discovery -> Fast Run)Phase 1: Codebase & Stack Inspection
Run the deterministic analyzer to inspect the project stack, existing test runners, and unmapped modules:
python <skill-dir>/scripts/analyze_codebase.py --target-dir .For JSON output for automated pipelines, add --json.
Inspection Checklist:
- Language & Framework Detection:
- Python: FastAPI, Flask, Django, Click/Typer, Pydantic, Celery.
- JS/TS: React, Next.js, Express, Fastify, Node.js.
- Runner & Config Audit:
- Check for
pytest.ini,pyproject.toml,vitest.config.ts,jest.config.js.
- Check for
- Module & Symbol Extraction:
- Functions, classes, async endpoints, React components, hooks.
Phase 2: Test Architecture & Gap Mapping
Structure the test suite using the Hermetic Test Pyramid:
| Layer | Directory | Purpose | Isolation Strategy |
| Smoke | tests/smoke/ | Import graph, configuration sanity, zero-side-effect startup | Isolated runtime |
| Unit | tests/unit/ | Pure business logic, algorithms, state reducers | 100% Mocked / In-memory |
| Components | tests/components/ | React DOM rendering, user events, accessibility roles | jsdom + @testing-library/react |
| Integration | tests/integration/ | API route contracts, database transactions | In-memory SQLite / MSW handlers |
| Fixtures | tests/fixtures/ | Typed mock data factories and payload generators | Hermetic factories |
Phase 3: Scaffolding Standardized /tests/ Suite
Execute the scaffolder script to create the full test harness:
# Preview proposed test files
python <skill-dir>/scripts/scaffold_tests.py --target-dir . --dry-run
# Generate the /tests/ structure and baseline test files
python <skill-dir>/scripts/scaffold_tests.py --target-dir .Generated Assets:
- Python:
tests/conftest.pywithautouseenvironment isolation fixture.pytest.iniwith standard markers (unit,integration,smoke,slow).tests/smoke/test_preflight_smoke.pyfor pre-flight import checks.tests/fixtures/factories.pyfor typed test payload generation.- Skeleton unit test files matching discovered modules (e.g.
tests/unit/test_<module>.py).
- TypeScript / React:
tests/setup.tswith global mock resets and DOM cleanup.vitest.config.tsconfigured forjsdomornode.tests/smoke/preflight_smoke.test.tsfor export sanity.tests/components/<Component>.test.tsxfor React components with RTL.tests/fixtures/factories.tsfor mock objects.
Phase 4: Fortifying Assertions & Boundary Edge Cases
Enhance generated test templates with deep assertions according to the language reference:
- Python Projects: Consult python-testing.md
- Add
@pytest.mark.asynciofor async endpoints. - Use
mocker.patch()to intercept network calls. - Add
@given(st.text())with Hypothesis for property-based fuzzing.
- Add
- TypeScript/React Projects: Consult typescript-react-testing.md
- Use
userEvent.setup()andscreen.getByRole()for RTL tests. - Add
toHaveNoViolations()withjest-axefor accessibility contracts. - Intercept API calls using Mock Service Worker (MSW).
- Use
- Boundary Checklist: Consult test-pyramid-and-mocking.md
- Verify empty inputs (
"",[],{},None/null), extreme numeric values, malformed JSON, and network timeouts.
- Verify empty inputs (
Phase 5: Pre-Flight Verification Pipeline
Execute the 4-stage pre-flight runner before launching the main application or deploying:
python <skill-dir>/scripts/run_preflight.py --target-dir .Pipeline Stages:
- Stage 1 (Syntax & Compilation): Validates that all files compile and parse without syntax errors (
py_compile/ AST). - Stage 2 (Test Discovery): Verifies test discovery and import graph (
pytest --collect-only/vitest --run). - Stage 3 (Hermetic Fast Run): Executes unit and smoke tests in sandbox isolation with strict timeouts.
- Stage 4 (Diagnostics): Generates a clear status scorecard and remediation action list.
========================================================================
🏁 [PRE-FLIGHT VERIFICATION SUMMARY]
========================================================================
Target Directory : /path/to/project
Total Duration : 1.42s
Overall Verdict : 🟢 READY FOR LAUNCH (PASS)
------------------------------------------------------------------------
Stage | Status | Duration
------------------------------------------------------------------------
Syntax & Compilation Integrity | ✅ PASS | 0.12s
Test Discovery & Import Graph | ✅ PASS | 0.45s
Hermetic Fast Test Execution | ✅ PASS | 0.85s
========================================================================Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

