test-data-creator
Skill1 starsChanged todayMITView on GitHub
Generates realistic, schema-conformant synthetic test data and high-volume performance test datasets across SQL (SQL Server, PostgreSQL, MySQL, Oracle, SQLite via sql-connector), JSON, and CSV formats. Supports streaming chunk generation for millions of records with constant memory, realistic cardinality and Zipfian/Pareto distributions, relational foreign-key integrity, boundary edge-case injection, and ephemeral in-memory SQLite fixtures for unit, integration, and load testing.
- Deletes or force-pushes
In this repository38 other files it gives its agents.
- AGENTS.mdAGENTS.md
- agent-orchestratorSkill
- algorithmic-complexity-analyzerSkill
- api-securitySkill
- architectural-planningSkill
- bdd-gherkin-testerSkill
- caching-queues-reviewerSkill
- code-reviewerSkill
- csharp-developerSkill
- design-pattern-detectorSkill
- detect-overengineeringSkill
- github-security-scanner-hookSkill
- java-developerSkill
- legacy-code-modernizationSkill
- llm-as-judgeSkill
- load-memorySkill
- memory-analyzerSkill
- nodejs-executorSkill
- playwright-testerSkill
- powershell-writerSkill
- python-api-writerSkill
- python-developerSkill
- python-linterSkill
- rag-pipelineSkill
- react-developerSkill
- remote-pod-log-analyzerSkill
- researcher-poc-testerSkill
- save-memorySkill
- scan-halucinated-testsSkill
- scan-repo-readmeSkill
- skill-creatorSkill
- skill-doctorSkill
- sql-connectorSkill
- sql-executorSkill
- sync-github-forksSkill
- trace-code-contextSkill
- troubleshoot-and-fixSkill
- typescript-developerSkill
The file
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 2 are distinctive, and 4 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
4
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 63
For Relational Databases: Use the sql-connector MCP server:
- Distinctive: line 118
When executing generated SQL statements into target databases:
- Very specific: line 120
Verify Query Safety: Run audit_query_safety via sql-connector to guarantee zero destructive mutations (DROP TABLE, DELETE *).
- Very specific: line 121
Transaction Scoping: Always execute inserts inside explicit BEGIN TRANSACTION.
- Very specific: line 140
Run verification query: SELECT status, COUNT(*) FROM customer_orders GROUP BY status.
- Distinctive: line 141
Verify Pareto distribution: Top 20% of customer IDs should account for ~80% of generated orders.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Test Data Creator
The Test Data Creator synthesizes realistic, high-fidelity mock datasets, API payloads, and high-volume performance test fixtures across relational databases (via sql-connector), JSON document trees, and CSV flat files.
Designed to prevent the pitfalls of trivial random test data, this skill enforces relational foreign key integrity, realistic distribution skew (Pareto 80/20 hot partitions, Zipfian distributions), boundary edge-case injection, and constant $O(1)$ memory streaming for high-volume load testing ($100\text{K} \dots 10\text{M}+$ rows).
When to Use This Skill
- Generating synthetic test data for relational databases (SQL Server, Oracle, PostgreSQL, MySQL, SQLite).
- Seeding ephemeral in-memory SQLite (
:memory:) databases for local unit and integration tests. - Synthesizing mock JSON payloads for REST APIs, message queues (Kafka, RabbitMQ), or NoSQL collections (MongoDB).
- Generating high-volume datasets for load, stress, and performance testing ($100\text{K} \dots 1\text{M}+$ rows).
- Modeling realistic data skew (Pareto 80/20, hot tenant partitions, index selectivity, null ratios).
- Injecting adversarial boundary test cases (special characters, SQL quote escaping, extreme timestamps, overflow numbers).
Core Capabilities
┌─────────────────────────────────┐
│ Schema & Table Inspection │ ──► Inspects target tables via `sql-connector` (`describe_table`)
└────────────────┬────────────────┘
│
▼
┌─────────────────────────────────┐
│ Data Modeling & Distribution │ ──► Configures cardinality, null ratios, and Pareto 80/20 skew
└────────────────┬────────────────┘
│
▼
┌─────────────────────────────────┐
│ Streaming Data Synthesis │ ──► Generates SQL batches, JSON/NDJSON, or CSV with O(1) memory
└────────────────┬────────────────┘
│
▼
┌─────────────────────────────────┐
│ Database Seeding & Verify │ ──► Executes chunked inserts (500–2k rows) within transaction blocks
└─────────────────────────────────┘- Relational Schema Fidelity: Inspects existing tables via
sql-connector(describe_table,list_tables) to match data types, column lengths, nullability constraints, and foreign key dependencies. - High-Throughput Chunked Batching: Emits multi-row
INSERT INTO ... VALUES (...), (...)in chunks of 500 to 2,000 rows inside transaction blocks, accelerating seeding by $20\text{x}\text{--}50\text{x}$ over single-row inserts. - Realistic Distribution Skew: Supports Pareto 80/20 power-law and normal/Gaussian distributions to simulate real-world hot tenant partitions and database lock contention.
- Zero-Dependency Ephemeral Fixtures: Seeds in-memory SQLite fixtures (
:memory:) with zero external infrastructure required. - Boundary & Injection Defense: Systematically injects apostrophes (
O'Connor), unicode/emojis (🔥), leap year timestamps (2024-02-29), and max integers to verify sanitization and edge-case handling.
5-Phase Workflow
Phase 1: Schema & Domain Inspection
Inspect the target schema to identify required fields, types, and constraints:
- For Relational Databases: Use the
sql-connectorMCP server:list_tables→ Identify target tables and topological parent-child hierarchy.describe_table→ Retrieve column names, data types, nullability, and primary/foreign keys.
- For JSON / Document APIs: Inspect TypeScript interfaces, Pydantic models, or sample JSON payloads.
Phase 2: Volume, Distribution & Profile Configuration
Select the appropriate volume tier and distribution profile:
| Volume Profile | Row Count | Target Evaluation | Distribution Setting |
| Smoke / Unit | $10 \dots 500$ | Local unit tests, mock API stubs | Uniform, strict boundary values |
| Integration | $1,000 \dots 25,000$ | API contract tests, join validation | Realistic foreign key relationships |
| Load / Performance | $100,000 \dots 1,000,000$ | Query optimization, index selectivity | Pareto 80/20 (--skew pareto), null ratios |
| Soak / Volume | $5,000,000+$ | Table partitioning, disk I/O, vacuuming | Streaming NDJSON / CSV direct bulk load |
Phase 3: Generator Synthesis & Chunk Streaming
Execute the streaming data generator script using the repository virtual environment:
# Generate 1,000 user accounts in JSON format:
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py \
--preset users --count 1000 --format json --output scratch/fixtures/users.json
# Generate 50,000 orders in SQL with 80/20 Pareto customer skew (batches of 1,000, target dialect):
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py \
--preset orders --count 50000 --format sql --dialect postgres --table-name customer_orders \
--batch-size 1000 --skew pareto --output scratch/fixtures/orders.sql
# Generate SQL Server T-SQL batch inserts:
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py \
--preset orders --count 10000 --format sql --dialect sqlserver --table-name customer_orders \
--output scratch/fixtures/orders_mssql.sql
# Generate IoT time-series sensor metrics:
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py \
--preset metrics --count 25000 --format csv --output scratch/fixtures/telemetry.csv
# Generate custom schema with specific fields:
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py \
--fields "id:id,name:name,email:email,status:choice[ACTIVE|PENDING|SUSPENDED],balance:float,created_at:datetime" \
--count 10000 --format sql --table-name accounts --output scratch/fixtures/accounts.sql(On POSIX, substitute .\.venv\Scripts\python.exe with .venv/bin/python).
Phase 4: Database Seeding & Safety Guardrails
When executing generated SQL statements into target databases:
- Verify Query Safety: Run
audit_query_safetyviasql-connectorto guarantee zero destructive mutations (DROP TABLE,DELETE *). - Transaction Scoping: Always execute inserts inside explicit
BEGIN TRANSACTION;andCOMMIT;blocks to prevent disk sync thrashing. - Topological Order: Seed parent tables first (Users $\to$ Products), followed by dependent child tables (Orders $\to$ OrderItems).
- Ephemeral SQLite In-Memory Seeding:
import { DatabaseSync } from 'node:sqlite'; const db = new DatabaseSync(':memory:'); db.exec(` CREATE TABLE users (id INTEGER PRIMARY KEY, username TEXT, email TEXT); BEGIN TRANSACTION; INSERT INTO users (id, username, email) VALUES (1, 'john.doe', 'john@example.com'), (2, 'jane.smith', 'jane@example.com'); COMMIT; `);
Phase 5: Verification & Memory Persistence
- Verify Ingested Counts & Distribution:
- Run verification query:
SELECT status, COUNT(*) FROM customer_orders GROUP BY status; - Verify Pareto distribution: Top 20% of customer IDs should account for ~80% of generated orders.
- Run verification query:
- Record Schema Seed Standards to Memory:
- Save permanent test data conventions and seed parameters into
ledgerusingsave-memory. - Save generation throughput benchmarks and lessons learned into
journal.
- Save permanent test data conventions and seed parameters into
Practical Examples
Example 1: Creating Relational Seed Data for Performance Testing
Goal: Generate 100,000 orders referencing 5,000 users with realistic Pareto distribution (testing database index efficiency).
# 1. Generate 5,000 parent users:
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py `
--preset users --count 5000 --format sql --table-name users `
--batch-size 1000 --output scratch/seed_users.sql
# 2. Generate 100,000 child orders referencing the 5,000 users with Pareto skew:
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py `
--preset orders --count 100000 --format sql --table-name orders `
--batch-size 1000 --skew pareto --null-ratio 0.05 --output scratch/seed_orders.sqlExample 2: Generating JSON Payload Mock for REST API / Kafka Stream
Goal: Stream 20,000 sensor telemetry records in NDJSON format for Kafka load testing.
.\.venv\Scripts\python.exe .github/skills/test-data-creator/scripts/generate-test-data.py `
--preset metrics --count 20000 --format ndjson `
--output scratch/telemetry.ndjsonReferences
- Test Data Creator Reference Guide — Comprehensive technical reference, distribution math, bulk loaders, and boundary edge cases.
- Streaming Data Generator Script — Universal streaming generator in Python.
- Sample Vocabularies Fixture — External JSON vocabulary fixture for names, domains, currencies, and devices.
- SQL Connector MCP Server — Database query execution, schema inspection, and safety firewall.
- SQL Executor — Multi-engine SQL query tuning and validation.
- Researcher PoC Tester — Empirical benchmarking and statistical test runner.
- Save Memory — Persist test data configurations and seed schemas into
ledger.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

