dataset-engineering
Skill1 starsChanged 3 months agoMITView on GitHub
Use when designing, auditing, expanding, or materializing datasets for SFT, RL, RLM, or coding-agent evals. Especially useful for held-out split design, OOD generalization, hard negatives, balanced tool coverage, leakage prevention, project-builder data, and datasets that expose real production behavior.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- ScientiaCapital/unsloth-mcp-servertheir own version, first seen Feb 21, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- ScientiaCapital/unsloth-mcp-server★ 2same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository32 other files it gives its agents.
- advertising-imagery-agentSkill
- agent-watchSkill
- aria-intelligenceSkill
- bambu-labsSkill
- cadSkill
- cad-viewerSkill
- design-business-agentSkill
- design-loopSkill
- dxfSkill
- elite-engineeringSkill
- emil-deign-skillSkill
- env-harnessSkill
- gcodeSkill
- human-writingSkill
- implicit-cadSkill
- inference-engineeringSkill
- kernel-engineeringSkill
- lamp-method-first-principles-thinkingSkill
- marketing-and-advertising-agentSkill
- model-evalSkill
- new-project-architectureSkill
- opportunity-hunterSkill
- rlmSkill
- rlSkill
- sdfSkill
- sendcutsendSkill
- sftSkill
- srdfSkill
- step-partsSkill
- synthesys-bug-triageSkill
- taste-maxSkill
- urdfSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 5 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈5
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- could be automatic: a setting could enforce it
- Distinctive: line 8
Use this skill when data quality determines the model signal.
- Distinctive: line 24
Do not merge these into a blurry aggregate until each component is understood.
- Distinctive: line 37
no duplicated prompt tails that teach echo
- Distinctive: line 39
If eval differs from train only by metadata, treat it as leakage.
- Distinctive: line 86
Before GPU time, check:
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Dataset Engineering
Use this skill when data quality determines the model signal.
Principle
A dataset is an experiment. Each corpus should expose one clear behavior, failure mode, or generalization question.
Dataset Types
- contract gate: exact known output format
- generalization gate: new wording, same behavior
- OOD gate: new problem families, same production bar
- hard-contrast gate: target against plausible wrong choices
- preservation gate: solved behavior that must not regress
- live canary: executable workflow and side-effect proof
- project-builder gate: inspect/build/test/expose/verify trajectories
Do not merge these into a blurry aggregate until each component is understood.
Split Discipline
Require:
- disjoint prompts between train and eval
- fresh OOD wording
- stable category labels across related gates
- balanced canonical tool coverage
- explicit action/tool ratio
- deterministic materialization
- known expected outputs
- no duplicated prompt tails that teach echo
If eval differs from train only by metadata, treat it as leakage.
Hard Negatives
Good negatives are tempting:
- correct tool with wrong arguments
- wrong tool with plausible arguments
- adjacent action family
- unsafe action instead of approval pause
- short shortcut against longer target
- generic checklist instead of project-specific plan
- prompt echo or wrapper continuation
Random bad answers teach little. Plausible bad answers teach taste.
Tool Data
For each canonical tool, verify:
- count per split
- schema completeness
- exact argument object
- execution validity
- safety side-effect expectations
- route ambiguity cases
Tool selection and argument exactness must be scored separately.
Project Data
Strong project-builder targets include:
- repo/workspace inspection
- bounded slice selection
- interface or contract definition
- implementation step
- deterministic tests
- endpoint/CLI/probe exposure
- user-perspective smoke
- numeric evidence or measurable signal
- limitations and rollback notes
Reject generic plans, toy-only tasks, unsafe host access, and unverified claims.
Pre-Training Validation
Before GPU time, check:
- sample counts
- split disjointness
- tool counts
- action/tool balance
- JSON parseability
- schema validity
- target family coverage
- hard-negative coverage
- expected failure mode
- artifact path and materialization command
Documentation
Record why the corpus exists, counts, split rules, held-out families, hard-negative design, validation commands, artifact path, baseline failure, and intended next training action.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.