validate-dbt
Skill3 starsChanged 31 days agoMITView on GitHub
Plans and runs the smallest safe dbt validation loop for changed or named resources. Use when the user asks to validate, compile, test, build, check, or verify dbt work, or asks the agent to fix failures and recheck them, including the deprecations a dbt package upgrade brings.
In this repository24 other files it gives its agents.
- analytics-requirementsSkill
- api-pipelineSkill
- bi-modelSkill
- build-modelSkill
- data-model-reviewSkill
- data-profileSkill
- data-quality-investigationSkill
- dbt-healthSkill
- explain-data-codeSkill
- freshness-investigationSkill
- incremental-modelSkill
- metric-definitionSkill
- migration-planSkill
- model-documentationSkill
- model-testsSkill
- project-profileSkill
- pr-reviewSkill
- refactor-modelSkill
- rename-modelSkill
- semantic-layerSkill
- source-setupSkill
- sql-reviewSkill
- trace-lineageSkill
- yaml-maintenanceSkill
The file
We split this skill into the rules it gives an agent
9 rules. About 0 are common, 4 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 18
Only an explicitly confirmed local or development target is eligible.
- Very specific: line 22
Read .ae/exercise.json and .ae/project.yml when present.
- Distinctive: line 23
Run the static inspectors before proposing execution:
- Very specific: line 31
Run static dbt parse and dbt ls steps one at a time when useful.
- Very specific: line 32
Before compile, show, test, run, build, debug, docs generate, source freshness, or freshness, however dbt is launched (dbt, dbtf, python -m …
- Very specific: line 58
Never auto-run deps, seed, snapshot, run-operation, a global command, or any command with --full-refresh.
- Distinctive: line 74
Every execution used an explicit non-production target and a non-empty selector.
- Distinctive: line 77
Every repair was either explicitly included in the request or separately approved before application.
- Distinctive: line 78
No credentials, profile values, raw rows, or unbounded logs appear in output.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
<!-- Generated by scripts/build_targets.py from src/skills. Edit the source and rebuild; changes made here are overwritten. -->Self-contained install.
<skill-dir>is the directory holding this file: substitute its absolute path in the commands below and run them from the repository root. Read<skill-dir>/references/operating-rules.mdbefore acting; it holds the toolkit's always-on rules, which this install cannot load on its own. This install has no safety hooks, so the approvals those rules describe rest on you and the agent rather than on an enforced gate.
Validate dbt
Close the loop from static evidence to a bounded, approved dbt result. Default silently to Work mode; use Learn or Assessment only when requested or required by .ae/exercise.json.
Boundary
- This is a free local workflow. Never require or call MCP.
- Static inspection and planning never contact a warehouse.
- A warehouse-capable command runs only after the exact command, selector, target, and likely side effects are previewed and approved.
- Only an explicitly confirmed local or development target is eligible. Never infer a target, use a production-like target, run globally without a selector, or use
--full-refresh.
Procedure
- Read
.ae/exercise.jsonand.ae/project.ymlwhen present. - Run the static inspectors before proposing execution:
python3 "<skill-dir>/scripts/inspect_dbt.py" <project-root> --prettypython3 "<skill-dir>/scripts/inspect_git.py" <project-root> --prettypython3 "<skill-dir>/scripts/dbt_capabilities.py" <project-root> --prettyfor the engine (dbt v1 or v2) and the evidence for it, whether JSON artifacts are written, and whether checks, static analysis, or dbt State apply. Use these answers instead of guessing from the CLI name.
- Determine whether the user named a scope. If not, derive it only from changed dbt resources. Classify YAML changes: formatting/comments and descriptions are static; generic, unit, and source tests validate only their named resources; source freshness and loaded-at changes validate the affected source tables; contract fields, renames, and deletions include downstream consumers; semantic/metric changes resolve affected consumers before any model build. Parse
ref()andsource()calls in changed singular tests and select the exact test. Macro, package, or project-config changes require explicit affected model scope before warehouse work. Do not turn an unrelated dirty worktree into validation scope. - Run
python3 "<skill-dir>/scripts/plan_dbt_validation.py" <project-root> --intent <intent> [--model <name>] [--paths <changed-path>] [--state <prior-artifacts>] [--target <dev-target>] --pretty. - Read
validation_steps, not justintent. A mixed change produces several independent bounded operations, and each one is there because a different resource changed; running only the broadest of them skips the others. Reviewrecommended_intentand its reason rather than trusting either blindly. The planner omits an unnecessarily broad warehouse command for metadata-only or test-only changes unless--allow-broader-intentis deliberately supplied. Confirm every selected node exists, YAML changes map to the reported models, renamed resources include downstream consumers, and the scope is not global. - Check
excluded_changes,unsupported_yaml, andunresolvable_deletionsbefore executing. A file excluded as belonging to another project in the same repository is not validated here; a YAML construct the reader cannot interpret makes that file's contribution unknown rather than absent; a deleted resource with no prior state needs explicit downstream scope, which--statecan often supply instead. - Run static
dbt parseanddbt lssteps one at a time when useful. Stop if selector resolution differs from the plan. - Before
compile,show,test,run,build,debug,docs generate,source freshness, orfreshness, however dbt is launched (dbt,dbtf,python -m dbt, or throughuv,pipx,conda, and similar runners), show one exact command and state:- why this is the smallest sufficient command;
- the explicit target and evidence it is local/development;
- resources and materializations in scope;
- whether warehouse objects may be created or replaced.
- Obtain approval, then run exactly that command, including its
--log-format-file json --log-patharguments. Do not batch a second dbt command into the same shell call. Where the plan lists several warehouse steps, approve and run them one at a time, in order. Note the UTC time immediately before each command; the artifact it writes is only evidence if it is newer than that. - Read the resulting artifacts with each step's
verification_command, or equivalentlypython3 "<skill-dir>/scripts/interpret_dbt_results.py" --project-dir <project-root> --expect-operation <step operation> --invocation-log <step log_path> --since <timestamp taken before the command> --command "<the exact command that ran>" [--log <path>] --pretty. Naming the operation reads only that operation's artifact, so a leftoverrun_results.jsoncannot be reported as the outcome of a freshness check. dbt chooses its own invocation ID, so the step's JSON log is how that ID is recovered; the timestamp is only a fallback. - Treat
verified: falseas unverified, not as failure. It means the artifact could not be tied to the command that just ran, so re-run the step or state that the outcome is unproven. Readstatususing dbt's own vocabulary:warnedis not a failure,no_opis a success,partialmeans inspectingbatchesfor which microbatches failed, andincompletemeans everything skipped. Under dbt State, nodes listed innot_executedwere reused or cloned rather than run, so they are not evidence about this change, andreusedmeans nothing ran at all.not_writtenmeans JSON artifacts were off for that command, so report its exit status and log rather than any file in the target directory.uninterpretedmeans dbt v2'sfreshness.jsonis current but its contents are not read; report the command's own output. - Read
required_evidence,satisfied_evidence, andunsatisfied_evidencebefore reporting success.artifact_verifiedproves the result came from the command that just ran; it does not prove the change was validated. Honor a narrower request —--intent testruns the tests — but say plainly what remains open: tests passing against the relation already in the warehouse do not establish that the edited model executes. - If validation fails, identify the smallest repair. When the request explicitly includes fixing failures, apply reversible project-owned SQL/YAML/test edits without a second approval. Otherwise preview the repair first because validation alone does not authorize edits. Contract changes, renames, deletions, and scope expansion always require preview, blast radius, and approval.
- Re-plan after a repair. Never broaden a selector silently. Stop after five execution attempts and report unresolved evidence rather than looping.
Command choice
parse: project structure, Jinja parsing, and graph construction.ls: prove a selector resolves the intended resources.check(dbt v2, when the project has checks): project-metadata rules, run locally without the warehouse.dbt buildruns them first unless--skip-checksis passed; never add that flag to get a build through.show --info <view>(dbt v2): reads the generated Information Schema locally.show --inlineandshow --selectrun SQL in the warehouse and need approval.compile: compiled SQL is needed to resolve uncertainty; treat as warehouse-capable because adapter macros may connect.test: existing relations are current and only assertions need checking.source freshness: a source's freshness or loaded-at configuration changed; select only that source table.run: build is needed but tests are intentionally out of scope.build: models and their tests both require validation; preferred final check for new model work.
For inferred Git scope, formatting/comments and descriptions recommend static parse/selector checks; changed singular tests and test-only model/source YAML recommend test; source freshness/loaded-at changes recommend source-specific freshness; model SQL or contract changes recommend build; and semantic, macro, package, and project-config changes require consumer resolution before warehouse work. These minima still apply when the caller names a model: --allow-broader-intent is the explicit upward override. Never override a narrower user-requested check merely because the planner recommends build.
Never auto-run deps, seed, snapshot, run-operation, a global command, or any command with --full-refresh.
Failure interpretation
Separate compile, database, permission, dependency, selector, and test-assertion failures. A failing uniqueness test is evidence about candidate grain; it is not merely a test to weaken. A failing not_null on a column documented as required is evidence of missing data, even when a note explains the gap; report it and ask rather than scoping the test. A missing column may mean stale lineage, a wrong target, or a model defect. Resolve those alternatives from files and artifacts before editing.
Package upgrades
When the request upgrades a dbt package and asks to fix what breaks, search the project being upgraded (models, macros, tests, snapshots, analyses, YAML) for every call into the package's namespace and check each macro against the target version's changelog, or the vendored copy under dbt_packages/, rather than memory. Major versions often remove or move whole families of macros; for example dbt_utils 1.0 moved its cross-database macros such as datediff to the dbt. namespace. Report each call you changed and each you left, with why it survives. Other projects in the same repository keep their own versions unless the request names them.
Output
Return # dbt Validation with: Changed, including any repairs; Why this was the smallest scope; Verified, including target, commands run, and exit status; Remaining uncertainty. Put any unsatisfied_evidence in Remaining uncertainty in plain words, such as "the tests passed against the existing relation; the edited model was not executed." If execution is awaiting approval or a failure remains, state the exact next command or repair instead of exposing internal workflow state.
Verification
- Every execution used an explicit non-production target and a non-empty selector.
- Planned and resolved selector sets agree.
- Reported pass/fail comes from the current command and artifact, not an older terminal transcript.
- Every repair was either explicitly included in the request or separately approved before application.
- No credentials, profile values, raw rows, or unbounded logs appear in output.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

