health-check
Skill4 starsChanged 7 days agoMITView on GitHub
Self-audit the BI team — repo structure lint, unfilled placeholders, metrics-catalog schema, governance policy and output scan, live connection re-tests, data freshness, knowledge-base staleness, eval drift, and inventory drift. Use as the post-setup smoke test, on a weekly schedule, and whenever the team's output starts feeling off.
21 other projects use a version of this.
Adopted by
Most recently taken up by
- codewithmukesh/dotnet-claude-kittheir own version, first seen Nov 8, 2026
- telerik/observability-skillstheir own version, first seen Oct 22, 2026
- allanninal/claude-code-skillstheir own version, first seen Oct 17, 2026
- hosaypeng/claude-skillstheir own version, first seen Oct 13, 2026
- Resgrid/Coretheir own version, first seen Oct 8, 2026
- kubehan/PromAItheir own version, first seen Oct 8, 2026
Used with
Projects with it also use testing.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
NVIDIASkilltesting
Run Python and Rust tests for physicsnemo-curator using uv + pytest with coverage reporting via pytest-cov, and cargo-nextest for Rust tests. Includes benchmark workflows with pytest-benchmark and criterion.
Also in 3 projects that use this
Other versions21 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 21
- zylos-ai/zylos-core★ 1.2ksame name
- codewithmukesh/dotnet-claude-kit★ 808same name
- bitwize-music-studio/claude-ai-music-skills★ 668same name
- Resgrid/Core★ 239same name
- kubehan/PromAI★ 129same name
- hoangsonww/AI-Agents-Orchestrator★ 83same name
- fagemx/edda★ 36same name
- weklund/fiduciary★ 10same name
- aws-samples/sample-aws-ops-skills-for-agents★ 8same name
- davidromeo/tradeblocks-skills★ 4same name
11 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository21 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- analyzeSkill
- build-dashboardSkill
- build-modelSkill
- build-pipelineSkill
- causal-analysisSkill
- connect-dataSkill
- define-kpisSkill
- evalSkill
- experimentSkill
- forecastSkill
- investigate-metricSkill
- make-deliverableSkill
- powerbiSkill
- privacy-reviewSkill
- research-domainSkill
- scorecardSkill
- setup-teamSkill
- tableauSkill
- triageSkill
- upgradeSkill
The file
We split this skill into the rules it gives an agent
6 rules. 0 are common, 3 are distinctive, and 3 are very specific to this project.
0
Common
Most of the 16 edited versions we read keep these.
3
Distinctive
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 62
If the warehouse is reached through MCP rather than a shell, test it by listing tables through the MCP tool and record the result the same …
- Very specific: line 87
The effective policy: If it reports template defaults, the policy was never filled in and the guards are running on fallbacks — say so …
- Distinctive: line 112
Read the dates and judge. There is no script for this because the question is whether the content is still *true*, not whether it was …
- Very specific: line 134
Every directory README (analyses/, pipelines/, dashboards/, models/, experiments/, scorecards/, deliverables/) is supposed to list what's …
- Very specific: line 157
Never fix a governance finding by lowering enforcement, widening the allowlist to make a scan pass, or deleting a failing eval.
- Distinctive: line 161
If everything passes, say so in one line.
“Common” is measured: kept by at least 6 in 10 of the 16 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Health Check — Does This Team Still Work?
A virtual BI team degrades quietly. A connection expires and nobody notices until a scorecard is wrong. A metric gets added to a dashboard but never to the catalog. The business context describes a company that reorganised six months ago. None of that throws an error; it just makes the output slowly less true.
This skill is the scheduled check that catches all of it. Run it after /setup-team,
weekly alongside the scorecard, and any time something feels stale.
Report what you find, fix what is unambiguous, and escalate the rest. Do not fix a finding by lowering the standard — deleting a failing check or loosening a threshold to get a green report is the one genuinely harmful outcome here.
Procedure
1. Repo integrity (fast, always run)
python scripts/lint_repo.py
python scripts/check_placeholders.py
python scripts/check_metrics.py
python scripts/check_governance.py
python scripts/check_provenance.py- lint_repo — agent and skill counts agreeing everywhere they're stated, every agent
and skill present in the
CLAUDE.mdrouting tables, frontmatter valid, cross-file links resolving. A skill missing from the routing table is invisible to the orchestrator; a broken link sends an agent looking for context that isn't there. - check_placeholders — post-setup mode. Any remaining
{{UPPER_SNAKE}}means an agent is reading a template string as if it were a fact about the business.{{lowercase prose}}blanks are expected and fine. - check_metrics — catalog entries parse, required keys present, names unique, and every metric referenced by a scorecard or dashboard spec exists in the catalog.
- check_governance — the policy block parses, every enabled regime has its pack, and no column that looks personal is documented in the Data Dictionary but missing from the Column Registry. An unregistered column is invisible to the query guard.
- check_provenance — findings and scorecards carry the footer. Warnings only, so work that predates the footer does not fail the run; a new document without one is worth a line in the report.
Fix mechanical failures directly (a stale count, a moved file, a missing routing row). Anything that needs a business answer becomes a question for the user.
2. Connections (the one that matters most)
python scripts/test_connection.pyRe-test every registered source. A source recorded as ✅ that now fails gets flipped to ❌ immediately, before anything else in this run — the whole team is downstream of that file. If a source has no registered command, note the coverage gap; an untestable connection is an untrusted one.
If the warehouse is reached through MCP rather than a shell, test it by listing tables through the MCP tool and record the result the same way.
3. Data freshness and health
Hand this to data-quality-engineer, or run its first two checks yourself if the sweep
is small: for every table the scorecard and live dashboards depend on, compare the max
load timestamp against the expected refresh window in knowledge/data-sources.md, and
compare the latest row counts against the trailing distribution.
Report staleness in business terms — "fct_orders is three days behind, so the revenue
line on the weekly scorecard and the exec dashboard are both wrong right now" — not as a
table of timestamps. Open items in knowledge/data-quality-log.md get re-checked here
too; anything that has recurred three times gets escalated as a design problem rather
than logged a fourth time.
4. Governance
python scripts/scan_outputs.py
python scripts/check_governance.py --list- Output scan over the work-product folders. Any finding in something already
shared is an incident (
knowledge/incident-runbook.md), not a to-do. - The effective policy. If it reports
template defaults, the policy was never filled in and the guards are running on fallbacks — say so plainly. - The audit trail (monthly, with
governance-officer): look through.audit/forrestricted_accessentries with"override": true. Each should have a reason a person would recognise; an override with none is a finding. Do not paste statement text from the log into the report. - New tables since the last run that have no Column Registry review.
- Exceptions past their expiry date in the policy's Exceptions Log.
5. Is the team still right?
python scripts/run_evals.py --check-references
python scripts/run_evals.py --historyThe first proves the eval answer key is still valid; the second shows the trend. Flag a
pass rate that has fallen between runs, and flag an eval suite that has not been run
since the last change to an agent, a skill, or a knowledge file (compare the latest
run's git_sha with git log). Running the full /eval costs model usage — recommend
it rather than starting it on a scheduled run. A team with no evals/local/ questions
is measuring the demo, not itself; say so once, not weekly.
6. Knowledge-base staleness
Read the dates and judge. There is no script for this because the question is whether the content is still true, not whether it was edited recently.
business-context.md— does it still describe the company? Priorities and the change ledger go stale fastest. A change ledger with no entries in three months is almost certainly untended rather than accurate, and root-cause analysis depends on it.metrics-catalog.md— any metric marked DRAFT for more than a quarter is either ratified or abandoned; ask which. Any metric withgeneric_sql: TODOis not usable.stakeholders.md— audiences and preferences drift with reorgs.industry-notes.md— briefings past their review-by date are flagged, not deleted.decision-log.md— open Observations that nobody triaged.data-sources.md— accepted quirks past their review date.knowledge/domains/— docs past the review date in their index. Then the check that matters most:git log --since="<last health check>" --name-only -- pipelines/and any mart definitions. A change to a pipeline or mart with no change to a knowledge file in the same or a following commit is drift — list each one. Documentation trailing the data model is the main way this team's accuracy decays, and it decays silently.governance-policy.md— roles that name people who have left; regimes that no longer match where the company operates.
7. Inventory drift
Every directory README (analyses/, pipelines/, dashboards/, models/,
experiments/, scorecards/, deliverables/) is supposed to list what's in that
directory. Compare each inventory against what's actually on disk in both directions:
work that exists but isn't listed is work the team will duplicate, and a listed artifact
that no longer exists is a broken promise to whoever goes looking.
While here, check for dead-end artifacts: dashboards nobody has opened, models nobody scores, scheduled scorecards nobody reads. Retiring one is a real win — it removes maintenance, refresh cost, and a chance to be wrong. Flag them; don't delete anything without asking.
8. Report
Lead with the verdict — healthy, degraded, or broken — and the single most important thing to fix. Then:
- what you fixed automatically
- what needs a human decision, with the specific question
- what you could not check, and why (coverage gaps are findings)
Log anything durable: data problems to knowledge/data-quality-log.md, methodological
or governance rulings to knowledge/decision-log.md.
Never fix a governance finding by lowering enforcement, widening the allowlist to make
a scan pass, or deleting a failing eval. Those are the same harmful move as loosening a
threshold to get a green report.
If everything passes, say so in one line. A health check that produces a page of prose when nothing is wrong trains people to stop reading it.
Running unattended
scheduling/ has the assets to run this weekly next to the scorecard. On a scheduled
run, stay quiet unless something is wrong: report only failures, changes since the last
run, and anything newly stale. See scheduling/SCHEDULING.md.
Created by Colin Beck — https://www.linkedin.com/in/beckcolin/
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

