agentleFS
zedarvateszedarvates/botte-secrete/skills/nn_audit/SKILL.md

nn_audit

Skill2 starsChanged 3 months agoMITView on GitHub

Audit the micro-NNs — is each model grounded in REAL data, or a synthetic copy of a hand-coded rule? Scans skills/botte_nn and reports, per model, the training data source (real/synthetic/unknown), whether the model file records provenance (trained_on/eval_accuracy), whether a test guards a real-world output, and a grounded/synthetic verdict. Deterministic, 0 cloud tokens. Use to tell which learned components are real vs placeholder, and which should be grounded or replaced by the rule they imitate.

The file

Files
SKILL.md

nn_audit — grounded NNs, or synthetic copies of rules?

A learned component only earns its place if it's trained on real data a rule can't capture. A net trained on np.random + hand-coded labels just approximates a deterministic function we already wrote — strictly worse than the rule (adds error + opacity, learns nothing). This audits which micro-NNs are real.

python -m skills.nn_audit.cli                 # audit skills/botte_nn
python -m skills.nn_audit.cli <dir> --json

Per model it reports:

  • data_source — real | synthetic | unknown, inferred from the training script's content (distill/corpus/labelled → real; np.random → synthetic).
  • wired — does any production file (not tests/training/registry) consume it, and which (usage)? Tells synthetic-but-driving-behaviour (real risk) apart from synthetic-but-orphan (dead weight).
  • has_provenance — does the .json record trained_on / eval_accuracy / data?
  • has_test_guard — does a test assert a specific real-world output for it?
  • verdict — grounded | synthetic — drives behaviour: ground it | synthetic + orphan: delete or wire | unknown; at_risk = synthetic AND wired.

The fix is decided by both axes: a wired synthetic net (e.g. binary_router in the routing belt) must be grounded on real data (distillation / active- learning, like error_classifier); an orphan synthetic net is dead weight to delete or wire. Exposed via [[llm_mcp]] as nn_audit; pairs with [[botte_nn]]. Pure file inspection, 0 cloud tokens.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.