product-output-safety
Skill2 starsChanged 31 days agoMITView on GitHub
Use when designing or shipping a feature whose OWN AI outputs or automated decisions reach end-users and could harm them: bias, hallucination as fact, missing AI disclosure, unsafe high-stakes automation. MAP, MEASURE, MANAGE with a human gate; never certifies safe. Opt-in: --with-output-safety.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- remigiusz-antczak/deep-code-reviewword for word, first seen Oct 16, 2026
Other versions1 project uses a version of this.
Word for word · 1
- remigiusz-antczak/deep-code-review★ 2100% same
Edited · 0
None found.
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository12 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- agentic-ceoSkill
- agentic-deliverySkill
- business-opsSkill
- communication-structureSkill
- contributionSkill
- deep-code-reviewSkill
- growth-analyticsSkill
- idea-criticSkill
- positioningSkill
- product-discoverySkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 21
Never certify: Do not declare a system "safe", "unbiased", "fair" or "compliant": no test proves it.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Product output safety
Governs the harm a product's own outputs and automated decisions do to its users: a feature that generates, recommends, ranks, scores or acts on a user's behalf can mislead, exclude or damage them even when the code is correct and secure. Code review and business-ops do not cover this axis. Persisted artifacts stay normal English. Lookup table: INDEX.md.
Hard rules
- MAP first, per feature. Before an output-facing feature ships, name its harms: bias / unfair outcomes, hallucination surfaced as fact, over-reliance, missing AI disclosure, deceptive patterns, unsafe automation of high-stakes or irreversible actions. For each, record who is affected and the worst realistic case. An empty inventory is not a pass.
- Never certify. Do not declare a system "safe", "unbiased", "fair" or "compliant": no test proves it. Red-team, measure, report residual risk, recommend controls; the ship decision is the owner's (
evals/evals.json:never-certifies-safe). - Human gate is the floor. Anything costly or hard to reverse (moving money, deleting data, sending on the user's behalf, high-stakes guidance) is gated behind human review or explicit confirmation by default; full autonomous execution of it is the failure. The floor never relaxes at any stage (
high-stakes-action-gated). - Never fabricate a metric. A bias, accuracy or hallucination rate that was not measured is
UNVERIFIED: give the eval to run and how to read it, never an invented "0% bias" or "99.9% accurate". No invented incident rate, benchmark or regulatory threshold (no-fabricated-harm-metric). - One test row per harm. Each row names the probe set (known-bad positive control, known-good negative control, sliced by subgroup), the metric, an owner-set pass threshold (zero tolerance only for a gated irreversible action; never invent a "typical" rate) and the stop condition that blocks launch or triggers rollback. Also sweep prompt injection, leakage of other users' data and excess agency. Never a bare "test for bias" (
one-runnable-test-row-per-harm-with-stop-condition). - Legal duty routes out. Whether an AI disclosure is legally required, whether a decision is regulated, or what a policy must say goes to counsel, and
business-opsnames the regime. Name the EU AI Act, ISO/IEC 42001 and NIST AI RMF, leading with the engineering obligations; timelines are amendment-sensitive: never assert that one binds and never encode a date or version. Recommend disclosure as a control (routes-legal-disclosure-duty,names-and-routes-ai-governance-regimes-never-dates-or-a-binds-verdict). - Confidence is a defined coarse tier, each tier defined and shown as text plus a cue, never a bare "87%" from the model; offer an override. Keep source reliability and claim corroboration as separate axes (
confidence-as-defined-tier-not-model-number). - Ship a model / data card (intended use and out-of-scope uses, subgroup performance, limits and known gaps, owner and model version, data composition, provenance and consent). No card is a transparency finding; a card stating an unmeasured metric is
UNVERIFIED, not a clearance (ai-feature-ships-with-a-model-and-data-card-no-fabricated-clearance). - Answer inside the stated constraint. Treat deadline, budget and goal as fixed inputs: lead with a plan that meets them (fallbacks needing no retraining, staged rollout, kill switch, dated steps, an owner each) and state its cost to the goal. A bare "do not ship" never answers the ask.
- Stage-aware, floors fixed. Early: a per-feature inventory plus a human gate on the single riskiest action is enough; a full governance program is premature. Name an owner for each shipped output-harm and re-review when the feature, model or data changes.
Excuse to rebuttal
- "Confirm it's unbiased so we can ship": I don't certify; here are the inventory, red-team and residual risk. The ship call is yours.
- "Just auto-send the payout / deletion": gate it by default.
- "Do we legally have to disclose it's AI?": counsel decides; I recommend it as a control.
- "Give us our hallucination rate":
UNVERIFIED; here is the eval to run. - "It's low-stakes, skip it": the gate and no-fabrication floors never relax.
Escalate: read this when
references/full-map.md: you are writing the harm inventory, the per-harm test plan, the model/data card, the confidence-tier rules, the stage call or the definition of done; or you need the full anti-rationalization table, the standards list (NIST AI RMF, model/data cards, Kent, Admiralty Code) or the verification cases. It holds the full text of every rule above.scripts/output_safety_check.py: run this when you have the product's output-handling facts as JSON; it checks only rules stated here and SKIPs any input you did not give.evals/evals.json: you change this skill and need the planted cases (certify ask, auto-action, legal duty, fabricated rate, missing card, regime coverage, bare confidence percentage, one test row per harm).
Output structure follows communication-structure if installed.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

