AI Governance Audit

Systematic trust and control assessment for AI/ML systems. Maps the AI system boundary, evaluates risk controls, and identifies gaps against governance frameworks (EU AI Act, NIST AI RMF, ISO 42001).

Usage

/ai-governance                   # full audit of the current project
/ai-governance <path>            # audit a specific ML service or module
/ai-governance --framework eu    # focus on EU AI Act risk classification
/ai-governance --framework nist  # focus on NIST AI RMF (Govern/Map/Measure/Manage)

Behavior

Step 1 — Map the AI system boundary

# Identify model artifacts, training scripts, inference code
find . -name "*.pkl" -o -name "*.pt" -o -name "*.onnx" -o -name "model.*" 2>/dev/null | head -10
grep -rn "model.predict\|model.generate\|pipeline\|inference" --include="*.py" -l . | head -10

# Find dataset references
grep -rn "pd.read_csv\|load_dataset\|dataloader\|train_data" --include="*.py" -l . | head -10

Document:

• What decisions does the model make? (classification, ranking, generation, recommendation)
• Who is affected? (internal tooling vs customer-facing vs high-stakes)
• What data does it consume?

Step 2 — Risk classification

Classify the system under the EU AI Act tiers:

Flag the applicable tier and list required controls.

Step 3 — Evaluate bias and fairness controls

# Look for fairness tooling
grep -rn "fairlearn\|aif360\|what-if-tool\|demographic\|protected\|sensitive_attr\|equalized_odds" \
  --include="*.py" -l . 2>/dev/null

# Check if sensitive attributes are in training data
grep -rn "gender\|race\|ethnicity\|age\|religion\|nationality" \
  --include="*.py" --include="*.csv" --include="*.json" -l . 2>/dev/null | head -10

Check for:

• Fairness metric definition (demographic parity, equalized odds, calibration)
• Bias testing in CI pipeline
• Disparate impact analysis on test set
• Sensitive attribute handling (removed, proxied, or legitimately included)

Step 4 — Evaluate explainability

grep -rn "shap\|lime\|captum\|eli5\|anchors\|explanation\|interpret" --include="*.py" -l . 2>/dev/null

Check:

• Is there a method to explain individual predictions?
• Are explanations surfaced to affected users or decision-makers?
• Is there a global feature importance report?
• For LLMs: are citations / sources provided for generated claims?

Step 5 — Evaluate model risk management

Check for model risk controls:

Step 6 — Evaluate data governance

# Check for data lineage documentation
find . -name "data_catalog*" -o -name "lineage*" -o -name "data_dictionary*" 2>/dev/null
grep -rn "consent\|gdpr\|pii\|personally_identifiable\|anonymize\|pseudonymize" --include="*.py" -l . 2>/dev/null

Check:

• Is training data provenance documented?
• Is PII in training data identified and handled?
• Is there a data retention / deletion policy?
• Can a subject request their data be removed from training?

Step 7 — Report findings

AI GOVERNANCE AUDIT — <system/path>

Risk Classification
  EU AI Act tier: LIMITED (chatbot with emotion recognition disclosure required)
  NIST AI RMF: Map phase incomplete — no impact assessment documented

Bias & Fairness (2 gaps)
  No fairness metrics defined — demographic parity not measured on test set
  gender column present in training features — document why or remove

Explainability (1 gap)
  No explanation method found — users cannot understand why they received a recommendation

Model Risk Management (3 gaps)
  No model card — intended use, limitations, and training data undocumented
  Model artifact (model.pkl) has no version tag — rollback impossible
  No drift monitoring configured — silent degradation in production undetected

Data Governance (1 gap)
  PII (email addresses) in training CSV with no anonymization documentation

Required actions for LIMITED tier compliance:
  [ ] Add disclosure that users are interacting with an AI system
  [ ] Document model limitations in model card
  [ ] Implement opt-out mechanism for data use in training

Edge Cases

• No ML code detected: Check for third-party model API calls (OpenAI, Bedrock, etc.) and apply governance review to the integration layer.
• Internal tooling only: Note reduced risk tier; focus on model risk management and explainability rather than regulatory compliance.
• Foundation model wrapper: Most governance obligations shift to the foundation model provider; focus audit on the application layer (context, output validation, misuse prevention).

Token Optimization

Expected range: 800–2,500 tokens (full audit); 300–600 tokens (single-framework focus)

Patterns used: Grep-before-Read, early exit (if no ML artifacts found, report and suggest scope), progressive disclosure (classification → gaps → required actions)

Caching: Caches system boundary map in .claude/cache/ai-governance/boundary.json (invalidated when model files or training scripts change).

Early exit: If no model artifacts, training scripts, or LLM client code found, report "No AI/ML system detected in this path" and exit.