agentleFS
santoshkanthetysantoshkanthety/powerbi-agent/skills/powerbi-data-governance-traceability/SKILL.md

powerbi-data-governance-traceability

Skill3 starsChanged 7 months agoMITView on GitHub

End-to-end data governance traceability for Power BI and Microsoft Fabric - GDPR/CCPA compliance, Purview lineage, sensitivity labels, retention, consent, and audit-ready evidence packages. Use when the user mentions: governance traceability, data lineage chain, GDPR, CCPA, HIPAA, data retention, right to erasure, data subject request, DSR, consent tracking, regulatory compliance, data classification, impact assessment, DPIA, breach detection, data residency, sovereignty, sensitivity label, Microsoft Purview, traceability report, evidence package.

The file

We split this skill into the rules it gives an agent

1 rule. About 0 are common, 1 is distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    All hops tracked by Microsoft Purview when connected to Fabric workspace.

    line 21

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Data Governance & Traceability in Power BI / Fabric

The Traceability Stack

Full traceability in Fabric means you can answer: "For any metric in a report — which source tables feed it, who can see it, how long is it kept, and can I prove it to a regulator?"

Source System (SQL Server / Snowflake / API)
  └─ Fabric Lakehouse (Bronze → Silver → Gold)
       └─ Semantic Model (Power BI Dataset)
            └─ Report / Dashboard
                 └─ Export / Embed → Consumer recorded in Purview

All hops tracked by Microsoft Purview when connected to Fabric workspace.

Data Classification with Sensitivity Labels

Microsoft Purview Sensitivity Labels propagate from data source → dataset → report → export.

# CLI: apply sensitivity label to a dataset
pbi-agent model set-label --workspace "Analytics" --dataset "Sales Model" --label "Confidential"

# List all datasets and their current sensitivity labels
pbi-agent model list-labels --workspace "Analytics"
# Power BI REST API: assign sensitivity label
import httpx, os

token = os.environ["FABRIC_TOKEN"]
headers = {"Authorization": f"Bearer {token}"}

# Apply "Confidential" label to a dataset
httpx.post(
    "https://api.powerbi.com/v1.0/myorg/datasets/{datasetId}/Default.SetLabels",
    headers=headers,
    json={"datasetId": "{datasetId}", "sensitivity": "Confidential"}
)

Label Hierarchy

LabelUse CasePower BI Behaviour
PublicMarketing contentNo restrictions
InternalInternal dashboardsWatermark on exports
ConfidentialPII, business metricsEncrypted export; logged
Highly ConfidentialFinancial, HR, legalExport blocked; alert on access

Microsoft Purview Integration

Connect Purview to scan and catalog all Fabric / Power BI assets:

# After connecting Purview to your Fabric workspace:

# 1. Trigger a manual scan
pbi-agent catalog purview-scan --workspace "Production"

# 2. View lineage for a report
pbi-agent catalog lineage --report "Revenue Dashboard" --workspace "Production"

# 3. Find all reports using a specific column
pbi-agent catalog impact --dataset "Sales Model" --column "customer_email"

Key Purview capabilities for Power BI:

  • Automatic scan of all datasets, dataflows, reports, dashboards
  • Column-level lineage from SQL source → Fabric table → DAX measure → report visual
  • Business glossary terms linked to Power BI tables and columns
  • Data quality scores surfaced in Purview Data Catalog

GDPR Right-to-Erasure Workflow

import httpx, os

def erase_subject_from_fabric(subject_email: str) -> dict:
    """
    Execute GDPR erasure across Fabric Lakehouse tables and invalidate
    cached Power BI reports containing subject data.
    """
    token    = os.environ["FABRIC_TOKEN"]
    headers  = {"Authorization": f"Bearer {token}"}
    evidence = {"subject": subject_email, "actions": []}

    # Step 1: Identify lakehouses containing this subject
    affected_tables = [
        {"lakehouse": "silver", "table": "customers",     "key_col": "email"},
        {"lakehouse": "silver", "table": "orders",        "key_col": "customer_email"},
        {"lakehouse": "gold",   "table": "customer_360",  "key_col": "email"},
    ]

    for item in affected_tables:
        # Delete via Fabric REST API (Spark SQL on Lakehouse)
        # In practice: run a Fabric notebook via API
        resp = httpx.post(
            f"https://api.fabric.microsoft.com/v1/workspaces/{{workspaceId}}/notebooks/erasure/jobs/instances",
            headers=headers,
            json={"parameters": {"subject_email": subject_email, **item}}
        )
        evidence["actions"].append({
            "type": "delete",
            "target": f"{item['lakehouse']}.{item['table']}",
            "status": resp.status_code
        })

    # Step 2: Trigger full dataset refresh (to flush cached PII from reports)
    httpx.post(
        "https://api.powerbi.com/v1.0/myorg/datasets/{datasetId}/refreshes",
        headers=headers
    )
    evidence["actions"].append({"type": "dataset_refresh", "target": "Sales Model"})

    return evidence
# CLI: erasure request
pbi-agent fabric erasure --subject customer@example.com --evidence-output erasure_2025.json

Data Subject Access Request (DSAR)

def compile_dsar_report(subject_email: str) -> dict:
    """Compile all Power BI / Fabric data held about a subject."""
    report = {"subject": subject_email, "data_found": []}

    # Query each Fabric lakehouse table
    tables_to_check = [
        "silver.customers", "silver.orders", "silver.support_tickets"
    ]

    for table in tables_to_check:
        # Run via Fabric notebook API or direct lakehouse query
        # Pseudocode: actual implementation via Fabric REST API
        results = spark.sql(f"SELECT * FROM {table} WHERE email = '{subject_email}'")
        if results.count() > 0:
            report["data_found"].append({
                "source": table,
                "records": results.toPandas().to_dict(orient="records")
            })

    return report

Column-Level Lineage Documentation

# Show full lineage chain for a DAX measure
pbi-agent model lineage --dataset "Sales Model" --measure "Total Revenue"

# Expected output:
# Total Revenue (DAX measure)
#   └─ orders[revenue] (Power BI column)
#        └─ silver.orders.revenue (Fabric Lakehouse)
#             └─ bronze.raw_orders.revenue_str (Bronze raw)
#                  └─ orders_api (Source API)

Track measure → column → lakehouse → source in CLAUDE.md or Purview glossary:

## Measure: Total Revenue
**Formula**: `SUMX(Orders, Orders[Revenue])`
**Lineage**: `orders_api → bronze.raw_orders → silver.orders → Orders[Revenue]`
**Owner**: analytics@company.com
**GDPR**: Contains transaction amounts (not PII). Retention: 7 years.
**Certified**: Yes — endorsed by Finance team 2025-01-15

Retention Policy Enforcement

# Apply retention labels via Purview (propagates to Power BI exports)
pbi-agent catalog retention --workspace "Production" --label "7-Year-Finance"

# List datasets with no retention label
pbi-agent catalog audit --workspace "Production" --check no-retention-label

Key retention triggers in Power BI:

  • Report exports inherit the dataset's sensitivity/retention label
  • Purview applies retention labels to Fabric OneLake files
  • Power BI activity log captures all export events for audit

Governance Maturity Assessment

# Run a full governance audit
pbi-agent catalog audit --workspace "Production" --output governance_report.md

# Checks performed:
# ✓ All datasets have a sensitivity label
# ✓ All datasets are endorsed (Promoted / Certified)
# ✓ All datasets have an owner assigned
# ✓ All certified datasets connected to Purview
# ✓ RLS applied to all customer-facing reports
# ✓ Column-level lineage captured in Purview
# ✓ Export audit log enabled for Confidential datasets

Audit-Ready Evidence Package

For regulatory audits (GDPR, SOC2, ISO 27001):

# Generate evidence package
pbi-agent catalog evidence --workspace "Production" \
  --regulation GDPR \
  --period 2025-01-01:2025-12-31 \
  --output gdpr_evidence_2025.zip

# Package contains:
# - All sensitivity label assignments + timestamps
# - Purview lineage screenshots
# - RLS rule definitions
# - Access audit log (who accessed what, when)
# - Erasure execution logs
# - Retention label assignments

Governance Traceability Checklist

  • All datasets have a Purview sensitivity label applied
  • Purview workspace scan scheduled (weekly minimum)
  • Column-level lineage visible in Purview for all certified datasets
  • RLS applied to all reports with customer or employee data
  • Erasure procedure documented and tested
  • DSAR report generation tested with a test subject
  • Retention labels applied to all Fabric OneLake tables
  • Power BI activity log exported to Fabric for long-term retention
  • Dataset endorsement status reviewed quarterly

CLI Reference

pbi-agent catalog lineage --report "Revenue Dashboard"       # Full report lineage
pbi-agent catalog impact --dataset "Sales" --column email    # Column impact analysis
pbi-agent catalog audit --workspace Production               # Governance audit
pbi-agent fabric erasure --subject user@example.com          # GDPR erasure
pbi-agent model list-labels --workspace Production           # Sensitivity labels
pbi-agent catalog evidence --workspace Production --regulation GDPR

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.