data-discovery
Skill0 starsChanged 11 days agoMITView on GitHub
Discover where sensitive data lives — structured (databases, warehouses) and unstructured (file shares, SharePoint/Drive, buckets, endpoints, logs, code repos) — using scanners (Macie, Purview, Google DLP/Sensitive Data Protection, open-source detectors), and build a sensitive-data inventory with owners. Use when preparing classification or DLP projects, building privacy records of processing, or scoping breach impact.
In this repository273 other files it gives its agents.
- ad-attack-path-analysisSkill
- adcs-security-reviewSkill
- ad-privileged-access-reviewSkill
- dcsync-detectionSkill
- gpo-security-reviewSkill
- kerberos-attack-detectionSkill
- agent-tool-permission-reviewSkill
- ai-incident-responseSkill
- llm-gateway-controlsSkill
- llm-security-evaluationSkill
- mcp-server-security-reviewSkill
- model-artifact-securitySkill
- prompt-injection-reviewSkill
- rag-security-reviewSkill
- api-abuse-detectionSkill
- api-authorization-testingSkill
- api-inventorySkill
- api-schema-securitySkill
- dast-baselineSkill
- sast-triageSkill
- sca-dependency-riskSkill
- secure-code-reviewSkill
- security-requirementsSkill
- threat-modelingSkill
- asset-criticality-tieringSkill
- asset-inventory-reconciliationSkill
- external-attack-surface-reviewSkill
- shadow-it-discoverySkill
- aws-credential-compromiseSkill
- aws-iam-reviewSkill
- aws-org-guardrailsSkill
- cloudtrail-investigationSkill
- guardduty-triageSkill
- s3-exposure-reviewSkill
- azure-activity-investigationSkill
- azure-rbac-reviewSkill
- conditional-access-reviewSkill
- entra-audit-investigationSkill
- entra-signin-investigationSkill
- backup-assuranceSkill
- critical-service-mappingSkill
- ransomware-recoverySkill
- recovery-exerciseSkill
- cloud-exposure-reviewSkill
- cloud-incident-responseSkill
- cloud-logging-baselineSkill
- cloud-posture-assessmentSkill
- audit-evidence-packageSkill
- continuous-control-monitoringSkill
- control-mappingSkill
- control-testing-samplingSkill
- container-incidentSkill
- dockerfile-hardeningSkill
- image-scanningSkill
- runtime-detection-falcoSkill
- db-audit-loggingSkill
- db-exfiltration-investigationSkill
- db-hardeningSkill
- db-privilege-reviewSkill
- data-classificationSkill
- The first 60 shown.
The file
Files
Sensitive Data Discovery
Tools by location
| Location | Tool |
| AWS S3 | Amazon Macie (managed identifiers + custom), automated sensitive data discovery |
| Azure / M365 / on-prem files | Microsoft Purview Data Map / Information Protection scanner, Content explorer |
| GCP | Sensitive Data Protection (Cloud DLP) discovery for BigQuery, Cloud SQL, Cloud Storage |
| Databases | Column-name and content sampling: information_schema.columns + regex on samples; commercial DSPM |
| Code/logs | Secret scanners (secret-scanning), log PII scanning in the pipeline |
| Endpoints | Endpoint DLP discovery, Purview endpoint content scanning |
Database quick pass (PostgreSQL):
SELECT table_schema, table_name, column_name, data_type
FROM information_schema.columns
WHERE column_name ~* '(ssn|social|passport|iban|card|cc_num|dob|birth|email|phone|address|tax|salary|diagnosis)'
ORDER BY 1, 2;Then sample values (with approval, read-only, small LIMIT) to confirm.
Method
- Prioritize by risk: internet-facing stores, shared drives with broad access, analytics copies, backups, dev/test environments with production data.
- Scan with sampling; record detector, confidence and counts — not raw values.
- Validate false positives (test data, synthetic records).
- Assign owner and classification for each store; record legal basis/retention where personal data.
- Remediate: delete unnecessary copies, mask/tokenize in non-prod, restrict access, encrypt.
Inventory record
Store | Location | Data types | Est. records | Classification | Owner | Access breadth | Encryption | Retention | Action
Safety
Discovery scanners read sensitive content: use least-privilege service identities, keep results metadata-only, protect findings (they are a map of sensitive data).
Related skills
data-classification, data-retention-review, dpia, secret-scanning
References
- Amazon Macie sensitive data discovery: https://docs.aws.amazon.com/macie/latest/user/data-classification.html
- Microsoft Purview Data Map: https://learn.microsoft.com/en-us/purview/concept-elastic-data-map
- Google Sensitive Data Protection discovery: https://cloud.google.com/sensitive-data-protection/docs/data-profiles
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

