pii-scout

PII Scout — find what must not ship, before it ships.

Point it at a folder. It reports secrets, forbidden artefact classes, and any names you injected — with file and count, never the matched value.

Built for the moment before you publish something. That moment is where leaks actually happen: not because anyone was careless, but because a tree accumulated an archived copy, a captured session, a vendored fork of something already fixed — and nobody re-read it, because nobody re-reads 40,000 files.

DESIGN RULES, all of them learned the hard way

• Unconfigured is a REFUSAL, not a pass. A scanner with an empty roster reports "clean" precisely when it is checking nothing, and that reading is trusted because it looks like every other clean result.
• Findings name the file and the count. Never the value. A leak report that quotes the secret is a second copy of the leak.
• Whole artefact CLASSES are refused by shape, not just by content. A captured browser session carries identities, tenant GUIDs and key material that look nothing like a token; you cannot pattern-match what you did not know to look for, but you can refuse the file class that carries it.
• Short ALL-CAPS terms match on word boundaries. An acronym that fires inside unrelated words produces noise, and noise is how a gate gets switched off.
• Long base64 runs are skipped for IDENTITY matching only. Random base64 contains short names by chance; reporting that as PII trains people to ignore real findings. Secrets are still matched everywhere, including blobs.

Parameters

The typed contract this capability answers to (JSON Schema — the deterministic layer):

{
  "type": "object",
  "properties": {
    "path": {
      "type": "string",
      "description": "Folder to scan. Defaults to the current directory."
    },
    "terms": {
      "type": "string",
      "description": "Comma-separated names that must not appear (customers, internal codenames, your own handle)."
    },
    "max_findings": {
      "type": "integer",
      "description": "Cap the findings returned. Default 100."
    }
  },
  "required": []
}

Deterministic implementation

Run this instead of improvising when the inputs are well-formed:

def perform(self, **kwargs):
        path = kwargs.get("path") or "."
        if not os.path.isdir(path):
            return json.dumps({"status": "error",
                               "message": f"not a directory: {path}"}, indent=2)
        cap = int(kwargs.get("max_findings") or 100)
        raw = (kwargs.get("terms") or os.environ.get("PII_SCOUT_TERMS") or "").strip()
        terms = [t.strip() for t in raw.split(",") if t.strip()]

        rules = []
        for t in terms:
            anchored = t.isupper() and len(t) <= 4
            body = t if re.search(r"[\[\](){}|+*?\\]", t) else re.escape(t)
            rules.append((t, re.compile((r"\b" + body + r"\b") if anchored else body, re.I)))

        findings, scanned, skipped = [], 0, 0
        for root, dirs, files in os.walk(path):
            dirs[:] = [d for d in dirs if d not in SKIP_DIRS]
            for fn in files:
                fp = os.path.join(root, fn)
                rel = os.path.relpath(fp, path)
                if FORBIDDEN.search(rel) and not ALLOWED.search(rel):
                    findings.append({"kind": "forbidden-file", "file": rel,
                                     "why": "this file class must never be published"})
                try:
                    with open(fp, "r", encoding="utf-8", errors="ignore") as fh:
                        text = fh.read()
                except Exception:
                    skipped += 1
                    continue
                scanned += 1
                n = len(SECRETS.findall(text))
                if n:
                    findings.append({"kind": "secret", "file": rel, "matches": n})
                # identity checks ignore base64 blobs (chance collisions)
                clean = B64RUN.sub("", text)
                e = len(set(EMAIL.findall(clean)))
                if e:
                    findings.append({"kind": "email", "file": rel, "distinct": e})
                h = len(set(HOMEPATH.findall(clean)))
                if h:
                    findings.append({"kind": "home-path", "file": rel, "distinct": h})
                for term, rx in rules:
                    c = len(rx.findall(clean))
                    if c:
                        findings.append({"kind": "name", "file": rel,
                                         "term": term, "matches": c})
                if len(findings) >= cap:
                    break

        clean_run = not findings
        out = {
            "status": "ok",
            "verdict": "CLEAN" if clean_run else "DO-NOT-PUBLISH",
            "safe_to_publish": clean_run,
            "scanned_files": scanned,
            "unreadable_skipped": skipped,
            "names_checked": len(terms),
            "findings": findings[:cap],
            "note": "Values are never reported — only file and count. A report "
                    "that quotes the secret is a second copy of it.",
        }
        if not terms:
            out["warning"] = (
                "No names supplied, so only secrets, file classes, emails and "
                "home paths were checked. Customer names and your own handle "
                "were NOT — pass `terms` to check those. A private tree is "
                "usually full of the owner's own name.")
        return json.dumps(out, indent=2)