agentleFS
joelaharris98joelaharris98/legal-ai-engineering-lab/.agents/skills/security-review/SKILL.md

security-review

Skill2 starsChanged 3 months agoMITView on GitHub

Automatically use when work touches real or potentially confidential legal, client, employer or personal data, external AI providers, databases, APIs, credentials, authentication, authorisation, AI tool/function access, writes/sends/deletes, multi-user access, deployment, or anything where a mistake could expose information or create a consequential side effect. Use it immediately, before reading anything, if the owner points at real client, employer or personal material, wherever it sits. data/real/, data/client/ and data/private/ name it, as does the commit guard firing or a new .repo-safety-allow entry. Also when getting data out of a real system arises - a matter or document management system, an export, a vendor API, a webhook; when retention, deletion or data protection arises, or an eval set is proposed from real matters; before a prototype moves toward real use, or anything here is published outside the machine; and whenever anonymising real material is proposed as a way to make it safe. Do not use for purely synthetic, local, read-only work with no credentials, external calls or side effects, nor for ordinary work in tmp/, which is not itself a trigger.

191 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use dependency-cve-sweep, fleet-standard-adoption and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • berntpoppSkill

    dependency-cve-sweep

    Use when handling Dependabot PRs, CodeQL/Trivy alerts, or doing a routine dependency/CVE sweep for this repo.

    Also in 12 projects that use this

  • berntpoppSkill

    fleet-standard-adoption

    Use when bringing this repo into compliance with a GeneFoundry fleet standard (tool-naming, response-envelope, container-hardening, versioning, MCP transport, or logging) or closing its tracking issue.

    Also in 12 projects that use this

  • xfstudioSkill

    linkerd-patterns

    Implement Linkerd service mesh patterns for lightweight, security-focused service mesh deployments. Use when setting up Linkerd, configuring traffic policies, or implementing zero-trust networking with minimal overhead.

    Also in 6 projects that use this

  • xfstudioSkill

    team-composition-analysis

    This skill should be used when the user asks to "plan team

    Also in 5 projects that use this

  • HABUBUSSSkill

    project-guidelines-example

    Also in 16 projects that use this

  • niloykumarbarmanSkill

    agent-sort

    Build an evidence-backed ECC install plan for a specific repo by sorting skills, commands, rules, hooks, and extras into DAILY vs LIBRARY buckets using parallel repo-aware review passes. Use when ECC should be trimmed to what a project actually needs instead of loading the full bundle.

    Also in 13 projects that use this

  • +2 more from berntpopp

The file

We split this skill into the rules it gives an agent

6 rules. 0 are common, 3 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 163 edited versions we read keep these.

  • 3

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Do not merely produce a security checklist.

    line 87
  2. Distinctive:

    Do not imply that every prototype requires production-grade security.

    line 102
  3. Distinctive:

    Do not use it for work that is entirely synthetic, entirely local, read-only, involves no credentials or external calls, and causes no side …

    line 152
  4. Very specific:

    If the required controls change how the application should be divided into modules, services, tools, or data-access boundaries, invoke …

    line 169
  5. Very specific:

    After implementing a material safeguard, use learning-review when there is a useful transferable concept to consolidate.

    line 170
  6. Very specific:

    If the project now has a defensible story about permissions, tool boundaries, human control, provenance, or auditability, surface it …

    line 171

“Common” is measured: kept by at least 6 in 10 of the 163 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md
<!-- Generated from ../../../.claude/skills/security-review/SKILL.md - do not edit this copy. Edit the source and run `uv run python scripts/sync_agent_files.py`. -->

Security Review

Review the current feature from the perspective of a legal-tech engineer responsible for protecting information and controlling what software and AI are allowed to do.

Explain findings in plain English. Separate realistic risks from remote theoretical ones. Controls should be proportionate to what the current project is actually doing.

Hold one fact steady throughout. This is a laboratory, not a system anybody is deploying. The work here runs locally on invented data; no client is affected, no organisation is relying on it, and nothing built here is waiting on anyone's approval. The purpose of this review is to build the instinct for where the risks in legal AI sit, and to produce a clear-eyed account of what would have to be different if a system like this one were ever used for real. It is not a compliance exercise, and it should never read as though one is under way.

If this fired because real material arrived, start here

Everything below assumes a design to review. If what changed is the data — the owner has referred to a real client, employer, or personal document, or pointed at one on disk — the review is short and it comes first.

Say where the contents would go if you read them, including to whatever model provider you are configured to use (AGENTS.md Safety invariants). Then offer the alternative rather than only the objection: a synthetic fixture carrying the same awkward structure usually tests the same thing, and data/synthetic/README.md explains how to write one that is genuinely difficult rather than merely invented. Do not open the file until the owner has decided.

Two things worth saying plainly at that moment, because they are easy to get wrong in either direction. tmp/ and data/real/ keep a file out of Git; they do nothing about a session. And experiments/<name>/examples/ is a declared-synthetic area — committing there asserts the content is invented, and the commit guard will not stop it, which makes it the worst place for a real document rather than a neutral one.

Consider, where relevant:

  • confidential or privileged information entering systems that are not approved to receive it;
  • how the data would leave the system it currently lives in, and on what terms - a manual export, a scheduled one, a vendor API, a webhook, or direct database access. This is a look-it-up, so it is yours rather than the owner's, and it belongs earlier than it feels: whether the data is reachable at all decides more about a design than anything downstream of it. Four questions settle it - whose credentials, what those can reach, read or write, and what the limits are at volume. docs/LEGAL_AI_DESIGN.md has the routes and what each costs;
  • personal or employer data being used in experiments;
  • personal data as a question distinct from confidentiality — purpose, minimisation, retention, whether every copy could be found for a subject access request, and whether any of it is special category data;
  • provider data-retention and processing boundaries, and whether the provider is acting solely on instructions or retaining and training on what it receives;
  • secrets and credentials;
  • authentication: who is the user?;
  • authorisation: what is that user actually allowed to access or do?;
  • cross-matter or cross-client information leakage;
  • AI or application access that is broader than the capability requires;
  • arbitrary SQL or unrestricted database access where narrow tools would suffice;
  • prompt injection through documents, emails, webpages, or retrieved text;
  • generated information being mistaken for verified or human-approved fact;
  • missing source provenance;
  • AI-derived data silently becoming authoritative data;
  • consequential actions occurring without suitable confirmation or validation;
  • retries accidentally duplicating writes, messages, submissions, or tasks;
  • audit records containing more confidential material than needed;
  • unsafe behaviour when a model or external system fails.

docs/LEGAL_AI_DESIGN.md holds the detail behind most of these: the consequence table and the staged workflow, what each tool boundary buys, logs versus audit events versus traces, the seven data-protection questions and what each changes about the build, the recurring legal failure modes, and the production-readiness questions. Read the relevant part rather than reasoning from memory.

For each material issue

Explain:

  1. the scenario in ordinary language;
  2. why it matters;
  3. the likely consequence;
  4. the smallest proportionate control;
  5. what must become true before the feature is safe enough for the next intended level of use.

Do not merely produce a security checklist. Tie every finding to the actual project.

Where an AI needs access to another system, prefer narrow capabilities such as get_matter, search_documents, or propose_deadline over unrestricted credentials or arbitrary queries. Explain that the model can request a capability while ordinary application code still decides whether the request is valid and authorised.

experiments/003-matter-tool-boundary/ is the worked example, and pointing at it beats restating the argument: the same request is refused identically whether the user made it or whether a file note inside a witness statement did, and the same material then walks out through execute_sql on the same credentials. It also shows the part that is usually missed — the audit entry degrades with the width of the interface for exactly the same reason the authorisation check does.

Where an action has meaningful consequences, consider staged control:

AI interprets/proposes → deterministic checks → preview/approval when appropriate → execute → audit

Do not imply that every prototype requires production-grade security. Make the boundary explicit: what is acceptable for synthetic/local experimentation, and what changes before real data or users are involved.

Questions this project cannot answer, and why that is worth knowing

Some of the most important questions in legal AI are not technical and are not the owner's. Whether a provider may receive client material, what the retention terms are, whether matter data may be anonymised and reused, what a client's engagement terms say — those belong to an organisation, and this repository does not have one.

So do not answer them. Do not guess, do not reason from what is typical elsewhere, and never present a technical control as though it stood in for a permission nobody has given. Encryption is not consent.

What to do instead is the interesting part, and it is an exercise rather than an errand: work out precisely what would have to be asked, of whom, and what would change depending on the answer. Set it out as though drafting it for a named person:

  • Who holds this fact. Information security, the data protection officer, risk and compliance, the practice group, the client relationship partner. Naming the right person is half the skill, and getting it wrong is one of the most common ways a legal-tech project stalls.
  • The question itself, one idea at a time, with a line on why it matters.
  • What turns on the answer. "If the answer is no, the extraction has to run locally and the model option disappears" is the sentence that makes the question worth asking.

Record it on the project brief's map as Not ours to settle, and treat it as a boundary rather than a blocker. The project does not wait for an answer that is not coming; it carries on with synthetic data, having marked where the edge is.

Two things this is not.

It is not a prompt to go and ask anybody. Nothing built here is ready to be put in front of an employer, and taking a half-formed experiment to a firm's risk function is a worse outcome than not asking at all. If real material is ever genuinely wanted, that is a separate decision taken outside this repository, and AGENTS.md Safety invariants sets out what would have to be settled first.

And it is not paperwork. Being able to say "this is the point where it stops being an engineering question — here is what would have to be approved, by whom, and what I would build differently depending on the answer" is one of the strongest things a legal professional can bring to a legal-tech conversation, precisely because most engineers cannot. It is a better answer to "what would need to change before production?" than any list of controls.

When not to use this skill

Controls must be proportionate, and that cuts both ways: reviewing a local experiment on invented data as though it were a production system teaches the wrong instinct and makes the review easy to start ignoring.

Do not use it for work that is entirely synthetic, entirely local, read-only, involves no credentials or external calls, and causes no side effect. Note the boundary in one line and carry on building.

Working in tmp/ is not a trigger. It holds one-off utilities — rename these files, check whether this export parses — which is ordinary work, and firing a review on it teaches the owner to stop reading reviews. What triggers this skill is real material, and it does so wherever that material sits: in tmp/, on the desktop, pasted into the conversation. The directory is not the signal; what the owner said about the file is.

The moment any one of those becomes untrue — a real document, an API key, an external provider, a write, a second user — the review applies immediately.

Handoff to other skills

Hand off to at most one of these, and note the rest rather than running them.

  • If the required controls change how the application should be divided into modules, services, tools, or data-access boundaries, invoke architecture-review.
  • After implementing a material safeguard, use learning-review when there is a useful transferable concept to consolidate.
  • If the project now has a defensible story about permissions, tool boundaries, human control, provenance, or auditability, surface it through interview-review at a meaningful milestone.

Then return to building. Security review is a recurring gate whenever the risk boundary changes, not a one-off document created at the beginning.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.