agentleFS
narenaryannarenaryan/agent-skills/skills/security/iam-policy-change-replay/SKILL.md

iam-policy-change-replay

Skill0 starsChanged 41 days agoMITView on GitHub

Use when tightening IAM permissions or reviewing generated infrastructure policies where a passing no-new-access check, aggregate denial, or historical access replay could hide an outage, new resource access, or missing runtime context.

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Add owner-specified required and forbidden requests before inspecting candidate results.

    line 21
  2. Very specific:

    After authorized rollout, verify effective policy identity and representative behavior through rbvm-verified-closure.

    line 24

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

IAM Policy Change Replay

Permission reduction and service continuity are separate claims. Compare identical requests against current and proposed policies; historical logged decisions belong to an older policy state.

Three independent checks

CheckProves within its scopeDoes not prove
No-new-access comparisonCandidate adds no permission relative to referenceRequired callers still work; existing excess access disappeared
Required-access replayNamed principal/action/resource/context cases still succeedUnobserved workflows work
Forbidden-access casesSpecified attack-path requests are deniedEvery possible attack path is blocked

Recipe

  1. Freeze current/candidate policy hashes, evaluator version, resource identities, runtime attributes, and request corpus. Refresh at review if any input changes. Replay both policies now; never substitute logged historical verdicts.
  2. Add owner-specified required and forbidden requests before inspecting candidate results. Include rare recovery jobs, resource boundaries, and relevant attribute variants. Deduplicating only principal/action/resource loses ABAC distinctions.
  3. For AWS simulation, compare per-action/per-resource decisions from ResourceSpecificResults. Current documentation describes a top-level aggregate across resources: one explicit denial can hide another resource's allowance. Check pagination and exact expected coverage; missing/duplicate results or MissingContextValues require investigation, not a pass.
  4. Report unexpected allow→deny and deny→allow transitions separately. A known forbidden request still allowed fails even without a delta. Keep incomplete context or unsupported policy layers unknown.
  5. After authorized rollout, verify effective policy identity and representative behavior through rbvm-verified-closure; simulated authorization neither executes the service operation nor validates its parameters.

Counterexample: denying every action satisfies no-new-access but breaks the deployment caller. Conversely, an aggregate denial can coexist with newly allowed access to another resource. The offline fixture catches both:

python3 skills/security/iam-policy-change-replay/scripts/check_replay.py

Sources: Uber policy replay (2025-09-18); AWS result semantics, reviewed 2026-10-11. Sources, versions, limits.

Pitfalls

  • Equal unknown outcomes are not evidence of equivalence.
  • Historical traffic omits future callers; preserve explicit contract cases.
  • AWS simulator coverage differs from live enforcement; unsupported layers remain a gap.
  • Access Analyzer custom checks can incur charges. This skill grants no API-spending or deployment permission.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.