agentleFS
jhkchanjhkchan/ast10-agent-skills/skills/AST04/SKILL.md

ast04-insecure-metadata

Skill2 starsChanged 3 months agoApache-2.0View on GitHub

Detect and triage OWASP AST04 Insecure Metadata — brand-impersonating skill names, permission-understating manifests, risk_tier spoofing, unsafe YAML/JSON/TOML deserialization in SKILL.md frontmatter or config, and ASCII/zero-width smuggling. Use when validating a skill manifest's parser and schema, when a manifest's declared permissions need cross-checking against observed behavior, when triaging a YAML-load code-execution report, or when deciding whether a config-format finding is YAML, JSON, or TOML shaped (they are not interchangeable).

The file

Files
SKILL.md

AST04 - Insecure Metadata

Unofficial community implementation of the OWASP® Agentic Skills Top 10 standard, v1.0. NOT an OWASP project; no OWASP endorsement. OWASP is a registered trademark of the OWASP Foundation, used here descriptively to identify the standard implemented. Source: https://owasp.org/www-project-agentic-skills-top-10/

Pattern: Knowledge. The decision rule that governs this whole category: metadata is the only signal most reviewers and installing agents act on, yet it is attacker-controlled and deserialized automatically — often silently, with the agent's full permission context. Mechanism (schema validation, unsafe-tag detection, Unicode normalization) lives in scripts/; frozen scenario tiers live in coverage-matrix.md.

Orientation — read this much first

Fires when you are validating a manifest's parser and schema, cross-checking a declared permission against bundled code, triaging a YAML/JSON/TOML load, or judging a skill name that reads like a vendor's.

Decides, from package bytes alone: five of AST04's seven scenarios — permission understating, risk-tier spoofing, YAML code execution, JSON prototype pollution, TOML config injection.

Does not decide: brand impersonation (needs a trademark corpus no package carries) and the staged loader (its payload resolves off-artifact at install). Nothing below settles those two; coverage-matrix.md holds their tier and the reason for it.

Route first. Reasoning for each cross-category call is in Distinguishing AST04 from its neighbors; this table is only the jump.

What you are holdingGo to
A tag the parser executed before any prose was read as instructionThe loader-name trap, then rules 1–2
Prose the agent read and obeyedAST01 — wrong skill, even in the same SKILL.md
A declared risk_tier below its permission floorRule 3
network: false shipped beside an egress call to an outside hostRule 4
An honest declaration that is merely too broadAST03 — breadth is theirs, misdeclaration is ours
Invisible Unicode with nothing decoded yetRule 5 — a precondition; the verdict is AST08-S02's
A scanner that should have caught any of the above and didn'tAST08 — a tool gap, not a second AST04 finding

Stop after Decision rules if you are producing a verdict. Read Where the shipped checks go quiet only when a check returned negative and you must decide whether that is a pass or a blind spot — it is a false-negative map, not background. NEVER is the last stop before you write a finding down: the eight ways a result of either sign gets over-read.

Layer 3 — load on condition, never by default.

  • coverage-matrix.md (249 lines) — load before re-tiering a scenario, quoting a coverage claim, or reporting an F1 number. Do NOT load it to route a finding: the table above is the whole answer.
  • scripts/detector.py (555 lines) — load before reproducing a false positive or negative at the line level, or editing a pattern table. Do NOT load it to learn what a check covers; every check's CHECK_COVERAGE claim is reproduced in coverage-matrix.md, and inferring coverage from code is how an artifact_signal gets quietly promoted into coverage.
  • Do NOT load either to answer "is this AST04, and does it fire" — this file closes that on its own.

The loader-name trap: why "which loader class does it name" is the wrong question

The tempting shortcut is a safety ladder over PyYAML loader names — UnsafeLoader bad, FullLoader fine, SafeLoader best. That ladder is wrong, and building a detector on it produces a confident pass over an RCE-capable call site.

FullLoader was introduced in PyYAML 5.1 as the safe-by-default replacement and then had two documented remote-code-execution bypasses against it — CVE-2020-1747 (fixed in 5.3.1) and CVE-2020-14343 (fixed in 5.4) — both reached through python/object/new constructor paths that FullLoader was believed to have closed. PyYAML 6.0 then removed the implicit default entirely and made the Loader argument mandatory, which is the upstream project's own statement that no default is defensible.

The decision rule that follows is version-independent, and it has to be: a skill package does not pin the host's PyYAML version, so "safe on 5.4+" is not a property the artifact can establish about the machine that will load it. Treat SafeLoader / safe_load as the only acceptor. Every other spelling — UnsafeLoader, FullLoader, a bare yaml.load() with no Loader=, or a Loader= bound to a name defined elsewhere — fires. That is what detect_yaml_injection implements, and it is why the check does not consult loader names as a hierarchy: it looks for the one construct that has never had an object-construction path, and treats its absence as the finding.

The same reasoning transfers, with different specifics, to the other two runtimes a skill may target: js-yaml and Ruby Psych each expose an opt-back-in unsafe API, and neither is scanned by this package at all (see the quiet list).

Decision rules

  1. The semantic layer and the parsing layer are two independent findings, not one. A field can lie (brand impersonation, permission understating, risk-tier spoofing) with zero parser involvement, and a parser can execute code with zero semantic deception (a technically honest manifest that still opts into an unsafe loader). Score them separately; a clean parsing check does not clear the semantic layer and vice versa.
  2. The three declared-format injection shapes are not the same vulnerability class and do not share a detector. YAML's risk is a deserialization construct (!!python/object/apply:...) reachable only through the legacy unsafe loader. JSON's risk (__proto__ prototype pollution) is not JSON.parse itself — parsing only creates an own property — it requires a separate, later unsafe recursive merge of the parsed object into a shared config object. TOML has no deserialization/code-execution construct at all; its risk is unvalidated key overrides injecting unexpected properties into a runner's config namespace when precedence rules between sources are not enforced. A single "config injection" check that doesn't distinguish these three will misattribute JSON pollution findings to a YAML-shaped fix, and will have no test at all for the TOML precedence case because there is no payload string to grep for.
  3. risk_tier spoofing is decided against a derived floor, and the comparison is deliberately one-directional. The floor this repository computes from the declared permission set (validators/usf.py::derive_risk_tier) is: L0 reads only; L1 an effective non-identity write and/or an allowlisted egress; L2 shell; L3 shell combined with write, or a granted write to an agent identity file. Only a declared tier strictly below that floor is the finding — declaring above it is conservative and must not fire, or every cautious author gets convicted and the signal dies. Three consequences follow that a reviewer has to supply by hand, because the check is silent for all three: a manifest with no risk_tier has nothing to contradict (that is metadata completeness, a different finding); a manifest with no permission block has no floor to derive; and a write entry that deny_write fully shadows grants no capability, so it correctly does not raise the floor — which means a broad write list paired with an equally broad deny list legitimately sits at L0 and the breadth never surfaces here. The derivation runs one way only, and that fixes what a correction is worth: raising the declared tier to meet the floor removes the contradiction and moves no permission, so the shell, the egress and the identity write that set the floor are all still granted and every finding resting on them still stands. Correcting the tier closes a misdeclaration here and buys nothing in any other category.
  4. A permission-understating manifest is caught by cross-referencing the declaration against the bundled code, not by validating the declaration. network: false declared alongside a script that calls curl to an external endpoint passes a schema-only validator cleanly — "is network a valid boolean" is satisfied — and that is why schema validation is the wrong instrument. But both sides of the contradiction ship in the same package: the declared permission in the manifest and the egress call site in the bundled script. The closing check is a declared-versus-code cross-check over package bytes, which is why coverage-matrix.md and scenarios/registry.yaml both tier AST04-S02 static-detectable and why no runtime observation is required to decide it. Sandboxed execution buys you breadth — it catches egress the static pass missed, through an interpreter, a vendored binary, or a hostname assembled at runtime — but it is a superset, not the closing condition. Do not read "a schema check does not catch this" as "only a behavioral check catches this"; that inference is what downgrades a static scenario into an unimplementable one.
  5. ASCII smuggling and zero-width payloads are invisible to a human reviewer by design, so they are a normalization requirement, not a review-process one — and the carrier's base rate is too high for its presence to convict. Prose that embeds base64 or control characters passes casual inspection precisely because it was authored to. Static normalization must therefore run before any human review step, never as a substitute for one. But the carrier is also ordinary: measured for this project, a plain-text extraction of the whitepaper's own PDF carries 860 code points in exactly the class detectors/scaffold.py scans for, all of them typesetting artifacts. That base rate is why the shared invisible-Unicode scan is declared a category precondition and never scenario coverage — presence is a prompt to decode and re-scan (AST08-S02's job), not a verdict.

Distinguishing AST04 from its neighbors

  • vs AST01: AST01's Instruction Override lives in the prose the agent reads and follows; AST04's YAML/JSON/TOML scenarios require the metadata to be deserialized by a parser, triggering before any prose is even read as instruction. A payload that only fires when the parser processes a specific tag is AST04; a payload that fires because the agent read and obeyed sentences is AST01, even if both live in the same SKILL.md file.
  • vs AST03 (Over-Privileged Skills): AST04 is the manifest misdeclaring its actual permission footprint (Permission Understating); AST03 is the manifest honestly declaring a footprint broader than the function needs. Same underlying risk (excessive access), different root cause and different fix — a schema/behavior cross-check fixes AST04; a scope-reduction review fixes AST03.
  • vs AST08 (Poor Scanning): metadata and deserialization attacks are explicitly named in the whitepaper as evading pattern-matching scanners — a missed AST04 finding due to scanner blindness (no Unicode normalization, no decode-and-rescan loop) is an AST08 gap in the tool, not a second AST04 finding on the same skill.

Where the shipped checks go quiet

  • The YAML loader scan reads .py only. A skill whose loader is JavaScript (js-yaml with a full schema) or Ruby (Psych with aliases enabled, or YAML.unsafe_load) has no scanned call site, and the payload half will not save you either: the tag scan matches !!python/… and !!ruby/… and nothing a JS loader would need. Read the loader by hand whenever the package ships a non-Python runtime.
  • The Loader= decision is made inside a 200-character window after each yaml.load(. A safe call whose Loader=yaml.SafeLoader sits beyond that window (a long multi-line call, a loader chosen above the call) false-positives; an unsafe call that happens to have the literal SafeLoader within that window — a comment, an adjacent safe call, or an import laundered as from yaml import UnsafeLoader as SafeLoader — clears. The window is a string match, not a resolution of the name; the only spellings resolved independently of it are a literal yaml.unsafe_load( and a literal Loader=yaml.UnsafeLoader.
  • .md frontmatter is only extracted when the file opens with --- on the first byte. A leading BOM or blank line yields no frontmatter block and the tag scan sees nothing. Body prose is excluded on purpose (this file discusses !!python/object and must not convict itself), so a payload parked in a fenced block in the body is out of scope by design — that is AST08's carrier-and-decode surface, not this one.
  • The prototype-pollution scan reads keys in shipped .json only, and a malformed JSON file is skipped silently. The polluting key delivered as a YAML or TOML value, or built at runtime, is not seen. The in-package recursive merge is corroborating evidence only — its absence is expected, because the exploiting merge usually lives in the host, and requiring it would miss the common shape of a skill that ships only the poisoned manifest.
  • The TOML check decides structure, not meaning. It fires on a redefined single-bracket table (found by text scan before tomllib, which raises on redefinition and used to swallow the finding) and on top-level keys outside a fixed six-name allowlist. A precedence attack expressed entirely within expected keys, or in an [[array_of_tables]], passes — array tables legitimately repeat.
  • AST04-S02 needs an egress primitive and an absolute URL on the same source line. A host in a variable, a base URL joined from parts, or an endpoint read from the environment yields no site to compare against the allowlist. And a manifest declaring unrestricted egress (allow: ["*"]) is not understating anything, so the check clears by design — that breadth is AST03's finding, and letting it read as an AST04 pass is the misread to guard against.

Scope and out-of-artifact boundary

Brand impersonation (a skill named google-workspace-integration with no affiliation to Google) is checkable against a trademark/brand list at publish time but requires an external reference corpus this artifact does not carry — whether that makes it static-detectable-with-a-supplied-list or agent-judgable is fixed in coverage-matrix.md, not decided here.

NEVER — how a check result becomes a wrong verdict

  • NEVER report a positive AST04-yaml-injection as "an executable payload ships in this package." The check ORs two independent halves — a !!python/…/!!ruby/… tag in shipped YAML or in .md frontmatter, and an unsafe loader call site in bundled .py — because a package can ship the payload for a host loader it does not bundle, and can bundle the loader for a payload that arrives later. A bare yaml.load(open(...)) with no tag anywhere produces the same detected=True as a planted !!python/object/apply. Read which half the evidence names: one fix pins SafeLoader, the other deletes bytes, and shipping the wrong one leaves the defect. Naming the half means naming the file it fired in — the tag arm reads every shipped YAML and .md frontmatter, so the payload can sit in a sidecar the loader never appears in. A remediation that does not say which file to open is a description of the check.
  • NEVER file an AST04-json-injection hit without locating the key. The three dangerous names are matched at any depth of any shipped .json, and two of them — constructor, prototype — are ordinary property names: a JSON Schema whose properties block declares a field called constructor comes back detected=True in the pollution wording. The cost runs both ways. A benign schema is convicted of AST04-S06, and because the check returns on its first hit, a genuine __proto__ elsewhere in the package is never named in the evidence you are reading.
  • NEVER read an "unexpected top-level key" as config injection before checking which file it fired on. The six-name allowlist (name, description, version, settings, permissions, metadata) is the skill manifest's schema, and detect_toml_injection applies it to every .toml in the package: a shipped pyproject.toml fires on ['build-system', 'project', 'tool'], a ruff.toml on ['line-length', 'lint']. The damage is not the noise. A reviewer who learns to wave this check through is the one who waves through the redefined [table] — the precedence violation the scenario is actually named for.
  • NEVER treat a parse failure as a scan. detect_json_injection skips a file json.loads rejects; detect_toml_injection skips one tomllib rejects, after its redefined-table text scan. Each then returns the same negative, in the same words, it returns for a genuinely clean package. A config the host's lenient parser accepts and the stdlib refuses is the most attacker-friendly shape in this category, and it is exactly the one whose Finding is indistinguishable from clean. Only the list of files actually parsed tells you which of the two you are holding.
  • NEVER read "no check fired" as "AST04 is clean." Six checks decide five of seven scenarios. AST04-S01 Brand Impersonation is agent-judgable — name, author and description all ship, but the trademark and vendor-namespace corpus that would settle them does not — and AST04-S05 Staged Loader is out-of-artifact; neither has a check, and detectors/engine.py::run_category raises OutOfArtifactFixtureError if a fixture is ever labeled under S05. A verdict that does not name the two questions nobody asked is a statement about the tooling wearing the package's name. Written in the checks' collective voice — all AST04 checks pass — it is worse than the bare negative it replaces, because it names no check, no file and no scenario while sounding like it enumerated all three. A clearance here is per check and per file or it is not one: which check read which bytes, and which of the seven scenarios that leaves standing.
  • NEVER decide the staged loader from the dependency's name. The tempting close on AST04-S05 is to read the referenced requirements.txt and judge what it lists. The staging structure is in the artifact; the payload is resolved from an index at install time, is version-range dependent, and can change between the review fetch and the install fetch with no edit to the skill. Judging the listed name is AST04-S01's problem restated with less evidence, and it costs the one control that does work — sandboxing the install-time resolve — by making it look already done.
  • NEVER let AST04-invisible-unicode-smuggling into an AST04 number or a verdict. Its CHECK_COVERAGE entry carries registry_ids: [] and covers: category-precondition, it has no fixture pair, and it is the single reason this module's F1_SCOPE reads mixed-proxy while every corpus-labeled check is covers: full. With rule 5's base rate, a hit is an instruction to decode and re-scan — AST08-S02's work — not a finding. Counted as coverage it inflates the category with a check that decides no named scenario in any category.
  • NEVER quote AST04's 1.00 as a rate. Each of the five labeled checks is scored over exactly one vulnerable and one clean case (coverage-matrix.md, reconciliation debt 1), and the fixtures share an author with the checks (debt 2), so the corpus cannot surface a blind spot both were built around. The number is evidence that the checks discriminate at all. Printed as precision it buys a reader's confidence with a measurement nobody made; the honest form is "five checks, one labeled pair each, written alongside them".

References

Full attack-scenario catalog and preventive-mitigation list are the whitepaper's own AST04 section (source: the OWASP Agentic Skills Top 10 publication, section AST04 (no local copy: this repository points at the publication rather than bundling it, so the authority chain cannot drift from a stale copy; the whitepaper is CC BY-SA 4.0)). This file is the delta on top of it. The PyYAML CVE identifiers and the 6.0 mandatory-Loader change above are upstream facts, not whitepaper content, and are stated here because the whitepaper's own framing of loader safety is not sufficient to build a correct check on.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.