ast04-insecure-metadata
Skill2 starsChanged 3 months agoApache-2.0View on GitHub
Detect and triage OWASP AST04 Insecure Metadata — brand-impersonating skill names, permission-understating manifests, risk_tier spoofing, unsafe YAML/JSON/TOML deserialization in SKILL.md frontmatter or config, and ASCII/zero-width smuggling. Use when validating a skill manifest's parser and schema, when a manifest's declared permissions need cross-checking against observed behavior, when triaging a YAML-load code-execution report, or when deciding whether a config-format finding is YAML, JSON, or TOML shaped (they are not interchangeable).
In this repository10 other files it gives its agents.
The file
Files
AST04 - Insecure Metadata
Unofficial community implementation of the OWASP® Agentic Skills Top 10 standard, v1.0. NOT an OWASP project; no OWASP endorsement. OWASP is a registered trademark of the OWASP Foundation, used here descriptively to identify the standard implemented. Source: https://owasp.org/www-project-agentic-skills-top-10/
Pattern: Knowledge. The decision rule that governs this whole category: metadata is
the only signal most reviewers and installing agents act on, yet it is
attacker-controlled and deserialized automatically — often silently, with the agent's
full permission context. Mechanism (schema validation, unsafe-tag detection, Unicode
normalization) lives in scripts/; frozen scenario tiers live in
coverage-matrix.md.
Orientation — read this much first
Fires when you are validating a manifest's parser and schema, cross-checking a declared permission against bundled code, triaging a YAML/JSON/TOML load, or judging a skill name that reads like a vendor's.
Decides, from package bytes alone: five of AST04's seven scenarios — permission understating, risk-tier spoofing, YAML code execution, JSON prototype pollution, TOML config injection.
Does not decide: brand impersonation (needs a trademark corpus no package carries)
and the staged loader (its payload resolves off-artifact at install). Nothing below
settles those two; coverage-matrix.md holds their tier and the reason for it.
Route first. Reasoning for each cross-category call is in Distinguishing AST04 from its neighbors; this table is only the jump.
| What you are holding | Go to |
| A tag the parser executed before any prose was read as instruction | The loader-name trap, then rules 1–2 |
| Prose the agent read and obeyed | AST01 — wrong skill, even in the same SKILL.md |
A declared risk_tier below its permission floor | Rule 3 |
network: false shipped beside an egress call to an outside host | Rule 4 |
| An honest declaration that is merely too broad | AST03 — breadth is theirs, misdeclaration is ours |
| Invisible Unicode with nothing decoded yet | Rule 5 — a precondition; the verdict is AST08-S02's |
| A scanner that should have caught any of the above and didn't | AST08 — a tool gap, not a second AST04 finding |
Stop after Decision rules if you are producing a verdict. Read Where the shipped checks go quiet only when a check returned negative and you must decide whether that is a pass or a blind spot — it is a false-negative map, not background. NEVER is the last stop before you write a finding down: the eight ways a result of either sign gets over-read.
Layer 3 — load on condition, never by default.
coverage-matrix.md(249 lines) — load before re-tiering a scenario, quoting a coverage claim, or reporting an F1 number. Do NOT load it to route a finding: the table above is the whole answer.scripts/detector.py(555 lines) — load before reproducing a false positive or negative at the line level, or editing a pattern table. Do NOT load it to learn what a check covers; every check'sCHECK_COVERAGEclaim is reproduced incoverage-matrix.md, and inferring coverage from code is how anartifact_signalgets quietly promoted into coverage.- Do NOT load either to answer "is this AST04, and does it fire" — this file closes that on its own.
The loader-name trap: why "which loader class does it name" is the wrong question
The tempting shortcut is a safety ladder over PyYAML loader names — UnsafeLoader bad,
FullLoader fine, SafeLoader best. That ladder is wrong, and building a detector on
it produces a confident pass over an RCE-capable call site.
FullLoader was introduced in PyYAML 5.1 as the safe-by-default replacement and then
had two documented remote-code-execution bypasses against it — CVE-2020-1747 (fixed in
5.3.1) and CVE-2020-14343 (fixed in 5.4) — both reached through python/object/new
constructor paths that FullLoader was believed to have closed. PyYAML 6.0 then removed
the implicit default entirely and made the Loader argument mandatory, which is the
upstream project's own statement that no default is defensible.
The decision rule that follows is version-independent, and it has to be: a skill package
does not pin the host's PyYAML version, so "safe on 5.4+" is not a property the artifact
can establish about the machine that will load it. Treat SafeLoader / safe_load as
the only acceptor. Every other spelling — UnsafeLoader, FullLoader, a bare
yaml.load() with no Loader=, or a Loader= bound to a name defined elsewhere —
fires. That is what detect_yaml_injection implements, and it is why the check does
not consult loader names as a hierarchy: it looks for the one construct that has never
had an object-construction path, and treats its absence as the finding.
The same reasoning transfers, with different specifics, to the other two runtimes a
skill may target: js-yaml and Ruby Psych each expose an opt-back-in unsafe API, and
neither is scanned by this package at all (see the quiet list).
Decision rules
- The semantic layer and the parsing layer are two independent findings, not one. A field can lie (brand impersonation, permission understating, risk-tier spoofing) with zero parser involvement, and a parser can execute code with zero semantic deception (a technically honest manifest that still opts into an unsafe loader). Score them separately; a clean parsing check does not clear the semantic layer and vice versa.
- The three declared-format injection shapes are not the same vulnerability class
and do not share a detector. YAML's risk is a deserialization construct
(
!!python/object/apply:...) reachable only through the legacy unsafe loader. JSON's risk (__proto__prototype pollution) is notJSON.parseitself — parsing only creates an own property — it requires a separate, later unsafe recursive merge of the parsed object into a shared config object. TOML has no deserialization/code-execution construct at all; its risk is unvalidated key overrides injecting unexpected properties into a runner's config namespace when precedence rules between sources are not enforced. A single "config injection" check that doesn't distinguish these three will misattribute JSON pollution findings to a YAML-shaped fix, and will have no test at all for the TOML precedence case because there is no payload string to grep for. risk_tierspoofing is decided against a derived floor, and the comparison is deliberately one-directional. The floor this repository computes from the declared permission set (validators/usf.py::derive_risk_tier) is:L0reads only;L1an effective non-identity write and/or an allowlisted egress;L2shell;L3shell combined with write, or a granted write to an agent identity file. Only a declared tier strictly below that floor is the finding — declaring above it is conservative and must not fire, or every cautious author gets convicted and the signal dies. Three consequences follow that a reviewer has to supply by hand, because the check is silent for all three: a manifest with norisk_tierhas nothing to contradict (that is metadata completeness, a different finding); a manifest with no permission block has no floor to derive; and awriteentry thatdeny_writefully shadows grants no capability, so it correctly does not raise the floor — which means a broad write list paired with an equally broad deny list legitimately sits atL0and the breadth never surfaces here. The derivation runs one way only, and that fixes what a correction is worth: raising the declared tier to meet the floor removes the contradiction and moves no permission, so the shell, the egress and the identity write that set the floor are all still granted and every finding resting on them still stands. Correcting the tier closes a misdeclaration here and buys nothing in any other category.- A permission-understating manifest is caught by cross-referencing the declaration
against the bundled code, not by validating the declaration.
network: falsedeclared alongside a script that callscurlto an external endpoint passes a schema-only validator cleanly — "isnetworka valid boolean" is satisfied — and that is why schema validation is the wrong instrument. But both sides of the contradiction ship in the same package: the declared permission in the manifest and the egress call site in the bundled script. The closing check is a declared-versus-code cross-check over package bytes, which is whycoverage-matrix.mdandscenarios/registry.yamlboth tierAST04-S02static-detectable and why no runtime observation is required to decide it. Sandboxed execution buys you breadth — it catches egress the static pass missed, through an interpreter, a vendored binary, or a hostname assembled at runtime — but it is a superset, not the closing condition. Do not read "a schema check does not catch this" as "only a behavioral check catches this"; that inference is what downgrades a static scenario into an unimplementable one. - ASCII smuggling and zero-width payloads are invisible to a human reviewer by
design, so they are a normalization requirement, not a review-process one — and the
carrier's base rate is too high for its presence to convict. Prose that embeds
base64 or control characters passes casual inspection precisely because it was
authored to. Static normalization must therefore run before any human review step,
never as a substitute for one. But the carrier is also ordinary: measured for this
project, a plain-text extraction of the whitepaper's own PDF carries 860 code points
in exactly the class
detectors/scaffold.pyscans for, all of them typesetting artifacts. That base rate is why the shared invisible-Unicode scan is declared a category precondition and never scenario coverage — presence is a prompt to decode and re-scan (AST08-S02's job), not a verdict.
Distinguishing AST04 from its neighbors
- vs AST01: AST01's Instruction Override lives in the prose the agent reads and follows; AST04's YAML/JSON/TOML scenarios require the metadata to be deserialized by a parser, triggering before any prose is even read as instruction. A payload that only fires when the parser processes a specific tag is AST04; a payload that fires because the agent read and obeyed sentences is AST01, even if both live in the same SKILL.md file.
- vs AST03 (Over-Privileged Skills): AST04 is the manifest misdeclaring its actual permission footprint (Permission Understating); AST03 is the manifest honestly declaring a footprint broader than the function needs. Same underlying risk (excessive access), different root cause and different fix — a schema/behavior cross-check fixes AST04; a scope-reduction review fixes AST03.
- vs AST08 (Poor Scanning): metadata and deserialization attacks are explicitly named in the whitepaper as evading pattern-matching scanners — a missed AST04 finding due to scanner blindness (no Unicode normalization, no decode-and-rescan loop) is an AST08 gap in the tool, not a second AST04 finding on the same skill.
Where the shipped checks go quiet
- The YAML loader scan reads
.pyonly. A skill whose loader is JavaScript (js-yamlwith a full schema) or Ruby (Psychwith aliases enabled, orYAML.unsafe_load) has no scanned call site, and the payload half will not save you either: the tag scan matches!!python/…and!!ruby/…and nothing a JS loader would need. Read the loader by hand whenever the package ships a non-Python runtime. - The
Loader=decision is made inside a 200-character window after eachyaml.load(. A safe call whoseLoader=yaml.SafeLoadersits beyond that window (a long multi-line call, a loader chosen above the call) false-positives; an unsafe call that happens to have the literalSafeLoaderwithin that window — a comment, an adjacent safe call, or an import laundered asfrom yaml import UnsafeLoader as SafeLoader— clears. The window is a string match, not a resolution of the name; the only spellings resolved independently of it are a literalyaml.unsafe_load(and a literalLoader=yaml.UnsafeLoader. .mdfrontmatter is only extracted when the file opens with---on the first byte. A leading BOM or blank line yields no frontmatter block and the tag scan sees nothing. Body prose is excluded on purpose (this file discusses!!python/objectand must not convict itself), so a payload parked in a fenced block in the body is out of scope by design — that is AST08's carrier-and-decode surface, not this one.- The prototype-pollution scan reads keys in shipped
.jsononly, and a malformed JSON file is skipped silently. The polluting key delivered as a YAML or TOML value, or built at runtime, is not seen. The in-package recursive merge is corroborating evidence only — its absence is expected, because the exploiting merge usually lives in the host, and requiring it would miss the common shape of a skill that ships only the poisoned manifest. - The TOML check decides structure, not meaning. It fires on a redefined
single-bracket table (found by text scan before
tomllib, which raises on redefinition and used to swallow the finding) and on top-level keys outside a fixed six-name allowlist. A precedence attack expressed entirely within expected keys, or in an[[array_of_tables]], passes — array tables legitimately repeat. AST04-S02needs an egress primitive and an absolute URL on the same source line. A host in a variable, a base URL joined from parts, or an endpoint read from the environment yields no site to compare against the allowlist. And a manifest declaring unrestricted egress (allow: ["*"]) is not understating anything, so the check clears by design — that breadth is AST03's finding, and letting it read as an AST04 pass is the misread to guard against.
Scope and out-of-artifact boundary
Brand impersonation (a skill named google-workspace-integration with no affiliation
to Google) is checkable against a trademark/brand list at publish time but requires an
external reference corpus this artifact does not carry — whether that makes it
static-detectable-with-a-supplied-list or agent-judgable is fixed in
coverage-matrix.md, not decided here.
NEVER — how a check result becomes a wrong verdict
- NEVER report a positive
AST04-yaml-injectionas "an executable payload ships in this package." The check ORs two independent halves — a!!python/…/!!ruby/…tag in shipped YAML or in.mdfrontmatter, and an unsafe loader call site in bundled.py— because a package can ship the payload for a host loader it does not bundle, and can bundle the loader for a payload that arrives later. A bareyaml.load(open(...))with no tag anywhere produces the samedetected=Trueas a planted!!python/object/apply. Read which half the evidence names: one fix pinsSafeLoader, the other deletes bytes, and shipping the wrong one leaves the defect. Naming the half means naming the file it fired in — the tag arm reads every shipped YAML and.mdfrontmatter, so the payload can sit in a sidecar the loader never appears in. A remediation that does not say which file to open is a description of the check. - NEVER file an
AST04-json-injectionhit without locating the key. The three dangerous names are matched at any depth of any shipped.json, and two of them —constructor,prototype— are ordinary property names: a JSON Schema whosepropertiesblock declares a field calledconstructorcomes backdetected=Truein the pollution wording. The cost runs both ways. A benign schema is convicted ofAST04-S06, and because the check returns on its first hit, a genuine__proto__elsewhere in the package is never named in the evidence you are reading. - NEVER read an "unexpected top-level key" as config injection before checking which
file it fired on. The six-name allowlist (
name,description,version,settings,permissions,metadata) is the skill manifest's schema, anddetect_toml_injectionapplies it to every.tomlin the package: a shippedpyproject.tomlfires on['build-system', 'project', 'tool'], aruff.tomlon['line-length', 'lint']. The damage is not the noise. A reviewer who learns to wave this check through is the one who waves through the redefined[table]— the precedence violation the scenario is actually named for. - NEVER treat a parse failure as a scan.
detect_json_injectionskips a filejson.loadsrejects;detect_toml_injectionskips onetomllibrejects, after its redefined-table text scan. Each then returns the same negative, in the same words, it returns for a genuinely clean package. A config the host's lenient parser accepts and the stdlib refuses is the most attacker-friendly shape in this category, and it is exactly the one whose Finding is indistinguishable from clean. Only the list of files actually parsed tells you which of the two you are holding. - NEVER read "no check fired" as "AST04 is clean." Six checks decide five of seven
scenarios.
AST04-S01Brand Impersonation is agent-judgable — name, author and description all ship, but the trademark and vendor-namespace corpus that would settle them does not — andAST04-S05Staged Loader is out-of-artifact; neither has a check, anddetectors/engine.py::run_categoryraisesOutOfArtifactFixtureErrorif a fixture is ever labeled under S05. A verdict that does not name the two questions nobody asked is a statement about the tooling wearing the package's name. Written in the checks' collective voice — all AST04 checks pass — it is worse than the bare negative it replaces, because it names no check, no file and no scenario while sounding like it enumerated all three. A clearance here is per check and per file or it is not one: which check read which bytes, and which of the seven scenarios that leaves standing. - NEVER decide the staged loader from the dependency's name. The tempting close on
AST04-S05is to read the referencedrequirements.txtand judge what it lists. The staging structure is in the artifact; the payload is resolved from an index at install time, is version-range dependent, and can change between the review fetch and the install fetch with no edit to the skill. Judging the listed name isAST04-S01's problem restated with less evidence, and it costs the one control that does work — sandboxing the install-time resolve — by making it look already done. - NEVER let
AST04-invisible-unicode-smugglinginto an AST04 number or a verdict. ItsCHECK_COVERAGEentry carriesregistry_ids: []andcovers: category-precondition, it has no fixture pair, and it is the single reason this module'sF1_SCOPEreadsmixed-proxywhile every corpus-labeled check iscovers: full. With rule 5's base rate, a hit is an instruction to decode and re-scan — AST08-S02's work — not a finding. Counted as coverage it inflates the category with a check that decides no named scenario in any category. - NEVER quote AST04's
1.00as a rate. Each of the five labeled checks is scored over exactly one vulnerable and one clean case (coverage-matrix.md, reconciliation debt 1), and the fixtures share an author with the checks (debt 2), so the corpus cannot surface a blind spot both were built around. The number is evidence that the checks discriminate at all. Printed as precision it buys a reader's confidence with a measurement nobody made; the honest form is "five checks, one labeled pair each, written alongside them".
References
Full attack-scenario catalog and preventive-mitigation list are the whitepaper's own
AST04 section (source: the OWASP Agentic Skills Top 10 publication, section AST04 (no local copy: this repository points at the publication rather than bundling it, so the authority chain cannot drift from a stale copy; the whitepaper is CC BY-SA 4.0)). This file is the delta on top of it. The PyYAML CVE
identifiers and the 6.0 mandatory-Loader change above are upstream facts, not
whitepaper content, and are stated here because the whitepaper's own framing of loader
safety is not sufficient to build a correct check on.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

