sc-ingest — Semantic Chunker (Pipeline Stage 1)

Splits a Solidity codebase into semantic chunks — one per function, modifier, constructor, fallback, receive, plus one __state__ chunk per contract for its storage layout. Each chunk is self-describing so that later stages never need to re-read whole files: they read chunks and the dependency graph instead. This is how the pipeline keeps the context window small while reasoning about a large codebase — the chunk bank is the "one big context file", normalized and machine-readable.

When to Use

• Starting an audit of a Solidity / EVM codebase (Foundry, Hardhat, plain .sol).
• The orchestrator (sc-audit) calls this first.
• User says: "chunk this codebase", "build the audit context", "ingest the contracts".

Announce

Begin with: "I'm using the sc-ingest skill to chunk this codebase into the audit context bank..."

Why deterministic (read before changing anything)

Downstream skills reference chunks by stable ID (Contract.func(arg1,arg2)). If chunk IDs change between runs, the dependency graph, hypotheses, and findings all break. The chunker is therefore a deterministic script — do not replace it with ad-hoc LLM chunking. LLM judgement is applied later (sc-map enrichment), never to the cut points themselves.

Workflow

1. Locate the project & toolchain

• Confirm the target directory holds .sol files (Glob "**/*.sol").
• Detect Foundry (foundry.toml, lib/, remappings.txt) or Hardhat (hardhat.config.*, node_modules/). Note src/, contracts/ layout.
• Check whether solc is available: solc --version. If present, you may pass --use-solc for an extra high-fidelity AST dump (optional; the chunker works fully without it).

2. Run the chunker

python skills/sc-ingest/scripts/chunk_solidity.py <project_dir>
# options:
#   --out <dir>          default <project_dir>/.audit
#   --use-solc           also dump solc --ast-compact-json per file (if solc present)
#   --include-tests      include *.t.sol / *.s.sol (default: excluded from audit set)
#   --include-deps       include lib/ node_modules/ dependencies (default: excluded)

Default behavior excludes tests, scripts, and third-party dependencies from the audit set (they are tagged, not chunked, unless flags are passed). Audit in-scope code first; pull in dependencies only when a hypothesis crosses into them.

3. Enforce the coverage gate (MANDATORY)

The script prints a coverage line and exits non-zero on mismatch:

coverage: 41/41 functions chunked (OK)

• If it prints MISMATCH or exits non-zero, stop and investigate before proceeding. A function that did not chunk is a function that will never be audited. Read manifest.json → coverage, find the offending file, and check for unusual syntax (assembly-heavy, exotic formatting, parse-defeating macros).
• Open manifest.json and sanity-check counts.contracts against what you see in the repo. A contract count of 0 means the source layout wasn't found.

4. Spot-check extraction quality

Read 2–3 representative chunks (especially the most complex external functions) and confirm:

• storage_reads / storage_writes look right.
• external_calls captured the real calls.
• access_control picked up the privilege modifiers.
• tags include the expected signals (see Tag Vocabulary below). If a chunk has a warnings entry (e.g. inline assembly), note it — those chunks need extra human/LLM attention later because heuristics are weaker there.

5. (Optional) Semantic enrichment

The script's read/write/call sets are heuristic (extraction: "heuristic"). You MAY enrich a chunk's JSON in place (set extraction: "llm-enriched") when:

• A chunk has inline assembly and you can determine the real storage touched.
• An external call was missed because it went through a function pointer or using X for Y library binding. Never rewrite IDs, source spans, or hashes during enrichment.

Output Artifacts

.audit/
  manifest.json              # index: files, contracts, inheritance, state vars, coverage
  chunks/<sanitized-id>.json  # one file per chunk
  ast/<file>.ast.json         # only with --use-solc, if solc present

The chunk schema is documented in templates/chunk.schema.json; the manifest in templates/manifest.schema.json. Full field reference in reference.md.

Tag Vocabulary (the signals downstream stages route on)

reentrancy-surface and unprivileged-reachable are the highest-signal tags for stage 4 (hypothesis generation) — they mark unguarded, state-mutating entry points that make external calls.

Coverage & Quality Gate (the contract this skill must honor)

MANDATORY before reporting stage 1 complete:

1. coverage.all_functions_covered == true in manifest.json.
2. counts.contracts > 0 and matches the visible source layout.
3. No silently-skipped in-scope .sol files (cross-check Glob vs manifest.files).
4. Every chunk has a non-empty source and a source_hash.

If any gate fails, the downstream audit is unsound — fix ingestion first.

Known Limitations (documented honestly; surfaced as chunk warnings where detectable)

• Inline assembly: storage reads/writes inside assembly {} blocks are not fully resolved → chunk carries a warning.
• Function pointers / using ... for: calls dispatched through these may be classified as internal or missed.
• Inheritance shadowing: a __state__ chunk lists a contract's own declarations; inherited storage is recorded via inherits_from and resolved in sc-graph, not here.
• Decimals/type inference: read/write detection is name-based, not type-checked. Treat as a strong hint, not proof.
• Overloaded functions: same canonical signature ⇒ later ones get #N suffix and a duplicate-or-overload-id tag.

These are intentional v1 trade-offs: determinism and offline operation over a full type-checker. sc-graph and sc-map layer semantic resolution on top.

Handoff

On success, tell the orchestrator: chunk count, contract count, coverage = OK, and the location of .audit/. Next stage: sc-map (build project-map.md).