wicked-garden-mem-ingest
Skill9 starsChanged 7 months agoMITView on GitHub
Ingest source files into the wicked-estate knowledge store as cited chunks. Handles text files (md, txt, csv, html, json, code) with deterministic chunking and binary documents (pdf, docx, pptx, xlsx, png, jpg, gif, webp) via LLM vision — the agent extracts and chunks, estate persists and ranks. Use when: dispatched by the wicked-garden-mem skill's ingest action, or directly for 'ingest this file', 'add this document to the knowledge base', 'index this directory', 'learn from this pdf'.
- Installs packages
In this repository142 other files it gives its agents.
- AGENTS.mdAGENTS.md
- .claude/CLAUDE.mdCLAUDE.md
- plugin-json-schemaSkill
- readme-style-guideSkill
- releaseSkill
- scaffoldSkill
- wicked-garden-agentic-agentic-patternsSkill
- wicked-garden-agentic-architectSkill
- wicked-garden-agentic-context-engineeringSkill
- wicked-garden-agentic-frameworksSkill
- wicked-garden-agentic-performance-analystSkill
- wicked-garden-agentic-review-methodologySkill
- wicked-garden-agentic-safety-reviewerSkill
- wicked-garden-agenticSkill
- wicked-garden-agentic-trust-and-safetySkill
- wicked-garden-appsSkill
- wicked-garden-archetypeSkill
- wicked-garden-build-guideSkill
- wicked-garden-classifySkill
- wicked-garden-coreSkill
- wicked-garden-data-engineerSkill
- wicked-garden-dataSkill
- wicked-garden-deliberateSkill
- wicked-garden-demoSkill
- wicked-garden-domain-coverageSkill
- wicked-garden-domain-extractorSkill
- wicked-garden-domain-modelerSkill
- wicked-garden-domainSkill
- wicked-garden-draftSkill
- wicked-garden-editor-scaffoldSkill
- wicked-garden-engineering-api-documentarianSkill
- wicked-garden-engineering-architectureSkill
- wicked-garden-engineering-backendSkill
- wicked-garden-engineering-debuggingSkill
- wicked-garden-engineering-docsSkill
- wicked-garden-engineering-frontendSkill
- wicked-garden-engineering-integrationSkill
- wicked-garden-engineering-large-scale-migrationSkill
- wicked-garden-engineering-migration-engineerSkill
- wicked-garden-engineering-patchSkill
- wicked-garden-engineeringSkill
- wicked-garden-engineering-solution-architectSkill
- wicked-garden-engineering-system-designSkill
- wicked-garden-engineering-unit-test-qualitySkill
- wicked-garden-governed-workerSkill
- wicked-garden-groundSkill
- wicked-garden-jam-brainstorm-facilitatorSkill
- wicked-garden-jam-councilSkill
- wicked-garden-jamSkill
- wicked-garden-learnSkill
- wicked-garden-mcp-scaffoldSkill
- wicked-garden-mcpSkill
- wicked-garden-mem-captureSkill
- wicked-garden-memSkill
- wicked-garden-personaSkill
- wicked-garden-platform-auditSkill
- wicked-garden-platform-compliance-officerSkill
- wicked-garden-platform-complianceSkill
- wicked-garden-platform-errorsSkill
- wicked-garden-platform-gate-benchmark-rebaselineSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 23
If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is …
- Distinctive: line 34
All writes go through the mem backend (stdio shim → estate MCP):
- Very specific: line 106
Do NOT ingest files one-by-one in conversation.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Mem Ingest Worker
This skill is designed to run as an isolated worker; when your harness cannot fork, run it inline and keep its output separate from the caller's.
You ingest source files into the wicked-estate knowledge store so later
knowledge.recall / cited-answer calls can return them with source
citations. This ports the brain-era ingest pipeline (FOLD-2, Phase 5-S7)
onto estate: you read, extract, and chunk (DEC-R: the agent is the
reasoner); the engine writes and ranks (knowledge.ingest /
knowledge.write — deterministic, never calls a model). Estate's embedded
method card skill://knowledge-ingest/SKILL.md is the chunking doctrine
this pipeline follows.
Runtime
Script-backed steps use the wicked-garden launcher: wicked-garden run <plugin-root-relative path, e.g. scripts/…> [args]. It is on PATH after npm i -g wicked-garden; otherwise use npx wicked-garden run …; inside a wicked-crew run it is "$WICKED_GARDEN_ROOT/scripts/wicked-garden".
If none of these is available, or Python 3 is missing, skip the script-backed step, say so, and follow the manual alternative where one is given next to it — never invent the script's output.
Relative paths in this skill are relative to the directory that contains this SKILL.md.
Parameters
- source (required): path to a file or directory
- scope (optional): estate scope, default
project:<cwd-basename> - title (optional): document title, default derived from the filename
The write seam
All writes go through the mem backend (stdio shim → estate MCP):
wicked-garden run \
scripts/mem/estate_memory.py ingest -with a JSON body on stdin (chunks are long — never inline them in argv):
{"title": "<doc title>", "chunks": ["<chunk 1>", "<chunk 2>"],
"scope": "<scope>", "source": "<absolute-or-repo-relative source path>"}Rules that make chunks worth storing:
- Self-contained — one idea per chunk (a section, a slide, a sheet, a table row group). Recall returns chunks verbatim; a fragment that needs its neighbours is a bad chunk.
- Source provenance always —
sourceis what a cited answer cites. An uncited chunk cannot backwicked-garden-search answer. Never omit it. - For a single standalone fact, use the
writeaction instead ({"content": ..., "class": "concept", "source": ...}).
Step 1 — assess the source
- Single file → Step 2 (text) or Step 3 (binary) by extension.
- Directory → Step 4 (batch).
- Health check when in doubt:
... estate_memory.py health '{}'— on"ok": false, stop and report the degrade (estate binary missing or store unreachable). Do not fake success.
Text extensions: .md .txt .csv .html .htm .json .py .js .jsx .ts .tsx .sh
.rs .go .java .yaml .yml .toml. Binary (vision) extensions: .pdf .docx
.pptx .xlsx .png .jpg .jpeg .gif .webp.
Step 2 — TEXT files (deterministic chunking)
- Read the file.
- Split:
- Markdown: on H1/H2 headings; sub-split any section over ~800 words at paragraph breaks.
- Code: one chunk per file (small files) or per top-level
class/function group; prepend a one-line
path — what it isheader so the chunk stands alone. - CSV/other text: paragraph groups of ~800 words; for CSV keep the header row with every chunk.
- Send title + chunks + scope + source through the write seam (one
ingestcall per file).
Step 3 — BINARY files (LLM vision)
You receive the document natively via Read — examine it visually and
extract content as markdown. This is the one genuinely agent-only stage of
the pipeline; everything around it stays deterministic.
- PDF: one chunk per logical section (or every 3–5 pages).
ReadPDFs with thepagesparameter for long documents. - PPTX: one chunk per slide or tight slide group; capture speaker-note intent, not just visible text.
- DOCX: one chunk per section heading.
- XLSX: one chunk per sheet; render data as markdown tables (downsample huge sheets: header + representative rows + totals).
- Images: one chunk describing the visual content, plus any legible text verbatim.
Preserve numbers, names, and tables exactly — a cited answer will quote
them. Then write through the same seam as Step 2, source = the original
binary's path.
Step 4 — directories (batch)
Do NOT ingest files one-by-one in conversation. Write a small Python batch
script to the session scratch dir that walks the tree (skip dotdirs,
node_modules, __pycache__, lockfiles), applies the Step-2 chunking to
each text file, and pipes one ingest JSON per file into
wicked-garden run scripts/mem/estate_memory.py ingest -. Run it, then:
- Report
files → chunkscounts from its output. - List the binary files it found and vision-ingest the important ones yourself (Step 3), or hand the list back for the caller to choose.
Step 5 — verify (seeded recall)
Pick 1–2 distinctive phrases from the ingested content and confirm they come back cited:
wicked-garden run \
scripts/mem/estate_memory.py sources '{"query":"<distinctive phrase>"}'The matching chunk must appear with the right source. If it doesn't,
say so — never report an unverified ingest as done.
Step 6 — report
{N} files ingested, {M} chunks written, scope {scope}- doc_ids returned by the backend
- binary files pending vision ingest (if any were deferred)
- verification: which seeded phrases came back, with their
source
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

