add-parser
Skill125 starsChanged 7 months agoMITView on GitHub
Use when adding a new file format to remindb's parser package — symptoms include "support .toml/.csv/.xml files", "ingest a new file type", "register an extension in ParseBytes", "wire up a new ContextNode source", or any task that touches `pkg/parser/` to extend supported source formats.
In this repository7 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- add-bench-scenarioSkill
- add-fuzz-targetSkill
- add-integration-testSkill
- add-mcp-toolSkill
- add-store-querySkill
- tune-temperature-policySkill
The file
We split this skill into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 105
Shadowing the package error sentinels: Use ErrInvalidUTF8, ErrUnsupportedExt, and ErrMalformed (skip-on-bad-content) from parser.go.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Add a parser for a new file format
remindb's pkg/parser/ package turns raw source bytes into a slice of *ContextNode trees. Adding a format means routing one new extension through ParseBytes and emitting nodes that match the conventions every other format already follows.
Where the format lands
Five files. Skipping any one of them ships a half-wired format.
| File | What changes |
pkg/parser/<format>.go | New file — the parser itself |
pkg/parser/parser.go | Add a case to the ParseBytes switch |
pkg/parser/<format>_test.go | New file — table tests over fixture strings |
pkg/parser/fuzz_test.go | Add 2–4 f.Add(...) seeds to FuzzParseBytes |
pkg/parser/testdata/ (optional) | Drop a sample file if tests are easier with a real fixture |
The parser file shape
Mirror pkg/parser/yaml.go exactly. The pattern is from .claude/rules/go-concise.md §4 ("Namespace prefix-sharing helpers via a struct"): an empty struct provides the namespace, a free function is the entry point, methods drop the prefix.
package parser
import "fmt"
type TomlParser struct{}
func parseToml(path string, data []byte) ([]*ContextNode, error) {
return TomlParser{}.parse(path, data)
}
func (p TomlParser) parse(path string, data []byte) ([]*ContextNode, error) {
// ... format-specific decode ...
if err != nil {
return nil, fmt.Errorf("failed to parse: toml %s: %w", path, err)
}
return []*ContextNode{buildNode(path, "", root, 1)}, nil
}Initialism rule (memory + go-concise.md): TomlParser not TOMLParser, JsonParser not JSONParser. Pascal-case file-format initialisms.
Error message rule (go-concise.md §5): action errors take a failed to <verb>: prefix and wrap with %w. Validation errors carry no prefix.
Abort vs. skip on bad content: by default a parse failure aborts the whole-workspace compile (the errgroup fails fast). If a malformed file of your format should instead be skipped with a warning — like JSON config that routinely carries comments (#171) — wrap the decode failure with the ErrMalformed sentinel (fmt.Errorf("%w: <format> %s: %v", ErrMalformed, path, err)) instead of the failed to <verb>: form. pkg/compiler/compiler.go already skips ErrMalformed exactly as it skips ErrUnsupportedExt; no compiler change needed. Note this drops the loud error for genuine corruption, so reserve it for formats where unparseable input is expected noise, not a signal.
The switch entry
Open pkg/parser/parser.go, find ParseBytes, add the case in the alphabetic position you'd expect a reader to scan:
switch ext {
case ".md", ".markdown":
return parseMarkdown(path, data)
case ".toml":
return parseToml(path, data) // <-- new line
case ".yml", ".yaml":
return parseYaml(path, data)
// ...
default:
return nil, fmt.Errorf("%w: %q", ErrUnsupportedExt, ext)
}Multi-extension formats list every variant in one case line (see .md/.markdown, .yml/.yaml, .jsonl/.ndjson).
The unit test
pkg/parser/yaml_test.go is the cleanest template. Table-driven, exercises the happy path plus at least one each of: empty input, malformed input, deeply nested input, and the format's specific edge case (BOM, frontmatter delimiter, mixed-type collection, etc.).
The fuzz seed corpus — do not skip this
pkg/parser/fuzz_test.go defines FuzzParseBytes, which scripts/fuzz.sh auto-discovers. Add seeds covering your format's shape variety:
f.Add("data.toml", []byte("title = \"Hello\"\n[section]\nkey = 1"))
f.Add("data.toml", []byte("")) // empty
f.Add("data.toml", []byte("[unclosed")) // malformed
f.Add("DATA.TOML", []byte("title = \"upper\"")) // extension caseThe fuzz invariant is "must never panic regardless of input." If your parser deref's a nil from the decoder or indexes past a slice, fuzz will find it. Run scripts/fuzz.sh 30s locally before committing to confirm the new seeds don't regress.
Quick reference
1. Create pkg/parser/<format>.go (empty struct + parseXxx + (p) parse)
2. Wire pkg/parser/parser.go (one case in ParseBytes)
3. Create pkg/parser/<format>_test.go (table tests)
4. Add seeds to pkg/parser/fuzz_test.go (FuzzParseBytes f.Add lines)
5. go test ./pkg/parser/... (must pass)
6. scripts/fuzz.sh 30s (must pass, no panics)Common mistakes
- Forgetting fuzz seeds. The fuzz test will still run and pass, but it'll never exercise the new format. The fuzz step is what catches "decoder returns nil and parser deref's it" before it ships.
- Returning
(nil, nil)for empty input but the test assertslen(nodes) == 0. Both work sincelen(nil) == 0, but be explicit in the test about which one your parser actually returns.yaml.goshort-circuits withnil, nilwhen the document is empty — mirror its style if your decoder gives you an obvious "no content" signal. - Setting
SourceFileto something other than thepatharg. Downstream code (search ranking,MemoryFetchancestor walks) keys on it. Passpathstraight through tobuildNode. - Shadowing the package error sentinels. Use
ErrInvalidUTF8,ErrUnsupportedExt, andErrMalformed(skip-on-bad-content) fromparser.go; don't redefine them per format. - Stutter naming.
parser.NewTomlParser()is wrong; the empty struct is exposed bare asTomlParserand constructed withTomlParser{}. Seego-concise.md§7.
Cross-references
.claude/rules/go-concise.md— error message format, naming, struct-with-methods pattern.claude/rules/git-versioning.md— commit each of (parser, switch, tests, fuzz seeds) as one logical chunk;feat(parser): add toml supportcovers all five files in one commit since they share the idea.claude/skills/add-fuzz-target/SKILL.md— for the broader fuzz discipline if you're adding a brand-new fuzz target rather than seeding an existing one
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

