brain-ingest-gate

---
name: brain-ingest-gate
version: 1.0.0
description: >
  Pre-write quality gate for content entering the brain. No raw copies: a bare
  cp/mv into the brain repo is a bug. Before any new page lands, resolve named
  entities registry-first (a vector score is a floor for prose, never a gate
  for named things), then run the read-the-top-hit dedup decision tree
  (clear-dup / plausible-dup / clear). Owns dedup; delegates enrichment to the
  shipped ingestion skills. Routing convention, not an operation-boundary
  enforcement.
triggers:
  - "move this to brain"
  - "migrate to brain"
  - "copy these files into the brain"
  - "is this already in the brain"
  - "check for duplicates before writing"
  - "dedup before saving"
  - "raw copy to brain"
mutating: true
writes_pages: true
writes_to:
  - people/
  - companies/
  - concepts/
  - projects/
upstream: brain-ingest-gate@fc834ee
# Brain-first applies in its purest form here: the entire gate IS a
# brain-first lookup performed at write time (entity card, alias-expanded
# search, read the top hit) before anything external or new is written.
brain_first: true
---

# Brain Ingest Gate — Resolve and Dedup Before Anything Enters the Brain

> **Convention:** see [conventions/brain-first.md](../conventions/brain-first.md) —
> the lookup chain (`gbrain entity` → `search` → `query` → `get`) is the same
> chain this gate runs before every write.
>
> **Convention:** see [_brain-filing-rules.md](../_brain-filing-rules.md) —
> when the gate's verdict is "write", the primary subject picks the directory.
>
> **Convention:** `skills/conventions/quality.md` owns the cross-cutting page
> rules (citations, Iron Law back-linking, notability) — every page the gate
> lets through follows them. Gate-specific delta: the gate only decides
> write/link/skip; the admitting skill applies the quality rules on write.

## The Rule

**No content enters the brain without passing this gate. A raw `cp` or `mv`
into the brain repo is a bug.**

One insight, one place. If it already exists, link to it — don't clone it.
Before any new page is written (file migration, bulk import, manual
`gbrain put`, subagent output), two checks run in order:

1. **Named-Entity Resolution Gate** — is this about a named thing that
   already has a page under its chosen name?
2. **Dedup Gate** — does the brain already state this insight somewhere?

**Scope honesty:** this gate is a routing convention — the harness resolves it
into context when an ingest-shaped intent matches, and a well-behaved agent
follows it. Nothing in the gbrain runtime mechanically blocks an unenriched or
duplicate write if the skill never loads.

## Why gbrain needs this gate

The native pipeline does NOT do semantic dedup for you:

- **`gbrain import` / `gbrain sync` skip only matching frontmatter IDs.**
  Identical content under a different slug or ID indexes twice — every
  duplicate becomes a second search hit competing with the canonical page.
- **`gbrain capture`'s dedup is a 24-hour exact content-hash** — it catches
  re-captures of identical bytes, not the same insight reworded.
- **The `remember` verb dedupes facts, not pages.**

Semantic dedup and named-entity resolution are this skill's job, in full.

## When This Gate Fires

1. **File migration** — moving files already in the workspace into the brain
   repo ("move this to brain").
2. **Bulk imports** — batch moves of any kind into brain directories, BEFORE
   `gbrain sync` or `gbrain import` indexes them. For batches, also read
   [conventions/test-before-bulk.md](../conventions/test-before-bulk.md):
   gate 3-5 items and inspect the decisions before running the rest.
3. **Manual writes** — `gbrain put` or `gbrain capture` of rich content, or
   direct file writes into the brain repo.
4. **Subagent output** — background agents writing notes or pages into the
   brain.

## What This Gate Owns vs Delegates

This skill is a **gate**, not a pipeline. It owns the pre-write checks below.
Everything downstream of a "write" verdict is delegated to shipped skills —
do not restate their steps here or inline:

| Concern | Delegate to |
|---|---|
| Routing new external content (meetings, articles, media) | [ingest](../ingest/SKILL.md) |
| Entity detection + notability on inbound content | [signal-detector](../signal-detector/SKILL.md) |
| Creating/updating person + company pages, tiered effort, backlinks | [enrich](../enrich/SKILL.md) |
| Concept pages, tiering, cluster synthesis | [concept-synthesis](../concept-synthesis/SKILL.md) |
| Back-link enforcement (Iron Law) | [conventions/quality.md](../conventions/quality.md) |
| Which directory the page lands in | [_brain-filing-rules.md](../_brain-filing-rules.md) |

## Named-Entity Resolution Gate (runs FIRST)

**Fires whenever the content is about a NAMED project, place, company, person,
or anything someone "wants to build / found / make."**

Vector similarity alone cannot be trusted to catch named-entity dupes: a page
stored under its chosen NAME will not embed close to the generic English
phrase someone happens to describe it with. The classic failure: a search for
a descriptive phrase scores the canonical named page below the prose floor, so
a duplicate stub gets written on top of a years-old page. Stored by named
meaning; retrieval attempted by literal generic phrase.

### The rules

1. **Resolve registry-first, not by the generic phrase.** gbrain's native
   registry is the entity surface:

   ```bash
   gbrain entity "<name>"        # zero-LLM card: page, aka list, near-miss suggestions
   ```

   A card hit means the page exists — STOP, link, don't clone. On a miss (or
   for concept-shaped nouns), fall through to `gbrain query "<name>" --limit 3`.
   If the brain also keeps an explicit index of named initiatives (e.g. a page
   under `concepts/`), read it before concluding anything is new.

2. **Expand through aliases before searching.** Named pages should carry an
   `aliases:` frontmatter list (generic label + chosen name + any nickname +
   signature phrase). Search EACH alias and the generic label, not just the
   phrase the user happened to say.

3. **A vector score is a floor for prose, NEVER a gate for named things.**
   If there is ANY plausible named match, open and read the candidate page
   (`gbrain get <slug>`) before concluding it doesn't exist. A named page can
   be the right answer at a score that would be a clear miss for prose.

4. **When a NEW named thing appears, bake its aliases in the same write.**
   Create the page with the full `aliases:` list so every future synonym
   resolves through `gbrain entity`. One frontmatter list covers all future
   phrasings — O(1), not a per-instance reminder.

### Why a gate and not a memory note

A memory reminder ("query the real name, not the generic phrase") is a
per-instance sticky note: it only works if it happens to be in hot context
that turn, doesn't generalize to the next named entity, and rots. This skill
loads when an ingest-shaped task routes here. Process rules belong in the
triggered gate, not in hot memory.

## Dedup Gate (runs SECOND)

Before writing ANY new page (for named things, the resolution gate above runs
first and takes precedence):

1. **Extract the core claim** — 1-2 sentences capturing what's novel about the
   new content.

2. **Search for it:**

   ```bash
   gbrain search "<core claim>" --limit 5
   ```

3. **OPEN AND READ the top hit** (`gbrain get <slug>`). Never band on the
   score alone. Donor systems publish cosine cutoffs for this step — do NOT
   port them: `gbrain search` returns fused hybrid rank scores, not cosine
   similarity, and no numeric threshold maps across. The band comes from
   reading, not from the number.

4. **Assign a band:**

   | Band | Meaning | Action |
   |---|---|---|
   | **clear-dup** | The top hit already states the same insight about the same subject | STOP. Link to the existing page (`gbrain link` / `gbrain timeline-add`) instead of writing. |
   | **plausible-dup** | Same territory; possibly a new angle | Read both fully. Same insight → link, don't write. Genuinely new angle → write WITH a cross-link to the existing page. |
   | **clear** | Nothing in the top results covers the claim | Write normally through the delegated enrichment skills. |

### Decision tree

```
New content to write
  ├─ Named thing? → Named-Entity Resolution Gate first
  │    (entity card → alias-expanded search → READ the candidate)
  ├─ Extract core claim (1-2 sentences)
  ├─ gbrain search "<core claim>" --limit 5
  └─ OPEN AND READ the top hit (gbrain get <slug>)
      ├─ clear-dup     → STOP. Link to existing. Report "duplicate".
      ├─ plausible-dup → Read both. Same insight?
      │    ├─ yes → STOP. Link to existing. Report "duplicate".
      │    └─ no  → Write with cross-link. Report "new angle".
      └─ clear         → Write via enrichment skills. Report "unique".
```

### When to skip dedup

- **Operational/state files** — time-series records, not knowledge.
- **Meeting transcripts** — each meeting is unique by definition (entities
  INSIDE it still go through the named-entity gate via the delegated skills).
- **Timeline entries on existing pages** — back-links are additive, not
  duplicative.
- **Media files** — dedup by filename/hash, not semantic similarity.

## Verification

After the batch, verify the gate's output holds:

```bash
gbrain check-backlinks check            # mentioned entities link back (fix with: check-backlinks fix)
gbrain backlinks <new-slug>             # each new page has inbound links
gbrain search "<core claim>" --limit 3  # the insight has exactly ONE home
```

If `check-backlinks check` reports gaps on pages the gate just admitted, the
enrichment delegation was skipped — route back through
[enrich](../enrich/SKILL.md) before declaring the ingest done.

## Contract

This skill guarantees:

- No new page enters the brain through this skill's flows without the
  named-entity resolution check and the dedup check running first.
- Every "duplicate" verdict names the matched slug and produces a link or
  timeline entry instead of a clone.
- New named-entity pages carry an `aliases:` frontmatter list in the same
  write that creates them.
- Dedup bands are assigned by READING the top hit, never by score alone; no
  numeric similarity thresholds are used against gbrain's fused scores.
- Enrichment is delegated to shipped skills (ingest, enrich, signal-detector,
  concept-synthesis) — never restated or reimplemented inline.
- Batches end with a `gbrain check-backlinks check` verification pass.
- Routing matches the canonical triggers in the frontmatter.
- Output written under the directories listed in `writes_to:`.
- Privacy contract preserved: no real names, no fork-specific filesystem path
  literals, no upstream-fork references.

The full behavior contract is documented in the body sections above; this
section exists for the conformance test.

## Output Format

One decision line per item checked, then the verification result:

```
Ingest gate — 3 item(s) checked

| item | entity resolution | band | action |
|---|---|---|---|
| notes-on-widget-co.md | resolved: companies/widget-co | clear-dup | linked (timeline entry on companies/widget-co) |
| pricing-thesis.md | n/a (prose) | plausible-dup | new angle — written to concepts/ with cross-link to concepts/pricing-power |
| charlie-example-intro.md | miss (near-miss: people/charlie-example) | — | read near-miss; same person → linked, no new page |

Verification: check-backlinks check → 0 gaps on admitted pages
```

Every "linked" or "duplicate" row MUST name the matched slug. If any row says
"written", the enrichment delegation (which skill handled it) should be
recoverable from the conversation.

## When it fails

Follow the [agent operator protocol](../conventions/agent-operator-protocol.md) for any gbrain error `code`, exit code, `[AGENT]` block or notice block. Specific to this skill:

- The "is this already in the brain?" search is empty with a degraded notice: an empty keyword-only result is not proof the content is missing. Check by slug, URL or exact title before importing.
- `gbrain capture` / `put` returns `write_pending` (exit 10): the content is accepted. Poll `gbrain write-request <request_id>`; do not capture it again.
- `source_binding_required` or `insufficient_scope` on an MCP write: this connection cannot write that source. Tell the user which source and scope are missing; the brain host's operator grants them.

## Anti-Patterns

- ❌ `cp file.md <brain-repo>/concepts/` — raw copy, no gate, no enrichment.
- ❌ Bulk `mv` of a folder into the brain repo, then `gbrain sync` — sync
  happily indexes every duplicate; matching-ID skip will not save you.
- ❌ Trusting a low vector score as proof a named thing has no page — named
  pages don't embed near generic descriptions of them.
- ❌ Banding on the search score without opening the top hit.
- ❌ Porting numeric dedup thresholds from other systems onto gbrain's fused
  scores.
- ❌ Writing a new named page without its `aliases:` list — the next synonym
  creates the next duplicate.
- ❌ Reimplementing entity detection, backlinking, or concept linking inline
  instead of delegating to the shipped skills.
- ❌ Skipping the gate because the write is "just one page" via `gbrain put` —
  single manual writes are where duplicate stubs come from.

## Dedup (sharp boundaries)

- **[capture](../capture/SKILL.md)** — the quick-save front door; its dedup is
  a 24h exact content-hash on identical bytes. This gate is the SEMANTIC +
  named-entity layer for content entering the brain as real pages (migrations,
  bulk imports, inbox graduation). "capture this thought" → capture; "migrate
  these files into the brain" → this gate.
- **[ingest](../ingest/SKILL.md)** — the router for NEW external content
  (meetings, articles, media) and its enrichment pipeline. ingest decides what
  to DO with content; this gate decides whether a page should EXIST at all.
  The gate fires before the write; ingest and its specialized skills handle
  everything after a "write" verdict.
- **[enrich](../enrich/SKILL.md)** — page creation/update mechanics (tiers,
  citations, timelines, backlinks) AFTER this gate says "write" or "link".
- **[concept-synthesis](../concept-synthesis/SKILL.md)** — retroactive,
  at-scale dedup of concept stubs that already slipped in. This gate is
  prevention at write time; concept-synthesis is the cleanup pass. "dedupe my
  existing concepts" → concept-synthesis.
- **frontmatter-guard (host-side)** — the same standalone-gate pattern on an
  orthogonal axis: structural validity of what's written vs (here) semantic
  novelty of whether to write.
- **[bulk-ingestion](../bulk-ingestion/SKILL.md)** — the bulk sibling. Its
  pipeline dedup key (`source + source_id`) only makes RE-RUNS idempotent; it
  does not catch cross-source duplicates or resolve named entities. This gate
  is the semantic + named-entity layer bulk-ingestion runs on its Phase 3 trial
  items and bakes into the codified pipeline (its Phase 1d/6). "Build a
  large-corpus pipeline" → bulk-ingestion; "does this page already exist before
  I write it" → this gate.
- **[data-loss-gate](../data-loss-gate/SKILL.md)** — the inverse gate: it
  stops data LEAVING the brain without confirmation; this gate stops data
  ENTERING without resolution + dedup.

More agent context in garrytan/gbrain

78 other files this repository gives its agents, the first 60 shown.