docs-bootstrap

Produce documentation whose primary consumer is a coding agent: layers linked by ids, every document carrying paths into the code, every claim checkable by machine.

[ Research — why the architecture is this and not that; verified vs hypothesis ]   optional
                              │
[ Feature — what the system does and why, + BDD scenarios = acceptance criteria ]
                              │
[ Screen / flow — what the user sees ]            only if there is a client
                              │
[ API — the contract: URL, auth tier, error codes ]
                              │
[ Service — who owns the data, config, deploy, quirks ]

Each layer answers its own question and points at its neighbour by id. Read SPEC.md before writing anything — it is the contract. example/ is a complete worked instance of it; when unsure what a filled-in document should look like, read the example rather than guessing.

The two rules everything else follows from

main describes what exists. An open pull request describes what will be.

Never document intent as fact. A feature that is designed but not shipped is status: draft and lives in an open pull request. What is not built yet is either that, or labelled target or hypothesis in the text itself.

What was verified is separated from what was assumed, explicitly.

Every status code, error string, field name and limit must be read out of the source before it is written down. A document that does not distinguish "I read this in the code" from "presumably" is worse than no document at all: it looks equally authoritative in both cases, and the next reader builds on the sand without knowing which part is sand. If you cannot find something, say the document does not cover it — an admitted gap costs a reader nothing, an invented detail costs them their trust in the whole file.

Procedure

0. Decide the scope before creating a single file

1. Layout. One product in one repository → docs/ inside it. A platform of three or more service repositories where a feature is smeared across them → a separate docs repository, and then the docs-first process of WORKFLOW.md applies. Start with docs/ inside the repository: moving out later is easy, moving back is not.
2. Greenfield or brownfield. If there is code, "verified fact" means read in this code. If there is no code yet, the things you can verify are external artefacts — the contents of a dependency's jar or klib, a package registry listing, the official documentation of the version you are actually pinning — and everything else is honestly called a decision or a hypothesis.
3. Is there a client anyone will change from the document? If not, no screens/ layer at all; how the UI is put together lives in services/<web>.md.
4. What already exists. README, the repository's agent instructions, ARCHITECTURE.md, comments in the code — these are research material, not rubble to clear. A contradiction between them and the code is a finding, and its place is the research document.

Ask the user only about what genuinely changes the layout — usually one question, inside the repository or a separate one. Everything else is a default you take yourself and say out loud in your answer.

Also ask what to cover if the scope is not obvious. Documenting everything at once produces thin documents; documenting one feature end to end across every layer produces a template for the rest.

1. Research, and write it into the file as you go

Research comes before the layer documents, and it is written while you read, not reconstructed afterwards. It is also the first thing anyone picking up a task will read. Scale it to the job: a platform is hours of reading code, one feature is a single pass.

• Every fact carries where it was verified. A | Fact | Where verified | table with a path, a class name inside an artefact, a URL. Verification addresses are what make the document re-checkable by the next person instead of merely believable.
• Your memory is not a source. Library versions, whether an API exists, how a framework behaves in a given version — this is exactly where memory is wrong most often, and exactly what an architecture decision then rests on. Unpack the artefact, list the registry, fetch the docs.
• Derive the consequence, separately. A fact on its own is inert: "JMX is resolved lazily" does nothing, while "therefore the base metric set must work without JMX" decides the architecture. The consequences are the valuable half.
• A hypothesis is called a hypothesis and gets an address — "check in M2". When that milestone closes, the hypothesis becomes a fact or a refutation in writing, not a silent deletion.
• A decision is recorded with its reason and the alternative you rejected. In six months the value is in why, not what; what is visible in the diff.
• A deviation from what you were asked for is the most valuable entry in the document. The user described X, research showed X impossible or harmful — that is a labelled deviation from the brief with its reasoning, never a quiet substitution.
• Risks come with mitigation machinery, not with a statement of concern. "Might not be fast enough" is not a risk; "UDP loss distorts the picture silently — mitigation: a sequence number per packet so the server shows gaps instead of drawing a clean graph over incomplete data" is.

Research is a living document. When the implementation diverges from it, amend the research, at the point of divergence: "this used to say take X — you cannot, because Y; the working replacement is Z". That keeps the document true and preserves why the first idea was wrong, which is what saves the next person from trying it again.

Which of the two forms you write — the permanent docs/research/research-architecture.md or the per-feature file that is deleted before its branch merges — follows from the layout you chose in step 0. They share a word and have opposite lifetimes: one says why the system is built this way and is amended as you learn, the other is the plan for one branch and is replaced by the code it asked for. SPEC.md §3.5 has both, with templates. If the project uses the second, give each service repository templates/workflow-research-guard.yaml — nothing in the documentation repository can see a file that lives in a service one.

2. Create the tree

docs/README.md      from templates/docs-readme.md
docs/research/  docs/features/  docs/screens/  docs/api/  docs/services/  docs/backlog/
docs/templates/     copy the templates in, so the format travels with the repository
backlog.md          the index page, with the BEGIN INDEX / END INDEX markers
Makefile            from templates/Makefile — the gate (step 9)
.github/workflows/check.yaml    from templates/workflow-check.yaml — CI, and the version of the checks (step 8)

Do not copy scripts/ in. This step used to say so, on the grounds that a check living elsewhere does not run. A check living elsewhere at a pinned version does; a copied one runs at the version of the day it was copied and never hears of a fix — 18 copies of one script were found across one portfolio in three versions, eleven of them without a guard that had been fixed upstream for weeks. The checks arrive at the ref the workflow pins, in CI and in a local make check alike.

Omit the layers the project does not have. A single-service tool has no screens/; a library has no api/. Do not rename them — the tooling looks for these names.

3. Write the layers bottom-up by how reliable the knowledge is

Not top-down by the diagram:

1. services/ — what the services or modules even are and who owns what. The most verifiable.
2. api/ — the contracts between them, written from the route code, because a mistake here is the most expensive one in the tree.
3. features/ — what it gives the user, plus BDD. Rests on the two layers below it.
4. screens/ — if there is a client.

Common to every layer:

• One document, one entity. A feature touching three services is one file with three entries in involved_services, not three files.
• id in the frontmatter equals the filename. Cross-layer links are ids in the frontmatter and ordinary markdown links in the body.
• Paths instead of copies, always. DTO fields, config keys, endpoint lists are not duplicated — a path is given. A pasted list is wrong within a sprint; a path stays right until the file is renamed, and there is a check for that.
• Language: the project's. Identifiers, URLs and HTTP header names verbatim as in the code.

Useful optional documents: infrastructure.md for everything that lives between services and therefore fits in none of them (environments, domains, credentials, deploy triggers), workflow.md if the process is not the obvious one. A screen built to a design does not get a design/ folder of mockups; it gets a design: block in its frontmatter (SPEC §3.2.1) pointing at the canvas and at the directory in the code where one reference PNG per state lives, with a state → stem map that the checker holds against section 1. The PNGs are code, kept next to the screenshot goldens, and the parity numbers stay in the pull request that measured them.

For each document: copy the template, fill the frontmatter, then fill the body from what you read in the code.

4. The code anchors table is not optional

Every document needs at least one table of paths into the code. This is the single thing that makes the documentation worth having for an agent, and it is the one structural rule the checker enforces. Point at the feature directory or the key file — not at a line number, which moves. Where the place in a file is the point (research citing what it read), loans.py:40-52 is checked as the file, and as missing once the file is shorter than the range (SPEC §4).

The one exception is research, which legitimately predates the code; there the checker warns instead of failing, and what you cite is the artefact you verified against — as an address inside it, with the separator a jar URL uses:

ktor-server-core-3.5.2.klib!/commonMain/io/ktor/server/engine/ShutdownHook.kt

Written as a bare path, that line is reported rotten for ever, because no tree here holds it; written this way it is reported as what it is. The left side has to name something a reader can fetch — a versioned file, a coordinate, or owner/repo — so the notation cannot be used to quiet an anchor that really has rotted. A repository is cited at the commit that was read, owner/repo@<sha> (or a version tag): an address at a branch, or at no ref, is listed as one at a ref that moves — and so is a -SNAPSHOT coordinate, or a file with no version in its name (x.jar). SPEC §4.1.

A document marked status: deprecated keeps its anchors as a record of where the behaviour was; the checker skips them and says so (SPEC §4).

A member written Type.member is resolved too: where Type is declared in the code, member has to be declared in the same file, or the reference is reported missing. That is what catches a field renamed under a document whose anchors table still resolves. A member the sentence says is gone is struck through, Footer.appTitle; another library's type of the same name leads with its package; a backlog item that is done or dropped is not looked in (SPEC §4.2).

An anchor resolves in its own repository only — the one its first segment names, else its service's, else the one the documentation lives in — so a path into another repository starts with that repository's name, or is an address when the checker will not have a clone of it. ./x is x; a path that climbs out with ../ is reported missing.

5. BDD scenarios are acceptance criteria

Write them against behaviour you have confirmed in the code: real status codes, real error strings. While the code does not exist, mark them target. A scenario covered by a test carries an **Automated:** line naming it — **Automated:** catalog-api LoanRoutesTest when several repositories are in play, or just the test when one is, in whatever form that project writes (tests/test_store.py::test_name and LoanRoutesTest.a renewal is refused are fine). A check that is a script rather than a test is named by its path, conformance/scripts/hashes.redis, or by its file name alone, negative-control.sh, and the report looks for that file as it looks for an anchor. Several tests are a comma-separated list, each in its own backticks; after a dash or a semicolon the line is commentary. The line goes under the scenario it automates — the same line in a business rule is not counted. The absence of that line means the check is manual, and that asymmetry is worth seeing.

Give every scenario its own ### Scenario: <name> heading, in English even in a document written in another language: the heading is what the tools count. The steps under it may be the template's bullets or a fenced gherkin block. Several scenarios in one block are counted as none, and the report then says 0 about a document full of them (SPEC.md §3.1).

6. Quirks sections are the highest-value content

An empty logout handler, a hard-coded test domain, a fire-and-forget sync, state kept in memory that no restart survives, a config key nothing reads — write these down. They are why someone opens the file at two in the morning. Do not quietly delete one when it is supposedly fixed; delete it after verifying the fix.

7. Backlog

Research without "what to do next" is an essay, not a working document. Size the backlog to the project:

• Up to about thirty items in one repository → BACKLOG.md at the root, items M-NN grouped under milestones M0…MN as checkboxes. A milestone closes as a whole and gets a summary line: what came out beyond the plan, and which research hypothesis was confirmed or refuted.
• Dozens of items, or several repositories → one file per item in docs/backlog/, flat, plus the generated index in backlog.md.

Either way: stage, priority and status are frontmatter fields, not directories, because documents cite items by id and re-prioritising must not break a link. There is no separate epic entity; epic: feature-<name> is the grouping.

The tables in backlog.md between the markers are generated. Edit the item, run the generator, commit both.

8. Wire the documentation into the repository

Without this step the documents exist and nobody opens them.

1. The repository's agent instructions (CLAUDE.md or equivalent) get a "how to start a session" section: research → backlog → the layer document the task belongs to. This is the single highest-leverage line in the whole exercise.
2. The product README links to docs/ in one line.

3. CI — copy templates/workflow-check.yaml to .github/workflows/check.yaml, unchanged, together with the Makefile of step 9. Do not write one from memory: several of the decisions in it are not the obvious ones, and each was paid for.

   • The version of the checks is written once, in its uses: youndie/docs-bootstrap@<tag> lines. CI runs the checks at that ref; the Makefile reads the same line and fetches the same ref for a local run. A version pinned in the workflow and again anywhere else is two pins, and two pins drift. Renovate's github-actions manager bumps the line, so a fix to the checks arrives as a pull request instead of never.
   • The action runs make check — the same target a contributor runs. A local set that differs from the CI set turns "green here, red there" into the normal state of affairs, and then neither is read. Checks of the project's own therefore go into the Makefile, not into the workflow. Before anything runs, the action asks the Makefile where its checks come from, so a pinned uses: over a Makefile that still runs copies fails instead of passing under a version it does not run.
   • No path filters. They save seconds and buy red default branches: a generated file diverges from its sources when something outside the listed paths moves, and a new directory is a list nobody remembers to update.
   • Two checks are branch-specific, and the action runs them around make check. status: draft is an error only on the default branch, because in a pull request it is the normal state. The other compares backlog item numbers with the base branch, and how much it buys depends on a decision made elsewhere: if every branch commits the regenerated index, two branches adding items always conflict in that one file, and the conflict — not this check — is what stops them. It earns its keep when the index is rebuilt on the default branch instead. Either way it runs before make check, because both catch the collision and only one of them says which branch took the number.
   • A pull request that cannot be merged runs no pull_request workflows at all. Nothing is reported and nothing is red; the checks are simply absent, exactly when the branches have diverged most. Anything gated on that event is a backstop rather than a guarantee.
   • Anchors run on a schedule, not on a pull request, and do not block. They rot because somebody refactored a different repository, so a pull request here is the least likely moment for one to break — and a red build nobody caused is a build people learn to ignore.

9. Run the checks and fix what they find

Copy templates/Makefile to the root of the repository — not this repository's own Makefile, which checks docs-bootstrap itself: copied into a project it prints "no docs tree — nothing checked" and goes green having checked nothing. Set the three lines that are the project's: DOCS and BACKLOG if the tree is not at docs/ and backlog.md, and BACKLOG_FORM — files, milestones or none, the answer of step 7.

pip install pyyaml
make check      # the gate and the reports: exactly what CI runs
make fix        # regenerate the backlog index, append missing coverage-map lines

The first run fetches the checks at the ref the workflow pins into .docs-bootstrap/, which ignores itself; make check then runs the same scripts CI runs. The gate opens with a guard that fails when its subject is not there — no docs/, or BACKLOG_FORM=files and no docs/backlog/B-*.md — because every script treats a missing tree as nothing to check, and a gate that cannot find its subject must not report success. The line it prints names the version of the checks that ran.

Only the goals that run the checks load them: check, gate, report and fix — the list in DOCS_BOOTSTRAP_GOALS — and check.mk's own docs- targets. The project's other targets below the template's head — a chart, a stand, a release — and make with no goal run without reading the pin and without the network: make fetches an included file before it runs any goal, so a Makefile that always included check.mk made every one of them download it on a fresh clone and fail offline. A goal of the project's own that leads to the checks (ci: check build) is added to DOCS_BOOTSTRAP_GOALS above the line that says nothing below is to be edited; left out, it stops with a message naming that variable. A Makefile copied from an earlier revision of the template keeps working, and check.mk warns that it is one; it is moved like a Makefile of copies in the section below — the new template, with the project's own lines carried over.

Checks of the project's own go under gate: in the Makefile, where CI runs them too. DOCS_BOOTSTRAP=<dir> make check runs the checks from a directory instead — a clone of docs-bootstrap being changed, or, offline or without GitHub Actions, a committed copy of its check.mk, scripts/ and .claude-plugin/. That is the copy route again, with its drift: the fallback, not the default.

The two reports are non-blocking by default. Demanding a percentage of automated scenarios is meaningless while acceptance is manual, and an anchor goes stale because of a refactor in somebody else's repository, not because of an edit here — a machine cannot tell a live path from one quoted as obsolete.

The anchors report can block, and once it can, it should. When it lists nothing and every path outside the repository is an address (SPEC §4.1) — inside an artefact, or owner/repo@<commit>!/… — no refactor elsewhere can move a path, and what can turn it red is a path of the repository's own, renamed or deleted without its document. Set ANCHORS_ARGS ?= --check in the Makefile, above the line that says nothing below is to be edited (the template carries it commented out): make check then fails on every pull request that breaks a path, and on a laptop, not only in the weekly job. A path quoted as obsolete is written as an address at a commit it existed in. make check ANCHORS_ARGS= or make report ANCHORS_ARGS= runs it as a report again for one run — while a neighbour's path is being rewritten as an address, say.

make fix regenerates the index and appends the coverage-map lines you forgot; the descriptions it writes are placeholders, and finishing them is yours.

When the repository already carries copies of the checks

Repositories documented before 0.3.0 copied scripts/ and a Makefile in. Moving one to the pinned checks, in one pull request:

• scripts/backlog_index.py, docs_check.py, coverage_map.py, bdd_report.py, code_anchors.py → delete them. A script of the project's own stays, and its line moves under gate: in the new Makefile.
• The Makefile → templates/Makefile. Carry over DOCS, BACKLOG, REPOS and the project's own gate lines; set BACKLOG_FORM; a target of the project's own that runs check, gate, report or fix goes into DOCS_BOOTSTRAP_GOALS. A hand-written guard (test -d docs, a count of docs/backlog/B-*.md) goes — docs-guard in check.mk does that now.
• .github/workflows/check.yaml → the steps setup-python, pip install pyyaml, the backlog_index.py --against step, make check and the docs_check.py --on-main step become one uses: youndie/docs-bootstrap@<tag>, the tag in templates/workflow-check.yaml; the anchors job's steps become the same line with target: report. Steps that are not the documentation gate — a formatter, a build — stay as they are.
• docs/templates/ → leave it, or refresh it from templates/: the copies of 0.3.0 and earlier tell an author to run python3 scripts/backlog_index.py, which is gone once the copies are; from 0.3.1 they say make fix and make check.
• Renovate needs nothing when the repository extends a preset with the github-actions manager on, which is the default.

Then make check locally and the check job in CI print the same docs-bootstrap <version> line.

When the documentation already exists

Do not rewrite it. Report the discrepancy between what a document claims and what the code does, and let the person who owns the meaning decide. The main invariant only holds while that is true; an agent that silently overwrites hand-written documents breaks it in one commit.

Extend instead: add the missing layer, add the anchors table to a document that has none, add the quirk you just discovered.

What not to do

• Do not put research off until after the structure is in place. Structure without research is empty files with the right names, and they begin lying by existing.
• Do not retell the code. If a paragraph can be replaced by a path, replace it.
• Do not enforce the section structure of the templates. Documents legitimately deviate — a feature that is mostly a reality check has no business-rules section. Check the substance instead: is there a path into the code?
• Do not treat an empty list as a mistake. client_entries: [] is the answer "this feature has no client surface".
• Do not invent a layer that is not there. No client, no screens/; one service, and services/ may be a single file or a section of the README.
• Do not write a number you have not seen. Measured figures name what was measured and how; everything else is a hypothesis and says so.