site
Skill0 starsChanged 4 months agoMITView on GitHub
Generate the static HTML site for a research project. Creates source/site/build_all.py and templates if they don't exist, then builds the site. Use when the user wants to create or rebuild a project site.
- Installs packages
- Commits and pushes
6 other projects use a version of this.
Adopted by
Most recently taken up by
- guiloureiromkt/ultra-skillstheir own version, first seen Oct 21, 2026
- grechj/Australian-Builders-Braintheir own version, first seen Oct 17, 2026
- 8bsite/8b-agent-plugintheir own version, first seen Oct 11, 2026
- okharedia/my-skillstheir own version, first seen Sep 5, 2026
- JustOneMoreTry/NoMoreHigstheir own version, first seen Aug 4, 2026
- anp2dev/anp2their own version, first seen May 24, 2026
Other versions6 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 6
- 8bsite/8b-agent-plugin★ 2same name
- anp2dev/anp2★ 2same name
- grechj/Australian-Builders-Brain★ 0same name
- guiloureiromkt/ultra-skills★ 0same name
- JustOneMoreTry/NoMoreHigs★ 0same name
- okharedia/my-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository39 other files it gives its agents.
- anecdotesSkill
- backupSkill
- checkSkill
- cite-syncSkill
- dataSkill
- diariosSkill
- distill-styleSkill
- driveSkill
- dropboxSkill
- fetch-annotationsSkill
- findings-auditSkill
- findingsSkill
- gmailSkill
- handoffSkill
- howtoSkill
- hypothesisSkill
- iatSkill
- ideaSkill
- inboxSkill
- institutionsSkill
- literatureSkill
- llmkitSkill
- meetSkill
- memory-auditSkill
- new-pipelineSkill
- new-projectSkill
- nextSkill
- otterSkill
- outlookSkill
- questionsSkill
- referee-panelSkill
- style-checkSkill
- style-reviewSkill
- style-reviseSkill
- tactiqSkill
- theorySkill
- validate-sectionSkill
- whatsappSkill
- zoomSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
20 rules. 0 are common, 8 are distinctive, and 12 are very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
8
Distinctive
This author's own choices. Worth a look.
12
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 9
Create and build a static HTML site for a research project.
- Distinctive: line 110
Read the target's CLAUDE.md to get the title and short description.
- Very specific: line 133
If source/site/build_all.py already exists:
- Distinctive: line 134
Ask the user if they want to rebuild only (run the existing script) or regenerate the scaffold.
- Distinctive: line 135
If rebuild only: skip to step 5.
- Distinctive: line 139
Create the following files using the canonical templates below.
- Distinctive: line 155
When creating the scaffold, customize these project-specific values:
- Very specific: line 178
Only include files that actually exist in the project's docs/ directory.
- Very specific: line 179
If there are .md files not in this table, add them with a sensible title and the "Reference" category.
- Very specific: line 228
When the project needs an extra section that doesn't generalize (e.g.
- Distinctive: line 316
All Documentation) — they differ only in what middle sections appear on the landing page and what auxiliary templates exist.
- Very specific: line 326
Theoretical: if the project's paper/ is a formal model (propositions, theorems, proofs), the docs include propositions.md / holdings.md / …
- Very specific: line 329
Empirical: if the project has descriptive figures/tables backed by data pipelines, source/figure/ and source/table/ scripts, and a summary/ …
- Very specific: line 338
build_all.py: update PROJECT_TITLE, PAPER_TITLE, DOC_REGISTRY, and the .tex filenames if they differ. - Very specific: line 348
Cite-ref machinery: if the project has (or will have) a docs/literature.md or docs/literature/index.md, the build must turn [cite:<bibkey>] …
- Very specific: line 370
Templates: update <title> tags and any paper/talk <h1> to use the new project's title.
- Distinctive: line 381
Every site must preserve these — they are what makes the projects look like one cohesive workspace.
- Very specific: line 493
Source-script pages: every .py / .R / .sh / .sql under source/ is rendered to its own HTML page at build/site/source/<subpath>.html with …
- Very specific: line 511
Grain-verification helper: when a DatasetConfig.key_columns field is declared (e.g.
- Very specific: line 608
Check that build/site/index.html exists and list the generated files.
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Project Site Generator
Create and build a static HTML site for a research project.
Status of the canonical implementation
The rendering machinery lives in the sitekit package under
research-kit/sitekit/. Each project's source/site/build_all.py is a
thin shim that hands a SiteConfig to sitekit.build_site().
Migrated to sitekit: audit, campaign-finance, deterrence,
electoral-justice, fisc, judgeGPT, poll-sponsor-bias, promotor,
serasa, vague, lawsuit, scheme, segredo, connect (minimal +
empirical), and bind (theoretical). For migrated projects, verification is
content-parity of each rendered doc body against the pre-migration output plus
intended convergence of the chrome onto the shared design system — NOT
byte-equivalence (that only held for serasa, whose templates were the
extraction source; other projects had drifted chrome the migration deliberately
normalizes). The standard acceptance check the recent migrations used: page-set
diff (0 lost), body-text parity (~1.0; sub-1.0 is pure CSS-var normalization),
and "new broken-link set ⊆ baseline broken-link set" (the migration introduces
no broken links; pre-existing stale source refs stay pre-existing).
Not yet migrated — each is a DEDICATED effort blocked on a specific host or build, NOT a batch slot (verified by attempted migrations 2026-08-31, which correctly refused to force a lossy/unverifiable rewrite of coauthor-facing sites):
procure— coauthor-facing + validation-gated; ~2,500 loc of bespoke subsystems to port as hooks: a paper page with validation-ledger UI + number-attribution tooltips + feedback-worker (the blocker — 850 loc of bespoke UI that can't be built/verified without a TeX host), a dataset portal (build/summary/*.json, different schema than the empirical archetype), a 1,111-locbuild_cases.pycases renderer, validation + election-RD pages, and a chip-filtered AN index. ~40% maps to stock. Needs a TeX-capable host. Deploys plaintext-public → run a PII/sensitive compliance audit before any redeploy (sitekit build newly copiesbuild/figure+build/analysisPNGs and stops shippingmain.pdf).ficha— empirical, 476 pages (231 script pages dominate), coauthor-facing. Needs a TeX host: its bespoke paper page (inline-footnote reconstruction,\newcommandmacro substitution, side-rail TOC, figure-path rewrite) has no cachedbuild/make4ht/paper.htmlin-sandbox, so it can't be diffed — don't port it blind. Also renders everything FLAT at site root with a bespoke autolink layer (an_file_map,link_h_refs,autolink_build_artifacts,rewrite_build_artifact_paths), so flat→nested is a full link-layer reimplementation (verify by new-broken⊆baseline), not a config migration. 4 bespoke builders to port as hooks (copy_build_artifactscopies PDF+PNG+tables tofigure/+table/,autolink_build_artifacts, ~300-linebuild_key_artifacts_page, note asset-folder copies) — the artifact copy needs a pre-docs hook (copy_site_figuresis pre-docs butconfig.hooksare post-docs); addSiteConfig.pre_docs_hookswith ficha as its first consumer (don't land it unexercised). Verifiable-in-sandbox parts: docs, 82 analyses, hypotheses, findings, subdirs, 231 script pages, 2 dataset pages, key_artifacts. Fold in a check of the possibly-staledocs/reference/analysis-index.yaml.saude— empirical; ~30% maps to stock (top-level docs + stock cached paper). LIVE bespoke subsystems needing hooks: a news timeline (references/news/stories.csv→news.html) and an 18-source references index (references/→references.html); notebooklm is dead (droppable). Dataset portal reads an empty summary cache in-sandbox → needs a data host to regenerate the cache and verify dataset pages.rolis intentionally NOT migrated (decision 2026-08-31, Henrik): it's not a docs site but a bespoke claim-graph knowledge base (14 typed node types, strand cross-cut indexes, an interactive graph explorer + generatedsite-data.js, a linear book). Its deploy is already on the sharedsite_deploy.sh, and a graph archetype would be single-consumer code in shared sitekit, so rol stays on its ownbuild_all.py. Do not force it ontominimal(catastrophic content loss) and do not re-open this without a second graph consumer to justify the archetype.
For unmigrated projects the old archetype-reference workflow is still
authoritative: copy build_all.py + templates/ from the matching
canonical reference and customize. For new projects, prefer the sitekit
path (see "Using sitekit" below).
The empirical archetype (fisc, poll-sponsor-bias, connect's docs-heavy
variant) and the theoretical archetype (bind — cases discovery +
per-case pages via a project cases.html, grouped cases index,
extra_tex_pages for holdings-style companion LaTeX, plus a citations hook)
are both real and exercised, as are the AN-page / cite-ref / anecdote /
findings / hypotheses / script-page passes. Still a stub: any graph
archetype (rol). sitekit also gained SiteConfig.exclude_stems (keep
specific subdir notes off a PUBLIC site) during this work.
Arguments
/site-- create site scaffold + build for the current project/site [project-slug]-- target a specific project
What the site contains
- Index page -- landing page with paper/talk hero cards and docs grouped by category
- Doc pages -- each
docs/*.mdfile rendered as HTML with a table of contents linking to h2/h3 sections, image lightbox, anchor linking, Hypothes.is annotation layer - Paper page -- LaTeX paper converted to HTML via make4ht, with inline footnote tooltips, MathJax, Hypothes.is annotation layer
- Talk page -- beamer slides converted to HTML via make4ht
Step-by-step
1. Locate project
Find workspace root by searching upward for CLAUDE.md next to projects/ and pipelines/.
Resolve the target: $ROOT/projects/{slug}/ or $ROOT/pipelines/{slug}/ — a slug can
name either. If /site foo is given, check both trees; pipelines are as valid a target as
projects.
Read the target's CLAUDE.md to get the title and short description.
Pipeline repos differ from project repos in three ways — mind them throughout:
- Deploy plumbing. Pipeline
build.shsources the sharedresearch-kit/tools/site_deploy.sh(providingsk_encrypt_site/sk_deploy_site) and setsSITE_TITLE+ aSITE_GATEDtoggle, rather than inlining the staticrypt/rsync logic. Public data-reconstruction pipelines setSITE_GATED=0(plaintext, e.g.lovhistorie); pipelines documenting access to sensitive/geoblocked raw data setSITE_GATED=1(default — gated, e.g.govspend). Model a new pipelinebuild.shonpipelines/lovhistorie/build.sh(public) orpipelines/govspend/build.sh(gated), NOT onserasa/build.sh. source/may not be a package. Pipelines often run scrape/clean scripts directly, sosource/__init__.pycan be absent;python3 -m source.site.build_allneeds it. Create emptysource/__init__.pyandsource/site/__init__.pyif missing.- Usual archetype is
minimal(docs-only) orempirical(data portal) — a pipeline rarely has apaper/, so setpaper_title=""and apaper_placeholder_msgthat says "this is a pipeline".lovhistorieis the canonical minimal pipeline;govspendthe canonical empirical pipeline (source cards over raw + clean tables — see "Empirical data portal for a pipeline" below).
2. Check if site already exists
If source/site/build_all.py already exists:
- Ask the user if they want to rebuild only (run the existing script) or regenerate the scaffold.
- If rebuild only: skip to step 5.
3. Create site scaffold
Create the following files using the canonical templates below.
Directory structure
source/site/
build_all.py # main generator script
templates/
index.html # landing page
doc.html # docs page (markdown rendered)
paper.html # paper page (make4ht content)
talk.html # talk/slides page (make4ht content)Customization points
When creating the scaffold, customize these project-specific values:
PROJECT_TITLE-- short title for the nav brand (e.g., "Causal Judge", "Corruption Networks"). Read from project'sCLAUDE.mdfirst line heading ordocs/summary.md.PAPER_TITLE-- full paper title for paper/talk page headers. Read frompaper/main.texorpaper/paper.tex(\title{...}), or fall back todocs/summary.mdheading.DOC_REGISTRY-- list of (path, title, description, category) tuples. Scandocs/for existing.mdfiles and register them using the standard mapping:
| File | Title | Category |
| summary.md | Research Summary | Reference |
| institutions.md | Institutional Background | Reference |
| data.md | Data Sources | Reference |
| methods.md | Methods | Reference |
| literature.md | Literature | Reference |
| thinking.md | Open Questions & Ideas | Working notes |
| decisions.md | Key Decisions | Working notes |
| outline.md | Paper Outline | Working notes |
| hypotheses.md | Hypotheses | Working notes |
| desiderata.md | Desiderata | Working notes |
| todo.md | Active Tasks | Tasks |
| done.md | Completed Tasks | Tasks |
| meetings.md | Meeting Notes | Communication |
| feedback.md | External Feedback | Communication |
Only include files that actually exist in the project's docs/ directory.
If there are .md files not in this table, add them with a sensible title and the "Reference" category.
4. Write the files
Using sitekit (preferred path for new projects and serasa-style minimal projects)
Two files in source/site/:
# source/site/build_all.py — thin shim
import sys
from pathlib import Path
_SITEKIT = (
Path(__file__).resolve().parent.parent.parent.parent.parent
/ "research-kit" / "sitekit"
)
if str(_SITEKIT) not in sys.path:
sys.path.insert(0, str(_SITEKIT))
from sitekit import build_site
from source.site.site import config
if __name__ == "__main__":
raise SystemExit(build_site(config))# source/site/site.py — config + project-local hooks
from pathlib import Path
from sitekit import SiteConfig
config = SiteConfig(
project_root=Path(__file__).parent.parent.parent,
project_title="…",
paper_title="…",
archetype="minimal", # or "empirical" / "theoretical" / "mixed"
paper_tex="paper.tex",
talk_tex="talk.tex",
doc_registry=[
("docs/summary.md", "Research Summary", "...", "Reference"),
# ...
],
)The sitekit package bundles default templates/. A project can override any
one by placing a same-named file under its source/site/templates/: project
files take precedence over package files.
When the project needs an extra section that doesn't generalize (e.g.
serasa's specs-for-Ramon dropdown, BCB results gallery), write it as a hook
callable in site.py and add it to config.hooks. The hook receives a
BuildContext and may return {"extra_cards": "<html>"} to inject content
above the index's standard doc-group grid.
On educloud the system venv is read-only, so the sys.path shim in
build_all.py replaces pip install -e. On a writable env you can
substitute pip install -e ../../research-kit/sitekit and drop the shim.
Empirical data portal (archetype="empirical") — extra wiring
The data portal (source-card landing grid → per-source pages → per-table pages
with columns / example rows / histograms) needs more than the site.py shim.
Gotchas, learned building govspend (canonical empirical pipeline):
- sitekit does NOT bundle
dataset.html,source.html, orindex.html. The empirical archetype raisesTemplateNotFoundError: dataset.htmlunless the project supplies them undersource/site/templates/. Copy all three fromprojects/fisc/source/site/templates/(the canonical empirical set), then edit onlyindex.html's<title>andsummary-boxfor the new project. The bundledindex.htmlhas NO<!-- INJECT_SOURCE_CARDS -->marker, so without a copiedindex.htmlthe landing page silently omits the source cards even though the source/dataset pages build fine — always verify the index actually linkssources/*.htmlafter building. - Two-stage build.
source/summary/{config,compute,build_all}.pycomputes a per-dataset JSON cache (source/summary/cache/*.json, committed to git so the site rebuilds without raw data); the site reads that cache.build.shneeds asummarymode (python3 -m source.summary.build_all [--force|--only <id>]) separate fromsite. Modelconfig.py/compute.pyonprojects/fisc/source/summary/. config.pycontract:SOURCES(list ofSourceConfig(id, name, description, categories)) +DATASETS(list ofDatasetConfig, each with acategoryin exactly one source'scategories) + aCACHE_DIR. Source group pages only build if every source has non-emptycategories.- Big / heterogeneous data (pipeline-scale): fisc's
compute.pydoes a singleread_parquet— fine for projects, not for a pipeline with 500M-row partitioned dirs or JSONL raw.govspend'scompute.pyis the reference for: exactrow_countfrom Parquet footer metadata (no data read) + a stratified sample of parts spread across the partition list (NOT the first N — a{uf}-{year}layout sampled first-N yields an all-Amazonas histogram); skip 0-row parts (BigQuery exports leave padding files that zero out stats); a JSONL reader; and CPF masking ([CPF]for CPF-shaped values + amask_columnslist for person-name columns) — do this even on a gated site. Firm CNPJs/names stay visible; individual CPFs never surface. Note CEIS/CNEP-style public naming registries are the exception — leave party names visible, mask only CPFs. Record astats_scopenote (which/how many parts scanned). - Full-text corpora (gazettes, court text, etc.) OOM a naive read. A
textcolumn of multi-KB documents times the scanned rows is gigabytes — a whole-partread_parquetgets the process silently OOM-killed (looks like an empty exit when piped throughgrep; symptom: only the first 1–2 caches written, no error).diario-municipal'scompute.pyis the reference: read a row-bounded first batch viapq.ParquetFile(f).iter_batches(batch_size=…)(a ~20k-row budget spread across the stratified scan parts) instead of loading whole parts, and truncate sample-row cells (~240 chars) so a dataset page doesn't embed megabytes of raw act text. When a build "completes" with a near-empty cache and no error, suspect OOM on atextcolumn — run the build in the foreground (unpiped) to see the real output. - Raw vs clean pages: show a raw table page only where the clean layer drops
fields the raw carries (or the raw has no clean equivalent); otherwise the clean
table is the faithful view. Channels whose raw is ZIP/nested dumps (not cleanly
tabular) are represented by their clean tables with a feeder note in
source_notes. - Turn off
build_descriptives/build_tablesinSiteConfigfor a pipeline with no paper-stylebuild/figure+build/table; the data portal is the content. A stray aggregatebuild/figure/*.pngstill gets copied intobuild/site/figures/(unencrypted on a gated site) — confirm it carries no PII or drop it. - Coverage checkerboard (high-value landing hero). Both
govspendanddiario-municipalship asource/summary/coverage.py(writes a committedcoverage.json: a UF × data-type × year cube) +source/figure/ coverage_matrix.py(small-multiples — each cell a mini year-histogram on a fixed shared x-axis, bar height ∝ √count normalized within column, tri-state green/amber/blank color), embedded on the index via an<img>the copiedindex.htmltemplate references. Copy both. Compute the cube cheaply/exactly: UF and year from{uf}-{year}partition filenames + footers where possible, else read just the uf + year/date columns; derive UF fromibge7(first two digits = IBGE state code) when there's no uf column. Pick the amber="partial" semantics per pipeline: govspend uses it for platform-limited/incomplete columns (BLL, aditivos), diario for recall-limited derived extractions vs complete corpus text. Name capability columns honestly ("Bids · incl. losers", not "full rolls") and verify surprising ones against the data before shipping.
Using the legacy fork-and-customize pattern (for archetype-rich projects until they migrate)
Pick the canonical reference that matches the project's archetype, then copy
its build_all.py and templates/ as a starting point. Both archetypes share
the same design system (palette, typography, nav, summary-box, Reading Guide,
All Documentation) — they differ only in what middle sections appear on the
landing page and what auxiliary templates exist.
| Archetype | Canonical reference | What's distinctive |
| Theoretical | /home/henrik/research/projects/bind/source/site/ | Cases section + per-case pages; Results at a Glance propositions table; briefs |
| Empirical | /home/henrik/research/projects/fisc/source/site/ | Data Sources grid + per-source / per-dataset pages; Descriptives + Tables pages |
How to tell which archetype:
- Theoretical if the project's
paper/is a formal model (propositions, theorems, proofs), the docs includepropositions.md/holdings.md/ case extractions, and there's nodata_catalog/-backed parquet dataset list. - Empirical if the project has descriptive figures/tables backed by data
pipelines,
source/figure/andsource/table/scripts, and asummary/cache of dataset metadata. - Mixed projects exist (e.g. an empirical paper with a small theory appendix). Pick the archetype that matches the primary output and add the secondary archetype's sections on top of it.
Then customize what you copied:
build_all.py: updatePROJECT_TITLE,PAPER_TITLE,DOC_REGISTRY, and the.texfilenames if they differ. Drop archetype-specific scaffolding the new project doesn't need (e.g.AREA_MAP/TOPIC_MAP/cases.htmlif you copied bind into an empirical project). Also make sure it: wipesSITE_DIRat the start of each build (so renamed/removed sources don't linger as stale files); does not copypaper.pdf/talk.pdfinto the site (staticrypt encrypts only HTML — a shipped PDF would be readable by direct URL; the make4ht HTML render is the paper page); and contains no deployment logic of its own (noDEPLOY_DIR, no rsync to~/hsigstad.github.io— deployment isbuild.sh's job, see step 6).- Cite-ref machinery: if the project has (or will have) a
docs/literature.mdordocs/literature/index.md, the build must turn[cite:<bibkey>]tokens into hyperlinks perresearch/rules/citations.md. The canonical implementation lives in/home/henrik/research/projects/connect/source/site/build_all.py— copy these helpers verbatim:_load_cite_map()+CITE_MAP(papers with their owndocs/literature/<key>.mdpage)_load_bib_authoryear()+BIB_AUTHORYEAR(author-year labels parsed frompaper/*.bib)_load_index_cite_map()+INDEX_CITE_MAP(papers that appear only as[cite:<key>]-tagged bullets on the literature index)_link_cite_refs(html, current_stem)— resolution order: page → index anchor → literal token_inject_index_cite_anchors(html)— addsid="cite-<key>"to<li>elements on the literature/index page Then wire_link_cite_refsinto the render pipeline (alongside the existing_link_an_refs/_link_h_refs/_link_anec_refspasses) and run_inject_index_cite_anchorsonly when renderingdocs/literature/index.md. The CSS rule.md-body li:target { background: #fff8c4; scroll-margin-top: 4rem; }intemplates/doc.htmlhighlights the bullet you jumped to.
- Templates: update
<title>tags and any paper/talk<h1>to use the new project's title. Keep all CSS in:rootand surface colors as-is — they are part of the contract below.
The canonical projects are the single source of truth for the templates; do not inline them in this skill. If you find a fully built site that diverges from the cream palette / nav style / summary-box+Reading Guide skeleton, that's drift — bring it back in line with the design-system invariants below.
Design system invariants
Every site must preserve these — they are what makes the projects look like one cohesive workspace. If you copy from any project that lacks them, fix it to match.
Color tokens (in every template's :root):
:root {
--bg: #faf9f6; /* warm cream background */
--card: #fffefa; /* slightly lighter cream for cards/dropdowns/page-headers */
--fg: #374151; /* soft near-black text */
--muted: #6b7280; /* secondary text */
--border: #e5e2db; /* warm cream border */
--accent: #2563eb; /* blue links / left-borders / primary actions */
--accent2: #198754; /* secondary green accent (rarely used) */
}Surface colors that pair with the cream palette (use these instead of
cool-gray #f8f9fa / #f0f4f8 / #eef1f5 / #fafbfc):
- Code-inline background:
#f0ede6 - Code-block / blockquote / table-header background:
#f5f3ee - Even-row striping:
var(--bg)(#faf9f6) - Hover surfaces (table rows, dropdown items):
#f5f3ee(light) or#f0ede6(slightly stronger)
Body text uses Georgia/Times serif at 1.06rem / line-height 1.78 on
.md-body / paper / brief / case pages; UI chrome (nav, headings, page-header,
table headers) stays on the system sans-serif.
Sticky top nav is background: #212529 with white brand on the left and
muted (#adb5bd) link text that brightens on hover. Active link is full white.
Landing page (index.html) structure — every project's landing page
follows this top-to-bottom shape:
<div class="summary-box">— one paragraph executive summary, with a blue 4px left border. Lives just inside<main>.<h2>Reading Guide</h2>followed by<div class="guide-grid">— a 2-column grid ofguide-cardlinks, each with a small uppercase priority label above the title (priority-startred,priority-mainblue,priority-refgray) and a one-line description below. The first 1–2 cards are the "start here" entry points (paper, talk, lead brief).- Archetype-specific sections — see below. Reuse
summary-box/ table / card vocabulary; never invent a new card style per project. <h2>All Documentation</h2>followed by<div class="doc-groups">— doc cards grouped by category, each group with a colored 3px top border.
Page width is max-width: 900px for the index, 42rem for doc/brief/case
pages. The body font is .88rem on the index, 1.06rem (Georgia serif) on
content pages.
Archetype-specific sections
The middle of the landing page (step 3 above) and the set of auxiliary
templates differ by archetype. Match these to whatever copy of build_all.py
you started from.
Theoretical projects (e.g. bind) typically include:
- Results at a Glance — one or two
<table class="results-table">blocks listing propositions/corollaries with a one-line "what it says" column and a<span class="badge badge-proved">/badge-draftstatus pill. Source: bindindex.htmllines 142–173. - Decisions Needed —
<ol class="decision-list">with each<li>styled as an amber-bordered card auto-numberedD1, D2, …via CSS counters. - Cases (or analogous content section): a separate
cases/index.htmlpage grouped by area/topic, plus per-case pages built fromcases/extractions/*.md. Add a "Cases" dropdown to the nav. - Auxiliary templates:
cases.html(grouped grid), and a paper template that handles multiple.texdocuments (paper + holdings, etc.).
Empirical projects (e.g. fisc) typically include:
- Data Sources —
<div class="data-grid">ofdata-cardlinks, one per source group, with adata-metarow-count badge ('533K','9.8M'). Each card links tosources/{id}.html. - Descriptives / Tables doc pages — auto-generated from
build/figure/PDFs (converted to PNG) andbuild/table/*.mdfiles; the build script appends them toDOC_REGISTRYunder a "Descriptives" category. Auxiliary templates:
dataset.html(Chart.js histograms + column-info table + sample-rows table),source.html(group overview with category rollups), and optional per-source narrative templates undertemplates/sources/{id}.htmlfor groups that have no parquet backing (e.g. a docs-only "diários" page). Thedataset.htmltemplate includes a Binary columns card above the categorical block: one horizontal bar per column showing the share of "true" (1 / "S" / "Y"), with NA% annotated per row. Binary columns are auto-detected bysource/summary/compute.py::detect_binaryfrom bool dtype, integer ⊆ {0,1,NA}, and string ⊆ {"S","N"} / {"Y","N"} / etc. — no manual registration. Detected binaries are excluded from the per-column value-count chart to avoid double-display.Canonical source of the binary helper: snippets live in
snippets/next to this skill —snippets/detect_binary.pyfor the Python detection logic andsnippets/binary_chart_block.htmlfor the JS chart block. When scaffolding a new project (or refreshing one whosecompute.py/dataset.htmlpredate the binary feature), paste from these files rather than copying from a random project. When updating the helper, edit the snippet first and propagate to projects.Example-value helper: the column-info table on each dataset page has an "Example" column showing one sampled non-null value per column. Canonical helper:
snippets/example_value.py. Adds anexamplefield to each entry in thecolumnslist of the cache JSON; the template renders it as the rightmost cell in the column-info table.Source-script pages: every
.py/.R/.sh/.sqlundersource/is rendered to its own HTML page atbuild/site/source/<subpath>.htmlwith per-line anchors (#L42). The dataset page's "Source:" line links to it (project-owned scripts only — upstreampipelines/*references stay plain text). Canonical helper:snippets/source_pages.py(functions forfind_source_scripts,script_out_name,build_source_pages, plus the per-line gutter CSS fordoc.html).Pseudocode block: assembled datasets get an optional
DatasetConfig.pseudocodefield — a short, hand-written summary of how the dataset is built, with input-dataset references in[brackets]that the template rewrites to hyperlinks. Renders as a<pre>block near the top of the dataset page. Canonical helper:snippets/pseudocode_block.md. Lets the long description / notes paragraphs come down — the construction logic lives in one place where every input is clickable.Grain-verification helper: when a
DatasetConfig.key_columnsfield is declared (e.g.key_columns=["cpf", "npu"]), the dataset page renders a tri-state badge in the page header — green ✓ "Unique on (...)" when the key holds, yellow "!" "Unique but null keys on (...)" when there are no duplicates but some rows have null values in a key column (typical for left-joined panels), red ✗ "Grain broken on (...)" when duplicate rows exist. Canonical pair:snippets/verify_grain.py(Python helper that emits agrain_checkblock in the cache JSON) andsnippets/grain_check_badge.html(the page-header JS block). The point is end-to-end data-quality observable on the page itself: if a future change to the assemble script silently breaks grain, the badge flips on the next cache rebuild — no separate ledger entry to maintain.- Talk page (
talk.html) — empirical projects usually have a beamer talk alongside the paper; theoretical ones often don't.
Mixed: layer the secondary archetype's sections on top of the primary. E.g. an empirical paper with theory: start from fisc, add a Results at a Glance table for the theoretical results above the Data Sources section.
5. Build the site
Run:
cd $PROJECT_ROOT && python3 -m source.site.build_allReport what was generated (number of doc pages, whether paper/talk were built).
6. Deploy: staticrypt-encrypted, to the project's gh-pages branch
This is the canonical deployment for every project: the site is
staticrypt-encrypted and pushed to the project's own gh-pages branch. One
self-contained command, a readable repo-name URL
(https://hsigstad.github.io/{repo}/), and a password gate so the private
research content isn't readable by anyone who stumbles on the URL. There is no
separate plaintext push or ~/hsigstad.github.io/{slug}/ personal-website
step — that legacy approach is retired.
The reference implementation is /home/henrik/research/projects/serasa/build.sh
(self-contained, project style). Port it (adjusting PROJECT_TITLE and the
paper/talk .tex names) to any project that doesn't have it yet — including
projects still on the old plaintext gh-pages push or the legacy
~/hsigstad.github.io/{slug}/ rsync.
Pipeline repos take the shared-helper route instead (see step 1): their
build.sh sources research-kit/tools/site_deploy.sh and calls its
sk_encrypt_site / sk_deploy_site — no inlined staticrypt/rsync. Set
SITE_GATED=0 for a public pipeline (pipelines/lovhistorie/build.sh) or leave
it gated (default 1) for a sensitive one (pipelines/govspend/build.sh). Same
gh-pages / repo-name-URL / password-gate outcome as below.
The self-contained build.sh wires three functions into a deploy mode:
build_site— make4ht for paper/talk, thenpython3 -m source.site.build_all→build/site/(plaintext).encrypt_site— staticrypt overbuild/site/→build/site-encrypted/:
staticrypt keeps the input dir's basename, so flattenSTATICRYPT_PASSWORD="$pw" npx --yes staticrypt build/site \ --recursive --directory build/site-encrypted \ --config build/.staticrypt.json \ --short --template-title "<PROJECT_TITLE>" \ --template-instructions "Enter the shared password to access the site." \ --remember 30build/site-encrypted/site/up one level afterward.--config build/.staticrypt.jsonkeeps the salt artifact out of the repo root.deploy_site— clone thegh-pagesbranch to a tmpdir (orphan-init it if absent),rsync -a --deletefrombuild/site-encrypted/(notbuild/site/), writerobots.txtwithDisallow: /, commit, push.
deploy mode chains them: build_site; encrypt_site; deploy_site. Run with:
cd $PROJECT_ROOT && bash build.sh deployRequirements: npx (for staticrypt) and git push access to the project repo.
site_deploy.sh clones/pushes gh-pages over SSH (git@github.com:…)
regardless of the repo's origin URL, so the deploy works even when origin is
HTTPS. But committing the source to an HTTPS-origin repo fails here (no
credential helper: could not read Username for 'https://github.com') — push via
token instead: git push "https://x-access-token:${GH_TOKEN}@github.com/<owner>/<repo>.git" HEAD:main
(leaves origin untouched). SSH-origin repos push normally.
The site password. Each project gets its own, stored in a gitignored
.site-password at the project root (also read from $STATICRYPT_PASSWORD).
Pick a memorable phrase tied to the project's topic, in three-word
hyphen-slug form (aaa-bbb-ccc) — for Brazil law/economics projects, a
Portuguese legal term or a phrase from the subject matter. Example: serasa uses
dano-moral-presumido (the "presumed moral damage" doctrine central to its
cases). When setting up a new site, propose a password in this style, write it
to .site-password, and tell the user what it is — it is not secret from the
user, they need it to view the site. Add both .site-password and
.staticrypt.json to the project .gitignore.
7. Verify
Check that build/site/index.html exists and list the generated files. After a
deploy, confirm https://hsigstad.github.io/{repo}/ serves the staticrypt
password gate (look for class="staticrypt-html" / "Enter the shared password")
— on a sub-page too, not just the index.
Important rules
- Never modify templates in other projects -- each project gets its own copy.
- The design system (CSS, nav bar, JS) must be identical across all projects for visual consistency.
- All sites include
robots.txtwithDisallow: /and<meta name="robots" content="noindex, nofollow">-- these are private research sites. - The deployed site is staticrypt-encrypted and pushed to the project's
gh-pagesbranch -- see step 6.bash build.sh deployis the one canonical deploy path; there is no plaintext or personal-website publish step. - Never ship sensitive non-HTML assets into
build/site/-- staticrypt encrypts only HTML, so PDFs/CSVs/images would be readable by direct URL. The only non-HTML file in a deployed site should berobots.txt. - Paper/talk pages are optional -- if
build/make4ht/doesn't exist, the build skips them gracefully and shows a placeholder card on the index. - Don't add data portal features (dataset cards, Chart.js) unless the user explicitly asks. The default is the docs-only pattern.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

