cogneuro-project
Skill3 starsChanged 15 days agoMITView on GitHub
Organize a cognitive-neuroscience fMRI project (Gilles's house style) — flat-package layout, Subject class as single source of truth, pipeline-stage submodules, co-located SLURM jobs, multi-env conda, BIDS derivatives discipline, plus conventions for using the in-house braincoder (PRF/encoding) and bauer (Bayesian behavioral) libraries. Use when bootstrapping a new project, refactoring an old one, adding a new analysis stage, or auditing a project for drift from the convention. Triggers on "set up a new project", "where should this script go", "how do I structure...", or working in any of ~/git/{neural_priors, abstract_values, retinonumeral, retsupp, value_capture, tms_risk, risk_experiment, bauer, braincoder}.
- Installs packages
In this repository14 other files it gives its agents.
- bayesian-workflowSkill
- braincoderSkill
- data-archivalSkill
- eyetracking-edfSkill
- finalize-projectSkill
- fmriprepSkill
- pycortexSkill
- sciencecloud-openstackSkill
- scienceclusterSkill
- scientific-figuresSkill
- scientific-presentationsSkill
- scientific-writingSkill
- snakemakeSkill
- spm-glmSkill
The file
We split this skill into the rules it gives an agent
11 rules. About 0 are common, 4 are distinctive, and 7 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 15
When working in an existing project, follow what's already there unless explicitly refactoring.
- Very specific: line 99
Every project has <package>/utils/data.py with a Subject class.
- Very specific: line 111
Never hardcode the cluster path inside data.py.
- Very specific: line 125
Don't return "either a DataFrame or a NIfTI dict" from one method — split them.
- Distinctive: line 162
Write each analysis output in the format that matches its *shape*, not whatever's one line to dump:
- Very specific: line 200
Not every project needs every stage — add as needed, use the canonical name.
- Distinctive: line 262
The submodule is for editable development: After building the env, override the pinned install:
- Very specific: line 271
Don't vendor copies. Don't rely on libs/braincoder being on sys.path (only works from repo root).
- Very specific: line 336
If you inherit a shared checkout you cannot untangle yet, guard it: a post-checkout hook in .git/modules/libs/<lib>/hooks/ printing the …
- Distinctive: line 350
PPCs before parameters, always: When a bauer fit lands, the first analysis is the posterior predictive check in the paradigm's native cells …
- Distinctive: line 368
Don't pull into the analysis env (PsychoPy has incompatible deps).
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Cognitive-neuroscience project organization
The house style for Gilles's fMRI projects, distilled from auditing neural_priors, abstract_values, retinonumeral, retsupp, tms_risk. Where the projects converge, that's the convention. Where they diverge, this skill picks one — usually the version the newest project settled on — and flags the others as drift to clean up next time that file is touched.
When working in an existing project, follow what's already there
unless explicitly refactoring; swapping slurm_jobs/ ↔ jobs/
mid-project just to match this skill is churn for its own sake.
Cross-skill pointers:
- SLURM internals (conda, GPU, NFS, log conventions) → sciencecluster
- fmriprep operational gotchas → fmriprep
- Figure aesthetics → scientific-figures
- NIfTI dtype trap, local-vs-cluster split → global CLAUDE.md
references/ are real exemplars. Most files are verbatim or
lightly-excerpted code from working projects (abstract_values,
tms_risk), each with a provenance header naming source path and git
SHA. Treat them as concrete reference, not blank-fill templates. The
genuinely synthetic templates that remain (subjects.yml, gitignore,
CLAUDE.md.template, setup.py, README.md) are marked as such.
Canonical layout
<project>/
├── setup.py # minimal: find_packages(), package_data=subjects.yml
├── README.md # paper-facing: science + setup + pipeline
├── CLAUDE.md # developer-facing: recipes, gotchas, model table
├── .gitignore # see references/gitignore
├── .gitmodules # libs/ submodules
├── libs/ # git submodules (braincoder, bauer, exptools2)
├── environment_cuda.yml # cluster GPU env (or environment_linux.yml,
│ # or inside create_env/ if it needs sbatch build)
├── environment_apple_silicon.yml # local mac dev
├── environment_psychopy.yml # optional, experiment-only
├── experiment/ # PsychoPy/exptools2 task code
├── notes/ # markdown working notes (NOT Jupyter)
│ ├── INDEX.md STATUS.md
│ ├── data/ # small TSVs aggregated from cluster
│ ├── figures/ # generated PDFs/PNGs
│ ├── analyses/ # focused per-analysis writeups
│ └── archive/ # superseded outputs
├── scripts/ # orchestrators, one-offs (NOT at repo root)
├── tests/ # pytest — at minimum, test_data.py
├── <project>/ # main package; same name as project dir
│ ├── __init__.py
│ ├── data/subjects.yml # per-subject session/run metadata
│ ├── utils/data.py # Subject class — SINGLE SOURCE OF TRUTH
│ ├── prepare/ # raw → BIDS, events files
│ ├── cluster_preproc/ # fmriprep wrapper, bids_filter.json
│ ├── nordic/ # NORDIC denoising (7T only)
│ ├── preprocess/ # confound regression, mean ts
│ ├── glm/ # GLMsingle single-trial betas
│ ├── modeling/ # PRF / encoding-model fits + decoding
│ ├── surface/ # FreeSurfer surface projection
│ ├── visualize/ # plotting helpers
│ ├── behavior/ # behavioral parsing + Bayesian models (bauer)
│ └── notebooks/ # exploratory Jupyter (inside the package)
└── derivatives/ # OPTIONAL: small version-controlled outputs only
# (not BIDS derivatives — those live on
# /shares/zne.uzh/gdehol/ds-<project>
# and /data/ds-<project>)Each pipeline submodule has its own slurm_jobs/ subdirectory with
matching .sh wrappers next to the .py scripts.
Naming standardization
Pick the right column for new code; leave existing code alone unless refactoring.
| Pick | Avoid | Why |
create_env/ | cluster_env_setup/ | shorter; retsupp's choice |
slurm_jobs/ | jobs/ | unambiguous |
modeling/ | encoding_model/, encoding_models/ | covers PRF + non-PRF encoding + decoding |
behavior/ | cogmodels/ | what abstract_values and retsupp use |
notes/ (markdown) | notebooks/ (Jupyter) for docs | markdown is durable; .ipynb diffs are noise |
<project>/notebooks/ | top-level notebooks/ | keep notebooks importable from the package |
scripts/<one_off>.sh | <one_off>.sh at repo root | abstract_values has 4 ad-hoc scripts at root — a mess |
--bids_folder vs --bids-folder: argparse handles both
identically with dest='bids_folder'. Most existing code uses
underscore — match it.
The Subject class — single source of truth
Every project has <package>/utils/data.py with a Subject class.
This is the only place that knows about disk paths, BIDS file
naming, per-subject quirks, and derivative caching. Analysis scripts
import Subject; they do NOT build paths themselves.
Real exemplar: references/subject_class.py (abstract_values). Six conventions:
- Constructor:
Subject(subject_id, bids_folder=DEFAULT). Accept both1and'01'; zero-pad internally. Pilots are strings ('pil01'). Defaultbids_folderis the local mac path (/data/ds-<project>); SLURM scripts override to the cluster path. Never hardcode the cluster path insidedata.py. - One method per data type, one keyword per BIDS entity.
Don't return "either a DataFrame or a NIfTI dict" from one method — split them. Don't shadow the builtinsub.get_preprocessed_bold(session, run=None, space='T1w') sub.get_single_trial_estimates(session, kind='stim', smoothed=False, roi=None) sub.get_prf_parameters_volume(model_label, smoothed=False) # NIfTI dict sub.get_prf_parameters_tsv(model_label, smoothed=False) # DataFrame sub.get_brain_mask(session=None, return_masker=True) sub.get_volume_mask(roi, session=1, return_masker=False) sub.get_confounds(session, kind='minimum') sub.get_onsets(session, run=None) sub.get_behavioral_data(session=None, tasks=None)type; usekind. - Encapsulate per-subject quirks here, never in scripts. retsupp
branches on
subject_id < 3(early sourcedata layout); retinonumeral has a pulse-t0 fix for sub-06 ses-2 run-5. These belong inSubject._apply_*_quirks()with a comment naming the quirk. Add a pytest case for each quirky path so regressions surface. - Read per-subject metadata from
<package>/data/subjects.yml— sessions, run counts, exclusions. Don't hardcode "8 runs" anywhere; callsub.get_runs(session). Schema: references/subjects.yml. - Cache the slow stuff at the TSV layer. PRF parameter NIfTIs are
big and most downstream code wants one row per voxel.
get_prf_parameters_tsv()readsderivatives/extracted_pars/*.tsv; the NIfTI variant is only for spatial analyses. neural_priors does this — adopt elsewhere. - Wrap dtype-safe NIfTI writes in
Subject._write_volume().
Without this, output inherits the (uint8) mask dtype and quantizes to ~256 values viaimg = masker.inverse_transform(values) img.set_data_dtype(np.float32) img.header.set_slope_inter(slope=1, inter=0) img.to_filename(path)scl_slope. See global CLAUDE.md ("NIfTI dtype trap"). Every project bites on this independently — solve once.
Optional patterns (derivative-name composition from boolean flags, module-level experimental constants, master-DataFrame entry point): references/subject_class_patterns.md.
Output formats — pick by the data's shape
Write each analysis output in the format that matches its shape, not whatever's one line to dump:
- Whole-brain / volumetric (per-voxel maps over the brain mask: PRF params, GLM betas, decoded maps) → NIfTI, always float32 + cleared scl (dtype trap above). It must live in brain space for masking, ROI extraction, and pycortex/freeview.
- Tabular (ROI-restricted, per-(sub, ROI), per-voxel-within-an-ROI, or group summaries) → tidy long-format TSV: one row per observation, with subject / ROI / model / fold / condition as columns. Greppable, opens anywhere, survives library upgrades, plots straight into seaborn. Keep logically distinct tables in separate TSVs (e.g. a small CV-R² table apart from a wider fitted-params table) so the one you replot constantly stays small.
- Ephemeral pipeline caches (concatenated cleaned BOLD, intermediate arrays) → npz / parquet, co-located with their source so provenance is obvious.
- Never pickle a durable result. pandas/numpy pickles break across library versions and are opaque/un-greppable. If a result is worth keeping it's worth a TSV (tabular) or NIfTI (volumetric); reserve pickle for throwaway within-run scratch. A nested payload (lists of per-fold DataFrames, ragged arrays) is a smell — flatten it to a long TSV with the fold/subject/ROI as columns.
Submodules by pipeline stage
| Submodule | Inputs | Outputs |
prepare/ | DICOM, behavior TSVs | BIDS dataset |
nordic/ | BIDS func | denoised func (7T only) |
cluster_preproc/ | BIDS | derivatives/fmriprep/ |
preprocess/ | fmriprep BOLD | derivatives/cleaned/ |
glm/ | cleaned BOLD | derivatives/glmsingle/ |
modeling/ | single-trial betas | derivatives/prf*/ |
surface/ | parameter NIfTIs | .func.gii files |
visualize/ | summary TSVs | PDFs in notes/figures/ |
behavior/ | events TSVs | bauer/HSSM trace files |
Not every project needs every stage — add as needed, use the canonical
name. For the surface/ projection recipe (nilearn vol_to_surf with
pial+white sampling → SurfaceTransform for fsaverage, pycortex
blend_curvature viewer): references/surface_sampling.md.
Analysis script CLI convention
Real exemplar: references/analysis_script_example.py
(abstract_values fit_glmsingle.py). Rules:
- Positional
subject(int or'pil01') first — never--subject. --bids_folderas the only path argument; default to the local mac path; SLURM scripts override.- One Python script per analysis variant, but model variants use
--model <label>not separate scripts (abstract_values:fit_aprf.py --model session-shift). - Defer heavyweight imports (
tensorflow,jax,pymc,glmsingle) until insidemain()— keepsimport <project>fast and surfaces argparse errors before GPU init. - Echo the resolved config at the top of
main()so SLURM logs are searchable and typos surface fast. - Hard-fail loudly on incomplete subjects
(
sub.require_complete_sessions()) with--allow-incompleteas the explicit escape hatch. Silent multi-session aggregation downstream is much worse than an early crash.
SLURM jobs co-located with Python modules
modeling/
├── fit_prf.py
├── decode.py
└── slurm_jobs/
├── fit_prf.sh
├── fit_prf_smoothed.sh
└── decode.shOne .sh wraps one .py script. Inputs are environment variables set
on the call (KEY=value sbatch job.sh; never an --export list, which causes
user env retrieval failed holds, see sciencecluster), not positional args. Naming: <analysis>[_<variant>].sh (e.g.,
fit_aprf.sh, fit_aprf_cv.sh). Avoid submit.sh, run.sh, job1.sh.
Real exemplar: references/slurm_job_example.sh
— shows the env-variable input pattern, dynamic ARGS=(...) array,
PYTHONUNBUFFERED=1 + direct env binary, log redirection.
The internals (account, conda activation, GPU constraints, log paths,
%150 throttling, dependency chains) live in the sciencecluster
skill — don't duplicate here.
libs/ — braincoder, bauer, exptools2
In-house libraries live both as git submodules under libs/ and as
pip-installed packages from git URLs in the env YML.
- The env YML pins a commit/branch for reproducibility. This is
what runtime imports use:
- pip: - "braincoder @ git+https://github.com/Gilles86/braincoder.git@v0.6.2" - The submodule is for editable development. After building the
env, override the pinned install:
Edits toconda activate <project> pip install -e libs/braincoderlibs/braincoder/braincoder/models.pyshow up immediately. Drop the editable install to freeze again.
Don't vendor copies. Don't rely on libs/braincoder being on
sys.path (only works from repo root).
One conda env per project, and per accelerator — never shared
A project owns its environments. Create them from the project's own YMLs and
install that project's own libs/ checkouts into them:
| Env name | Built from | For |
<project> | environment_apple_silicon.yml | local Mac dev |
<project>_cpu | create_env/environment_cpu.yml | cluster CPU jobs |
<project>_cuda | create_env/environment_cuda.yml | cluster GPU jobs (build ON a GPU node) |
<project>_behavior | environment_cloud_behavior.yml | behaviour-only box; no TF/CUDA, solves in seconds |
conda env create -f environment_cuda.yml # name: <project>_cuda
conda activate <project>_cuda
pip install -e . # the project package
pip install -e libs/bauer libs/braincoder # THIS project's checkoutsThen verify the env points where you think it does — one line, run it every time you build an env:
python -c "import bauer, braincoder, pathlib; \
print(bauer.__file__); print(braincoder.__file__)"
# both paths MUST be inside this project's libs/. If they name another
# project's repo, the env is borrowing someone else's library.Why this is a rule and not a preference. pip install -e writes an
absolute path into site-packages, so an env built by pointing at a different
project's libs/ is permanently coupled to it. Audit an existing machine with:
for e in $CONDA_ROOT/envs/*/; do
p=$($e/bin/python -c "import bauer; print(bauer.__file__)" 2>/dev/null)
[ -n "$p" ] && echo "$(basename $e) -> $p"
doneOn sciencecluster in 2026-08 that returned fourteen envs across five
projects — retsupp_*, soglio_cuda, value_capture, tf2-* and the
tms_risk_* family — all resolving to ~/git/tms_risk/libs/bauer. A
git checkout in that one directory silently repoints the library for every
one of them, including jobs already running, and nothing raises.
Pinning a version, wherever you are. Even with dedicated envs, don't pin by
checking out inside libs/ — that mutates the tree your editable install
tracks. Clone and use PYTHONPATH, which precedes site-packages and so wins
without touching anything:
git clone <repo>/libs/bauer /scratch/$USER/bauer_<tag>
git -C /scratch/$USER/bauer_<tag> checkout <commit>
PYTHONPATH=/scratch/$USER/bauer_<tag> python -m <project>.behavior.fit_model ...Stamp the library commit into every artifact you fit (bauer traces do this
in trace.posterior.attrs). A stored fit only means something against the
commit that produced it, and a mismatched reload predicts plausible-looking
wrong numbers rather than raising.
If you inherit a shared checkout you cannot untangle yet, guard it: a
post-checkout hook in .git/modules/libs/<lib>/hooks/ printing the
dependent-env list, plus an untracked SHARED_CHECKOUT_README.md at the
library root — untracked so a checkout cannot remove it.
The three libraries:
- braincoder — PRF / encoding-model fitting (Keras 3
multi-backend). Read the braincoder skill first: manual, traps,
decoding, runtimes. Lives in
<package>/modeling/. Real fit example: references/braincoder_prf_example.py shows the canonical 3-stage fit (fit_grid→refine_baseline_and_amplitude→fit). Never skip the refine step — it makes gradient-descent warm-start dramatically better. - PPCs before parameters, always. When a bauer fit lands, the first analysis is the posterior predictive check in the paradigm's native cells (simulated outcomes, per-draw aggregation, band coverage printed) — NOT the ELPD ladder, NOT parameter posteriors. ELPD ties can hide offsetting cell-wise misfits, and group-mean parameter readouts hit the mean-parameter trap. Order: PPC → ELPD → parameters. (Gilles, 2026-08-22.)
- bauer fits on any cluster: set
PYTENSOR_FLAGS="base_compiledir=/scratch/$USER/pytensor/<jobid>_<taskid>"in the SLURM script — pytensor's default$HOME/.pytensorcache blows home quotas and serializes on one lock when many fits compile at once (bit tms_risk 2026-08-22; details in the sciencecluster skill). This applies even with--backend numpyro. - bauer — Bayesian behavioral models (PyMC / numpyro). Lives in
<package>/behavior/. Real fit example with model dispatch + DDM/RDM handling: references/bauer_cogmodel_example.py. - exptools2 — PsychoPy + ioHub experiment runtime. Lives in
experiment/(top-level), used on the experiment machine only. Don't pull into the analysis env (PsychoPy has incompatible deps).
notes/ for markdown, not notebooks
The single best refinement from retsupp:
notes/INDEX.md— manually curated list of every doc innotes/notes/STATUS.md— what's done, in progress, blockednotes/<topic>.md— focused writeups (one per analysis decision)notes/analyses/<analysis>/— per-analysis subfoldersnotes/data/— TSVs aggregated on the cluster, rsync'd back for local plotting (cf. global CLAUDE.md)notes/figures/— generated PDFs/PNGs (gitignore the regenerable scratch ones, commit the paper-figure outputs)notes/archive/— superseded analyses, kept for provenance
Notebooks still belong in the project, but inside the package at
<package>/notebooks/. Top-level notebooks/ makes
from <project> import utils brittle.
Reference index
Load on demand. Most are real-code excerpts (see provenance headers).
Code exemplars
subject_class.py— Subject class (abstract_values)analysis_script_example.py— analysis script (abstract_values fit_glmsingle.py)slurm_job_example.sh— SLURM .sh wrapper (abstract_values)braincoder_prf_example.py— braincoder 3-stage fit (abstract_values fit_aprf.py)bauer_cogmodel_example.py— bauer model dispatch (tms_risk fit_model.py)ingest_new_session.sh— pipeline orchestrator (abstract_values)test_data.py— Subject pytest suite (tms_risk)environment_cuda.yml— cluster CUDA stack (abstract_values environment_linux.yml)environment_apple_silicon.yml— Mac M-series, optional metal GPU
Deeper conventions (load when needed)
envs.md— when to split into 2 / 3 / 4 envs; canonical TF/Keras/JAX stacksubject_class_patterns.md— optional Subject patterns (deriv-name composition, module constants, master-DataFrame entry)surface_sampling.md— volume → fsnative → fsaverage projection recipeclaude_md_checklist.md— what to put in a project's CLAUDE.mdaudit_punchlist.md— house-style gaps to clean up when touching a project
Synthetic templates (copy and edit)
CLAUDE.md.template— starting CLAUDE.mdsubjects.yml— per-subject metadata schemagitignore— sensible defaultsetup.py— minimalfind_packages()README.md— bootstrap recipe for a new project
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

