bayesian-hierarchical-process-models
Skill7 starsChanged 6 months agoApache-2.0View on GitHub
Fit Bayesian hierarchical (multi-level) regression models on process data using PyMC: campaign / regime random intercepts, censored likelihoods, AR(1) residuals, weakly informative priors. Use when you need calibrated uncertainty across regimes (campaigns, units, operators) or for partially pooled estimates with few observations per group. Don't use for plain OLS (use engineering-statistics), or as a substitute for first-principles modelling.
In this repository31 other files it gives its agents.
- absorption-stripping-designSkill
- caustic-merox-extractionSkill
- censored-regressionSkill
- compositional-data-analysisSkill
- control-valve-sizing-isa75Skill
- convective-heat-transfer-correlationsSkill
- design-of-experimentsSkill
- dimensionless-numbersSkill
- distillation-shortcut-designSkill
- distributed-lag-modelsSkill
- engg-skills-commonSkill
- engineering-skill-creatorSkill
- engineering-statisticsSkill
- equation-of-state-selectionSkill
- heat-exchanger-sizingSkill
- literature-search-engineeringSkill
- material-energy-balancesSkill
- pipe-flow-pressure-dropSkill
- process-causal-inference-dagsSkill
- process-units-conversionSkill
- reactor-sizing-and-kineticsSkill
- relief-valve-sizing-api520Skill
- separator-vessel-sizingSkill
- sour-gas-amine-treatingSkill
- statistical-process-controlSkill
- steam-tables-iapwsSkill
- thermo-process-propertiesSkill
- time-series-process-data-analysisSkill
- two-phase-flowSkill
- uvSkill
- vle-flash-calculationsSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 88
Write the spec YAML. Start with weakly informative priors.
- Very specific: line 94
If diagnostics fail, increase target_accept, tighten priors, or reparameterise.
- Distinctive: line 117
If PyMC is unavailable, surface to the user.
- Very specific: line 124
Run the listed script with representative inputs and an --output file when a deterministic calculation is available.
- Distinctive: line 126
Check result magnitudes against the stated assumptions, references, and a hand calculation or known operating range before reporting them.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Bayesian Hierarchical Process Models
Overview
Fits hierarchical (multi-level) Bayesian regressions of the form
y_t | x_t, group ~ Normal(alpha_{group[t]} + x_t' beta, sigma)with optional censoring on y_t and an AR(1) residual structure. Group
intercepts are drawn from a hyperprior, enabling partial pooling across
regimes.
The implementation uses PyMC (NUTS sampler) via a thin wrapper. The script accepts a YAML model specification so analysts can iterate on priors and likelihoods without editing code.
Prerequisites
uvavailable.- The script declares
pymc,arviz,pandas,pyyamlin PEP-723; first run installs them (sizeable: ~500 MB combined). - On first use, writes
LICENSE_NOTIFICATION.txt.
When to Use
- Data has natural groupings (campaigns, operators, instruments) and you want partial pooling.
- You need calibrated credible intervals that propagate censoring, autocorrelation, and shrinkage.
- The model needs informative priors (e.g. from engineering judgement).
Don't use for
- Plain OLS or single-level GLMs (
engineering-statistics). - Time-series forecasting (use a dedicated state-space tool).
- Extremely large datasets (>1M rows) without thinning — NUTS is not designed for that scale.
Utility Scripts
uv run scripts/bhm.py fit --data data.csv --spec spec.yaml --output /tmp/fit.json
Example spec.yaml:
response: log_S_total
predictors: [z_heavy, source_split, lean_loading]
group: campaign
censoring:
lower_col: LOQ_log
ar1: true
priors:
beta: {dist: normal, mu: 0, sigma: 5}
sigma: {dist: half_normal, sigma: 1}
tau_group: {dist: half_normal, sigma: 0.5}
sampler:
draws: 2000
tune: 1000
chains: 4
target_accept: 0.95Procedure
- Build a tidy CSV: one row per observation; columns for response, predictors, group, and censoring bounds.
- Write the spec YAML. Start with weakly informative priors.
- Run
fit. Inspect:- R-hat (should be < 1.01 for all parameters).
- Effective sample size (ESS bulk and tail > 400).
- Divergences (should be 0).
- Posterior predictive checks.
- If diagnostics fail, increase
target_accept, tighten priors, or reparameterise. - Report posterior means with 94% HDI; never report point estimates alone.
Pitfalls
- Using flat priors and being surprised by funnel shapes. Weakly
informative priors (e.g.
Normal(0, 5)on standardised predictors) work better. - Failing to standardise predictors; PyMC + NUTS works much better with scaled inputs.
- Treating the posterior mean as the only output; report the HDI.
- Running with
chains=1; convergence diagnostics need at least 2 chains. - Ignoring divergences; they indicate biased posterior geometry.
- Using non-centred parameterisation for groups with many observations but the centred version for groups with few; pick consistently and re-fit if diagnostics fail.
- Forgetting to write the InferenceData artefact (NetCDF) to disk — re-fitting is expensive.
Fallback Strategies
- If PyMC is unavailable, surface to the user; there is no clean pure-Python fallback for hierarchical Bayesian with censoring.
- For sensitivity, fit the same model with at least two prior choices and compare.
Verification
- Run the listed script with representative inputs and an
--outputfile when a deterministic calculation is available. - Confirm the JSON result contains
ok: true, expected units, and no unhandled warnings. - Check result magnitudes against the stated assumptions, references, and a hand calculation or known operating range before reporting them.
References
references/spec_format.md— full spec.yaml grammar.- Gelman, Carlin, Stern, Dunson, Vehtari, Rubin, Bayesian Data Analysis (3rd ed).
- PyMC docs: https://www.pymc.io/
Anti-Patterns
- Reporting Bayesian posterior intervals as if they were classical CIs without naming the model.
- Skipping convergence diagnostics.
- Treating sampling failure as a tuning issue without reviewing the model geometry.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

