2 The Stan Ecosystem

By the end of this section, you’ll understand that the Stan ecosystem is a stack of related projects mainly developed under stan-dev, ranging from the compiler and math library to CmdStan, language interfaces, and workflow packages. You’ll be able to place a typical modeling task within that stack and identify which layer each piece belongs to.

2.1 A minimal Stan workflow

We start our workflow by writing a minimal Stan model in a .stan file.

R · Stan model

stan_code <- "
data {
  int<lower=0> N;
  vector[N] y;
}
parameters {
  real mu;
  real<lower=0> sigma;
}
model {
  mu ~ normal(0, 10);      // prior
  sigma ~ exponential(1);  // prior
  target += normal_lpdf(y | mu, sigma);  // likelihood
}
generated quantities {
  vector[N] log_lik;
  for (n in 1:N) {
    log_lik[n] = normal_lpdf(y[n] | mu, sigma);
  }
}
"
writeLines(stan_code, "model.stan")

Additionally, we provide some data for the input and save it as a JSON file.

R · data

library(cmdstanr)

stan_data <- list(N = 20, y = rnorm(20, mean = 5, sd = 2))
cmdstanr::write_stan_json(stan_data, "data.json")

> stan_data
$N
[1] 20

$y
 [1] 4.543589 4.899750 4.081620 8.698614 4.827003 2.903813 8.659615 5.165155
 [9] 5.148127 1.206675 2.368544 7.873981 5.071258 5.322390 6.913431 7.336083
[17] 3.462279 3.717310 1.620141 3.577615

2.1.1 Language interfaces

Next we can compile and sample from the Stan model. To do this, we use an interface, such as cmdstanr for R or cmdstanpy for Python, that wraps CmdStan.

CmdStan invokes

stanc3 to transpile the Stan model into a C++ file, which is then compiled and linked against
the math library (implementing built-in functions for distributions, linear algebra, ODE solvers, etc., along with automatic differentiation) and
the stan library (implementing the inference algorithms, such as HMC/NUTS).

The result is a standalone executable that the interface then runs to produce posterior samples.

2.1.1.1 The R interface `cmdstanr`

Compile and sample from the Stan model in R with cmdstanr. Use the fitted model to extract posterior draws, plot MCMC trace plots with bayesplot, and run approximate leave-one-out cross-validation with loo.

R · cmdstanr

library(cmdstanr)
library(bayesplot)
library(loo)

# compile Stan model
mod <- cmdstanr::cmdstan_model("model.stan")

# sample from the model
fit <- mod$sample(data = "data.json", chains = 4, iter_sampling = 1000)

# get draws
draws <- fit$draws(variables = c("mu", "sigma"))

# plotting and cross-validation
bayesplot::mcmc_trace(draws)
loo::loo(fit$draws("log_lik"))

2.1.1.2 The Python interface `cmdstanpy`

Compile and sample from the Stan model in Python with cmdstanpy. Use the fitted model to extract posterior draws, plot MCMC trace plots, and run approximate leave-one-out cross-validation using arviz.

Python  · cmdstanpy

from cmdstanpy import CmdStanModel
import arviz as az

# compile Stan model
mod = CmdStanModel(stan_file="model.stan")

# sample from the model
fit = mod.sample(data="data.json", chains=4, iter_sampling=1000)

# get draws
idata = az.from_cmdstanpy(fit, log_likelihood="log_lik")

# plotting and cross-validation
az.plot_trace(idata, var_names=["mu", "sigma"])
az.loo(idata)

2.1.1.3 The Command-line interface `CmdStan`

cmdstanr and cmdstanpy are thin language wrappers around CmdStan, the command-line interface to Stan. CmdStan itself invokes stanc3 to compile a .stan file into a standalone executable, which can be run directly from the command line to sample from (or optimize, etc.) the model.

Bash

# run from the CmdStan directory
make model

./model sample num_chains=4 num_samples=1000 \
  data file=data.json \
  output file=output.csv

$CMDSTAN/bin/stansummary output_*.csv
$CMDSTAN/bin/diagnose output_*.csv

2.1.2 Higher-level modelling frameworks

Instead of writing a Stan model from scratch, you can use higher-level modelling frameworks that let you specify your model using formula syntax. Two example frameworks are

brms, which dynamically generates and compiles custom Stan code for your specified model, and
rstanarm, which fits your model using a set of pre-compiled Stan programs for common model families (avoiding compilation time, at the cost of some flexibility).

R · brms

library(brms)

dat <- data.frame(y = jsonlite::fromJSON("data.json")$y)

fit <- brms::brm(
  formula = y ~ 1,
  data = dat,
  family = gaussian(),
  prior = c(
    prior(normal(0, 10), class = Intercept),
    prior(exponential(1), class = sigma)
  ),
  chains = 4,
  iter = 2000
)

R · rstanarm

library(rstanarm)

dat <- data.frame(y = jsonlite::fromJSON("data.json")$y)

fit <- rstanarm::stan_glm(
  formula = y ~ 1,
  data = dat,
  family = gaussian(),
  prior_intercept = normal(0, 10, autoscale = FALSE),
  prior_aux = exponential(1, autoscale = FALSE),
  chains = 4,
  iter = 2000
)

2.2 The Stan ecosystem and its libraries

To summarize what we have seen so far, we can outline the different layers in the Stan ecosystem and how they relate to one another.

Different layers in the Stan ecosystem

Layer 1: Compiler & math: stanc3 · stan · math
- Contribute to the fundamental building blocks of the Stan language: compiler transpilation, autodiff, or core inference algorithms.
Layer 2: Command-line engine: cmdstan
- Contribute to the command-line interface that drives compilation and execution.
Layer 3: Language interfaces: cmdstanr · cmdstanpy · rstan · …
- Contribute to environment-specific host language wrappers (R, Python, Julia, etc.) that interface with Stan.
Layer 4: Workflow & analysis: posterior · bayesplot · loo · priorsense · …
- Contribute to downstream packages used among others for visualization, posterior diagnostics, cross-validation, and sensitivity analysis.

2.2.1 Compiler, math, and CmdStan

Package	Description	Relations	Language
`math`	Autodiff library for distributions, linear algebra, and gradients	Used by generated model code	C++
`stanc3`	Compiler that translates `.stan` to C++	Feeds `cmdstan` and `rstan`	OCaml
`stan`	Inference algorithms and services (HMC/NUTS, etc.)	Built on `math`; used via `cmdstan` / `rstan`	C++
`cmdstan`	Command-line engine to compile and run Stan models	Wraps `stanc3` + `stan` + `math`; backend for `cmdstanr` / `cmdstanpy`	C++ / CLI

2.2.2 Language interfaces

Package	Description	Relations	Language
`cmdstanr`	R wrapper around CmdStan; returns `CmdStanMCMC`	Calls `cmdstan`; used with `posterior`, `bayesplot`, `loo`	R
`cmdstanpy`	Python wrapper around CmdStan; returns `CmdStanMCMC`	Calls `cmdstan`; used with `arviz`	Python
`rstan`	R interface that embeds Stan in-process; returns `stanfit`	Alternative to CmdStan; used by `rstanarm` and `rstantools` packages	R

2.2.3 Workflow, modeling, and packaging

Package	Description	Relations	Language
`brms`	Flexible multilevel / distributional regression via formulas; returns `brmsfit`	Generates Stan; backends `cmdstanr` / `rstan`; used with `loo`, `bayesplot`	R
`rstanarm`	Precompiled applied regression models; returns `stanreg`	Built on `rstan`; used with `loo`, `bayesplot`, `projpred`	R
`posterior`	Tools to manipulate and summarize draws (R-hat, ESS, etc.)	Shared draws API for `cmdstanr` and analysis packages	R
`bayesplot`	ggplot2 plots for MCMC diagnostics, PPC, and posteriors	Uses draws from interfaces / `posterior`	R

2.2.4 Workflow, modeling, and packaging (continued I)

Package	Description	Relations	Language
`loo`	Approximate LOO-CV (PSIS-LOO), WAIC, and model weights	Needs pointwise `log_lik` from fitted models	R
`arviz`	Diagnostics, plots, and model comparison in Python; returns `xr.DataTree`	Python counterpart to `posterior` / `bayesplot` / parts of `loo`	Python
`priorsense`	Prior and likelihood sensitivity analysis (power-scaling)	Uses `posterior`; related to PSIS methods in `loo`	R
`projpred`	Projection predictive variable selection	Uses `stanreg` (`rstanarm`) or `brmsfit` (`brms`) as reference	R

2.2.5 Workflow, modeling, and packaging (continued II)

Package	Description	Relations	Language
`shinystan`	Interactive Shiny GUI for MCMC diagnostics	Uses `stanfit` / `stanreg`; overlaps with `bayesplot`	R
`rstantools`	Tools to ship Stan models inside R packages; models typically return `stanfit`	Scaffolding used by packages such as `rstanarm`	R
`posteriordb`	Database of models, data, and reference posteriors	Shared resource for testing, teaching, and CI	R / Python / data