MCMC

Markov Chain Monte Carlo (MCMC) is a family of algorithms used to draw samples from probability distributions that are difficult or impossible to sample from directly. The core idea is to construct a Markov chain — a sequence of random states where each state depends only on the previous one — that eventually converges to the desired target distribution. Once the chain has run long enough to reach its stationary distribution, the samples it produces are representative of the target.

The two most widely used MCMC algorithms are the Metropolis-Hastings algorithm and Gibbs sampling. Metropolis-Hastings proposes a candidate move at each step and accepts or rejects it based on a probability ratio, guaranteeing convergence to the target distribution regardless of the proposal mechanism. Gibbs sampling is a special case that works when the conditional distributions of individual variables are easy to sample from, iterating through each variable in turn.

MCMC is foundational in Bayesian statistics, where posterior distributions over model parameters rarely have closed-form solutions. It is also used in physics simulations, computational biology, and probabilistic programming frameworks like Stan and PyMC. In machine learning, MCMC appears in Bayesian neural networks, latent variable models, and uncertainty quantification. The main practical challenges are diagnosing convergence, managing autocorrelation between samples, and computational cost — MCMC chains can require millions of steps to produce reliable estimates for high-dimensional problems.