This post is to expose the inner workings of Bayesian inference.

Markov-Chain Monte Carlo

For the uninitiated

  1. MCMC Sampling for Dummies by Thomas Wiecki
  2. Introduction to Markov Chain Monte Carlo by Charles Geyer

Hamiltonian Monte Carlo and the No-U-Turn Sampler

  1. A Conceptual Introduction to Hamiltonian Monte Carlo by Michael Betancourt
  2. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo by Matthew Hoffman and Andrew Gelman
  3. MCMC Using Hamiltonian Dynamics by Radford Neal
  4. Hamiltonian Monte Carlo explained
  5. Hamiltonian Monte Carlo in PyMC3 by Colin Carroll

Sequential Monte Carlo and particle filters

  1. An Introdution to Sequential Monte Carlo Methods by Arnaud Doucet, Nando de Freitas and Neil Gordon
  2. Sequential Monte Carlo Methods & Particle Filters Resources by Arnaud Doucet

Other sampling methods

  1. Chapter 11 (Sampling Methods) of Pattern Recognition and Machine Learning by Christopher Bishop
  2. The Markov-chain Monte Carlo Interactive Gallery by Chi Feng

Variational Inference

For the uninitiated

  1. Variational Inference: A Review for Statisticians by David Blei, Alp Kucukelbir and Jon McAuliffe
  2. Chapter 10 (Approximate Inference) of Pattern Recognition and Machine Learning by Christopher Bishop

Open-Source Packages

  1. Stan
  2. PyMC3
  3. Pyro
  4. Edward
  5. Tensorflow Probability
  6. Infer.NET
  7. BUGS
  8. JAGS

Further Topics

Approximate Bayesian computation and likelihood-free methods

  1. Likelihood-free Monte Carlo by Scott Sisson and Yanan Fan

Expectation propagation

  1. Expectation propagation as a way of life: A framework for Bayesian inference on partitioned data