Handbook of
Markov Chain
Monte Carlo
Chapman & Hall/CRC
Handbooks of Modern Statistical Methods
Series Editor
Garrett Fitzmaurice
Department o f Biostatistics
Harvard School o f Public Health
Boston, MA, U.S.A.
Aims and Scope
The objective o f the series is to provide high-quality volumes covering the state-of-the-art in the theory
and applications o f statistical methodology. The hooks in the series are thoroughly edited and present
comprehensive, coherent, and unified summaries o f specific methodological topics from statistics. The
chapters are written by the leading researchers in the field, and present a good balance o f theory and
application through a synthesis o f the key methodological developments and examples and case studies
using real data.
The scope o f the series is wide, covering topics o f statistical methodology that are well developed and
find application in a range o f scientific disciplines. The volumes are primarily o f interest to researchers
and graduate students from statistics and biostatistics, but also appeal to scientists from fields where the
methodology is applied to real problems, including medical research, epidemiology and public health,
engineering, biological science, environmental science, and the social sciences.
Published Titles
Longitudinal Data Analysis
Edited by Garrett Fitzmaurice, Marie Davidian,
Geert Verheke, and Geert Molenherghs
Handbook of Spatial Statistics
Edited by Alan E, Gelfand, Peter J. Diggle,
Montserrat Fuentes, and Peter Guttorp
Handbook of M arkov Chain Monte Carlo
Edited by Steve Brooks, Andrew Gelman,
Galin L. Jones, andXiao-Li Meng
Chapman & Hall/CRC
Handbooks of Modern
Statistical Methods
Handbook of
Markov Chain
Monte Carlo
E dited by
Steve Brooks
Andrew Gelman
Galin L. Jones
Xiao-Li Meng
CRC Press
Taylor S. Francis Group
Boca Ralon London New York
C R C Press is an im p rin l of the
Taylor & Francis Croup, an informa business
A C H A P M A N & H AL L BO O K
M A TLA B” is a tradem ark o f T he M athW orks, Inc. and is used with perm ission, T he M athW orks does not w arrant the
accuracy o f the text or exercises in this book. T his book's use or discussion o f M A T L A B 7, softw are or related products
does not constitute endorsem ent or sponsorship by T he M athW orks o f a particular pedagogical approach or particular
use o f the M A TLA IT softw are.
Chapm an & Hall/CRC
Taylor & Francis Group
6 0 0 0 Broken Sound Parkway NW, Suite 3 0 0
Boca Raton, FL 33487-2742
© 2011 by Taylor and Francis Group, LLC
Chapm an & Hall/CRC Ls an im print o f Taylor & Francis Group, an Info m i a business
No claim to original U.S. Governm ent works
International Standard Book Number-13: 978-1-4200-7942-5 (eBook - PDF)
T his book contains inform ation obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and inform ation, but the author and publisher cannot assume responsibility for the valid
ity o f all m aterials or the consequences o f their use. T he authors and publishers have attem pted to trace the copyright
holders o f all m aterial reproduced in this publication and apologize to copyright holders if perm ission to publish in this
form has not been obtained. If any copyright m aterial has not been acknowledged please w rite and let us know so we may
rectify in any future reprint.
Except as perm itted under U.S. Copyright Law, no part o f th is book may be reprinted, reproduced, transm itted, or uti
lized in any form by any electronic, m echanical, or other m eans, now known or hereafter invented, including photocopy
ing, m icrofilm ing, and recording, or in any inform ation storage or retrieval system, without w ritten perm ission from the
publishers.
For perm ission to photocopy or use m aterial electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright C learance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, M A 01923,
97 8 -750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety o f users. For
organizations that have been granted a photocopy license by the CCC , a separate system o f payment has been arranged.
T rad em ark N otice: Product or corporate names may be tradem arks or registered tradem arks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http: //w w w,taylor a nd f ranci s .com
and the CRC Press Web site at
http://www,crcpress,com
Contents
Preface................................................................................................................................................... xix
Editors...................................................................................................................................................xxi
Contributors...................................................................................................................................... xxiii
Part I Foundations, Methodology, and Algorithms
1. Introduction to Markov Chain Monte Carlo..........................................................................3
1.10 Variance Estimation
Charles J. Geyer
1.1 History....................................................................................................................................3
1.2 Markov Chains.....................................................................................................................4
1.3 Computer Programs and Markov Chains
.....................................................................5
1.4 Stationaiity............................................................................................................................ 5
1.5 Reversibility..........................................................................................................................6
1.6 Functionals............................................................................................................................ 6
1.7 The Theory of Ordinary Monte Carlo
...........................................................................6
1.8 The Theory of M CM C........................................................................................................ 8
1.8.1 Multivariate Theory...............................................................................................8
1.8.2 The Autocovariance Function............................................................................. 9
1.9 AR{1) Exam ple.....................................................................................................................9
1.9.1 A Digression, on Toy Problem s...................................................................... 10
1.9.2
Supporting Technical R ep o rt.........................................................................11
1.9.3 The Exam ple...................................................................................................... 11
........................................................................................................13
1.10.1 Nonoverlapping Batch M ean s...................................................................... 13
Initial Sequence Methods................................................................................ 16
1.10.2
1.10.3
Initial Sequence Methods and Batch M e a n s..............................................17
1 11 The Practice of M C M C ................................................................................................. 17
1.11.1 Black Box MCMC
............................................................................................ 18
1.11.2 Pseudo-Convergence........................................................................................18
1.11.3 One Long Run versus Many Short Runs..................................................... 18
1.11.4 Bu rn-In................................................................................................................ 19
1.11.5 Diagnostics.........................................................................................................21
1.12 Elementary Theory of M CM C..................................................................................... 22
1.12.1 The Metropolis-Hastings Update................................................................. 22
1.12.2 The Metropolis-Hastings Theorem...............................................................23
1.12.3 The Metropolis U p d ate...................................................................................24
1.12.4 The Gibbs U p d ate............................................................................................ 24
1.12.5 Vaiiable-at-a-Time Metropolis-Hastings..................................................... 25
1.12.6 Gibbs Is a Special Case of Metropolis-Hastings
.......................................26
1.12.7 Combining Updates.......................................................................................... 26
1.12.7.1 Composition...................................................................................... 26
1.12.7.2 Palindromic Com position..............................................................26
1.12.8 State-Independent M ixing..............................................................................26
1.12.9 Subsampling.......................................................................................................27
1.12.10 Gibbs and Metropolis Revisited.................................................................... 28
v
vi
Contents
1.13 A Metropolis Example..................................................................................................... 29
1.14 Checkpointing..................................................................................................................34
1.15 Designing M C M C C ode................................................................................................ 35
1.16 Validating and Debugging MCMC C od e...................................................................36
1.17 The Metropolis-Hasting^ Green Algorithm..............................................................37
1.17.1 State-Dependent Mixing
................................................................................38
1.17.2 Radon-Nikodym Derivatives.........................................................................39
.................................................. 40
1.17.3 Measure-Theoretic Metropolis-Hastings
1.17.3.1 Metropolis-Hastings-Green Elementary U p d ate....................40
1.17.3.2 The MHG Theorem ..........................................................................42
1.17.4 MHG with Jacobians and Augmented State S p a c e .................................45
1.17.4.1 The MHGJ Theorem ........................................................................46
Acknowledgments.................................................................................................................... 47
References...................................................................................................................................47
2. A Short History of MCMC: Subjective Recollections from Incomplete D ata........49
2.2.1
2.2.2
Christian. Rd>ert and George CaseUn
2.1
Introduction.......................................................................................................................49
2.2 Before the Revolution..................................................................................................... 50
The Metropolis et al. (1953) Paper.................................................................50
The Hastings (1970) P ap er............................................................................. 52
2.3 Seeds of the Revolution...................................................................................................53
Besag and the Fundamental (Missing) Theorem ......................................53
2.3.1
2.3.2
EM and Its Simulated Versions as Precursors...........................................53
2.3.3 Gibbs and Beyond.............................................................................................54
2.4 The Revolution..................................................................................................................54
2.4.1 Advances in MCMC Theory...........................................................................56
2.4.2 Advances in MCMC A pplications.............................................................. 57
2.5 After the Revolution........................................................................................................58
2.5.1 A Brief Glimpse atPartide System s............................................................58
2.5.2
Perfect Sam pling...............................................................................................58
2.5.3 Reversible Jump and Variable Dimensions................................................59
2.5.4 Regeneration and the Central Limit Theorem ...........................................59
2.6 Conclusion.........................................................................................................................60
Acknowledgments.................................................................................................................... 61
References...................................................................................................................................61
3. Reversible Jump MCMC..........................................................................................................67
Yimtvi F(m mid Scott A. Sisson
3.1
3.2
Introduction.......................................................................................................................67
3.1.1
From Metropolis-Hastmgs to Reversible Jum p........................................ 67
3.1.2 Application A re a s............................................................................................ 68
Im plementation............................................................................................................... 71
3.2.1 Mapping Functions and Proposal Distributions......................................72
3.2.2 Marginalization and Augmentation............................................................ 73
3.2.3 Centering and Order M ethods......................................................................74
3.2.4 Multi-Step Proposals....................................................................................... 77
3.2.5 Generic Samplers...............................................................................................78