第1页 / 共294页
第2页 / 共294页
第3页 / 共294页
第4页 / 共294页
第5页 / 共294页
第6页 / 共294页
第7页 / 共294页
第8页 / 共294页
Introducing Monte Carlo Methods with R 原版PDF by Robert & Casella.pdf
Christian P. Robert · George Casella
Introducing Monte Carlo
Methods with R
123
George Casella
Department of Statistics
University of Florida
103 Griffin-Floyd Hall
Gainesville FL 32611-8545
USA
casella@stat.ufl.edu
Kurt Hornik
Department of Statistik and Mathematik
Wirtshchaftsuniversit¨at Wien Augasse 2-6
A-1090 Wien
Austria
Christian P. Robert
Universit´e Paris Dauphine
UMR CNRS 7534
CEREMADE
place du Mar´echal de Lattre
de Tassigny
75775 Paris cedex 16
France
xian@ceremade.dauphine.fr
Series Editors
Robert Gentleman
Department of Bioinformatics
and Computational Biology
Genentech South San Francisco
CA 94080
USA
Giovanni Parmigiani
Department of Biostatistics
and Computational Biology
Dana-Farber Cancer Institute
44 Binney Street
Boston, MA 02115
USA
ISBN 978-1-4419-1575-7
DOI 10.1007/978-1-4419-1576-4
Springer New York Dordrecht Heidelberg London
e-ISBN 978-1-4419-1576-4
Library of Congress Control Number: 2009941076
c Springer Science+Business Media, LLC 2010
All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the
publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief
excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and
retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified
as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
Preface
“After that, it was down to attitude.”
Ian Rankin
Black & Blue
The purpose of this book is to provide a self-contained entry into Monte Carlo
computational techniques. First and foremost, it must not be confused with
a programming addendum to our earlier book Monte Carlo Statistical Meth-
ods whose second edition came out in 2004. The current book has a different
purpose, namely to make a general audience familiar with the programming
aspects of Monte Carlo methodology through practical implementation. Not
only have we introduced R at the core of this book, but the emphasis and
contents have changed drastically from Monte Carlo Statistical Methods, even
though the overall vision remains the same. Theoretical foundations are in-
tentionally avoided in the current book.
Indeed, the emphasis on practice is a major feature of this book in that
its primary audience consists of graduate students in statistics, biostatistics,
engineering, etc., who need to learn how to use simulation methods as a tool
to analyze their experiments and/or datasets. The book should appeal to
scientists in all fields, given the versatility of these Monte Carlo tools. It can
also be used for a more classical statistics audience when aimed at teaching a
quick entry into modern computational methods based on R, at the end of an
undergraduate program for example, even though this may prove challenging
for some students.
The choice of the programming language R, as opposed to faster alterna-
tives such as Matlab or C and more structured constructs such as BUGS, is
due to its pedagogical simplicity and its versatility. Readers can easily con-
duct experiments in their own programming language by translating the ex-
amples provided in this book. (Obviously, the book can also supplement other
textbooks on Bayesian modeling at the graduate level, including our books
Bayesian Choice (Robert, 2001) and Monte Carlo Statistical Methods (Robert
viii
Preface
and Casella, 2004).) This book can also be viewed as a companion to, rather
than a competitor of, Jim Albert’s Use R! book Bayesian Computation with
R (Albert, 2009). Indeed, taken as a pair, these two books can provide a fairly
thorough introduction to Monte Carlo methods and Bayesian modeling.
We stress that, at a production level (that is, when using advanced Monte
Carlo techniques or analyzing large datasets), R cannot be recommended as
the default language, but the expertise gained from this book should make
the switch to another language seamless.
Contrary to usual practice, many exercises are interspersed within the
chapters rather than postponed until the end of each chapter. There are two
reasons for this stylistic choice. First, the results or developments contained in
those exercises are often relevant for upcoming points in the chapter. Second,
they signal to the student (or to any reader) that some pondering over the
previous pages may be useful before moving to the following topic and so may
act as self-checking gateways. Additional exercises are found at the end of
each chapter, with abridged solutions of the odd-numbered exercises provided
on our Webpages as well as Springer’s.
Thanks
We are immensely grateful to colleagues and friends for their help with this
book, in particular to the following people: Ed George for his comments on the
general purpose of the book and some exercises in particular; Jim Hobert and
Fernando Quintana for helpful discussions on the Monte Carlo EM; Alessandra
Iacobucci for signaling in due time a fatal blunder; Jean-Michel Marin for
letting us recycle the first chapter of Bayesian Core (Marin and Robert, 2007)
into our introduction to R and for numerous pieces of R advice, as well as
pedagogical suggestions; Antonietta Mira for pointing out mistakes during
a session of an MCMC conference in Warwick; Fran¸cois Perron for inviting
CPR to Montr´eal and thus providing him with a time window to complete
Chapter 8 (only shortened by an ice-climbing afternoon in Qu´eb´ec!), and also
Fran¸cois Perron and Cl´ementine Trimont for testing the whole book from
the perspectives of a professor and a student, respectively; Martyn Plummer
for answering queries about coda; Jeff Rosenthal for very helpful exchanges
on amcmc; Dimitris Rizopoulos for providing Exercise 7.21; and Phil Spector
from Berkeley for making available his detailed and helpful notes and slides
on R and C, now partly published as Spector (2009). Comments from both
reviewers were especially useful in finalizing the book. We are also grateful to
John Kimmel of Springer for his advice and efficiency, as well as for creating
the visionary Use R! series and supporting the development of the R language
that way. From a distance, we also wish to thank Professors Gentleman and
Ihaka for creating the R language and for doing it in open-source, as well as
the entire R community for contributing endlessly to its development.
Sceaux and Gainesville
October 18, 2009
Christian P. Robert and George Casella
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
ii
ii
List of Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
1
1 Basic R Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.2 Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.3 R objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
1.3.1 The vector class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
1.3.2 The matrix, array, and factor classes . . . . . . . . . . . . . . . . . .
9
1.3.3 The list and data.frame classes . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Probability distributions in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.5 Basic and not-so-basic statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.6 Graphical facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.7 Writing new R functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
1.8 Input and output in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.9 Administration of R objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.10 The mcsm package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.11 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2 Random Variable Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.1.1 Uniform simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.1.2 The inverse transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.2 General transformation methods. . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.2.1 A normal generator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.2.2 Discrete distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.2.3 Mixture representations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.3 Accept–reject methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
x
Contents
2.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3 Monte Carlo Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.2 Classical Monte Carlo integration . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3 Importance sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.3.1 An arbitrary change of reference measure . . . . . . . . . . . . . 69
3.3.2 Sampling importance resampling . . . . . . . . . . . . . . . . . . . . 75
3.3.3 Selection of the importance function . . . . . . . . . . . . . . . . . 78
3.4 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4 Controlling and Accelerating Convergence . . . . . . . . . . . . . . . . . 89
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.2 Monitoring variation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.3 Asymptotic variance of importance sampling estimators . . . . . . 92
4.4 Effective sample size and perplexity . . . . . . . . . . . . . . . . . . . . . . . . 98
4.5 Simultaneous monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.6 Rao–Blackwellization and deconditioning . . . . . . . . . . . . . . . . . . . 107
4.7 Acceleration methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.7.1 Correlated simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.7.2 Antithetic variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.7.3 Control variates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.8 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5 Monte Carlo Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
5.2 Numerical optimization methods . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.3 Stochastic search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.3.1 A basic solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.3.2 Stochastic gradient methods . . . . . . . . . . . . . . . . . . . . . . . . 136
5.3.3 Simulated annealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.4 Stochastic approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
5.4.1 Optimizing Monte Carlo approximations . . . . . . . . . . . . . 146
5.4.2 Missing-data models and demarginalization . . . . . . . . . . . 150
5.4.3 The EM algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.4.4 Monte Carlo EM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
5.5 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6 Metropolis–Hastings Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
6.2 A peek at Markov chain theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
6.3 Basic Metropolis–Hastings algorithms . . . . . . . . . . . . . . . . . . . . . . 170
6.3.1 A generic Markov chain Monte Carlo algorithm . . . . . . . 171
6.3.2 The independent Metropolis–Hastings algorithm . . . . . . 175
6.4 A selection of candidates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
Contents
xi
6.4.1 Random walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6.4.2 Alternative candidates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
6.5 Acceptance rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
6.6 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
7 Gibbs Samplers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
7.2 The two-stage Gibbs sampler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
7.3 The multistage Gibbs sampler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
7.4 Missing data and latent variables . . . . . . . . . . . . . . . . . . . . . . . . . . 209
7.5 Hierarchical structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.6 Other considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
7.6.1 Reparameterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
7.6.2 Rao–Blackwellization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
7.6.3 Metropolis within Gibbs and hybrid strategies . . . . . . . . 230
7.6.4 Improper priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
7.7 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
8 Monitoring and Adaptation for MCMC Algorithms . . . . . . . . 237
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
8.2 Monitoring what and why . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
8.2.1 Convergence to the stationary distribution . . . . . . . . . . . . 238
8.2.2 Convergence of averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.2.3 Approximating iid sampling . . . . . . . . . . . . . . . . . . . . . . . . 240
8.2.4 The coda package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
8.3 Monitoring convergence to stationarity . . . . . . . . . . . . . . . . . . . . . 242
8.3.1 Graphical diagnoses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
8.3.2 Nonparametric tests of stationarity . . . . . . . . . . . . . . . . . . 243
8.3.3 Spectral analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
8.4 Monitoring convergence of averages . . . . . . . . . . . . . . . . . . . . . . . . 250
8.4.1 Graphical diagnoses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
8.4.2 Within and between variances . . . . . . . . . . . . . . . . . . . . . . 253
8.4.3 Effective sample size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
8.4.4 Fixed-width batch means . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
8.5 Adaptive MCMC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
8.5.1 Cautions about adaptation . . . . . . . . . . . . . . . . . . . . . . . . . 258
8.5.2 The amcmc package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
8.6 Additional exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Index of R Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Index of Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
