Springer Texts in Statistics
Advisors:
George Casella Stephen Fienberg
Ingram Olkin
Springer Texts in Statistics
Alfred: Elements of Statistics for the Life and Social Sciences
Athreya and Lahiri: Measure Theory and Probability Theory
Berger: An Introduction to Probability and Stochastic Processes
Bilodeau and Brenner: Theory of Multivariate Statistics
Blom: Probability and Statistics: Theory and Applications
Brockwell and Davis: Introduction to Times Series and Forecasting, Second Edition
Carmona: Statistical Analysis of Financial Data in S-Plus
Chow and Teicher: Probability Theory: Independence, Interchangeability,
Martingales, Third Edition
Christensen: Advanced Linear Modeling: Multivariate, Time Series, and
Spatial Data—Nonparametric Regression and Response Surface
Maximization, Second Edition
Christensen: Log-Linear Models and Logistic Regression, Second Edition
Christensen: Plane Answers to Complex Questions: The Theory of Linear
Models, Third Edition
Creighton: A First Course in Probability Models and Statistical Inference
Davis: Statistical Methods for the Analysis of Repeated Measurements
Dean and Voss: Design and Analysis of Experiments
du Toit, Steyn, and Stumpf: Graphical Exploratory Data Analysis
Durrett: Essentials of Stochastic Processes
Edwards: Introduction to Graphical Modelling, Second Edition
Finkelstein and Levin: Statistics for Lawyers
Flury: A First Course in Multivariate Statistics
Ghosh, Delampady and Samanta: An Introduction to Bayesian Analysis:
Theory and Methods
Gut: Probability: A Graduate Course
Heiberger and Holland: Statistical Analysis and Data Display:
An Intermediate Course with Examples in S-PLUS, R, and SAS
Jobson: Applied Multivariate Data Analysis, Volume I: Regression and
Experimental Design
Jobson: Applied Multivariate Data Analysis, Volume II: Categorical and
Multivariate Methods
Kalbfleisch: Probability and Statistical Inference, Volume I: Probability,
Second Edition
Kalbfleisch: Probability and Statistical Inference, Volume II: Statistical
Inference, Second Edition
Karr: Probability
Keyfitz: Applied Mathematical Demography, Second Edition
Kiefer: Introduction to Statistical Inference
Kokoska and Nevison: Statistical Tables and Formulae
Kulkarni: Modeling, Analysis, Design, and Control of Stochastic Systems
Lange: Applied Probability
Lange: Optimization
Lehmann: Elements of Large-Sample Theory
(continued after index)
Krishna B. Athreya
Soumendra N. Lahiri
Measure Theory
and Probability Theory
Krishna B. Athreya
Department of Mathematics and
Department of Statistics
Iowa State University
Ames, IA 50011
kba@iastate.edu
Soumendra N. Lahiri
Department of Statistics
Iowa State University
Ames, IA 50011
snlahiri@iastate.edu
Editorial Board
George Casella
Department of Statistics
University of Florida
Gainesville, FL 32611-8545 Pittsburgh, PA 15213-3890
USA
Stephen Fienberg
Department of Statistics
Carnegie Mellon University Stanford University
Stanford, CA 94305
USA
Ingram Olkin
Department of Statistics
USA
Library of Congress Control Number: 2006922767
ISBN-10: 0-387-32903-X
ISBN-13: 978-0387-32903-1
e-ISBN: 0-387-35434-4
Printed on acid-free paper.
©2006 Springer Science+Business Media, LLC
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street,
New York, NY 10013, USA), except for brief excepts in connection with reviews or scholarly
analysis. Use in connection with any form of information storage and retrieval, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of opinion as to whether or
not they are subject to proprietary rights.
Printed in the United States of America.
(MVY)
9 8 7 6 5 4 3 2 1
springer.com
Dedicated to our wives
Krishna S. Athreya and Pubali Banerjee
and
to the memory of
Uma Mani Athreya and Narayani Ammal
Preface
This book arose out of two graduate courses that the authors have taught
during the past several years; the first one being on measure theory followed
by the second one on advanced probability theory.
The traditional approach to a first course in measure theory, such as in
Royden (1988), is to teach the Lebesgue measure on the real line, then the
differentation theorems of Lebesgue, Lp-spaces on R, and do general mea-
sure at the end of the course with one main application to the construction
of product measures. This approach does have the pedagogic advantage
of seeing one concrete case first before going to the general one. But this
also has the disadvantage in making many students’ perspective on mea-
sure theory somewhat narrow. It leads them to think only in terms of the
Lebesgue measure on the real line and to believe that measure theory is
intimately tied to the topology of the real line. As students of statistics,
probability, physics, engineering, economics, and biology know very well,
there are mass distributions that are typically nonuniform, and hence it is
useful to gain a general perspective.
This book attempts to provide that general perspective right from the
beginning. The opening chapter gives an informal introduction to measure
and integration theory. It shows that the notions of σ-algebra of sets and
countable additivity of a set function are dictated by certain very natu-
ral approximation procedures from practical applications and that they
are not just some abstract ideas. Next, the general extension theorem of
Carathedory is presented in Chapter 1. As immediate examples, the con-
struction of the large class of Lebesgue-Stieltjes measures on the real line
and Euclidean spaces is discussed, as are measures on finite and countable
viii
Preface
spaces. Concrete examples such as the classical Lebesgue measure and var-
ious probability distributions on the real line are provided. This is further
developed in Chapter 6 leading to the construction of measures on sequence
spaces (i.e., sequences of random variables) via Kolmogorov’s consistency
theorem.
After providing a fairly comprehensive treatment of measure and inte-
gration theory in the first part (Introduction and Chapters 1–5), the focus
moves onto probability theory in the second part (Chapters 6–13). The fea-
ture that distinguishes probability theory from measure theory, namely, the
notion of independence and dependence of random variables (i.e., measure-
able functions) is carefully developed first. Then the laws of large numbers
are taken up. This is followed by convergence in distribution and the central
limit theorems. Next the notion of conditional expectation and probability
is developed, followed by discrete parameter martingales. Although the de-
velopment of these topics is based on a rigorous measure theoretic founda-
tion, the heuristic and intuitive backgrounds of the results are emphasized
throughout. Along the way, some applications of the results from probabil-
ity theory to proving classical results in analysis are given. These include,
for example, the density of normal numbers on (0,1) and the Wierstrass
approximation theorem. These are intended to emphasize the benefits of
studying both areas in a rigorous and combined fashion. The approach
to conditional expectation is via the mean square approximation of the
“unknown” given the “known” and then a careful approximation for the
L1-case. This is a natural and intuitive approach and is preferred over the
“black box” approach based on the Radon-Nikodym theorem.
The final part of the book provides a basic outline of a number of special
topics. These include Markov chains including Markov chain Monte Carlo
(MCMC), Poisson processes, Brownian motion, bootstrap theory, mixing
processes, and branching processes. The first two parts can be used for a
two-semester sequence, and the last part could serve as a starting point for
a seminar course on special topics.
This book presents the basic material on measure and integration theory
and probability theory in a self-contained and step-by-step manner. It is
hoped that students will find it accessible, informative, and useful and also
that they will be motivated to master the details by carefully working out
the text material as well as the large number of exercises. The authors hope
that the presentation here is found to be clear and comprehensive without
being intimidating.
Here is a quick summary of the various chapters of the book. After giving
an informal introduction to the ideas of measure and integration theory,
the construction of measures starting with set functions on a small class of
sets is taken up in Chapter 1 where the Caratheodory extension theorem is
proved and then applied to construct Lebesgue-Stieltjes measures. Integra-
tion theory is taken up in Chapter 2 where all the basic convergence the-
orems including the MCT, Fatou, DCT, BCT, Egorov’s, and Scheffe’s are