Convergence of Probability Measures
WILEY SERIES IN PROBABILITY AND STATISTICS
PROBABILITY AND STATISTICS SECTION
Established by WALTER A. SHEWHART and SAMUEL S. WILKS
Editors: Vic Barnett, Noel A. C. Cressie, Nicholas I. Fisher,
Iain M, Johnstone, J. B. Kadane, David G. Kendall, David W. Scott,
Bernard W. Silverman, Adrian F. M. Smith, Jozef L. Teugels;
Ralph A. Bradley, Emeritus, J. Stuart Hunter, Emeritus
A complete list of the titles in this series appears at the end of this volume.
Convergence of
Probability Measures
Second Edition
PATRICK BILLINGSLEY
TEe University of Chicago
Chicago, Illinois
A Wiley-Interscience Publication
JOHN WILEY & SONS, INC.
NewYork Chichester Weinheim Brisbane
Singapore Toronto
This text is printed on acid-free paper. 8
Copyright 0 1999 by John Wiley & Sons, Inc.
All rights reserved. Published simultaneously in Canada.
No patt of this publication may be reproduced, stored in a retrieval system or transmitted in any
form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise,
except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without
either the prior written permission of the Publisher, or authorization through payment of the
appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA
01923, (978) 750-8400, fax (978) 750-4744. Requests to the Publisher for permission should be
addressed to the Permissions Department, John Wiley & Sons, Inc., 605 Third Avenue, New York,
NY 10158-0012, (212) 850-601 1, fax (212) 850-6008, E-Mail: PERMREQ @, WILEY.COM.
For ordering and customer service, call 1-800-CALL-WILEY.
Library of Congress Cataloging in Publication Data:
Billingsley, Patrick.
Convergence of probability measures / Patrick Billingsley. - 2nd
ed.
p. cm. - (Wiley series in probability and statistics.
Probability and statistics)
“Wiley-Interscience publication.”
Includes bibliographical references and indexes.
ISBN 0-471-19745-9 (alk. paper)
1. Probability measures. 2. Metric spaces. 3. Convergence.
I. Title. 11. Series: Wiley series in probability and statistics.
Probability and statistics.
QA273.6.BS5 1999
5 1 9 . 2 4 ~ 2 1
99-30372
CIP
Printed in the United States of America
1 0 9 8 7 6
PREFACE
From the preface to the first edition. Asymptotic distribution theo-
rems in probability and statistics have from the beginning depended
on the classical theory of weak convergence of distribution functions
in Euclidean spat-onvergence, that is, at continuity points of the
limit function. The past several decades have seen the creation and
extensive application of a more inclusive theory of weak convergence
of probability measures on metric spaces. There are many asymptotic
results that can be formulated within the classical theory but require
for their proofs this more general theory, which thus does not merely
study itself. This book is about weak-convergence methods in metric
spaces, with applications sufficient to show their power and utility.
The second edition. A person who read the first edition of this
book when it appeared thirty years ago could move directly on to
the periodical literature and to research in the subject. Although the
book no longer takes the reader to the current boundary of what is
known in this area of probability theory, I think it is still useful as a
textbook one can study before tackling the industrial-strength treatises
now available. For the second edition I have reworked most of the
sections, clarifying some and shortening others (most notably the ones
on dependent random variables) by using discoveries of the last thirty
years, and I have added some new topics. I have written with students
in mind; for example, instead of going directly to the space D [ 0, oo),
I have moved, in what I hope are easy stages, from C[O, I] to D[O, 11
to D[O,oo). In an earlier book of mine, I said that I had tried to
follow the excellent example of Hardy and Wright, who wrote their
Introduction to the Theory of Numbers with the avowed aim, as they
say in the preface, of producing an interesting book, and I have again
taken them as my model.
Chicago,
January 1999
Patrick Billingsley
V
For mathematical information and advice, I thank Richard Arratia,
Peter Donnelly, Walter Philipp, Simon Tavar6, and Michael Wichura.
For essential help on Tex and the figures, I thank Marty Billingsley
and Mitzi Nakatsuka.
PB
CONTENTS
Introduction
Chapter 1. Weak convergence in Metric Spaces
Section 1. Measures on Metric Spaces, 7
1
7
Measures and Integrals. Tightness. Some Examples. Problems.
Section 2. Properties of Weak Convergence, 14
The Portmanteau Theorem. Other Criteria.
Theorem. Product Spaces. Problems.
,The Mapping
Section 3. Convergence in Distribution, 24
Random Elements. Convergence in Distribution. Convergence
in Probability. Local us. Integral Laws. Integration to the Limit.
Relative measure.* Three Lemmas.* Problems.
Section 4. Long Cycles and Large Divisors,* 38
Long Cycles. The Space A. The Poisson-Dirichlet Distribu-
tion. Size-Biased Sampling. Large Prime Divisors. Technical
Arguments. Problems.
Section 5 . Prohorov’s Theorem, 57
Relative Compactness. Tightness. The Proof. Problems.
Section 6. A Miscellany,* 65
The Ball a-Field. Slcorohod’s Representation Theorem. The
Prohorov Metric. A Coupling Theorem. Problems.
Chapter 2. The Space C
Section 7 . Weak Convergence and Tightness in C, 80
* Starred topics can be omitted on a first reading
80
vii
viii
CONTENTS
Tightness and Compactness in C . Random Functions. Coordi-
nate Variables. Problems.
Section 8. Wiener Measure and Donsker’s Theorem, 86
Wiener Measure. Construction of Wiener Measure. Donsker ’s
Theorem. A n Application. The Brownian Bridge. Problems.
Section 9. Functions of Brownian Motion Paths, 94
Maximum and Minimum. The Arc Sane Law. The Brownian
Bridge. Problems.
Section 10. Maximal Inequalities,lO5
Maxima of Partial Sums. A More General Inequality. A Fur-
ther Inequality. Problems.
Section 11. Trigonometric Series:
113
Lacunary Series. Incommensurabl e Arguments. Problem.
Chapter 3. The Space D
Section 12. The Geometry of D, 121
121
The Definition. The Skorohod Topology. Separability and Com-
pleteness of D. Compactness in D. A Second Characterization
of Compactness. Finite-Dimensional Sets. Random Functions
in D. The Poisson Lamat.* Problems.
Section 13. Weak Convergence and Tightness in D, 138
Finite-Dimensional Distributions. Tightness. A Criterion for
Convergence. A Criterion for Existence.“ Problem.
Section 14. Applications, 146
Donsker’s Theorem Again. A n Extension. Dominated Mea-
sures. Empirical Distribution Functions. Random Change of
Time. Renewal Theory. Problems.
Section 15. Uniform Topologies,” 156
The Uniform Metric on D[ 0,1]. A Theorem of Dudley’s. Em-
pirical Processes Indexed by Convex Sets.
Section 16. The Space D[ 0, oo), 166
Definitions. Properties of the Metric. SeparabilitQ and Com-
pleteness. Compactness. Finite-Dimensional Sets. Weak Con-
vergence. Tightness. Aldous ’s Tightness Criterion.