Springer Texts in Statistics
Advisors:
George Casella Stephen Fienberg
Ingram Olkin
Springer Texts in Statistics
Alfred: Elements of Statistics for the Life and Social Sciences
Berger: An Introduction to Probability and Stochastic Processes
Bilodeau and Brenner: Theory of Multivariate Statistics
Blom: Probability and Statistics: Theory and Applications
Brockwell and Davis: Introduction to Times Series and Forecasting, Second
Edition
Chow and Teicher: Probability Theory: Independence, Interchangeability,
Martingales, Third Edition
Christensen: Advanced Linear Modeling: Multivariate, Time Series, and Spatial
Data: Nonparametric Regression and Response Surface Maximization, Second
Edition
Christensen: Log-Linear Models and Logistic Regression, Second Edition
Christensen: Plane Answers to Complex Questions: The Theory of Linear
Models, Third Edition
Creighton: A First Course in Probability Models and Statistical Inference
Davis: Statistical Methods for the Analysis of Repeated Measurements
Dean and Voss: Design and Analysis of Experiments
du Toit, Steyn, and Stumpf: Graphical Exploratory Data Analysis
Durrett: Essentials of Stochastic Processes
Edwards: Introduction to Graphical Modelling, Second Edition
Finkelstein and Levin: Statistics for Lawyers
Flury: A First Course in Multivariate Statistics
Jobson: Applied Multivariate Data Analysis, Volume I: Regression and
Jobson: Applied Multivariate Data Analysis, Volume II: Categorical and
Experimental Design
Multivariate Methods
Edition
Kalbfleisch: Probability and Statistical Inference, Volume I: Probability, Second
Kalbfleisch: Probability and Statistical Inference, Volume II: Statistical
Inference, Second Edition
Karr: Probability
Keyfitz: Applied Mathematical Demography, Second Edition
Kiefer: Introduction to Statistical Inference
Kokoska and Nevison: Statistical Tables and Formulae
Kulkarni: Modeling, Analysis, Design, and Control of Stochastic Systems
Lange: Applied Probability
Lehmann: Elements of Large-Sample Theory
Lehmann: Testing Statistical Hypotheses, Second Edition
Lehmann and Casella: Theory of Point Estimation, Second Edition
Lindman: Analysis of Variance in Experimental Design
Lindsey: Applying Generalized Linear Models
(continued after index)
Jun Shao
Mathematical Statistics
Second Edition
Jun Shao
Department of Statistics
University of Wisconsin, Madison
Madison, WI 53706-1685
USA
shao@stat.wisc.edu
Editorial Board
George Casella
Department of Statistics
University of Florida
Gainesville, FL 32611-8545
USA
With 7 figures.
Stephen Fienberg
Department of Statistics
Carnegie Mellon University Stanford University
Pittsburgh, PA 15213-3890
Stanford, CA 94305
USA
USA
Ingram Olkin
Department of Statistics
Library of Congress Cataloging-in-Publication Data
Shao, Jun.
Mathematical statistics / Jun Shao.—2nd ed.
p.
cm.— (Springer texts in statistics)
Includes bibliographical references and index.
ISBN 0-387-95382-5 (alk. paper)
1. Mathematical statistics.
QA276.S458
519.5—dc21
I. Title.
2003
2003045446
II. Series.
ISBN 0-387-95382-5
Printed on acid-free paper.
ISBN-13 978-0-387-95382-3
© 2003 Springer Science+Business Media, LLC.
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer Science+Business Media, LLC., 233 Spring St.,
New York, N.Y., 10013, USA), except for brief excerpts in connection with reviews or scholarly
analysis.
information storage and retrieval, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of opinion as to whether or
not they are subject to proprietary rights.
Use in connection with any form of
Printed in the United States of America.
9 8 7 6 5 4(corrected printing as of 4 th printing, 2007)
springer.com
To Guang, Jason, and Annie
Preface to the First
Edition
This book is intended for a course entitled Mathematical Statistics offered
at the Department of Statistics, University of Wisconsin-Madison. This
course, taught in a mathematically rigorous fashion, covers essential ma-
terials in statistical theory that a first or second year graduate student
typically needs to learn as preparation for work on a Ph.D. degree in statis-
tics. The course is designed for two 15-week semesters, with three lecture
hours and two discussion hours in each week. Students in this course are
assumed to have a good knowledge of advanced calculus. A course in real
analysis or measure theory prior to this course is often recommended.
Chapter 1 provides a quick overview of important concepts and results
in measure-theoretic probability theory that are used as tools in math-
ematical statistics. Chapter 2 introduces some fundamental concepts in
statistics, including statistical models, the principle of sufficiency in data
reduction, and two statistical approaches adopted throughout the book:
statistical decision theory and statistical inference. Each of Chapters 3
through 7 provides a detailed study of an important topic in statistical de-
cision theory and inference: Chapter 3 introduces the theory of unbiased
estimation; Chapter 4 studies theory and methods in point estimation un-
der parametric models; Chapter 5 covers point estimation in nonparametric
settings; Chapter 6 focuses on hypothesis testing; and Chapter 7 discusses
interval estimation and confidence sets. The classical frequentist approach
is adopted in this book, although the Bayesian approach is also introduced
(§2.3.2, §4.1, §6.4.4, and §7.1.3). Asymptotic (large sample) theory, a cru-
cial part of statistical inference, is studied throughout the book, rather than
in a separate chapter.
About 85% of the book covers classical results in statistical theory that
are typically found in textbooks of a similar level. These materials are in the
Statistics Department’s Ph.D. qualifying examination syllabus. This part
of the book is influenced by several standard textbooks, such as Casella and
vii
viii
Preface to the First Edition
Berger (1990), Ferguson (1967), Lehmann (1983, 1986), and Rohatgi (1976).
The other 15% of the book covers some topics in modern statistical theory
that have been developed in recent years, including robustness of the least
squares estimators, Markov chain Monte Carlo, generalized linear models,
quasi-likelihoods, empirical likelihoods, statistical functionals, generalized
estimation equations, the jackknife, and the bootstrap.
In addition to the presentation of fruitful ideas and results, this book
emphasizes the use of important tools in establishing theoretical results.
Thus, most proofs of theorems, propositions, and lemmas are provided
or left as exercises. Some proofs of theorems are omitted (especially in
Chapter 1), because the proofs are lengthy or beyond the scope of the
book (references are always provided). Each chapter contains a number of
examples. Some of them are designed as materials covered in the discussion
section of this course, which is typically taught by a teaching assistant (a
senior graduate student). The exercises in each chapter form an important
part of the book. They provide not only practice problems for students,
but also many additional results as complementary materials to the main
text.
The book is essentially based on (1) my class notes taken in 1983-84
when I was a student in this course, (2) the notes I used when I was a
teaching assistant for this course in 1984-85, and (3) the lecture notes I
prepared during 1997-98 as the instructor of this course. I would like to
express my thanks to Dennis Cox, who taught this course when I was
a student and a teaching assistant, and undoubtedly has influenced my
teaching style and textbook for this course.
I am also very grateful to
students in my class who provided helpful comments; to Mr. Yonghee Lee,
who helped me to prepare all the figures in this book; to the Springer-Verlag
production and copy editors, who helped to improve the presentation; and
to my family members, who provided support during the writing of this
book.
Madison, Wisconsin
January 1999
Jun Shao
Preface to the Second
Edition
In addition to correcting typos and errors and making a better presentation,
the main effort in preparing this new edition is adding some new material
to Chapter 1 (Probability Theory) and a number of new exercises to each
chapter. Furthermore, two new sections are created to introduce semipara-
metric models and methods (§5.1.4) and to study the asymptotic accuracy
of confidence sets (§7.3.4). The structure of the book remains the same.
In Chapter 1 of the new edition, moment generating and characteristic
functions are treated in more detail and a proof of the uniqueness theorem
is provided; some useful moment inequalities are introduced; discussions
on conditional independence, Markov chains, and martingales are added,
as a continuation of the discussion of conditional expectations; the con-
cepts of weak convergence and tightness are introduced; proofs to some key
results in asymptotic theory, such as the dominated convergence theorem
and monotone convergence theorem, the L´evy-Cram´er continuity theorem,
the strong and weak laws of large numbers, and Lindeberg’s central limit
theorem, are included; and a new section (§1.5.6) is created to introduce
Edgeworth and Cornish-Fisher expansions. As a result, Chapter 1 of the
new edition is self-contained for important concepts, results, and proofs in
probability theory with emphasis in statistical applications.
Since the original book was published in 1999, I have been using it as
a textbook for a two-semester course in mathematical statistics. Exercise
problems accumulated during my teaching are added to this new edition.
Some exercises that are too trivial have been removed.
In the original book, indices on definitions, examples, theorems, propo-
sitions, corollaries, and lemmas are included in the subject index. In the
new edition, they are in a separate index given in the end of the book (prior
to the author index). A list of notation and a list of abbreviations, which
are appendices of the original book, are given after the references.
ix