Springer Texts in Statistics
Advisors:
George Casella Stephen Fienberg Ingram Olkin
Springer Texts in Statistics
AJfred: Elements of Statistics for the Life and Social Sciences
Berger: An Introduction to Probability and Stochastic Processes
Bilodeau and Brenner: Theory of Multivariate Statistics
BIom: Probability and Statistics: Theory and Applications
Brockwell and Davis: Introduction to Times Series and Forecasting,
Second Edition
Chow and Teicher: Probability Theory: Independence, Interchangeability,
Martingales, Third Edition
Christensen: Advanced Linear Modeling: Multivariate, Time Series, and
Spatial Data; Nonparametric Regression and Response Surface
Maximization, Second Edition
Christensen: Log-Linear Models and Logistic Regression, Second Edition
Christensen: Plane Answers to Complex Questions: The Theory of Linear
Models, Third Edition
Creighton: A First Course in Probability Models and Statistical Inference
Davis: Statistical Methods for the Analysis of Repeated Measurements
Dean and Voss: Design and Analysis of Experiments
du Toit, Steyn, and Stumpf Graphical Exploratory Data Analysis
Durrett: Essentials of Stochastic Processes
Edwards: Introduction to Graphical Modelling, Second Edition
Finkelstein and Levin: Statistics for Lawyers
Flury: A First Course in Multivariate Statistics
Jobson: Applied Multivariate Data Analysis, Volume I: Regression and
Experimental Design
Jobson: Applied Multivariate Data Analysis, Volume II: Categorical and
Multivariate Methods
Kalbfleisch: Probability and Statistical Inference, Volume I: Probability,
Second Edition
Kalbfleisch: Probability and Statistical Inference, Volume II : Statistical Inference,
Second Edition
Karr: Probability
Keyfitz: Applied Mathematical Demography, Second Edition
Kiefer: Introduction to Statistical Inference
Kokoska and Nevison: Statistical Tables and Fonnulae
Kulkarni: Modeling, Analysis, Design, and Control of Stochastic Systems
Lange: Applied Probability
Lehmann: Elements of Large-Sample Theory
Lehmann: Testing Statistical Hypotheses, Second Edition
Lehmann and Casella: Theory of Point Estimation, Second Edition
Lindman: Analysis of Variance in Experimental Design
Lindsey: Applying Generalized Linear Models
(continued after index)
Larry Wassennan
All of Statistics
A Concise Course in Statistical Inference
With 95 Figures
,
Springer
Larry W assennan
Department of Statistics
Carnegie Mellon University
Baker Hali 228A
Pittsburgh, PA 15213-3890
USA
larry@ stat.cmu.edu
Editorial Board
George Casella
Department of Statistics
University of Florida
Gainesville, FL 32611-8545
USA
Stephen Fienberg
Department of Statistics
Carnegie Mellon University
Pittsburgh, PA 15213-3890
USA
Ingram Olkin
Department of Statistics
Stanford University
Stanford, CA 94305
USA
Library of Congress Cataloging-in-Publication Data
Wasserman, Larry A. (Larry Alan), 1959-
All of statistics: a concise course in statistica! inference 1 Larry a. W asserman.
p. cm. -
(Springer texts in statistics)
Includes bibliographical references and index.
1. Mathematical statistics. 1. Title.
QA276.12.W37 2003
519.5-dc21
Il. Series.
2003062209
ISBN 978-0-387-21736-9 (eBook)
ISBN 978-1-4419-2322-6
DOI 10.1007/978-0-387-21736-9
© 2004 Springer Science+Business Media New York
Originally published by Springer Science+Business Media, !ne in 2004
Softcover reprint of the hardcover 1 st edition 2004
All rights reserved. This work may not be translated or copied in whole or in part without the
written permission of the publisher (Springer Science+ Business Media, LLC ), except for brief
excerpts in connection with reviews or scholarly analysis.
Use in connection with any form of information storage and retrieval, electronic adaptation, com-
puter software, or by similar or dissimilar methodology now known or hereafter developed is for-
bidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of opinion as to whether or not
they are subject to proprietary rights.
9 8 7 6 5 4 3
(Corrected second printing, 2005)
springeronline.com
To Isa
Preface
Taken literally, the title "All of Statistics" is an exaggeration. But in spirit,
the title is apt, as the book does cover a much broader range of topics than a
typical introductory book on mathematical statistics.
This book is for people who want to learn probability and statistics quickly.
It is suitable for graduate or advanced undergraduate students in computer
science, mathematics, statistics, and related disciplines. The book includes
modern topics like nonparametric curve estimation, bootstrapping, and clas-
sification, topics that are usually relegated to follow-up courses. The reader is
presumed to know calculus and a little linear algebra. No previous knowledge
of probability and statistics is required.
Statistics, data mining, and machine learning are all concerned with
collecting and analyzing data. For some time, statistics research was con-
ducted in statistics departments while data mining and machine learning re-
search was conducted in computer science departments. Statisticians thought
that computer scientists were reinventing the wheel. Computer scientists
thought that statistical theory didn't apply to their problems.
Things are changing. Statisticians now recognize that computer scientists
are making novel contributions while computer scientists now recognize the
generality of statistical theory and methodology. Clever data mining algo-
rithms are more scalable than statisticians ever thought possible. Formal sta-
tistical theory is more pervasive than computer scientists had realized.
Students who analyze data, or who aspire to develop new methods for
analyzing data, should be well grounded in basic probability and mathematical
statistics. Using fancy tools like neural nets, boosting, and support vector
viii
Preface
machines without understanding basic statistics is like doing brain surgery
before knowing how to use a band-aid.
But where can students learn basic probability and statistics quickly? Nowhere.
At least, that was my conclusion when my computer science colleagues kept
asking me: "Where can I send my students to get a good understanding of
modern statistics quickly?" The typical mathematical statistics course spends
too much time on tedious and uninspiring topics (counting methods, two di-
mensional integrals, etc.) at the expense of covering modern concepts (boot-
strapping, curve estimation, graphical models, etc. ). So I set out to redesign
our undergraduate honors course on probability and mathematical statistics.
This book arose from that course. Here is a summary of the main features of
this book.
1. The book is suitable for graduate students in computer science and
honors undergraduates in math, statistics, and computer science. It is
also useful for students beginning graduate work in statistics who need
to fill in their background on mathematical statistics.
2. I cover advanced topics that are traditionally not taught in a first course.
For example, nonparametric regression, bootstrapping, density estima-
tion, and graphical models.
3. I have omitted topics in probability that do not play a central role in
statistical inference. For example, counting methods are virtually ab-
sent.
4. Whenever possible, I avoid tedious calculations in favor of emphasizing
concepts.
5. I cover nonparametric inference before parametric inference.
6. I abandon the usual "First Term = Probability" and "Second Term
= Statistics" approach . Some students only take the fi rst half and it
would be a crime if they did not see any statistical theory. Furthermore,
probability is more engaging when students can see it put to work in the
context of statistics. An exception is the topic of stochastic processes
which is included ill the later material.
7. T he course moves very quickly and covers much material. My colleagues
joke that I cover all of statistics in this course and hence the title. The
course is demanding but I have worked hard to make the material as
intuitive as possible so that the material is very understandable despite
the fast pace.
8. Rigor and clarity are not synonymous. I have tried to strike a good
balance. To avoid getting bogged down in uninteresting technical details,
many results are stated without proof. The bibliographic references at
the end of each chapter point the student to appropriate sources.