Springer Texts in Statistics
Series Editors:
G. Casella
S. Fienberg
I. Olkin
For further volumes:
http://www.springer.com/series/417
Richard Durrett
Essentials of Stochastic
Processes
Second Edition
123
Richard Durrett
Duke University
Department of Mathematics
Box 90320
Durham
North Carolina
USA
ISSN 1431-875X
ISBN 978-1-4614-3614-0
DOI 10.1007/978-1-4614-3615-7
Springer New York Heidelberg Dordrecht London
ISBN 978-1-4614-3615-7 (eBook)
Library of Congress Control Number: 2012937472
© Springer Science+Business Media, LLC 1999, 2012
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
70
60
50
40
30
20
10
0
500
10-Sep
10-Jun
10-May
at expiry
520
540
560
580
600
620
640
660
680
700
Preface
Between the first undergraduate course in probability and the first graduate course
that uses measure theory, there are a number of courses that teach Stochastic
Processes to students with many different interests and with varying degrees of
mathematical sophistication. To allow readers (and instructors) to choose their own
level of detail, many of the proofs begin with a nonrigorous answer to the question
“Why is this true?” followed by a Proof that fills in the missing details. As it is
possible to drive a car without knowing about the working of the internal combustion
engine, it is also possible to apply the theory of Markov chains without knowing
the details of the proofs. It is my personal philosophy that probability theory was
developed to solve problems, so most of our effort will be spent on analyzing
examples. Readers who want to master the subject will have to do more than a
few of the 20 dozen carefully chosen exercises.
This book began as notes I typed in the spring of 1997 as I was teaching ORIE
361 at Cornell for the second time. In Spring 2009, the mathematics department
there introduced its own version of this course, MATH 474. This started me on
the task of preparing the second edition. The plan was to have this finished in
Spring 2010 after the second time I taught the course, but when May rolled around
completing the book lost out to getting ready to move to Durham after 25 years
in Ithaca. In the Fall of 2011, I taught Duke’s version of the course, Math 216, to
20 undergrads and 12 graduate students and over the Christmas break the second
edition was completed.
The second edition differs substantially from the first, though curiously the length
and the number of problems has remained roughly constant. Throughout the book
there are many new examples and problems, with solutions that use the TI-83 to
eliminate the tedious details of solving linear equations by hand. My students tell
me I should just use MATLAB and maybe I will for the next edition.
The Markov chains chapter has been reorganized. The chapter on Poisson
processes has moved up from third to second, and is now followed by a treatment of
the closely related topic of renewal theory. Continuous time Markov chains remain
fourth, with a new section on exit distributions and hitting times, and reduced
coverage of queueing networks. Martingales, a difficult subject for students at this
vii