Probability and
Stochastic Processes
Features of this Text
Who will benefit from using this text?
What’s New?
This text can be used in Junior, Senior or graduate level courses in probability, stochastic
process, random signal processing and queuing theory. The mathematical exposition will
appeal to students and practioners in many areas. The examples, quizzes, and problems are
typical of those encountered by practicing electrical and computer engineers. Professionals
in the telecommunications and wireless industry will find it particularly useful.
This text has been expanded greatly with new material:
• Matlab examples and problems give students hands-on access to theory and ap-
plications. Every chapter includes guidance on how to use MATLAB to perform
calculations and simulations relevant to the subject of the chapter.
• A new chapter on Random Vectors
• Expanded and enhanced coverage of Random Signal Processing
• Streamlined exposition of Markov Chains and Queuing Theory provides quicker
access to theories of greatest practical importance
Notable Features
The Friendly Approach
The friendly and accessible writing style gives students an intuitive feeling for the
formal mathematics.
Quizzes and Homework Problems
An extensive collection of in-chapter Quizzes provides check points for readers to
gauge their understanding. Hundreds of end-of-chapter problems are clearly marked
as to their degree of difficulty from beginner to expert.
Website for Students http://www.wiley.com/college/yates
Available for download: All Matlab m-files in the text, the Quiz Solutions Manual
Instructor Support
Instructors should register at the Instructor Companion Site (ISC) at Wiley in order to
obtain supplements. The ISC can be reached by accessing the text’s companion web page
http://www.wiley.com/college/yates
• Unparalleled in its offerings, this Second Edition provides a web-based interface for
instructors to create customized solutions documents that output in PDF or PostScript.
• Extensive PowerPoint slides are available.
Probability and
Stochastic Processes
A Friendly Introduction
for Electrical and Computer Engineers
Second Edition
Roy D. Yates
Rutgers, The State University of New Jersey
David J. Goodman
Polytechnic University
JOHN WILEY & SONS, INC.
EXECUTIVE EDITOR
MARKETING MANAGER
PRODUCTION EDITOR
COVER DESIGNER
Bill Zobrist
Jennifer Powers
Ken Santor
Dawn Stanley
This book was set in Times Roman by the authors using LATEXand printed and bound by
Malloy, Inc. The cover was printed by Lehigh Press.
About the cover: The cover shows a cutaway view of a bivariate Gaussian probability den-
sity function. The bell-shaped cross-sections show that the marginal densities are Gaussian.
This book is printed on acid-free paper. ∞
Copyright c 2005 John Wiley & Sons, Inc. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system or transmitted
in any form or by any means, electronic, mechanical, photocopying, recording, scanning
or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States
Copyright Act, without either the prior written permission of the Publisher, or authorization
through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222
Rosewood Drive, Danvers, MA 01923, (978)750-8400, fax (978)750-4470. Requests to the
Publisher for permission should be addressed to the Permissions Department, John Wiley
& Sons, Inc., 111 River Street, Hoboken, NJ 07030 (201)748-6011, fax (201)748-6008,
E-Mail: PERMREQ@WILEY.COM.
To order books or for customer service call 1-800-CALL WILEY (225-5945).
ISBN 0-471-27214-0
WIE 0-471-45259-9
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
To our children,
Tony, Brett, and Zachary Yates
Leila and Alissa Goodman
Preface
What’s new in the second edition?
We are happy to introduce you to the second edition of our textbook. Students and instructors
using the first edition have responded favorably to the “friendly” approach that couples
engineering intuition to mathematical principles. They are especially pleased with the
abundance of exercises in the form of “examples,” “quizzes,” and “problems,” many of
them very simple. The exercises help students absorb the new material in each chapter and
gauge their grasp of it.
Aiming for basic insight, the first edition avoided exercises that require complex com-
putation. Although most of the original exercises have evident engineering relevance, they
are considerably simpler than their real-world counterparts. This second edition adds a
large set of Matlab programs offering students hands-on experience with simulations and
calculations. Matlab bridges the gap between the computationally simple exercises and
the more complex tasks encountered by engineering professionals. The Matlab section
at the end of each chapter presents programs we have written and also guides students to
write their own programs.
Retaining the friendly character of the first edition, we have incorporated into this edition
the suggestions of many instructors and students. In addition to the Matlab programs,
new material includes a presentation of multiple random variables in vector notation. This
format makes the math easier to grasp and provides a convenient stepping stone to the chapter
on stochastic processes, which in turn leads to an expanded treatment of the application of
probability theory to digital signal processing.
Why did we write the book?
When we started teaching the course Probability and Stochastic Processes to Rutgers un-
dergraduates in 1991, we never dreamed we would write a textbook on the subject. Our
bookshelves contain more than a twenty probability texts, many of them directed at elec-
trical and computer engineering students. We respect most of them. However, we have
yet to find one that works well for Rutgers students. We discovered to our surprise that
the majority of our students have a hard time learning the subject. Beyond meeting degree
requirements, the main motivation of most of our students is to learn how to solve practical
problems. For the majority, the mathematical logic of probability theory is, in itself, of
minor interest. What the students want most is an intuitive grasp of the basic concepts and
lots of practice working on applications.
The students told us that the textbooks we assigned, for all their mathematical elegance,
didn’t meet their needs. To help them, we distributed copies of our lecture notes, which
gradually grew into this book. We also responded to students who find that although much
of the material appears deceptively simple, it takes a lot of careful thought and practice to
vii
viii
PREFACE
use the mathematics correctly. Even when the formulas are simple, knowing which ones to
use is difficult. This is a reversal from some mathematics courses, where the equations are
given and the solutions are hard to obtain.
What is distinctive about this book?
• The entire text adheres to a single model that begins with an experiment consisting
of a procedure and observations.
• The mathematical logic is apparent to readers. Every fact is identified clearly as a
definition, an axiom, or a theorem. There is an explanation, in simple English, of the
intuition behind every concept when it first appears in the text.
• The mathematics of discrete random variables are introduced separately from the
mathematics of continuous random variables.
• Stochastic processes and statistical inference fit comfortably within the unifying
model of the text.
• An abundance of exercises puts the theory to use. New ideas are augmented with
detailed solutions of numerical examples. Each section concludes with a simple quiz
to help students gauge their grasp of the new material. The book’s Web site contains
complete solutions of all of the quizzes.
• Each problem at the end of a chapter is labeled with a reference to a section in the
chapter and a degree of difficulty ranging from “easy” to “experts only.”
• There is considerable support on the World Wide Web for students and instructors,
including Matlab files and problem solutions.
How is the book organized?
We estimate that the material in this book represents about 150% of a one-semester under-
graduate course. We suppose that most instructors will spend about two-thirds of a semester
covering the material in the first five chapters. The remainder of a course will be devoted
to about half of the material in the final seven chapters, with the selection depending on the
preferences of the instructor and the needs of the students. Rutgers electrical and computer
engineering students take this course in the first semester of junior year. The following
semester they use much of the material in Principles of Communications.
We have also covered the entire book in one semester in an entry-level graduate course that
places more emphasis on mathematical derivations and proofs than does the undergraduate
course. Although most of the early material in the book is familiar in advance to many
graduate students, the course as a whole brings our diverse graduate student population up
to a shared level of competence.
The first five chapters carry the core material that is common to practically all intro-
ductory engineering courses in probability theory. Chapter 1 examines probability models
defined on abstract sets. It introduces the set theory notation used throughout the book and
states the three axioms of probability and several theorems that follow directly from the ax-
ioms. It defines conditional probability, the Law of Total Probability, Bayes’ theorem, and