Solution Manual
to accompany
Pattern
Classification
(2nd ed.)
David G. Stork
Solution Manual to accompany
Pattern Classification (2nd ed.)
by R. O. Duda, P. E. Hart and D. G. Stork
David G. Stork
Copyright 2001. All rights reserved.
This Manual is for the sole use of designated educators
and must not be distributed to students
except in short, isolated portions and in conjunction
with the use of Pattern Classification (2nd ed.)
June 18, 2003
Preface
In writing this Solution Manual I have learned a very important lesson. As a
student, I thought that the best way to master a subject was to go to a superb
university and study with an established expert. Later, I realized instead that the
best way was to teach a course on the subject. Yet later, I was convinced that the
best way was to write a detailed and extensive textbook. Now I know that all these
years I have been wrong:
in fact the best way to master a subject is to write the
Solution Manual.
In solving the problems for this Manual I have been forced to confront myriad
technical details that might have tripped up the unsuspecting student. Students and
teachers can thank me for simplifying or screening out problems that required pages of
unenlightening calculations. Occasionally I had to go back to the text and delete the
word “easily” from problem references that read “it can easily be shown (Problem ...).”
Throughout, I have tried to choose data or problem conditions that are particularly
instructive. In solving these problems, I have found errors in early drafts of this text
(as well as errors in books by other authors and even in classic refereed papers), and
thus the accompanying text has been improved for the writing of this Manual.
I have tried to make the problem solutions self-contained and self-explanatory. I
have gone to great lengths to ensure that the solutions are correct and clearly presented
— many have been reviewed by students in several classes. Surely there are errors and
typos in this manuscript, but rather than editing and rechecking these solutions over
months or even years, I thought it best to distribute the Manual, however flawed,
as early as possible. I accept responsibility for these inevitable errors, and humbly
ask anyone finding them to contact me directly. (Please, however, do not ask me to
explain a solution or help you solve a problem!) It should be a small matter to change
the Manual for future printings, and you should contact the publisher to check that
you have the most recent version. Notice, too, that this Manual contains a list of
known typos and errata in the text which you might wish to photocopy and distribute
to students.
I have tried to be thorough in order to help students, even to the occassional fault of
verbosity. You will notice that several problems have the simple “explain your answer
in words” and “graph your results.” These were added for students to gain intuition
and a deeper understanding. Graphing per se is hardly an intellectual challenge, but
if the student graphs functions, he or she will develop intuition and remember the
problem and its results better. Furthermore, when the student later sees graphs of
data from dissertation or research work, the link to the homework problem and the
material in the text will be more readily apparent. Note that due to the vagaries
of automatic typesetting, figures may appear on pages after their reference in this
Manual; be sure to consult the full solution to any problem.
I have also included worked examples and so sample final exams with solutions to
1
2
cover material in text. I distribute a list of important equations (without descriptions)
with the exam so students can focus understanding and using equations, rather than
memorizing them. I also include on every final exam one problem verbatim from a
homework, taken from the book. I find this motivates students to review carefully
their homework assignments, and allows somewhat more difficult problems to be in-
cluded. These will be updated and expanded; thus if you have exam questions you
find particularly appropriate, and would like to share them, please send a copy (with
solutions) to me.
It should be noted, too, that a set of overhead transparency masters of the figures
from the text are available to faculty adopters. I have found these to be invaluable
for lecturing, and I put a set on reserve in the library for students. The files can be
accessed through a standard web browser or an ftp client program at the Wiley STM
ftp area at:
ftp://ftp.wiley.com/public/sci tech med/pattern/
or from a link on the Wiley Electrical Engineering software supplements page at:
http://www.wiley.com/products/subject/engineering/electrical/
software supplem elec eng.html
I have taught from the text (in various stages of completion) at the University
of California at Berkeley (Extension Division) and in three Departments at Stan-
ford University: Electrical Engineering, Statistics and Computer Science. Numerous
students and colleagues have made suggestions. Especially noteworthy in this re-
gard are Sudeshna Adak, Jian An, Sung-Hyuk Cha, Koichi Ejiri, Rick Guadette,
John Heumann, Travis Kopp, Yaxin Liu, Yunqian Ma, Sayan Mukherjee, Hirobumi
Nishida, Erhan Oztop, Steven Rogers, Charles Roosen, Sergio Bermejo Sanchez, God-
fried Toussaint, Namrata Vaswani, Mohammed Yousuf and Yu Zhong. Thanks too
go to Dick Duda who gave several excellent suggestions.
I would greatly appreciate notices of any errors in this Manual or the text itself. I
would be especially grateful for solutions to problems not yet solved. Please send any
such information to me at the below address. I will incorporate them into subsequent
releases of this Manual.
This Manual is for the use of educators and must not be distributed in bulk to
students in any form. Short excerpts may be photocopied and distributed, but only
in conjunction with the use of Pattern Classification (2nd ed.).
I wish you all the best of luck in teaching and research.
Ricoh Innovations, Inc.
2882 Sand Hill Road Suite 115
Menlo Park, CA 94025-7022 USA
stork@rii.ricoh.com
David G. Stork
Contents
Preface
1 Introduction
2 Bayesian decision theory
Problem Solutions
Computer Exercises
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
7
7
74
3 Maximum likelihood and Bayesian parameter estimation
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
77
77
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Problem Solutions
Computer Exercises
4 Nonparametric techniques
Problem Solutions
Computer Exercises
131
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
5 Linear discriminant functions
Problem Solutions
Computer Exercises
177
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
6 Multilayer neural networks
Problem Solutions
Computer Exercises
219
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
7 Stochastic methods
Problem Solutions
Computer Exercises
255
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
8 Nonmetric methods
Problem Solutions
Computer Exercises
277
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
9 Algorithm-independent machine learning
295
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
Problem Solutions
Computer Exercises
10 Unsupervised learning and clustering
Problem Solutions
Computer Exercises
305
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
Sample final exams and solutions
357
3
4
Worked examples
CONTENTS
415
Errata and ammendations in the text
417
First and second printings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
Fifth printing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
Chapter 1
Introduction
Problem Solutions
There are neither problems nor computer exercises in Chapter 1.
5
6
CHAPTER 1. INTRODUCTION