Estimation with Applications
To Tracking and Navigation
Estimation with Applications
To Tracking and Navigation
Yaakov Bar-Shalom
X.-Rong Li
Thiagalingam
Kirubarajan
A Wiley-Interscience
JOHN WILEY
New York
l Chichester
l Weinheim
l Brisbane
l Singapore
l Toronto
Publication
& SONS, INC.
This
text
is printed
on acid-free
paper. @
Copyright
0 2001 by John Wiley & Sons,
Inc.
All
rights
reserved. Published
simultaneously
in Canada.
the prior written
No part of this publication
may be reproduced,
form or by any means, electronic, mechanical,
except as permitted
either
appropriate
0 1923,
addressed
NY 10158-0012,
under Section
permission
fax (978) 750-4744.
fee to the Copyright
to the Permissions
(2 12) 850-6011,
(978) 750-8400,
per-copy
stored
photocopying,
in a retrieval
system or transmitted
in any
recording,
scanning or otherwise,
Act, without
107 or 108 of the 1976 United States Copyright
of the Publisher,
or authorization
Clearance Center,
through
222 Rosewood
of the
payment
Drive, Danvers, MA
should be
Requests
to the Publisher
for permission
Department,
John Wiley & Sons,
fax (2 12) 850-6008,
E-Mail:
Inc., 605 Third Avenue, New York,
PERMREQ
@ WILEY.COM.
For ordering
and customer
service,
call 1-800-CALL-WILEY.
Library
of Congress Cataloging-in-Publication
Data
Bar-Shalom,
Yaakov.
Estimation
with applications
to tracking
and navigation
/ by Yaakov
Bar-Shalom,
X.-Rong
Li, Thiagalingam
Kirubarajan.
p. cm.
Includes bibliographical
ISBN O-47 l-4 1655-X
1. Motion
control
devices.
references
and index.
(cloth)
Robots-Control
Kirubarajan,
systems.
Thiagalingam.
2. Remote
control.
3. Telecommunication
systems.
4.
5. Process control.
6. Estimation
theory.
I. Li, X.-Rong.
II.
III. Title.
.B37 2001
TJ214.5
68 1’.2-dc2
1
2001022366
Printed
in the United States of America.
10987654321
To Eva, Tali, Yael and Michael
To Peizhu, Helen and Linda
To Appa, Amma, Ketha, Abi and Arun
YBS
XRL
TK
Lemma 1.
Make
things as simple as possible but not simpler.
A. Einstein
Theorem 1.
By making
things absolutely clear,
people will become confused.
A Chinese
fortune cookie
Corollary 1.
We wi 11 make things
simple
but not too simple,
clear
but not too c
lear.
Lemma 2.
Uncertainty
is everywhere.
Theorem 2.
Uncertainty
cannot be conquered.
Corollary 2.
Embrace
it!
Paraphrased after Michael Moschen,
professonal
juggler.
1
INTRODUCTION
1.1
1.2
1.3
1.4
of State Estimation:
Vehicle
Collision
Avoidance
Prerequisites
ALGEBRA
AND
LINEAR
SYSTEMS
Algebra
the Determinant
Operations
of a Matrix
and
Projection
of Vectors
Jacobian
Eigenvectors,
Linear
and Hessian
and Quadratic
Dynamic
Systems
Forms
-
Controllability
and Observability
Linear
Dynamic
Systems
- Controllability
and Observability
Contents
Areas
PREFACE
ACRONYMS
MATHEMATICAL
NOTATIONS
and Related
REVIEW
OF LINEAR
and Chapter
and Notations
Objectives
Overview
OF PROBABILITY
and
the Axioms
of Estimation
of Estimation/Filtering
Estimation
Applications
Preview
An Example
Definitions
Some Linear
Inversion
Orthogonal
The Gradient,
Eigenvalues,
Continuous-Time
Discrete-Time
BACKGROUND
1.1.1
1.1.2
1.1.3
1.1.4
SCOPE OF THE TEXT
1.2.1
1.2.2
BRIEF
1.3.1
1.3.2
1.3.3
1.3.4
1.35
1.3.6
1.3.7
1.3.8
BRIEF
1.4.1
1.4.2
1.4.3
1.4.4
1.4.5
1.4.6
1.4.7
1.48
1.4.9
1.4.10
1.4.11
1.4.12
1.4.13
1.4.14
1.4.15
1.4.16
1.4.17
1.4.18
1.4.19
1.4.20
1.4.21
1.4.22
REVIEW
Events
Random
Probability
Mixed
Expectations
and Moments
Joint PDF of Two Random
Independent
Vector-Valued
Conditional
The Total Probability
Bayes’
Conditional
Gaussian
Joint
Expected
Mixture
Chi-Square
Weighted
Random
Random Walk
Markov
Random
Processes
Sequences,
Random
Probability
Mass Function
and Conditional
Events
and
Formula
Random
Variables
Value
Probability
of Quadratic
Density
Sum of Chi-Square
Processes
THEORY
of Probability
Variables
and Probability
Density
Function
Random
Variable
and Mixed
Probability-PDF
of a Scalar Random
Variables
Independent
Variables
and PDF
Random
and Their Moments
Variable
Variables
Theorem
Expectations
and Their
Smoothing
Property
Gaussian
Random
and Quartic
Functions
Variables
Forms
Variables
Random
Variables
Process
Distributed
Random
and
the Wiener
Markov
Sequences
and Markov
Chains
xvii
xxi
xxii
1
1
1
3
4
10
15
15
16
19
19
20
21
23
24
25
27
29
31
31
33
35
36
37
38
41
41
44
45
47
50
51
52
54
55
57
60
61
65
66
69
ix
X
CONTENTS
1.5
1.6
1.4.23
BRIEF
1.5.1
1.5.2
1.5.3
1.5.4
NOTES
1.6.1
1.6.2
The Law
of Large Numbers
and
the Central
Limit
Theorem
REVIEW
OF STATISTICS
Testing
Regions
Runs
Carlo
of the Chi-Square
Hypothesis
Confidence
Monte
Tables
AND
PROBLEMS
Bibliographical
Problems
Notes
and Significance
and Comparison
and Gaussian
of Algorithms
Distributions
IN ESTIMATION
Outline
Basic Concepts
- Summary
of Objectives
ESTIMATION
OF PARAMETER
for Estimation
of a Parameter
LIKELIHOOD
Definitions
MLE
MAP
MAP
The Sufficient
vs. MAP
Estimator
Estimator
of ML and MAP
Estimators
with Gaussian
Prior
Estimator
with One-Sided
with Diffuse
Statistic
and
Exponential
Prior
Prior
the Likelihood
MEAN
SQUARE
Equation
of LS and MMSE
Estimators
AND MINIMUM
SQUARES
Definitions
Some LS Estimators
MMSE
vs. MAP
ESTIMATORS
Estimator
in Gaussian
Noise
and a MAP
of an ML
the ML Estimation
AND MSE OF AN ESTIMATOR
Estimator
of Two Parameters
of Estimator
of Variances
Variances
of an ML and a MAP
and Sample
the Sample Mean
of
the Probability
EFFICIENCY
of an Event
OF ESTIMATORS
Definitions
Comparison
The Variances
Estimation
of
AND
Consistency
The Cramer-Rao
Proof
An Example
Large Sample
Lower
the Cramer-Rao
of Efficient
Properties
of
Bound
Lower
Estimator
of
the Fisher
and
Bound
the ML Estimator
of Estimators
of Estimator
Properties
PROBLEMS
Notes
Definitions
Models
CONCEPTS
INTRODUCTION
2.1.1
2.1.2
THE PROBLEM
2.2.1
2.2.2
MAXIMUM
2.3.1
2.3.2
2.3.3
2.3.4
2.3.5
LEAST
2.4.1
2.4.2
2.4.3
UNBIASED
Definition
2.5.1
Unbiasedness
2.5.2
Bias
2.5.3
THE VARIANCE
2.6.1
2.6.2
2.6.3
2.6.4
CONSISTENCY
2.7.1
2.7.2
2.7.3
2.7.4
2.7.5
SUMMARY
2.8.1
2.8.2
NOTES
2.9.1
2.9.2
Summary
Summary
AND
in
Bibliographical
Problems
2
BASIC
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
3
LINEAR
3.1
3.2
3.3
AND MAXIMUM
A POSTERIOR1
ESTIMATORS
ERROR
ESTIMATION
Estimator
Variance
Information
Matrix
Outline
Linear
ESTIMATION
INTRODUCTION
3.1.1
3.1.2
ESTIMATION
3.2.1
3.2.2
LINEAR
3.3.1
IN STATIC
SYSTEMS
Estimation
in Static
Systems
-
Summary
of Objectives
OF GAUSSIAN
The Conditional
Estimation
MINIMUM
The Principle
Mean
of Gaussian
MEAN
SQUARE
of Orthogonality
VECTORS
RANDOM
and Covariance
Random
Vectors
for Gaussian
Random
Vectors
-
Summary
ERROR
ESTIMATION
70
72
72
74
79
82
85
85
85
89
89
89
89
90
90
91
92
92
92
94
95
96
98
98
100
100
101
101
102
102
104
104
105
106
107
108
108
109
110
112
113
114
114
115
115
115
116
121
121
121
121
122
122
123
123
123