i
i
i
\sm2"
2004/2/22
page i
i
SPECTRAL ANALYSIS OF
SIGNALS
Petre Stoica and Randolph Moses
PRENTICE HALL, Upper Saddle River, New Jersey 07458
i
i
i
i
i
i
i
\sm2"
2004/2/22
page ii
i
Library of Congress Cataloging-in-Publication Data
Spectral Analysis of Signals/Petre Stoica and Randolph Moses
p. cm.
Includes bibliographical references index.
ISBN 0-13-113956-8
1. Spectral theory (Mathematics) I. Moses, Randolph II. Title
512’{dc21
QA814.G27
2005
00-055035
CIP
Acquisitions Editor: Tom Robbins
Editor-in-Chief: ?
Assistant Vice President of Production and Manufacturing: ?
Executive Managing Editor: ?
Senior Managing Editor: ?
Production Editor: ?
Manufacturing Buyer: ?
Manufacturing Manager: ?
Marketing Manager: ?
Marketing Assistant: ?
Director of Marketing: ?
Editorial Assistant: ?
Art Director: ?
Interior Designer: ?
Cover Designer: ?
Cover Photo: ?
c 2005 by Prentice Hall, Inc.
Upper Saddle River, New Jersey 07458
All rights reserved. No part of this book may
be reproduced, in any form or by any means,
without permission in writing from the publisher.
Printed in the United States of America
10
2
1
9
8
7
6
5
4
3
ISBN 0-13-113956-8
Pearson Education LTD., London
Pearson Education Australia PTY, Limited, Sydney
Pearson Education Singapore, Pte. Ltd
Pearson Education North Asia Ltd, Hong Kong
Pearson Education Canada, Ltd., Toronto
Pearson Educacion de Mexico, S.A. de C.V.
Pearson Education - Japan, Tokyo
Pearson Education Malaysia, Pte. Ltd
i
i
i
i
i
i
i
\sm2"
2004/2/22
page iii
i
Contents
1 Basic Concepts
1.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Energy Spectral Density of Deterministic Signals . . . . . . . . . . .
1.3 Power Spectral Density of Random Signals
. . . . . . . . . . . . . .
1.3.1 First Denition of Power Spectral Density . . . . . . . . . . .
Second Denition of Power Spectral Density . . . . . . . . . .
1.3.2
1.4 Properties of Power Spectral Densities . . . . . . . . . . . . . . . . .
1.5 The Spectral Estimation Problem . . . . . . . . . . . . . . . . . . . .
1.6 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6.1 Coherency Spectrum . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.7 Exercises
2 Nonparametric Methods
2.1
2.2 Periodogram and Correlogram Methods
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
2.2.1 Periodogram . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.2 Correlogram . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3 Periodogram Computation via FFT . . . . . . . . . . . . . . . . . .
2.3.1 Radix{2 FFT . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3.2 Zero Padding . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4 Properties of the Periodogram Method . . . . . . . . . . . . . . . . .
2.4.1 Bias Analysis of the Periodogram . . . . . . . . . . . . . . . .
2.4.2 Variance Analysis of the Periodogram . . . . . . . . . . . . .
2.5 The Blackman{Tukey Method . . . . . . . . . . . . . . . . . . . . . .
2.5.1 The Blackman{Tukey Spectral Estimate . . . . . . . . . . . .
2.5.2 Nonnegativeness of the Blackman{Tukey Spectral Estimate .
2.6 Window Design Considerations . . . . . . . . . . . . . . . . . . . . .
2.6.1 Time{Bandwidth Product and Resolution{Variance Trade-
os in Window Design . . . . . . . . . . . . . . . . . . . . . .
Some Common Lag Windows . . . . . . . . . . . . . . . . . .
2.6.2
2.6.3 Window Design Example . . . . . . . . . . . . . . . . . . . .
2.6.4 Temporal Windows and Lag Windows . . . . . . . . . . . . .
2.7 Other Rened Periodogram Methods . . . . . . . . . . . . . . . . . .
2.7.1 Bartlett Method . . . . . . . . . . . . . . . . . . . . . . . . .
2.7.2 Welch Method . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7.3 Daniell Method . . . . . . . . . . . . . . . . . . . . . . . . . .
2.8 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Sample Covariance Computation via FFT . . . . . . . . . . .
2.8.1
2.8.2 FFT{Based Computation of Windowed Blackman{Tukey Pe-
1
1
3
4
6
7
8
12
12
12
14
22
22
22
22
23
25
26
27
28
28
32
37
37
39
39
40
41
45
47
48
49
50
52
55
55
riodograms
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
2.8.3 Data and Frequency Dependent Temporal Windows: The
Apodization Approach . . . . . . . . . . . . . . . . . . . . . .
59
i
i
iii
i
i
i
i
i
\sm2"
2004/2/22
page iv
i
iv
2.8.4 Estimation of Cross{Spectra and Coherency Spectra . . . . .
2.8.5 More Time{Bandwidth Product Results . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9 Exercises
64
66
71
3 Parametric Methods for Rational Spectra
3.5 Order{Recursive Solutions to the Yule{Walker Equations
3.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Signals with Rational Spectra . . . . . . . . . . . . . . . . . . . . . .
3.3 Covariance Structure of ARMA Processes . . . . . . . . . . . . . . .
3.4 AR Signals
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4.1 Yule{Walker Method . . . . . . . . . . . . . . . . . . . . . . .
3.4.2 Least Squares Method . . . . . . . . . . . . . . . . . . . . . .
. . . . . .
3.5.1 Levinson{Durbin Algorithm . . . . . . . . . . . . . . . . . . .
3.5.2 Delsarte{Genin Algorithm . . . . . . . . . . . . . . . . . . . .
86
86
87
88
90
90
91
94
96
97
3.6 MA Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.7 ARMA Signals
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.7.1 Modied Yule{Walker Method . . . . . . . . . . . . . . . . . 103
3.7.2 Two{Stage Least Squares Method . . . . . . . . . . . . . . . 106
3.8 Multivariate ARMA Signals . . . . . . . . . . . . . . . . . . . . . . . 109
3.8.1 ARMA State{Space Equations . . . . . . . . . . . . . . . . . 109
Subspace Parameter Estimation | Theoretical Aspects . . . 113
3.8.2
3.8.3
Subspace Parameter Estimation | Implementation Aspects . 115
3.9 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.9.1 The Partial Autocorrelation Sequence . . . . . . . . . . . . . 117
3.9.2
Some Properties of Covariance Extensions . . . . . . . . . . . 118
3.9.3 The Burg Method for AR Parameter Estimation . . . . . . . 119
3.9.4 The Gohberg{Semencul Formula . . . . . . . . . . . . . . . . 122
3.9.5 MA Parameter Estimation in Polynomial Time . . . . . . . . 125
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.10 Exercises
4 Parametric Methods for Line Spectra
144
4.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.2 Models of Sinusoidal Signals in Noise . . . . . . . . . . . . . . . . . . 148
4.2.1 Nonlinear Regression Model . . . . . . . . . . . . . . . . . . . 148
4.2.2 ARMA Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
4.2.3 Covariance Matrix Model
. . . . . . . . . . . . . . . . . . . . 149
4.3 Nonlinear Least Squares Method . . . . . . . . . . . . . . . . . . . . 151
4.4 High{Order Yule{Walker Method . . . . . . . . . . . . . . . . . . . . 155
4.5 Pisarenko and MUSIC Methods . . . . . . . . . . . . . . . . . . . . . 159
4.6 Min{Norm Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
4.7 ESPRIT Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.8 Forward{Backward Approach . . . . . . . . . . . . . . . . . . . . . . 168
4.9 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
4.9.1 Mean Square Convergence of Sample Covariances for Line
Spectral Processes . . . . . . . . . . . . . . . . . . . . . . . . 170
4.9.2 The Caratheodory Parameterization of a Covariance Matrix . 172
i
i
i
i
i
i
i
\sm2"
2004/2/22
page v
i
v
4.9.3 Using the Unwindowed Periodogram for Sine Wave Detection
in White Noise . . . . . . . . . . . . . . . . . . . . . . . . . . 174
4.9.4 NLS Frequency Estimation for a Sinusoidal Signal with Time-
Varying Amplitude . . . . . . . . . . . . . . . . . . . . . . . . 177
4.9.5 Monotonically Descending Techniques for Function Minimiza-
tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
4.9.6 Frequency-selective ESPRIT-based Method . . . . . . . . . . 185
4.9.7 A Useful Result for Two-Dimensional (2D) Sinusoidal Signals 193
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
4.10 Exercises
5 Filter Bank Methods
207
5.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.2 Filter Bank Interpretation of the Periodogram . . . . . . . . . . . . . 210
5.3 Rened Filter Bank Method . . . . . . . . . . . . . . . . . . . . . . . 212
5.3.1
Slepian Baseband Filters . . . . . . . . . . . . . . . . . . . . . 213
5.3.2 RFB Method for High{Resolution Spectral Analysis . . . . . 216
5.3.3 RFB Method for Statistically Stable Spectral Analysis . . . . 218
5.4 Capon Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
5.4.1 Derivation of the Capon Method . . . . . . . . . . . . . . . . 222
5.4.2 Relationship between Capon and AR Methods
. . . . . . . . 228
5.5 Filter Bank Reinterpretation of the Periodogram . . . . . . . . . . . 231
5.6 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
5.6.1 Another Relationship between the Capon and AR Methods . 235
5.6.2 Multiwindow Interpretation of Daniell and Blackman{Tukey
Periodograms . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
5.6.3 Capon Method for Exponentially Damped Sinusoidal Signals 241
5.6.4 Amplitude and Phase Estimation Method (APES) . . . . . . 244
5.6.5 Amplitude and Phase Estimation Method for Gapped Data
(GAPES) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
5.6.6 Extensions of Filter Bank Approaches to Two{Dimensional
5.7 Exercises
Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
6 Spatial Methods
6.1
6.2 Array Model
263
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
6.2.1 The Modulation{Transmission{Demodulation Process . . . . 266
6.2.2 Derivation of the Model Equation . . . . . . . . . . . . . . . 268
6.3 Nonparametric Methods . . . . . . . . . . . . . . . . . . . . . . . . . 273
6.3.1 Beamforming . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6.3.2 Capon Method . . . . . . . . . . . . . . . . . . . . . . . . . . 279
6.4 Parametric Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
6.4.1 Nonlinear Least Squares Method . . . . . . . . . . . . . . . . 281
6.4.2 Yule{Walker Method . . . . . . . . . . . . . . . . . . . . . . . 283
6.4.3 Pisarenko and MUSIC Methods . . . . . . . . . . . . . . . . . 284
6.4.4 Min{Norm Method . . . . . . . . . . . . . . . . . . . . . . . . 285
6.4.5 ESPRIT Method . . . . . . . . . . . . . . . . . . . . . . . . . 285
i
i
i
i
i
i
i
\sm2"
2004/2/22
page vi
i
vi
6.5 Complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
6.5.1 On the Minimum Norm Constraint . . . . . . . . . . . . . . . 286
6.5.2 NLS Direction-of-Arrival Estimation for a Constant-Modulus
Signal
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
6.5.3 Capon Method: Further Insights and Derivations . . . . . . . 290
6.5.4 Capon Method for Uncertain Direction Vectors . . . . . . . . 294
6.5.5 Capon Method with Noise Gain Constraint . . . . . . . . . . 298
6.5.6
. . . . . . 305
6.5.7 The CLEAN Algorithm . . . . . . . . . . . . . . . . . . . . . 312
6.5.8 Unstructured and Persymmetric ML Estimates of the Covari-
Spatial Amplitude and Phase Estimation (APES)
6.6 Exercises
ance Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
APPENDICES
A Linear Algebra and Matrix Analysis Tools
328
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
A.2 Range Space, Null Space, and Matrix Rank . . . . . . . . . . . . . . 328
A.3 Eigenvalue Decomposition . . . . . . . . . . . . . . . . . . . . . . . . 330
A.3.1 General Matrices . . . . . . . . . . . . . . . . . . . . . . . . . 331
A.3.2 Hermitian Matrices . . . . . . . . . . . . . . . . . . . . . . . . 333
A.4 Singular Value Decomposition and Projection Operators . . . . . . . 336
A.5 Positive (Semi)Denite Matrices
. . . . . . . . . . . . . . . . . . . . 341
A.6 Matrices with Special Structure . . . . . . . . . . . . . . . . . . . . . 345
A.7 Matrix Inversion Lemmas . . . . . . . . . . . . . . . . . . . . . . . . 347
A.8 Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . 347
A.8.1 Consistent Systems . . . . . . . . . . . . . . . . . . . . . . . . 347
A.8.2 Inconsistent Systems . . . . . . . . . . . . . . . . . . . . . . . 350
A.9 Quadratic Minimization . . . . . . . . . . . . . . . . . . . . . . . . . 353
B Cramer{Rao Bound Tools
355
B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
B.2 The CRB for General Distributions . . . . . . . . . . . . . . . . . . . 358
B.3 The CRB for Gaussian Distributions . . . . . . . . . . . . . . . . . . 359
B.4 The CRB for Line Spectra . . . . . . . . . . . . . . . . . . . . . . . . 364
B.5 The CRB for Rational Spectra . . . . . . . . . . . . . . . . . . . . . 365
B.6 The CRB for Spatial Spectra . . . . . . . . . . . . . . . . . . . . . . 367
C Model Order Selection Tools
377
C.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
C.2 Maximum Likelihood Parameter Estimation . . . . . . . . . . . . . . 378
C.3 Useful Mathematical Preliminaries and Outlook . . . . . . . . . . . . 381
C.3.1 Maximum A Posteriori (MAP) Selection Rule . . . . . . . . . 382
C.3.2 Kullback-Leibler Information . . . . . . . . . . . . . . . . . . 384
C.3.3 Outlook: Theoretical and Practical Perspectives
. . . . . . . 385
C.4 Direct Kullback-Leibler (KL) Approach: No-Name Rule . . . . . . . 386
i
i
i
i
i
i
i
\sm2"
2004/2/22
page vii
i
C.5 Cross-Validatory KL Approach: The AIC Rule . . . . . . . . . . . . 387
C.6 Generalized Cross-Validatory KL Approach: the GIC Rule . . . . . . 391
C.7 Bayesian Approach: The BIC Rule . . . . . . . . . . . . . . . . . . . 392
C.8 Summary and the Multimodel Approach . . . . . . . . . . . . . . . . 395
C.8.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
C.8.2 The Multimodel Approach . . . . . . . . . . . . . . . . . . . 397
vii
D Answers to Selected Exercises
Bibliography
References Grouped by Subject
Index
399
401
413
420
i
i
i
i
i
i
viii
i
\sm2"
2004/2/22
page viii
i
i
i
i
i