CONTENTS
PREFACE
PART I BASIC METHODS
INTRODUCTION
The Spectral Estimation Problem and its Application
On the Uses o f Spectral Estimation, 6
Principal Approaches to Spectral Estimation, 7
Comparison o f Spectral Estimators, 8
A Common Test Case, 11
References, 13
Problems,14
Appendix 1A Listing o f Test Data Sets. 15
2
REVIEW OF LINEAR AND MATRIX ALGEBRA
Introduction, 17
2.1
2.2 Definitions 17
2.3 Special Matrices, 19
2.4 M atrix Manipulations and Formulas. 23
2.5
Important Theorems, 25
2.6 Eigendecompo5>ition o f Matrices, 26
2.7 Solutions o f Linear Equations, 28
2.8 Minimization o f Quadratic and Hcrmitian Functions, 31
References, 33
Problems, 34
Appendix 2A Computer Program for FFT. 36
Appendix 2B Computer Program for Cholesky Solution
o f Simultaneous Linear Equations, 38
3
REVIEW OF PROBABILITY, STATISTICS, AND
RANDOM PROCESSES
Introduction, 41
3.1
3.2 Some Useful Probability Density Functions,41
3.3 Estimation Theory, 45
3.4 Random Process Characlcrization, 51
3.5 Some Important Random Processes, 54
3.6 Hrgodicity o f the Autocorrelation Function, 58
3.7 An Alternative Definition o f the Power Spectral Density,
59
References, 60
Problems, 60
4
CLASSICAL SPECTRAL ESTIMATION
Introduction, 63
4.1
4.2 Summary. 64
4.3 Pcriodogram, 65
4.4 Averaged Periodogram, 72
4.5 Blackman-Tukey Spectral Estimation. 77
4.6 Computer Simulation Examples, 82
References. 94
Problems, 94
Appendix 4A Bias and Variance o f Periodogram, %
Appendix 4B Bias and Variance of Blackman-Tukey
Spectral Estimator. 98
Appendix 4C Computer Program for the Pcriodogram.
100
Appendix 4D Computer Program for the Correlation
Estimate, 102
Appendix 4E Computer Program for the Blackman-
Tukey Spcctral Estimator,103
41
63
vi
Contents
PARAMETRIC MODELING
106
Estimator, 140
5.8 Model Parameter Determination Based on PSD or ACF,
Introduction, 106
5.1
5.2 Summary, 108
5.3 Rational Transfer Function Models, 109
5.4 Model Parameter Relationships to Autocorrelation, 114
5.5 Examples o f AR M A ,AR. and M A Processes, 118
5.6 Model Fitting, 131
5.7 MA Modeling and the Blackman-Tukey Spectral
141
Rcfcrences, 143
Problems • 143
Appendix 5A Computer Program to Generate Real White
Gaussian Noise,145
Appendix 5B Computer Program to Generate Time
Scries, 147
Appendix 5C Computer Program to Compute PSD
Values, 150
6
AUTOREGRESSIVE SPECTRAL ESTIMATION: GENERAL
153
Introduction, 153
Summary. 154
Properties of AR Processes, 156
Properties o f the AR Spectral Estimator, 178
Estimation o f AR Paramclcrs and Reflection
Coefficients, 185
Estimation o f the AR Power Spectral Dcnsily. 193
Effect o f Noise on the AR Spectral Estimator, 195
Considerations in Model Order Selection, 206
Rcfcrcnces, 207
Problems, 209
Appendix 6A Derivation o f Cramer- Rao Lower Bounds
for AR Parameter Estimators, 211
Appendix 6B Computer Program for the Ixvinson
Recursion. 213
Appendix 6C Computer Program for Step-Down
Procedure,214
7
AUTOREGRESSIVE SPECTRAL ESTIMATION: METHODS
217
Introduction. 217
7.1
7.2 Summary, 218
7.3 Autocorrelation Method, 221
Contents
vii
7.4 Covariance Method, 222
7.5 Modified Covariance Method, 225
7.6 Burg Method. 228
7.7 Recursive M LE . 232
7.8 Model Order Selection,234
7.9 Spectral Estimation o f Noisy AR Processes, 237
7.10 Computer Simulation Examples, 240
References, 253
Problems, 256
Appendix 7A Development o f Akaike Information
Criterion, 258
Appendix 7B Computer Program for Autocorrelation
Method, 260
Appendix 7C Computer Program for Covariance and
Modified Covariance Methods, 262
Appendix 7D Computer Program for Burg Method,265
Appendix 7E Computer Progrdm for Recursive M LE
Method, 267
8
MOVING AVERAGE SPECTRAL ESTIMATION
271
Introduction, 271
8.1
8.2 Summary, 271
8.3 The M A Spectral Estimator, 272
8.4 Maximum Likelihood Estimation: Durbin's Method, 273
8.5 Statistics o f the M A Parameter and Spcctral Estimators,
277
8.6 Mode! Order Selection, 279
8.7 Other MA Estimators, 280
8.8 Computer Simulation Examples, 282
References, 287
Problems, 287
Appendix 8A Computer Program for Durbin Method, 288
AUTOREGRESSIVE MOVING AVERAGE SPECTRAL
ESTIMATION: GENERAL
9
290
Introduction, 290
Summary, 290
Maximum Likelihood Estimation, 291
Statistics o f the Maximum Likelihood Estimator, 293
Numerator Determination for Known Autoregressive
Parameters,2%
Model Order Selection, 297
A Special ARM A Model. 299
References, 300
Problems, 301
Appendix 9A Derivation o f the Cramer- Rao Bounds for
ARMA Parameter Estimators, 302
AUTOREGRESSIVE MOVING AVERAGE SPECTRAL
ESTIMATION: METHODS
Introduction, 306
Summary, 307
Akaike Approximate M LE , 309
Modified Yule-W alkcr Equations,312
Least Squares Modified Yule-W alker Equations, 316
Input-O utput Identification Approaches, 318
Computer Simulation Examples, 322
References, 342
Problems, 344
Appendix 10A Evaluation o f Partial Derivatives for
Akaike M LE , 346
Appendix 10B Positive Definite Property o f Approximate
Hessian, 350
Appendix 10C Computer Program fo r Akaike M LE , 351
Appendix 10D Computer Program fo r Modified Y ule-
Walker Equations Method. 358
Appendix 10E Computer Program for Least Squares
Modified Yule-W alker Equations Method, 361
Appendix 10F Computer Program fo r Mayne-Firoozan
Method, 364
11
M INIM UM VARIANCE SPECTRAL ESTIMATION
370
I
n
I
.
I
n
1
1
1
1
1
n
Introduction, 370
Summary, 370
Maximum Likelihood Estimation o f Signal Amplitude,
372
Filtering Interpretation o f the Linear Minimum Variance
Unbiased Estimator,374
The Minimum Variance Spectral Estimator, 378
Comparison o f the MVSE and AR Spectral Estimators,
380
Computer Simulation Examples,383
References. 391
Problems, 392
Appendix 11A Computer Program for Minimum
Variance Spcctral Estimator, 393
Contents
1
1
1
12
SUMMARY OF SPECTRAL ESTIMATORS
396
12.1 丨mroduction, 3%
12.2 Test Case Data Comparison, 3%
12.3 General Comparison, 401
References, 402
Problems, 403
PART II ADVANCED CONCEPTS
13
SINUSOIDAL PARAMETER ESTIMATION
Iniroduction. 407
Summary. 408
Maximum Likelihood Estimation, 408
Cramer-Rao Bounds, 413
Approximate M LE Methods, 416
Frequency Estimation by Spectral Estimation, 420
Properties o f the Autocorrelation M atrix, 422
Principal Component Frequency Estimation,425
Noise Sub^pacc Frequency Estimation, 429
Model Order Selection, 434
Computer Simulation Examples, 436
References • 438
Problems, 440
Appendix 13A Proof o f Spanning Property o f Principal
Eigenveclon>, 442
Appendix 13B Proof o f Pisarenko Property, 443
3
3
3
_
3
1
■
•
«
2
•
«
3
•
H
-
I
4
•
5
•
14
MULTICHANNEL SPECTRAL ESTIMATION
3
Introduction, 446
Summary, 447
Review o f Linear Systems and Fourier Transforms, 447
Review o f Random Processes• 452
Classical Spectral Estimation, 455
Rational Transfer Function Models, 457
Autoregressive Spectral Estimation, 460
Autoregressive Moving Average Spectral Estimation.
465
Minimum Variance Spectral Estimation, 468
Computer Simulation Examples. 469
References, 471
6
•
7
•
8
•
9
3
3
9
14
14
比•
癱
3
405
407
446
Contents
Problems, 473
Appendix 14A Derivaiion o f Levinson Algorithm for
Solution o f the Multichannel Yule-W alkcr Equations,
475
15
TWO-DIMENSIONAL SPECTRAL ESTIMATION
479
5.
1
IX
i
.si
1
f
l/^5.
lrl
1
5
5
5.
1
1
3,
1^ 2.
4-
6-
^
8-
7
Inlroduction. 479
Summary,480
Review o f Linear Systems and Fourier Transforms, 480
Review o f Random Processes, 487
Classical Spectral Estimation, 489
Autoregressive Spectral Estimation. 492
Maximum Entropy Spectral Eslimation, 505
Minimum Variance Spectral Estimation, 506
Sinusoidal Parameter Estimation, 508
Computer Simulation Examples, 509
o
References, 512
Problems. 514
16
OTHER APPLICATIONS OF SPECTRAL ESTIMATION
METHODS
1
1
518
6
参
•
•
6
•參
I
Introduction,518
Time Series Extrapolation and Interpolation, 518
Signal Detection, 519
Bandwidth Compression, 520
Spectral Smoothing and Modeling, 521
Beamforming/Direction Finding. 522
Lattice Filters, 524
Other Applications, 524
References. 525
1
2
3
4
APPENDIX 1 SUMMARY OF COMPUTER PROGRAMS
6
APPENDIX 2 GLOSSARY OF SYMBOLS,
n
ABBREVIATIONS, AND NOTATIONAL CONVENTIONS
APPENDIX 3 DESCRIPTION OF FLOPPY DISK
6
CONTENTS
1
APPENDIX 4 DESCRIPTION OF MENU-DRIVEN
SPECTRAL ESTIMATION SOFTWARE
6
INDEX
5
6
••秦
6
1
7
Contents
6
8
527
533
537
539
541
xi
Chapter 1
Introduction
THE SPECTRAL ESTIMATION PROBLEM AND ITS
APPLICATION
The general problem o f spcctral estimation is that o f determining the spectral
contcnt o f a random process based on a finite set o f observations from that pro
cess. Form ally, the power spectral density (PSD),which w ill be denoted by 尸„(/),
o f a complex wide sense stationary (WSS) random process x[n] is defined as
P x x if ) = 2 ^xx[^l exp
k--- *
f
(1.1)
where r „fA l is the autocorrelation function (ACF) o f x[/i] defined as
r«[A:】= f (x^[n]x[n 十 A:J)
(1.2)
and t is the cxpcclalion operator. The frequency f may either be thought o f as
the fraction o f the sampling frequency used in obtaining the data samples from a
continuous random process or as the number of cycles/sample. The PSD function
describes the distribution o f power with frequency o f the random process. Phys
ically. we could determine this distribution by filtering the random process with
a bandpass filte r that is centered at / = / 0 and has a sufficiently narrow filte r
bandwidth, and then measuring the power at its output. The power is then divided
by the filte r bandwidth. This procedure would be repeated for all center fre
quencies - I s: f 0 ^
This mclhod. however, presupposes that the obsened
random process w ill be o f sufficient duration to allow the filte r transients to decay.
The narrower the filte r bandwidth, the longer the observation interval must be.