S E C O N D E D I T I O N
IMAGE ANALYSIS,
CLASSIFICATION, and
CHANGE DETECTION
in REMOTE SENSING
with Algorithms for ENVI/IDL
S E C O N D E D I T I O N
IMAGE ANALYSIS,
CLASSIFICATION, and
CHANGE DETECTION
in REMOTE SENSING
with Algorithms for ENVI/IDL
Morton J. Canty
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2009 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20131120
International Standard Book Number-13: 978-1-4200-8714-7 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Preface to the Second Edition....................................................
xi
Preface to the First Edition ....................................................... xiii
1
1. Images, Arrays, and Matrices ................................................
2
1.1 Multispectral Satellite Images ...........................................
5
1.2 Algebra of Vectors and Matrices ........................................
6
1.2.1 Elementary Properties ...........................................
1.2.2 Square Matrices ...................................................
8
1.2.3 Singular Matrices ................................................. 10
1.2.4 Symmetric, Positive Definite Matrices ......................... 11
1.2.5 Linear Dependence and Vector Spaces ........................ 12
1.3 Eigenvalues and Eigenvectors .......................................... 13
1.4 Singular Value Decomposition.......................................... 16
1.5 Vector Derivatives ........................................................ 18
1.6 Finding Minima and Maxima ........................................... 19
1.7 Exercises .................................................................... 25
2. Image Statistics ................................................................. 27
2.1 Random Variables ........................................................ 27
2.1.1 Discrete Random Variables ..................................... 28
2.1.2 Continuous Random Variables ................................. 29
2.1.3 Normal Distribution .............................................. 32
2.2 Random Vectors ........................................................... 34
2.3 Parameter Estimation..................................................... 39
2.3.1 Sampling a Distribution ......................................... 39
2.3.2
Interval Estimation ............................................... 42
2.3.3 Provisional Means ................................................ 43
2.4 Hypothesis Testing and Sample Distribution Functions ............ 44
2.4.1 Chi-Square Distribution ......................................... 48
2.4.2 Student-t Distribution ............................................ 49
2.4.3 F-Distribution ..................................................... 50
2.5 Conditional Probabilities, Bayes’ Theorem, and Classification .... 51
2.6 Ordinary Linear Regression ............................................. 55
2.6.1 One Independent Variable ...................................... 55
2.6.2 More Than One Independent Variable ........................ 57
2.6.3 Regularization, Duality, and the Gram Matrix ............... 60
2.7 Entropy and Information ................................................ 62
2.7.1 Kullback–Leibler Divergence ................................... 64
2.7.2 Mutual Information .............................................. 64
2.8 Exercises .................................................................... 65
v
vi
Contents
3. Transformations................................................................ 69
3.1 Discrete Fourier Transform .............................................. 69
3.2 Discrete Wavelet Transform ............................................. 73
3.2.1 Haar Wavelets ..................................................... 75
3.2.2
Image Compression .............................................. 79
3.2.3 Multiresolution Analysis ........................................ 82
3.2.3.1 Dilation Equation and Refinement
Coefficients .............................................. 83
3.2.3.2 Cascade Algorithm ..................................... 84
3.2.3.3 Mother Wavelet ......................................... 85
3.2.3.4 Daubechies D4 Scaling Function ..................... 87
3.3 Principal Components .................................................... 89
3.3.1 Primal Solution .................................................... 91
3.3.2 Dual Solution ...................................................... 91
3.4 Minimum Noise Fraction ................................................ 93
3.4.1 Additive Noise .................................................... 93
3.4.2 Minimum Noise Fraction Transformation in ENVI ......... 96
3.5 Spatial Correlation ........................................................ 98
3.5.1 Maximum Autocorrelation Factor.............................. 98
3.5.2 Noise Estimation .................................................. 101
3.6 Exercises .................................................................... 103
4. Filters, Kernels, and Fields ................................................... 107
4.1 Convolution Theorem .................................................... 107
4.2 Linear Filters ............................................................... 111
4.3 Wavelets and Filter Banks ............................................... 113
4.3.1 One-Dimensional Arrays ........................................ 115
4.3.2 Two-Dimensional Arrays........................................ 120
4.4 Kernel Methods ........................................................... 122
4.4.1 Valid Kernels ...................................................... 124
4.4.2 Kernel PCA ........................................................ 127
4.5 Gibbs–Markov Random Fields .......................................... 130
4.6 Exercises .................................................................... 135
5. Image Enhancement and Correction ....................................... 139
5.1 Lookup Tables and Histogram Functions ............................. 139
5.2 Filtering and Feature Extraction ........................................ 141
5.2.1 Edge Detection .................................................... 141
5.2.2
Invariant Moments ............................................... 145
5.3 Panchromatic Sharpening................................................ 150
5.3.1 HSV Fusion ........................................................ 151
5.3.2 Brovey Fusion ..................................................... 152
5.3.3 PCA Fusion ........................................................ 153
5.3.4 DWT Fusion ....................................................... 154
5.3.5 À Trous Fusion..................................................... 155
Contents
vii
5.3.6 Quality Index ...................................................... 157
5.4 Topographic Correction .................................................. 159
5.4.1 Rotation, Scaling, and Translation ............................. 159
5.4.2
Imaging Transformations........................................ 160
5.4.3 Camera Models and RFM Approximations................... 161
5.4.4 Stereo Imaging and Digital Elevation Models ................ 163
5.4.5 Slope and Aspect.................................................. 167
5.4.6
Illumination Correction .......................................... 170
5.5 Image–Image Registration ............................................... 175
5.5.1 Frequency-Domain Registration................................ 176
5.5.2 Feature Matching ................................................. 177
5.5.2.1 High-Pass Filtering ..................................... 178
5.5.2.2 Closed Contours ........................................ 179
5.5.2.3 Chain Codes and Moments ........................... 179
5.5.2.4 Contour Matching ...................................... 180
5.5.2.5 Consistency Check...................................... 180
5.5.2.6 Implementation in IDL................................. 181
5.5.3 Resampling and Warping ....................................... 182
5.6 Exercises .................................................................... 183
6. Supervised Classification: Part 1 ............................................ 187
6.1 Maximum a Posteriori Probability....................................... 188
6.2 Training Data and Separability ......................................... 189
6.3 Maximum Likelihood Classification ................................... 193
6.3.1 ENVI’s Maximum Likelihood Classifier ...................... 195
6.3.2 Modified Maximum Likelihood Classifier .................... 196
6.4 Gaussian Kernel Classification .......................................... 198
6.5 Neural Networks .......................................................... 202
6.5.1 Neural Network Classifier ...................................... 207
6.5.2 Cost Functions..................................................... 209
6.5.3 Backpropagation .................................................. 212
6.5.4 Overfitting and Generalization ................................. 216
6.6 Support Vector Machines ................................................ 219
6.6.1 Linearly Separable Classes ...................................... 220
6.6.1.1 Primal Formulation..................................... 221
6.6.1.2 Dual Formulation ....................................... 222
6.6.1.3 Quadratic Programming and Support Vectors..... 224
6.6.2 Overlapping Classes.............................................. 225
6.6.3 Solution with Sequential Minimal Optimization ............ 227
6.6.4 Multiclass SVMs .................................................. 228
6.6.5 Kernel Substitution ............................................... 230
6.6.6 Modified SVM Classifier......................................... 231
6.7 Exercises .................................................................... 232