Remote Sensing
Image Fusion
A Practical Guide
Remote Sensing
Image Fusion
A Practical Guide
Christine Pohl
John van Genderen
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2017 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20160609
International Standard Book Number-13: 978-1-4987-3002-0 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Names: Pohl, Christine, 1966- author. | Van Genderen, J. L. (John L.) author.
Title: Remote sensing image fusion : a practical guide / Christine Pohl and
John Van Genderen.
Description: Boca Raton, FL : Taylor & Francis, 2016. | Includes
bibliographical references and index.
Identifiers: LCCN 2016013861 | ISBN 9781498730020
Subjects: LCSH: Remote sensing--Handbooks, manuals, etc. | Image
processing--Handbooks, manuals, etc.
Classification: LCC G70.4 .P65 2016 | DDC 621.36/78028566--dc23
LC record available at https://lccn.loc.gov/2016013861
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Foreword .................................................................................................................xi
Preface ................................................................................................................... xiii
Acknowledgments ................................................................................................xv
Authors ................................................................................................................ xvii
Acronyms ............................................................................................................. xix
1. Introduction .....................................................................................................1
1.1 Outline of the Book ...............................................................................1
1.2 Remote Sensing .....................................................................................3
1.3 Multi-Sensor Data .................................................................................5
1.3.1 Optical Remote Sensing ..........................................................7
1.3.2 Radar Remote Sensing .......................................................... 14
1.3.3 Hyperspectral Remote Sensing ........................................... 19
1.3.4 Thermal Remote Sensing ...................................................... 24
1.3.5 LiDAR ......................................................................................29
Image and Data Fusion ...................................................................... 32
1.4.1 Objectives of Image Fusion...................................................34
1.4.2 Types of Fusion ......................................................................36
1.4.3 Benefits .................................................................................... 39
1.4.4 Limitations .............................................................................. 41
1.5 Definitions and Terminology for This Book ................................... 41
1.5.1 Remote Sensing Images ........................................................42
1.5.2 Resolution ...............................................................................43
1.6
Summary ..............................................................................................46
References .......................................................................................................46
1.4
2. Fusion Levels ................................................................................................. 51
2.1 Data Fusion .......................................................................................... 52
Fusion in Remote Sensing ..................................................................53
2.2
2.3
Subpixel Level ......................................................................................56
2.4 Pixel Level ............................................................................................ 57
2.5
Feature Level........................................................................................59
2.6 Decision Level...................................................................................... 62
2.7 Hybrid-Level Fusion ...........................................................................64
2.8
Summary .............................................................................................. 67
References ....................................................................................................... 67
3. Preprocessing ................................................................................................71
3.1 Data Selection ......................................................................................71
3.2 General Workflow ...............................................................................73
v
vi
3.3
Contents
Sensor-Specific Corrections ............................................................... 74
3.3.1 Radiometric Distortions in VIR Data and Their
Corrections .............................................................................. 74
3.3.2 Radiometric Corrections of SAR Images ............................ 76
3.3.2.1 Multi-Look Processing ...........................................77
3.3.2.2 Spatial Averaging Method ....................................77
3.3.2.3 Filtering Techniques ..............................................77
3.3.2.4 Antenna Pattern ..................................................... 78
3.3.2.5 16-Bit to 8-Bit Data Conversion .............................79
3.3.2.6 Calibration ...............................................................79
3.3.3 Geometric Corrections ..........................................................80
3.3.4 Geometric Distortions in VIR Imagery ..............................80
3.3.5 Geometric Effects in SAR Data ............................................ 81
3.4 Geometric Corrections .......................................................................82
3.4.1 Terms .......................................................................................82
3.4.2 Geometric Models ..................................................................83
3.4.2.1 Polynomial Rectification .......................................84
3.4.2.2 Sensor Model ..........................................................85
3.4.3 Resampling .............................................................................86
Image Enhancement ...........................................................................88
3.5.1 Spatial Filters ..........................................................................88
3.5.2 Frequency Domain ................................................................89
3.5.3 Histogram Matching .............................................................89
3.6
Summary ..............................................................................................89
References .......................................................................................................90
3.5
4. Fusion Techniques ........................................................................................93
4.1 Categorizations of Image Fusion Techniques .................................95
4.2 Role of Color in Image Fusion ...........................................................96
4.3 Component Substitutions ..................................................................99
4.3.1
Intensity Hue Saturation Transform ................................. 101
4.3.2 Principal Component Substitution .................................... 105
4.3.3 Gram–Schmidt Transform .................................................. 109
4.4 Numerical Methods .......................................................................... 112
4.4.1
Sum ........................................................................................ 112
4.4.2 Multiplication ....................................................................... 113
4.4.3
Subtractive Resolution Merge and Ratioing .................... 113
4.4.4 Brovey Transform ................................................................ 115
4.4.5 Modulation-Based Techniques .......................................... 116
4.4.6 Color Normalized Spectral Sharpening ........................... 116
4.4.7 High-Pass Filtering .............................................................. 117
Statistical Image Fusion ................................................................... 119
4.5.1
Spatially Adaptive Image Fusion....................................... 119
4.5.2 Regression Variable Substitution ....................................... 120
4.5
Contents
vii
4.5.3 Principal Component Analysis .......................................... 120
4.5.4 Minimum Mean-Square Error Approach ........................ 121
4.5.5 FuzeGoTM ............................................................................... 121
4.5.6 Regression Fusion ................................................................ 121
4.6 Multiresolution Approaches ............................................................ 123
4.6.1 Principles ............................................................................... 124
4.6.2 Fusion Process ...................................................................... 125
4.6.3 Advances in MRA Fusion ................................................... 125
4.7 Hybrid Techniques ............................................................................ 127
4.7.1 Ehlers Fusion ........................................................................ 127
4.7.2
IHS–BT Fusion ...................................................................... 127
4.7.3 Wavelet-IHS Fusion ............................................................. 129
4.7.4 Wavelet-PCA Fusion ............................................................ 129
4.7.5 Modified BT-Wavelet Fusion .............................................. 130
4.7.6 Laplacian Filtering and Multiple Regression ................... 131
4.8 Other Fusion Techniques ................................................................. 132
4.8.1 Classification ......................................................................... 133
4.8.2 Spatial and Temporal Data Fusion Approaches .............. 134
Selection Approach ........................................................................... 136
4.9.1 VIR/SAR ................................................................................ 137
4.9.2 Feature Extraction ................................................................ 138
4.9.3 Optimum Index Factor ........................................................ 138
4.10 Comparison and Popularity of Fusion Techniques ..................... 139
4.11 Summary ............................................................................................ 141
References ..................................................................................................... 142
4.9
5. Quality Assessment ................................................................................... 151
5.1 Overview ............................................................................................ 151
5.2
Image Quality Parameters ............................................................... 154
5.3 Acceptable Quality Measures in RSIF ........................................... 155
5.3.1 Mean Bias .............................................................................. 155
5.3.2 Mean Square Error............................................................... 156
5.3.3 Root Mean Square Error ..................................................... 156
5.3.4 Normalized Root Mean Square Error ............................... 156
5.3.5 Correlation Coefficient ........................................................ 156
5.3.6 Universal Image Quality Index ......................................... 157
5.3.7 UIQI for Four Band Imagery .............................................. 157
5.3.8 Spectral Angle Mapper ....................................................... 158
5.3.9 Relative Average Spectral Error ......................................... 158
5.3.10 Relative Dimensionless Global Error of Synthesis .......... 159
5.3.11 Structure Similarity Index Metric ..................................... 159
5.4 RSIF Quality Assessment Protocols ............................................... 159
5.4.1 Wald’s Protocol ..................................................................... 160
5.4.2 Zhou’s Protocol ..................................................................... 160
viii
Contents
5.4.3 Quality with No Reference Protocol ................................. 161
5.4.4 Khan’s Protocol..................................................................... 162
5.5 Palubinskas’ Generic Framework for Pansharpening
Quality Assessment .......................................................................... 162
5.6 Value and Requirements for Visual Evaluation ............................ 165
5.7
Summary ............................................................................................ 167
References ..................................................................................................... 167
6. Applications ................................................................................................. 171
6.1 Urban Studies .................................................................................... 172
6.1.1 Global Urban Extent Mapping ........................................... 173
6.1.2 Local Urban Mapping ......................................................... 174
6.1.3 Outlook .................................................................................. 176
6.2 Agriculture ......................................................................................... 178
6.2.1 Mapping Growth Cycles, Plant Health, and
Cropland Extent ................................................................... 178
6.2.2 Yield Prediction .................................................................... 180
6.3 Land Use/Land Cover Mapping..................................................... 181
6.4 Change Detection .............................................................................. 183
6.4.1 Fusion Techniques in Change Detection .......................... 185
6.4.2 Examples of Change Detection .......................................... 185
6.5 Geology ............................................................................................... 186
6.6 Vegetation Monitoring ..................................................................... 190
6.6.1
Spatial and Temporal Data Fusion Approach .................. 190
6.6.2 Multimodal Classification .................................................. 192
6.7
Forest Monitoring ............................................................................. 194
6.8 Natural Hazards and Disasters ...................................................... 195
6.8.1 Floods .................................................................................... 197
6.8.2 Volcano Eruptions ................................................................ 198
6.8.3 Landslides ............................................................................. 199
6.9 Coastal Zone Monitoring .................................................................200
6.10 Coal Fire Monitoring ........................................................................203
6.11 Detection of Landmines and Minefields ....................................... 207
6.12 Oceanography .................................................................................... 212
6.13 Security and Defense ........................................................................ 213
6.14 Missing Information Reconstruction ............................................. 214
6.15 Other Applications ............................................................................ 215
6.16 Summary ............................................................................................ 217
References .....................................................................................................220
7. Conclusions and Trends ............................................................................233
7.1 Challenges ..........................................................................................233
7.2 Trends .................................................................................................234
7.3 Data Mining .......................................................................................236