in computervision
Richard Hartley and Andrew Zisserman
CAMBRIDGE
ENGINEERING LIBRARY
Multiple View Geometry in Computer Vision
Second Edition
Richard Hartley
Australian National University,
Canberra, Australia
Andrew Zisserman
University of Oxford, UK
C A M B R I D GE
UNIVERSITY PRESS
PUBLISHED BY THE PRESS SYNDICATE OF THE UNIVERSITY OF CAMBRIDGE
The Pitt Building, Trumpington Street, Cambridge, United Kingdom
CAMBRIDGE UNIVERSITY PRESS
The Edinburgh Building, Cambridge, CB2 2RU, UK
40 West 20th Street, New York, NY 10011-4211, USA
477 Wiiiiamstown Road, Port Melbourne, VIC 3207, Australia
Ruiz de Alarcon 13, 28014 Madrid, Spain
Dock House, The Waterfront, Cape Town 8001, South Africa
http://www.cambridge.org
© Cambridge University Press 2000, 2003
This book is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the
written permission of Cambridge University Press.
First Published 2000
Reprinted 2001, 2002
Second Edition 2003
Printed in the United Kingdom at the University Press, Cambridge
A catalogue record for this book is available from the British Library
Library of Congress Cataloguing in Publication data
ISBN 0521 54051 8 hardback
This book
led us intc
Dedication
This book is dedicated to Joe Mundy whose vision and constant search for new ideas
led us into this field.
Contents
Foreword
Preface
1
Introduction - the ubiquitous projective geometry
Camera projections
Reconstruction from more than one view
Three-view geometry
Four view geometry and n-view reconstruction
Transfer
Euclidean reconstruction
Introduction - a Tour of Multiple View Geometry
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8 Auto-calibration
1.9
The reward 1: 3D graphical models
1.10 The reward II: video augmentation
-•&'
page xi
xiii
1
1
6
10
12
13
14
16
17
18
19
PART 0: The Background: Projective Geometry, Transformations and Esti
mation
Outline
2 Projective Geometry and Transformations of 2D
Planar geometry
The 2D projective plane
Projective transformations
2.1
2.2
2.3
2.4 A hierarchy of transformations
2.5
The projective geometry of ID
Topology of the projective plane
2.6
2.7
Recovery of affine and metric properties from images
2.8 More properties of conies
2.9
2.10 Closure
Fixed points and lines
3 Projective Geometry and Transformations of 3D
3.1
3.2
Points and projective transformations
Representing and transforming planes, lines and quadrics
v
23
24
25
25
26
32
37
44
46
47
58
61
62
65
65
66
Contents
3.3
3.4
3.5
3.6
3.7
3.8
Twisted cubics
The hierarchy of transformations
The plane at infinity
The absolute conic
The absolute dual quadric
Closure
4 Estimation - 2D Projective Transformations
The Direct Linear Transformation (DLT) algorithm
4.1
4.2 Different cost functions
4.3
4.4
4.5
4.6
4.7
4.8 Automatic computation of a homography
4.9
Statistical cost functions and Maximum Likelihood estimation
Transformation invariance and normalization
Iterative minimization methods
Experimental comparison of the algorithms
Robust estimation
Closure
5 Algorithm Evaluation and Error Analysis
Bounds on performance
Covariance of the estimated transformation
5.1
5.2
5.3 Monte Carlo estimation of covariance
5.4
Closure
PART I: Camera Geometry and Single View Geometry
Outline
6 Camera Models
Finite cameras
The projective camera
Cameras at infinity
6.1
6.2
6.3
6.4 Other camera models
6.5
Closure
7 Computation of the Camera Matrix P
7.1
Basic equations
7.2 Geometric error
7.3
7.4
7.5
Restricted camera estimation
Radial distortion
Closure
8 More Single View Geometry
Images of smooth surfaces
8.1 Action of a projective camera on planes, lines, and conies
8.2
8.3 Action of a projective camera on quadrics
8.4
8.5
The importance of the camera centre
Camera calibration and the image of the absolute conic
75
77
79
81
83
85
87
88
93
102
104
110
115
116
123
127
132
132
138
149
150
151
152
153
153
158
166
174
176
178
178
180
184
189
193
195
195
200
201
202
208
Contents
8.6 Vanishing points and vanishing lines
8.7 Affine 3D measurements and reconstruction
8.8 Determining camera calibration K from a single view
8.9
8.10 The calibrating conic
8.11 Closure
Single view reconstruction
PART II: Two-View Geometry
Outline
9 Epipolar Geometry and the Fundamental Matrix
9.1
Epipolar geometry
9.2
The fundamental matrix F
9.3
Fundamental matrices arising from special motions
9.4 Geometric representation of the fundamental matrix
9.5
9.6
9.7
Retrieving the camera matrices
The essential matrix
Closure
10 3D Reconstruction of Cameras and Structure
10.1 Outline of reconstruction method
10.2 Reconstruction ambiguity
10.3 The projective reconstruction theorem
10.4 Stratified reconstruction
10.5 Direct reconstruction - using ground truth
10.6 Closure
11 Computation of the Fundamental Matrix F
11.1 Basic equations
11.2 The normalized 8-point algorithm
11.3 The algebraic minimization algorithm
11.4 Geometric distance
11.5 Experimental evaluation of the algorithms
11.6 Automatic computation of F
11.7 Special cases of F-computation
11.8 Correspondence of other entities
11.9 Degeneracies
11.10 A geometric interpretation of F-computation
11.11 The envelope of epipolar lines
11.12 Image rectification
11.13 Closure
12 Structure Computation
12.1 Problem statement
12.2 Linear triangulation methods
12.3 Geometric error cost function
12.4 Sampson approximation (first-order geometric correction)
VI]
213
220
223
229
231
233
237
238
239
239
241
247
250
253
257
259
262
262
264
266
267
275
276
279
279
281
282
284
288
290
293
294
295
297
298
302
308
310
310
312
313
314
Vlll
Contents
12.5 An optimal solution
12.6 Probability distribution of the estimated 3D point
12.7 Line reconstruction
12.8 Closure
13 Scene planes and homographies
13.1 Homographies given the plane and vice versa
13.2 Plane induced homographies given F and image correspondences
13.3 Computing F given the homography induced by a plane
13.4 The infinite homography HQO
13.5 Closure
14 Affine Epipolar Geometry
14.1 Affine epipolar geometry
14.2 The affine fundamental matrix
14.3 Estimating FA from image point correspondences
14.4 Triangulation
14.5 Affine reconstruction
14.6 Necker reversal and the bas-relief ambiguity
14.7 Computing the motion
14.8 Closure
PART III: Three-View Geometry
Outline
15 The Trifocal Tensor
15.1 The geometric basis for the trifocal tensor
15.2 The trifocal tensor and tensor notation
15.3 Transfer
15.4 The fundamental matrices for three views
15.5 Closure
16 Computation of the Trifocal Tensor T
16.1 Basic equations
16.2 The normalized linear algorithm
16.3 The algebraic minimization algorithm
16.4 Geometric distance
16.5 Experimental evaluation of the algorithms
16.6 Automatic computation of T
16.7 Special cases of T-computation
16.8 Closure
PART IV: N-View Geometry
Outline
17 ^-Linearities and Multiple View Tensors
17.1 Bilinear relations
17.2 Trilinear relations
315
321
321
323
325
326
329
334
338
340
344
344
345
347
353
353
355
357
360
363
364
365
365
376
379
383
387
391
391
393
395
396
399
400
404
406
409
410
411
411
414