An Introduction to
Numerical Analysis
Endre S¨uli and David F. Mayers
University of Oxford
published by the press syndicate of the university of cambridge
The Pitt Building, Trumpington Street, Cambridge, United Kingdom
cambridge university press
The Edinburgh Building, Cambridge CB2 2RU, UK
40 West 20th Street, New York, NY 10011-4211, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
Ruiz de Alarc´on 13, 28014 Madrid, Spain
Dock House, The Waterfront, Cape Town 8001, South Africa
http://www.cambridge.org
C Cambridge University Press, 2003
This book is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without
the written permission of Cambridge University Press.
First published 2003
Printed in the United Kingdom at the University Press, Cambridge
Typeface CMR 10/13 pt
System LATEX 2ε [TB]
A catalogue record for this book is available from the British Library
Library of Congress Cataloguing in Publication data
ISBN 0 521 81026 4 hardback
ISBN 0 521 00794 1 paperback
Contents
Solution of equations by iteration
Introduction
Simple iteration
Iterative solution of equations
Relaxation and Newton’s method
The secant method
The bisection method
Preface
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7 Global behaviour
1.8
Notes
Exercises
Solution of systems of linear equations
Introduction
2
2.1
2.2 Gaussian elimination
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10 Notes
LU factorisation
Pivoting
Solution of systems of equations
Computational work
Norms and condition numbers
Hilbert matrix
Least squares method
Exercises
Special matrices
Introduction
Symmetric positive definite matrices
Tridiagonal and band matrices
3
3.1
3.2
3.3
iii
page vii
1
1
2
17
19
25
28
29
32
35
39
39
44
48
52
55
56
58
72
74
79
82
87
87
87
93
iv
Contents
3.4 Monotone matrices
3.5
Notes
Exercises
Simultaneous nonlinear equations
Introduction
Simultaneous iteration
Relaxation and Newton’s method
4
4.1
4.2
4.3
4.4 Global convergence
4.5
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.7.1 The QR factorisation revisited
5.7.2 The definition of the QR algorithm
Inverse iteration for the eigenvectors
5.8
5.9
The Rayleigh quotient
5.10 Perturbation analysis
5.11 Notes
98
101
102
104
104
106
116
123
124
Notes
Exercises
126
Eigenvalues and eigenvectors of a symmetric matrix 133
Introduction
133
137
The characteristic polynomial
137
Jacobi’s method
145
The Gerschgorin theorems
Householder’s method
150
156
Eigenvalues of a tridiagonal matrix
162
The QR algorithm
162
164
166
170
172
174
175
179
179
180
185
187
191
194
195
200
200
201
204
208
209
Notes
Exercises
Numerical integration – I
Introduction
Newton–Cotes formulae
Error estimates
The Runge phenomenon revisited
Composite formulae
Exercises
Polynomial interpolation
Introduction
Lagrange interpolation
Convergence
Hermite interpolation
6
6.1
6.2
6.3
6.4
6.5 Differentiation
6.6
7
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7.8
8
8.1
8.2
8.3
8.4
8.5
8.6
Contents
The Euler–Maclaurin expansion
Extrapolation methods
Notes
Exercises
Polynomial approximation in the ∞-norm
Introduction
Normed linear spaces
Best approximation in the ∞-norm
Chebyshev polynomials
Interpolation
Notes
Exercises
Approximation in the 2-norm
Introduction
Inner product spaces
Best approximation in the 2-norm
9
9.1
9.2
9.3
9.4 Orthogonal polynomials
9.5
9.6
Comparisons
Notes
Exercises
Introduction
10 Numerical integration – II
10.1
10.2 Construction of Gauss quadrature rules
10.3 Direct construction
10.4 Error estimation for Gauss quadrature
10.5 Composite Gauss formulae
10.6 Radau and Lobatto quadrature
10.7 Note
Exercises
Piecewise polynomial approximation
Introduction
11
11.1
11.2 Linear interpolating splines
11.3 Basis functions for the linear spline
11.4 Cubic splines
11.5 Hermite cubic splines
11.6 Basis functions for cubic splines
11.7 Notes
Exercises
v
211
215
219
220
224
224
224
228
241
244
247
248
252
252
253
256
259
270
272
273
277
277
277
280
282
285
287
288
288
292
292
293
297
298
300
302
306
307
vi
Contents
Initial value problems for ODEs
Introduction
12
12.1
12.2 One-step methods
12.3 Consistency and convergence
12.4 An implicit one-step method
12.5 Runge–Kutta methods
12.6 Linear multistep methods
12.7 Zero-stability
12.8 Consistency
12.9 Dahlquist’s theorems
12.10 Systems of equations
12.11 Stiff systems
12.12 Implicit Runge–Kutta methods
12.13 Notes
Exercises
Boundary value problems for ODEs
Introduction
13
13.1
13.2 A model problem
13.3 Error analysis
13.4 Boundary conditions involving a derivative
13.5 The general self-adjoint problem
13.6 The Sturm–Liouville eigenvalue problem
13.7 The shooting method
13.8 Notes
Exercises
The finite element method
Introduction: the model problem
14
14.1
14.2 Rayleigh–Ritz and Galerkin principles
14.3 Formulation of the finite element method
14.4 Error analysis of the finite element method
14.5 A posteriori error analysis by duality
14.6 Notes
Exercises
Appendix A An overview of results from real analysis
Appendix B WWW-resources
Bibliography
Index
310
310
317
321
324
325
329
331
337
340
341
343
349
353
355
361
361
361
364
367
370
373
375
380
381
385
385
388
391
397
403
412
414
419
423
424
429
1
Solution of equations by iteration
1.1 Introduction
Equations of various kinds arise in a range of physical applications and
a substantial body of mathematical research is devoted to their study.
Some equations are rather simple: in the early days of our mathematical
education we all encountered the single linear equation ax+b = 0, where
a and b are real numbers and a = 0, whose solution is given by the
formula x = −b/a. Many equations, however, are nonlinear: a simple
example is ax2 + bx + c = 0, involving a quadratic polynomial with real
coefficients a, b, c, and a = 0. The two solutions to this equation, labelled
x1 and x2, are found in terms of the coefficients of the polynomial from
the familiar formulae
−b +
−b − √
√
b2 − 4ac
2a
,
x2 =
x1 =
b2 − 4ac
2a
.
(1.1)
It is less likely that you have seen the more intricate formulae for the
solution of cubic and quartic polynomial equations due to the sixteenth
century Italian mathematicians Niccolo Fontana Tartaglia (1499–1557)
and Lodovico Ferrari (1522–1565), respectively, which were published
by Girolamo Cardano (1501–1576) in 1545 in his Artis magnae sive de
regulis algebraicis liber unus. In any case, if you have been led to believe
that similar expressions involving radicals (roots of sums of products of
coefficients) will supply the solution to any polynomial equation, then
you should brace yourself for a surprise: no such closed formula exists
for a general polynomial equation of degree n when n ≥ 5. It transpires
that for each n ≥ 5 there exists a polynomial equation of degree n with
1