OT98_LevequeFM2.qxp 6/4/2007 10:20 AM Page 1
Finite Difference Methods
for Ordinary and Partial
Differential Equations
OT98_LevequeFM2.qxp 6/4/2007 10:20 AM Page 2
OT98_LevequeFM2.qxp 6/4/2007 10:20 AM Page 3
Finite Difference Methods
for Ordinary and Partial
Differential Equations
Steady-State and Time-Dependent Problems
Randall J. LeVeque
University of Washington
Seattle, Washington
Society for Industrial and Applied Mathematics • Philadelphia
OT98_LevequeFM2.qxp 6/4/2007 10:20 AM Page 4
Copyright © 2007 by the Society for Industrial and Applied Mathematics.
10 9 8 7 6 5 4 3 2 1
All rights reserved. Printed in the United States of America. No part of this book may be
reproduced, stored, or transmitted in any manner without the written permission of the
publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600
University City Science Center, Philadelphia, PA 19104-2688.
Trademarked names may be used in this book without the inclusion of a trademark symbol.
These names are used in an editorial context only; no infringement of trademark is intended.
MATLAB is a registered trademark of The MathWorks, Inc. For MATLAB product information,
please contact The MathWorks, Inc., 3 Apple Hill Drive, Natick, MA 01760-2098 USA, 508-
647-7000, Fax: 508-647-7101, info@mathworks.com, www.mathworks.com.
Library of Congress Cataloging-in-Publication Data
LeVeque, Randall J., 1955-
Finite difference methods for ordinary and partial differential equations : steady-state and
time-dependent problems / Randall J. LeVeque.
p.cm.
Includes bibliographical references and index.
ISBN 978-0-898716-29-0 (alk. paper)
1. Finite differences. 2. Differential equations. I. Title.
QA431.L548 2007
515’.35—dc22
2007061732
Partial royalties from the sale of this book are placed in a fund to help students
attend SIAM meetings and other SIAM-related activities. This fund is administered
by SIAM, and qualified individuals are encouraged to write directly to SIAM for
guidelines.
is a registered trademark.
OT98_LevequeFM2.qxp 6/4/2007 10:20 AM Page 5
To my family,
Loyce, Ben, Bill, and Ann
i
i
i
“rjlfdm”
2007/6/1
page vii
i
Contents
Preface
Boundary Value Problems and Iterative Methods
Finite Difference Approximations
1.1
.
1.2
1.3
1.4
1.5
.
Truncation errors .
.
.
Deriving finite difference approximations .
.
.
Second order derivatives .
.
Higher order derivatives .
.
A general approach to deriving the coefficients .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
I
1
2
xiii
1
3
5
.
7
.
8
.
9
.
. 10
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Steady States and Boundary Value Problems
2.1
.
.
2.2
.
2.3
.
2.4
2.5
.
.
2.6
.
2.7
.
2.8
.
2.9
2.10
.
2.11
2.12
2.13
2.14
2.15
2.16
13
. 13
.
.
.
The heat equation .
.
. 14
.
Boundary conditions .
.
.
. 14
The steady-state problem .
.
.
. 15
A simple finite difference method .
.
. 17
.
Local truncation error
.
.
. 18
.
.
.
.
Global error
.
. 18
.
.
.
Stability .
.
.
. 19
.
.
.
Consistency .
.
. 19
.
Convergence .
.
.
.
. 20
Stability in the 2-norm .
.
.
. 22
Green’s functions and max-norm stability .
. 29
.
.
Neumann boundary conditions
.
. 32
.
Existence and uniqueness .
.
.
.
. 34
Ordering the unknowns and equations .
.
.
. 35
.
A general linear second order equation .
. 37
Nonlinear equations .
.
.
Discretization of the nonlinear boundary value problem . 38
2.16.1
Nonuniqueness .
. 40
.
2.16.2
. 41
2.16.3
Accuracy on nonlinear equations
. 43
Singular perturbations and boundary layers .
2.17.1
.
. 46
Interior layers
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2.17
i
i
vii
i
i
i
i
viii
2.18
2.19
2.20
2.21
.
.
.
.
.
.
.
.
.
Nonuniform grids .
2.18.1
Continuation methods .
Higher order methods .
2.20.1
2.20.2
2.20.3
Spectral methods .
.
Adaptive mesh selection .
.
.
.
.
.
.
Fourth order differencing .
.
Extrapolation methods .
.
.
Deferred corrections .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4.3
4.4
4.5
4.6
3
4
II
5
i
i
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Contents
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 49
. 51
. 52
. 52
. 52
. 53
. 54
. 55
59
. 59
. 60
. 61
. 63
. 64
. 66
. 66
. 68
69
. 69
. 71
. 74
. 76
. 78
. 79
. 83
. 86
. 88
. 93
. 96
. 96
. 99
. 100
. 101
. 103
. 103
. 106
111
113
. 114
. 115
. 116
i
“rjlfdm”
2007/6/1
page viii
i
i
i
.
.
.
Elliptic Equations
3.1
3.2
3.3
3.4
3.5
3.6
3.7
Steady-state heat conduction .
.
The 5-point stencil for the Laplacian .
Ordering the unknowns and equations .
.
Accuracy and stability .
.
.
.
The 9-point Laplacian .
.
Other elliptic equations
.
Solving the linear system .
.
3.7.1
.
.
.
.
.
.
.
Sparse storage in MATLAB .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Iterative Methods for Sparse Linear Systems
.
4.1
4.2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Jacobi and Gauss–Seidel .
.
.
.
.
.
.
Analysis of matrix splitting methods
.
.
.
.
Rate of convergence .
4.2.1
.
.
.
.
4.2.2
.
Successive overrelaxation .
.
.
Descent methods and conjugate gradients .
.
.
.
The method of steepest descent
4.3.1
.
The A-conjugate search direction .
4.3.2
.
The conjugate-gradient algorithm .
4.3.3
.
Convergence of conjugate gradient
4.3.4
.
4.3.5
Preconditioners
.
Incomplete Cholesky and ILU preconditioners .
4.3.6
The Arnoldi process and GMRES algorithm .
.
4.4.1
.
4.4.2
Newton–Krylov methods for nonlinear problems .
.
Multigrid methods .
.
4.6.1
4.6.2
.
.
.
.
.
.
.
.
.
.
.
.
.
Krylov methods based on three term recurrences .
Other applications of Arnoldi
.
.
.
.
.
.
Slow convergence of Jacobi
The multigrid approach .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Initial Value Problems
The Initial Value Problem for Ordinary Differential Equations
.
5.1
.
.
Linear ordinary differential equations .
.
5.1.1
Lipschitz continuity .
.
Duhamel’s principle .
.
5.2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
i
i
i
i
Contents
5.3
5.4
5.5
5.6
5.7
5.8
5.9
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Existence and uniqueness of solutions .
Systems of equations
.
Significance of the Lipschitz constant .
.
Limitations .
.
.
.
.
.
5.2.1
5.2.2
5.2.3
5.2.4
.
Some basic numerical methods .
.
.
Truncation errors .
.
One-step errors .
.
.
.
Taylor series methods .
Runge–Kutta methods .
.
5.7.1
One-step versus multistep methods .
.
Linear multistep methods .
5.9.1
.
5.9.2
5.9.3
5.9.4
.
.
.
.
.
.
.
.
.
Embedded methods and error estimation .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Local truncation error
.
Characteristic polynomials .
Starting values .
.
.
Predictor-corrector methods .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Zero-Stability and Convergence for Initial Value Problems
.
6.1
.
6.2
.
6.3
.
.
Convergence .
.
The test problem .
.
One-step methods .
6.3.1
6.3.2
6.3.3
6.3.4
.
Zero-stability of linear multistep methods .
6.4.1
.
.
.
.
.
.
Euler’s method on linear problems
.
Relation to stability for boundary value problems .
.
Euler’s method on nonlinear problems
.
.
General one-step methods .
.
.
.
Solving linear difference equations .
6.4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
ix
. 116
. 117
. 118
. 119
. 120
. 121
. 122
. 123
. 124
. 128
. 130
. 131
. 132
. 133
. 134
. 135
137
. 137
. 138
. 138
. 138
. 140
. 141
. 142
. 143
. 144
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6
7
8
.
.
.
.
.
.
.
.
.
.
Absolute Stability for Ordinary Differential Equations
7.1
7.2
7.3
7.4
Unstable computations with a zero-stable method .
.
Absolute stability .
.
.
Stability regions for linear multistep methods .
.
.
Systems of ordinary differential equations
7.4.1
.
.
.
.
.
.
7.4.2
.
.
.
7.4.3
.
.
.
Practical choice of step size .
Plotting stability regions .
.
.
.
.
7.6.1
7.6.2
Relative stability regions and order stars
149
. 149
. 151
. 153
. 156
. 157
. 158
. 160
. 161
. 162
The boundary locus method for linear multistep methods . 162
. 163
Plotting stability regions of one-step methods .
.
. 164
.
Chemical kinetics
Linear systems .
.
Nonlinear systems .
.
.
7.5
7.6
7.7
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Stiff Ordinary Differential Equations
.
8.1
8.2
.
8.3
.
Numerical difficulties .
.
Characterizations of stiffness
.
Numerical methods for stiff problems .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
167
. 168
. 169
. 170
i
“rjlfdm”
2007/6/1
page ix
i
i
i