INTRODUCTI
N
TO
LINEAR ALGEBRA
Fifth Edition
GILBERT STRANG
Massachusetts
Institute
of Technology
WELLESLEY -CAMBRIDGE PRESS
MA 02482
Box 812060 Wellesley
Introduction
Copyright
©2016 by Gilbert
ISBN 978-0-9802327-
to Linear Algebra,
Strang
7-6
5th Edition
reserved.
All rights
by any means, including
Press.
Wellesley
authorized
are arranged
BTEX typesetting
Printed
-Cambridge
translations
in the United States
of America
No part of this book may be reproduced
from
prohibited
photocopying,
without
Translation
permission
is strictly
or stored
written
or transmitted
in any language
by the publisher.
-
by Ashley C. Fernandes (info@problemsolvingpathway.com)
QA184.S78
2016 512'.5 93-14092
9876543
Other texts from Wellesley
-Cambridge
Press
Computational
Science
and Engineering
, Gilbert
Strang
ISBN 978-0-9614088-1-
7
Wavelets
and Filter
Banks, Gilbert
Strang
and Truong Nguyen
ISBN 978-0-9614088-
7-9
Introduction
to Applied
Mathematics,
Gilbert
Strang
ISBN 978-0-9614088-0-0
Calculus
Third Edition
(2017),
Gilbert
Strang
ISBN 978-0-9802327-5-2
Algorithms
for Global Positioning,
Kai Borre & Gilbert
Strang ISBN 978-0-9802327-3-8
Essays in Linear Algebra,
Gilbert
Strang
ISBN 978-0-9802327-6-9
Differential
Equations
and Linear Algebra,
Gilbert
Strang ISBN 978-0-9802327-9-0
An Analysis
of the Finite
Element
Method,
2008 edition,
Gilbert
Strang
and George Fix
ISBN 978-0-9802327-0-
7
Press
-Cambridge
Wellesley
Box 812060
Wellesley
www.wellesleycambridge.com
linearalgebrabook@gmail.com
math.mit.edu/�gs
phone(781)431-8488
fax (617) 253-4358
MA 02482 USA
The website
The Solution
for this book is math.mit.edu/linearalgebra
Manual can be printed
from that website.
.
Course material
are available
including
syllabus
and exams and also videotaped
lectures
on the book website
and the teaching
website:
web.mit.edu/18.06
is included
Linear
Algebra
This provides
MATLAB® is a registered
video lectures
of the full linear
algebra
course
of The Math Works, Inc.
trademark
in MIT's OpenCourseWare
site ocw.mit.edu
18.06 and 18.06 SC.
.
a central idea of linear
algebra.
The front cover captures
Ax = bis solvable
One particular
Add any vector
The complete
solution
z from the (green)
solution
when bis in the (red) column space of A.
y is in the (yellow)
row space: Ay = b.
nullspace
of A: Az = 0.
is x = y + z. Then Ax = Ay + Az = b.
The cover design
was the inspiration
of Lois Sellers
and Gail Corbett.
Table of Contents
1
Introduction to Vectors
1.1 Vectors and Linear Combinations .
1.2 Lengths and Dot Products .
1.3 Matrices . ... .. ...
2 Solving Linear Equations
2.1 Vectors and Linear Equations .
2.2 The Idea of Elimination . . .
2.3 Elimination Using Matrices .
2.4 Rules for Matrix Operations
2.5
Inverse Matrices . . . . . . .
2.6 Elimination = Factorization: A = LU
2.7 Transposes and Permutations . .. . ..
3 Vector Spaces and Subspaces
3.1 Spaces of Vectors . . . . . . . ............
3.2 The Nullspace of A: Solving Ax = 0 and Rx = 0
3.3 The Complete Solution to Ax = b .
3.4
Independence, Basis and Dimension
3.5 Dimensions of the Four Subspaces
4 Orthogonality
4.1 Orthogonality of the Four Subspaces
4.2 Projections ... . .. . . .. ...
4.3 Least Squares Approximations . . .
4.4 Orthonormal Bases and Gram-Schmidt .
5 Determinants
5.1 The Properties of Determinants .
5.2 Permutations and Cofactors . . .
5.3 Cramer's Rule, Inverses, and Volumes
iii
1
2
11
22
31
31
46
58
70
83
97
109
123
123
135
150
164
181
194
194
206
219
233
247
247
258
273
iv
Table of Contents
6 Eigenvalues and Eigenvectors
6.1
Introduction to Eigenvalues . .
6.2 Diagonalizing a Matrix . . . .
6.3 Systems of Differential Equations
6.4 Symmetric Matrices . . . . . . . .
6.5 Positive Definite Matrices . . . . .
7 The Singular Value Decomposition (SVD)
Image Processing by Linear Algebra . . . .
7 .1
7 .2 Bases and Matrices in the SVD . . . . . . .
7 .3 Principal Component Analysis (PCA by the SVD) .
7.4 The Geometry of the SVD
. . . . . . . . . . . . .
8 Linear Transformations
8.1 The Idea of a Linear Transformation
8.2 The Matrix of a Linear Transformation .
8.3 The Search for a Good Basis .
. .. . .
9 Complex Vectors and Matrices
9.1 Complex Numbers . . . . . .
9.2 Hermitian and Unitary Matrices
9.3 The Fast Fourier Transform .
10 Applications
10.l Graphs and Networks . ... . ... .. .. .
10.2 Matrices in Engineering . . . . . . . . . . . .
10.3 Markov Matrices, Population, and Economics
10.4 Linear Programming . . . . . . . . . . .
. .
10.5 Fourier Series: Linear Algebra for Functions .
10.6 Computer Graphics . . . . . . . .
10.7 Linear Algebra for Cryptography .. . ... .
11 Numerical Linear Algebra
11 .1 Gaussian Elimination in Practice
11.2 Norms and Condition Numbers .
11.3 Iterative Methods and Preconditioners
12 Linear Algebra in Probability & Statistics
12.1 Mean, Variance, and Probability
. . . . . .
12.2 Covariance Matrices and Joint Probabilities
12.3 Multivariate Gaussian and Weighted Least Squares
Matrix Factorizations
Index
Six Great Theorems I Linear Algebra in a Nutshell
288
288
304
319
338
350
364
364
371
382
392
401
401
411
421
430
431
438
445
452
452
462
474
483
490
496
502
508
508
518
524
535
535
546
555
563
565
574
Preface
I am happy for you to see this Fifth Edition
This is the text for my video lectures
also YouTube).
I hope those lectures
will be useful
on MIT's OpenCourseWare
of Introduction
to Linear Algebra.
(ocw.mit.edu
and
to you (maybe even enjoyable
!).
Hundreds
of coll�ges and universities
have chosen this textbook
for their basic linear
A sabbatical
gave me a chance to prepare
two new chapters
course.
algebra
probability
probably
and statistics
and understanding
data. Thousands
of other improvements
only noticed
by the author.
. . Here is a new addition
for students
and all readers:
about
too
opens with a brief summary
Every section
read a new section,
it in your mind, those lines are a quick guide and an aid to memory.
its contents.
When you
to review
to explain
a section
and when you revisit
and organize
big change comes on this book's
to the Problem
Another
now contains
much more flexible
than printing
solutions
Sets in the book. With unlimited
There are three key websites
short solutions.
space,
this is
:
website
math.mit.edu/linearalgebra.
That site
come from thousands
of students
and faculty about linear
site. The 18.06 and 18.06 SC courses
include
algebra
video lectures
of
Messages
ocw.mit.edu
on this OpenCourseWare
a complete
subject
be 2 a.m. (The reader
world have seen these videos
semester
doesn't
of classes.
based on this textbook-the
professor's
Those lectures
offer an independent
review
time stays free and the student's
time can
around the
viewers
of the whole
have to be in a class at all.)
Six million
(amazing). I hope you find them helpful.
as it is taught,
web.mit.edu/18.06
course
Teaching
as useful
Codes, and short essays
to you as possible,
This site has homeworks
and exams (with solutions)
for the current
and as far back as 1996. There are also review
questions,
Java demos,
(and the video lecture
s). My goal is to make this book
we can provide.
with all the course
material
math.mit.edu/linearalgebr
space to explain
to Exercises-with
ferent sources-practice
development
problems,
and Julia and Python, plus whole collections
a This has become an active
of textbook
of exams (18.06
ideas.
examples,
codes in MATLAB
and others)
for review.
There are also new exercises
from many dif
website.
It now has Solutions
Please
visit
this linear
algebra
site. Send suggestions to linearalgebrabook@gm
ail.com
V
vi
Preface
The Fifth Edition
The cover shows the Four Fundamental
on the left side, the column space and the nulls
to put the central
in Chapter
ideas of the subject
3, you will understand
why that picture
Those were named the Four Fundamental
on display
Subspaces-the row space and nullspace
are
pace of AT are on the right.
like this! When you meet those four spaces
It is not usual
from a matrix A. Each row of A is a vector
has m rows, each column is a vector
linear
of a matrix-vector
multiplication.
is to take linear
algebra
in m-dimensional
of column vectors.
combinations
Ax is a combination
of the columns
of A.
space. The crucial
This is exactly
operation
the result
in
Subspaces
in n-dimensional
in my first book, and they start
space. When the matrix
is so central
to linear algebra.
When we take all combinations
Ax of the column vectors,
we get the column space.
If this space
includes
the vector
b, we can solve the equation
Ax = b.
May I call special
attention
to Section
1.3, where these ideas come early-with two
You are not expected
examples.
specific
But you will see the first matrices
There is even an inverse matrix
language
of linear
algebra
and its connection
to calculus.
way: by using it.
in the best and most efficient
spaces
in one day!
of their column spaces.
You will be learning
the
to catch every detail
of vector
in the book, and a picture
Every section
of the basic course
ends with a large collection
of review
problems.
ask you to use the ideas in that section--the
that space,
look for computations
Challenge
Problems go a step further,
by hand on a small matrix,
the rank and inverse
and determinant
and sometimes
and eigenvalues
and they have been highly
of A. Many problems
praised.
The
deeper.
Let me give four examples:
dimension
of the column space,
They
a basis for
Section
2.1: Which row exchanges
of a Sudoku matrix
produce
another
Sudoku matrix?
Section
2.7: If Pis a permutation
matrix,
why is some power pk equal to I?
Section
3.4: If Ax= band Cx = b have the same solutions
for every b, does A equal C?
Section
the row space,
4.1: What conditions
the nullspace,
on the four vectors
r, n, c, £ allow them to be bases for
the column space,
and the left nullspace
of a 2 by
2 matrix?
The Start of the Course
The equation
Ax is a combination
b. The solution
produces
Ax = b uses the language
of linear
combinations
of the columns
of A. The equation
is asking
right away. The vector
for a combination
that
vector
x comes at three levels
and all are important:
1.Direct
solution
to find x by forward
elimination
and back substitution.
2.Matrix solution
using the inverse
matrix:
x = A-1b (if A has an inverse).
3.Particular
solution
(to Ay = b) plus nullspace
solution
(to Az = 0).
That vector
space solution
x = y + z is shown on the cover of the book.
Preface
vii
Direct
elimination
is the most frequently
triangular-then
used algorithm
come quickly.
in scientific
We also see bases for the four
computing.
The
A becomes
matrix
subspaces.
solutions
on practicing
But don't spend forever
elimination
. . .
good ideas are coming.
The speed of every new supercomputer
is tested
on Ax = b : pure linear
algebra.
But
even a supercomputer
doesn't
formula x = A-lb but not the top speed.
even slower-there
determinant
of an n by n matrix.
is no way a linear
And everyone
must know that determinants
algebra
course
should
Those formulas
have a place,
are
begin with formulas
for the
but not first place.
want the inverse
matrix:
too slow. Inverses
give the simplest
Structure
of the Textbook
in this preface,
Already
to explain
of linear
numbers
this beautiful
algebra
to vectors
Here are 12 points
you can see the style of the book and its goal. That goal is serious,
and usefulpart
You will see how the applications
of mathematics.
reinforce
the key ideas.
This book moves gradually
to subspaces-each
level comes naturally
from this book
and everyone
:
and teaching
about learning
and steadily
can get it.
from
1.Chapter
1 starts
with vectors
and dot products.
If the class has met them before,
focus quickly
vectors
vectors
on linear
combinations.
Section
1.3 provides
three independent
whose combinations
in a plane.
Those two examples
space,
are the beginning
fill all of 3-dimensional
and three dependent
of linear algebra.
2.Chapter
2 shows the row picture
and the column picture
of Ax = b. The heart of
is in that connection
linear algebra
the same numbers
an elimination
the whole process-start
between
pictures.
A to produce
matrix E multiplies
but very different
with A, multiply
by E's, end with U.
the rows of A and the columns of
of matrices:
the algebra
Then begins
A :
a zero. The goal is to capture
Elimination
the forward elimination
steps,
is seen in the beautiful
form A = LU. The lower triangular
L holds
and U is upper triangular
for back substitution.
algebra
3.Chapter
all linear
columns
the key information
3 is linear
combinations
are needed?
of the columns.
The answer tells
at the best level:
question
is: How many of those
The crucial
us the dimension of the
column space,
of Linear
Theorem
about A. We reach the Fundamental
subspaces.
The column space contains
Algebra.
and
4.With more equations
than unknowns,
it is almost
We cannot throw out every measurement
When we solve by least squares,
matrix
the key will be the matrix
mathematics,
that is close but not perfectly
exact!
AT A. This wonderful
when A is rectangular.
sure that Ax = b has no solution.
everywhere
in applied
appears
5.Determinants
give formulas
for all that has come before-Cramer's
Rule,
volumes
matrices,
inverse
pute. They slow us down. But det A = 0 tells when a matrix
the key to eigenvalues.
We don't need those formulas
to com
is singular
: this is
inn dimensions.
vm
Preface
6.Section
6 .1 explains
eigenvalues
eigenvalues
early.
because
It is completely
the determinant
is easy for a 2 by 2 matrix.
for 2 by 2 matrices.
reasonable
to come here directly
The key equation
from Chapter
want to see
3,
is Ax= >.x.
Many courses
and eigenvectors
Eigenvalues
They are not for Ax = b, they are for dynamic
The idea is always the same: follow
A acts like a single
number (the eigenvalue
the eigenvectors.
>.) and the problem
are an astonishing
equations
a square
like du/ dt = Au.
way to understand
matrix.
In those special
directions,
is one-dimensional.
highlight
An essential
When all the eigenvalues
idea connects
and energy.
of Chapter
are positive,
6 is diagonalizing
is "positive
the matrix
a symmetric
definite".
matrix.
This key
and determinants
the whole course-positive
pivots
I work hard to reach this point in the book and to explain
and eigenvalues
it by examples.
7.Chapter
7 is new. It introduces
singular
all martices
into simple
one way to compress
values
and singular
They separate
You will see
ranked in order of their importance.
full of data.
you can analyze
vectors.
a matrix
an image. Especially
pieces,
8.Chapter
8 explains
linear
transformations.
This is geometry
without
with no coordinates.
When we choose a basis,
we reach the best possible
axes, algebra
matrix.
9.Chapter
9 moves from real numbers
and vectors
to complex
The Fourier
the Fast Fourier Transform (multiplying
quickly
F is the most important
matrix
complex
matrix
by F and p-1) is revolutionary.
and matrices.
vectors
we will ever see. And
10.Chapter
10 is full of applications,
more than any single
course
could need:
in Engineering-differential
to the edge-node
equations
matrix
parallel
for Kirchhoff's Laws
to matrix
equations
in Google's
Programming-a
10.1 Graphs and Networks-leading
10.2 Matrices
10.3 Markov Matrices-as
10.4 Linear
10.5 Fourier Series-linear
algebra
10.6 Computer
Algebra in Cryptography-this
10.7 Linear
is not too secure.
Cipher
Multiplication
gives 4 x 5
for functions
move and rotate
Graphics-matrices
It uses modular
new requirement
PageRank
algorithm
x 2'. 0 and minimization
processing
images
and compress
and digital
signal
of the cost
new section
was fun to write.
arithmetic:
1 (mod 19). For decoding
integers
this gives 4-1
The Hill
from O to p -1.
5.
=
=
11.How should
computing
be included
in a linear
algebra
understanding
Mathematica
accessible
of matrices-every
are powerful in
on the Web. Those newer languages
class will
different
are powerful
too !
ways. Julia and P ython are free and directly
find a balance.
course?
It can open a new
MATLAB and Maple and
Basic commands
gorithms.You
begin in Chapter
2. Then Chapter
11 moves toward
professional
al
can upload and download
codes for this course on the website.
12.Chapter
12 on Probability
and Statistics
is new, with truly important
When random variables
they are symmetric
positive
we get covariance
in Chapter
algebra
6 is needed
now.
are not independent
definite.
The linear
applications.
Fortunately
matrices.