第1页 / 共72页
第2页 / 共72页
第3页 / 共72页
第4页 / 共72页
第5页 / 共72页
第6页 / 共72页
第7页 / 共72页
第8页 / 共72页
矩阵公式大全.pdf
Basics
Trace
Determinant
The Special Case 2x2
Derivatives
Derivatives of a Determinant
Derivatives of an Inverse
Derivatives of Eigenvalues
Derivatives of Matrices, Vectors and Scalar Forms
Derivatives of Traces
Derivatives of vector norms
Derivatives of matrix norms
Derivatives of Structured Matrices
Inverses
Basic
Exact Relations
Implication on Inverses
Approximations
Generalized Inverse
Pseudo Inverse
Complex Matrices
Complex Derivatives
Higher order and non-linear derivatives
Inverse of complex sum
Solutions and Decompositions
Solutions to linear equations
Eigenvalues and Eigenvectors
Singular Value Decomposition
Triangular Decomposition
LU decomposition
LDM decomposition
LDL decompositions
Statistics and Probability
Definition of Moments
Expectation of Linear Combinations
Weighted Scalar Variable
Multivariate Distributions
Cauchy
Dirichlet
Normal
Normal-Inverse Gamma
Gaussian
Multinomial
Student's t
Wishart
Wishart, Inverse
Gaussians
Basics
Moments
Miscellaneous
Mixture of Gaussians
Special Matrices
Block matrices
Discrete Fourier Transform Matrix, The
Hermitian Matrices and skew-Hermitian
Idempotent Matrices
Orthogonal matrices
Positive Definite and Semi-definite Matrices
Singleentry Matrix, The
Symmetric, Skew-symmetric/Antisymmetric
Toeplitz Matrices
Transition matrices
Units, Permutation and Shift
Vandermonde Matrices
Functions and Operators
Functions and Series
Kronecker and Vec Operator
Vector Norms
Matrix Norms
Rank
Integral Involving Dirac Delta Functions
Miscellaneous
One-dimensional Results
Gaussian
One Dimensional Mixture of Gaussians
Proofs and Details
Misc Proofs
The Matrix Cookbook
[ http://matrixcookbook.com ]
Kaare Brandt Petersen
Michael Syskind Pedersen
Version: November 15, 2012
1
Introduction
What is this? These pages are a collection of facts (identities, approxima-
tions, inequalities, relations, ...) about matrices and matters relating to them.
It is collected in this form for the convenience of anyone who wants a quick
desktop reference .
Disclaimer: The identities, approximations and relations presented here were
obviously not invented but collected, borrowed and copied from a large amount
of sources. These sources include similar but shorter notes found on the internet
and appendices in books - see the references for a full list.
Errors: Very likely there are errors, typos, and mistakes for which we apolo-
gize and would be grateful to receive corrections at cookbook@2302.dk.
Its ongoing: The project of keeping a large repository of relations involving
matrices is naturally ongoing and the version will be apparent from the date in
the header.
Suggestions: Your suggestion for additional content or elaboration of some
topics is most welcome acookbook@2302.dk.
Keywords: Matrix algebra, matrix relations, matrix identities, derivative of
determinant, derivative of inverse matrix, differentiate a matrix.
Acknowledgements: We would like to thank the following for contributions
and suggestions: Bill Baxter, Brian Templeton, Christian Rishøj, Christian
Schr¨oppel, Dan Boley, Douglas L. Theobald, Esben Hoegh-Rasmussen, Evripidis
Karseras, Georg Martius, Glynne Casteel, Jan Larsen, Jun Bin Gao, J¨urgen
Struckmeier, Kamil Dedecius, Karim T. Abou-Moustafa, Korbinian Strimmer,
Lars Christiansen, Lars Kai Hansen, Leland Wilkinson, Liguo He, Loic Thibaut,
Markus Froeb, Michael Hubatka, Miguel Bar˜ao, Ole Winther, Pavel Sakov,
Stephan Hattinger, Troels Pedersen, Vasile Sima, Vincent Rabaud, Zhaoshui
He. We would also like thank The Oticon Foundation for funding our PhD
studies.
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 2
CONTENTS
Contents
1 Basics
CONTENTS
1.1 Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 The Special Case 2x2 . . . . . . . . . . . . . . . . . . . . . . . . .
2 Derivatives
2.1 Derivatives of a Determinant
. . . . . . . . . . . . . . . . . . . .
2.2 Derivatives of an Inverse . . . . . . . . . . . . . . . . . . . . . . .
2.3 Derivatives of Eigenvalues . . . . . . . . . . . . . . . . . . . . . .
2.4 Derivatives of Matrices, Vectors and Scalar Forms
. . . . . . . .
2.5 Derivatives of Traces . . . . . . . . . . . . . . . . . . . . . . . . .
2.6 Derivatives of vector norms . . . . . . . . . . . . . . . . . . . . .
2.7 Derivatives of matrix norms . . . . . . . . . . . . . . . . . . . . .
2.8 Derivatives of Structured Matrices
. . . . . . . . . . . . . . . . .
3 Inverses
3.1 Basic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Exact Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3
Implication on Inverses . . . . . . . . . . . . . . . . . . . . . . . .
3.4 Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5 Generalized Inverse . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6 Pseudo Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Complex Matrices
4.1 Complex Derivatives . . . . . . . . . . . . . . . . . . . . . . . . .
4.2 Higher order and non-linear derivatives . . . . . . . . . . . . . . .
4.3
Inverse of complex sum . . . . . . . . . . . . . . . . . . . . . . .
5 Solutions and Decompositions
5.1 Solutions to linear equations . . . . . . . . . . . . . . . . . . . . .
5.2 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . .
5.3 Singular Value Decomposition . . . . . . . . . . . . . . . . . . . .
5.4 Triangular Decomposition . . . . . . . . . . . . . . . . . . . . . .
5.5 LU decomposition . . . . . . . . . . . . . . . . . . . . . . . . . .
5.6 LDM decomposition . . . . . . . . . . . . . . . . . . . . . . . . .
5.7 LDL decompositions . . . . . . . . . . . . . . . . . . . . . . . . .
6 Statistics and Probability
6.1 Definition of Moments . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Expectation of Linear Combinations . . . . . . . . . . . . . . . .
6.3 Weighted Scalar Variable
. . . . . . . . . . . . . . . . . . . . . .
7 Multivariate Distributions
7.1 Cauchy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2 Dirichlet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.3 Normal
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.4 Normal-Inverse Gamma . . . . . . . . . . . . . . . . . . . . . . .
7.5 Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.6 Multinomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
6
6
7
8
8
9
10
10
12
14
14
14
17
17
18
20
20
21
21
24
24
26
27
28
28
30
31
32
32
33
33
34
34
35
36
37
37
37
37
37
37
37
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 3
CONTENTS
CONTENTS
7.7 Student’s t
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.8 Wishart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7.9 Wishart, Inverse
. . . . . . . . . . . . . . . . . . . . . . . . . . .
8 Gaussians
8.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2 Moments
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.3 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.4 Mixture of Gaussians . . . . . . . . . . . . . . . . . . . . . . . . .
9 Special Matrices
9.1 Block matrices
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.2 Discrete Fourier Transform Matrix, The . . . . . . . . . . . . . .
9.3 Hermitian Matrices and skew-Hermitian . . . . . . . . . . . . . .
9.4
Idempotent Matrices . . . . . . . . . . . . . . . . . . . . . . . . .
9.5 Orthogonal matrices . . . . . . . . . . . . . . . . . . . . . . . . .
9.6 Positive Definite and Semi-definite Matrices . . . . . . . . . . . .
9.7 Singleentry Matrix, The . . . . . . . . . . . . . . . . . . . . . . .
9.8 Symmetric, Skew-symmetric/Antisymmetric . . . . . . . . . . . .
9.9 Toeplitz Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.10 Transition matrices . . . . . . . . . . . . . . . . . . . . . . . . . .
9.11 Units, Permutation and Shift . . . . . . . . . . . . . . . . . . . .
9.12 Vandermonde Matrices . . . . . . . . . . . . . . . . . . . . . . . .
10 Functions and Operators
10.1 Functions and Series . . . . . . . . . . . . . . . . . . . . . . . . .
10.2 Kronecker and Vec Operator
. . . . . . . . . . . . . . . . . . . .
10.3 Vector Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.4 Matrix Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.5 Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10.6 Integral Involving Dirac Delta Functions . . . . . . . . . . . . . .
10.7 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A One-dimensional Results
A.1 Gaussian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A.2 One Dimensional Mixture of Gaussians . . . . . . . . . . . . . . .
B Proofs and Details
B.1 Misc Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
38
39
40
40
42
44
44
46
46
47
48
49
49
50
52
54
54
55
56
57
58
58
59
61
61
62
62
63
64
64
65
66
66
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 4
CONTENTS
CONTENTS
Notation and Nomenclature
A
Aij
Ai
Aij
An
A−1
A+
A1/2
(A)ij
Aij
[A]ij
a
ai
ai
a
z
z
Z
z
z
Z
Matrix
Matrix indexed for some purpose
Matrix indexed for some purpose
Matrix indexed for some purpose
Matrix indexed for some purpose or
The n.th power of a square matrix
The inverse matrix of the matrix A
The pseudo inverse matrix of the matrix A (see Sec. 3.6)
The square root of a matrix (if unique), not elementwise
The (i, j).th entry of the matrix A
The (i, j).th entry of the matrix A
The ij-submatrix, i.e. A with i.th row and j.th column deleted
Vector (column-vector)
Vector indexed for some purpose
The i.th element of the vector a
Scalar
Real part of a scalar
Real part of a vector
Real part of a matrix
Imaginary part of a scalar
Imaginary part of a vector
Imaginary part of a matrix
Trace of the matrix A
Eigenvalues of the matrix A
det(A) Determinant of A
Tr(A)
diag(A) Diagonal matrix of the matrix A, i.e. (diag(A))ij = δijAij
eig(A)
vec(A) The vector-version of the matrix A (see Sec. 10.2.2)
sup
||A||
AT
A−T
A∗
AH
A ◦ B Hadamard (elementwise) product
A ⊗ B Kronecker product
Supremum of a set
Matrix norm (subscript if any denotes what norm)
Transposed matrix
The inverse of the transposed and vice versa, A−T = (A−1)T = (AT )−1.
Complex conjugated matrix
Transposed and complex conjugated matrix (Hermitian)
0
I
Jij
Σ
Λ
The null matrix. Zero in all entries.
The identity matrix
The single-entry matrix, 1 at (i, j) and zero elsewhere
A positive definite matrix
A diagonal matrix
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 5
1 Basics
1.1 Trace
(AB)−1 = B−1A−1
(ABC...)−1 = ...C−1B−1A−1
(AT )−1 = (A−1)T
(A + B)T = AT + BT
(AB)T = BT AT
(ABC...)T = ...CT BT AT
(AH )−1 = (A−1)H
(A + B)H = AH + BH
(AB)H = BH AH
(ABC...)H = ...CH BH AH
Tr(A) =
Tr(A) =
iAii
iλi,
Tr(A) = Tr(AT )
Tr(AB) = Tr(BA)
λi = eig(A)
Tr(A + B) = Tr(A) + Tr(B)
Tr(ABC) = Tr(BCA) = Tr(CAB)
aT a = Tr(aaT )
1.2 Determinant
Let A be an n × n matrix.
det(A) =
iλi
λi = eig(A)
if A ∈ Rn×n
det(cA) = cn det(A),
det(AT ) = det(A)
det(AB) = det(A) det(B)
det(A−1) = 1/ det(A)
det(An) = det(A)n
det(I + uvT ) = 1 + uT v
For n = 2:
For n = 3:
det(I + A) = 1 + det(A) + Tr(A)
1 BASICS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
(18)
(19)
(20)
(21)
(22)
(23)
(24)
(25)
det(I + A) = 1 + det(A) + Tr(A) +
1
2
Tr(A)2 − 1
2
Tr(A2)
(26)
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 6
1.3 The Special Case 2x2
1 BASICS
For n = 4:
det(I + A) = 1 + det(A) + Tr(A) +
1
2
Tr(A2)
+Tr(A)2 − 1
2
Tr(A)3 − 1
2
1
6
+
Tr(A)Tr(A2) +
1
3
Tr(A3)
(27)
For small ε, the following approximation holds
det(I + εA) ∼= 1 + det(A) + εTr(A) +
1
2
ε2Tr(A)2 − 1
2
ε2Tr(A2)
(28)
1.3 The Special Case 2x2
Consider the matrix A
A =
Determinant and trace
A11 A12
A21 A22
det(A) = A11A22 − A12A21
Tr(A) = A11 + A22
(29)
(30)
Eigenvalues
λ1 =
Tr(A) +Tr(A)2 − 4 det(A)
2
λ1 + λ2 = Tr(A)
v1 ∝
A12
λ1 − A11
Eigenvectors
λ2 − λ · Tr(A) + det(A) = 0
λ2 =
Tr(A) −Tr(A)2 − 4 det(A)
2
λ1λ2 = det(A)
Inverse
A−1 =
1
det(A)
A12
λ2 − A11
v2 ∝
A22 −A12
−A21 A11
(31)
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 7
2 DERIVATIVES
2 Derivatives
This section is covering differentiation of a number of expressions with respect to
a matrix X. Note that it is always assumed that X has no special structure, i.e.
that the elements of X are independent (e.g. not symmetric, Toeplitz, positive
definite). See section 2.8 for differentiation of structured matrices. The basic
assumptions can be written in a formula as
= δikδlj
(32)
that is for e.g. vector forms,
∂x
∂y
i
=
∂xi
∂y
∂x
∂y
ij
=
∂xi
∂yj
=
∂x
∂yi
∂Xkl
∂Xij
∂x
∂y
i
The following rules are general and very useful when deriving the differential of
an expression ([19]):
∂A = 0
∂(αX) = α∂X
(A is a constant)
∂(XY) = (∂X)Y + X(∂Y)
∂(X + Y) = ∂X + ∂Y
∂(Tr(X)) = Tr(∂X)
∂(X ◦ Y) = (∂X) ◦ Y + X ◦ (∂Y)
∂(X ⊗ Y) = (∂X) ⊗ Y + X ⊗ (∂Y)
∂(X−1) = −X−1(∂X)X−1
∂(det(X)) = Tr(adj(X)∂X)
∂(det(X)) = det(X)Tr(X−1∂X)
∂(ln(det(X))) = Tr(X−1∂X)
∂XT = (∂X)T
∂XH = (∂X)H
2.1 Derivatives of a Determinant
2.1.1 General form
Y−1 ∂Y
∂x
= det(Y)Tr
∂ det(Y)
k
∂x
∂ det(X)
∂Xik
∂2 det(Y)
∂x2
Xjk = δij det(X)
= det(Y)
Tr
Y−1 ∂ ∂Y
∂x
∂x
Y−1 ∂Y
∂x
Y−1 ∂Y
∂x
Tr
Y−1 ∂Y
∂x
Y−1 ∂Y
∂x
+Tr
−Tr
(33)
(34)
(35)
(36)
(37)
(38)
(39)
(40)
(41)
(42)
(43)
(44)
(45)
(46)
(47)
(48)
Petersen & Pedersen, The Matrix Cookbook, Version: November 15, 2012, Page 8
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
