Linear Algebra (一)

A B = C AB = C A B =C
[ a 11 ⋯ a 1 n ⋮ ⋱ ⋮ a m 1 ⋯ a m n ] [ b 11 ⋯ b 1 p ⋮ ⋱ ⋮ b n 1 ⋯ b n p ] = [ c 11 ⋯ c 1 p ⋮ ⋱ ⋮ c m 1 ⋯ c m p ] \begin{bmatrix} a_{11} & \cdots & a_{1n} \ \vdots & \ddots & \vdots \ a_{m1} & \cdots & a_{mn}\end{bmatrix}\begin{bmatrix} b_{11} & \cdots & b_{1p} \ \vdots & \ddots & \vdots \ b_{n1} & \cdots & b_{np}\end{bmatrix}=\begin{bmatrix} c_{11} & \cdots & c_{1p} \ \vdots & \ddots & \vdots \ c_{m1} & \cdots & c_{mp}\end{bmatrix}⎣⎢⎡​a 1 1 ​⋮a m 1 ​​⋯⋱⋯​a 1 n ​⋮a m n ​​⎦⎥⎤​⎣⎢⎡​b 1 1 ​⋮b n 1 ​​⋯⋱⋯​b 1 p ​⋮b n p ​​⎦⎥⎤​=⎣⎢⎡​c 1 1 ​⋮c m 1 ​​⋯⋱⋯​c 1 p ​⋮c m p ​​⎦⎥⎤​

矩阵相乘的5种视角，互相等价

• 常规视角
• c i j = ∑ k = 1 n a i k b k j c_{ij} = \sum_{k=1}^n a_{ik}b_{kj}c i j ​=∑k =1 n ​a i k ​b k j ​
• 通过矩阵和向量乘法
• 右乘：C 的每个列向量c j \bold{c}j c j ​由A的列向量a k , 1 ≤ k ≤ n \bold{a}_k,1\le k\le n a k ​,1 ≤k ≤n的线性组合构成 c j = ∑ k = 1 n b k j a k \bold{c}_j = \sum{k=1}^n b_{kj} \bold{a}_k c j ​=∑k =1 n ​b k j ​a k ​
• 左乘：C 的没个行向量c i \bold{c}i c i ​ 由B的行向量b k , 1 ≤ k ≤ p \bold{b}_k,1\le k\le p b k ​,1 ≤k ≤p的线性组合构成c i = ∑ k = 1 p a i k b k \bold{c}_i=\sum{k=1}^pa_{ik}\bold{b}_k c i ​=∑k =1 p ​a i k ​b k ​
• A B = ∑ i ( c o l u m n i O f A ) ( r o w i O f B ) AB = \sum_i (column_iOf A)(row_iOf B)A B =∑i ​(c o l u m n i ​O f A )(r o w i ​O f B )
• By Blocks

左逆矩阵（Left Inverse）：

A − 1 A = I A^{-1}A = I A −1 A =I

右逆矩阵（Right Inverse）

A A − 1 = I AA^{-1}=I A A −1 =I

Dependence：

向量V 1 , V 2 , . . , V n V_1,V_2,..,V_n V 1 ​,V 2 ​,..,V n ​，存在组合非全零实数c 1 , c 2 , . . . , c n c_1, c_2,...,c_n c 1 ​,c 2 ​,...,c n ​，满足∑ i = 1 n c i V i = 0 \sum_{i=1}^nc_iV_i = 0 ∑i =1 n ​c i ​V i ​=0，则称向量线性相关。

矩阵行列式的三个性质：

• d e t ( I ) = 1 det(I) = 1 d e t (I )=1
• Exchange rows , reverse sign of determinant
• Linear for each row

∣ t ∗ a t ∗ b c d ∣ = t ∗ ∣ a b c d ∣ \left|\begin{array}{cccc} ta & tb \ c & d \ \end{array}\right| = t * \left|\begin{array}{cccc} a & b \ c & d \ \end{array}\right|∣∣∣∣​t ∗a c ​t ∗b d ​∣∣∣∣​=t ∗∣∣∣∣​a c ​b d ​∣∣∣∣​

∣ a + a ′ b + b ′ c d ∣ = ∣ a b c d ∣ + ∣ a ′ b ′ c d ∣ \left|\begin{array}{cccc} a + a' & b + b' \ c & d \ \end{array}\right| = \left|\begin{array}{cccc} a & b \ c & d \ \end{array}\right| + \left|\begin{array}{cccc} a' & b' \ c & d \ \end{array}\right|∣∣∣∣​a +a ′c ​b +b ′d ​∣∣∣∣​=∣∣∣∣​a c ​b d ​∣∣∣∣​+∣∣∣∣​a ′c ​b ′d ​∣∣∣∣​

由此三个性质推导出行列式的以下性质：

• two equal rows => det = 0
• subtract l × r o w i l\times row_i l ×r o w i ​ from r o w j row_j r o w j ​, det does not change.

∣ a b c − l ∗ a d − l ∗ b ∣ = ∣ a b c d ∣ + ∣ a b − l ∗ a − l ∗ b ∣ \left|\begin{array}{cccc} a & b \ c -la& d-lb \ \end{array}\right| = \left|\begin{array}{cccc} a & b \ c & d \ \end{array}\right| + \left|\begin{array}{cccc} a & b \ -la & -lb \ \end{array}\right|∣∣∣∣​a c −l ∗a ​b d −l ∗b ​∣∣∣∣​=∣∣∣∣​a c ​b d ​∣∣∣∣​+∣∣∣∣​a −l ∗a ​b −l ∗b ​∣∣∣∣​
= ∣ a b c d ∣ − l ∗ ∣ a b a b ∣ = ∣ a b c d ∣ =\left|\begin{array}{cccc} a & b \ c & d \ \end{array}\right| -l * \left|\begin{array}{cccc} a & b \ a & b \ \end{array}\right| = \left|\begin{array}{cccc} a & b \ c & d \ \end{array}\right|=∣∣∣∣​a c ​b d ​∣∣∣∣​−l ∗∣∣∣∣​a a ​b b ​∣∣∣∣​=∣∣∣∣​a c ​b d ​∣∣∣∣​

• Row of zeros => det = 0

∣ 0 ∗ a 0 ∗ b c d ∣ = 0 ∗ ∣ a b c d ∣ = 0 \left|\begin{array}{cccc} 0a & 0b \ c & d \ \end{array}\right| = 0 * \left|\begin{array}{cccc} a & b \ c & d \ \end{array}\right| = 0 ∣∣∣∣​0 ∗a c ​0 ∗b d ​∣∣∣∣​=0 ∗∣∣∣∣​a c ​b d ​∣∣∣∣​=0

• Trianglar matrix =>d e t = d 1 ∗ d 2 ∗ d 3... d n det = d1d2d3...dn d e t =d 1 ∗d 2 ∗d 3 ...d n

∣ d 1 ⋯ ⋯ ⋯ ⋯ 0 d 2 ⋯ ⋯ ⋯ 0 0 d 3 ⋯ ⋯ ⋮ ⋮ ⋮ ⋮ ⋮ 0 0 0 ⋯ d n ∣ = ∏ i = 1 n d i \left|\begin{array}{cccc} d1 & \cdots & \cdots & \cdots & \cdots\ 0 & d2 & \cdots & \cdots & \cdots\ 0 & 0 & d3 & \cdots & \cdots \ \vdots & \vdots & \vdots & \vdots & \vdots \ 0 & 0 & 0 & \cdots & dn \ \end{array}\right| = \prod_{i=1}^n di ∣∣∣∣∣∣∣∣∣∣∣​d 1 0 0 ⋮0 ​⋯d 2 0 ⋮0 ​⋯⋯d 3 ⋮0 ​⋯⋯⋯⋮⋯​⋯⋯⋯⋮d n ​∣∣∣∣∣∣∣∣∣∣∣​=i =1 ∏n ​d i

• det = 0 exactly when A is singular（d e t ≠ 0 det\ne 0 d e t ​=0 when A is invertible）
• d e t ( A B ) = d e t ( A ) d e t ( B ) det(AB) = det(A)det(B)d e t (A B )=d e t (A )d e t (B )
  d e t A − 1 = ( d e t A ) − 1 det A^{-1} = (det A)^{-1}d e t A −1 =(d e t A )−1
  d e t A 2 = ( d e t A ) 2 det A^2 = (det A)^2 d e t A 2 =(d e t A )2
  d e t 2 A = 2 n d e t A det 2A = 2^n det A d e t 2 A =2 n d e t A
• d e t A T = d e t A detA^T = det A d e t A T =d e t A

d e t A T = d e t U T L T = d e t U d e t L = d e t L U = d e t A det A^T = det U^TL^T = det U det L = det LU = det A d e t A T =d e t U T L T =d e t U d e t L =d e t L U =d e t A

行列式的定义

d e t ( A ) = ∑ n ! a 1 α a 2 β a 1 γ . . . a n ω det(A) = \sum_{n!} a_{1\alpha}a_{2\beta}a_{1\gamma}...a_{n\omega}d e t (A )=n !∑​a 1 α​a 2 β​a 1 γ​...a n ω​

( α , β , γ , . . . , ω ) (\alpha,\beta,\gamma,...,\omega)(α,β,γ,...,ω)is permutation of (1,2,3,...,n)

Exist a none zero vector X， satisfiy A x = 0 Ax = 0 A x =0，then A is singular.

定义：A是n阶矩阵，若实数λ \lambda λ和n维非零向量α \alpha α满足A α = λ α A\alpha = \lambda\alpha A α=λα，则称λ \lambda λ为A的特征值，α \alpha α为A的特征向量。

• If A is singular, the λ = 0 \lambda=0 λ=0 is an eigenvalue.

• Trace：∑ i λ i = ∑ a i i \sum_i \lambda_i = \sum a_{ii}∑i ​λi ​=∑a i i ​

• Determinant :d e t = ∏ i λ i det = \prod_i \lambda_i d e t =∏i ​λi ​
• 对称或近似对称，特征值是实数，否则可能是复数。

对于对称举矩阵

• the eigenvalues are also Real
• the eigenvectors are Perpendicular

Usual case:

A = S Λ S − 1 A = S\Lambda S^{-1}A =S ΛS −1

Symmetric case:

A = Q Λ Q − 1 = Q Λ Q T A=Q\Lambda Q^{-1} = Q\Lambda Q^T A =Q ΛQ −1 =Q ΛQ T

定义：矩阵的奇异值分解是指，将一个非零的m × n m\times n m ×n实矩阵A , A ∈ R m × n A,A\in R^{m\times n}A ,A ∈R m ×n，表示为以下三个实矩阵乘积形式的运算，即进行矩阵的因子分解：

A = U Σ V T A= U\Sigma V^T A =U ΣV T

其中U U U是m m m阶正交矩阵，V V V是n n n阶正交矩阵，Σ \Sigma Σ是由降序排列的非负的对角线元素组成的m × n m\times n m ×n矩形对角阵，满足

U U T = I V V T = I Σ = d i a g ( σ 1 , σ 2 , . . . , σ p ) UU^T=I\VV^T=I\ \Sigma=diag(\sigma_1,\sigma_2,...,\sigma_p)U U T =I V V T =I Σ=d i a g (σ1 ​,σ2 ​,...,σp ​)

σ 1 ≥ σ 2 ≥ . . . ≥ σ p ≥ 0 \sigma_1\ge \sigma_2\ge ...\ge\sigma_p\ge 0 σ1 ​≥σ2 ​≥...≥σp ​≥0

p = m i n ( m , n ) p=min(m,n)p =m i n (m ,n )

U Σ V T U\Sigma V^T U ΣV T称为矩阵A的奇异值分解（singular value decomposition）,σ i \sigma_i σi ​称为矩阵A的奇异值（singular value），U U U的列向量称为左奇异向量，V的列向量称为右奇异向量。

A = U Σ V T A=U\Sigma V^T A =U ΣV T

其中U U U是m阶正交矩阵，V V V是n阶正交矩阵，Σ \Sigma Σ是m × n m\times n m ×n矩形对角矩阵，其对角线元素非负，且降序排列。

Original: https://blog.csdn.net/gaofeipaopaotang/article/details/123776108
Author: jony0917
Title: Linear Algebra (一)