Basic Matrix Operations (easy to use/read)

$0_{m \times n}$  denotes the  $m \times n$  zero matrix, with all entries zero.

$I_{n}$  denotes the $n \times n$  identity matrix, with $I_{ij}= \cases{ 1 & i = j \cr 0 & i $\ne$ j }$

For example;  $0_{2 \times 3} = \left\lbrack \matrix{ 0 & 0 & 0 \cr 0 & 0 & 0 } \right\rbrack,$  $I_{2} =  \left\lbrack \matrix{ 1 & 0 \cr 0 & 1 } \right\rbrack$

Transpose of Matrix 

$A = \left\lbrack \matrix{ a & b & c \cr d & e & f } \right\rbrack {⇒}$  ${A^T} = \left\lbrack \matrix{ a & d \cr b & e \cr c & f } \right\rbrack$

Summation of Matrices (entrywise)

$\left\lbrack \matrix{ a & b \cr c & d } \right\rbrack + \left\lbrack \matrix{ x & y \cr z & t} \right\rbrack = \left\lbrack \matrix{ a+x & b+y \cr c+z & d+t } \right\rbrack$

Subtraction of Matrices (entrywise)

$\left\lbrack \matrix{ a & b \cr c & d } \right\rbrack - \left\lbrack \matrix{ x & y \cr z & t} \right\rbrack = \left\lbrack \matrix{ a-x & b-y \cr c-z & d-t } \right\rbrack$

Properties of matrix addition

• commutative: $A + B = B + A$
• associative: $( A + B ) + C = A + ( B + C )$ , so we can write as  $A + B + C$
• $A + 0 = 0 + A = A; A −A = 0$
• $( A + B )^T = A^T + B^T$

Scalar Multiplication

$s \left\lbrack \matrix{ x & y \cr z & t} \right\rbrack = \left\lbrack \matrix{ sx & sy \cr sz & st } \right\rbrack$

Dot Product

$a$  and  $b$  are $n$ dimensional vectors;

$a = \lbrack a_{1}, a_{2}, ⋯ ,a_{n} \rbrack ,$  $b =  \lbrack b_{1}, b_{2}, ⋯ , b_{n} \rbrack$

Dot product of these two vectors is a scalar;

$a \cdot b = \langle a, b \rangle = \sum\limits_{i=1}^n a_i b_i = a_1 b_1 + a_2 b_2 + ⋯ + a_n b_n$

Also;  $a \cdot b = a^T b$ , which means, same as above;

$a^T b = \lbrack \matrix{ a_{1} & a_{2} & ⋯ & a_{n}} \rbrack \left\lbrack \matrix{b_{1} \cr b_{2} \cr ⋮\cr b_{n}} \right\rbrack = \sum\limits_{i=1}^n a_i b_i = a_1 b_1 + a_2 b_2 + ⋯ + a_n b_n$

Matrix - Vector Product

$\left\lbrack \matrix{ a_{11} & a_{12} & ⋯ & a_{1n} \cr a_{21} & c_{22} & ⋯ &a_{2n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr a_{m1} & a_{m2} &  ⋯& a_{mn} } \right\rbrack \left\lbrack \matrix{ x_{1} \cr x_{2} \cr ⋮ \cr x_{n} } \right\rbrack\ = \left\lbrack \matrix{ a_{11}x_{1} + a_{12}x_{2} + ⋯ + a_{1n}x_{n} \cr a_{21}x_{1} + a_{22}x_{2} + ⋯ + a_{2n}x_{n} \cr ⋮ \cr a_{m1}x_{1} + a_{m2}x_{2} +  ⋯ + a_{mn}x_{n} } \right\rbrack$

Matrix multiplication

First one is  $m \times p$ , second one is  $p \times n$  and output matrix is $m \times n$ .

$\left\lbrack \matrix{ a_{11} & a_{12} & ⋯ & a_{1p} \cr a_{21} & a_{22} & ⋯ &a_{2p} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr a_{m1} & a_{m2} &  ⋯&a_{mp} } \right\rbrack \left\lbrack \matrix{ b_{11} & b_{12} & ⋯ & b_{1n} \cr b_{21} & b_{22} & ⋯ &b_{2n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr b_{p1} & b_{p2} &  ⋯& b_{pn} } \right\rbrack = \left\lbrack \matrix{ c_{11} & c_{12} & ⋯ & c_{1n} \cr c_{21} & c_{22} & ⋯ &c_{2n} \cr ⋮ & ⋮ & ⋱ & ⋮ \cr c_{m1} & c_{m2} &  ⋯& c_{mn} } \right\rbrack$

$\langle$ Column Vector $,$ Row Vector $\rangle$ = Scalar ; (Gives value at intersection point of those column and row in output matrix);

$\lbrack \matrix{ a_{1} & a_{2} & ⋯ & a_{n}} \rbrack \left\lbrack \matrix{b_{1} \cr b_{2} \cr ⋮\cr b_{n}} \right\rbrack = a_{1} b_{1} + a_{2} b_{2} + ⋯ + a_{n} b_{n}$

shows;

$c_{ij} = \langle a_{row_i}, b_{col_j} \rangle = a_{i1} b_{1j} + a_{i2} b_{2j} + ~ . . . ~+ ~a_{ip} b_{pj} , 1 \le i \le m ; 1 \le j \le n$

so, output will be equal to;

$\left\lbrack \matrix{ \langle a_{row_1},b_{col_1} \rangle & \langle a_{row_1},b_{col_2} \rangle & ⋯ & \langle a_{row_1},b_{col_n} \rangle \cr \langle a_{row_2},b_{col_1} \rangle & \langle a_{row_2},b_{col_2} \rangle & ⋯ & \langle a_{row_2},b_{col_n} \rangle \cr ⋮ &⋮ & ⋱ &⋮ \cr \langle a_{row_m},b_{col_1} \rangle & \langle a_{row_m},b_{col_2} \rangle & ⋯ & \langle a_{row_m},b_{col_n} \rangle } \right\rbrack$

• $0A = 0,$  $A0 = 0$    (here $0$ can be scalar, or a compatible matrix)
• $IA = A,$  $AI = A$
• $A(BC) = (AB)C$  so we can write $ABC$, but, order must be same.
• $\alpha (AB) =  (\alpha A ) B$ , where $\alpha$ is a scalar.
• $A (B+C) = AB + AC, (A+B)C = AC + BC$
• $(AB)^T = B^T A^T$

Matrix powers

 $A$  must be $n \times n$ sized matrix, so it can be multiplicative by itself;

 $A^k = \underbrace{A A ⋯ A }_{k\rm\ times}$

• $A^0 = I$
• $A^k A^l = A^{k + l}$

Determinant 

For two dimensional matrix $A$;

$A = \left\lbrack \matrix{ a & b \cr c & d } \right\rbrack {⇒}$ $\det(A) = |A| = \left| \matrix{ a & b \cr c & d } \right| = ad - bc$

For three dimensional matrix $M$;

$M = \left\lbrack \matrix{ \color{#67D4FF}{m_{11}} & \color{#FF8B8B}{m_{12}} & m_{13} \cr m_{21} & \color{#67D4FF}{m_{22}} & \color{#FF8B8B}{m_{23}} \cr \color{#FF8B8B}{m_{31}} & m_{32} & \color{#67D4FF}{m_{33}} } \right\rbrack {⇒}$ $\det(M) = |M| = \left| \matrix{ \color{#67D4FF}{\color{#67D4FF}{m_{11}}} & \color{#FF8B8B}{m_{12}} & m_{13} \cr m_{21} & \color{#67D4FF}{m_{22}} & \color{#FF8B8B}{m_{23}} \cr \color{#FF8B8B}{m_{31}} & m_{32} & \color{#67D4FF}{m_{33}} } \right|$

$= \color{#67D4FF}{m_{11}} \left| \matrix{ \color{#67D4FF}{m_{22}} & \color{#FF8B8B}{m_{23}} \cr m_{32} & \color{#67D4FF}{m_{33}} } \right| - \color{#FF8B8B}{m_{12}} \left| \matrix{ m_{21} & \color{#FF8B8B}{m_{23}} \cr \color{#FF8B8B}{m_{31}} & \color{#67D4FF}{m_{33}} } \right| + m_{13}  \left| \matrix{ m_{21} & \color{#67D4FF}{m_{22}} \cr \color{#FF8B8B}{m_{31}} & m_{32} } \right| = \color{#67D4FF}{m_{11}}(\color{#67D4FF}{m_{22}} \color{#67D4FF}{m_{33}} − \color{#FF8B8B}{m_{23}} m_{32}) − \color{#FF8B8B}{m_{12}}(m_{21} \color{#67D4FF}{m_{33}} − \color{#FF8B8B}{m_{23}} \color{#FF8B8B}{m_{31}}) + m_{13}(m_{21} m_{32} − \color{#67D4FF}{m_{22}} \color{#FF8B8B}{m_{31}})$

$\color{#67D4FF}{m_{11}} \color{#67D4FF}{m_{22}} \color{#67D4FF}{m_{33}} + \color{#FF8B8B}{m_{12}} \color{#FF8B8B}{m_{23}} \color{#FF8B8B}{m_{31}} + m_{13} m_{21} m_{32} - m_{13} \color{#67D4FF}{m_{22}} \color{#FF8B8B}{m_{31}} - \color{#FF8B8B}{m_{12}} m_{21} \color{#67D4FF}{m_{33}} - \color{#67D4FF}{m_{11}} \color{#FF8B8B}{m_{23}} m_{32}$

Matrix inverse

Again, $A$  must be $n \times n$ sized matrix,  $F$  satisfies  $F A = I$;
$F$ is called the inverse of $A$, and is denoted $A^{-1}$,
The matrix $A$ is called invertible or nonsingular.

Assuming  $A$,  $B$ are invertible, $x \in \mathbf{R}^n, \alpha \ne 0$;

• $A^{−k} = ({A^{−1}}) ^k$
• $({A^{−1}}) ^{-1} = A$
• $(AB) ^{-1} = B^{-1}A^{-1}$
• $({A^{T}}) ^{-1} = ({A^{−1}}) ^T$
• $I^{−1} = I$
• $(\alpha A)^{−1}  =  (1 / \alpha )A^{−1}$
• if  $y = Ax \Rightarrow x = A^{-1}y \Rightarrow A^{−1}y = A^{−1}Ax = Ix = x$

TODO Add more

Sources

http://www.onemathematicalcat.org/MathJaxDocumentation/TeXSyntax.htm
https://cms.inonu.edu.tr/uploads/old/5/357/matrislerde-islem-1.pdf
http://ee263.stanford.edu/notes/matrix-primer-lect2.pdf
http://mathworld.wolfram.com/MatrixInverse.html
http://mathinsight.org/matrix_vector_multiplication
https://en.wikipedia.org/wiki/Determinant
https://en.wikipedia.org/wiki/Dot_product