Day 2

Three base concepts of matrices

1. Additions and subtractions. The rule is the row and column of both matrix are the same.

$$\begin{align*} A_{mn} + B_{mn} &= C_{mn} \end{align*}$$

1. Multiplication (Dot Product). The rule is the number of columns of the first matrix (In this case A) needs to have the same value as the row of the second matrix (In this case B).

$$\begin{align*} A_{mn} \cdot B_{nk} &= C_{mk} \end{align*}$$

The result will return a new matrix with the size of (row A x Column B) 3. The Identity Matrix A NxN-matrix with the main diagonal being filled with 1 while the rest are zeros.

$$I_3 = \begin{pmatrix} 1 & 0 & 0 \\\ 0 & 1 & 0 \\\ 0 & 0 & 1 \end{pmatrix}$$

The main diagonal is when the row and column index is the same. So row 1 column 1, row 2 column 2, etc.

Properties of Matrices

Much like the base concepts, there are three properties described in the book before we delve into Inverse and Transpose.

Associativity

$$\forall A \in \mathbb{R}^{m \times n}, \; B \in \mathbb{R}^{n \times p}, \;\\\ C \in \mathbb{R}^{p \times q} \; : \; (AB)C = A(BC)$$

Don't be scared or annoyed by these symbols! This just means for all (V) A, B, and C that belongs to real numbers (R) with the dimensions of mxn,nxp,pxq respectively will create the equation.

P.S. If you weren't scared or annoyed, congratulations you're not me.

Distributivity

$$\forall A,B \in \mathbb{R}^{m \times n}, \;\\\ C,D \in \mathbb{R}^{n \times p} \;\\\ : (A+B)C = AB+BC\\\ : A(C+D) = AC+CD$$

Much like scaling the matrix, when there's both addition/subtraction along side multiplication, the multiplication is distributed throughout the addition/subtraction.

Multiplication with identity matrix

$$\forall A \in \mathbb{R}^{m \times n}\\\ : I_{m \times m} A_{m \times n} = A_{m \times n} I_{n \times n} = A_{m \times n}$$

When a matrix is multiplied by an identity matrix, the resulting matrix won't change. The same goes for an identity matrix multiplied by a matrix, the result will not change the shape nor the values of the matrix multiplied.

Acknowledgement

I can't overstate this: I'm truly grateful for this book being open-sourced for everyone. Many people will be able to learn and understand machine learning on a fundamental level. Whether changing careers, demystifying AI, or just learning in general, this book offers immense value even for fledgling composer such as myself. So, Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, thank you for this book.

Source: Deisenroth, M. P., Faisal, A. A., & Ong, C. S. (2020). Mathematics for Machine Learning. Cambridge: Cambridge University Press. https://mml-book.com (opens in a new tab)