Vectors

Vector: Any object that can be added together and multiplied by scalars. Examples of ‘vectors’:

• Geometric Vectors: Directed segments (i.e., $\vec{x}, \vec{y}$)
• Polynomials: Two polynomials can be added together or scaled with a scalar
• Audio Signals: These can be represented by a series of numbers. You can add them together and you can scale them
• More abstractly, tuples of n real numbers. This is the most common concept focused on in ML. Example below:

$$ a = \begin{bmatrix}1 \\2 \\3 \\\end{bmatrix} \in \mathbb{R}^3 $$

Matrices

Matrices are systems of vectors. Properties of matrices:

• Associativity: $(AB)C = A(BC)$ given $A,B,C$ are matrices with compatible dimensions
• Distributivity: $AB + AC = A(B + C)$ given $A,B,C$ are matrices with compatible dimensions
• Identity: There exist $I$ (identity matrices) that return the original matrix when multiplied. Note $I_m \neq I_n$ for $m \neq n$.

$$ \forall A \in \mathbb{R}^{m \times n} : I_m A = A I_n = A $$

Additionally, matrices have features of Inverses and Transposes:

• Inverse: $AA^{-1}=I$. A few points to note:
  • To be invertible, a matrix must be square.
  • Not all matrices have inverses. A matrix is said to be singular / noninvertible if it doesn’t have any inverse. Note, a matrix must have a non-zero determinant and have linearly independence to be invertible.
  • If matrices $A,B$ are both square matrices and it is such that $AB =I$, then you also know that $BA =I$.
  • To compute the inverse of a matrix, can use Gaussian Elimination to convert an invertible matrix $A$ into $I$
• Transpose: $A^T$ is the transpose of matrix $A$ where the columns and rows are swapped.
  • A matrix is considered ‘symmetric’ if $A=A^T$. Only square matrices can be symmetric. Also, if $A$ is invertible, then so is $A^T$ and $(A^{-1})^T = (A^T)^{-1}=:A^{-T}$
  • The sum of symmetric matrices is always symmetric. The product however is not.

Properties of Inverses and Transposes:

• $AA^{-1}=I=A^{-1}A$