Motivation

The concept of ‘angle’ and ‘length’ in a vector space is defined by the inner product. This in turn gives rise to many important properties and applications.

A major purpose of the inner product (beyond defining length) is to determine if vectors are orthogonal to each other.

Norms

A norm on a vector space is a function $\| \cdot \| : V \rightarrow \R$ which assigns a length to a vector. For all $\lambda \in \R$ and $x, y \in V$ the following hold:

• Absolutely Homogenous: $\| \lambda x \| = | \lambda | \| x \|$
• Triangle Inequality: $\| x + y \| \le \| x \| + \| y \|$
• Positive Definite: $\| x \| \le 0 \text { and } \| x \| = 0 \iff x=0$

Examples of norms:

1. Manhattan Norm (L-1 Norm)

$$ \| x \|1 := \sum{i=1}^n |x_i| $$

1. Euclidean Norm (L-2 Norm)

$$ \| x \|2 := \sqrt{ \sum{i=1}^n x_i^2} = \sqrt{x^Tx} $$

Inner Products

The dot product (aka scalar product) is the most common inner product.

$$ x^Ty = \sum_{i=1}^n x_iy_i $$