Classify your data by drawing a line(s) that splits up your classes

<aside> 💡 While elementary, building an understanding of these simple classifiers will provide a strong foundation prior to analyzing more complex models.

</aside>

The key difference between a Linear Classifier and Non-Linear Classifier is that Non-Linear Classifiers have some element that introduces non-linearity → this can be from an activation function in a neural network to different rules-based approaches made by a decision tree.

Contents:

1. What is a Linear Classifier?
2. Overview of Linear Classifiers
   1. Binary Linear Classifiers
      1. Math Explained: Binary Classifiers
   2. Multi-Class Linear Classifiers
   3. Hard vs. Soft Classifiers
3. Implementations
   1. Perceptron
   2. Logistic Linear Regression
   3. Support Vector Machines

What is a Linear Classifier

You have two classes - we’ll denote them as ‘+’ or ‘-’. This could be determining if a financial transaction is fraud or not - maybe classifying if a picture is of a cat or not. Tons of possibilities.

Given a dataset of features in 2-dimensions $(\R^2)$, you try to create a model that can correctly predict of the data will be ‘+’ or ‘-’.

Take this below dataset for example:

2-dimensional dataset with labels denoted with ‘+’ or ‘-’.

The first approach you might think of is simply drawing a line that separates the data. This would be a ‘binary linear classifier’ — it is classifying the data into 1 of 2 categories (hence the ‘binary’). This can look like the following:

Linear classifier defined with the function $y = sign(\theta^Tx + \theta_0)$. We’ll cover this math later.

We’ll spend the next few sections breaking this down further.

Overview of Linear Classifiers

We’ll split the idea of a linear classifier into two groups:

1. Binary Linear Classifier (determining if something is one of two classes)
2. Multi-Class Linear Classifier (determining which of many classes a datapoint is)

Within these, there are different styles as well: