Outline of the slecture

• Linear discriminant functions
• Summary
• References

Linear classification Problem Statement

In a linear classification problem, the feature space can be divided into different regions by hyperplanes. In this lecture, we will take a two-catagory case to illustrate. Given training samples $ \vec{y}_1,\vec{y}_2,...\vec{y}_n \in \mathbb{R}^p $, each $ \vec{y}_i $ is a p-dimensional vector and belongs to either class $ w_1 $ or $ w_2 $. The goal is to find the maximum-margin hyperplane that separate the points in the feature space that belong to class $ w_1 $ from those belong to class$ w_2 $. The discriminate function can be written as

$ g(\vec{y}) = c\cdot\vec{y} $

We want to find $ c\in\mathbb{R}^{n+1} $ so that a testing data point $ \vec{y}_i $ is labelled

$ {w_1} $ if $ c\cdot\vec{y}>0 $

$ {w_2} $ if $ c\cdot\vec{y}<0 $

We can apply a trick here to replace all $ \vec{y} $'s in class $ w_2 $ by $ -\vec{y} $, then the task is looking for $ c $ so that $ c\cdot \vec{y}>0, \forall \vec{y} \in $new sample space. Then hyperplane through origin is defined by $ c\cdot \vec{y} = 0 $, where c is the normal of the plane lying on the positive side of every hyperplane.

You might have already observe the ambiguity of c in the above discussion.