From Duda, Hart & Stock Textbook chapter 5.2


Discriminant fuction that is a linear combination of the component x
$ g(x)=w^Tx +w_o $

$ w=[w1, w2, w3,...,wd] $ =>weight vector
$ w_o $ =>bias or threshold weight
Lin classifier OldKiwi.jpg

Two-catergory case
Decide w1 if $ g(x)>0 =>w^Tx > -wo $
Decide w2 if $ g(x) <0 =>w^Tx < -wo $
If $ g(x)=0 $ then it can we assigned to any class or be left undefined

$ g(x)=0 $ define a decision surface (which is a hyperplane) that separates w1 and w2 points

The hyperplane divides the feature space into two halfs: region R1 for w1 and region R2 for w2
w is normal to any vector laying in the hyperplane

The distance from x to the hyperplane is $ g(x)/||w|| $
The distance from the origin to the hyperplane is $ w_o/||w|| $

Proof:
  $ x=xp+r(w/||w||) $

  • xp = normal projectionof x onto hyperplane(H)
  • r = desired algebraic distance
  • positive if on the positive side and negative if on the negative side

  Because $ g(xp) =0 $
  $ g(x) =w^Tx + wo =r||w|| $
  therefore $ r = g(x)/||w|| $

Multicategory case

Some ways to devise multicategory classifiers

  • reduce the problem to c-1 two-class problems; where teh ith problem is solvd by a linear discriminant function that separates points assigned to w_i from those not assigned to w_i
  • use c(c-1)/2 linear discriminants; one for every pair of classes
  • defining c linear discriminant functions

$ g_i(x)=w^Tx_i+w_{i_o} $ $ i=1,\ldots,c $
assigning x to w_i if $ g_i(x)>g_j(x) {\forall} j !=i $

A linear machine divides the feature space into c decision regions

If Region i and region j are contiguous, the boundary between them is a portion of the hyperplane Hij defined by $ g_i(x)=g_j(x) $ => $ (w_i-w_j)^Tx+(w_{i_o}-w_{j_o})=0 $

$ w_i-w_j $ is normal to Hij
$ (g_i-g_j)/||w_i-w_j|| $ is the distance from x to $ H_{ij} $


More on Linear Discriminant Functions

[Chap5.Part1.pdf]

[LDF.pdf]

Alumni Liaison

Ph.D. 2007, working on developing cool imaging technologies for digital cameras, camera phones, and video surveillance cameras.

Buyue Zhang