Revision as of 11:19, 25 March 2008 by Guptam (Talk)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Introduction

==

Fisher's linear discriminant is a classification method that projects high-dimensional data onto a line and performs classification in this one-dimensional space. The projection maximizes the distance between the means of the two classes while minimizing the variance within each class. See [Lecture 10] for detailed explanation.

Finding the separating hyperplane and finding the projection direction are dual problems

=======================================

Fisher's Linear Discriminant relies on dimension reduction. If we want to project D-dimensional data down to one dimension we can use $ y=\omega^T \vec{x} $ . After that, by determining a threshold $ \omega_0 $, we can select class 1 if $ y>\omega_0 $ and class 2 if $ y<\omega_0> $.

However, dimension reduction can cause considerable amount of information loss leading not being able to separate the data in reduced dimension, while the data is separable in D dimension. Fisher's Linear Discriminant method uses that idea by adjusting the weight vector $ \omega $ for projection to minimize the amount of overlapping data in reduced dimension. Here are some figures from C. M. Bishop's book to understand the idea completely.

FLD Old Kiwi.jpg


    • Context:** Classical [Discriminant Analysis] Problem

Given data $ y_1,\cdots, y_d \in \mathbb{R}^n $, from 2 classes. When n is big, it may be difficult to separate classes, because of computational issues.

In this case we try to find a projection $ \pi: R^n \rightarrow R^k, k <n $ s.t . $ \pi(y_1), \cdots, \pi(y_d) $ can be separated.


Derivation of Fisher's Linear Discriminant

================

Main article: [Derivation of Fisher's Linear Discriminant]


References

=

"`Two variations on Fisher's linear discriminant for pattern recognition <http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/34/21179/00982904.pdf>`_" is a nice journal article. The paper provides two fast and simple techniques for improving on the classification performance provided by Fisher's linear discriminant for two classes. See the following link for the detailed information: `Two variations on Fisher's linear discriminant for pattern recognition <http://ieeexplore.ieee.org/Xplore/login.jsp?url=/iel5/34/21179/00982904.pdf>`_.

Alumni Liaison

Abstract algebra continues the conceptual developments of linear algebra, on an even grander scale.

Dr. Paul Garrett