| Line 18: | Line 18: | ||
<div style="margin-left: 25em;"> | <div style="margin-left: 25em;"> | ||
<math>g_i(\mathbf{Y}) = - \frac{||\mathbf{x} - \boldsymbol{\mu}_i||^2 }{2\boldsymbol{\sigma}_i^{2}} + \ln P(w_i) </math> | <math>g_i(\mathbf{Y}) = - \frac{||\mathbf{x} - \boldsymbol{\mu}_i||^2 }{2\boldsymbol{\sigma}_i^{2}} + \ln P(w_i) </math> | ||
| + | </div> | ||
| + | |||
| + | where ||.|| denotes the''Euclidean norm, that is, | ||
| + | |||
| + | <div style="margin-left: 25em;"> | ||
| + | <math>||\mathbf{x} - \boldsymbol{\mu}_i|| = \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t (\mathbf{x} - \boldsymbol{\mu}_i) </math> | ||
</div> | </div> | ||
Revision as of 05:48, 13 April 2013
Discriminant Functions For The Normal Density - Part 1
Continuing from where we left of in Part 1, in a problem with feature vector y and state of nature variable w, we can represent the discriminant function as:
$ g_i(\mathbf{Y}) = - \frac{1}{2} \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t\boldsymbol{\Sigma}_i^{-1} \left (\mathbf{x} - \boldsymbol{\mu}_i \right) - \frac{d}{2} \ln 2\pi - \frac{1}{2} \ln |\boldsymbol{\Sigma}_i| + \ln P(w_i) $
we will now look at the multiple cases for a multivariate normal distribution.
Case 1: Σi = σ2I
This is the simplest case and it occurs when the features are statistically independent and each feature has the same variance, σ2. Here, the covariance matrix is diagonal since its simply σ2 times the identity matrix I. This means that each sample falls into equal sized clusters that are centered about their respective mean vectors. The computation of the determinant and the inverse |Σi| = σ2d and Σi-1 = (1/σ2)I. Because both |Σi| and the (d/2) ln 2π term in the equation above are independent of i, we can ignore them and thus we obtain this simplified discriminant function:
$ g_i(\mathbf{Y}) = - \frac{||\mathbf{x} - \boldsymbol{\mu}_i||^2 }{2\boldsymbol{\sigma}_i^{2}} + \ln P(w_i) $
where ||.|| denotes theEuclidean norm, that is,
$ ||\mathbf{x} - \boldsymbol{\mu}_i|| = \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t (\mathbf{x} - \boldsymbol{\mu}_i) $
