| Line 20: | Line 20: | ||
</div> | </div> | ||
| − | where ||.|| denotes the''Euclidean norm, that is, | + | where ||.|| denotes the ''Euclidean norm'', that is, |
<div style="margin-left: 25em;"> | <div style="margin-left: 25em;"> | ||
| Line 26: | Line 26: | ||
</div> | </div> | ||
| + | |||
| + | If theprior probabilities are not equal, then the discriminant function shows that the squared distance ||'''x - μ||<sup>2</sup> must be normalized by the variance σ<sup>2</sup> and offset by adding ln ''P''(''w<sub>i</sub>''; therefore if '''x''' is equally near two different mean vectors, the optimal decision will favor the priori more likely. | ||
Revision as of 06:00, 13 April 2013
Discriminant Functions For The Normal Density - Part 1
Continuing from where we left of in Part 1, in a problem with feature vector y and state of nature variable w, we can represent the discriminant function as:
$ g_i(\mathbf{Y}) = - \frac{1}{2} \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t\boldsymbol{\Sigma}_i^{-1} \left (\mathbf{x} - \boldsymbol{\mu}_i \right) - \frac{d}{2} \ln 2\pi - \frac{1}{2} \ln |\boldsymbol{\Sigma}_i| + \ln P(w_i) $
we will now look at the multiple cases for a multivariate normal distribution.
Case 1: Σi = σ2I
This is the simplest case and it occurs when the features are statistically independent and each feature has the same variance, σ2. Here, the covariance matrix is diagonal since its simply σ2 times the identity matrix I. This means that each sample falls into equal sized clusters that are centered about their respective mean vectors. The computation of the determinant and the inverse |Σi| = σ2d and Σi-1 = (1/σ2)I. Because both |Σi| and the (d/2) ln 2π term in the equation above are independent of i, we can ignore them and thus we obtain this simplified discriminant function:
$ g_i(\mathbf{Y}) = - \frac{||\mathbf{x} - \boldsymbol{\mu}_i||^2 }{2\boldsymbol{\sigma}_i^{2}} + \ln P(w_i) $
where ||.|| denotes the Euclidean norm, that is,
$ ||\mathbf{x} - \boldsymbol{\mu}_i|| = \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t (\mathbf{x} - \boldsymbol{\mu}_i) $
If theprior probabilities are not equal, then the discriminant function shows that the squared distance ||x - μ||2 must be normalized by the variance σ2 and offset by adding ln P(wi; therefore if x is equally near two different mean vectors, the optimal decision will favor the priori more likely.
