| Line 3: | Line 3: | ||
---- | ---- | ||
| − | Continuing from where we left of in [[Discriminant Functions For The Normal(Gaussian) Density|Part 1]], | + | Continuing from where we left of in [[Discriminant Functions For The Normal(Gaussian) Density|Part 1]], in a problem with feature vector '''y''' and state of nature variable ''w'', we can represent the discriminant function as: |
<div style="margin-left: 25em;"> | <div style="margin-left: 25em;"> | ||
| Line 14: | Line 14: | ||
'''Case 1: Σ<sub>i</sub> = σ<sup>2</sup>I''' | '''Case 1: Σ<sub>i</sub> = σ<sup>2</sup>I''' | ||
| − | This is the simplest case and it occurs when the features are statistically independent and each feature has the same variance, σ<sup>2</sup>. Here, the covariance matrix is diagonal since its simply σ<sup>2</sup> times the identity matrix '''I'''. This means that each sample falls into equal sized clusters that are centered about their respective mean vectors. The computation of the determinant and the inverse |Σ<sub>i</sub>| = σ<sup>2d</sup> and Σ<sub>i</sub><sup>-1</sup> = (1/σ<sup>2</sup>)I. Because both |Σ<sub>i</sub>| and the (d/2) ln 2π term in the equation above are independent of i, we can ignore them and thus we obtain this simplified discriminant function: | + | This is the simplest case and it occurs when the features are statistically independent and each feature has the same variance, σ<sup>2</sup>. Here, the covariance matrix is diagonal since its simply σ<sup>2</sup> times the identity matrix '''I'''. This means that each sample falls into equal sized clusters that are centered about their respective mean vectors. The computation of the determinant and the inverse |Σ<sub>i</sub>| = σ<sup>2d</sup> and Σ<sub>i</sub><sup>-1</sup> = (1/σ<sup>2</sup>)I. Because both |Σ<sub>i</sub>| and the (d/2) ln 2π term in the equation above are independent of ''i'', we can ignore them and thus we obtain this simplified discriminant function: |
| − | + | <div style="margin-left: 25em;"> | |
| + | <math>g_i(\mathbf{Y}) = - \frac{1}{2} \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t\boldsymbol{\Sigma}_i^{-1} \left (\mathbf{x} - \boldsymbol{\mu}_i \right) - \frac{d}{2} \ln 2\pi - \frac{1}{2} \ln |\boldsymbol{\Sigma}_i| + \ln P(w_i) </math> | ||
| + | </div> | ||
Revision as of 05:44, 13 April 2013
Discriminant Functions For The Normal Density - Part 1
Continuing from where we left of in Part 1, in a problem with feature vector y and state of nature variable w, we can represent the discriminant function as:
$ g_i(\mathbf{Y}) = - \frac{1}{2} \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t\boldsymbol{\Sigma}_i^{-1} \left (\mathbf{x} - \boldsymbol{\mu}_i \right) - \frac{d}{2} \ln 2\pi - \frac{1}{2} \ln |\boldsymbol{\Sigma}_i| + \ln P(w_i) $
we will now look at the multiple cases for a multivariate normal distribution.
Case 1: Σi = σ2I
This is the simplest case and it occurs when the features are statistically independent and each feature has the same variance, σ2. Here, the covariance matrix is diagonal since its simply σ2 times the identity matrix I. This means that each sample falls into equal sized clusters that are centered about their respective mean vectors. The computation of the determinant and the inverse |Σi| = σ2d and Σi-1 = (1/σ2)I. Because both |Σi| and the (d/2) ln 2π term in the equation above are independent of i, we can ignore them and thus we obtain this simplified discriminant function:
$ g_i(\mathbf{Y}) = - \frac{1}{2} \left (\mathbf{x} - \boldsymbol{\mu}_i \right)^t\boldsymbol{\Sigma}_i^{-1} \left (\mathbf{x} - \boldsymbol{\mu}_i \right) - \frac{d}{2} \ln 2\pi - \frac{1}{2} \ln |\boldsymbol{\Sigma}_i| + \ln P(w_i) $
