# Discriminant Functions For The Normal Density - Part 1

**Introduction to Normal or Gaussian Distribution**

Before talking about discriminant functions for the normal density, we first need to know what a normal distribution is and how it is represented for just a single variable, and for a vector variable. Lets begin with the continuous univariate normal or Gaussian density.

$ f_x = \frac{1}{\sqrt{2 \pi} \sigma} \exp \left [- \frac{1}{2} \left ( \frac{x - \mu}{\sigma} \right)^2 \right ] $

for which the expected value of *x* is

$ \mu = \mathcal{E}[x] =\int\limits_{-\infty}^{\infty} xp(x)\, dx $

and where the expected squared deviation or *variance* is

$ \sigma^2 = \mathcal{E}[(x- \mu)^2] =\int\limits_{-\infty}^{\infty} (x- \mu)^2 p(x)\, dx $

The univariate normal density is completely specified by two parameters; its mean *μ * and variance *σ ^{2}*. The function f

_{x}can be written as

*N(μ,σ)*which says that

*x*is distributed normally with mean

*μ*and variance

*σ*. Samples from normal distributions tend to cluster about the mean with a spread related to the standard deviation

^{2}*σ*.

For the multivariate normal density in *d* dimensions, f_{x} is written as

$ f_x = \frac{1}{(2 \pi)^ \frac{d}{2} |\boldsymbol{\Sigma}|^\frac{1}{2}} \exp \left [- \frac{1}{2} (\mathbf{x} -\boldsymbol{\mu})^t\boldsymbol{\Sigma}^{-1} (\mathbf{x} -\boldsymbol{\mu}) \right] $

where **x** is a *d*-component column vector, **μ** is the *d*-component mean vector, **Σ** is the *d*-by-*d* covariance matrix, and **|Σ|** and **Σ ^{-1}** are its determinant and inverse respectively. Also,(

**x - μ**)

^{t}denotes the transpose of (

**x - μ**).

and

$ \boldsymbol{\Sigma} = \mathcal{E} \left [(\mathbf{x} - \boldsymbol{\mu})(\mathbf{x} - \boldsymbol{\mu})^t \right] = \int(\mathbf{x} - \boldsymbol{\mu})(\mathbf{x} - \boldsymbol{\mu})^t p(\mathbf{x})\, dx $

where the expected value of a vector or a matrix is found by taking the expected value of the individual components. i.e if *x _{i}* is the

*i*th component of

**x**,

*μ*the

_{i}*i*th component of

**μ**, and

*σ*the

_{ij}*ij*th component of

**Σ**, then

$ \mu_i = \mathcal{E}[x_i] $

and

$ \sigma_{ij} = \mathcal{E}[(x_i - \mu_i)(x_j - \mu_j)] $

The covariance matrix **Σ** is always symmetric and positive definite which means that the determinant of **Σ** is strictly positive. The diagonal elements *σ _{ii}* are the variances of the respective

*x*( i.e.,

_{i}*σ*), and the off-diagonal elements

^{2}*σ*are the covariances of

_{ij}*x*and

_{i}*x*. If

_{j}*x*and

_{i}*x*are statistically independent, then

_{j}*σ*= 0. If all off-diagonanl elements are zero,

_{ij}*p*(

**x**) reduces to the product of the univariate normal densities for the components of

**x**.

**Discriminant Functions**

Discriminant functions are used to find the minimum probability of error in decision making problems. In a problem with feature vector **y** and state of nature variable *w*, we can represent the discriminant function as:

$ g_i(\mathbf{Y}) = \ln p(\mathbf{Y}|w_i) + \ln P(w_i) $

where from previous essays we defined p(**Y**|*w _{i}*) as the conditional probability density function for

**Y**with

*w*being the state of nature, and

_{i}*P*(

*w*) is the prior probability that nature is in state

_{j}*w*. If we take p(

_{j}**Y**|

*w*) as multivariate normal distributions. That is if p(

_{i}**Y**|

*w*) =

_{i}*N(*. Then the discriminant function changes to;

**μ**,**σ**)$ g_i(\mathbf{Y}) = - \frac{||\mathbf{x} - \boldsymbol{\mu}_i||^2}{\boldsymbol{\sigma}_i } + \ln P(w_i) $,

where ||.|| denotes the *Euclidean norm*, that is,

Next week, we will look more in depth into discriminant functions for the normal density, looking at the special cases of the covariance.