Topic 10: Characteristic Functions

## Characteristic Functions

The pdf f$_X$ of a random variable X is a function of a real valued variable x. It is sometimes useful to work with a "frequency domain" representation of f$_X$. The characteristic function gives us this representation.

Definition $\qquad$ Z:SC defined on (S,F,P) is a complex random variable if

$Z = X+iY$

where X and Y are real valued random variables on (S,F,P).

Using the linearity of E[],

$E[Z] = E[X] + iE[Y] \$

Now consider the complex random variable Z = $e^{i\omega X}$, where $\omega$R is a "frequency" variable. We can write Z as

$Z=e^{i\omega X}=\cos(\omega X)+i\sin(\omega X) \$

and

$E[Z]=E[e^{i\omega X}]=E[\cos(\omega X)]+iE[\sin(\omega X)] \$

This expectation depends on $\omega$R and will be the characteristic function of X.

Definition $\qquad$ Let X be a random variable on (S,F,P). The characteristic function X is given by

$\Phi_X(\omega)\equiv E[e^{i\omega X}]\qquad\forall\omega\in\mathbb R$

If X is continuous, we have

$\Phi_X(\omega)= \int_{-\infty}^{\infty}e^{i\omega x}f_X(x)dx$

And if X is discrete, then we use

$\Phi_X(\omega)= \sum_{x\in\mathcal R_x}e^{i\omega x}p_X(x)$

Note: The characteristic function looks like the Fourier Transform of f$_X$, with opposite sign in the exponent. We can show that

$f_X(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty}\Phi_X(\omega)e^{-i\omega x}d\omega$

## Moments

Definition $\qquad$ The Moment Generating Function (mgf) of random variable X is given by

$\phi_X(s)\equiv E[e^{sX}]\qquad\forall s\in\mathbb C$

Moment Theorem $\qquad$ The Moment Theorem (or Moment Generating Theorem) shows us how to use the mgf to find moments of X:
given a random variable X with mgf $\phi_X$, the nth moment pf X is given by

\begin{align} \mu_n &= E[X^n] \\ \\ &=\phi^{(n)}(0) \\ \\ &= \frac{d^n\phi_X(s)}{ds^n}|_{s=0} \end{align}

Proof:
Differentiating $\phi_X$ with respect to s n times gives

\begin{align} \phi_X^{(n)}(s)&=\frac{d^n}{ds^n}E[e^{sX}] \\ \\ &=E[\frac{d^n}{ds^n}e^{sX}] \\ \\ &=E[X^ne^{sX}] \end{align}

So,

$\phi_X^{(n)}(0)=E[X^n]$

This result can be written in terms of the characteristic function:

$\mu_n = \frac{1}{i^n}\;\frac{d^n}{d\omega^n}\Phi_X(\omega)|_{\omega = 0}$

Example $\qquad$ X is an exponential random variable. We can show that

$\Phi_X(\omega) = \frac{1/\mu}{1/\mu - i\omega}$

since

$f_X(x) = \frac{1}{\mu}e^{-\frac{x}{\mu}}u(x)$

Now,

$\Phi_X'(\omega) = \frac{(1/\mu)\;i}{(1/\mu-i\omega)^2}$

and

$\Phi_X''(\omega) = \frac{-2/\mu}{(1/\mu-i\omega)^3}$

So,

$E[X]=\frac{1}{i}(\frac{(1/\mu)\;i}{(1/\mu)^2})=\mu$

and

$E[X^2]=\frac{1}{i^2}(\frac{-2/\mu}{(1/\mu)^3})=2\mu^2$

Then

$Var(X) = 2\mu^2-\mu^2 =\mu^2 \$