Topic 15: Conditional Distributions for Two Random Variables

## Conditional Distributions

There are many applications of probability theory where we want to know the probabilistic behavior of a random variable Y given the value of another random variable X. We get this using conditional distributions.

Definition $\qquad$ For random variables X and Y defined on (S,F,P), the joint cdf of X and Y given an event M ∈ F, with P(M) >0, is

$F_{XY}(x,y|M)=\frac{P(\{X\leq x,Y\leq y\}\cap M)}{P(M)},$

A special case of great interest is:

$\lim_{x\rightarrow\infty}F_{XY}(x,y|\{x_1<X\leq x_2\}) = F_Y(y|x_1<X\leq x_2)$

where x$_1$ < x$_2$. In this case, we have, assuming P(x$_1$ < X ≤ x$_2$),

\begin{align} F_Y(y|x_1<X\leq x_2) &= \frac{P(Y\leq y,x_1<X\leq x_2)}{P(x_1<X\leq x_2)} \\ &= \frac{F_{XY}(x_2,y)-F_{XY}(x_1,y)}{F_X(x_2)-F_X(x_1)} \end{align}

What we really want is f_$_Y$(y|x$_1$ < X ≤ x$_2$), so we need to differentiate with respect to y. We do not have a name for this partial differentiation and we have not talked about how to find it, but we will need it here.

Now, writing

$F_{XY}(x,y)=\int_{-\infty}^y\int_{-\infty}^xf_{XY}(x',y')dx'dy'$

we have

\begin{align} f_Y(y|x_1<X\leq x_2) &= \frac{\frac{\partial}{\partial y}\left[ \int_{-\infty}^{x_2}\int_{-\infty}^yf_{XY}(x,y')dxdy'-\int_{-\infty}^{x_1}\int_{-\infty}^yf_{XY}(x,y')dxdy'\right]}{F_X(x_2)-F_X(x_1)} \\ \\ &=\frac{\int_{-\infty}^{x_2}f_{XY}(x,y)dx-\int_{-\infty}^{x_1}f_{XY}(x,y)dx}{F_X(x_2)-F_X(x_1)} \end{align}
$f_Y(y|x_1<X\leq x_2)=\frac{\int_{x_1}^{x_2}f_{XY}(x,y)dx}{F_X(x_2)-F_X(x_1)}$

But what we really want is f$_Y$(y|X = x) ∀x ∈ R. We cannot set x$_1$ = x$_2$ in the above equation, so instead, let

$f_Y(y|X=x)=\lim_{\Delta x\rightarrow 0}f_Y(y|x<X\leq x+\Delta x)$

Setting x$_1$ = x and x$_2$ = x + Δx,

$f_Y(y|x<X\leq x+\Delta x)=\frac{\int_x^{x+\Delta x}f_{XY}(x',y)dx'}{F_X(x+\Delta x)-F_X(x)}$

Multiplying the denominator and numerator by 1/Δx and taking the limit,

$f_Y(y|X=x) = \lim_{\Delta x\rightarrow 0}\frac {\frac{1}{\Delta x}\int_x^{x+\Delta x}f_{XY}(x',y)dx'} {\frac{1}{\Delta x}[F_X(x+\Delta x)-F_X(x)]}$

Now let

$f_Y(y|X=x) = \frac{\lim_{\Delta x\rightarrow 0}\frac{1}{\Delta x}[F(x+\Delta x)-F(x)]}{\lim_{\Delta x\rightarrow 0}\frac{1}{\Delta x}[F_X(x+\Delta x)-F_X(x)]}$

The numerator is the derivative of F(x) with respect to x and the denominator is the derivative of F$_X$(x) with respect to x, so

$f_Y(y|X=x) = \frac{f_{XY}(x,y)}{f_X(x)}$

Similarly,

$f_X(x|Y=y) = \frac{f_{XY}(x,y)}{f_Y(y)}$

Notation

$f_X(x|Y=y)\equiv f_{X|Y}(x|y) = f(x|y)$

Writing two equations above in terms of f$_{XY}$ and setting them equal to each other gives Bayes' formula:

$f_{X|Y}(x|y) = \frac{f_{Y|X}(y|x)f_X(x)}{f_Y(y)}$

We also have a Total Probability Law:

\begin{align} f_Y(y)&=\int_{-\infty}^{\infty}f_{XY}(x,y)dx \\ &=\int_{-\infty}^{\infty}f_{Y|X}(y|x)f_X(x)dx \end{align}

So we can write f${X|Y}$(x|y) in terms of f${Y|X}$(x|y)f$_X$(x), which can be very useful.

Note that if X and Y are independent, we have

\begin{align} f_{Y|X}(y|x)&=\frac{f_{XY}(x,y)}{f_X(x)} \\ &=\frac{f_X(x)f_Y(y)}{f_X(x)} \\ &=f_Y(y) \end{align}

So f${Y|X}$(x|y) does not depend on x.

Summary of three forms of Bayes' formula that we have derived:

$\bullet P(A|B)=\frac{P(B|A)P(A)}{P(B)}\qquad A,B\in\mathcal F$
Use this form when X and y are discrete with A = {Y = y}, B={X = x}, so
$\;p_{Y|X}(y|x) =\frac{p_{X|Y}(x|y)p_Y(y)}{p_X(x)}$
where p$_{Y|X}$(y|x) ≡ P(Y=y|X=x) and p$_{X|Y}$(x|y) ≡ P(X=x|Y=y) are conditional pdfs.
$\bullet P(M|Y=y)=\frac{f_Y(y|M)P(M)}{f_Y(y)}\qquad M\in\mathcal F$
Use this when Y is continuous and X is discrete with M = {X = x}, so
$\;p_{X|Y}(x|y) =\frac{f_{Y|X}(y|x)p_X(x)}{f_Y(y)}$
$\bullet f_{Y|X}(y|x)=\frac{f_{X|Y}(x|y)f_Y(y)}{f_X(x)}$
Use this when X,Y are both continuous.