Back to all ECE 600 notes

The Comer Lectures on Random Variables and Signals

Slectures by Maliha Hossain


Topic 15: Conditional Distributions for Two Random Variables



Conditional Distributions

There are many applications of probability theory where we want to know the probabilistic behavior of a random variable Y given the value of another random variable X. We get this using conditional distributions.

Definition $ \qquad $ For random variables X and Y defined on (S,F,P), the joint cdf of X and Y given an event M ∈ F, with P(M) >0, is

$ F_{XY}(x,y|M)=\frac{P(\{X\leq x,Y\leq y\}\cap M)}{P(M)}, $

A special case of great interest is:

$ \lim_{x\rightarrow\infty}F_{XY}(x,y|\{x_1<X\leq x_2\}) = F_Y(y|x_1<X\leq x_2) $

where x$ _1 $ < x$ _2 $. In this case, we have, assuming P(x$ _1 $ < X ≤ x$ _2 $),

$ \begin{align} F_Y(y|x_1<X\leq x_2) &= \frac{P(Y\leq y,x_1<X\leq x_2)}{P(x_1<X\leq x_2)} \\ &= \frac{F_{XY}(x_2,y)-F_{XY}(x_1,y)}{F_X(x_2)-F_X(x_1)} \end{align} $

What we really want is f_$ _Y $(y|x$ _1 $ < X ≤ x$ _2 $), so we need to differentiate with respect to y. We do not have a name for this partial differentiation and we have not talked about how to find it, but we will need it here.

Now, writing

$ F_{XY}(x,y)=\int_{-\infty}^y\int_{-\infty}^xf_{XY}(x',y')dx'dy' $

we have

$ \begin{align} f_Y(y|x_1<X\leq x_2) &= \frac{\frac{\partial}{\partial y}\left[ \int_{-\infty}^{x_2}\int_{-\infty}^yf_{XY}(x,y')dxdy'-\int_{-\infty}^{x_1}\int_{-\infty}^yf_{XY}(x,y')dxdy'\right]}{F_X(x_2)-F_X(x_1)} \\ \\ &=\frac{\int_{-\infty}^{x_2}f_{XY}(x,y)dx-\int_{-\infty}^{x_1}f_{XY}(x,y)dx}{F_X(x_2)-F_X(x_1)} \end{align} $
$ f_Y(y|x_1<X\leq x_2)=\frac{\int_{x_1}^{x_2}f_{XY}(x,y)dx}{F_X(x_2)-F_X(x_1)} $

But what we really want is f$ _Y $(y|X = x) ∀x ∈ R. We cannot set x$ _1 $ = x$ _2 $ in the above equation, so instead, let

$ f_Y(y|X=x)=\lim_{\Delta x\rightarrow 0}f_Y(y|x<X\leq x+\Delta x) $

Setting x$ _1 $ = x and x$ _2 $ = x + Δx,

$ f_Y(y|x<X\leq x+\Delta x)=\frac{\int_x^{x+\Delta x}f_{XY}(x',y)dx'}{F_X(x+\Delta x)-F_X(x)} $

Multiplying the denominator and numerator by 1/Δx and taking the limit,

$ f_Y(y|X=x) = \lim_{\Delta x\rightarrow 0}\frac {\frac{1}{\Delta x}\int_x^{x+\Delta x}f_{XY}(x',y)dx'} {\frac{1}{\Delta x}[F_X(x+\Delta x)-F_X(x)]} $

Now let

$ f_Y(y|X=x) = \frac{\lim_{\Delta x\rightarrow 0}\frac{1}{\Delta x}[F(x+\Delta x)-F(x)]}{\lim_{\Delta x\rightarrow 0}\frac{1}{\Delta x}[F_X(x+\Delta x)-F_X(x)]} $

The numerator is the derivative of F(x) with respect to x and the denominator is the derivative of F$ _X $(x) with respect to x, so

$ f_Y(y|X=x) = \frac{f_{XY}(x,y)}{f_X(x)} $

Similarly,

$ f_X(x|Y=y) = \frac{f_{XY}(x,y)}{f_Y(y)} $

Notation

$ f_X(x|Y=y)\equiv f_{X|Y}(x|y) = f(x|y) $


Writing two equations above in terms of f$ _{XY} $ and setting them equal to each other gives Bayes' formula:

$ f_{X|Y}(x|y) = \frac{f_{Y|X}(y|x)f_X(x)}{f_Y(y)} $

We also have a Total Probability Law:

$ \begin{align} f_Y(y)&=\int_{-\infty}^{\infty}f_{XY}(x,y)dx \\ &=\int_{-\infty}^{\infty}f_{Y|X}(y|x)f_X(x)dx \end{align} $

So we can write f$ {X|Y} $(x|y) in terms of f$ {Y|X} $(x|y)f$ _X $(x), which can be very useful.

Note that if X and Y are independent, we have

$ \begin{align} f_{Y|X}(y|x)&=\frac{f_{XY}(x,y)}{f_X(x)} \\ &=\frac{f_X(x)f_Y(y)}{f_X(x)} \\ &=f_Y(y) \end{align} $

So f$ {Y|X} $(x|y) does not depend on x.


Summary of three forms of Bayes' formula that we have derived:

$ \bullet P(A|B)=\frac{P(B|A)P(A)}{P(B)}\qquad A,B\in\mathcal F $
Use this form when X and y are discrete with A = {Y = y}, B={X = x}, so
$ \;p_{Y|X}(y|x) =\frac{p_{X|Y}(x|y)p_Y(y)}{p_X(x)} $
where p$ _{Y|X} $(y|x) ≡ P(Y=y|X=x) and p$ _{X|Y} $(x|y) ≡ P(X=x|Y=y) are conditional pdfs.
$ \bullet P(M|Y=y)=\frac{f_Y(y|M)P(M)}{f_Y(y)}\qquad M\in\mathcal F $
Use this when Y is continuous and X is discrete with M = {X = x}, so
$ \;p_{X|Y}(x|y) =\frac{f_{Y|X}(y|x)p_X(x)}{f_Y(y)} $
$ \bullet f_{Y|X}(y|x)=\frac{f_{X|Y}(x|y)f_Y(y)}{f_X(x)} $
Use this when X,Y are both continuous.



References



Questions and comments

If you have any questions, comments, etc. please post them on this page



Back to all ECE 600 notes

Alumni Liaison

Followed her dream after having raised her family.

Ruth Enoch, PhD Mathematics