Back to all ECE 600 notes

The Comer Lectures on Random Variables and Signals

Slectures by Maliha Hossain


Topic 16: Conditional Expectation for Two Random Variables



If X and Y are random variables on (S,F,P) and ∈ F with P(M) > 0, then,

$ E[g(X,Y)|M] = \int\int_{\mathbb R^2} g(x,y)f_{XY}(x,y|M)dxdy $

One important case is when M = {Y = y} for some y ∈ R. Then we have that

$ E[g(X,Y)|Y=y]=\int\int_{\mathbb R^2}g(x,y')f_{XY}(x,y'|Y=y)dxdy' $

Using our old trick, let

$ E[g(X,Y)|Y=y]=\lim_{\Delta y\rightarrow 0}\int\int_{\mathbb R^2}g(x,y')f_{XY}(x,y'|y<Y\leq y+\Delta y)dxdy' $

Using this approach, it can be shown that

$ \begin{align} E[g(X,Y)|Y=y]&=E[g(X,y)|Y=y] \\ &=\int_{-\infty}^{\infty}g(x,y)f_{X|Y}(x|y)dx \\ \end{align} $


Another important case: g(X,Y)=g(X)

$ \begin{align} E[g(X)|M]&=\int\int g(x)f_{XY}(x,y|M)dxdy \\ &=\int g(x)\left[\int f_{XY}(x,y|M)dy\right]dx \\ &=\int g(x)f_X(x|M)dx \end{align} $

Note that this is the same equation we had, for example

$ E[g(X)|Y=y]=\int g(x)f_{X|Y}(x|y)dx $



Iterated Expectation

Sometimes we want to work with f$ _{Y|X} $(y|x) and f$ _X $(x) instead of f$ _{XY} $(x,y). This can make computation of E[g(X,Y)] easier in some cases. We can write

$ \begin{align} E[g(X,Y)]&=\int\int_{\mathbb R^2}g(x,y)f_{XY}(x,y)dxdy \\ &=\int\int_{\mathbb R^2}f_X(x)g(x,y)f_{Y|X}(y|x)dydx \\ &=\int_{\mathbb R}f_X(x)E[g(X,Y)|X=x]dx \end{align} $

Note that E[g(X,Y)|X=x] is a function of x ∈ R. We will call this function h.

$ h(x)=E[g(X,Y)|X=x]\qquad h:\mathbb R\rightarrow\mathbb R $

We can create a random variable h(X). We will use the notation

$ E[g(X,Y)|X]\equiv h(X) $

So we have

$ h(x) = E[g(X,Y)|X=x] \ $

which is a real-valued function of x ∈ R, and h(X), which is a random variable since it is a function of random variable X.

Now we can write

$ \begin{align} E[g(X,Y)]&=\int_{\mathbb R}E[g(X,Y)|X=x]f_X(x)dx \\ &= \int_{\mathbb R}h(x)f_X(x)dx \\ &=E[h(X)] \end{align} $

So,

$ E[g(X,Y)] = E[E[g(X,Y)|X]] \ $

We call this iterated expectation.

An important special case is when g(X,Y)=Y, in which case, we have

$ E[Y]=E[E[Y|X]] \ $


Example $ \qquad $ Suppose we have a stick of length l. We break the stick at a uniformly chosen point Y, then again at a uniformly chosen point X. Find E[X].


Fig 1: example problem


$ \begin{align} E[X]&=\int\int_{\mathbb R^2}xf_{XY}(x,y)dxdy \\ &=\int_{\mathbb R}xf_X(x)dx \end{align} $

We do not know f$ _{XY} $ or f$ _X $, but we know f$ _{X|Y} $ or f$ _Y $

$ \begin{align} f_Y(y)&=\frac{1}{l}\qquad\ &0\leq y\leq l \\ f_{X|Y}(x|y)&=\frac{1}{y}\qquad &0\leq x\leq y \end{align} $

Use E[X]=E[E[X|Y]]. Now

$ h(y)\equiv E[X|Y=y] = \frac{y}{2} $

since X is uniform on [0,y] given Y=y. So,

$ h(Y)=\frac{Y}{2} $

Then

$ E[X]=E\left[\frac{Y}{2}\right]=\frac{1}{2}E[Y]=\frac{1}{2}.\frac{l}{2}=\frac{l}{4} $



References



Questions and comments

If you have any questions, comments, etc. please post them on this page



Back to all ECE 600 notes

Alumni Liaison

Basic linear algebra uncovers and clarifies very important geometry and algebra.

Dr. Paul Garrett