Topic 16: Conditional Expectation for Two Random Variables

If X and Y are random variables on (S,F,P) and ∈ F with P(M) > 0, then,

$E[g(X,Y)|M] = \int\int_{\mathbb R^2} g(x,y)f_{XY}(x,y|M)dxdy$

One important case is when M = {Y = y} for some y ∈ R. Then we have that

$E[g(X,Y)|Y=y]=\int\int_{\mathbb R^2}g(x,y')f_{XY}(x,y'|Y=y)dxdy'$

Using our old trick, let

$E[g(X,Y)|Y=y]=\lim_{\Delta y\rightarrow 0}\int\int_{\mathbb R^2}g(x,y')f_{XY}(x,y'|y<Y\leq y+\Delta y)dxdy'$

Using this approach, it can be shown that

\begin{align} E[g(X,Y)|Y=y]&=E[g(X,y)|Y=y] \\ &=\int_{-\infty}^{\infty}g(x,y)f_{X|Y}(x|y)dx \\ \end{align}

Another important case: g(X,Y)=g(X)

\begin{align} E[g(X)|M]&=\int\int g(x)f_{XY}(x,y|M)dxdy \\ &=\int g(x)\left[\int f_{XY}(x,y|M)dy\right]dx \\ &=\int g(x)f_X(x|M)dx \end{align}

Note that this is the same equation we had, for example

$E[g(X)|Y=y]=\int g(x)f_{X|Y}(x|y)dx$

## Iterated Expectation

Sometimes we want to work with f$_{Y|X}$(y|x) and f$_X$(x) instead of f$_{XY}$(x,y). This can make computation of E[g(X,Y)] easier in some cases. We can write

\begin{align} E[g(X,Y)]&=\int\int_{\mathbb R^2}g(x,y)f_{XY}(x,y)dxdy \\ &=\int\int_{\mathbb R^2}f_X(x)g(x,y)f_{Y|X}(y|x)dydx \\ &=\int_{\mathbb R}f_X(x)E[g(X,Y)|X=x]dx \end{align}

Note that E[g(X,Y)|X=x] is a function of x ∈ R. We will call this function h.

$h(x)=E[g(X,Y)|X=x]\qquad h:\mathbb R\rightarrow\mathbb R$

We can create a random variable h(X). We will use the notation

$E[g(X,Y)|X]\equiv h(X)$

So we have

$h(x) = E[g(X,Y)|X=x] \$

which is a real-valued function of x ∈ R, and h(X), which is a random variable since it is a function of random variable X.

Now we can write

\begin{align} E[g(X,Y)]&=\int_{\mathbb R}E[g(X,Y)|X=x]f_X(x)dx \\ &= \int_{\mathbb R}h(x)f_X(x)dx \\ &=E[h(X)] \end{align}

So,

$E[g(X,Y)] = E[E[g(X,Y)|X]] \$

We call this iterated expectation.

An important special case is when g(X,Y)=Y, in which case, we have

$E[Y]=E[E[Y|X]] \$

Example $\qquad$ Suppose we have a stick of length l. We break the stick at a uniformly chosen point Y, then again at a uniformly chosen point X. Find E[X].

Fig 1: example problem

\begin{align} E[X]&=\int\int_{\mathbb R^2}xf_{XY}(x,y)dxdy \\ &=\int_{\mathbb R}xf_X(x)dx \end{align}

We do not know f$_{XY}$ or f$_X$, but we know f$_{X|Y}$ or f$_Y$

\begin{align} f_Y(y)&=\frac{1}{l}\qquad\ &0\leq y\leq l \\ f_{X|Y}(x|y)&=\frac{1}{y}\qquad &0\leq x\leq y \end{align}

Use E[X]=E[E[X|Y]]. Now

$h(y)\equiv E[X|Y=y] = \frac{y}{2}$

since X is uniform on [0,y] given Y=y. So,

$h(Y)=\frac{Y}{2}$

Then

$E[X]=E\left[\frac{Y}{2}\right]=\frac{1}{2}E[Y]=\frac{1}{2}.\frac{l}{2}=\frac{l}{4}$