(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
=5 Exams=
 
=5 Exams=
  
−
=Example. Addition of two independent Poisson random variables=
+
Problem Examples
  
−
Let <math>\mathbf{Z}=\mathbf{X}+\mathbf{Y}</math>  where <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  are independent Poisson random variables with means <math>\lambda</math>  and <math>\mu</math> , respectively.
+
*[[ECE 600 Exams Addition of two independent Poisson random variables|Addition of two independent Poisson random variables]]
 +
*[[ECE 600 Exams Addition of two independent Gaussian random variables|Addition of two independent Gaussian random variables]]
 +
*[[ECE 600 Exams Addition of two jointly distributed Gaussian random variables|Addition of two jointly distributed Gaussian random variables]]
 +
*[[ECE 600 Exams Two jointly distributed random variables|Two jointly distributed random variables]]
 +
*[[ECE 600 Exams Two jointly distributed independent random variables|Two jointly distributed independent random variables]]
 +
*[[ECE 600 Exams Two jointly distributed random variables (Joint characteristic function)|Two jointly distributed random variables (Joint characteristic function)]]
 +
*[[ECE 600 Exams Geometric random variable|Geometric random variable]]
 +
*[[ECE 600 Exams Sequence of binomially distributed random variables|Sequence of binomially distributed random variables]]
 +
*[[ECE 600 Exams Sequence of exponentially distributed random variables|Sequence of exponentially distributed random variables]]
 +
*[[ECE 600 Exams Sequence of uniformly distributed random variables|Sequence of uniformly distributed random variables]]
 +
*[[ECE 600 Exams Mean of iid random variables|Mean of iid random variables]]
 +
*[[ECE 600 Exams A sum of a random number of iid Gaussian random variables|A sum of a random number of iid Gaussian random variables]]
  
−
(a)
+
----
−
 
+
[[ECE600|Back to ECE600]]
−
Find the pmf of <math>\mathbf{Z}</math> .
+
−
 
+
−
According to the characteristic function of Poisson random variable
+
−
 
+
−
<math>\Phi_{\mathbf{X}}(\omega)=e^{-\lambda\left(1-e^{i\omega}\right)},\Phi_{\mathbf{Y}}(\omega)=e^{-\mu\left(1-e^{i\omega}\right)}</math>.
+
−
 
+
−
<math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  are independent <math>\Longrightarrow  \mathbf{X}</math>  and <math>\mathbf{Y}</math>  are uncorrelated <math>\Longrightarrow  e^{i\omega\mathbf{X}}</math>  and <math>e^{i\omega\mathbf{Y}}</math>  are uncorrelated.
+
−
 
+
−
<math>\Phi_{\mathbf{Z}}(\omega)=E\left[e^{i\omega\mathbf{Z}}\right]=E\left[e^{i\omega\left(\mathbf{X}+\mathbf{Y}\right)}\right]=E\left[e^{i\omega\mathbf{X}}e^{i\omega\mathbf{Y}}\right]=E\left[e^{i\omega\mathbf{X}}\right]\cdot E\left[e^{i\omega\mathbf{Y}}\right]</math><math>=e^{-\lambda\left(1-e^{i\omega}\right)}\cdot e^{-\mu\left(1-e^{i\omega}\right)}=e^{-\left(\lambda+\mu\right)\left(1-e^{i\omega}\right).}</math>
+
−
 
+
−
Now, we know that \mathbf{Z}  is a Poisson random variable with mean <math>\lambda+\mu</math> .
+
−
 
+
−
<math>\therefore p_{\mathbf{Z}}(k)=\frac{e^{-\left(\lambda+\mu\right)}\left(\lambda+\mu\right)^{k}}{k!}.</math>
+
−
 
+
−
(b)
+
−
 
+
−
Show that the conditional pmf of <math>\mathbf{X}</math>  conditioned on the event <math>\left\{ \mathbf{Z}=n\right\}</math>  is binomially distributed, and determine the parameters of binomial distribution (<math>n</math>  and <math>p</math> ).
+
−
 
+
−
<math>P_{\mathbf{X}}\left(\mathbf{X}|\left\{ \mathbf{Z}=n\right\} \right)=P\left(\left\{ \mathbf{X}=k\right\} |\left\{ \mathbf{Z}=n\right\} \right)=\frac{P\left(\left\{ \mathbf{X}=k\right\} \cap\left\{ \mathbf{Z}=n\right\} \right)}{P\left(\left\{ \mathbf{Z}=n\right\} \right)}=\frac{P\left(\left\{ \mathbf{X}=k\right\} \cap\left\{ \mathbf{Y}=n-k\right\} \right)}{P\left(\left\{ \mathbf{Z}=n\right\} \right)}</math><math>=\frac{\frac{e^{-\lambda}\lambda^{k}}{k!}\cdot\frac{e^{-\mu}\mu^{n-k}}{\left(n-k\right)!}}{\frac{e^{-\left(\lambda+\mu\right)}\left(\lambda+\mu\right)^{n}}{n!}}=\left(\frac{n!}{k!\left(n-k\right)!}\right)\left(\frac{\lambda}{\lambda+\mu}\right)^{k}\left(\frac{\mu}{\lambda+\mu}\right)^{n-k}</math><math>=\left(\begin{array}{c}
+
−
n\\
+
−
k
+
−
\end{array}\right)\left(\frac{\lambda}{\lambda+\mu}\right)^{k}\left(\frac{\mu}{\lambda+\mu}\right)^{n-k}\;,\; k=0,\,1,\,2,\,\cdots</math>
+
−
 
+
−
This is a binomial pmf <math>b(n,p)</math>  with parameters <math>n</math>  and <math>p=\frac{\lambda}{\lambda+\mu}</math> .
+
−
 
+
−
=Example. Addition of two independent Gaussian random variables=
+
−
 
+
−
<math>\mathbf{X}\sim\mathcal{N}\left(0,\sigma_{\mathbf{X}}^{2}\right),\;\mathbf{N}\sim\mathcal{N}\left(0,\sigma_{\mathbf{N}}^{2}\right),\;\mathbf{Y}=\mathbf{X}+\mathbf{N}.</math>
+
−
 
+
−
(a)
+
−
 
+
−
Correlation coefficient between <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math> .
+
−
 
+
−
<math>\sigma_{\mathbf{Y}}=\sqrt{\sigma_{\mathbf{X}}^{2}+2r_{\mathbf{XN}}\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}+\sigma_{\mathbf{N}}^{2}}=\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}</math>
+
−
 
+
−
because <math>\mathbf{X}</math>  and <math>\mathbf{N}</math>  are independnet <math>\Longrightarrow</math> uncorrelated <math>\Longrightarrow r_{\mathbf{XN}}=0</math> .
+
−
 
+
−
<math>r_{\mathbf{XY}}=\frac{\text{cov}(\mathbf{X},\mathbf{Y})}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}\left(\mathbf{X}+\mathbf{N}\right)\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}^{2}\right]+E\left[\mathbf{XN}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}</math><math>=\frac{\sigma_{\mathbf{X}}^{2}+E\left[\mathbf{X}\right]E\left[\mathbf{N}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\qquad\because E\left[\mathbf{X}\right]=0.</math>
+
−
 
+
−
(b)
+
−
 
+
−
Conditional pmf of <math>\mathbf{X}</math>  conditioned on the event <math>\left\{ \mathbf{Y}=y\right\}</math>  .
+
−
 
+
−
<math>f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{Y}}(y)}=\frac{\frac{1}{2\pi\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} }{\frac{1}{\sqrt{2\pi}\sigma_{Y}}\exp\left\{ \frac{-y^{2}}{2\sigma_{Y}^{2}}\right\} }</math><math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}-\frac{\left(1-r^{2}\right)y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\}</math> <math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{r^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\}</math> <math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left[x^{2}-\frac{2r\sigma_{\mathbf{X}}xy}{\sigma_{\mathbf{Y}}}+\frac{r^{2}\sigma_{\mathbf{X}}^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\}</math> <math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left(x-\frac{r\sigma_{\mathbf{X}}y}{\sigma_{\mathbf{Y}}}\right)^{2}\right\}</math>
+
−
 
+
−
Noting that <math>\sqrt{1-r^{2}}=\sigma_{\mathbf{X}}\sqrt{1-\left(\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}\right)^{2}}=\sqrt{1-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}=\sqrt{\frac{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}-\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}</math>  and
+
−
 
+
−
<math>r\cdot\frac{\sigma_{\mathbf{X}}}{\sigma_{\mathbf{Y}}}=\frac{\sigma_{X}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\cdot\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}.</math> <math>\therefore f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{1}{\sqrt{2\pi}\cdot\frac{\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\mathbf{\sigma}_{\mathbf{N}}^{\mathbf{2}}}}}\exp\left\{ \frac{-1}{2\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\left(x-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y\right)^{2}\right\}</math> 
+
−
 
+
−
(c)
+
−
 
+
−
What kind of pdf is the pdf you determined in part (b)? What is the mean and variance of a random variable with this pdf?
+
−
 
+
−
This is a Gaussian pdf with mean <math>\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y</math>  and variance <math>\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}</math> .
+
−
 
+
−
(d)
+
−
 
+
−
What is the minimum mean-square estimate of <math>\mathbf{X}</math>  given that <math>\left\{ \mathbf{Y}=y\right\}</math>  ?
+
−
 
+
−
The minimum mean-square error estimate of <math>\mathbf{X}</math>  given <math>\mathbf{Y}=y</math>  is
+
−
 
+
−
<math>\hat{x}_{MMS}(y)=E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\int_{-\infty}^{\infty}x\cdot f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)dx=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y</math>  from part (b).
+
−
 
+
−
(e)
+
−
 
+
−
What is the maximum a posteriori estimate of <math>\mathbf{X}</math>  given that <math>\left\{ \mathbf{Y}=y\right\}</math>  ?
+
−
 
+
−
<math>\hat{x}_{MAP}(y)=\arg\max_{x\in\mathbf{R}}\left\{ f_{\mathbf{X}}\left(x|\left\{ Y=y\right\} \right)\right\} =\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y</math>
+
−
 
+
−
as a Gaussian pdf takes on its maximum value at its mean.
+
−
 
+
−
(f)
+
−
 
+
−
Given that I observe <math>\mathbf{Y}=y</math> , what is <math>E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]</math> ?
+
−
 
+
−
<math>E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y</math> from part (d).
+
−
 
+
−
=Example. Addition of two jointly distributed Gaussian random variables=
+
−
 
+
−
Let <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  be two jointly distributed Gaussian random variables. Assume <math>\mathbf{X}</math>  has mean <math>\mu_{\mathbf{X}}</math>  and variance <math>\sigma_{\mathbf{X}}^{2} , \mathbf{Y}</math>  has mean <math>\mu_{\mathbf{Y}}</math>  and variance <math>\sigma_{\mathbf{Y}}^{2}</math> , and that the correlation coefficient between <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  is <math>r</math> . Define a new random variable <math>\mathbf{Z}=\mathbf{X}+\mathbf{Y}</math> .
+
−
 
+
−
(a)
+
−
 
+
−
Show that <math>\mathbf{Z}</math>  is a Gaussian random variable.
+
−
 
+
−
If <math>\mathbf{Z}</math>  is a Guassian random variable, then it has a characteristic function of the form
+
−
 
+
−
<math>\Phi_{\mathbf{Z}}\left(\omega\right)=e^{i\mu_{\mathbf{Z}}\omega}e^{-\frac{1}{2}\sigma_{\mathbf{Z}}^{2}\omega^{2}}.</math>
+
−
 
+
−
<math>\Phi_{\mathbf{Z}}\left(\omega\right)</math>
+
−
 
+
−
where <math>\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)</math>  is the joint characteristic function of <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math> , defined as
+
−
 
+
−
<math>\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)=E\left[e^{i\left(\mathbf{\omega_{1}X}+\omega_{2}\mathbf{Y}\right)}\right].</math>
+
−
 
+
−
Now because <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  are jointly Gaussian with the given parameters, we know that
+
−
 
+
−
<math>\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)=e^{i\left(\mu_{X}\omega_{1}+\mu_{Y}\omega_{2}\right)}e^{-\frac{1}{2}\left(\sigma_{X}^{2}\omega_{1}^{2}+2r\sigma_{X}\sigma_{Y}\omega_{1}\omega_{2}+\sigma_{Y}^{2}\omega_{2}^{2}\right)}.</math>
+
−
 
+
−
Thus,
+
−
 
+
−
<math>\Phi_{\mathbf{Z}}\left(\omega\right)=\Phi_{\mathbf{XY}}\left(\omega,\omega\right)=e^{i\left(\mu_{X}\omega+\mu_{Y}\omega\right)}e^{-\frac{1}{2}\left(\sigma_{X}^{2}\omega^{2}+2r\sigma_{X}\sigma_{Y}\omega^{2}+\sigma_{Y}^{2}\omega^{2}\right)}</math><math>=e^{i\left(\mu_{X}+\mu_{Y}\right)\omega}e^{-\frac{1}{2}\left(\sigma_{X}^{2}+2r\sigma_{X}\sigma_{Y}+\sigma_{Y}^{2}\right)\omega^{2}}=e^{i\mu_{Z}\omega}e^{-\frac{1}{2}\sigma_{Z}^{2}\omega^{2}}</math>
+
−
 
+
−
where <math>\mu_{Z}=\mu_{X}+\mu_{Y}</math>  and <math>\sigma_{Z}^{2}=\sigma_{X}^{2}+2r\sigma_{X}\sigma_{Y}+\sigma_{Y}^{2}</math> .
+
−
 
+
−
<math>\mathbf{Z}</math>  is a Gaussian random variable with E\left[\mathbf{Z}\right]=\mu_{X}+\mu_{Y}  and Var\left[\mathbf{Z}\right]=\sigma_{X}^{2}+2r\sigma_{X}\sigma_{Y}+\sigma_{Y}^{2} .
+
−
 
+
−
(b)
+
−
 
+
−
Find the variance of <math>\mathbf{Z}</math> .
+
−
 
+
−
As show in part (a) <math>Var\left[\mathbf{Z}\right]=\sigma_{\mathbf{X}}^{2}+2r\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}+\sigma_{\mathbf{Y}}^{2}</math> .
+
−
 
+
−
=Example. Two jointly distributed random variables=
+
−
 
+
−
Two joinly distributed random variables <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  have joint pdf
+
−
 
+
−
<math>f_{\mathbf{XY}}\left(x,y\right)=\begin{cases}
+
−
\begin{array}{ll}
+
−
c  ,\text{ for }x\geq0,y\geq0,\textrm{ and }x+y\leq1\\
+
−
0  ,\text{ elsewhere.}
+
−
\end{array}\end{cases}</math>
+
−
 
+
−
(a)
+
−
 
+
−
Find the constant c  such that <math>f_{\mathbf{XY}}(x,y)</math>  is a valid pdf.
+
−
 
+
−
[[Image:002.eps]]
+
−
 
+
−
<math>\iint_{\mathbf{R}^{2}}f_{\mathbf{XY}}\left(x,y\right)=c\cdot Area=1</math>  where <math>Area=\frac{1}{2}</math> .
+
−
 
+
−
<math>\therefore c=2</math>
+
−
 
+
−
(b)
+
−
 
+
−
Find the conditional density of \mathbf{Y}  conditioned on \mathbf{X}=x .
+
−
 
+
−
f_{\mathbf{Y}}\left(y|\left\{ \mathbf{X}=x\right\} \right)=\frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{X}}(x)}.
+
−
 
+
−
f_{\mathbf{X}}(x)=\int_{-\infty}^{\infty}f_{\mathbf{XY}}\left(x,y\right)dy=\int_{0}^{1-x}2dy=2\left(1-x\right)\cdot\mathbf{1}_{\left[0,1\right]}(x).
+
−
 
+
−
f_{\mathbf{Y}}\left(y|\left\{ \mathbf{X}=x\right\} \right)=\frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{X}}(x)}=\frac{2}{2\left(1-x\right)}=\frac{1}{1-x}\textrm{ where }0\leq y\leq1-x\Longrightarrow\frac{1}{1-x}\cdot\mathbf{1}_{\left[0,1-x\right]}\left(y\right).
+
−
 
+
−
(c)
+
−
 
+
−
Find the minimum mean-square error estimator \hat{y}_{MMS}\left(x\right)  of \mathbf{Y}  given that \mathbf{X}=x .
+
−
 
+
−
\hat{y}_{MMS}\left(x\right)=E\left[\mathbf{Y}|\left\{ \mathbf{X}=x\right\} \right]=\int_{\mathbf{R}}yf_{\mathbf{Y}}\left(y|\left\{ \mathbf{X}=x\right\} \right)dy=\int_{0}^{1-x}\frac{y}{1-x}dy=\frac{y^{2}}{2\left(1-x\right)}\biggl|_{0}^{1-x}=\frac{1-x}{2}.
+
−
 
+
−
(d)
+
−
 
+
−
Find a maximum aposteriori probability estimator.
+
−
 
+
−
\hat{y}_{MAP}\left(x\right)=\arg\max_{y}\left\{ f_{Y}\left(y|\left\{ \mathbf{X}=x\right\} \right)\right\}  but f_{Y}\left(y|\left\{ \mathbf{X}=x\right\} \right)=\frac{1}{1-x}\cdot\mathbf{1}_{\left[0,1-x\right]}\left(y\right) . Any \hat{y}\in\left[0,1-x\right]  is a MAP estimator. The MAP estimator is NOT unique.
+
−
 
+
−
Example. Two jointly distributed independent random variables
+
−
 
+
−
Let \mathbf{X}  and \mathbf{Y}  be two jointly distributed, independent random variables. The pdf of \mathbf{X}  is
+
−
 
+
−
f_{\mathbf{X}}\left(x\right)=xe^{-x^{2}/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(x\right), and \mathbf{Y}  is a Gaussian random variable with mean 0  and variance 1 . Let \mathbf{U}  and \mathbf{V}  be two new random variables defined as \mathbf{U}=\sqrt{\mathbf{X}^{2}+\mathbf{Y}^{2}}  and \mathbf{V}=\lambda\mathbf{Y}/\mathbf{X}  where \lambda  is a positive real number.
+
−
 
+
−
(a)
+
−
 
+
−
Find the joint pdf of \mathbf{U}  and \mathbf{V} . (Direct pdf method)
+
−
 
+
−
f_{\mathbf{UV}}\left(u,v\right)=f_{\mathbf{XY}}\left(x\left(u,v\right),y\left(u,v\right)\right)\left|\frac{\partial\left(x,y\right)}{\partial\left(u,v\right)}\right|
+
−
 
+
−
Solving for x  and y  in terms of u  and v , we have u^{2}=x^{2}+y^{2}  and v^{2}=\frac{\lambda^{2}y^{2}}{x^{2}}\Longrightarrow y^{2}=\frac{v^{2}x^{2}}{\lambda^{2}} .
+
−
 
+
−
Now, u^{2}=x^{2}+y^{2}=x^{2}+\frac{v^{2}x^{2}}{\lambda^{2}}=x^{2}\left(1+v^{2}/\lambda^{2}\right)\Longrightarrow x=\frac{u}{\sqrt{1+v^{2}/\lambda^{2}}}\Longrightarrow x\left(u,v\right)=\frac{u}{\sqrt{1+v^{2}/\lambda^{2}}} .
+
−
 
+
−
Thus, y=\frac{vx}{\lambda}=\frac{vu}{\lambda\sqrt{1+v^{2}/\lambda^{2}}}\Longrightarrow y\left(u,v\right)=\frac{vu}{\lambda\sqrt{1+v^{2}/\lambda^{2}}} .
+
−
 
+
−
Computing the Jacobian.
+
−
 
+
−
\frac{\partial\left(x,y\right)}{\partial\left(u,v\right)}
+
−
 
+
−
Because \mathbf{X}  and \mathbf{Y}  are statistically independent
+
−
 
+
−
f_{\mathbf{XY}}\left(x,y\right)=f_{\mathbf{X}}\left(x\right)f_{\mathbf{Y}}\left(y\right)=xe^{-x^{2}/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(x\right)\cdot\frac{1}{\sqrt{2\pi}}e^{-y^{2}/2}=\frac{x}{\sqrt{2\pi}}e^{-\left(x^{2}+y^{2}\right)/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(x\right).
+
−
 
+
−
Substituting these quantities, we get
+
−
 
+
−
f_{\mathbf{UV}}\left(u,v\right)
+
−
 
+
−
(b)
+
−
 
+
−
Are \mathbf{U}  and \mathbf{V}  statistically independent? Justify your answer.
+
−
 
+
−
\mathbf{U}  and \mathbf{V}  are statistically independent iff f_{\mathbf{UV}}\left(u,v\right)=f_{\mathbf{U}}\left(u\right)f_{\mathbf{V}}\left(v\right) .
+
−
 
+
−
Now from part (a), we see thatf_{\mathbf{UV}}\left(u,v\right)=c_{1}g_{1}\left(u\right)\cdot c_{2}g_{2}\left(v\right) where g_{1}\left(u\right)=u^{2}e^{-u^{2}/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(u\right)  and g_{2}\left(v\right)=\frac{1}{\left(\lambda^{2}+v^{2}\right)^{\frac{3}{2}}}  with c_{1}  and c_{2}  selected such that f_{\mathbf{U}}\left(u\right)=c_{1}g_{1}\left(u\right)  and f_{\mathbf{V}}\left(v\right)=c_{2}g_{2}\left(v\right)  are both valid pdfs.
+
−
 
+
−
\therefore  \mathbf{U}  and \mathbf{V}  are statistically independent.
+
−
 
+
−
Example. Two jointly distributed random variables (Joint characteristic function)
+
−
 
+
−
Let \mathbf{X}  and \mathbf{Y}  be tweo jointly distributed random variables having joint characteristic function
+
−
 
+
−
\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)=\frac{1}{\left(1-i\omega_{1}\right)\left(1-i\omega_{2}\right)}.
+
−
 
+
−
(a)
+
−
 
+
−
Calculate E\left[\mathbf{X}\right] .
+
−
 
+
−
\Phi_{\mathbf{X}}\left(\omega\right)=\Phi_{\mathbf{XY}}\left(\omega,0\right)=\frac{1}{1-i\omega}=\left(1-i\omega\right)^{-1}
+
−
 
+
−
E\left[\mathbf{X}\right]=\frac{d}{d\left(i\omega\right)}\Phi_{\mathbf{X}}\left(\omega\right)|_{i\omega=0}=(-1)(1-i\omega)^{-2}(-1)|_{i\omega=0}=1
+
−
 
+
−
(b)
+
−
 
+
−
Calculate E\left[\mathbf{Y}\right]
+
−
 
+
−
E\left[\mathbf{Y}\right]=1
+
−
 
+
−
(c)
+
−
 
+
−
Calculate E\left[\mathbf{XY}\right] .
+
−
 
+
−
E\left[\mathbf{XY}\right]=\frac{\partial^{2}}{\partial\left(i\omega_{1}\right)\partial\left(i\omega_{2}\right)}\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)|_{i\omega_{1}=i\omega_{2}=0}=\left(1-i\omega_{1}\right)^{-2}\left(1-i\omega_{2}\right)^{-2}|_{i\omega_{1}=i\omega_{2}=0}=1
+
−
 
+
−
(d)
+
−
 
+
−
Calculate E\left[\mathbf{X}^{j}\mathbf{Y}^{k}\right] .
+
−
 
+
−
E\left[\mathbf{X}^{j}\mathbf{Y}^{k}\right]
+
−
 
+
−
(e)
+
−
 
+
−
Calculate the correlation coefficient r_{\mathbf{XY}}  between \mathbf{X}  and \mathbf{Y} .
+
−
 
+
−
r_{\mathbf{XY}}=\frac{Cov\left(\mathbf{X},\mathbf{Y}\right)}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{1-1\cdot1}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=0.
+
−
 
+
−
Example. Geometric random variable
+
−
 
+
−
Let \mathbf{X}  be a random variable with probability mass function
+
−
 
+
−
p_{\mathbf{X}}\left(k\right)=\alpha\left(1-\alpha\right)^{k-1},k=1,2,3,\cdots
+
−
 
+
−
where 0<\alpha<1 .
+
−
 
+
−
Note
+
−
 
+
−
This is a geometric random variable with success probability \alpha .
+
−
 
+
−
(a) Find the characteristic function of \mathbf{X} .
+
−
 
+
−
\Phi_{\mathbf{X}}\left(\omega\right)
+
−
 
+
−
since \left|e^{i\omega}\left(1-\alpha\right)\right|<1 .
+
−
 
+
−
\because0<1-\alpha<1  and the real term of e^{i\omega}=\cos\omega+i\sin\omega  is \left|\cos\omega\right|<1 .
+
−
 
+
−
(b) Find the mean of \mathbf{X} .
+
−
 
+
−
E\left[\mathbf{X}\right]
+
−
 
+
−
Note
+
−
 
+
−
You can see the other approach to find E\left[\mathbf{X}\right] and Var\left[\mathbf{X}\right]  [CS1GeometricDistribution].
+
−
 
+
−
(c) Find the variance of \mathbf{X} .
+
−
 
+
−
E\left[\mathbf{X}^{2}\right]
+
−
 
+
−
because
+
−
 
+
−
 
+
−
 
+
−
\frac{d}{d\left(i\omega\right)}e^{i\omega2}\left(1-e^{i\omega}\left(1-\alpha\right)\right)^{-2}\left|_{i\omega=0}\right.
+
−
 
+
−
Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=\frac{2-\alpha}{\alpha^{2}}-\frac{1}{\alpha^{2}}=\frac{1-\alpha}{\alpha^{2}}.
+
−
 
+
−
Example. Secquence of binomially distributed random variables
+
−
 
+
−
Let \left\{ \mathbf{X}_{n}\right\} _{n\geq1}  be a sequence of binomially distributed random variables, with the n_{th}  random variable \mathbf{X}_{n}  having pmf
+
−
 
+
−
P_{\mathbf{X}_{n}}(k)=P\left(\left\{ \mathbf{X}_{n}=k\right\} \right)=\left(\begin{array}{c}
+
−
n\\
+
−
k
+
−
\end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}\;,\; k=0,1,\cdots,n,\; p_{n}\in\left(0,1\right). Show that, if the p_{n}  have the property that np_{n}\rightarrow\lambda  as n\rightarrow\infty , where \lambda  is a positive constant, then the sequence \left\{ \mathbf{X}_{n}\right\} _{n\leq1}  converges in distribution to a Poisson random variable \mathbf{X}  with mean \lambda .
+
−
 
+
−
Hint:
+
−
 
+
−
You may find the following fact useful:
+
−
 
+
−
\lim_{n\rightarrow\infty}\left(1+\frac{x}{n}\right)^{n}=e^{x}.
+
−
 
+
−
Solution
+
−
 
+
−
If \mathbf{X}_{n}  converges to \mathbf{X}  in distribution, then F_{\mathbf{X}_{n}}(x)\rightarrow F_{\mathbf{X}}(x)  \forall x\in\mathbf{R} , where F_{\mathbf{X}}(x)  is continuous. This occurs iff \Phi_{\mathbf{X}_{n}}(\omega)\rightarrow\Phi_{\mathbf{X}}(\omega)  \forall x\in\mathbf{R} . We will show that \Phi_{\mathbf{X}_{n}}(\omega)  converges to e^{-\lambda\left(1-e^{i\omega}\right)}  as n\rightarrow\infty , which is the characteristic function of a Poisson random variable with mean \lambda .
+
−
 
+
−
\Phi_{\mathbf{X}_{n}}(\omega)
+
−
 
+
−
Now as n\rightarrow\infty , np_{n}\rightarrow\lambda\Rightarrow p_{n}\rightarrow\frac{\lambda}{n} .
+
−
 
+
−
\lim_{n\rightarrow\infty}\Phi_{\mathbf{X}_{n}}(\omega)=\lim_{n\rightarrow\infty}\left(1+p_{n}\left(e^{i\omega}-1\right)\right)^{n}=\lim_{n\rightarrow\infty}\left(1+\frac{\lambda}{n}\left(e^{i\omega}-1\right)\right)^{n}=e^{\lambda\left(e^{i\omega}-1\right)}=e^{-\lambda\left(1-e^{i\omega}\right)},
+
−
 
+
−
which is the characteristic function of Poisson random variable with mean \lambda .
+
−
 
+
−
c.f.
+
−
 
+
−
The problem 2 of the August 2007 QE [CS1QE2007August] is identical to this example.
+
−
 
+
−
Example. Secquence of exponentially distributed random variables
+
−
 
+
−
Let \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  be a collection of i.i.d.  exponentially distributed random variables, each having mean \mu . Define
+
−
 
+
−
\mathbf{Y}=\max\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\} 
+
−
 
+
−
and
+
−
 
+
−
\mathbf{Z}=\min\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\} .
+
−
 
+
−
(a) Find the pdf of \mathbf{Y} .
+
−
 
+
−
F_{\mathbf{Y}}\left(y\right)
+
−
 
+
−
f_{\mathbf{Y}}(y)
+
−
 
+
−
(b) Find the pdf of \mathbf{Z}
+
−
 
+
−
F_{\mathbf{Z}}(z)
+
−
 
+
−
f_{Z}(z)
+
−
 
+
−
(c) In words, give as complete a description of the random variable \mathbf{Z}  as you can.
+
−
 
+
−
\mathbf{Z}  is an exponetially distributed random variable with mean \frac{\mu}{n} .
+
−
 
+
−
Example. Secquence of uniformly distributed random variables
+
−
 
+
−
Let \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  be n  i.i.d.  jointly distributed random variables, each uniformly distributed on the interval \left[0,1\right] . Define the new random variables \mathbf{W}=\max\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\} 
+
−
 
+
−
and
+
−
 
+
−
\mathbf{Z}=\min\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\} .
+
−
 
+
−
(a) Find the pdf of \mathbf{W} .
+
−
 
+
−
F_{\mathbf{W}}(w)
+
−
 
+
−
where f_{\mathbf{X}}(x)=\mathbf{1}_{\left[0,1\right]}(x)  and F_{X}\left(x\right)=\left\{ \begin{array}{ll}
+
−
0 & ,x<0\\
+
−
x & ,0\leq x<1\\
+
−
1 & ,x\geq1
+
−
\end{array}\right. .
+
−
 
+
−
f_{\mathbf{W}}\left(w\right)
+
−
 
+
−
(b) Find the pdf of \mathbf{Z} .
+
−
 
+
−
F_{\mathbf{Z}}(z)
+
−
 
+
−
f_{\mathbf{Z}}(z)
+
−
 
+
−
(c) Find the mean of \mathbf{W} .
+
−
 
+
−
E\left[\mathbf{W}\right]
+
−
 
+
−
Example. Mean of i.i.d.  random variables
+
−
 
+
−
Let \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  be M  jointly distributed i.i.d.  random variables with mean \mu  and variance \sigma^{2} . Let \mathbf{Y}_{M}=\frac{1}{M}\sum_{n=0}^{M}\mathbf{X}_{n} .
+
−
 
+
−
(a) Find the variance of \mathbf{Y}_{M} .
+
−
 
+
−
Var\left[\mathbf{Y}_{M}\right]=E\left[\mathbf{Y}_{M}^{2}\right]-\left(E\left[\mathbf{Y}_{M}\right]\right)^{2}.
+
−
 
+
−
E\left[\mathbf{Y}_{M}\right]=E\left[\frac{1}{M}\sum_{n=0}^{M}\mathbf{X}_{n}\right]=\frac{1}{M}\sum_{n=0}^{M}E\left[\mathbf{X}_{n}\right]=\frac{1}{M}\cdot M\cdot\mu=\mu.
+
−
 
+
−
E\left[\mathbf{Y}_{M}^{2}\right]=E\left[\frac{1}{M^{2}}\sum_{m=1}^{M}\sum_{n=1}^{M}\mathbf{X}_{m}\mathbf{X}_{n}\right]=\frac{1}{M^{2}}\sum_{m=1}^{M}\sum_{n=1}^{M}E\left[\mathbf{X}_{m}\mathbf{X}_{n}\right].
+
−
 
+
−
Now E\left[\mathbf{X}_{m}\mathbf{X}_{n}\right]=\begin{cases}
+
−
\begin{array}{ll}
+
−
E\left[\mathbf{X}_{m}^{2}\right] & ,m=n\\
+
−
E\left[\mathbf{X}_{m}\right]E\left[\mathbf{X}_{n}\right] & ,m\neq n
+
−
\end{array}\end{cases}  because when m\neq n , \mathbf{X}_{m}  and \mathbf{X}_{n}  are independent \Rightarrow  \mathbf{X}_{m}  and \mathbf{X}_{n}  are uncorrelated.
+
−
 
+
−
E\left[\mathbf{Y}_{M}^{2}\right]=\frac{1}{M^{2}}\left[M\left(\mu^{2}+\sigma^{2}\right)+M\left(M-1\right)\mu^{2}\right]=\frac{\left(\mu^{2}+\sigma^{2}\right)+\left(M-1\right)\mu^{2}}{M}=\frac{M\mu^{2}+\sigma^{2}}{M}.
+
−
 
+
−
Var\left[\mathbf{Y}_{M}\right]=\frac{M\mu^{2}+\sigma^{2}-M\mu^{2}}{M}=\frac{\sigma^{2}}{M}.
+
−
 
+
−
(b) Now assume that the \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  are identically distributed with with mean \mu  and variance \sigma^{2} , but they are only correlated rather than independent. Find the variance of \mathbf{Y}_{M} .
+
−
 
+
−
Again, Var\left[\mathbf{Y}_{M}\right]=\frac{\sigma^{2}}{M} , because only uncorrelatedness was used in part (a).
+
−
 
+
−
Example. A sum of a random number of i.i.d.  Gaussians
+
−
 
+
−
Let \left\{ \mathbf{X}_{n}\right\}  be a sequence of i.i.d.  Gaussian random variables, each having characteristic function
+
−
 
+
−
\Phi_{\mathbf{X}}\left(\omega\right)=e^{i\mu\omega}e^{-\frac{1}{2}\sigma^{2}\omega^{2}}. Let \mathbf{N}  be a Poisson random variable with pmf
+
−
 
+
−
p(n)=\frac{e^{-\lambda}\lambda^{n}}{n!},\; n=0,1,2,\cdots,\;\lambda>0, and assume \mathbf{N}  is statistically independent of \left\{ \mathbf{X}_{n}\right\}  . Define a new random variable
+
−
 
+
−
\mathbf{Y}=\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{N}.
+
−
 
+
−
Note
+
−
 
+
−
If \mathbf{N}=0 , then \mathbf{Y}=0 .
+
−
 
+
−
(a) Find the mean of \mathbf{Y} .
+
−
 
+
−
• Probability generating function of \mathbf{N}  is P_{\mathbf{N}}\left(z\right)=E\left[z^{\mathbf{N}}\right]=\sum_{n=0}^{\infty}z^{n}\frac{e^{-\lambda}\lambda^{n}}{n!}=e^{-\lambda}\sum_{n=0}^{\infty}\frac{\left(z\lambda\right)^{n}}{n!}=e^{-\lambda}e^{z\lambda}=e^{-\lambda\left(1-z\right)}.
+
−
 
+
−
• The characteristic function of \mathbf{Y}  is \Phi_{\mathbf{Y}}\left(\omega\right)=P_{\mathbf{N}}\left(z\right)\Bigl|_{z=\Phi_{\mathbf{X}}\left(\omega\right)}=e^{-\lambda\left(1-z\right)}\Bigl|_{z=e^{i\mu\omega}e^{-\frac{1}{2}\sigma^{2}\omega^{2}}}=e^{-\lambda\left(1-e^{i\mu\omega}e^{-\frac{1}{2}\sigma^{2}\omega^{2}}\right)}.
+
−
 
+
−
• Now, we can get the mean of \mathbf{Y}  using the characteristic function. E\left[\mathbf{Y}\right]
+

Latest revision as of 11:57, 30 November 2010

5 Exams

Problem Examples

Back to ECE600

Retrieved from "https://www.projectrhea.org/rhea/index.php?title=ECE_600_Exams&oldid=42975"
Shortcuts
Help Main Wiki Page Random Page Special Pages Log in
Search

Alumni Liaison

Basic linear algebra uncovers and clarifies very important geometry and algebra.

Dr. Paul Garrett