(New page: == Example 1 == <math> \mathbf{X}\sim\mathcal{N}\left(0,\sigma_{\mathbf{X}}^{2}\right),\;\mathbf{N}\sim\mathcal{N}\left(0,\sigma_{\mathbf{N}}^{2}\right),\;\mathbf{Y}=\mathbf{X}+\mathbf{N}...) |
|||
| Line 1: | Line 1: | ||
| − | == Example | + | == Example. Addition of two independent Gaussian random variables == |
<math> | <math> | ||
\mathbf{X}\sim\mathcal{N}\left(0,\sigma_{\mathbf{X}}^{2}\right),\;\mathbf{N}\sim\mathcal{N}\left(0,\sigma_{\mathbf{N}}^{2}\right),\;\mathbf{Y}=\mathbf{X}+\mathbf{N}. | \mathbf{X}\sim\mathcal{N}\left(0,\sigma_{\mathbf{X}}^{2}\right),\;\mathbf{N}\sim\mathcal{N}\left(0,\sigma_{\mathbf{N}}^{2}\right),\;\mathbf{Y}=\mathbf{X}+\mathbf{N}. | ||
| − | </math> | + | </math> |
| − | === (a) === | + | === (a) === |
| − | Correlation coefficient between <math>\mathbf{X}</math> and <math>\mathbf{Y}</math>. | + | |
| + | Correlation coefficient between <math>\mathbf{X}</math> and <math>\mathbf{Y}</math>. | ||
<math> | <math> | ||
\sigma_{\mathbf{Y}}=\sqrt{\sigma_{\mathbf{X}}^{2}+2r_{\mathbf{XN}}\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}+\sigma_{\mathbf{N}}^{2}}=\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}} | \sigma_{\mathbf{Y}}=\sqrt{\sigma_{\mathbf{X}}^{2}+2r_{\mathbf{XN}}\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}+\sigma_{\mathbf{N}}^{2}}=\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}} | ||
| − | </math> | + | </math> |
| − | because <math>\mathbf{X}</math> and <math>\mathbf{N}</math> are independnet <math>\Longrightarrow</math> uncorrelated <math>\Longrightarrow r_{\mathbf{XN}}=0</math>. | + | because <math>\mathbf{X}</math> and <math>\mathbf{N}</math> are independnet <math>\Longrightarrow</math> uncorrelated <math>\Longrightarrow r_{\mathbf{XN}}=0</math>. |
<math> | <math> | ||
r_{\mathbf{XY}} =\frac{\textrm{cov}(\mathbf{X},\mathbf{Y})}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}\left(\mathbf{X}+\mathbf{N}\right)\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}^{2}\right]+E\left[\mathbf{XN}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}} | r_{\mathbf{XY}} =\frac{\textrm{cov}(\mathbf{X},\mathbf{Y})}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}\left(\mathbf{X}+\mathbf{N}\right)\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}^{2}\right]+E\left[\mathbf{XN}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}} | ||
| − | </math> | + | </math> |
<math> | <math> | ||
=\frac{\sigma_{\mathbf{X}}^{2}+E\left[\mathbf{X}\right]E\left[\mathbf{N}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\qquad\because E\left[\mathbf{X}\right]=0. | =\frac{\sigma_{\mathbf{X}}^{2}+E\left[\mathbf{X}\right]E\left[\mathbf{N}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\qquad\because E\left[\mathbf{X}\right]=0. | ||
| − | </math> | + | </math> |
| − | === (b) === | + | === (b) === |
| − | Conditional pmf of <math>\mathbf{X}</math> conditioned on the event <math>\left\{ \mathbf{Y}=y\right\}</math>. | + | |
| + | Conditional pmf of <math>\mathbf{X}</math> conditioned on the event <math>\left\{ \mathbf{Y}=y\right\}</math>. | ||
<math> | <math> | ||
f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right) = \frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{Y}}(y)}=\frac{\frac{1}{2\pi\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} }{\frac{1}{\sqrt{2\pi}\sigma_{Y}}\exp\left\{ \frac{-y^{2}}{2\sigma_{Y}^{2}}\right\} } | f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right) = \frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{Y}}(y)}=\frac{\frac{1}{2\pi\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} }{\frac{1}{\sqrt{2\pi}\sigma_{Y}}\exp\left\{ \frac{-y^{2}}{2\sigma_{Y}^{2}}\right\} } | ||
| − | </math> | + | </math> |
<math> | <math> | ||
=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}-\frac{\left(1-r^{2}\right)y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} | =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}-\frac{\left(1-r^{2}\right)y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} | ||
| − | </math> | + | </math> |
<math> | <math> | ||
=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{r^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} | =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{r^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} | ||
| − | </math> | + | </math> |
<math> | <math> | ||
=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left[x^{2}-\frac{2r\sigma_{\mathbf{X}}xy}{\sigma_{\mathbf{Y}}}+\frac{r^{2}\sigma_{\mathbf{X}}^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} | =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left[x^{2}-\frac{2r\sigma_{\mathbf{X}}xy}{\sigma_{\mathbf{Y}}}+\frac{r^{2}\sigma_{\mathbf{X}}^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} | ||
| − | </math> | + | </math> |
<math> | <math> | ||
=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left(x-\frac{r\sigma_{\mathbf{X}}y}{\sigma_{\mathbf{Y}}}\right)^{2}\right\} | =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left(x-\frac{r\sigma_{\mathbf{X}}y}{\sigma_{\mathbf{Y}}}\right)^{2}\right\} | ||
| − | </math> | + | </math> |
Noting that | Noting that | ||
| Line 49: | Line 51: | ||
<math> | <math> | ||
\sqrt{1-r^{2}}=\sigma_{\mathbf{X}}\sqrt{1-\left(\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}\right)^{2}}=\sqrt{1-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}=\sqrt{\frac{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}-\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}} | \sqrt{1-r^{2}}=\sigma_{\mathbf{X}}\sqrt{1-\left(\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}\right)^{2}}=\sqrt{1-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}=\sqrt{\frac{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}-\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}} | ||
| − | </math> | + | </math> |
and | and | ||
| Line 55: | Line 57: | ||
<math> | <math> | ||
r\cdot\frac{\sigma_{\mathbf{X}}}{\sigma_{\mathbf{Y}}}=\frac{\sigma_{X}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\cdot\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}. | r\cdot\frac{\sigma_{\mathbf{X}}}{\sigma_{\mathbf{Y}}}=\frac{\sigma_{X}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\cdot\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}. | ||
| − | </math> | + | </math> |
<math> | <math> | ||
\therefore f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{1}{\sqrt{2\pi}\cdot\frac{\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\mathbf{\sigma}_{\mathbf{N}}^{\mathbf{2}}}}}\exp\left\{ \frac{-1}{2\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\left(x-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y\right)^{2}\right\} | \therefore f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{1}{\sqrt{2\pi}\cdot\frac{\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\mathbf{\sigma}_{\mathbf{N}}^{\mathbf{2}}}}}\exp\left\{ \frac{-1}{2\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\left(x-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y\right)^{2}\right\} | ||
| − | </math> | + | </math> |
| − | === (c) === | + | === (c) === |
| − | + | ||
| − | + | What kind of pdf is the pdf you determined in part (b)? What is the mean and variance of a random variable with this pdf? | |
| − | + | This is a Gaussian pdf with mean <math>\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y</math> and variance <math>\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}</math>. | |
| − | + | ||
| − | The minimum mean-square error estimate of <math>\mathbf{X}</math> given <math>\mathbf{Y}=y</math> is | + | === (d) === |
| + | |||
| + | What is the minimum mean-square estimate of <math>\mathbf{X}</math> given that <math>\left\{ \mathbf{Y}=y\right\}</math>? | ||
| + | |||
| + | The minimum mean-square error estimate of <math>\mathbf{X}</math> given <math>\mathbf{Y}=y</math> is | ||
<math> | <math> | ||
\hat{x}_{MMS}(y)=E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\int_{-\infty}^{\infty}x\cdot f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)dx=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y | \hat{x}_{MMS}(y)=E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\int_{-\infty}^{\infty}x\cdot f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)dx=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y | ||
| − | </math> | + | </math> |
| − | from part (b). | + | from part (b). |
| + | <br> | ||
| − | === (e) === | + | === (e) === |
| − | What is the maximum a posteriori estimate of <math>\mathbf{X}</math> given that <math>\left\{ \mathbf{Y}=y\right\}</math>? | + | |
| + | What is the maximum a posteriori estimate of <math>\mathbf{X}</math> given that <math>\left\{ \mathbf{Y}=y\right\}</math>? | ||
<math> | <math> | ||
\hat{x}_{MAP}(y)=\arg\max_{x\in\mathbf{R}}\left\{ f_{\mathbf{X}}\left(x|\left\{ Y=y\right\} \right)\right\} =\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y | \hat{x}_{MAP}(y)=\arg\max_{x\in\mathbf{R}}\left\{ f_{\mathbf{X}}\left(x|\left\{ Y=y\right\} \right)\right\} =\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y | ||
| − | </math> | + | </math> |
| + | |||
| + | as a Gaussian pdf takes on its maximum value at its mean. | ||
| − | + | === (f) === | |
| − | + | Given that I observe <math>\mathbf{Y}=y</math>, what is <math>E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]</math>? | |
| − | Given that I observe <math>\mathbf{Y}=y</math>, what is <math>E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]</math>? | + | |
<math> | <math> | ||
E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y | E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y | ||
| − | </math> | + | </math> |
from part (d). | from part (d). | ||
Latest revision as of 04:20, 15 November 2010
Contents
Example. Addition of two independent Gaussian random variables
$ \mathbf{X}\sim\mathcal{N}\left(0,\sigma_{\mathbf{X}}^{2}\right),\;\mathbf{N}\sim\mathcal{N}\left(0,\sigma_{\mathbf{N}}^{2}\right),\;\mathbf{Y}=\mathbf{X}+\mathbf{N}. $
(a)
Correlation coefficient between $ \mathbf{X} $ and $ \mathbf{Y} $.
$ \sigma_{\mathbf{Y}}=\sqrt{\sigma_{\mathbf{X}}^{2}+2r_{\mathbf{XN}}\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}+\sigma_{\mathbf{N}}^{2}}=\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}} $
because $ \mathbf{X} $ and $ \mathbf{N} $ are independnet $ \Longrightarrow $ uncorrelated $ \Longrightarrow r_{\mathbf{XN}}=0 $.
$ r_{\mathbf{XY}} =\frac{\textrm{cov}(\mathbf{X},\mathbf{Y})}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}\left(\mathbf{X}+\mathbf{N}\right)\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}^{2}\right]+E\left[\mathbf{XN}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}} $
$ =\frac{\sigma_{\mathbf{X}}^{2}+E\left[\mathbf{X}\right]E\left[\mathbf{N}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\qquad\because E\left[\mathbf{X}\right]=0. $
(b)
Conditional pmf of $ \mathbf{X} $ conditioned on the event $ \left\{ \mathbf{Y}=y\right\} $.
$ f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right) = \frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{Y}}(y)}=\frac{\frac{1}{2\pi\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} }{\frac{1}{\sqrt{2\pi}\sigma_{Y}}\exp\left\{ \frac{-y^{2}}{2\sigma_{Y}^{2}}\right\} } $
$ =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}-\frac{\left(1-r^{2}\right)y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} $
$ =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{r^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} $
$ =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left[x^{2}-\frac{2r\sigma_{\mathbf{X}}xy}{\sigma_{\mathbf{Y}}}+\frac{r^{2}\sigma_{\mathbf{X}}^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} $
$ =\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left(x-\frac{r\sigma_{\mathbf{X}}y}{\sigma_{\mathbf{Y}}}\right)^{2}\right\} $
Noting that
$ \sqrt{1-r^{2}}=\sigma_{\mathbf{X}}\sqrt{1-\left(\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}\right)^{2}}=\sqrt{1-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}=\sqrt{\frac{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}-\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}} $
and
$ r\cdot\frac{\sigma_{\mathbf{X}}}{\sigma_{\mathbf{Y}}}=\frac{\sigma_{X}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\cdot\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}. $
$ \therefore f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{1}{\sqrt{2\pi}\cdot\frac{\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\mathbf{\sigma}_{\mathbf{N}}^{\mathbf{2}}}}}\exp\left\{ \frac{-1}{2\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\left(x-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y\right)^{2}\right\} $
(c)
What kind of pdf is the pdf you determined in part (b)? What is the mean and variance of a random variable with this pdf?
This is a Gaussian pdf with mean $ \frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y $ and variance $ \frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}} $.
(d)
What is the minimum mean-square estimate of $ \mathbf{X} $ given that $ \left\{ \mathbf{Y}=y\right\} $?
The minimum mean-square error estimate of $ \mathbf{X} $ given $ \mathbf{Y}=y $ is
$ \hat{x}_{MMS}(y)=E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\int_{-\infty}^{\infty}x\cdot f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)dx=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y $
from part (b).
(e)
What is the maximum a posteriori estimate of $ \mathbf{X} $ given that $ \left\{ \mathbf{Y}=y\right\} $?
$ \hat{x}_{MAP}(y)=\arg\max_{x\in\mathbf{R}}\left\{ f_{\mathbf{X}}\left(x|\left\{ Y=y\right\} \right)\right\} =\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y $
as a Gaussian pdf takes on its maximum value at its mean.
(f)
Given that I observe $ \mathbf{Y}=y $, what is $ E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right] $?
$ E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y $
from part (d).
