| (One intermediate revision by one other user not shown) | |||
| Line 2: | Line 2: | ||
From the [[ECE_600_Prerequisites|ECE600 Pre-requisites notes]] of [[user:han84|Sangchun Han]], [[ECE]] PhD student. | From the [[ECE_600_Prerequisites|ECE600 Pre-requisites notes]] of [[user:han84|Sangchun Han]], [[ECE]] PhD student. | ||
---- | ---- | ||
| − | Let <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> be two jointly-distributed RVs, suppose we want to estimate the value of <math>\mathbf{Y}</math> given the value of <math>\mathbf{X}</math> (i.e. given that we observe<math>\left\{ \mathbf{X}=x\right\}</math> ). What is the “best” estimate of <math>\mathbf{Y}</math> ? One commonly used error criterion is square-error. The goal then becomes to minimize the mean-square error. We wish to find a function <math>c\left(x\right)</math> to estimate <math>\mathbf{Y}</math> given that <math>\mathbf{X}=x</math> such that <math>\epsilon=E\left[\left(\mathbf{Y}-c\left(\mathbf{X}\right)\right)^{2}\right]</math> is minimized. | + | Let <math class="inline">\mathbf{X}</math> and <math class="inline">\mathbf{Y}</math> be two jointly-distributed RVs, suppose we want to estimate the value of <math class="inline">\mathbf{Y}</math> given the value of <math class="inline">\mathbf{X}</math> (i.e. given that we observe<math class="inline">\left\{ \mathbf{X}=x\right\}</math> ). What is the “best” estimate of <math class="inline">\mathbf{Y}</math> ? One commonly used error criterion is square-error. The goal then becomes to minimize the mean-square error. We wish to find a function <math class="inline">c\left(x\right)</math> to estimate <math class="inline">\mathbf{Y}</math> given that <math class="inline">\mathbf{X}=x</math> such that <math class="inline">\epsilon=E\left[\left(\mathbf{Y}-c\left(\mathbf{X}\right)\right)^{2}\right]</math> is minimized. |
Claim | Claim | ||
| − | The mean-square error is minimized by the function <math>c\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right]</math>. | + | The mean-square error is minimized by the function <math class="inline">c\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right]</math>. |
We will use the following notation. | We will use the following notation. | ||
| − | <math>\hat{y}_{MMS}\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right]</math> | + | <math class="inline">\hat{y}_{MMS}\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right]</math> |
| − | <math>\hat{x}_{MMS}\left(y\right)=E\left[\mathbf{X}|\mathbf{Y}=y\right]</math> | + | <math class="inline">\hat{x}_{MMS}\left(y\right)=E\left[\mathbf{X}|\mathbf{Y}=y\right]</math> |
Maximum Aposteriori Probability estimator | Maximum Aposteriori Probability estimator | ||
| − | <math>\hat{y}_{MAP}\left(x\right)=\arg\max_{y}\left\{ f_{\mathbf{Y}}\left(y|x\right)\right\}</math> | + | <math class="inline">\hat{y}_{MAP}\left(x\right)=\arg\max_{y}\left\{ f_{\mathbf{Y}}\left(y|x\right)\right\}</math> |
| − | <math>\hat{x}_{MAP}\left(y\right)=\arg\max_{x}\left\{ f_{\mathbf{X}}\left(x|y\right)\right\}</math> | + | <math class="inline">\hat{x}_{MAP}\left(y\right)=\arg\max_{x}\left\{ f_{\mathbf{X}}\left(x|y\right)\right\}</math> |
| + | ---- | ||
| + | [[ECE600|Back to ECE600]] | ||
| + | |||
| + | [[ECE 600 Prerequisites|Back to ECE 600 Prerequisites]] | ||
Latest revision as of 11:33, 30 November 2010
1.12 Minimum Mean-Square Error Estimation
From the ECE600 Pre-requisites notes of Sangchun Han, ECE PhD student.
Let $ \mathbf{X} $ and $ \mathbf{Y} $ be two jointly-distributed RVs, suppose we want to estimate the value of $ \mathbf{Y} $ given the value of $ \mathbf{X} $ (i.e. given that we observe$ \left\{ \mathbf{X}=x\right\} $ ). What is the “best” estimate of $ \mathbf{Y} $ ? One commonly used error criterion is square-error. The goal then becomes to minimize the mean-square error. We wish to find a function $ c\left(x\right) $ to estimate $ \mathbf{Y} $ given that $ \mathbf{X}=x $ such that $ \epsilon=E\left[\left(\mathbf{Y}-c\left(\mathbf{X}\right)\right)^{2}\right] $ is minimized.
Claim
The mean-square error is minimized by the function $ c\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right] $.
We will use the following notation.
$ \hat{y}_{MMS}\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right] $
$ \hat{x}_{MMS}\left(y\right)=E\left[\mathbf{X}|\mathbf{Y}=y\right] $
Maximum Aposteriori Probability estimator
$ \hat{y}_{MAP}\left(x\right)=\arg\max_{y}\left\{ f_{\mathbf{Y}}\left(y|x\right)\right\} $
$ \hat{x}_{MAP}\left(y\right)=\arg\max_{x}\left\{ f_{\mathbf{X}}\left(x|y\right)\right\} $
