(New page: ='''1.12 Minimum Mean-Square Error Estimation'''= Let <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> be two jointly-distributed RVs, suppose we want to estimate the value of <math>...) |
|||
| Line 1: | Line 1: | ||
='''1.12 Minimum Mean-Square Error Estimation'''= | ='''1.12 Minimum Mean-Square Error Estimation'''= | ||
| − | + | From the [[ECE_600_Prerequisites|ECE600 Pre-requisites notes]] of [[user:han84|Sangchun Han]], [[ECE]] PhD student. | |
| + | ---- | ||
Let <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> be two jointly-distributed RVs, suppose we want to estimate the value of <math>\mathbf{Y}</math> given the value of <math>\mathbf{X}</math> (i.e. given that we observe<math>\left\{ \mathbf{X}=x\right\}</math> ). What is the “best” estimate of <math>\mathbf{Y}</math> ? One commonly used error criterion is square-error. The goal then becomes to minimize the mean-square error. We wish to find a function <math>c\left(x\right)</math> to estimate <math>\mathbf{Y}</math> given that <math>\mathbf{X}=x</math> such that <math>\epsilon=E\left[\left(\mathbf{Y}-c\left(\mathbf{X}\right)\right)^{2}\right]</math> is minimized. | Let <math>\mathbf{X}</math> and <math>\mathbf{Y}</math> be two jointly-distributed RVs, suppose we want to estimate the value of <math>\mathbf{Y}</math> given the value of <math>\mathbf{X}</math> (i.e. given that we observe<math>\left\{ \mathbf{X}=x\right\}</math> ). What is the “best” estimate of <math>\mathbf{Y}</math> ? One commonly used error criterion is square-error. The goal then becomes to minimize the mean-square error. We wish to find a function <math>c\left(x\right)</math> to estimate <math>\mathbf{Y}</math> given that <math>\mathbf{X}=x</math> such that <math>\epsilon=E\left[\left(\mathbf{Y}-c\left(\mathbf{X}\right)\right)^{2}\right]</math> is minimized. | ||
Revision as of 11:13, 17 November 2010
1.12 Minimum Mean-Square Error Estimation
From the ECE600 Pre-requisites notes of Sangchun Han, ECE PhD student.
Let $ \mathbf{X} $ and $ \mathbf{Y} $ be two jointly-distributed RVs, suppose we want to estimate the value of $ \mathbf{Y} $ given the value of $ \mathbf{X} $ (i.e. given that we observe$ \left\{ \mathbf{X}=x\right\} $ ). What is the “best” estimate of $ \mathbf{Y} $ ? One commonly used error criterion is square-error. The goal then becomes to minimize the mean-square error. We wish to find a function $ c\left(x\right) $ to estimate $ \mathbf{Y} $ given that $ \mathbf{X}=x $ such that $ \epsilon=E\left[\left(\mathbf{Y}-c\left(\mathbf{X}\right)\right)^{2}\right] $ is minimized.
Claim
The mean-square error is minimized by the function $ c\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right] $.
We will use the following notation.
$ \hat{y}_{MMS}\left(x\right)=E\left[\mathbf{Y}|\mathbf{X}=x\right] $
$ \hat{x}_{MMS}\left(y\right)=E\left[\mathbf{X}|\mathbf{Y}=y\right] $
Maximum Aposteriori Probability estimator
$ \hat{y}_{MAP}\left(x\right)=\arg\max_{y}\left\{ f_{\mathbf{Y}}\left(y|x\right)\right\} $
$ \hat{x}_{MAP}\left(y\right)=\arg\max_{x}\left\{ f_{\mathbf{X}}\left(x|y\right)\right\} $
