| Line 14: | Line 14: | ||
<math>\hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x)</math> | <math>\hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x)</math> | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
== '''Law Of Iterated Expectation''' == | == '''Law Of Iterated Expectation''' == | ||
| Line 72: | Line 49: | ||
<math>\mbox{Overall P(err)} = P_{\theta}(\theta_{0})Pr\Big[\mbox{Say }H_{1}|H_{0}\Big] +P_{\theta}(\theta_{1})Pr\Big[\mbox{Say }H_{0}|H_{1}\Big] </math> | <math>\mbox{Overall P(err)} = P_{\theta}(\theta_{0})Pr\Big[\mbox{Say }H_{1}|H_{0}\Big] +P_{\theta}(\theta_{1})Pr\Big[\mbox{Say }H_{0}|H_{1}\Big] </math> | ||
| + | |||
| + | ==Likelihood Ratio TEST== | ||
| + | |||
| + | '''''How to find a good rule?''''' | ||
| + | --[[User:Khosla|Khosla]] 16:44, 13 December 2008 (UTC) | ||
| + | |||
| + | <math>\ L(x) = \frac{P_{\rm X|\theta} (x|\theta_1)}{P_{\rm X|\theta} (x|\theta_1)} </math> | ||
| + | |||
| + | Choose threshold (T), | ||
| + | |||
| + | <math>\mbox{Say } | ||
| + | \begin{cases} | ||
| + | H_{1}; \mbox{ if } L(x) > T\\ | ||
| + | H_{0}; \mbox{ if } L(x) < T | ||
| + | \end{cases}</math> | ||
| + | |||
| + | so ML Rule is an LRT with T = 1 | ||
| + | |||
| + | as T increases Type I Error Increases | ||
| + | |||
| + | as T increases Type II Error Decreases | ||
| + | |||
| + | & Vice Versa | ||
| + | |||
| + | so ML Rule is an Likelihood Ratio Test with T = 1 | ||
Revision as of 14:46, 13 December 2008
Contents
- 1 Maximum Likelihood Estimation (ML)
- 2 Maximum A-Posteriori Estimation (MAP)
- 3 Minimum Mean-Square Estimation (MMSE)
- 4 Law Of Iterated Expectation
- 5 Mean square error :
- 6 Linear Minimum Mean-Square Estimation (LMMSE)
- 7 Hypothesis Testing: ML Rule
- 8 Hypothesis Testing: MAP Rule
- 9 Likelihood Ratio TEST
Maximum Likelihood Estimation (ML)
$ \hat a_{ML} = \text{max}_a ( f_{X}(x_i;a)) $ continuous
$ \hat a_{ML} = \text{max}_a ( Pr(x_i;a)) $ discrete
Maximum A-Posteriori Estimation (MAP)
$ \hat \theta_{MAP}(x) = \text{arg max}_\theta P_{X|\theta}(x|\theta)P_ {\theta}(\theta) $
$ \hat \theta_{MAP}(x) = \text{arg max}_\theta f_{X|\theta}(x|\theta)P_ {\theta}(\theta) $
Minimum Mean-Square Estimation (MMSE)
$ \hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x) $
Law Of Iterated Expectation
Unconditional Expectaion--$ \ E[X] = E[E[x|\theta]] $
--Umang 16:10, 13 December 2008 (UTC)umang
Mean square error :
$ MSE = E[(\theta - \hat \theta(x))^2] $
Linear Minimum Mean-Square Estimation (LMMSE)
$ \hat{y}_{\rm LMMSE}(x) = E[\theta]+\frac{COV(x,\theta)}{Var(x)}*(x-E[x]) $
Law of Iterated Expectation: E[E[X|Y]]=E[X]
Hypothesis Testing: ML Rule
Given a value of X, we will say H1 is true if X is in region R, else will will say H0 is true.
Type I error
Say H1 when truth is H0. Probability of this is: $ Pr(\mbox{Say } H_1|H_0) = Pr(x \in R|\theta_0) $
Type II error
Say H0 when truth is H1. Probability of this is: $ Pr(\mbox{Say }H_0|H_1) = Pr(x \in R^C|\theta_1) $
Hypothesis Testing: MAP Rule
$ \mbox{Overall P(err)} = P_{\theta}(\theta_{0})Pr\Big[\mbox{Say }H_{1}|H_{0}\Big] +P_{\theta}(\theta_{1})Pr\Big[\mbox{Say }H_{0}|H_{1}\Big] $
Likelihood Ratio TEST
How to find a good rule? --Khosla 16:44, 13 December 2008 (UTC)
$ \ L(x) = \frac{P_{\rm X|\theta} (x|\theta_1)}{P_{\rm X|\theta} (x|\theta_1)} $
Choose threshold (T),
$ \mbox{Say } \begin{cases} H_{1}; \mbox{ if } L(x) > T\\ H_{0}; \mbox{ if } L(x) < T \end{cases} $
so ML Rule is an LRT with T = 1
as T increases Type I Error Increases
as T increases Type II Error Decreases
& Vice Versa
so ML Rule is an Likelihood Ratio Test with T = 1
