| Line 1: | Line 1: | ||
| − | |||
==Maximum Likelihood Estimation (ML)== | ==Maximum Likelihood Estimation (ML)== | ||
| Line 45: | Line 44: | ||
Say H0 when truth is H1. Probability of this is: Pr(Say H0|H1) = Pr(X is NOT in R|theta1) | Say H0 when truth is H1. Probability of this is: Pr(Say H0|H1) = Pr(X is NOT in R|theta1) | ||
| + | |||
| + | == How to find a good rule? | ||
| + | Likelihood Ratio TEST == | ||
| + | |||
| + | <math>\ L(x) = P_(X|\theta) (x|\theta1) / P_(X|\theta) (x|\theta1) | ||
==Hypothesis Testing: MAP Rule== | ==Hypothesis Testing: MAP Rule== | ||
Overall P(err) = <math>P_{\theta}(\theta_{0})Pr[Say H_{1}|H_{0}]+P_{\theta}(\theta_{1})Pr[Say H_{0}|H_{1}]</math> | Overall P(err) = <math>P_{\theta}(\theta_{0})Pr[Say H_{1}|H_{0}]+P_{\theta}(\theta_{1})Pr[Say H_{0}|H_{1}]</math> | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Revision as of 12:26, 13 December 2008
Contents
Maximum Likelihood Estimation (ML)
$ \hat a_{ML} = \text{max}_a ( f_{X}(x_i;a)) $ continuous
$ \hat a_{ML} = \text{max}_a ( Pr(x_i;a)) $ discrete
Maximum A-Posteriori Estimation (MAP)
$ \hat \theta_{MAP}(x) = \text{arg max}_\theta P_{X|\theta}(x|\theta)P_ {\theta}(\theta) $
$ \hat \theta_{MAP}(x) = \text{arg max}_\theta f_{X|\theta}(x|\theta)P_ {\theta}(\theta) $
Minimum Mean-Square Estimation (MMSE)
$ \hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x) $
Law Of Iterated Expectation
Unconditional Expectaion--E[X] = E{E[x|theta]}--Umang 16:10, 13 December 2008 (UTC)umang
Mean square error :
Headline text
$ MSE = E[(\theta - \hat \theta(x))^2] $
Linear Minimum Mean-Square Estimation (LMMSE)
$ \hat{y}_{\rm LMMSE}(x) = E[\theta]+\frac{COV(x,\theta)}{Var(x)}*(x-E[x]) $
Hypothesis Testing: ML Rule
Given a value of X, we will say H1 is true if X is in region R, else will will say H0 is true.
Type I error
Say H1 when truth is H0. Probability of this is: Pr(Say H1|H0) = Pr(X is in R|theta0)
Type II error
Say H0 when truth is H1. Probability of this is: Pr(Say H0|H1) = Pr(X is NOT in R|theta1)
== How to find a good rule? Likelihood Ratio TEST ==
$ \ L(x) = P_(X|\theta) (x|\theta1) / P_(X|\theta) (x|\theta1) ==Hypothesis Testing: MAP Rule== Overall P(err) = <math>P_{\theta}(\theta_{0})Pr[Say H_{1}|H_{0}]+P_{\theta}(\theta_{1})Pr[Say H_{0}|H_{1}] $
