(Maximum Likelihood Estimation (ML))
Line 1: Line 1:
 
==Maximum Likelihood Estimation (ML)==
 
==Maximum Likelihood Estimation (ML)==
<math>\hat a_{ML} = \text{max}_a ( f_{X}(x_i;a))</math> continuous
+
:<math>\hat a_{ML} = \overset{max}{a}  f_{X}(x_i;a)</math> continuous
  
<math>\hat a_{ML} = \text{max}_a ( Pr(x_i;a))</math> discrete
+
:<math>\hat a_{ML} = \overset{max}{a}  Pr(x_i;a)</math> discrete
  
 
==Maximum A-Posteriori Estimation (MAP)==
 
==Maximum A-Posteriori Estimation (MAP)==

Revision as of 15:51, 13 December 2008

Maximum Likelihood Estimation (ML)

$ \hat a_{ML} = \overset{max}{a} f_{X}(x_i;a) $ continuous
$ \hat a_{ML} = \overset{max}{a} Pr(x_i;a) $ discrete

Maximum A-Posteriori Estimation (MAP)

$ \hat \theta_{MAP}(x) = \text{arg max}_\theta P_{X|\theta}(x|\theta)P_ {\theta}(\theta) $

$ \hat \theta_{MAP}(x) = \text{arg max}_\theta f_{X|\theta}(x|\theta)P_ {\theta}(\theta) $

Minimum Mean-Square Estimation (MMSE)

$ \hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x) $

Law Of Iterated Expectation

Unconditional Expectaion--$ \ E[X] = E[E[x|\theta]] $

--Umang 16:10, 13 December 2008 (UTC)umang


Mean square error :

$ MSE = E[(\theta - \hat \theta(x))^2] $

Linear Minimum Mean-Square Estimation (LMMSE)

$ \hat{y}_{\rm LMMSE}(x) = E[\theta]+\frac{COV(x,\theta)}{Var(x)}*(x-E[x]) $

Law of Iterated Expectation: E[E[X|Y]]=E[X]

Hypothesis Testing: ML Rule

Given a value of X, we will say H1 is true if X is in region R, else will will say H0 is true.

Type I error

Say H1 when truth is H0. Probability of this is: $ Pr(\mbox{Say } H_1|H_0) = Pr(x \in R|\theta_0) $

Type II error

Say H0 when truth is H1. Probability of this is: $ Pr(\mbox{Say }H_0|H_1) = Pr(x \in R^C|\theta_1) $

Hypothesis Testing: MAP Rule

$ \mbox{Overall P(err)} = P_{\theta}(\theta_{0})Pr\Big[\mbox{Say }H_{1}|H_{0}\Big] +P_{\theta}(\theta_{1})Pr\Big[\mbox{Say }H_{0}|H_{1}\Big] $

Likelihood Ratio TEST

How to find a good rule? --Khosla 16:44, 13 December 2008 (UTC)

$ \ L(x) = \frac{P_{\rm X|\theta} (x|\theta_1)}{P_{\rm X|\theta} (x|\theta_1)} $

Choose threshold (T),

$ \mbox{Say } \begin{cases} H_{1}; \mbox{ if } L(x) > T\\ H_{0}; \mbox{ if } L(x) < T \end{cases} $

so ML Rule is an LRT with T = 1

as T increases Type I Error Increases

as T increases Type II Error Decreases

& Vice Versa

so ML Rule is an Likelihood Ratio Test with T = 1

Alumni Liaison

Ph.D. on Applied Mathematics in Aug 2007. Involved on applications of image super-resolution to electron microscopy

Francisco Blanco-Silva