(Linear Minimum Mean-Square Estimation (LMMSE))
Line 15: Line 15:
 
<math>\hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x)</math>
 
<math>\hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x)</math>
  
 +
==Likelihood Ratio TEST==
  
 +
'''''How to find a good rule?'''''
 +
--[[User:Khosla|Khosla]] 16:44, 13 December 2008 (UTC)
 +
 +
<math>\ L(x) = P_{\rm X|\theta} (x|\theta1) / P_{\rm X|\theta} (x|\theta1) </math>
 +
 +
Choose threshold  (T),
 +
 +
say H_1 if L(x) > T
 +
 +
say H_0 if L(x) < T
 +
 +
so ML Rule is an LRT with T = 1
 +
 +
as T increases Type I Error Increases
 +
 +
as T increases Type II Error Decreases
 +
 +
& Vice Versa
 +
 +
so ML Rule is an LRT with T =1
 
== '''Law Of Iterated Expectation''' ==
 
== '''Law Of Iterated Expectation''' ==
  
Line 46: Line 67:
 
Say H0 when truth is H1. Probability of this is: Pr(Say H0|H1) = Pr(X is NOT in R|theta1)
 
Say H0 when truth is H1. Probability of this is: Pr(Say H0|H1) = Pr(X is NOT in R|theta1)
  
== How to find a good rule?
 
Likelihood Ratio TEST ==
 
  
<math> L(x) = P_(X|\theta) (x|\theta1) / P_(X|\theta) (x|\theta1) </math>
 
  
 
==Hypothesis Testing: MAP Rule==
 
==Hypothesis Testing: MAP Rule==
  
 
Overall P(err) = <math>P_{\theta}(\theta_{0})Pr[Say H_{1}|H_{0}]+P_{\theta}(\theta_{1})Pr[Say H_{0}|H_{1}]</math>
 
Overall P(err) = <math>P_{\theta}(\theta_{0})Pr[Say H_{1}|H_{0}]+P_{\theta}(\theta_{1})Pr[Say H_{0}|H_{1}]</math>

Revision as of 12:44, 13 December 2008

Maximum Likelihood Estimation (ML)

$ \hat a_{ML} = \text{max}_a ( f_{X}(x_i;a)) $ continuous

$ \hat a_{ML} = \text{max}_a ( Pr(x_i;a)) $ discrete

Maximum A-Posteriori Estimation (MAP)

$ \hat \theta_{MAP}(x) = \text{arg max}_\theta P_{X|\theta}(x|\theta)P_ {\theta}(\theta) $

$ \hat \theta_{MAP}(x) = \text{arg max}_\theta f_{X|\theta}(x|\theta)P_ {\theta}(\theta) $

Minimum Mean-Square Estimation (MMSE)

$ \hat{y}_{\rm MMSE}(x) = \int\limits_{-\infty}^{\infty}\ {y}{f}_{\rm Y|X}(y|x)\, dy={E}(Y|X=x) $

Likelihood Ratio TEST

How to find a good rule? --Khosla 16:44, 13 December 2008 (UTC)

$ \ L(x) = P_{\rm X|\theta} (x|\theta1) / P_{\rm X|\theta} (x|\theta1) $

Choose threshold (T),

say H_1 if L(x) > T

say H_0 if L(x) < T

so ML Rule is an LRT with T = 1

as T increases Type I Error Increases

as T increases Type II Error Decreases

& Vice Versa

so ML Rule is an LRT with T =1

Law Of Iterated Expectation

Unconditional Expectaion--E[X] = E{E[x|theta]}--Umang 16:10, 13 December 2008 (UTC)umang


Mean square error :

Headline text

$ MSE = E[(\theta - \hat \theta(x))^2] $

Linear Minimum Mean-Square Estimation (LMMSE)

$ \hat{y}_{\rm LMMSE}(x) = E[\theta]+\frac{COV(x,\theta)}{Var(x)}*(x-E[x]) $

Law of Iterated Expectation: E[E[X|Y]]=E[X]

Hypothesis Testing: ML Rule

Given a value of X, we will say H1 is true if X is in region R, else will will say H0 is true.

Type I error

Say H1 when truth is H0. Probability of this is: Pr(Say H1|H0) = Pr(X is in R|theta0)

Type II error

Say H0 when truth is H1. Probability of this is: Pr(Say H0|H1) = Pr(X is NOT in R|theta1)


Hypothesis Testing: MAP Rule

Overall P(err) = $ P_{\theta}(\theta_{0})Pr[Say H_{1}|H_{0}]+P_{\theta}(\theta_{1})Pr[Say H_{0}|H_{1}] $

Alumni Liaison

Questions/answers with a recent ECE grad

Ryne Rayburn