Line 27: Line 27:
 
** Let <math>D={x_1,x_2,...,x_N}</math>, <math>x_j</math> is drown independently from some probability law.  
 
** Let <math>D={x_1,x_2,...,x_N}</math>, <math>x_j</math> is drown independently from some probability law.  
 
** Choose parametric from <math>\rho(x|\theta)</math> for the pdf of x or <math>Prob(x|\theta)</math> for the probability of x <math>\rightarrow</math> an unknown parametric vector
 
** Choose parametric from <math>\rho(x|\theta)</math> for the pdf of x or <math>Prob(x|\theta)</math> for the probability of x <math>\rightarrow</math> an unknown parametric vector
 +
**Use <math>D</math> to estimate <math>\theta</math>
 +
 +
*The maximum likelihood estimate of <math>\theta</math> is the value <math>\hat{\theta}</math> that maximize
 +
<math>\rho_D(D|\theta)</math>, if x is continuous R.V.,
 +
or <math>Prob(D|\theta)</math>, if x is discrete R.V.
  
  

Revision as of 21:15, 5 May 2014


Expected Value of MLE estimate over standard deviation and expected deviation

A slecture by ECE student Zhenpeng Zhao

Partly based on the ECE662 Spring 2014 lecture material of Prof. Mireille Boutin.




1. Motivation

  • Most likely converge as number of number of training sample increase.
  • Simpler than alternate methods such as Bayesian technique.



2. Motivation

  • Statistical Density Theory Context
    • Given c classes + some knowledge about features $ x \in \mathbb{R}^n $ (or some other space)
    • Given training data, $ x_j\sim\rho(x)=\sum\limits_{i=1}^n\rho(x|w_i) Prob(w_i) $, unknown class $ w_{ij} $ for $ x_j $ is know, $ \forall{j}=1,...,N $ (N hopefully large enough)
    • In order to make decision, we need to estimate $ \rho(x|w_i) $, $ Prob(w_i) $ $ \rightarrow $ use Bayes rule, or $ \rho(x|w_i) $ $ \rightarrow $ use Neyman-Pearson Criterion
    • To estimate the above two, use training data.
  • The parametric pdf|Prob estimation problem
    • Let $ D={x_1,x_2,...,x_N} $, $ x_j $ is drown independently from some probability law.
    • Choose parametric from $ \rho(x|\theta) $ for the pdf of x or $ Prob(x|\theta) $ for the probability of x $ \rightarrow $ an unknown parametric vector
    • Use $ D $ to estimate $ \theta $
  • The maximum likelihood estimate of $ \theta $ is the value $ \hat{\theta} $ that maximize

$ \rho_D(D|\theta) $, if x is continuous R.V., or $ Prob(D|\theta) $, if x is discrete R.V.


Zhenpeng Selecture 1.png Zhenpeng Selecture 2.png Zhenpeng Selecture 3.png Zhenpeng Selecture 4.png Zhenpeng Selecture 5.png



(create a question page and put a link below)

Questions and comments

If you have any questions, comments, etc. please post them on https://kiwi.ecn.purdue.edu/rhea/index.php/ECE662Selecture_ZHenpengMLE_Ques.


Back to ECE662, Spring 2014

Alumni Liaison

Abstract algebra continues the conceptual developments of linear algebra, on an even grander scale.

Dr. Paul Garrett