Maximum Likelihood Estimation (MLE): its properties and examples

A slecture by graduate student Keehwan Park

Loosely based on the ECE662 Spring 2014 lecture material of Prof. Mireille Boutin.



Part 1: Basic Setup

Link to Video on Youtube


Part 2: Properties of MLE

Link to Video on Youtube


Part 3: Examples of MLE (Analytically Tractable Cases)

  • Binomial($ n=1 $,$ p $)
Link to Video on Youtube

  • Gamma($ k=2 $,$ \theta $)
Link to Video on Youtube

  • Normal($ \mu=0 $, $ \sigma^2 $)
Link to Video on Youtube


Part 4: Summary of MLE and Numerical Optimization Options

Link to Video on Youtube


References

  • Mireille Boutin, "ECE662: Statistical Pattern Recognition and Decision Making Processes," Purdue University, Spring 2014.
  • R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification, Wiley New York, 2nd Edition, 2000.
  • Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journal of Mathematical Psychology 47.1 (2003): 90-100. Print.
  • Panchenko, Dmitry. "Lecture 3: Properties of MLE: consistency, asymptotic normality. Fisher information," "18-443: Statistics for Applications," MIT, Fall 2006.
  • Golder, Matt, "Maximum Likelihood Estimation (MLE)," Pennsylvania State University.
  • Dietze, Michael, "Lesson 7 Intractable MLEs: Basics of Numerical Optimization," "Statistical Modeling", University of Illinois at Urbana-Champaign.
  • "1.3.6.5.2. Maximum Likelihood." N.p., n.d. Web. 29 Apr. 2014.
  • "Maximum likelihood." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 April 2014. Web. 29 Apr. 2014.
  • "Cramér–Rao bound." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 28 October 2013. Web. 29 Apr. 2014.
  • "Expectation–maximization algorithm." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 3 April 2014. Web. 29 Apr. 2014.
  • C. Couvreur. The EM algorithm: A guided tour. In Proc. 2d IEEE European Workshop on Computationaly Intensive Methods in Control and Signal Processing (CMP’96), pages 115–120, Pragues, Czech Republik, August 1996.


Review and Comments


Back to ECE662, Spring 2014

Alumni Liaison

Basic linear algebra uncovers and clarifies very important geometry and algebra.

Dr. Paul Garrett