Maximum Likelihood Estimation (MLE): its properties and examples
A slecture by graduate student Keehwan Park
Loosely based on the ECE662 Spring 2014 lecture material of Prof. Mireille Boutin.
Contents
Part 1: Basic Setup
Part 2: Properties of MLE
Part 3: Examples of MLE (Analytically Tractable Cases)
- Binomial($ n=1 $,$ p $)
- Gamma($ k=2 $,$ \theta $)
- Normal($ \mu=0 $, $ \sigma^2 $)
Part 4: Summary of MLE and Numerical Optimization Options
References
- Mireille Boutin, "ECE662: Statistical Pattern Recognition and Decision Making Processes," Purdue University, Spring 2014.
- R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification, Wiley New York, 2nd Edition, 2000.
- Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journal of Mathematical Psychology 47.1 (2003): 90-100. Print.
- Panchenko, Dmitry. "Lecture 3: Properties of MLE: consistency, asymptotic normality. Fisher information," "18-443: Statistics for Applications," MIT, Fall 2006.
- Golder, Matt, "Maximum Likelihood Estimation (MLE)," Pennsylvania State University.
- Dietze, Michael, "Lesson 7 Intractable MLEs: Basics of Numerical Optimization," "Statistical Modeling", University of Illinois at Urbana-Champaign.
- "1.3.6.5.2. Maximum Likelihood." N.p., n.d. Web. 29 Apr. 2014.
- "Maximum likelihood." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 April 2014. Web. 29 Apr. 2014.
- "Cramér–Rao bound." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 28 October 2013. Web. 29 Apr. 2014.
- "Expectation–maximization algorithm." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 3 April 2014. Web. 29 Apr. 2014.
- C. Couvreur. The EM algorithm: A guided tour. In Proc. 2d IEEE European Workshop on Computationaly Intensive Methods in Control and Signal Processing (CMP’96), pages 115–120, Pragues, Czech Republik, August 1996.
