| (15 intermediate revisions by 3 users not shown) | |||
| Line 9: | Line 9: | ||
</font size> | </font size> | ||
| − | A [ | + | A [http://www.projectrhea.org/learning/slectures.php slecture] by graduate student Keehwan Park |
| − | Loosely based on the [[ | + | Loosely based on the [[2014_Spring_ECE_662_Boutin_Statistical_Pattern_recognition_slectures|ECE662 Spring 2014 lecture]] material of [[user:mboutin|Prof. Mireille Boutin]]. |
</center> | </center> | ||
---- | ---- | ||
---- | ---- | ||
==Part 1: Basic Setup== | ==Part 1: Basic Setup== | ||
| + | [http://youtu.be/YeYpWL7LKUs Link to Video on Youtube] | ||
| + | <center> | ||
| + | <youtube>YeYpWL7LKUs</youtube> | ||
| + | </center> | ||
---- | ---- | ||
==Part 2: Properties of MLE== | ==Part 2: Properties of MLE== | ||
| + | [http://youtu.be/iQzfGPXGkQ4 Link to Video on Youtube] | ||
| + | |||
| + | <center> | ||
| + | <youtube>iQzfGPXGkQ4</youtube> | ||
| + | </center> | ||
---- | ---- | ||
==Part 3: Examples of MLE (Analytically Tractable Cases)== | ==Part 3: Examples of MLE (Analytically Tractable Cases)== | ||
| + | *Binomial(<math>n=1</math>,<math>p</math>) | ||
| + | ::[http://youtu.be/Yz2zJgNnXMM Link to Video on Youtube] | ||
| + | <center> | ||
| + | <youtube>Yz2zJgNnXMM</youtube> | ||
| + | </center> | ||
| + | |||
| + | *Gamma(<math>k=2</math>,<math>\theta</math>) | ||
| + | ::[http://youtu.be/GyEYKasQTFg Link to Video on Youtube] | ||
| + | <center> | ||
| + | <youtube>GyEYKasQTFg</youtube> | ||
| + | </center> | ||
| + | |||
| + | *Normal(<math>\mu=0</math>, <math>\sigma^2</math>) | ||
| + | ::[http://youtu.be/w8YhiTAX4Cg Link to Video on Youtube] | ||
| + | |||
| + | <center> | ||
| + | <youtube>w8YhiTAX4Cg</youtube> | ||
| + | </center> | ||
---- | ---- | ||
==Part 4: Summary of MLE and Numerical Optimization Options== | ==Part 4: Summary of MLE and Numerical Optimization Options== | ||
| + | [http://youtu.be/y6wtaX5GyXE Link to Video on Youtube] | ||
| + | |||
| + | <center> | ||
| + | <youtube>y6wtaX5GyXE</youtube> | ||
| + | </center> | ||
---- | ---- | ||
==References== | ==References== | ||
| Line 29: | Line 61: | ||
*Panchenko, Dmitry. "[http://ocw.mit.edu/courses/mathematics/18-443-statistics-for-applications-fall-2006/lecture-notes/lecture3.pdf Lecture 3: Properties of MLE: consistency, asymptotic normality. Fisher information]," "18-443: Statistics for Applications," MIT, Fall 2006. | *Panchenko, Dmitry. "[http://ocw.mit.edu/courses/mathematics/18-443-statistics-for-applications-fall-2006/lecture-notes/lecture3.pdf Lecture 3: Properties of MLE: consistency, asymptotic normality. Fisher information]," "18-443: Statistics for Applications," MIT, Fall 2006. | ||
*Golder, Matt, "[https://files.nyu.edu/mrg217/public/mle_introduction1.pdf Maximum Likelihood Estimation (MLE)]," Pennsylvania State University. | *Golder, Matt, "[https://files.nyu.edu/mrg217/public/mle_introduction1.pdf Maximum Likelihood Estimation (MLE)]," Pennsylvania State University. | ||
| − | + | *Dietze, Michael, "[http://www.life.illinois.edu/dietze/Lectures2012/Lesson07_Optim.pdf Lesson 7 Intractable MLEs: Basics of Numerical Optimization]," "Statistical Modeling", University of Illinois at Urbana-Champaign. | |
*"[http://www.itl.nist.gov/div898/handbook/eda/section3/eda3652.htm 1.3.6.5.2. Maximum Likelihood.]" N.p., n.d. Web. 29 Apr. 2014. | *"[http://www.itl.nist.gov/div898/handbook/eda/section3/eda3652.htm 1.3.6.5.2. Maximum Likelihood.]" N.p., n.d. Web. 29 Apr. 2014. | ||
| + | *"Maximum likelihood." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 April 2014. Web. 29 Apr. 2014. | ||
| + | *"Cramér–Rao bound." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 28 October 2013. Web. 29 Apr. 2014. | ||
| + | *"Expectation–maximization algorithm." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 3 April 2014. Web. 29 Apr. 2014. | ||
| + | *C. Couvreur. The EM algorithm: A guided tour. In Proc. 2d IEEE European Workshop on Computationaly Intensive Methods in Control and Signal Processing (CMP’96), pages 115–120, Pragues, Czech Republik, August 1996. | ||
---- | ---- | ||
---- | ---- | ||
| − | ==Comments | + | ==[[SlectureKeehwanECE662Spring14Review|Review and Comments]]== |
| − | + | ||
| − | + | ||
| − | + | ||
---- | ---- | ||
[[2014_Spring_ECE_662_Boutin|Back to ECE662, Spring 2014]] | [[2014_Spring_ECE_662_Boutin|Back to ECE662, Spring 2014]] | ||
Latest revision as of 09:50, 22 January 2015
Maximum Likelihood Estimation (MLE): its properties and examples
A slecture by graduate student Keehwan Park
Loosely based on the ECE662 Spring 2014 lecture material of Prof. Mireille Boutin.
Contents
Part 1: Basic Setup
Part 2: Properties of MLE
Part 3: Examples of MLE (Analytically Tractable Cases)
- Binomial($ n=1 $,$ p $)
- Gamma($ k=2 $,$ \theta $)
- Normal($ \mu=0 $, $ \sigma^2 $)
Part 4: Summary of MLE and Numerical Optimization Options
References
- Mireille Boutin, "ECE662: Statistical Pattern Recognition and Decision Making Processes," Purdue University, Spring 2014.
- R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification, Wiley New York, 2nd Edition, 2000.
- Myung, In Jae. "Tutorial on Maximum Likelihood Estimation." Journal of Mathematical Psychology 47.1 (2003): 90-100. Print.
- Panchenko, Dmitry. "Lecture 3: Properties of MLE: consistency, asymptotic normality. Fisher information," "18-443: Statistics for Applications," MIT, Fall 2006.
- Golder, Matt, "Maximum Likelihood Estimation (MLE)," Pennsylvania State University.
- Dietze, Michael, "Lesson 7 Intractable MLEs: Basics of Numerical Optimization," "Statistical Modeling", University of Illinois at Urbana-Champaign.
- "1.3.6.5.2. Maximum Likelihood." N.p., n.d. Web. 29 Apr. 2014.
- "Maximum likelihood." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 26 April 2014. Web. 29 Apr. 2014.
- "Cramér–Rao bound." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 28 October 2013. Web. 29 Apr. 2014.
- "Expectation–maximization algorithm." Wikipedia: The Free Encyclopedia. Wikimedia Foundation, Inc. 3 April 2014. Web. 29 Apr. 2014.
- C. Couvreur. The EM algorithm: A guided tour. In Proc. 2d IEEE European Workshop on Computationaly Intensive Methods in Control and Signal Processing (CMP’96), pages 115–120, Pragues, Czech Republik, August 1996.
