| Line 11: | Line 11: | ||
== ''' 1. Outline of the slecture'''<br> == | == ''' 1. Outline of the slecture'''<br> == | ||
| − | Receiver Operating Characteristic (ROC) curve is often used as an important tool to visualize the performance of a binary classifier. The use of ROC curves can be originated from signal detection theory that developed during World War II for radar analysis [2]. What will be covered in the slecture is listed as: | + | Receiver Operating Characteristic (ROC) curve is often used as an important tool to visualize the performance of a binary classifier. The use of ROC curves can be originated from signal detection theory that developed during World War II for radar analysis [2]. What will be covered in the slecture is listed as: |
| − | *A quick example about ROC in binary classification | + | *Basics in measuring binary classification |
| − | *Some statistics behind ROC curves | + | *A quick example about ROC in binary classification |
| + | *Some statistics behind ROC curves | ||
*Neyman-Pearson Criterion<br> | *Neyman-Pearson Criterion<br> | ||
| Line 20: | Line 21: | ||
---- | ---- | ||
| + | |||
| + | == ''' 2. Introduction '''<br> == | ||
| + | |||
| + | A ROC curve shows graphically about the trade-off between the true positive rate (TPR) and the false positive rate (FPR). Now assume that we have a two-class prediction problem (binary classification), in which the outcomes are labeled either as class 1 (C1) or class 2 (C2). There are four possible outcomes from a binary classifier. If the outcome from a prediction is C1 and the actual value is also C1, then it is called a true positive (TP); however if the actual value is C2 then it is said to be a false positive (FP). Conversely, a true negative (TN) has occurred when both the prediction outcome and the actual value are C2, and false negative (FN) is when the prediction outcome is C2 while the actual value is C1. Usually, the outcomes can be summarized into a contingency table or a Confusion Matrix: | ||
| + | |||
| + | {| width="600" border="1" cellpadding="1" cellspacing="1" | ||
| + | |+ A Confusion Matrix | ||
| + | |- | ||
| + | | Actual class \ Predicted class | ||
| + | | C1 | ||
| + | | C2 | ||
| + | |- | ||
| + | | C1 | ||
| + | | True Positives (TP) | ||
| + | | False Negatives (FN) | ||
| + | |- | ||
| + | | C2 | ||
| + | | False Positives (FP) | ||
| + | | True Negatives (TN) | ||
| + | |} | ||
| + | |||
| + | |||
| + | |||
| + | <br> | ||
---- | ---- | ||
Revision as of 17:42, 29 April 2014
ROC curve and Neyman Pearsom Criterion
Partly based on the ECE662 Spring 2014 lecture material of Prof. Mireille Boutin.
1. Outline of the slecture
Receiver Operating Characteristic (ROC) curve is often used as an important tool to visualize the performance of a binary classifier. The use of ROC curves can be originated from signal detection theory that developed during World War II for radar analysis [2]. What will be covered in the slecture is listed as:
- Basics in measuring binary classification
- A quick example about ROC in binary classification
- Some statistics behind ROC curves
- Neyman-Pearson Criterion
2. Introduction
A ROC curve shows graphically about the trade-off between the true positive rate (TPR) and the false positive rate (FPR). Now assume that we have a two-class prediction problem (binary classification), in which the outcomes are labeled either as class 1 (C1) or class 2 (C2). There are four possible outcomes from a binary classifier. If the outcome from a prediction is C1 and the actual value is also C1, then it is called a true positive (TP); however if the actual value is C2 then it is said to be a false positive (FP). Conversely, a true negative (TN) has occurred when both the prediction outcome and the actual value are C2, and false negative (FN) is when the prediction outcome is C2 while the actual value is C1. Usually, the outcomes can be summarized into a contingency table or a Confusion Matrix:
| Actual class \ Predicted class | C1 | C2 |
| C1 | True Positives (TP) | False Negatives (FN) |
| C2 | False Positives (FP) | True Negatives (TN) |
Reference
[1] Mireille Boutin, "ECE662: Statistical Pattern Recognition and Decision Making Processes," Purdue University, Spring 2014.
[2] Jiawei Han. 2005. Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
[3] Richard O. Duda, Peter E. Hart, and David G. Stork. 2000. Pattern Classification. Wiley-Interscience.
[4] Detection Theory. http://www.ece.iastate.edu/~namrata/EE527_Spring08/l5c_2.pdf.
[5] The Neyman-Pearson Criterion. http://cnx.org/content/m11548/1.2/.
Questions and comments
If you have any questions, comments, etc. please post them on this page.
