Latest revision as of 06:19, 25 June 2012

Third Homework, ECE662 Spring 2012

• Drop test set output, predicted accuracy, and proposed nickname into Daniel's drop box before 11:59pm, Friday April 27, 2012.
• Report due before 11:59pm, Monday April 30, in your instructor's Rhea dropbox. Make sure to drop it in the correct homework folder!!!!. It is the somewhere in the middle of the page.

Automatic Pattern Recognition Contest!

An anonymous company has agreed to share real data with us, so we are going to have a little contest using this data! The data comes from a five-class classification problem using 13 features. We are looking for the student who will design the most accurate classifier using this data.

The training data consists of 550 data points (i.e. 550 points in a 13 dimensional space) along with the correct label for each point. Use this data, along with any method of your choice, to design what you think is an accurate classifier.

Part I

When you are done designing your classifier, write down what accuracy the classifier has, according to the training data. (This is what we call the "predicted accuracy".) Then use your classifier to classify the following test set vectors. Your colleague Daniel has kindly volunteered to collect all the answers and summarize their accuracy in the table below. In order for Daniel to do that, we ask that you send him

• The nickname you want him to use to identify your work in the table below;
• Your predicted accuracy;
• The labels you obtain for each test set vector.

Hand in everything in Daniel's drop box in a single file following this syntax:

Joe Blo
77%
1
0
1
2
3
4
2
3
1
2
0
2
1

Part II

After the deadline for the homework has passed, I will release the ground truth labels for the test set vectors here. (If you are done designing your classifier before 8pm on Thursday April 26, you may obtain the ground truth labels by emailing your instructor.) Compare the labels obtained using your classifier with the ground truth labels: the number of correctly classified vectors is the "test" accuracy. Summarize your method and results in a report. Make sure your report includes all your code, your predicted accuracy, and the confusion matrix of your classifier on the test data. Drop a pdf of your report in the HW3 assignment box in your instructor's dropbox before 11:59pm, Monday April 30.

Part III (optional)

If you feel like sharing your results and methods publicly, feel free to post a summary of your method and replace your nickname by your true name in the table below. But please only do this after April 30. If you do not wish to be identified but still would like a summary of your method to appear in the table below, send it to Daniel (by email) and he will post it for you.

The discussion page for this homework is here.

Result Summary (kindly complied by your colleague Daniel)

The following results were computed from the submissions in drugeles dropbox. Refer to HW3 discussion for details or questions.

Finally our competition has come to an end. However you are more than welcome to keep experimenting on the dataset, or trying to find trends in the results from our ece662 alumni. Many of the submissions did not follow the guidelines and therefore there might be some results missing, if this is your case, please let me know, I will recompute the results and we might even find a new winner.

All the data used in testing the competition and the script can be found at here.
Only unzip and execute in the shell
$ python contest.py

Please do not forget to share results and why not, a link to your work. We will be happy to see Purdue's talent in pattern recognition.

Results from Pattern Recognition Contest!
Position	Nickname (add link to method summary or report)	Confusion Matrix	Predicted Accuracy	Test Set Accuracy		Within 10%
1	shera (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 45& 13& 24& 8& 0\\ \mathbf{1}& 3& 6& 2& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43.64%	50.5%
2	ck910525 (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.73%	47.52%
3	Tim Chen (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.5455%	47.52%
4	Sherlock Holmes (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 25& 7& 0\\ \mathbf{1}& 0& 0& 1& 1& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	40%	47.52%
5	Katniss Everdeen (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.44%	47.52%
6	Joy (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42%	47.52%
7	ECE_662_is_cool (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 37& 10& 16& 3& 0\\ \mathbf{1}& 1& 5& 4& 4& 0\\ \mathbf{2}& 10& 4& 6& 1& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	61.45%	47.52%
8	Chris (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43.4%	47.52%
9	BY SVM 1-vs-1 RBF 6 (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.7007%	47.52%
10	BY SVM 1-vs-1 Poly 2nd (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.6314%	47.52%
11	Alvin (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.55%	47.52%
12	Alpha Zeta (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 48& 19& 26& 8& 0\\ \mathbf{1}& 0& 0& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.5%	47.52%
13	parriky (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 46& 18& 26& 8& 0\\ \mathbf{1}& 1& 1& 0& 0& 0\\ \mathbf{2}& 1& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42%	46.53%
14	chotu (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 46& 18& 25& 7& 0\\ \mathbf{1}& 2& 1& 1& 0& 0\\ \mathbf{2}& 0& 0& 0& 1& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	40%	46.53%
15	Zoe (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 46& 18& 24& 6& 0\\ \mathbf{1}& 2& 1& 2& 0& 0\\ \mathbf{2}& 0& 0& 0& 2& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43%	46.53%
16	Shelan (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 43& 15& 21& 4& 0\\ \mathbf{1}& 5& 4& 5& 2& 0\\ \mathbf{2}& 0& 0& 0& 2& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.91%	46.53%
17	Danny (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 46& 18& 26& 8& 0\\ \mathbf{1}& 2& 1& 0& 0& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	41.73%	46.53%
18	BY SVM 1-vs-all RBF 6 (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 43& 17& 21& 5& 0\\ \mathbf{1}& 3& 2& 3& 1& 0\\ \mathbf{2}& 2& 0& 2& 2& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	41.7482%	46.53%
19	starry (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 38& 11& 18& 7& 0\\ \mathbf{1}& 8& 6& 7& 1& 0\\ \mathbf{2}& 2& 2& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	45.27%	44.55%
20	Doctor Who (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 44& 19& 25& 8& 0\\ \mathbf{1}& 3& 0& 0& 0& 0\\ \mathbf{2}& 1& 0& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	47%	44.55%
21	Czardas (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 38& 11& 17& 7& 0\\ \mathbf{1}& 8& 6& 8& 1& 0\\ \mathbf{2}& 2& 2& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	45.45%	44.55%
22	BY SVM 1-vs-all Poly 2nd (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 42& 16& 21& 5& 0\\ \mathbf{1}& 3& 0& 1& 3& 0\\ \mathbf{2}& 3& 3& 3& 0& 0\\ \mathbf{3}& 0& 0& 1& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	47.0474%	44.55%
23	NaiveBoys (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 40& 15& 22& 8& 0\\ \mathbf{1}& 4& 2& 2& 0& 0\\ \mathbf{2}& 3& 2& 2& 0& 0\\ \mathbf{3}& 1& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.55%	43.56%
24	ARM (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 41& 17& 21& 8& 0\\ \mathbf{1}& 4& 1& 2& 0& 0\\ \mathbf{2}& 2& 1& 2& 0& 0\\ \mathbf{3}& 1& 0& 1& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42%	43.56%
25	cheese (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 36& 14& 17& 6& 0\\ \mathbf{1}& 3& 1& 3& 2& 0\\ \mathbf{2}& 9& 4& 6& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	44%	42.57%
26	Mc Awesome Man Random Forest report	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 41& 16& 25& 7& 0\\ \mathbf{1}& 4& 2& 1& 1& 0\\ \mathbf{2}& 2& 1& 0& 0& 0\\ \mathbf{3}& 1& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	41%	42.57%
27	Mao (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 34& 9& 20& 8& 0\\ \mathbf{1}& 10& 9& 6& 0& 0\\ \mathbf{2}& 4& 1& 0& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43%	42.57%
28	Jin (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 42& 19& 24& 8& 0\\ \mathbf{1}& 5& 0& 1& 0& 0\\ \mathbf{2}& 1& 0& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	63.8%	42.57%
29	Gemini (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 34& 11& 13& 7& 0\\ \mathbf{1}& 9& 6& 10& 1& 0\\ \mathbf{2}& 3& 2& 3& 0& 0\\ \mathbf{3}& 2& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42%	42.57%
30	timetorun (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 34& 12& 13& 7& 0\\ \mathbf{1}& 9& 4& 9& 1& 0\\ \mathbf{2}& 5& 3& 4& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	47%	41.58%
31	confusion matrix (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 37& 16& 18& 7& 0\\ \mathbf{1}& 8& 2& 4& 1& 0\\ \mathbf{2}& 2& 1& 3& 0& 0\\ \mathbf{3}& 1& 0& 1& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.55%	41.58%
32	asvyatko (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 39& 15& 21& 8& 0\\ \mathbf{1}& 4& 1& 2& 0& 0\\ \mathbf{2}& 3& 1& 2& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 2& 2& 1& 0& 0\\ \end{array} \right) $	44%	41.58%
33	Ming (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 39& 15& 21& 7& 0\\ \mathbf{1}& 4& 2& 4& 1& 0\\ \mathbf{2}& 5& 2& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42.886%	41.58%
34	DanRugeles Optimized KNN <br\> K=14 <br\> Distance metric=L1/2 <br\> Weighting Function= Adaptive Gaussian <br\> Feature selection by ranking (Features 8,9,10).<br\> Ranking=Bhattacharyya distance.<br\> Report	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 35& 12& 14& 5& 0\\ \mathbf{1}& 6& 2& 7& 2& 0\\ \mathbf{2}& 7& 5& 5& 1& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43.54%	41.58%
35	MM (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 30& 8& 14& 3& 0\\ \mathbf{1}& 10& 7& 8& 1& 0\\ \mathbf{2}& 6& 4& 4& 4& 0\\ \mathbf{3}& 2& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	45%	40.59%
36	chimp Bayes Discriminant Functions Lazy Learners Decision Rules Decision Trees Report	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 26& 10& 9& 6& 0\\ \mathbf{1}& 9& 5& 8& 1& 0\\ \mathbf{2}& 13& 3& 9& 1& 0\\ \mathbf{3}& 0& 1& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	41%	39.6%
37	Min (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 34& 11& 19& 8& 0\\ \mathbf{1}& 7& 4& 5& 0& 0\\ \mathbf{2}& 7& 4& 2& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43.27%	39.6%
38	M Usman Sadiq (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 23& 10& 6& 2& 0\\ \mathbf{1}& 2& 1& 0& 0& 0\\ \mathbf{2}& 20& 8& 16& 5& 0\\ \mathbf{3}& 3& 0& 3& 0& 0\\ \mathbf{4}& 0& 0& 1& 1& 0\\ \end{array} \right) $	46.6667%	39.6%
39	Marshall Zigler (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 37& 16& 19& 6& 0\\ \mathbf{1}& 10& 1& 5& 2& 0\\ \mathbf{2}& 1& 2& 2& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	43.45%	39.6%
40	WorldPeace (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 36& 16& 20& 8& 0\\ \mathbf{1}& 8& 2& 5& 0& 0\\ \mathbf{2}& 4& 1& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	45%	38.61%
41	K Marshall (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 34& 14& 16& 7& 0\\ \mathbf{1}& 10& 2& 6& 1& 0\\ \mathbf{2}& 3& 2& 3& 0& 0\\ \mathbf{3}& 1& 1& 1& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	40%	38.61%
42	ellie (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 32& 12& 17& 4& 0\\ \mathbf{1}& 14& 5& 8& 4& 0\\ \mathbf{2}& 2& 2& 1& 0& 0\\ \mathbf{3}& 0& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	40.16%	37.62%
43	BY KNN (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 34& 14& 18& 7& 0\\ \mathbf{1}& 9& 2& 7& 1& 0\\ \mathbf{2}& 3& 3& 1& 0& 0\\ \mathbf{3}& 2& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	48.54%	36.63%
44	shawn (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 25& 10& 14& 6& 0\\ \mathbf{1}& 9& 4& 7& 1& 0\\ \mathbf{2}& 12& 3& 4& 1& 0\\ \mathbf{3}& 2& 2& 1& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	36%	32.67%
45	Vicky (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 26& 12& 17& 3& 0\\ \mathbf{1}& 13& 4& 7& 4& 0\\ \mathbf{2}& 8& 3& 2& 1& 0\\ \mathbf{3}& 1& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	42%	31.68%
46	KH7 (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 24& 9& 9& 1& 0\\ \mathbf{1}& 7& 1& 5& 1& 0\\ \mathbf{2}& 0& 0& 0& 0& 0\\ \mathbf{3}& 17& 9& 12& 6& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	24.59%	30.69%
47	JRM (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 24& 15& 13& 4& 0\\ \mathbf{1}& 16& 3& 9& 4& 0\\ \mathbf{2}& 7& 1& 4& 0& 0\\ \mathbf{3}& 1& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	66%	30.69%
48	Dom (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 26& 11& 18& 6& 0\\ \mathbf{1}& 12& 3& 6& 1& 0\\ \mathbf{2}& 7& 5& 2& 1& 0\\ \mathbf{3}& 3& 0& 0& 0& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	30%	30.69%
49	neergil (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 20& 7& 12& 4& 0\\ \mathbf{1}& 14& 4& 10& 3& 0\\ \mathbf{2}& 9& 5& 4& 0& 0\\ \mathbf{3}& 5& 3& 0& 1& 0\\ \mathbf{4}& 0& 0& 0& 0& 0\\ \end{array} \right) $	47%	28.71%
50	Luck (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 21& 9& 13& 3& 0\\ \mathbf{1}& 17& 3& 8& 3& 0\\ \mathbf{2}& 9& 4& 3& 2& 0\\ \mathbf{3}& 1& 2& 1& 0& 0\\ \mathbf{4}& 0& 1& 1& 0& 0\\ \end{array} \right) $	40%	26.73%
51	Anshu K=3 (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 21& 9& 13& 3& 0\\ \mathbf{1}& 17& 3& 8& 3& 0\\ \mathbf{2}& 9& 4& 3& 2& 0\\ \mathbf{3}& 1& 2& 1& 0& 0\\ \mathbf{4}& 0& 1& 1& 0& 0\\ \end{array} \right) $	42%	26.73%
52	Anshu K=1 (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 21& 9& 13& 3& 0\\ \mathbf{1}& 17& 3& 8& 3& 0\\ \mathbf{2}& 9& 4& 3& 2& 0\\ \mathbf{3}& 1& 2& 1& 0& 0\\ \mathbf{4}& 0& 1& 1& 0& 0\\ \end{array} \right) $	38%	26.73%
53	OCEAN'S FOURTEEN (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 15& 8& 11& 6& 0\\ \mathbf{1}& 15& 4& 10& 1& 0\\ \mathbf{2}& 7& 2& 3& 0& 0\\ \mathbf{3}& 6& 2& 1& 1& 0\\ \mathbf{4}& 5& 3& 1& 0& 0\\ \end{array} \right) $	42%	22.77%
54	MS (method summary)	$ \left( \begin{array}{cccccc}& \mathbf{0}&\mathbf{1}&\mathbf{2}&\mathbf{3}&\mathbf{4}\\ \mathbf{0}& 2& 0& 0& 0& 0\\ \mathbf{1}& 21& 10& 12& 6& 0\\ \mathbf{2}& 1& 0& 3& 0& 0\\ \mathbf{3}& 21& 9& 9& 1& 0\\ \mathbf{4}& 3& 0& 2& 1& 0\\ \end{array} \right) $	42%	15.84%

Petition:
Can you give some extra statistics on the results of the class e.g, frequency distribution of three-types of students as follows:
Type-1: how many students accuracy increased from what they predicted?
Type-2: how many students accuracy decreased from what they predicted?
Type-3: how many students accuracy remained within 5% of what they predicted?

Seeing this result would actually give an insight on the percentage of people who successfully designed a good classifier.

Thanks,
chimp

Response:
I added to columns to satisfy your petition and here are the answers to your question: 25 Classifiers where within 10% of their estimation.
23 Classifiers where overstimated.
31 Classifiers where understimated.

Petition:
<Please post your own petitions here, or send an email to drugeles@purdue.edu>

Comments:
<Please post your coments here, or send an email to drugeles@purdue.edu>