Revision as of 06:24, 29 April 2010

What is your favorite decision method?

Student poll for ECE662, Spring 2010.

• Coin flipping. ostava
  • Interesting. What is the expected rate of error for this method? -pm
  • I would think the expected error would be .5. Assume if heads decide class 1, if tails decide class 2. So P(error) = P(Heads)P(Class 2) + P(Tails)P(Class 1). I'll assume you have a fair coin so P(Heads) = P(Tails) = .5. Also, if there's only two classes, P(Class 2) + P(Class 1) = 1. Thus from the above formula, P(error) = .5(P(Class1) + P(Class2)) = .5 -ATH
  • Actually, a loaded coin might be better! Looking at the relative frequency of the training data points, one can estimate the priors and bias the coin accordingly. -Satyam.
  • Actually, not flipping a coin (or equivalently flipping a coin that is so biased that it lands on one side with certainty) will be best! Biasing the coin to match the priors is better than flipping a fair coin, but will still give an expected error rate greater than or equal to the expected error rate of always choosing the class with the higher prior. Bayes rule is optimal. - jvaught
  • Good point Jim! Somehow, I get a feeling that stating coin flipping was intended to be humorous. Of course, we (engineers) have steered it in a different direction! Ondrej, comment? -Satyam.
  • Well, it wasn't really meant seriously but it is true that I've used this method quite a lot of times in my life. But it is usually somehow difficult to measure error in these real life situations .. sometimes I think the error rate actually converges to 1 :) -ostava

• Nearest neighbors. It reminds me of human behavior in that if we don't know what to do in certain situations (social ones in particular), we'll look at those around us to decide what to do. -ATH
• Kernel methods in general (SVM, KDE, KPCA, etc..) since we can handle non-linearly separable data easier. I also feel that clustering techniques are very useful in my research area. --ilaguna
• Nearest neighbor. From practical point of view, it is easy to implement and quite fast (and, surprisingly, not too bad in terms of errors). -Satyam.
• Decision trees. Linear discriminants are not expressive enough for practical data, but nonlinear are unwieldy and more prone to misbehave computationally. Decision trees give the expressiveness of nonlinear discriminants with the efficiency of linear discriminants. Humans implicitly use this approach in the games "20 Questions" and "Guess Who?" and field identification guides are organized this way. -jvaught
• Decision Trees with AdaBoost. Boosting will raise the performance of decision tree significantly and this method is generally tested out to be consistently reliable for many applications. Besides, the recent theoretical research of consistency of AdaBoost shows promising application of the Boosting methods. -yuanl
• write your opinion here. sign your name/nickname.

Back to 2010 Spring ECE 662 mboutin