Bayes' classification is an ideal classification technique when the true distribution of the data is known. Although it can rarely be used in practice, it represents an ideal classification rate which other algorithms may attempt to achieve.
Lectures discussing this technique:
Bayes rule
(From Lecture 3 - Bayes classification_OldKiwi)
Bayes rule addresses the predefined classes classification problem. Given value of X for an object, assign one of the k classes to the object
Bayes rule is used for discrete feature vectors, that is, Bayes rule is to do the following: Given $ x $, choose the most likely class $ E{\lbrace}w_1,...,w_k{\rbrace} $
$ w: E{\lbrace}w_1,...,w_k{\rbrace} $ ie. choose $ w_i $ such that the $ P(w_i|x) \geq P(w_j|x), {\forall}j $
$ posterior = \frac{(likelihood)(prior)}{(evidence)} $
$ posterior = P(w_i|x)= \frac{p(x|w_i)P(w_i)}{P(x)} $
Bayes rule: choose the class $ w_i $ that maximizes the $ p(x|w_i)P(w_i) $
Example: Given 2 class decision problems $ w_1 = $ women & $ w_2 $= men, $ L = hair length $ choose $ w_1 $, if $ P(w_1|L) \geq P(w_2|L) $ else choose $ w_2 $ or
choose $ w_1 $ if $ p(L|w_1)P(w_1)>p(L|w_2)P(w_2) $
else choose $ w_2 $
Minimum probability of error is the error made when $ w = w_2 $ and decided $ w_1 $
Special cases
If $ P(w_1) = P(w_2) $
$ p(x|w_1)P(w_1) \geq p(x|w_2)P(w_2), {\forall j} $
$ p(x|w_1) \geq p(x|w_2) $ decision is based on the likelihood
-If $ p(x|w_1)=p(x|w_2) $
$ p(x|w_1)P(w_1) \geq p(x|w_2)P(w_2), {\forall j} $
$ P(w_1) \geq P(w_2) $ decision is based on the prior
See Also
Lectures discussing this technique:
