(Began new page.) |
(Copy in some material) |
||
| Line 1: | Line 1: | ||
| − | '''Bayes' | + | '''Bayes' classification''' is an ideal classification technique when the true distribution of the data is known. Although it can rarely be used in practice, it represents an ideal classification rate which other algorithms may attempt to achieve. |
Lectures discussing this technique: | Lectures discussing this technique: | ||
| Line 5: | Line 5: | ||
* [[Lecture 4 - Bayes Classification_OldKiwi]] | * [[Lecture 4 - Bayes Classification_OldKiwi]] | ||
| + | == Bayes rule == | ||
| + | (From [[Lecture 3 - Bayes classification_OldKiwi]]) | ||
| − | === See Also | + | Bayes rule addresses the predefined classes classification problem. |
| + | Given value of X for an object, assign one of the k classes to the object | ||
| + | |||
| + | Bayes rule is used for discrete feature vectors, that is, Bayes rule is to do the following: Given <math>x</math>, choose the most likely class <math>E{\lbrace}w_1,...,w_k{\rbrace}</math> | ||
| + | |||
| + | <math>w: E{\lbrace}w_1,...,w_k{\rbrace}</math> | ||
| + | ie. choose <math>w_i</math> such that the <math>P(w_i|x) \geq P(w_j|x), {\forall}j</math> | ||
| + | |||
| + | <math>posterior = \frac{(likelihood)(prior)}{(evidence)}</math> | ||
| + | |||
| + | <math>posterior = P(w_i|x)= \frac{p(x|w_i)P(w_i)}{P(x)}</math> | ||
| + | |||
| + | Bayes rule: choose the class <math>w_i</math> that maximizes the <math>p(x|w_i)P(w_i)</math> | ||
| + | |||
| + | Example: Given 2 class decision problems <math>w_1 = </math> women & <math>w_2 </math>= men, <math>L = hair length </math> | ||
| + | choose <math>w_1</math>, if <math>P(w_1|L) \geq P(w_2|L)</math> | ||
| + | else choose <math>w_2</math> | ||
| + | or | ||
| + | |||
| + | choose <math>w_1</math> if <math>p(L|w_1)P(w_1)>p(L|w_2)P(w_2)</math> | ||
| + | |||
| + | else choose <math> w_2 </math> | ||
| + | |||
| + | Minimum probability of error is the error made when <math> w = w_2 </math> and decided <math> w_1 </math> | ||
| + | |||
| + | Special cases <br> | ||
| + | If <math> P(w_1) = P(w_2)</math> <br> | ||
| + | <math>p(x|w_1)P(w_1) \geq p(x|w_2)P(w_2), {\forall j}</math><br> | ||
| + | <math>p(x|w_1) \geq p(x|w_2)</math> decision is based on the likelihood<br> | ||
| + | <br> | ||
| + | -If <math>p(x|w_1)=p(x|w_2)</math><br> | ||
| + | <math>p(x|w_1)P(w_1) \geq p(x|w_2)P(w_2), {\forall j}</math><br> | ||
| + | <math>P(w_1) \geq P(w_2)</math> decision is based on the prior<br> | ||
| + | |||
| + | |||
| + | == See Also == | ||
Lectures discussing this technique: | Lectures discussing this technique: | ||
* [[Lecture 3 - Bayes classification_OldKiwi]] | * [[Lecture 3 - Bayes classification_OldKiwi]] | ||
* [[Lecture 4 - Bayes Classification_OldKiwi]] | * [[Lecture 4 - Bayes Classification_OldKiwi]] | ||
Revision as of 07:57, 17 April 2008
Bayes' classification is an ideal classification technique when the true distribution of the data is known. Although it can rarely be used in practice, it represents an ideal classification rate which other algorithms may attempt to achieve.
Lectures discussing this technique:
Bayes rule
(From Lecture 3 - Bayes classification_OldKiwi)
Bayes rule addresses the predefined classes classification problem. Given value of X for an object, assign one of the k classes to the object
Bayes rule is used for discrete feature vectors, that is, Bayes rule is to do the following: Given $ x $, choose the most likely class $ E{\lbrace}w_1,...,w_k{\rbrace} $
$ w: E{\lbrace}w_1,...,w_k{\rbrace} $ ie. choose $ w_i $ such that the $ P(w_i|x) \geq P(w_j|x), {\forall}j $
$ posterior = \frac{(likelihood)(prior)}{(evidence)} $
$ posterior = P(w_i|x)= \frac{p(x|w_i)P(w_i)}{P(x)} $
Bayes rule: choose the class $ w_i $ that maximizes the $ p(x|w_i)P(w_i) $
Example: Given 2 class decision problems $ w_1 = $ women & $ w_2 $= men, $ L = hair length $ choose $ w_1 $, if $ P(w_1|L) \geq P(w_2|L) $ else choose $ w_2 $ or
choose $ w_1 $ if $ p(L|w_1)P(w_1)>p(L|w_2)P(w_2) $
else choose $ w_2 $
Minimum probability of error is the error made when $ w = w_2 $ and decided $ w_1 $
Special cases
If $ P(w_1) = P(w_2) $
$ p(x|w_1)P(w_1) \geq p(x|w_2)P(w_2), {\forall j} $
$ p(x|w_1) \geq p(x|w_2) $ decision is based on the likelihood
-If $ p(x|w_1)=p(x|w_2) $
$ p(x|w_1)P(w_1) \geq p(x|w_2)P(w_2), {\forall j} $
$ P(w_1) \geq P(w_2) $ decision is based on the prior
See Also
Lectures discussing this technique:
