Difference between revisions of "Bayes rule in practice" - Rhea

Revision as of 11:58, 21 April 2014

Bayes rule in practice
A slecture by Lu Wang

(partially based on Prof. Mireille Boutin's ECE 662 lecture)

1. Bayes rule for Gaussian data

Given data x ∈ R $d$ and N categories {w $i$ }, i=1,2,…,N, we decide which category the data corresponds to by computing the probability of the N events. We’ll pick the category with the largest probability. Mathematically, this can be interpreted as:

\max_{w_{i}} \rho \left(w_{i}|x\right)

According to Bayes rule:

\max_{w_{i}} \rho \left(w_{i}|x\right) = \max_{w_{i}} \rho \left(x|w_{i}\right)Prob(w_{i})

In our case, the data is distributed as Gaussian. So we have,

\rho \left(x|w_{i}\right) = \frac{1}{(2\pi)^{\frac{n}{2}}|\mathbf{\Sigma}|^{\frac{1}{2}}}\mbox{exp}\left[{-\frac{1}{2}(x - \mu)^T\mathbf{\Sigma}^{-1}(x - \mu)}\right]

Let

\begin{align}g_{i}(x) &= ln(\rho \left(w_{i}|x\right)) \\ &= ln(\rho \left(x|w_{i}\right)Prob(w_{i})) \\ &= -\frac{n}{2}ln(2\pi)-\frac{1}{2}ln(|\mathbf{\Sigma}|)-{\frac{1}{2}(x - \mu)^T\mathbf{\Sigma}^{-1}(x - \mu)} \end{align}

Now we have,

\begin{align}\max_{w_{i}} \rho \left(w_{i}|x\right) &= \max_{w_{i}} \rho \left(x|w_{i}\right)Prob(w_{i}) \\ &= \max_{w_{i}} g_{i}(x) \end{align}

For two-class case, generate the discriminant function as

g\left(x\right) = g_{1}\left(x\right) - g_{2}\left(x\right);

decide w₁ if g(x) > 0;
else decide w₂.

@@ Line 36: / Line 36: @@
 \end{align}
 </math></center>
-<br> Generate the discriminant function as
+<br>For two-class case, generate the discriminant function as
-<center><math>g\left(x\right) = g_{1}\left(x\right) - g_{2}\left(x\right);</math><br></center> <center>decide w<sub>1</sub>&nbsp;if <br></center>
+<center><math>g\left(x\right) = g_{1}\left(x\right) - g_{2}\left(x\right);</math><br></center> <center>decide w<sub>1</sub>&nbsp;if g(x) &gt;&nbsp;0;<br></center> <center>else decide w<sub>2</sub>.</center>

Difference between revisions of "Bayes rule in practice" - Rhea

Revision as of 11:58, 21 April 2014

1. Bayes rule for Gaussian data

Alumni Liaison