Bayes rule in practice - Rhea

Bayes rule in practice
A slecture by Lu Wang

(partially based on Prof. Mireille Boutin's ECE 662 lecture)

1. Bayes rule for Gaussian data

Given data x ∈ R $d$ and N categories {w $i$ }, i=1,2,…,N, we decide which category the data corresponds to by computing the probability of the N events. We’ll pick the category with the largest probability. Mathematically, this can be interpreted as:

\max_{w_{i}} \rho \left(w_{i}|x\right)

According to Bayes rule:

\max_{w_{i}} \rho \left(w_{i}|x\right) = \max_{w_{i}} \rho \left(x|w_{i}\right)Prob(w_{i})

In our case, the data is distributed as Gaussian. So we have,

\rho \left(x|w_{i}\right) = \frac{1}{(2\pi)^{\frac{n}{2}}|\mathbf{\Sigma}|^{\frac{1}{2}}}\mbox{exp}\left[{-\frac{1}{2}(x - \mu)^T\mathbf{\Sigma}^{-1}(x - \mu)}\right]

Let

\begin{align}g_{i}(x) &= ln(\rho \left(w_{i}|x\right)) \\ &= ln(\rho \left(x|w_{i}\right)Prob(w_{i})) \\ &= -\frac{n}{2}ln(2\pi)-\frac{1}{2}ln(|\mathbf{\Sigma}|)-{\frac{1}{2}(x - \mu)^T\mathbf{\Sigma}^{-1}(x - \mu)} \end{align}

Now we have,

\begin{align}\max_{w_{i}} \rho \left(w_{i}|x\right) &= \max_{w_{i}} \rho \left(x|w_{i}\right)Prob(w_{i}) \\ &= \max_{w_{i}} g_{i}(x) \end{align}

For two-class case, generate the discriminant function as

g\left(x\right) = g_{1}\left(x\right) - g_{2}\left(x\right);

decide w₁ if g(x) > 0;
else decide w₂.

Bayes rule in practice - Rhea

1. Bayes rule for Gaussian data

Alumni Liaison