| Line 2: | Line 2: | ||
---- | ---- | ||
| + | |||
Lets begin with the continuous univariate normal or Gaussian density. | Lets begin with the continuous univariate normal or Gaussian density. | ||
| Line 37: | Line 38: | ||
</div> | </div> | ||
| − | where the expected value of a vector or a matrix is found by taking the expected value of the individual components. i.e if ''x<sub>i</sub>'' is the ''i''th component of '''x''', ''μ<sub>i</sub>'' the ''i''th component of '''μ''', and ''σ<sub>ij</sub> the ''ij''th component of '''Σ''', then | + | where the expected value of a vector or a matrix is found by taking the expected value of the individual components. i.e if ''x<sub>i</sub>'' is the ''i''th component of '''x''', ''μ<sub>i</sub>'' the ''i''th component of '''μ''', and ''σ<sub>ij</sub>'' the ''ij''th component of '''Σ''', then |
<div style="margin-left: 25em;"> | <div style="margin-left: 25em;"> | ||
| Line 48: | Line 49: | ||
<math>\sigma_{ij} = \mathcal{E}[(x_i - \mu_i)(x_j - \mu_j)] </math> | <math>\sigma_{ij} = \mathcal{E}[(x_i - \mu_i)(x_j - \mu_j)] </math> | ||
</div> | </div> | ||
| + | |||
| + | The covariance matrix '''Σ''' is always symmetric and positive definite which means that the determinant of '''Σ''' is strictly positive. The diagonal elements ''σ<sub>ii</sub>'' are the variances of the respective ''x<sub>i</sub>'' ( i.e., ''σ<sup>2</sup>''), and the off-diagonal elements ''σ<sub>ij</sub>'' are the covariances of ''x<sub>i</sub>'' and ''x<sub>j</sub>''. If ''x<sub>i</sub>'' and ''x<sub>j</sub>'' are statistically independent, then ''σ<sub>ij</sub>'' = 0. If all off-diagonanl elements are zero, ''p''('''x''') reduces to the product of the univariate normal densities for the components of '''x'''. | ||
Revision as of 18:40, 4 April 2013
Discriminant Functions For The Normal Density
Lets begin with the continuous univariate normal or Gaussian density.
$ f_x = \frac{1}{\sqrt{2 \pi} \sigma} \exp \left [- \frac{1}{2} \left ( \frac{x - \mu}{\sigma} \right)^2 \right ] $
for which the expected value of x is
$ \mu = \mathcal{E}[x] =\int\limits_{-\infty}^{\infty} xp(x)\, dx $
and where the expected squared deviation or variance is
$ \sigma^2 = \mathcal{E}[(x- \mu)^2] =\int\limits_{-\infty}^{\infty} (x- \mu)^2 p(x)\, dx $
The univariate normal density is completely specified by two parameters; its mean μ and variance σ2. The function fx can be written as N(μ,σ) which says that x is distributed normally with mean μ and variance σ2. Samples from normal distributions tend to cluster about the mean with a spread related to the standard deviation σ.
For the multivariate normal density in d dimensions, fx is written as
$ f_x = \frac{1}{(2 \pi)^ \frac{d}{2} |\boldsymbol{\Sigma}|^\frac{1}{2}} \exp \left [- \frac{1}{2} (\mathbf{x} -\boldsymbol{\mu})^t\boldsymbol{\Sigma}^{-1} (\mathbf{x} -\boldsymbol{\mu}) \right] $
where x is a d-component column vector, μ is the d-component mean vector, Σ is the d-by-d covariance matrix, and |Σ| and Σ-1 are its determinant and inverse respectively. Also,(x - μ)t denotes the transpose of (x - μ).
and
$ \boldsymbol{\Sigma} = \mathcal{E} \left [(\mathbf{x} - \boldsymbol{\mu})(\mathbf{x} - \boldsymbol{\mu})^t \right] = \int(\mathbf{x} - \boldsymbol{\mu})(\mathbf{x} - \boldsymbol{\mu})^t p(\mathbf{x})\, dx $
where the expected value of a vector or a matrix is found by taking the expected value of the individual components. i.e if xi is the ith component of x, μi the ith component of μ, and σij the ijth component of Σ, then
$ \mu_i = \mathcal{E}[x_i] $
and
$ \sigma_{ij} = \mathcal{E}[(x_i - \mu_i)(x_j - \mu_j)] $
The covariance matrix Σ is always symmetric and positive definite which means that the determinant of Σ is strictly positive. The diagonal elements σii are the variances of the respective xi ( i.e., σ2), and the off-diagonal elements σij are the covariances of xi and xj. If xi and xj are statistically independent, then σij = 0. If all off-diagonanl elements are zero, p(x) reduces to the product of the univariate normal densities for the components of x.
