2.5 Central limit theorem
Let $ \left\{ \mathbf{X}_{n}\right\} $ be a sequence of $ i.i.d. $ random vectors with mean $ \mu $ and variance $ \sigma^{2} $ , such that $ 0<\sigma^{2}<\infty $ . Then if
$ \mathbf{Z}_{n}\triangleq\frac{\left(\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n}\right)-n\mu}{\sigma\sqrt{n}} $
then $ \mathbf{Z}_{n} $ converges in distribution to a random variable $ \mathbf{Z} $ that is Gaussian with mean 0 and variance 1 .
In fact, $ \mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n} $ has the mean $ n\mu $ and the variance $ n\sigma^{2} $ .
i.e. $ F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx $ as $ n\rightarrow\infty , \forall z\in\mathbf{R} $ .
Proof
We will show that $ \Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow e^{-\frac{1}{2}\omega^{2}} , \forall\omega\in\mathbf{R} $ .
$ \Phi_{\mathbf{Z}_{n}}\left(\omega\right) $
We can expand the exponential as a power series (in $ \omega $ about $ \omega=0 $ ).
$ E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right] $
It can be shown that $ \frac{E\left[R\left(\omega\right)\right]}{\omega^{2}/2n}\longrightarrow0 $ as $ n\rightarrow\infty , \forall\omega\in\mathbf{R} $ .
Thus we have
$ E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]=1-\frac{\omega^{2}}{2n}+E\left[R\left(\omega\right)\right]\backsimeq1-\frac{\omega^{2}}{2n}\text{ as }n\rightarrow\infty. $
$ \Phi_{\mathbf{Z}_{n}}\left(\omega\right)\backsimeq\left(1-\frac{\omega^{2}}{2n}\right)^{n}=\left(1+\frac{\left(-\omega^{2}/2\right)}{n}\right)^{n}\longrightarrow e^{-\omega^{2}/2}\text{ as }n\rightarrow\infty. $
$ \therefore\Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow e^{-\omega^{2}/2}\Longrightarrow F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx. $
We have “proved” the central limit theorem for $ i.i.d. $ random variables. It can be proven under much more general circumstances (e.g. sums of correlated sequences of random variables). Lindeberg-Feller central limit theorem works for some classes of correlated random variables.
Gaussian normalization
Gaussian distribution $ \left(\mathbf{X}\right) $ with $ \mu $ and $ \sigma^{2} \Longrightarrow $ Standard normal distribution $ \left(\mathbf{Z}\right) $ that has mean 0 and variance 1
$ \mathbf{Z}=\frac{\mathbf{X}-\mu}{\sigma}\text{ and }\mathbf{X}=\sigma\mathbf{Z}+\mu. $
