| Line 1: | Line 1: | ||
| − | 2.5 Central limit theorem | + | 2.5 Central limit theorem |
| − | Let <math>\left\{ \mathbf{X}_{n}\right\}</math> | + | Let <math>\left\{ \mathbf{X}_{n}\right\}</math> be a sequence of <span class="texhtml">''i''.''i''.''d''.</span> random vectors with mean <span class="texhtml">μ</span> and variance <span class="texhtml">σ<sup>2</sup></span> , such that <span class="texhtml">0 < σ<sup>2</sup> < ∞</span> . Then if |
<math>\mathbf{Z}_{n}\triangleq\frac{\left(\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n}\right)-n\mu}{\sigma\sqrt{n}}</math> | <math>\mathbf{Z}_{n}\triangleq\frac{\left(\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n}\right)-n\mu}{\sigma\sqrt{n}}</math> | ||
| − | then <math>\mathbf{Z}_{n}</math> | + | then <math>\mathbf{Z}_{n}</math> converges in distribution to a random variable <math>\mathbf{Z}</math> that is Gaussian with mean 0 and variance 1 . |
| − | In fact, <math>\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n}</math> | + | In fact, <math>\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n}</math> has the mean <span class="texhtml">''n''μ</span> and the variance <span class="texhtml">''n''σ<sup>2</sup></span> . |
| − | i.e. <math>F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx</math> | + | i.e. <math>F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx</math> as <math>n\rightarrow\infty , \forall z\in\mathbf{R}</math> . |
| − | Proof | + | Proof |
| − | We will show that <math>\Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow e^{-\frac{1}{2}\omega^{2}} , \forall\omega\in\mathbf{R}</math> . | + | We will show that <math>\Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow e^{-\frac{1}{2}\omega^{2}} , \forall\omega\in\mathbf{R}</math> . |
| − | <math>\Phi_{\mathbf{Z}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{Z}}\right]=E\left[\exp\left\{ \frac{i\omega}{\sigma\sqrt{n}}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right\} \right]=E\left[\prod_{k=1}^{n}e^{i\omega\left(\mathbf{X}_{k}-\mu\right)/\sigma\sqrt{n}}\right]=\prod_{k=1}^{n}E\left[e^{i\omega\left(\mathbf{X}_{k}-\mu\right)/\sigma\sqrt{n}}\right]=\cdots=\left(E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]\right)^{n}.</math> | + | <math>\Phi_{\mathbf{Z}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{Z}}\right]=E\left[\exp\left\{ \frac{i\omega}{\sigma\sqrt{n}}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right\} \right]=E\left[\prod_{k=1}^{n}e^{i\omega\left(\mathbf{X}_{k}-\mu\right)/\sigma\sqrt{n}}\right]</math><math>=\prod_{k=1}^{n}E\left[e^{i\omega\left(\mathbf{X}_{k}-\mu\right)/\sigma\sqrt{n}}\right]=\cdots=\left(E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]\right)^{n}.</math> |
| − | We can expand the exponential as a power series (in < | + | We can expand the exponential as a power series (in <span class="texhtml">ω</span> about <span class="texhtml">ω = 0</span> ). |
<math>E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]</math> | <math>E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]</math> | ||
| − | It can be shown that <math>\frac{E\left[R\left(\omega\right)\right]}{\omega^{2}/2n}\longrightarrow0</math> | + | It can be shown that <math>\frac{E\left[R\left(\omega\right)\right]}{\omega^{2}/2n}\longrightarrow0</math> as <math>n\rightarrow\infty , \forall\omega\in\mathbf{R}</math> . |
| − | Thus we have | + | Thus we have |
<math>E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]=1-\frac{\omega^{2}}{2n}+E\left[R\left(\omega\right)\right]\backsimeq1-\frac{\omega^{2}}{2n}\text{ as }n\rightarrow\infty.</math> | <math>E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]=1-\frac{\omega^{2}}{2n}+E\left[R\left(\omega\right)\right]\backsimeq1-\frac{\omega^{2}}{2n}\text{ as }n\rightarrow\infty.</math> | ||
| Line 31: | Line 31: | ||
<math>\therefore\Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow e^{-\omega^{2}/2}\Longrightarrow F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx.</math> | <math>\therefore\Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow e^{-\omega^{2}/2}\Longrightarrow F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx.</math> | ||
| − | We have “proved” the central limit theorem for < | + | We have “proved” the central limit theorem for <span class="texhtml">''i''.''i''.''d''.</span> random variables. It can be proven under much more general circumstances (e.g. sums of correlated sequences of random variables). Lindeberg-Feller central limit theorem works for some classes of correlated random variables. |
| − | Gaussian normalization | + | Gaussian normalization |
| − | Gaussian distribution <math>\left(\mathbf{X}\right)</math> | + | Gaussian distribution <math>\left(\mathbf{X}\right)</math> with <span class="texhtml">μ</span> and <span class="texhtml">σ<sup>2</sup>⇒</span> Standard normal distribution <math>\left(\mathbf{Z}\right)</math> that has mean 0 and variance 1 |
<math>\mathbf{Z}=\frac{\mathbf{X}-\mu}{\sigma}\text{ and }\mathbf{X}=\sigma\mathbf{Z}+\mu.</math> | <math>\mathbf{Z}=\frac{\mathbf{X}-\mu}{\sigma}\text{ and }\mathbf{X}=\sigma\mathbf{Z}+\mu.</math> | ||
Revision as of 11:33, 17 November 2010
2.5 Central limit theorem
Let $ \left\{ \mathbf{X}_{n}\right\} $ be a sequence of i.i.d. random vectors with mean μ and variance σ2 , such that 0 < σ2 < ∞ . Then if
$ \mathbf{Z}_{n}\triangleq\frac{\left(\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n}\right)-n\mu}{\sigma\sqrt{n}} $
then $ \mathbf{Z}_{n} $ converges in distribution to a random variable $ \mathbf{Z} $ that is Gaussian with mean 0 and variance 1 .
In fact, $ \mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{n} $ has the mean nμ and the variance nσ2 .
i.e. $ F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx $ as $ n\rightarrow\infty , \forall z\in\mathbf{R} $ .
Proof
We will show that $ \Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow e^{-\frac{1}{2}\omega^{2}} , \forall\omega\in\mathbf{R} $ .
$ \Phi_{\mathbf{Z}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{Z}}\right]=E\left[\exp\left\{ \frac{i\omega}{\sigma\sqrt{n}}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right\} \right]=E\left[\prod_{k=1}^{n}e^{i\omega\left(\mathbf{X}_{k}-\mu\right)/\sigma\sqrt{n}}\right] $$ =\prod_{k=1}^{n}E\left[e^{i\omega\left(\mathbf{X}_{k}-\mu\right)/\sigma\sqrt{n}}\right]=\cdots=\left(E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]\right)^{n}. $
We can expand the exponential as a power series (in ω about ω = 0 ).
$ E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right] $
It can be shown that $ \frac{E\left[R\left(\omega\right)\right]}{\omega^{2}/2n}\longrightarrow0 $ as $ n\rightarrow\infty , \forall\omega\in\mathbf{R} $ .
Thus we have
$ E\left[e^{i\omega\left(\mathbf{X}-\mu\right)/\sigma\sqrt{n}}\right]=1-\frac{\omega^{2}}{2n}+E\left[R\left(\omega\right)\right]\backsimeq1-\frac{\omega^{2}}{2n}\text{ as }n\rightarrow\infty. $
$ \Phi_{\mathbf{Z}_{n}}\left(\omega\right)\backsimeq\left(1-\frac{\omega^{2}}{2n}\right)^{n}=\left(1+\frac{\left(-\omega^{2}/2\right)}{n}\right)^{n}\longrightarrow e^{-\omega^{2}/2}\text{ as }n\rightarrow\infty. $
$ \therefore\Phi_{\mathbf{Z}_{n}}\left(\omega\right)\longrightarrow e^{-\omega^{2}/2}\Longrightarrow F_{\mathbf{Z}_{n}}\left(z\right)\longrightarrow\mathbf{\Phi}\left(z\right)=\int_{-\infty}^{z}\frac{1}{\sqrt{2\pi}}e^{-x^{2}/2}dx. $
We have “proved” the central limit theorem for i.i.d. random variables. It can be proven under much more general circumstances (e.g. sums of correlated sequences of random variables). Lindeberg-Feller central limit theorem works for some classes of correlated random variables.
Gaussian normalization
Gaussian distribution $ \left(\mathbf{X}\right) $ with μ and σ2⇒ Standard normal distribution $ \left(\mathbf{Z}\right) $ that has mean 0 and variance 1
$ \mathbf{Z}=\frac{\mathbf{X}-\mu}{\sigma}\text{ and }\mathbf{X}=\sigma\mathbf{Z}+\mu. $
