Difference between revisions of "ECE 600 Weak law of large numbers" - Rhea

Revision as of 12:29, 22 November 2010

2.3 Weak law of large numbers

Let $ \left\{ \mathbf{X}_{n}\right\} $ be a sequence of $ i.i.d. $ random variables with mean $ \mu $ and variance $ \sigma^{2} $ . Define $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k},\quad n=1,2,\cdots $ . Then for any $ \epsilon>0 $ , $ p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|>\epsilon\right\} \right)\rightarrow0 $ as $ n\rightarrow\infty $(convergence in probability).

$ \mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty. $

Proof

$ E\left[\mathbf{Y}_{n}\right]=E\left[\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\frac{1}{n}\cdot\left(n\mu\right)=\mu. $

$ Var\left[\mathbf{Y}_{n}\right] $

By the Chebyshev inequality,

$ p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|\geq\epsilon\right\} \right)\leq\frac{\sigma^{2}}{n\epsilon^{2}}\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow0. $

$ \Longrightarrow\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.\blacksquare $

Note

You can show this is true as long as the mean exists. The variance need not exist. Proof for this is harder and not responsible for this.

Note

There are also stronger forms of the law of large numbers. Strong one uses coveriance $ \left(a.e.\right) $ as well as weak one uses coveriance $ \left(p\right) $ .

Back to ECE600

Back to Sequences of Random Variables