2.3 Weak law of large numbers
Let $ \left\{ \mathbf{X}_{n}\right\} $ be a sequence of $ i.i.d. $ random variables with mean $ \mu $ and variance $ \sigma^{2} $ . Define $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k},\quad n=1,2,\cdots $ . Then for any $ \epsilon>0 $ , $ p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|>\epsilon\right\} \right)\rightarrow0 $ as $ n\rightarrow\infty $(convergence in probability).
$ \mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty. $
Proof
$ E\left[\mathbf{Y}_{n}\right]=E\left[\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\frac{1}{n}\cdot\left(n\mu\right)=\mu. $
$ Var\left[\mathbf{Y}_{n}\right] $
By the Chebyshev inequality,
$ p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|\geq\epsilon\right\} \right)\leq\frac{\sigma^{2}}{n\epsilon^{2}}\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow0. $
$ \Longrightarrow\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.\blacksquare $
Note
You can show this is true as long as the mean exists. The variance need not exist. Proof for this is harder and not responsible for this.
Note
There are also stronger forms of the law of large numbers. Strong one uses coveriance $ \left(a.e.\right) $ as well as weak one uses coveriance $ \left(p\right) $ .
