Example. Mean of i.i.d. random variables

Let $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n} $ be $ M $ jointly distributed i.i.d. random variables with mean $ \mu $ and variance $ \sigma^{2} $ . Let $ \mathbf{Y}_{M}=\frac{1}{M}\sum_{n=0}^{M}\mathbf{X}_{n} $ .

(a) Find the variance of $ \mathbf{Y}_{M} $ .

$ Var\left[\mathbf{Y}_{M}\right]=E\left[\mathbf{Y}_{M}^{2}\right]-\left(E\left[\mathbf{Y}_{M}\right]\right)^{2}. $

$ E\left[\mathbf{Y}_{M}\right]=E\left[\frac{1}{M}\sum_{n=0}^{M}\mathbf{X}_{n}\right]=\frac{1}{M}\sum_{n=0}^{M}E\left[\mathbf{X}_{n}\right]=\frac{1}{M}\cdot M\cdot\mu=\mu. $

$ E\left[\mathbf{Y}_{M}^{2}\right]=E\left[\frac{1}{M^{2}}\sum_{m=1}^{M}\sum_{n=1}^{M}\mathbf{X}_{m}\mathbf{X}_{n}\right]=\frac{1}{M^{2}}\sum_{m=1}^{M}\sum_{n=1}^{M}E\left[\mathbf{X}_{m}\mathbf{X}_{n}\right]. $

Now $ E\left[\mathbf{X}_{m}\mathbf{X}_{n}\right]=\begin{cases} \begin{array}{ll} E\left[\mathbf{X}_{m}^{2}\right] ,m=n\\ E\left[\mathbf{X}_{m}\right]E\left[\mathbf{X}_{n}\right] ,m\neq n \end{array}\end{cases} $ because when $ m\neq n $ , $ \mathbf{X}_{m} $ and $ \mathbf{X}_{n} $ are independent $ \Rightarrow \mathbf{X}_{m} $ and $ \mathbf{X}_{n} $ are uncorrelated.

$ E\left[\mathbf{Y}_{M}^{2}\right]=\frac{1}{M^{2}}\left[M\left(\mu^{2}+\sigma^{2}\right)+M\left(M-1\right)\mu^{2}\right]=\frac{\left(\mu^{2}+\sigma^{2}\right)+\left(M-1\right)\mu^{2}}{M}=\frac{M\mu^{2}+\sigma^{2}}{M}. $

$ Var\left[\mathbf{Y}_{M}\right]=\frac{M\mu^{2}+\sigma^{2}-M\mu^{2}}{M}=\frac{\sigma^{2}}{M}. $

(b) Now assume that the $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n} $ are identically distributed with with mean $ \mu $ and variance $ \sigma^{2} $ , but they are only correlated rather than independent. Find the variance of $ \mathbf{Y}_{M} $ .

Again, $ Var\left[\mathbf{Y}_{M}\right]=\frac{\sigma^{2}}{M} $ , because only uncorrelatedness was used in part (a).