Communication, Networking, Signal and Image Processing (CS)
Question 1: Probability and Random Processes
August 2001
3. (30 Points)
Let $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ be a sequence of random variables that are not necessarily statistically independent, but that each have identical mean $ \mu $ and variance $ \sigma^{2} $ . Let $ \mathbf{Y}_{1},\cdots,\mathbf{Y}_{n},\cdots $ be a sequence of random variable with $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}. $
(a)
Given that $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ are uncorrelated, determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.
(b)
Given that the covariance between $ \mathbf{X}_{j} $ and $ \mathbf{X}_{k} $ is given by
$ cov\left(\mathbf{X}_{j},\mathbf{X}_{k}\right)=\begin{cases} \begin{array}{lll} \sigma^{2} \text{, for }j=k\\ r\sigma^{2} \text{, for }\left|j-k\right|=1\\ 0 \text{, elsewhere, } \end{array}\end{cases} $
where $ -1\leq r\leq1 $ , determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.
Solution 1
(a)
$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}. $
$ E\left[\mathbf{Y}_{n}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\mu. $
$ E\left[\mathbf{Y}_{n}^{2}\right]=E\left[\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}\mathbf{X}_{k}\mathbf{X}_{l}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\mathbf{X}_{k}\mathbf{X}_{l}\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\mathbf{X}_{k}\right]E\left[\mathbf{X}_{l}\right] $$ =\frac{1}{n}\left(\mu^{2}+\sigma^{2}\right)+\frac{n\left(n-1\right)}{n^{2}}\mu^{2}=\frac{1}{n}\mu^{2}+\frac{1}{n}\sigma^{2}+\mu^{2}-\frac{1}{n}\mu^{2} $$ =\frac{\sigma^{2}}{n}+\mu^{2}. $
$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}=\frac{\sigma^{2}}{n}+\mu^{2}-2\mu\cdot\mu+\mu^{2}=\frac{\sigma^{2}}{n}. $ $ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $
(b)
Given that the covariance between $ \mathbf{X}_{j} $ and $ \mathbf{X}_{k} $ is given by
$ cov\left(\mathbf{X}_{j},\mathbf{X}_{k}\right)=\begin{cases} \begin{array}{lll} \sigma^{2} \text{, for }j=k\\ r\sigma^{2} \text{, for }\left|j-k\right|=1\\ 0 \text{, elsewhere, } \end{array}\end{cases} $
where $ -1\leq r\leq1 $ , determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.
$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\left|\frac{1}{n}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right|^{2}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}. $
$ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}\right)=0. $
Thus, $ \mathbf{Y}_{n} $ converges in the mean square sense to $ \mu $ .
Solution 2
(a)
$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\left|\frac{1}{n}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right|^{2}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\mathbf{X}_{k}-\mu\right]E\left[\mathbf{X}_{l}-\mu\right] $$ =\frac{1}{n^{2}}\cdot n\cdot\sigma^{2}+\frac{1}{n^{2}}\cdot n\left(n-1\right)\cdot0^{2}=\frac{\sigma^{2}}{n}. $
$ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $
