7.3 QE 2001 August

1. (10 Points)

Consider the following random experiment: A fair coin is repeatedly tossed until the same outcome (H or T) appears twice in a row.

(a)

What is the probability that this experiment terminates on or before the seventh coin toss?

Let N be the number of toss until the same outcome appears twice in a row.

$ P\left(\left\{ N=n\right\} \right)=\frac{2}{2^{n}}=\frac{1}{2^{n-1}}\text{ for }n\geq2. $

$ P\left(\left\{ N\leq7\right\} \right)=\sum_{k=2}^{7}\frac{1}{2^{k-1}}=\sum_{k=1}^{6}\left(\frac{1}{2}\right)^{k}=\frac{\frac{1}{2}\left(1-\left(\frac{1}{2}\right)^{6}\right)}{1-\frac{1}{2}}=1-\frac{1}{64}=\frac{63}{64}. $

(b)

What is the probability that this experiment terminates with an even number of coin tosses?

$ P\left(\left\{ N\text{ is even}\right\} \right)=\sum_{k=1}^{\infty}\frac{1}{2^{2k-1}}=2\sum_{k=1}^{\infty}\left(\frac{1}{4}\right)^{k}=2\cdot\frac{\frac{1}{4}}{1-\frac{1}{4}}=2\cdot\frac{1}{3}=\frac{2}{3}. $

2. (25 Points)

Let $ \mathbf{X} $ and $ \mathbf{Y} $ be independent Poisson random variables with mean $ \lambda $ and $ \mu $ , respectively. Let $ \mathbf{Z} $ be a new random variable defined as $ \mathbf{Z}=\mathbf{X}+\mathbf{Y}. $

Note

This problem is identical to the example: Addition of two independent Poisson random variables.

(a)

Find the probability mass function (pmf) of $ \mathbf{Z} $ .

(b)

Find the conditional probability mass function (pmf) of $ \mathbf{X} $ conditional on the event $ \left\{ \mathbf{Z}=n\right\} $ . Identify the type of pmf that this is, and fully specify its parameters.

3. (30 Points)

Let $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ be a sequence of random variables that are not necessarily statistically independent, but that each have identical mean $ \mu $ and variance $ \sigma^{2} $ . Let $ \mathbf{Y}_{1},\cdots,\mathbf{Y}_{n},\cdots $ be a sequence of random variable with $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}. $

(a)

Given that $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ are uncorrelated, determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}. $

$ E\left[\mathbf{Y}_{n}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\mu. $

$ E\left[\mathbf{Y}_{n}^{2}\right]=E\left[\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}\mathbf{X}_{k}\mathbf{X}_{l}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\mathbf{X}_{k}\mathbf{X}_{l}\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\mathbf{X}_{k}\right]E\left[\mathbf{X}_{l}\right] $$ =\frac{1}{n}\left(\mu^{2}+\sigma^{2}\right)+\frac{n\left(n-1\right)}{n^{2}}\mu^{2}=\frac{1}{n}\mu^{2}+\frac{1}{n}\sigma^{2}+\mu^{2}-\frac{1}{n}\mu^{2} $$ =\frac{\sigma^{2}}{n}+\mu^{2}. $

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}=\frac{\sigma^{2}}{n}+\mu^{2}-2\mu\cdot\mu+\mu^{2}=\frac{\sigma^{2}}{n}. $ $ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $

Another approach

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\left|\frac{1}{n}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right|^{2}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\mathbf{X}_{k}-\mu\right]E\left[\mathbf{X}_{l}-\mu\right] $$ =\frac{1}{n^{2}}\cdot n\cdot\sigma^{2}+\frac{1}{n^{2}}\cdot n\left(n-1\right)\cdot0^{2}=\frac{\sigma^{2}}{n}. $

$ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $

(b)

Given that the covariance between $ \mathbf{X}_{j} $ and $ \mathbf{X}_{k} $ is given by
$ cov\left(\mathbf{X}_{j},\mathbf{X}_{k}\right)=\begin{cases} \begin{array}{lll} \sigma^{2} \text{, for }j=k\\ r\sigma^{2} \text{, for }\left|j-k\right|=1\\ 0 \text{, elsewhere, } \end{array}\end{cases} $
where $ -1\leq r\leq1 $ , determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\left|\frac{1}{n}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right|^{2}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}. $

$ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}\right)=0. $

Thus, $ \mathbf{Y}_{n} $ converges in the mean square sense to $ \mu $ .

4. (35 Points)

Let $ \left\{ t_{k}\right\} $ be the set of Poisson points corresponding to a homogeneous Poisson process with parameters $ \lambda $ on the real line such that if $ \mathbf{N}\left(t_{1},t_{2}\right) $ is defined as the number of points in the interval $ \left[t_{1},t_{2}\right) $ , then $ P\left(\left\{ N\left(t_{1},t_{2}\right)=k\right\} \right)=\frac{\left[\lambda\left(t_{2}-t_{1}\right)\right]^{k}e^{-\lambda\left(t_{2}-t_{1}\right)}}{k!}\;,\qquad k=0,1,2,\cdots,\; t_{2}>t_{1}\geq0. Let \mathbf{X}\left(t\right)=\mathbf{N}\left(0,t\right) $ be the Poisson counting process for $ t>0 $ (note that $ \mathbf{X}\left(0\right)=0 $ ).

(a)

Find the (first order) characteristic function of $ \mathbf{X}\left(t\right) $ .

$ \Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{\infty}e^{i\omega k}\frac{\left(\lambda t\right)^{k}e^{-\lambda t}}{k!}=e^{-\lambda t}\sum_{k=0}^{\infty}\frac{\left(\lambda te^{i\omega}\right)^{k}}{k!}=e^{-\lambda t}e^{\lambda te^{i\omega}}=e^{-\lambda t\left(1-e^{i\omega}\right)}. $

(b)

Find the mean and variance of $ \mathbf{X}\left(t\right) $ .

$ E\left[\mathbf{X}\left(t\right)\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\biggl|_{i\omega=0}=\frac{d}{di\omega}e^{-\lambda t}e^{\lambda te^{i\omega}}\biggl|_{i\omega=0}=e^{-\lambda t}\cdot\frac{d}{di\omega}e^{\lambda te^{i\omega}}\biggl|_{i\omega=0} $$ =e^{-\lambda t}\cdot e^{\lambda te^{i\omega}}\cdot\lambda te^{i\omega}\biggl|_{i\omega=0}=e^{-\lambda t}\cdot e^{\lambda t}\cdot\lambda t=\lambda t. $

$ E\left[\mathbf{X}^{2}\left(t\right)\right]=\frac{d}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\biggl|_{i\omega=0}=\frac{d}{di\omega}\lambda te^{-\lambda t}e^{\lambda te^{i\omega}}e^{i\omega}\biggl|_{i\omega=0} $$ =\lambda te^{-\lambda t}\cdot\frac{d}{di\omega}e^{\lambda te^{i\omega}}e^{i\omega}\biggl|_{i\omega=0} $$ =\lambda te^{-\lambda t}\left(e^{\lambda te^{i\omega}}\lambda te^{i\omega}e^{i\omega}+e^{\lambda te^{i\omega}}e^{i\omega}\right)\biggl|_{i\omega=0} $$ =\lambda te^{-\lambda t}\left(\lambda te^{\lambda te^{i\omega}}e^{2i\omega}+e^{\lambda te^{i\omega}}e^{i\omega}\right)\biggl|_{i\omega=0}=\lambda te^{-\lambda t}\left(\lambda te^{\lambda t}+e^{\lambda t}\right) $$ =\lambda t\left(\lambda t+1\right)=\left(\lambda t\right)^{2}+\lambda t. $

$ Var\left[\mathbf{X}\left(t\right)\right]=E\left[\mathbf{X}^{2}\left(t\right)\right]-\left(E\left[\mathbf{X}\left(t\right)\right]\right)^{2}=\left(\lambda t\right)^{2}+\lambda t-\left(\lambda t\right)^{2}=\lambda t. $

(c)

Deriven an expression for the autocorrelation function of $ \mathbf{X}\left(t\right) $ .

$ R_{\mathbf{XX}}\left(t_{1},t_{2}\right) $

(d)

Assuming that $ t_{2}>t_{1} $ , find an expression for $ P\left(\left\{ \mathbf{X}\left(t_{1}\right)=m\right\} \cap\left\{ \mathbf{X}\left(t_{2}\right)=n\right\} \right) $ , for all $ m=0,1,2,\cdots $ and $ n=0,1,2,\cdots $ .

$ P\left(\left\{ \mathbf{X}\left(t_{1}\right)=m\right\} \cap\left\{ \mathbf{X}\left(t_{2}\right)=n\right\} \right) $

Back to ECE600

Back to my ECE 600 QE page

Back to the general ECE PHD QE page (for problem discussion)