7.3 QE 2001 August
1. (10 Points)
Consider the following random experiment: A fair coin is repeatedly tossed until the same outcome (H or T) appears twice in a row.
(a)
What is the probability that this experiment terminates on or before the seventh coin toss?
Let N be the number of toss until the same outcome appears twice in a row.
\begin{center} \begin{tabular}{|c|c|c|c|c|} \hline $N$th & $\left(N-1\right)$th & $\left(N-2\right)$th & $\left(N-3\right)$th & $\cdots$\tabularnewline \hline \hline H & H & T & H & $\cdots$\tabularnewline \hline T & T & H & T & $\cdots$\tabularnewline \hline \end{tabular} \par\end{center}
$ P\left(\left\{ N=n\right\} \right)=\frac{2}{2^{n}}=\frac{1}{2^{n-1}}\text{ for }n\geq2. $
$ P\left(\left\{ N\leq7\right\} \right)=\sum_{k=2}^{7}\frac{1}{2^{k-1}}=\sum_{k=1}^{6}\left(\frac{1}{2}\right)^{k}=\frac{\frac{1}{2}\left(1-\left(\frac{1}{2}\right)^{6}\right)}{1-\frac{1}{2}}=1-\frac{1}{64}=\frac{63}{64}. $
(b)
What is the probability that this experiment terminates with an even number of coin tosses?
$ P\left(\left\{ N\text{ is even}\right\} \right)=\sum_{k=1}^{\infty}\frac{1}{2^{2k-1}}=2\sum_{k=1}^{\infty}\left(\frac{1}{4}\right)^{k}=2\cdot\frac{\frac{1}{4}}{1-\frac{1}{4}}=2\cdot\frac{1}{3}=\frac{2}{3}. $
2. (25 Points)
Let $ \mathbf{X} $ and $ \mathbf{Y} $ be independent Poisson random variables with mean $ \lambda $ and $ \mu $ , respectively. Let $ \mathbf{Z} $ be a new random variable defined as $ \mathbf{Z}=\mathbf{X}+\mathbf{Y}. $
Note
This problem is identical to the example [CS1AdditionOfTwoIndependentPoissonRV].
(a)
Find the probability mass function (pmf) of $ \mathbf{Z} $ .
(b)
Find the conditional probability mass function (pmf) of $ \mathbf{X} $ conditional on the event $ \left\{ \mathbf{Z}=n\right\} $ . Identify the type of pmf that this is, and fully specify its parameters.
3. (30 Points)
Let $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ be a sequence of random variables that are not necessarily statistically independent, but that each have identical mean $ \mu $ and variance $ \sigma^{2} $ . Let $ \mathbf{Y}_{1},\cdots,\mathbf{Y}_{n},\cdots $ be a sequence of random variable with $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}. $
(a)
Given that $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ are uncorrelated, determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.
$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}. $
$ E\left[\mathbf{Y}_{n}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\mu. $
$ E\left[\mathbf{Y}_{n}^{2}\right] $
$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}=\frac{\sigma^{2}}{n}+\mu^{2}-2\mu\cdot\mu+\mu^{2}=\frac{\sigma^{2}}{n}. \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $
Another approach
E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]
\lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0.
(b)
Given that the covariance between \mathbf{X}_{j} and \mathbf{X}_{k} is given by cov\left(\mathbf{X}_{j},\mathbf{X}_{k}\right)=\begin{cases} \begin{array}{lll} \sigma^{2} & & \textnormal{, for }j=k\\ r\sigma^{2} & & \textnormal{, for }\left|j-k\right|=1\\ 0 & & \textnormal{, elsewhere, } \end{array}\end{cases} where -1\leq r\leq1 , determine whether or not \left\{ \mathbf{Y}_{n}\right\} converges to \mu in the mean square sense.
E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]
\lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}\right)=0.
Thus, \mathbf{Y}_{n} converges in the mean square sense to \mu .
4. (35 Points)
Let \left\{ t_{k}\right\} be the set of Poisson points corresponding to a homogeneous Poisson process with parameters \lambda on the real line such that if \mathbf{N}\left(t_{1},t_{2}\right) is defined as the number of points in the interval \left[t_{1},t_{2}\right) , then P\left(\left\{ N\left(t_{1},t_{2}\right)=k\right\} \right)=\frac{\left[\lambda\left(t_{2}-t_{1}\right)\right]^{k}e^{-\lambda\left(t_{2}-t_{1}\right)}}{k!}\;,\qquad k=0,1,2,\cdots,\; t_{2}>t_{1}\geq0. Let \mathbf{X}\left(t\right)=\mathbf{N}\left(0,t\right) be the Poisson counting process for t>0 (note that \mathbf{X}\left(0\right)=0 ).
(a)
Find the (first order) characteristic function of \mathbf{X}\left(t\right) .
\Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{\infty}e^{i\omega k}\frac{\left(\lambda t\right)^{k}e^{-\lambda t}}{k!}=e^{-\lambda t}\sum_{k=0}^{\infty}\frac{\left(\lambda te^{i\omega}\right)^{k}}{k!}=e^{-\lambda t}e^{\lambda te^{i\omega}}=e^{-\lambda t\left(1-e^{i\omega}\right)}.
(b)
Find the mean and variance of \mathbf{X}\left(t\right) .
E\left[\mathbf{X}\left(t\right)\right]
E\left[\mathbf{X}^{2}\left(t\right)\right]
Var\left[\mathbf{X}\left(t\right)\right]=E\left[\mathbf{X}^{2}\left(t\right)\right]-\left(E\left[\mathbf{X}\left(t\right)\right]\right)^{2}=\left(\lambda t\right)^{2}+\lambda t-\left(\lambda t\right)^{2}=\lambda t.
(c)
Deriven an expression for the autocorrelation function of \mathbf{X}\left(t\right) .
R_{\mathbf{XX}}\left(t_{1},t_{2}\right)
(d)
Assuming that t_{2}>t_{1} , find an expression for P\left(\left\{ \mathbf{X}\left(t_{1}\right)=m\right\} \cap\left\{ \mathbf{X}\left(t_{2}\right)=n\right\} \right) , for all m=0,1,2,\cdots and n=0,1,2,\cdots .
P\left(\left\{ \mathbf{X}\left(t_{1}\right)=m\right\} \cap\left\{ \mathbf{X}\left(t_{2}\right)=n\right\} \right)
