7.12 QE 2006 August
1
Let $ \mathbf{U}_{n} $ be a sequence of independent, identically distributed zero-mean, unit-variance Gaussian random variables. The sequence $ \mathbf{X}_{n} $ , $ n\geq1 $ , is given by $ \mathbf{X}_{n}=\frac{1}{2}\mathbf{U}_{n}+\left(\frac{1}{2}\right)^{2}\mathbf{U}_{n-1}+\cdots+\left(\frac{1}{2}\right)^{n}\mathbf{U}_{1}. $
(a) (15 points)
Find the mean and variance of $ \mathbf{X}_{n} $ .
i) Find $ E\left[\mathbf{X}_{n}\right] $
$ \mathbf{X}_{n}=\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{k+1}\mathbf{U}_{n-k}. E\left[\mathbf{X}_{n}\right]=E\left(\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{k+1}\mathbf{U}_{n-k}\right)=\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{k+1}E\left[\mathbf{U}_{n-k}\right]=0. $
ii) Find $ E\left[\mathbf{X}_{n}^{2}\right] $
$ E\left[\mathbf{X}_{n}^{2}\right]=E\left[\left(\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{k+1}\mathbf{U}_{n-k}\right)^{2}\right]=E\left[\sum_{k=0}^{n-1}\sum_{j=0}^{n-1}\left(\frac{1}{2}\right)^{k+1}\left(\frac{1}{2}\right)^{j+1}\mathbf{U}_{n-k}\mathbf{U}_{n-j}\right] $$ =E\left[\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{2k+2}\mathbf{U}_{n-k}^{2}+\underset{k\neq j}{\sum_{k=0}^{n-1}\sum_{j=0}^{n-1}}\left(\frac{1}{2}\right)^{k+1}\left(\frac{1}{2}\right)^{j+1}\mathbf{U}_{n-k}\mathbf{U}_{n-j}\right] $$ =\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{2k+2}E\left[\mathbf{U}_{n-k}^{2}\right]+\underset{k\neq j}{\sum_{k=0}^{n-1}\sum_{j=0}^{n-1}}\left(\frac{1}{2}\right)^{k+1}\left(\frac{1}{2}\right)^{j+1}E\left[\mathbf{U}_{n-k}\right]E\left[\mathbf{U}_{n-j}\right] $$ =\sum_{k=0}^{n-1}\left(\frac{1}{2}\right)^{2k+2}=\sum_{k=1}^{n}\left(\frac{1}{2}\right)^{2k}=\frac{\left(\frac{1}{2}\right)^{2}\left(1-\left(\frac{1}{2}\right)^{2n}\right)}{1-\left(\frac{1}{2}\right)^{2}}=\frac{1}{3}\left(1-\left(\frac{1}{2}\right)^{2n}\right). $
iii) Find $ Var\left[\mathbf{X}_{n}\right] $
$ Var\left[\mathbf{X}_{n}\right]=E\left[\mathbf{X}_{n}^{2}\right]-\left(E\left[\mathbf{X_{n}}\right]\right)^{2}=\frac{1}{3}\left(1-\left(\frac{1}{2}\right)^{2n}\right). $
(b) (15 points)
Find the characteristic function of $ \mathbf{X}_{n} $ .
Since $ \mathbf{U}_{n} $ is a sequence of i.i.d. Gaussian random variables, $ \mathbf{X}_{n} $ is a sequence of Gaussian random variables with zero mean and variance $ \sigma_{\mathbf{X}_{n}}^{2}=\frac{1}{3}\left(1-\left(\frac{1}{2}\right)^{2n}\right) $ . Hence the characteristic function of $ \mathbf{X}_{n} $ is $ \Phi_{\mathbf{X}_{n}}\left(\omega\right)=\exp\left(i\mu_{\mathbf{X}_{n}}\omega-\frac{1}{2}\sigma_{\mathbf{X}_{n}}^{2}\omega^{2}\right)=\exp\left(-\frac{\omega^{2}}{6}\left(1-\left(\frac{1}{2}\right)^{2n}\right)\right). $
(c) (10 points)
Does the sequence $ \mathbf{X}_{n} $ converge in distribution? A simple yes or no answer is not sufficient. You must justify your answer.
$ \Phi=F_{\mathbf{X}_{n}}\left(x\right)=\int_{-\infty}^{x}\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}_{n}}}\exp\left(-\frac{x'^{2}}{2\sigma_{\mathbf{X}_{n}}^{2}}\right)dx' $ where $ \sigma_{\mathbf{X}_{n}}^{2}=\frac{1}{3}\left(1-\left(\frac{1}{2}\right)^{2n}\right) $ .
Since $ \lim_{n\rightarrow\infty}\sigma_{\mathbf{X}_{n}}^{2}=\frac{1}{3} , \lim_{n\rightarrow\infty}F_{\mathbf{X}_{n}}=\int_{-\infty}^{x}\frac{1}{\sqrt{\frac{2\pi}{3}}}\exp\left(-\frac{x'^{2}}{2\sigma_{\mathbf{X}_{n}}^{2}}\right)dx'=F_{\mathbf{X}}\left(x\right). $
$ \therefore $ The squance $ \mathbf{X}_{n} $ converges in distribution.
2
Let $ \Phi $ be the standard normal distribution, i.e., the distribution function of a zero-mean, unit-variance Gaussian random variable. Let $ \mathbf{X} $ be a normal random variable with mean $ \mu $ and variance 1 . We want to find $ E\left[\Phi\left(\mathbf{X}\right)\right] $ .
(a) (10 points)
First show that $ E\left[\Phi\left(\mathbf{X}\right)\right]=P\left(\mathbf{Z}\leq\mathbf{X}\right) $ , where $ \mathbf{Z} $ is a standard normal random variable independent of $ \mathbf{X} $ . Hint: Use an intermediate random variable $ \mathbf{I} $ defined as
$ \mathbf{I}=\left\{ \begin{array}{lll} 1 & & \textrm{if }\mathbf{Z}\leq\mathbf{X}\\ 0 & & \textrm{if }\mathbf{Z}>\mathbf{X}. \end{array}\right. $
$ P\left(\mathbf{Z}\leq\mathbf{X}\right)=\int_{-\infty}^{\infty}P\left(\mathbf{Z}\leq x|\mathbf{X}=x\right)\cdot f_{\mathbf{X}}\left(x\right)dx=\int_{-\infty}^{\infty}\Phi\left(x\right)\cdot f_{\mathbf{X}}\left(x\right)dx=E\left[\Phi\left(\mathbf{X}\right)\right]. $
(b) (10 points)
Now use the result from Part (a) to show that $ E\left[\Phi\left(\mathbf{X}\right)\right]=\Phi\left(\frac{\mu}{\sqrt{2}}\right) $ .
Let $ \mathbf{Y}=\mathbf{Z}-\mathbf{X} $ . Since $ \mathbf{Z} $ and $ \mathbf{X} $ are Gaussian random variables, $ \mathbf{Y} $ is also a Gaussian random variable. $ E\left[\mathbf{Y}\right]=E\left[\mathbf{Z}\right]-E\left[\mathbf{X}\right]=-\mu. $
$ Var\left[\mathbf{Y}\right]=E\left[\left(\mathbf{Y}-E\left[\mathbf{Y}\right]\right)^{2}\right]=E\left[\left(\mathbf{Z}-\left(\mathbf{X}-\mu\right)\right)^{2}\right]=E\left[\mathbf{Z}^{2}\right]+E\left[\left(\mathbf{X}-\mu\right)^{2}\right]-2E\left[\mathbf{Z}\right]E\left[\mathbf{X}-\mu\right] $$ =E\left[\mathbf{Z}^{2}\right]-E\left[\mathbf{Z}\right]E\left[\mathbf{X}-\mu\right]+E\left[\left(\mathbf{X}-\mu\right)^{2}\right]-E\left[\mathbf{Z}\right]E\left[\mathbf{X}-\mu\right] $$ =E\left[\mathbf{Z}^{2}\right]-\left(E\left[\mathbf{Z}\right]\right)^{2}+E\left[\left(\mathbf{X}-\mu\right)^{2}\right]-\left(E\left[\mathbf{X}-\mu\right]\right)^{2}=Var\left[\mathbf{Z}\right]+Var\left[\mathbf{X}\right]=2. $
$ E\left[\Phi\left(\mathbf{X}\right)\right]=P\left(\left\{ \mathbf{Z}\leq\mathbf{X}\right\} \right)=P\left(\left\{ \mathbf{Y}\leq0\right\} \right)=\Phi\left(\frac{0-\left(-\mu\right)}{\sqrt{2}}\right)=\Phi\left(\frac{\mu}{\sqrt{2}}\right). $
3 (15 points)
Let \mathbf{Y}(t) be the output of linear system with impulse response h\left(t\right) and input \mathbf{X}\left(t\right)+\mathbf{N}\left(t\right) , where \mathbf{X}\left(t\right) and \mathbf{N}\left(t\right) are jointly wide-sense stationary independent random processes. If \mathbf{Z}\left(t\right)=\mathbf{X}\left(t\right)-\mathbf{Y}\left(t\right) , find the power spectral density S_{\mathbf{Z}}\left(\omega\right) in terms of S_{\mathbf{X}}\left(\omega\right) , S_{\mathbf{N}}\left(\omega\right) , m_{\mathbf{X}}=E\left[\mathbf{X}\right] , and m_{\mathbf{Y}}=E\left[\mathbf{Y}\right] .
Solution
Let \mathbf{M}\left(t\right)=\mathbf{X}\left(t\right)+\mathbf{N}\left(t\right) . Since \mathbf{X}\left(t\right) and \mathbf{N}\left(t\right) are jointly wide-sense stationary. \mathbf{M}\left(t\right) is also a wide-sense stationary random process. \mathbf{Y}\left(t\right)=\mathbf{M}\left(t\right)*h\left(t\right). R_{\mathbf{Y}}\left(\tau\right)=\left(R_{\mathbf{M}}*h*\widetilde{h}\right)\left(\tau\right)\textrm{ where }\left(\widetilde{h}\left(t\right)=h\left(-t\right)\right). R_{\mathbf{M}}\left(\tau\right) R_{\mathbf{XY}}\left(\tau\right)
R_{\mathbf{Z}}\left(\tau\right) S_{\mathbf{Z}}\left(\omega\right) \because m_{\mathbf{Y}}=m_{\mathbf{M}}*h\left(t\right)=\int_{-\infty}^{\infty}\left(m_{\mathbf{X}}+m_{\mathbf{N}}\right)h\left(t\right)dt=\left(m_{\mathbf{X}}+m_{\mathbf{N}}\right)H\left(0\right)\Rightarrow m_{\mathbf{N}}H\left(0\right)=m_{\mathbf{Y}}-m_{\mathbf{X}}H\left(0\right).
4
Suppose customer orders arrive according to an i.i.d. Bernoulli random process \mathbf{X}_{n} with parameter p . Thus, an order arrives at time index n (i.e., \mathbf{X}_{n}=1 ) with probability p ; if an order does not arrive at time index n , then \mathbf{X}_{n}=0 . When an order arrives, its size is an exponential random variable with parameter \lambda . Let \mathbf{S}_{n} be the total size of all orders up to time n .
(a) (20 points)
Find the mean and autocorrelation function of \mathbf{S}_{n} .
Let \mathbf{Y}_{n} be the size of an order at time index n , then \mathbf{Y}_{n} is a sequence of i.i.d. exponential random variables. \mathbf{S}_{n}=\sum_{k=1}^{n}\mathbf{X}_{n}\mathbf{Y}_{n}. E\left[\mathbf{S}_{n}\right]=\sum_{k=1}^{n}E\left[\mathbf{X}_{n}\right]E\left[\mathbf{Y}_{n}\right]=\sum_{k=1}^{n}p\cdot\frac{1}{\lambda}=\frac{np}{\lambda}. R_{\mathbf{S}}\left(n,m\right)=E\left[\mathbf{S}_{n}\mathbf{S}_{m}\right]=\sum_{k=1}^{n}\sum_{l=1}^{m}E\left[\mathbf{X}_{n}\right]E\left[\mathbf{X}_{m}\right]E\left[\mathbf{Y}_{n}\right]E\left[\mathbf{Y}_{m}\right]=\sum_{k=1}^{n}\sum_{l=1}^{m}\frac{p^{2}}{\lambda^{2}}=nm\frac{p^{2}}{\lambda^{2}}.
(b) (5 points)
Is \mathbf{S}_{n} a stationary random process? Explain.
• Approach 1: \mathbf{S}_{n} is not a stationary random process since R_{\mathbf{S}}\left(n,m\right) does not depend on only m-n .
• Approach 2: \mathbf{S}_{n} is not a stationary random process since E\left[\mathbf{S}_{n}\right] is not constant.
