@@ Line 1: / Line 1: @@
 =5 Exams=
-=Example. Addition of two independent Poisson random variables=
+Problem Examples
-Let <math>\mathbf{Z}=\mathbf{X}+\mathbf{Y}</math>  where <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  are independent Poisson random variables with means <math>\lambda</math>  and <math>\mu</math> , respectively.
+*[[ECE 600 Exams Addition of two independent Poisson random variables|Addition of two independent Poisson random variables]]
+*[[ECE 600 Exams Addition of two independent Gaussian random variables|Addition of two independent Gaussian random variables]]
+*[[ECE 600 Exams Addition of two jointly distributed Gaussian random variables|Addition of two jointly distributed Gaussian random variables]]
+*[[ECE 600 Exams Two jointly distributed random variables|Two jointly distributed random variables]]
+*[[ECE 600 Exams Two jointly distributed independent random variables|Two jointly distributed independent random variables]]
+*[[ECE 600 Exams Two jointly distributed random variables (Joint characteristic function)|Two jointly distributed random variables (Joint characteristic function)]]
+*[[ECE 600 Exams Geometric random variable|Geometric random variable]]
+*[[ECE 600 Exams Sequence of binomially distributed random variables|Sequence of binomially distributed random variables]]
+*[[ECE 600 Exams Sequence of exponentially distributed random variables|Sequence of exponentially distributed random variables]]
+*[[ECE 600 Exams Sequence of uniformly distributed random variables|Sequence of uniformly distributed random variables]]
+*[[ECE 600 Exams Mean of iid random variables|Mean of iid random variables]]
+*[[ECE 600 Exams A sum of a random number of iid Gaussian random variables|A sum of a random number of iid Gaussian random variables]]
-(a)
+----
+[[ECE600|Back to ECE600]]
-Find the pmf of <math>\mathbf{Z}</math> .
-According to the characteristic function of Poisson random variable
-<math>\Phi_{\mathbf{X}}(\omega)=e^{-\lambda\left(1-e^{i\omega}\right)},\Phi_{\mathbf{Y}}(\omega)=e^{-\mu\left(1-e^{i\omega}\right)}</math>.
-<math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  are independent <math>\Longrightarrow  \mathbf{X}</math>  and <math>\mathbf{Y}</math>  are uncorrelated <math>\Longrightarrow  e^{i\omega\mathbf{X}}</math>  and <math>e^{i\omega\mathbf{Y}}</math>  are uncorrelated.
-<math>\Phi_{\mathbf{Z}}(\omega)=E\left[e^{i\omega\mathbf{Z}}\right]=E\left[e^{i\omega\left(\mathbf{X}+\mathbf{Y}\right)}\right]=E\left[e^{i\omega\mathbf{X}}e^{i\omega\mathbf{Y}}\right]=E\left[e^{i\omega\mathbf{X}}\right]\cdot E\left[e^{i\omega\mathbf{Y}}\right]</math><math>=e^{-\lambda\left(1-e^{i\omega}\right)}\cdot e^{-\mu\left(1-e^{i\omega}\right)}=e^{-\left(\lambda+\mu\right)\left(1-e^{i\omega}\right).}</math>
-Now, we know that \mathbf{Z}  is a Poisson random variable with mean <math>\lambda+\mu</math> .
-<math>\therefore p_{\mathbf{Z}}(k)=\frac{e^{-\left(\lambda+\mu\right)}\left(\lambda+\mu\right)^{k}}{k!}.</math>
-(b)
-Show that the conditional pmf of <math>\mathbf{X}</math>  conditioned on the event <math>\left\{ \mathbf{Z}=n\right\}</math>   is binomially distributed, and determine the parameters of binomial distribution (<math>n</math>  and <math>p</math> ).
-<math>P_{\mathbf{X}}\left(\mathbf{X}|\left\{ \mathbf{Z}=n\right\} \right)=P\left(\left\{ \mathbf{X}=k\right\} |\left\{ \mathbf{Z}=n\right\} \right)=\frac{P\left(\left\{ \mathbf{X}=k\right\} \cap\left\{ \mathbf{Z}=n\right\} \right)}{P\left(\left\{ \mathbf{Z}=n\right\} \right)}=\frac{P\left(\left\{ \mathbf{X}=k\right\} \cap\left\{ \mathbf{Y}=n-k\right\} \right)}{P\left(\left\{ \mathbf{Z}=n\right\} \right)}</math><math>=\frac{\frac{e^{-\lambda}\lambda^{k}}{k!}\cdot\frac{e^{-\mu}\mu^{n-k}}{\left(n-k\right)!}}{\frac{e^{-\left(\lambda+\mu\right)}\left(\lambda+\mu\right)^{n}}{n!}}=\left(\frac{n!}{k!\left(n-k\right)!}\right)\left(\frac{\lambda}{\lambda+\mu}\right)^{k}\left(\frac{\mu}{\lambda+\mu}\right)^{n-k}</math><math>=\left(\begin{array}{c}
-n\\
-k
-\end{array}\right)\left(\frac{\lambda}{\lambda+\mu}\right)^{k}\left(\frac{\mu}{\lambda+\mu}\right)^{n-k}\;,\; k=0,\,1,\,2,\,\cdots</math>
-This is a binomial pmf <math>b(n,p)</math>  with parameters <math>n</math>  and <math>p=\frac{\lambda}{\lambda+\mu}</math> .
-=Example. Addition of two independent Gaussian random variables=
-<math>\mathbf{X}\sim\mathcal{N}\left(0,\sigma_{\mathbf{X}}^{2}\right),\;\mathbf{N}\sim\mathcal{N}\left(0,\sigma_{\mathbf{N}}^{2}\right),\;\mathbf{Y}=\mathbf{X}+\mathbf{N}.</math>
-(a)
-Correlation coefficient between <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math> .
-<math>\sigma_{\mathbf{Y}}=\sqrt{\sigma_{\mathbf{X}}^{2}+2r_{\mathbf{XN}}\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}+\sigma_{\mathbf{N}}^{2}}=\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}</math>
-because <math>\mathbf{X}</math>  and <math>\mathbf{N}</math>  are independnet <math>\Longrightarrow</math> uncorrelated <math>\Longrightarrow r_{\mathbf{XN}}=0</math> .
-<math>r_{\mathbf{XY}}=\frac{\text{cov}(\mathbf{X},\mathbf{Y})}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}\left(\mathbf{X}+\mathbf{N}\right)\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{E\left[\mathbf{X}^{2}\right]+E\left[\mathbf{XN}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}</math><math>=\frac{\sigma_{\mathbf{X}}^{2}+E\left[\mathbf{X}\right]E\left[\mathbf{N}\right]}{\sigma_{\mathbf{X}}\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\qquad\because E\left[\mathbf{X}\right]=0.</math>
-(b)
-Conditional pmf of <math>\mathbf{X}</math>  conditioned on the event <math>\left\{ \mathbf{Y}=y\right\}</math>  .
-<math>f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{Y}}(y)}=\frac{\frac{1}{2\pi\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\} }{\frac{1}{\sqrt{2\pi}\sigma_{Y}}\exp\left\{ \frac{-y^{2}}{2\sigma_{Y}^{2}}\right\} }</math><math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{y^{2}}{\sigma_{\mathbf{Y}}^{2}}-\frac{\left(1-r^{2}\right)y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\}</math> <math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)}\left[\frac{x^{2}}{\sigma_{\mathbf{X}}^{2}}-\frac{2rxy}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}+\frac{r^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\}</math> <math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left[x^{2}-\frac{2r\sigma_{\mathbf{X}}xy}{\sigma_{\mathbf{Y}}}+\frac{r^{2}\sigma_{\mathbf{X}}^{2}y^{2}}{\sigma_{\mathbf{Y}}^{2}}\right]\right\}</math> <math>=\frac{1}{\sqrt{2\pi}\sigma_{\mathbf{X}}\sqrt{1-r^{2}}}\exp\left\{ \frac{-1}{2\left(1-r^{2}\right)\sigma_{\mathbf{X}}^{2}}\left(x-\frac{r\sigma_{\mathbf{X}}y}{\sigma_{\mathbf{Y}}}\right)^{2}\right\}</math>
-Noting that <math>\sqrt{1-r^{2}}=\sigma_{\mathbf{X}}\sqrt{1-\left(\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}\right)^{2}}=\sqrt{1-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}}=\sqrt{\frac{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}-\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}</math>  and
-<math>r\cdot\frac{\sigma_{\mathbf{X}}}{\sigma_{\mathbf{Y}}}=\frac{\sigma_{X}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\cdot\frac{\sigma_{\mathbf{X}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}.</math> <math>\therefore f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)=\frac{1}{\sqrt{2\pi}\cdot\frac{\sigma_{\mathbf{X}}\sigma_{\mathbf{N}}}{\sqrt{\sigma_{\mathbf{X}}^{2}+\mathbf{\sigma}_{\mathbf{N}}^{\mathbf{2}}}}}\exp\left\{ \frac{-1}{2\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}}\left(x-\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y\right)^{2}\right\}</math>
-(c)
-What kind of pdf is the pdf you determined in part (b)? What is the mean and variance of a random variable with this pdf?
-This is a Gaussian pdf with mean <math>\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}\cdot y</math>  and variance <math>\frac{\sigma_{\mathbf{X}}^{2}\sigma_{\mathbf{N}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{N}}^{2}}</math> .
-(d)
-What is the minimum mean-square estimate of <math>\mathbf{X}</math>  given that <math>\left\{ \mathbf{Y}=y\right\}</math>  ?
-The minimum mean-square error estimate of <math>\mathbf{X}</math>  given <math>\mathbf{Y}=y</math>  is
-<math>\hat{x}_{MMS}(y)=E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\int_{-\infty}^{\infty}x\cdot f_{\mathbf{X}}\left(x|\left\{ \mathbf{Y}=y\right\} \right)dx=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y</math>  from part (b).
-(e)
-What is the maximum a posteriori estimate of <math>\mathbf{X}</math>  given that <math>\left\{ \mathbf{Y}=y\right\}</math>  ?
-<math>\hat{x}_{MAP}(y)=\arg\max_{x\in\mathbf{R}}\left\{ f_{\mathbf{X}}\left(x|\left\{ Y=y\right\} \right)\right\} =\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y</math>
-as a Gaussian pdf takes on its maximum value at its mean.
-(f)
-Given that I observe <math>\mathbf{Y}=y</math> , what is <math>E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]</math> ?
-<math>E\left[\mathbf{X}|\left\{ \mathbf{Y}=y\right\} \right]=\frac{\sigma_{\mathbf{X}}^{2}}{\sigma_{\mathbf{X}}^{2}+\sigma_{\mathbf{Y}}^{2}}\cdot y</math> from part (d).
-=Example. Addition of two jointly distributed Gaussian random variables=
-Let <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  be two jointly distributed Gaussian random variables. Assume <math>\mathbf{X}</math>  has mean <math>\mu_{\mathbf{X}}</math>  and variance <math>\sigma_{\mathbf{X}}^{2} , \mathbf{Y}</math>  has mean <math>\mu_{\mathbf{Y}}</math>  and variance <math>\sigma_{\mathbf{Y}}^{2}</math> , and that the correlation coefficient between <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  is <math>r</math> . Define a new random variable <math>\mathbf{Z}=\mathbf{X}+\mathbf{Y}</math> .
-(a)
-Show that <math>\mathbf{Z}</math>  is a Gaussian random variable.
-If <math>\mathbf{Z}</math>  is a Guassian random variable, then it has a characteristic function of the form
-<math>\Phi_{\mathbf{Z}}\left(\omega\right)=e^{i\mu_{\mathbf{Z}}\omega}e^{-\frac{1}{2}\sigma_{\mathbf{Z}}^{2}\omega^{2}}.</math>
-<math>\Phi_{\mathbf{Z}}\left(\omega\right)</math>
-where <math>\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)</math>  is the joint characteristic function of <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math> , defined as
-<math>\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)=E\left[e^{i\left(\mathbf{\omega_{1}X}+\omega_{2}\mathbf{Y}\right)}\right].</math>
-Now because <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  are jointly Gaussian with the given parameters, we know that
-<math>\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)=e^{i\left(\mu_{X}\omega_{1}+\mu_{Y}\omega_{2}\right)}e^{-\frac{1}{2}\left(\sigma_{X}^{2}\omega_{1}^{2}+2r\sigma_{X}\sigma_{Y}\omega_{1}\omega_{2}+\sigma_{Y}^{2}\omega_{2}^{2}\right)}.</math>
-Thus,
-<math>\Phi_{\mathbf{Z}}\left(\omega\right)=\Phi_{\mathbf{XY}}\left(\omega,\omega\right)=e^{i\left(\mu_{X}\omega+\mu_{Y}\omega\right)}e^{-\frac{1}{2}\left(\sigma_{X}^{2}\omega^{2}+2r\sigma_{X}\sigma_{Y}\omega^{2}+\sigma_{Y}^{2}\omega^{2}\right)}</math><math>=e^{i\left(\mu_{X}+\mu_{Y}\right)\omega}e^{-\frac{1}{2}\left(\sigma_{X}^{2}+2r\sigma_{X}\sigma_{Y}+\sigma_{Y}^{2}\right)\omega^{2}}=e^{i\mu_{Z}\omega}e^{-\frac{1}{2}\sigma_{Z}^{2}\omega^{2}}</math>
-where <math>\mu_{Z}=\mu_{X}+\mu_{Y}</math>  and <math>\sigma_{Z}^{2}=\sigma_{X}^{2}+2r\sigma_{X}\sigma_{Y}+\sigma_{Y}^{2}</math> .
-<math>\mathbf{Z}</math>  is a Gaussian random variable with E\left[\mathbf{Z}\right]=\mu_{X}+\mu_{Y}  and Var\left[\mathbf{Z}\right]=\sigma_{X}^{2}+2r\sigma_{X}\sigma_{Y}+\sigma_{Y}^{2} .
-(b)
-Find the variance of <math>\mathbf{Z}</math> .
-As show in part (a) <math>Var\left[\mathbf{Z}\right]=\sigma_{\mathbf{X}}^{2}+2r\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}+\sigma_{\mathbf{Y}}^{2}</math> .
-=Example. Two jointly distributed random variables=
-Two joinly distributed random variables <math>\mathbf{X}</math>  and <math>\mathbf{Y}</math>  have joint pdf
-<math>f_{\mathbf{XY}}\left(x,y\right)=\begin{cases}
-\begin{array}{ll}
-c  ,\text{ for }x\geq0,y\geq0,\textrm{ and }x+y\leq1\\
-  ,\text{ elsewhere.}
-\end{array}\end{cases}</math>
-(a)
-Find the constant c  such that <math>f_{\mathbf{XY}}(x,y)</math>  is a valid pdf.
-[[Image:002.eps]]
-<math>\iint_{\mathbf{R}^{2}}f_{\mathbf{XY}}\left(x,y\right)=c\cdot Area=1</math>  where <math>Area=\frac{1}{2}</math> .
-<math>\therefore c=2</math>
-(b)
-Find the conditional density of \mathbf{Y}  conditioned on \mathbf{X}=x .
-f_{\mathbf{Y}}\left(y|\left\{ \mathbf{X}=x\right\} \right)=\frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{X}}(x)}.
-f_{\mathbf{X}}(x)=\int_{-\infty}^{\infty}f_{\mathbf{XY}}\left(x,y\right)dy=\int_{0}^{1-x}2dy=2\left(1-x\right)\cdot\mathbf{1}_{\left[0,1\right]}(x).
-f_{\mathbf{Y}}\left(y|\left\{ \mathbf{X}=x\right\} \right)=\frac{f_{\mathbf{XY}}\left(x,y\right)}{f_{\mathbf{X}}(x)}=\frac{2}{2\left(1-x\right)}=\frac{1}{1-x}\textrm{ where }0\leq y\leq1-x\Longrightarrow\frac{1}{1-x}\cdot\mathbf{1}_{\left[0,1-x\right]}\left(y\right).
-(c)
-Find the minimum mean-square error estimator \hat{y}_{MMS}\left(x\right)  of \mathbf{Y}  given that \mathbf{X}=x .
-\hat{y}_{MMS}\left(x\right)=E\left[\mathbf{Y}|\left\{ \mathbf{X}=x\right\} \right]=\int_{\mathbf{R}}yf_{\mathbf{Y}}\left(y|\left\{ \mathbf{X}=x\right\} \right)dy=\int_{0}^{1-x}\frac{y}{1-x}dy=\frac{y^{2}}{2\left(1-x\right)}\biggl|_{0}^{1-x}=\frac{1-x}{2}.
-(d)
-Find a maximum aposteriori probability estimator.
-\hat{y}_{MAP}\left(x\right)=\arg\max_{y}\left\{ f_{Y}\left(y|\left\{ \mathbf{X}=x\right\} \right)\right\}  but f_{Y}\left(y|\left\{ \mathbf{X}=x\right\} \right)=\frac{1}{1-x}\cdot\mathbf{1}_{\left[0,1-x\right]}\left(y\right) . Any \hat{y}\in\left[0,1-x\right]  is a MAP estimator. The MAP estimator is NOT unique.
-Example. Two jointly distributed independent random variables
-Let \mathbf{X}  and \mathbf{Y}  be two jointly distributed, independent random variables. The pdf of \mathbf{X}  is
-f_{\mathbf{X}}\left(x\right)=xe^{-x^{2}/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(x\right), and \mathbf{Y}  is a Gaussian random variable with mean 0  and variance 1 . Let \mathbf{U}  and \mathbf{V}  be two new random variables defined as \mathbf{U}=\sqrt{\mathbf{X}^{2}+\mathbf{Y}^{2}}  and \mathbf{V}=\lambda\mathbf{Y}/\mathbf{X}  where \lambda  is a positive real number.
-(a)
-Find the joint pdf of \mathbf{U}  and \mathbf{V} . (Direct pdf method)
-f_{\mathbf{UV}}\left(u,v\right)=f_{\mathbf{XY}}\left(x\left(u,v\right),y\left(u,v\right)\right)\left|\frac{\partial\left(x,y\right)}{\partial\left(u,v\right)}\right|
-Solving for x  and y  in terms of u  and v , we have u^{2}=x^{2}+y^{2}  and v^{2}=\frac{\lambda^{2}y^{2}}{x^{2}}\Longrightarrow y^{2}=\frac{v^{2}x^{2}}{\lambda^{2}} .
-Now, u^{2}=x^{2}+y^{2}=x^{2}+\frac{v^{2}x^{2}}{\lambda^{2}}=x^{2}\left(1+v^{2}/\lambda^{2}\right)\Longrightarrow x=\frac{u}{\sqrt{1+v^{2}/\lambda^{2}}}\Longrightarrow x\left(u,v\right)=\frac{u}{\sqrt{1+v^{2}/\lambda^{2}}} .
-Thus, y=\frac{vx}{\lambda}=\frac{vu}{\lambda\sqrt{1+v^{2}/\lambda^{2}}}\Longrightarrow y\left(u,v\right)=\frac{vu}{\lambda\sqrt{1+v^{2}/\lambda^{2}}} .
-Computing the Jacobian.
-\frac{\partial\left(x,y\right)}{\partial\left(u,v\right)}
-Because \mathbf{X}  and \mathbf{Y}  are statistically independent
-f_{\mathbf{XY}}\left(x,y\right)=f_{\mathbf{X}}\left(x\right)f_{\mathbf{Y}}\left(y\right)=xe^{-x^{2}/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(x\right)\cdot\frac{1}{\sqrt{2\pi}}e^{-y^{2}/2}=\frac{x}{\sqrt{2\pi}}e^{-\left(x^{2}+y^{2}\right)/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(x\right).
-Substituting these quantities, we get
-f_{\mathbf{UV}}\left(u,v\right)
-(b)
-Are \mathbf{U}  and \mathbf{V}  statistically independent? Justify your answer.
-\mathbf{U}  and \mathbf{V}  are statistically independent iff f_{\mathbf{UV}}\left(u,v\right)=f_{\mathbf{U}}\left(u\right)f_{\mathbf{V}}\left(v\right) .
-Now from part (a), we see thatf_{\mathbf{UV}}\left(u,v\right)=c_{1}g_{1}\left(u\right)\cdot c_{2}g_{2}\left(v\right) where g_{1}\left(u\right)=u^{2}e^{-u^{2}/2}\cdot\mathbf{1}_{\left[0,\infty\right)}\left(u\right)  and g_{2}\left(v\right)=\frac{1}{\left(\lambda^{2}+v^{2}\right)^{\frac{3}{2}}}  with c_{1}  and c_{2}  selected such that f_{\mathbf{U}}\left(u\right)=c_{1}g_{1}\left(u\right)  and f_{\mathbf{V}}\left(v\right)=c_{2}g_{2}\left(v\right)  are both valid pdfs.
-\therefore  \mathbf{U}  and \mathbf{V}  are statistically independent.
-Example. Two jointly distributed random variables (Joint characteristic function)
-Let \mathbf{X}  and \mathbf{Y}  be tweo jointly distributed random variables having joint characteristic function
-\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)=\frac{1}{\left(1-i\omega_{1}\right)\left(1-i\omega_{2}\right)}.
-(a)
-Calculate E\left[\mathbf{X}\right] .
-\Phi_{\mathbf{X}}\left(\omega\right)=\Phi_{\mathbf{XY}}\left(\omega,0\right)=\frac{1}{1-i\omega}=\left(1-i\omega\right)^{-1}
-E\left[\mathbf{X}\right]=\frac{d}{d\left(i\omega\right)}\Phi_{\mathbf{X}}\left(\omega\right)|_{i\omega=0}=(-1)(1-i\omega)^{-2}(-1)|_{i\omega=0}=1
-(b)
-Calculate E\left[\mathbf{Y}\right]
-E\left[\mathbf{Y}\right]=1
-(c)
-Calculate E\left[\mathbf{XY}\right] .
-E\left[\mathbf{XY}\right]=\frac{\partial^{2}}{\partial\left(i\omega_{1}\right)\partial\left(i\omega_{2}\right)}\Phi_{\mathbf{XY}}\left(\omega_{1},\omega_{2}\right)|_{i\omega_{1}=i\omega_{2}=0}=\left(1-i\omega_{1}\right)^{-2}\left(1-i\omega_{2}\right)^{-2}|_{i\omega_{1}=i\omega_{2}=0}=1
-(d)
-Calculate E\left[\mathbf{X}^{j}\mathbf{Y}^{k}\right] .
-E\left[\mathbf{X}^{j}\mathbf{Y}^{k}\right]
-(e)
-Calculate the correlation coefficient r_{\mathbf{XY}}  between \mathbf{X}  and \mathbf{Y} .
-r_{\mathbf{XY}}=\frac{Cov\left(\mathbf{X},\mathbf{Y}\right)}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{E\left[\mathbf{XY}\right]-E\left[\mathbf{X}\right]E\left[\mathbf{Y}\right]}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=\frac{1-1\cdot1}{\sigma_{\mathbf{X}}\sigma_{\mathbf{Y}}}=0.
-Example. Geometric random variable
-Let \mathbf{X}  be a random variable with probability mass function
-p_{\mathbf{X}}\left(k\right)=\alpha\left(1-\alpha\right)^{k-1},k=1,2,3,\cdots
-where 0<\alpha<1 .
-Note
-This is a geometric random variable with success probability \alpha .
-(a) Find the characteristic function of \mathbf{X} .
-\Phi_{\mathbf{X}}\left(\omega\right)
-since \left|e^{i\omega}\left(1-\alpha\right)\right|<1 .
-\because0<1-\alpha<1  and the real term of e^{i\omega}=\cos\omega+i\sin\omega  is \left|\cos\omega\right|<1 .
-(b) Find the mean of \mathbf{X} .
-E\left[\mathbf{X}\right]
-Note
-You can see the other approach to find E\left[\mathbf{X}\right]  and Var\left[\mathbf{X}\right]  [CS1GeometricDistribution].
-(c) Find the variance of \mathbf{X} .
-E\left[\mathbf{X}^{2}\right]
-because
-\frac{d}{d\left(i\omega\right)}e^{i\omega2}\left(1-e^{i\omega}\left(1-\alpha\right)\right)^{-2}\left|_{i\omega=0}\right.
-Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=\frac{2-\alpha}{\alpha^{2}}-\frac{1}{\alpha^{2}}=\frac{1-\alpha}{\alpha^{2}}.
-Example. Secquence of binomially distributed random variables
-Let \left\{ \mathbf{X}_{n}\right\} _{n\geq1}  be a sequence of binomially distributed random variables, with the n_{th}  random variable \mathbf{X}_{n}  having pmf
-P_{\mathbf{X}_{n}}(k)=P\left(\left\{ \mathbf{X}_{n}=k\right\} \right)=\left(\begin{array}{c}
-n\\
-k
-\end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}\;,\; k=0,1,\cdots,n,\; p_{n}\in\left(0,1\right). Show that, if the p_{n}  have the property that np_{n}\rightarrow\lambda  as n\rightarrow\infty , where \lambda  is a positive constant, then the sequence \left\{ \mathbf{X}_{n}\right\} _{n\leq1}  converges in distribution to a Poisson random variable \mathbf{X}  with mean \lambda .
-Hint:
-You may find the following fact useful:
-\lim_{n\rightarrow\infty}\left(1+\frac{x}{n}\right)^{n}=e^{x}.
-Solution
-If \mathbf{X}_{n}  converges to \mathbf{X}  in distribution, then F_{\mathbf{X}_{n}}(x)\rightarrow F_{\mathbf{X}}(x)  \forall x\in\mathbf{R} , where F_{\mathbf{X}}(x)  is continuous. This occurs iff \Phi_{\mathbf{X}_{n}}(\omega)\rightarrow\Phi_{\mathbf{X}}(\omega)  \forall x\in\mathbf{R} . We will show that \Phi_{\mathbf{X}_{n}}(\omega)  converges to e^{-\lambda\left(1-e^{i\omega}\right)}  as n\rightarrow\infty , which is the characteristic function of a Poisson random variable with mean \lambda .
-\Phi_{\mathbf{X}_{n}}(\omega)
-Now as n\rightarrow\infty , np_{n}\rightarrow\lambda\Rightarrow p_{n}\rightarrow\frac{\lambda}{n} .
-\lim_{n\rightarrow\infty}\Phi_{\mathbf{X}_{n}}(\omega)=\lim_{n\rightarrow\infty}\left(1+p_{n}\left(e^{i\omega}-1\right)\right)^{n}=\lim_{n\rightarrow\infty}\left(1+\frac{\lambda}{n}\left(e^{i\omega}-1\right)\right)^{n}=e^{\lambda\left(e^{i\omega}-1\right)}=e^{-\lambda\left(1-e^{i\omega}\right)},
-which is the characteristic function of Poisson random variable with mean \lambda .
-c.f.
-The problem 2 of the August 2007 QE [CS1QE2007August] is identical to this example.
-Example. Secquence of exponentially distributed random variables
-Let \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  be a collection of i.i.d.  exponentially distributed random variables, each having mean \mu . Define
-\mathbf{Y}=\max\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\}
-and
-\mathbf{Z}=\min\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\} .
-(a) Find the pdf of \mathbf{Y} .
-F_{\mathbf{Y}}\left(y\right)
-f_{\mathbf{Y}}(y)
-(b) Find the pdf of \mathbf{Z}
-F_{\mathbf{Z}}(z)
-f_{Z}(z)
-(c) In words, give as complete a description of the random variable \mathbf{Z}  as you can.
-\mathbf{Z}  is an exponetially distributed random variable with mean \frac{\mu}{n} .
-Example. Secquence of uniformly distributed random variables
-Let \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  be n  i.i.d.  jointly distributed random variables, each uniformly distributed on the interval \left[0,1\right] . Define the new random variables \mathbf{W}=\max\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\}
-and
-\mathbf{Z}=\min\left\{ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}\right\} .
-(a) Find the pdf of \mathbf{W} .
-F_{\mathbf{W}}(w)
-where f_{\mathbf{X}}(x)=\mathbf{1}_{\left[0,1\right]}(x)  and F_{X}\left(x\right)=\left\{ \begin{array}{ll}
-& ,x<0\\
-x & ,0\leq x<1\\
-& ,x\geq1
-\end{array}\right. .
-f_{\mathbf{W}}\left(w\right)
-(b) Find the pdf of \mathbf{Z} .
-F_{\mathbf{Z}}(z)
-f_{\mathbf{Z}}(z)
-(c) Find the mean of \mathbf{W} .
-E\left[\mathbf{W}\right]
-Example. Mean of i.i.d.  random variables
-Let \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  be M  jointly distributed i.i.d.  random variables with mean \mu  and variance \sigma^{2} . Let \mathbf{Y}_{M}=\frac{1}{M}\sum_{n=0}^{M}\mathbf{X}_{n} .
-(a) Find the variance of \mathbf{Y}_{M} .
-Var\left[\mathbf{Y}_{M}\right]=E\left[\mathbf{Y}_{M}^{2}\right]-\left(E\left[\mathbf{Y}_{M}\right]\right)^{2}.
-E\left[\mathbf{Y}_{M}\right]=E\left[\frac{1}{M}\sum_{n=0}^{M}\mathbf{X}_{n}\right]=\frac{1}{M}\sum_{n=0}^{M}E\left[\mathbf{X}_{n}\right]=\frac{1}{M}\cdot M\cdot\mu=\mu.
-E\left[\mathbf{Y}_{M}^{2}\right]=E\left[\frac{1}{M^{2}}\sum_{m=1}^{M}\sum_{n=1}^{M}\mathbf{X}_{m}\mathbf{X}_{n}\right]=\frac{1}{M^{2}}\sum_{m=1}^{M}\sum_{n=1}^{M}E\left[\mathbf{X}_{m}\mathbf{X}_{n}\right].
-Now E\left[\mathbf{X}_{m}\mathbf{X}_{n}\right]=\begin{cases}
-\begin{array}{ll}
-E\left[\mathbf{X}_{m}^{2}\right] & ,m=n\\
-E\left[\mathbf{X}_{m}\right]E\left[\mathbf{X}_{n}\right] & ,m\neq n
-\end{array}\end{cases}  because when m\neq n , \mathbf{X}_{m}  and \mathbf{X}_{n}  are independent \Rightarrow  \mathbf{X}_{m}  and \mathbf{X}_{n}  are uncorrelated.
-E\left[\mathbf{Y}_{M}^{2}\right]=\frac{1}{M^{2}}\left[M\left(\mu^{2}+\sigma^{2}\right)+M\left(M-1\right)\mu^{2}\right]=\frac{\left(\mu^{2}+\sigma^{2}\right)+\left(M-1\right)\mu^{2}}{M}=\frac{M\mu^{2}+\sigma^{2}}{M}.
-Var\left[\mathbf{Y}_{M}\right]=\frac{M\mu^{2}+\sigma^{2}-M\mu^{2}}{M}=\frac{\sigma^{2}}{M}.
-(b) Now assume that the \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n}  are identically distributed with with mean \mu  and variance \sigma^{2} , but they are only correlated rather than independent. Find the variance of \mathbf{Y}_{M} .
-Again, Var\left[\mathbf{Y}_{M}\right]=\frac{\sigma^{2}}{M} , because only uncorrelatedness was used in part (a).
-Example. A sum of a random number of i.i.d.  Gaussians
-Let \left\{ \mathbf{X}_{n}\right\}   be a sequence of i.i.d.  Gaussian random variables, each having characteristic function
-\Phi_{\mathbf{X}}\left(\omega\right)=e^{i\mu\omega}e^{-\frac{1}{2}\sigma^{2}\omega^{2}}. Let \mathbf{N}  be a Poisson random variable with pmf
-p(n)=\frac{e^{-\lambda}\lambda^{n}}{n!},\; n=0,1,2,\cdots,\;\lambda>0, and assume \mathbf{N}  is statistically independent of \left\{ \mathbf{X}_{n}\right\}  . Define a new random variable
-\mathbf{Y}=\mathbf{X}_{1}+\mathbf{X}_{2}+\cdots+\mathbf{X}_{N}.
-Note
-If \mathbf{N}=0 , then \mathbf{Y}=0 .
-(a) Find the mean of \mathbf{Y} .
-• Probability generating function of \mathbf{N}  is P_{\mathbf{N}}\left(z\right)=E\left[z^{\mathbf{N}}\right]=\sum_{n=0}^{\infty}z^{n}\frac{e^{-\lambda}\lambda^{n}}{n!}=e^{-\lambda}\sum_{n=0}^{\infty}\frac{\left(z\lambda\right)^{n}}{n!}=e^{-\lambda}e^{z\lambda}=e^{-\lambda\left(1-z\right)}.
-• The characteristic function of \mathbf{Y}  is \Phi_{\mathbf{Y}}\left(\omega\right)=P_{\mathbf{N}}\left(z\right)\Bigl|_{z=\Phi_{\mathbf{X}}\left(\omega\right)}=e^{-\lambda\left(1-z\right)}\Bigl|_{z=e^{i\mu\omega}e^{-\frac{1}{2}\sigma^{2}\omega^{2}}}=e^{-\lambda\left(1-e^{i\mu\omega}e^{-\frac{1}{2}\sigma^{2}\omega^{2}}\right)}.
-• Now, we can get the mean of \mathbf{Y}  using the characteristic function. E\left[\mathbf{Y}\right]

Difference between revisions of "ECE 600 Exams" - Rhea

Latest revision as of 10:57, 30 November 2010

5 Exams

Alumni Liaison