Communication, Networking, Signal and Image Processing (CS)
Question 1: Probability and Random Processes
August 2005
1. (30 Points)
Assume that $ \mathbf{X} $ is a binomial distributed random variable with probability mass function (pmf) given by $ p_{n}\left(k\right)=\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n $ where $ 0<p<1 $ .
(a)
Find the characteristic function of $ \mathbf{X} $ . (You must show how you derive the characteristic function.)
$ \Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{n}e^{i\omega k}\cdot\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}=\sum_{k=0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)\left(p\cdot e^{i\omega}\right)^{k}\left(1-p\right)^{n-k} $$ =\left(p\cdot e^{i\omega}+1-p\right)^{n}. $
(b)
Compute the standard deviation of $ \mathbf{X} $ .
$ E\left[\mathbf{X}\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=n\left(pe^{i\omega}+1-p\right)^{n-1}\cdot pe^{i\omega}\Bigl|_{i\omega=0}=np. $
$ E\left[\mathbf{X}^{2}\right]=\frac{d^{2}}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=\frac{d}{di\omega}npe^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0} $$ =np\left[\frac{d}{di\omega}e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}\right] $$ =np\left[e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}+e^{i\omega}(n-1)\left(pe^{i\omega}+1-p\right)^{n-2}\cdot pe^{i\omega}\Bigl|_{i\omega=0}\right] $$ =np\left[1+\left(n-1\right)p\right]=np+n\left(n-1\right)p^{2}. $
$ Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=np+n\left(n-1\right)p^{2}-\left(np\right)^{2}=np+n^{2}p^{2}-np^{2}-n^{2}p^{2}=np\left(1-p\right). $
$ \therefore\sigma_{\mathbf{X}}=\sqrt{np\left(1-p\right)}. $
(c)
Find the value or values of $ k $ for which $ p_{n}\left(k\right) $ is maximum, and express the answer in terms of $ p $ and $ n $ . Give the most complete answer to this question that you can.
$ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}=\frac{\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}}{\left(\begin{array}{c} n\\ k-1 \end{array}\right)p^{k-1}\left(1-p\right)^{n-k+1}}=\frac{\frac{n!}{\left(n-k\right)!k!}p^{k}\left(1-p\right)^{n-k}}{\frac{n!}{\left(n-k+1\right)!\left(k-1\right)!}p^{k-1}\left(1-p\right)^{n-k+1}} $$ =\frac{n!}{\left(n-k\right)!k!}\cdot\frac{\left(n-k+1\right)!\left(k-1\right)!}{n!}\cdot\frac{p}{1-p}=\frac{n-k+1}{k}\cdot\frac{p}{1-p}. $
$ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)} $ is monotonically descreasing function of $ k $ . Hence, $ p_{n}\left(k\right) $ is maximized by the largest $ k $ such that $ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}\geq1 $ .$ \left(n-k+1\right)p\geq k\left(1-p\right)\Longrightarrow np-kp+p\geq k-kp\Longrightarrow\left(n+1\right)p\geq k $. $ k $ should be the integer. Thus, $ k=\left\lfloor \left(n+1\right)p\right\rfloor $ maximize $ p_{n}\left(k\right) $ . If $ p_{n}\left(k\right)=p_{n}\left(k-1\right) $ , both $ k=\left(n+1\right)p $ and $ k=\left(n+1\right)p-1 $ maximize $ p_{n}\left(k\right) $ .
