7.10 QE 2005 August
1. (30 Points)
Assume that $ \mathbf{X} $ is a binomial distributed random variable with probability mass function (pmf) given by $ p_{n}\left(k\right)=\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n $ where $ 0<p<1 $ .
(a)
Find the characteristic function of $ \mathbf{X} $ . (You must show how you derive the characteristic function.)
$ \Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{n}e^{i\omega k}\cdot\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}=\sum_{k=0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)\left(p\cdot e^{i\omega}\right)^{k}\left(1-p\right)^{n-k} $$ =\left(p\cdot e^{i\omega}+1-p\right)^{n}. $
(b)
Compute the standard deviation of $ \mathbf{X} $ .
$ E\left[\mathbf{X}\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=n\left(pe^{i\omega}+1-p\right)^{n-1}\cdot pe^{i\omega}\Bigl|_{i\omega=0}=np. $
$ E\left[\mathbf{X}^{2}\right]=\frac{d^{2}}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=\frac{d}{di\omega}npe^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0} $$ =np\left[\frac{d}{di\omega}e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}\right] $$ =np\left[e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}+e^{i\omega}(n-1)\left(pe^{i\omega}+1-p\right)^{n-2}\cdot pe^{i\omega}\Bigl|_{i\omega=0}\right] $$ =np\left[1+\left(n-1\right)p\right]=np+n\left(n-1\right)p^{2}. $
$ Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=np+n\left(n-1\right)p^{2}-\left(np\right)^{2}=np+n^{2}p^{2}-np^{2}-n^{2}p^{2}=np\left(1-p\right). $
$ \therefore\sigma_{\mathbf{X}}=\sqrt{np\left(1-p\right)}. $
(c)
Find the value or values of $ k $ for which $ p_{n}\left(k\right) $ is maximum, and express the answer in terms of $ p $ and $ n $ . Give the most complete answer to this question that you can.
$ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}=\frac{\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}}{\left(\begin{array}{c} n\\ k-1 \end{array}\right)p^{k-1}\left(1-p\right)^{n-k+1}}=\frac{\frac{n!}{\left(n-k\right)!k!}p^{k}\left(1-p\right)^{n-k}}{\frac{n!}{\left(n-k+1\right)!\left(k-1\right)!}p^{k-1}\left(1-p\right)^{n-k+1}} $$ =\frac{n!}{\left(n-k\right)!k!}\cdot\frac{\left(n-k+1\right)!\left(k-1\right)!}{n!}\cdot\frac{p}{1-p}=\frac{n-k+1}{k}\cdot\frac{p}{1-p}. $
$ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)} $ is monotonically descreasing function of $ k $ . Hence, $ p_{n}\left(k\right) $ is maximized by the largest $ k $ such that $ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}\geq1 $ .$ \left(n-k+1\right)p\geq k\left(1-p\right)\Longrightarrow np-kp+p\geq k-kp\Longrightarrow\left(n+1\right)p\geq k $. $ k $ should be the integer. Thus, $ k=\left\lfloor \left(n+1\right)p\right\rfloor $ maximize $ p_{n}\left(k\right) $ . If $ p_{n}\left(k\right)=p_{n}\left(k-1\right) $ , both $ k=\left(n+1\right)p $ and $ k=\left(n+1\right)p-1 $ maximize $ p_{n}\left(k\right) $ .
2. (30 Points)
Let $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots $ be a sequence of binomially distributed random variables, with $ \mathbf{X}_{n} $ having probability mass function $ p_{n}\left(k\right)=\left(\begin{array}{c} n\\ k \end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n, $ where $ 0<p_{n}<1 $ for all $ n=1,2,3,\cdots $ . Show that if $ np_{n}\rightarrow\lambda\text{ as }n\rightarrow\infty, $ then the random sequence $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots $ converges in distribution to a Poisson random variable having mean $ \lambda $ .
$ \Phi_{\mathbf{X}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}_{n}}\right]=\sum_{k=0}^{n}e^{i\omega k}\left(\begin{array}{c} n\\ k \end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}=\sum_{k=0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)\left(e^{i\omega}p_{n}\right)^{k}\left(1-p_{n}\right)^{n-k} $$ =\left(e^{i\omega}p_{n}+1-p_{n}\right)^{n}=\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}. $
Since $ np_{n}\rightarrow\lambda $ as $ n\rightarrow\infty $ , $ p_{n}\rightarrow\lambda/n $ . $ \lim_{n\rightarrow\infty}\Phi_{\mathbf{X}_{n}}\left(\omega\right)=\lim_{n\rightarrow\infty}\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}=\lim_{n\rightarrow\infty}\left(1-\frac{\lambda}{n}\left(e^{i\omega}-1\right)\right)^{n}=e^{\lambda\left(e^{i\omega}-1\right)}. $
$ \because\lim_{n\rightarrow\infty}\left(1+\frac{x}{n}\right)^{n}=e^{x}. $
$ e^{\lambda\left(e^{i\omega}-1\right)} $ is the characteristic function of a Poisson random variable with mean $ \lambda $ . Thus, as $ n\rightarrow\infty $ , $ \mathbf{X}_{n} $ converges in distribution to a Poisson random variable with mean $ \lambda $ .
3. (40 Points)
Consider a homogeneous Poisson point process with rate $ \lambda $ and points (event occurrence times) $ \mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots $ .
(a)
Derive the pdf $ f_{k}\left(t\right) $ of the $ k $ -th point $ \mathbf{T}_{k} $ for arbitrary $ k $ .
• The cdf is $ F_{\mathbf{T}_{k}}\left(t\right)=P\left(\mathbf{T}_{k}\leq t\right)=P\left(\text{at least }k\text{ points within }t\right)=\sum_{j=k}^{\infty}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!} $$ =1-P\left(\mathbf{T}_{k}>t\right)=1-P\left(\text{less than }k\text{ points within }t\right)=1-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}. $
• The pdf by differentiating the cdf $ isf_{\mathbf{T}_{k}}\left(t\right)=\frac{dF_{\mathbf{T}_{k}}}{dt}=-\sum_{j=0}^{k-1}\frac{\left(-\lambda\right)e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot j\left(\lambda t\right)^{j-1}\cdot\lambda}{j!} $$ =\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{j\left(\lambda t\right)^{j-1}}{j!}=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{\left(\lambda t\right)^{j-1}}{\left(j-1\right)!} $$ =\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=0}^{k-2}\frac{\left(\lambda t\right)^{j}}{j!}=\lambda e^{-\lambda t}\frac{\left(\lambda t\right)^{k-1}}{\left(k-1\right)!}. $
– This is Erlang distribution.
(b)
What kind of distribution does $ \mathbf{T}_{1} $ have?
• If $ k=1 $ , then $ f_{\mathbf{T}_{1}}\left(t\right)=\lambda e^{-\lambda t} $ . Thus $ \mathbf{T}_{1} $ is a exponential random variable with parameter $ \lambda $ .
(c)
What is the conditional pdf of $ \mathbf{T}_{k} $ given $ \mathbf{T}_{k-1}=t_{0} $ , where $ t_{0}>0 $ ? (You can give the answer without derivation if you know it.)
• The conditional cdf is
$ F_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(\mathbf{T}_{k}\leq t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(N\left(t_{k},t_{0}\right)\geq1\right) $$ =1-P\left(N\left(t_{k},t_{0}\right)=0\right)=1-e^{-\lambda\left(t_{k}-t_{0}\right)}. $
• The conditional pdf by differentiating the conditional cdf is
$ \therefore f_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=\lambda e^{-\lambda\left(t_{k}-t_{0}\right)}. $
(d)
Suppose you have a random number generator that produces independent, identically distributed (i.i.d. ) random variables $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots $ that are uniformaly distributed on the interval $ \left(0,1\right) $ . Explain how you could use these to simulate the Poisson points $ \mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots $ describe above. Provide as complete an explanation as possible.
• This problem is similar to QE 2003 August Problem 2.
