| Line 3: | Line 3: | ||
<font size="4">[[ECE PhD Qualifying Exams|ECE Ph.D. Qualifying Exam]] </font> | <font size="4">[[ECE PhD Qualifying Exams|ECE Ph.D. Qualifying Exam]] </font> | ||
| − | <font size="4">Communication, Networking, Signal and Image Processing (CS)</font> | + | <font size="4">Communication, Networking, Signal and Image Processing (CS)</font> |
<font size="4">Question 1: Probability and Random Processes </font> | <font size="4">Question 1: Probability and Random Processes </font> | ||
| Line 21: | Line 21: | ||
Problem statement: Let <span class="texhtml">''X''</span> be a continuous or discrete random variable with mean <span class="texhtml">μ</span> and variance <span class="texhtml">σ<sup>2</sup></span>. Then, <math>\forall \varepsilon >0</math>, we have<br> <math> P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2}</math><br> | Problem statement: Let <span class="texhtml">''X''</span> be a continuous or discrete random variable with mean <span class="texhtml">μ</span> and variance <span class="texhtml">σ<sup>2</sup></span>. Then, <math>\forall \varepsilon >0</math>, we have<br> <math> P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2}</math><br> | ||
| − | ===== <math>\color{blue}\text{Solution 1:}</math><br> ===== | + | ===== <math>\color{blue}\text{Solution 1:}</math><br> ===== |
| + | |||
| + | *Continuous case:<br> | ||
| + | |||
| + | Consider two functions <span class="texhtml">''g''<sub>''1''</sub>(''x'')</span> and <span class="texhtml">''g''<sub>''2''</sub>(''x'')</span> | ||
| + | |||
| + | <span class="texhtml">''g''<sub>''1''</sub>(''x'')</span> <math>=</math> <span class="texhtml">''1''<sub>''R''</sub>(''x'')</span><br> | ||
| + | where, <math>R = \{x: \mid x-\mu\mid \geq \varepsilon\}</math> | ||
| + | |||
| + | |||
| + | <math>g_{1}(x)= 1_{\{x: \mid x-\mu\mid \geq \varepsilon\}}(x)</math> | ||
| + | |||
| + | <math>g_{2}(x)=\frac{(x-\mu)^2}{\varepsilon^2}</math> | ||
| − | |||
| − | |||
===== <math>\color{blue}\text{Solution 2:}</math> ===== | ===== <math>\color{blue}\text{Solution 2:}</math> ===== | ||
| Line 32: | Line 42: | ||
Let <span class="texhtml">''p''<sub>''X''</sub>(''x'')</span> be the pmf of X. The probability that <span class="texhtml">''X''</span> differs from <span class="texhtml">μ</span> by at least <math>\varepsilon </math> is <br> <math> P(|X-\mu| \geq \varepsilon)= \sum_{|X-\mu| \geq \varepsilon}p_{X}(x)</math><br> Based on the definition of the variance, we have<br> <span class="texhtml"> | Let <span class="texhtml">''p''<sub>''X''</sub>(''x'')</span> be the pmf of X. The probability that <span class="texhtml">''X''</span> differs from <span class="texhtml">μ</span> by at least <math>\varepsilon </math> is <br> <math> P(|X-\mu| \geq \varepsilon)= \sum_{|X-\mu| \geq \varepsilon}p_{X}(x)</math><br> Based on the definition of the variance, we have<br> <span class="texhtml"> | ||
| − | </span> | + | </span> |
{| | {| | ||
|- style="text-align: center;" | |- style="text-align: center;" | ||
| σ<sup>2</sup> = | | σ<sup>2</sup> = | ||
| − | | <span style="font-size: x-large; font-family: serif;">∑</span> | + | | <span style="font-size: x-large; font-family: serif;">∑</span> |
| (''x'' − μ)<sup>2</sup>''p''<sub>''X''</sub>(''x'') | | (''x'' − μ)<sup>2</sup>''p''<sub>''X''</sub>(''x'') | ||
|- style="text-align: center; vertical-align: top;" | |- style="text-align: center; vertical-align: top;" | ||
| | | | ||
| − | | ''x'' | + | | ''x'' |
| | | | ||
|} | |} | ||
Revision as of 12:09, 26 January 2014
Communication, Networking, Signal and Image Processing (CS)
Question 1: Probability and Random Processes
August 2012
Problem 2
Problem statement: Let X be a continuous or discrete random variable with mean μ and variance σ2. Then, $ \forall \varepsilon >0 $, we have
$ P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} $
$ \color{blue}\text{Solution 1:} $
- Continuous case:
Consider two functions g1(x) and g2(x)
g1(x) $ = $ 1R(x)
where, $ R = \{x: \mid x-\mu\mid \geq \varepsilon\} $
$ g_{1}(x)= 1_{\{x: \mid x-\mu\mid \geq \varepsilon\}}(x) $
$ g_{2}(x)=\frac{(x-\mu)^2}{\varepsilon^2} $
$ \color{blue}\text{Solution 2:} $
- Discrete Case:
Let pX(x) be the pmf of X. The probability that X differs from μ by at least $ \varepsilon $ is
$ P(|X-\mu| \geq \varepsilon)= \sum_{|X-\mu| \geq \varepsilon}p_{X}(x) $
Based on the definition of the variance, we have
| σ2 = | ∑ | (x − μ)2pX(x) |
| x |
Let a set $ A= \{ x|\,|x-\mu| \geq \varepsilon \} $. We have
$ \sigma^2 = \sum_{x}(x-\mu)^2 p_{X}(x)= \sum_{x \in A}(x-\mu)^2 p_{X}(x)+\sum_{x \notin A}(x-\mu)^2 p_{X}(x) $
$ \Rightarrow\sigma^2 \geq \sum_{x \in A}(x-\mu)^2 p_{X}(x) $
Since, in set A, we have $ |x-\mu| \geq \varepsilon $, we have
$ \Rightarrow\sigma^2 \geq \sum_{x \in A}\varepsilon^2 p_{X}(x)= \varepsilon^2 \sum_{x \in A}p_{X}(x)=\varepsilon^2 P(x \in A) =\varepsilon^2 P(|X-\mu| \geq \varepsilon) $
That is
$ P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} $
- Continuous Case:
Let fX(x) be the pdf of X.
$ \sigma^2=\int_{-\infty}^{\infty}(x-\mu)^2f_{X}(x) \,dx \geq \int_{-\infty}^{\mu-\varepsilon}(x-\mu)^2f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty}(x-\mu)^2f_{X}(x) \,dx $
The last inequality holds since we integrate a positive function. Since $ x \leq \mu-\varepsilon $ or $ x \geq \mu+\varepsilon $
$ \Rightarrow |x-\mu| \geq \varepsilon \Rightarrow (x-\mu)^2 \geq \varepsilon^2 $
Based on the above equation, we have
$ \sigma^2 \geq \int_{-\infty}^{\mu-\varepsilon}\varepsilon^2 f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty} \varepsilon^2 f_{X}(x) \,dx $
$ = \varepsilon^2 \left( \int_{-\infty}^{\mu-\varepsilon}f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty} f_{X}(x) \,dx \right) = \varepsilon^2 P \bigg( X \leq (\mu-\varepsilon)\, \text{or} \, X \geq (\mu+\varepsilon) \bigg) = \varepsilon^2 P(|X-\mu| \geq \varepsilon) $
$ \Rightarrow P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} $
