| Line 37: | Line 37: | ||
<math> \sigma^2 = \sum_{x}(x-\mu)^2 p_{X}(x)= \sum_{x \in A}(x-\mu)^2 p_{X}(x)+\sum_{x \notin A}(x-\mu)^2 p_{X}(x)</math><br> | <math> \sigma^2 = \sum_{x}(x-\mu)^2 p_{X}(x)= \sum_{x \in A}(x-\mu)^2 p_{X}(x)+\sum_{x \notin A}(x-\mu)^2 p_{X}(x)</math><br> | ||
<math> \Rightarrow\sigma^2 \geq \sum_{x \in A}(x-\mu)^2 p_{X}(x)</math><br> | <math> \Rightarrow\sigma^2 \geq \sum_{x \in A}(x-\mu)^2 p_{X}(x)</math><br> | ||
| + | Since, in set <math class="inline">A</math>, we have <math class="inline">|x-\mu| \geq \varepsilon</math>, we have<br> | ||
| + | <math> \Rightarrow\sigma^2 \geq \sum_{x \in A}\varepsilon^2 p_{X}(x)= \varepsilon^2 \sum_{x \in A}p_{X}(x)=\varepsilon^2 P(x \in A) =\varepsilon^2 P(|X-\mu| \geq \varepsilon)</math><br> | ||
| + | That is <br> | ||
| + | <math> P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2}</math><br> | ||
Continuous Case:<br> | Continuous Case:<br> | ||
| + | Let <math class="inline">f_{X}(x)</math> be the pdf of X. <br> | ||
| + | <math> \sigma^2=\int_{-\infty}^{\infty}(x-\mu)^2f_{X}(x) \,dx \geq \int_{-\infty}^{\mu-\varepsilon}(x-\mu)^2f_{X}(x) \,dx+ \int_{-\mu-\varepsilon}^{\infty}(x-\mu)^2f_{X}(x) \,dx</math><br> | ||
| + | The last inequality holds since we integrate a positive function. Since <math class="inline">x \leq \mu-\varepsilon</math> or <math class="inline">x \geq \mu+\varepsilon</math><br> | ||
| + | <math> \Rightarrow |x-\mu| \geq \varepsilon \Rightarrow (x-\mu)^2 \geq \varepsilon^2 </math><br> | ||
| + | Based on the above equation, we have <br> | ||
| + | <math> \sigma^2 \geq \int_{-\infty}^{\mu-\varepsilon}\varepsilon^2 f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty} \varepsilon^2 f_{X}(x) \,dx</math><br> | ||
| + | <math> = \varepsilon^2 \left( \int_{-\infty}^{\mu-\varepsilon}f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty} f_{X}(x) \,dx \right) = \varepsilon^2 P \bigg( X \leq (\mu-\varepsilon)\, \text{or} \, X \geq (\mu+\varepsilon) \bigg) = \varepsilon^2 P(|X-\mu| \geq \varepsilon)</math><br> | ||
| + | <math> \Rightarrow P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} | ||
Revision as of 20:14, 25 January 2014
Communication, Networking, Signal and Image Processing (CS)
Question 1: Probability and Random Processes
August 2012
Problem 2
Problem statement: Let $ X $ be a continuous or discrete random variable with mean $ \mu $ and variance $ \sigma^2 $. Then, $ \forall \varepsilon >0 $, we have
$ P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} $
$ \color{blue}\text{Solution 1:} $
$ \color{blue}\text{Solution 2:} $
Discrete Case:
Let $ p_{X}(x) $ be the pmf of X. The probability that $ X $ differs from $ \mu $ by at least $ \varepsilon $ is
$ P(|X-\mu| \geq \varepsilon)= \sum_{|X-\mu| \geq \varepsilon}p_{X}(x) $
Based on the definition of the variance, we have
$ \sigma^2 = \sum_{x}(x-\mu)^2 p_{X}(x) $
Let a set $ A= \{ x|\,|x-\mu| \geq \varepsilon \} $. We have
$ \sigma^2 = \sum_{x}(x-\mu)^2 p_{X}(x)= \sum_{x \in A}(x-\mu)^2 p_{X}(x)+\sum_{x \notin A}(x-\mu)^2 p_{X}(x) $
$ \Rightarrow\sigma^2 \geq \sum_{x \in A}(x-\mu)^2 p_{X}(x) $
Since, in set $ A $, we have $ |x-\mu| \geq \varepsilon $, we have
$ \Rightarrow\sigma^2 \geq \sum_{x \in A}\varepsilon^2 p_{X}(x)= \varepsilon^2 \sum_{x \in A}p_{X}(x)=\varepsilon^2 P(x \in A) =\varepsilon^2 P(|X-\mu| \geq \varepsilon) $
That is
$ P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} $
Continuous Case:
Let $ f_{X}(x) $ be the pdf of X.
$ \sigma^2=\int_{-\infty}^{\infty}(x-\mu)^2f_{X}(x) \,dx \geq \int_{-\infty}^{\mu-\varepsilon}(x-\mu)^2f_{X}(x) \,dx+ \int_{-\mu-\varepsilon}^{\infty}(x-\mu)^2f_{X}(x) \,dx $
The last inequality holds since we integrate a positive function. Since $ x \leq \mu-\varepsilon $ or $ x \geq \mu+\varepsilon $
$ \Rightarrow |x-\mu| \geq \varepsilon \Rightarrow (x-\mu)^2 \geq \varepsilon^2 $
Based on the above equation, we have
$ \sigma^2 \geq \int_{-\infty}^{\mu-\varepsilon}\varepsilon^2 f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty} \varepsilon^2 f_{X}(x) \,dx $
$ = \varepsilon^2 \left( \int_{-\infty}^{\mu-\varepsilon}f_{X}(x) \,dx+ \int_{\mu+\varepsilon}^{\infty} f_{X}(x) \,dx \right) = \varepsilon^2 P \bigg( X \leq (\mu-\varepsilon)\, \text{or} \, X \geq (\mu+\varepsilon) \bigg) = \varepsilon^2 P(|X-\mu| \geq \varepsilon) $
$ \Rightarrow P(|X-\mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2} $
