ECE600 F13 Conditional Distributions for Two Random Variables mhossain - Rhea

Topic 15: Conditional Distributions for Two Random Variables

Conditional Distributions

There are many applications of probability theory where we want to know the probabilistic behavior of a random variable Y given the value of another random variable X. We get this using conditional distributions.

Definition $\qquad$ For random variables X and Y defined on (S,F,P), the joint cdf of X and Y given an event M ∈ F, with P(M) >0, is

F_{XY}(x,y|M)=\frac{P(\{X\leq x,Y\leq y\}\cap M)}{P(M)},

A special case of great interest is:

\lim_{x\rightarrow\infty}F_{XY}(x,y|\{x_1<X\leq x_2\}) = F_Y(y|x_1<X\leq x_2)

where x $$ _1 $$ < x $$ _2 $$ . In this case, we have, assuming P(x $$ _1 $$ < X ≤ x $$ _2 $$ ),

\begin{align} F_Y(y|x_1<X\leq x_2) &= \frac{P(Y\leq y,x_1<X\leq x_2)}{P(x_1<X\leq x_2)} \\ &= \frac{F_{XY}(x_2,y)-F_{XY}(x_1,y)}{F_X(x_2)-F_X(x_1)} \end{align}

What we really want is f_ $$ _Y $$ (y|x $$ _1 $$ < X ≤ x $$ _2 $$ ), so we need to differentiate with respect to y. We do not have a name for this partial differentiation and we have not talked about how to find it, but we will need it here.

Now, writing

F_{XY}(x,y)=\int_{-\infty}^y\int_{-\infty}^xf_{XY}(x',y')dx'dy'

we have

\begin{align} f_Y(y|x_1<X\leq x_2) &= \frac{\frac{\partial}{\partial y}\left[ \int_{-\infty}^{x_2}\int_{-\infty}^yf_{XY}(x,y')dxdy'-\int_{-\infty}^{x_1}\int_{-\infty}^yf_{XY}(x,y')dxdy'\right]}{F_X(x_2)-F_X(x_1)} \\ \\ &=\frac{\int_{-\infty}^{x_2}f_{XY}(x,y)dx-\int_{-\infty}^{x_1}f_{XY}(x,y)dx}{F_X(x_2)-F_X(x_1)} \end{align}

f_Y(y|x_1<X\leq x_2)=\frac{\int_{x_1}^{x_2}f_{XY}(x,y)dx}{F_X(x_2)-F_X(x_1)}

But what we really want is f $$ _Y $$ (y|X = x) ∀x ∈ R. We cannot set x $$ _1 $$ = x $$ _2 $$ in the above equation, so instead, let

f_Y(y|X=x)=\lim_{\Delta x\rightarrow 0}f_Y(y|x<X\leq x+\Delta x)

Setting x $$ _1 $$ = x and x $$ _2 $$ = x + Δx,

f_Y(y|x<X\leq x+\Delta x)=\frac{\int_x^{x+\Delta x}f_{XY}(x',y)dx'}{F_X(x+\Delta x)-F_X(x)}

Multiplying the denominator and numerator by 1/Δx and taking the limit,

f_Y(y|X=x) = \lim_{\Delta x\rightarrow 0}\frac {\frac{1}{\Delta x}\int_x^{x+\Delta x}f_{XY}(x',y)dx'} {\frac{1}{\Delta x}[F_X(x+\Delta x)-F_X(x)]}

Now let

f_Y(y|X=x) = \frac{\lim_{\Delta x\rightarrow 0}\frac{1}{\Delta x}[F(x+\Delta x)-F(x)]}{\lim_{\Delta x\rightarrow 0}\frac{1}{\Delta x}[F_X(x+\Delta x)-F_X(x)]}

The numerator is the derivative of F(x) with respect to x and the denominator is the derivative of F $$ _X $$ (x) with respect to x, so

f_Y(y|X=x) = \frac{f_{XY}(x,y)}{f_X(x)}

Similarly,

f_X(x|Y=y) = \frac{f_{XY}(x,y)}{f_Y(y)}

Notation

f_X(x|Y=y)\equiv f_{X|Y}(x|y) = f(x|y)

Writing two equations above in terms of f $_{XY}$ and setting them equal to each other gives Bayes' formula:

f_{X|Y}(x|y) = \frac{f_{Y|X}(y|x)f_X(x)}{f_Y(y)}

We also have a Total Probability Law:

\begin{align} f_Y(y)&=\int_{-\infty}^{\infty}f_{XY}(x,y)dx \\ &=\int_{-\infty}^{\infty}f_{Y|X}(y|x)f_X(x)dx \end{align}

So we can write f $$ {X|Y} $$ (x|y) in terms of f $$ {Y|X} $$ (x|y)f $$ _X $$ (x), which can be very useful.

Note that if X and Y are independent, we have

\begin{align} f_{Y|X}(y|x)&=\frac{f_{XY}(x,y)}{f_X(x)} \\ &=\frac{f_X(x)f_Y(y)}{f_X(x)} \\ &=f_Y(y) \end{align}

So f $$ {Y|X} $$ (x|y) does not depend on x.

Summary of three forms of Bayes' formula that we have derived:

\bullet P(A|B)=\frac{P(B|A)P(A)}{P(B)}\qquad A,B\in\mathcal F

Use this form when X and y are discrete with A = {Y = y}, B={X = x}, so

\;p_{Y|X}(y|x) =\frac{p_{X|Y}(x|y)p_Y(y)}{p_X(x)}

where p

_{Y|X}

(y|x) ≡ P(Y=y|X=x) and p

_{X|Y}

(x|y) ≡ P(X=x|Y=y) are conditional pdfs.

\bullet P(M|Y=y)=\frac{f_Y(y|M)P(M)}{f_Y(y)}\qquad M\in\mathcal F

Use this when Y is continuous and X is discrete with M = {X = x}, so

\;p_{X|Y}(x|y) =\frac{f_{Y|X}(y|x)p_X(x)}{f_Y(y)}

\bullet f_{Y|X}(y|x)=\frac{f_{X|Y}(x|y)f_Y(y)}{f_X(x)}

Use this when X,Y are both continuous.

References

M. Comer. ECE 600. Class Lecture. Random Variables and Signals. Faculty of Electrical Engineering, Purdue University. Fall 2013.

Questions and comments

If you have any questions, comments, etc. please post them on this page

Back to all ECE 600 notes

ECE600 F13 Conditional Distributions for Two Random Variables mhossain - Rhea

Conditional Distributions

References

Questions and comments

Alumni Liaison