View source for ECE600 F13 rv Functions of random variable mhossain - Rhea

[[Category:ECE600]]
[[Category:Lecture notes]]

[[ECE600_F13_notes_mhossain|Back to all ECE 600 notes]]<br/>
[[ECE600_F13_rv_conditional_distribution_mhossain|Previous Topic: Conditional Distributions]]<br/>
[[ECE600_F13_Expectation_mhossain|Next Topic: Expectation]]
----
[[Category:ECE600]]
[[Category:probability]]
[[Category:lecture notes]]
[[Category:slecture]]

<center><font size= 4>
[[ECE600_F13_notes_mhossain|'''The Comer Lectures on Random Variables and Signals''']]
</font size>

[https://www.projectrhea.org/learning/slectures.php Slectures] by [[user:Mhossain | Maliha Hossain]] 


<font size= 3> Topic 8: Functions of Random Variables</font size>
</center>

----
----

We often do not work with the random variable we observe directly, but with some function of that random variable. So, instead of working with a random variable X, we might instead have some random variable Y=g(X) for some function g:'''R''' → '''R'''.<br/>
In this case, we might model Y directly to get f<math>_Y</math>(y), especially if we do not know g. Or we might have a model for X and find f<math>_Y</math>(y) (or p<math>_Y</math>(y)) as a function of f<math>_X</math> (or p<math>_X</math> and g. <br/>
We will discuss the latter approach here.

More formally, let X be a random variable on (''S,F'',P) and consider a mapping g:'''R''' → '''R'''. Then let Y<math>(\omega)=</math>g(X(<math>\omega))</math> ∀<math>\omega</math> ∈ ''S''.<br/>
We normally write this as Y=g(X).

Graphically, 

<center>[[Image:fig1_functions_on_rv.png|400px|thumb|left|Fig 1: Mapping from ''S'' to X to Y under g]]</center>


Is ''Y'' a random variable? We must have Y<math>^{-1}</math>(A) ≡ {<math>\omega</math> ∈ ''S'': Y<math>(\omega)</math> ∈ A} = {<math>\omega</math> ∈ ''S'': g(X<math>(\omega)</math>) ∈ A} be an element of ''F'' ∀A ∈ B('''R''') (Y must be Borel measurable). <br/>
We will only consider functions g in this class for which  Y<math>^{-1}</math>(A) ∈ ''F'' ∀A ∈ B('''R'''), so that if Y=g(X) for some random variable X, Y will be a random variable. 

What is the distribution of Y? Consider 3 cases:
# X discrete, Y discrete
# X continuous, Y discrete
# X continuous, Y continuous

Note: you cannot have a continuous Y from a discrete X.


----

==Case 1: X and Y Discrete ==

Let <math>R_X</math> ≡ X(''S'') be the range space of X and <math>R_Y</math> ≡ g(X(''S'')) be the range space of Y (i.e. the image of X(''S'') under g). Then the pmf of Y is <br/>
<center>p<math>_Y</math>(y) = P(Y=y) = P(g(X)=y)</center>
But this means that <br/>
<center><math>p_Y(y) = \sum_{x\in\mathcal{R}_X:g(x)=y}p_X(x)\;\;\forall y\in\mathcal{R}_Y</math></center>


'''Example''' <math>\quad</math> Let X be the value rolled on a die and <br/>
<center><math>Y = \begin{cases}
1 & \mbox{if}\;X\;\mbox{is odd} \\ 
0 & \mbox{if}\;X\;\mbox{is even} 
\end{cases}</math></center>

Then ''R''<math>_X</math> = {0,1,2,3,4,5,6} and ''R''<math>_Y</math> = {0,1} and g(x) = x % 2.

Now <center><math>p_Y(y) = \sum_{x\in\mathcal{R}_X:g(x)=y}p_X(x)</math></center>
<center><math>\begin{align}
\Rightarrow p_Y(0) &= p_X(2)+p_X(4)+p_X(6) \\
p_Y(1) &= p_X(1)+p_X(3)+p_X(5)
\end{align}</math></center>


----

==Case 2: X Continuous, Y Discrete ==

The pmf of Y in this case is <br/>
<center>p<math>_Y</math>(y) = P(g(X)=y) = P(X ∈ D<math>_y</math>)</center><br/>
<center>where D<math>_y</math> ≡ {x ∈ '''R''': g(x)=y} ∀y ∈ ''R''<math>_y</math></center>
i.e. for a given y ∈ ''R''<math>_y</math>, D<math>_y</math> is the set of all x ∈ '''R''' such that g(x) = y.

Then, <br/>
<center><math> p_Y(y) = \int_{D_y}f_X(x)dx</math></center>

'''Example''' Let g(x) = u(x - x<math>_0</math>) for some x<math>_0</math> ∈ '''R''', and let Y=g(X). Then <math>R_Y</math> = {0,1} and <br/>
<center> D<math>_0</math> = {x ∈ '''R''': x < x<math>_0</math>} = (-∞, x<math>_0</math>)</center>
<center> D<math>_1</math> = {x ∈ '''R''': x ≥ x<math>_0</math>} = [ x<math>_0</math>, ∞)</center>
So, <br/>
<center><math> p_Y(y) = \begin{cases}
\int_{-\infty}^{x_0} f_X(x)dx & y=0\\
\\
\int_{x_0}^{-\infty} f_X(x)dx & y=1
\end{cases}</math></center>


----

==Case 3: X and Y Continuous ==

We will discuss 2 methods for finding f<math>_Y</math> in this case. 

'''Approach 1'''<br/>
First, find the cdf F<math>_Y</math>. <br/>
<center>F<math>_Y</math>(y) = P(g(X) ≤ y) = P(X ∈ D<math>_y</math>) <br/>
where D<math>_y</math> = {x ∈ '''R''': g(x) ≤ y}.</center>
i.e. for a given y ∈ '''R''', D<math>_y</math> is the set of all x ∈ '''R''' such that g(x) ≤ y.

Then <br/>
<center><math>F_Y(f) = \int_{D_y}f_X(x)dx</math></center>

Differentiate F<math>_Y</math> to get f<math>_y</math>.

You can find D<math>_Y</math> graphically or analytically


'''Example'''

<center>[[Image:fig2_functions_on_rv.png|400px|thumb|left|Fig 2: This plot of g(x) can be used to derive D<math>_Y</math> graphically]]</center>


For y = y<math>_1</math> and y = y<math>_2</math>, <br/>
<center><math>\begin{align}
D_{y_1} &= \{x:\;x \leq x_1\} \\
D_{y_2} &= \{x:\;x\leq x_2'\} \cup \{x:\;x_2''<x\leq x_2'''\}
\end{align}</math></center>

Then <br/>
<center><math>\begin{align}
F_Y(y_1) &= \int_{-\infty}^{x_1}f_X(x)dx \\
\\
F_Y(y_2) &= \int_{-\infty}^{x_2'}f_X(x)dx + \int_{x_2''}^{x_2'''}f_X(x)dx
\end{align}</math></center>


'''Example''' Y = aX + b, a,b ∈ '''R''', a ≠ 0<br/>
<center>F<math>_Y</math>(y) = P(aX + b ≤ y)</center>
So, <br/>
<center><math>\begin{align}
D_y&=\{x\in\mathbb R: x\leq\frac{y-b}{a}\}\quad\mbox{if}\;a>0 \\
D_y&=\{x\in\mathbb R: x\geq\frac{y-b}{a}\}\quad\mbox{if}\;a<0
\end{align}</math></center>
Then <br/>
<center><math>F_Y(y)=\begin{cases}
\int_{-\infty}^{\frac{y-b}{a}}f_X(x)dx & \mbox{if }\;a>0 \\
\\
\int_{\frac{y-b}{a}}^{-\infty}f_X(x)dx & \mbox{if }\;a<0
\end{cases}</math></center>


'''Example''' Y = X<math>^2</math> <br/>

<center>[[Image:fig3_functions_on_rv.png|400px|thumb|left|Fig 3: Y = X<math>^2</math>]]</center>


For y < 0, D<math>_y</math> = ø <br/>
For y ≥ 0, <br/>
<center><math>\begin{align}
F_Y(y) &= P(X^2\leq y) \\
&= P(-\sqrt{y} <X\leq \sqrt{y})
\end{align}</math></center>
So, <br/>
<center><math> D_y = (-\sqrt{y},\sqrt{y})</math></center>
and <br/>
<center><math>F_Y(y) = \int_{-\sqrt{y}}^{\sqrt{y}}f_X(x)dx</math></center>

For general y, we need to find subsets of the y-axis that have solutions of the same form and solve the problems separately for the different subsets. 


'''Approach 2'''

Use a formula for f<math>_y</math> in terms of f<math>_X</math>. To derive the formula, assume the inverse function g<math>^{-1}</math> exists, so if y = g(x), then x = g<math>^{-1}</math>(y). Also assume g and  g<math>^{-1}</math> are differentiable. Then, if Y = g(X), we have that <br/>
<center><math> f_Y(y) = \frac{f_X(g^{-1}(y))}{|\frac{dy}{dx}|_{x=g^{-1}(y)}}</math></center>

'''Proof:'''<br/>
First consider g monotone (strictly monotone) increasing (note that for differentiable and hence continuous functions defined for a given interval, injection implies monotonicity, hence it is sufficient to limit our analysis to monotonic functions only).

<center>[[Image:fig4_functions_on_rv.png|400px|thumb|left|Fig 4: Function g is strictly increasing on its domain.]]</center>


Since {y < Y ≤ y + Δy} = {x < X ≤ x + Δx}, we have that P(y < Y ≤ y + Δy) = P(x < X ≤ x + Δx). 

Use the following approximations:<br/>
*P(y < Y ≤ y + Δy) ≈ f<math>_Y</math>(y)Δy
*P(x < X ≤ x + Δx) ≈ f<math>_X</math>(x)Δx

<center>[[Image:fig5_functions_on_rv.png|400px|thumb|left|Fig 5: P(y < Y ≤ y + Δy) ≈ f<math>_Y</math>(y)Δy]]</center>


Since the left hand sides are equal, <br/>
<center><math>f_Y(y)\Delta y \approx f_X(x)\Delta x</math></center>

Now as Δy → 0, we also have that Δx → 0 since g is continuous, and the approximations above become equalities. We rename Δy, Δx as dy and dx respectively, so letting Δy → 0, we get<br/>
<center><math>\begin{align}
f_Y(y)dy &= f_X(x)dx \\
\Rightarrow f_Y(y)&=f_X(x)\frac{dx}{dy}
\end{align}</math></center>


We normally write this as <br/>
<center><math> f_Y(y) = \frac{f_X(g^{-1}(y))}{\frac{dy}{dx}|_{x=g^{-1}(y)}}</math></center>

A similar derivation for g monotone decreasing gives us the general result for invertible g:<br/>
<center><math> f_Y(y) = \frac{f_X(g^{-1}(y))}{|\frac{dy}{dx}|_{x=g^{-1}(y)}}</math></center>

Note this result can be extended to the case where y = g(x) has n solutions x<math>_1</math>,...,x<math>_n</math>, in which case, <br/>
<center><math>f_Y(y) = \sum_{i=1}^n\frac{f_X(x_i)}{|\frac{dy}{dx}|_{x=x_i}}</math></center>

For example, if Y = X<math>^2</math>,<br/>
<center><math>x_1 = -\sqrt{y},\;\;x_2 = \sqrt{y}</math></center>
<center><math>\Rightarrow f_Y(y) = \frac{f_X(-\sqrt{y})}{2\sqrt{y}}+\frac{f_X(\sqrt{y})}{2\sqrt{y}}</math></center>



----

== References ==

* [https://engineering.purdue.edu/~comerm/ M. Comer]. ECE 600. Class Lecture. [https://engineering.purdue.edu/~comerm/600 Random Variables and Signals]. Faculty of Electrical Engineering, Purdue University. Fall 2013.


----

==[[Talk:ECE600_F13_rv_Functions_of_random_variable_mhossain|Questions and comments]]==

If you have any questions, comments, etc. please post them on [[Talk:ECE600_F13_rv_Functions_of_random_variable_mhossain|this page]]


----


[[ECE600_F13_notes_mhossain|Back to all ECE 600 notes]]<br/>
[[ECE600_F13_rv_conditional_distribution_mhossain|Previous Topic: Conditional Distributions]]<br/>
[[ECE600_F13_Expectation_mhossain|Next Topic: Expectation]]