Random Variables and Signals
Topic 14: Joint Expectation
Contents
Joint Expectation
Given random variables X and Y, let Z = g(X,Y) for some g:R$ _2 $→R. Then E[Z] can be computed using f$ _Z $(z) or p$ _Z $(z) in the original definition of E[ ]. Or, we can use
or
The proof is in Papoulis.
We will use joint expectation to define some important moments that help characterize the joint behavior of X and Y.
Note that joint expectation is a linear operator, so if g$ _1 $,...,g$ _n $ are n functions from R$ ^2 $ to R and a$ _1 $,...,a$ _n $ are constants, then
Important Moments for X,Y
We still need $ \mu_X $, $ \mu_Y $, $ \sigma_X $$ ^2 $, $ \sigma_X $$ ^2 $, the means and variances of X and Y. Other moments that are of great interest are:
- The correlation between X and Y is
- The covariance of X and Y
- The correlation coefficient of X and Y is
- Note:
- |r$ _{XY} $| ≤ 1 (proof)
- If X and Y are independent, then r$ _{XY} $ = 0. The converse is not true in general.
- If r$ _{XY} $=0, then X and Y are said to be uncorrelated. It can be shown that
- X and Yare uncorrelated iff Cov(X,Y)=0 (proof).
- X and Y are uncorrelated iff E[XY] = $ \mu_X\mu_Y $ (proof).
- X and Y are orthogonal if E[XY]=0.
The Cauchy Schwarz Inequality
For random variables X and Y,
with equality iff Y = a$ _0 $X with probability 1, where a$ _0 $ is a constant. Note that "equality with probability 1" will be defined later.
Proof: $ \qquad $ We start by considering
which is a quadratic function of a ∈ R. Consider two cases:
- (i)$ \; $ E[(aX-Y)^2] > 0
- (ii) E[(aX-Y)^2] = 0
Case (i):
Since this quadratic is greater than 0 for all a, there are no real roots. There must be two complex roots. From the quadratic equation, this means that
Case (ii):
In this case
This means that ∃a ∈ R such that
It can be shown that if a random variable X has E[X$ ^2 $]=0, then X=0 except possibly on a set of probability 0. Note that previously, we have defined equality between random variables X and Y to mean
With the same notion of equality, we would have that
However, the result for the case E[X$ ^2 $]=0 is that X=0 with probability 1, which means that there is a set A = {$ \omega $ ∈ S: X($ \omega $) = 0} with P(A) = 1.
Returning to the case E[(a$ _0 $X-Y)$ ^2 $]=0, we get that Y = a$ _0 $X for some a$ _0 $ ∈ R, with probability 1.
Thus we have shown that
with equality iff Y = a$ _0 $X with probability 1.
The Correlation Coefficient for Jointly Gaussian Random Variables X and Y
We have previously seen that the joint density function for Gaussian X and Y contains a parameter r. It can be shown that this r is the correlation coeffecient of X and Y:
Note that one way to show this is to use a concept called iterated expectation, which we will cover later.
Joint Moments
Definition $ \qquad $ The joint moments of X and Y are
and the joint central moments are
for j = 0,1,...; k = 0,1,...
Joint Characteristic Function
Definition $ \qquad $ The joint characteristic function of X and Y is
for $ \omega_1,\omega_2 $ ∈ R.
If X and Y are continuous, we write this as
If X and Y are discrete, we use
Note:
- $ \bullet\; \Phi_X(\omega)=\Phi_{XY}(\omega,0);\quad\Phi_Y(\omega)=\Phi_{XY}(0,\omega) $
- (proof)
- $ \bullet\; Z = aX+bY \;\Rightarrow\;\Phi_Z(\omega) =\Phi_{XY}(a\omega,b\omega) $
- (proof)
- $ \bullet\; X \perp\!\!\!\perp Y \;\mbox{iff}\;\Phi_{XY}(\omega_1,\omega_2)=\Phi_X(\omega_1)\Phi_Y(\omega_2) $
- (proof)
- $ \bullet\; X \perp\!\!\!\perp Y \;\mbox{and}\;Z=X+Y\;\Rightarrow\Phi_{Z}(\omega)=\Phi_X(\omega)\Phi_Y(\omega) $
- (proof)
Moment Generating Function
The joint moment generating function of X and Y is
Then it can be shown that
References
- M. Comer. ECE 600. Class Lecture. Random Variables and Signals. Faculty of Electrical Engineering, Purdue University. Fall 2013.
Questions and comments
If you have any questions, comments, etc. please post them on this page
