3.2 Systems with Stochastic Inputs

Given a random process $ \mathbf{X}\left(t\right) $ , if we assign a new sample function $ \mathbf{Y}\left(t,\omega\right) $ to each sample function $ \mathbf{X}\left(t,\omega\right) $ , then we have a new random process $ \mathbf{Y}\left(t\right) $ : $ \mathbf{Y}\left(t\right)=T\left[\mathbf{X}\left(t\right)\right] $ .

Note

We will assume that $ T $ is deterministic (NOT random). Think of $ \mathbf{X}\left(t\right)=\text{input to a system} $. $ \mathbf{Y}\left(t\right)=\text{output of a system} $.

$ \mathbf{Y}\left(t,\omega\right)=T\left[\mathbf{X}\left(t,\omega\right)\right],\quad\forall\omega\in\mathcal{S} $.

In ECE, we are often interested in finding a statistical description of $ \mathbf{Y}\left(t\right) $ in terms of that of $ \mathbf{X}\left(t\right) $ . For general $ T\left[\cdot\right] $ , this is very difficult. We will look at two special cases:

1. Memoryless system

2. Linear time-invariant system

3.2.1 Memoryless System

Definition

A system is called memoryless if its output $ \mathbf{Y}\left(t\right)=g\left(\mathbf{X}\left(t\right)\right) $ , where $ g:\mathbf{R}\rightarrow\mathbf{R} $ is only a function of its current argument $ x $ .

$ g\left(\cdot\right) $ is not a function of the past or future values of input.

$ \mathbf{Y}\left(t\right)=g\left(\mathbf{X}\left(t\right)\right) $ depends only on the instantaneous value of $ \mathbf{X}\left(t\right) $ at time $ t $ .

Example

Square is a memoryless system.

$ g\left(x\right)=x^{2}. $

$ \mathbf{Y}\left(t\right)=g\left(\mathbf{X}\left(t\right)\right)=\mathbf{X}^{2}\left(t\right). $

Example

Integrators are NOT memoryless. They have memory of the past.

$ \mathbf{Y}\left(t\right)=\int_{-\infty}^{t}\mathbf{X}\left(\alpha\right)d\alpha. $

$ \mathbf{Y}\left(t,\omega\right)=\int_{-\infty}^{t}\mathbf{X}\left(\alpha,\omega\right)d\alpha. $

Note

For memoryless systems, the first-order density $ f_{\mathbf{Y}\left(t\right)}\left(y\right) $ of $ \mathbf{Y}\left(t\right) $ can be expressed in terms of the first-order density of $ \mathbf{X}\left(t\right) $ and $ g\left(\cdot\right) $ . This is just a simple function of a random variable. Also $ E\left[\mathbf{Y}\left(t\right)\right]=E\left[g\left(\mathbf{X}\left(t\right)\right)\right]=\int_{-\infty}^{\infty}g\left(x\right)f_{\mathbf{X}\left(t\right)}\left(x\right)dx $ and $ R_{\mathbf{YY}}\left(t_{1},t_{2}\right)=E\left[\mathbf{Y}\left(t_{1}\right)\mathbf{Y}\left(t_{2}\right)\right]=\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}g\left(x_{1}\right)g\left(x_{2}\right)f_{\mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)}\left(x_{1},x_{2}\right)dx_{1}dx_{2}. $

Also, you can get the n-th order pdf of $ \mathbf{Y}\left(t\right) $ using the mapping $ \mathbf{Y}\left(t_{1}\right)=g\left(\mathbf{X}\left(t_{1}\right)\right),\cdots,\mathbf{Y}\left(t_{n}\right)=g\left(\mathbf{X}\left(t_{n}\right)\right) $ .

Theorem

Let $ \mathbf{X}\left(t\right) $ be a S.S.S. random process that is the input to a memoryless system. Then the output \mathbf{Y}\left(t\right) is also a S.S.S. random process.

Example. Hard limiter

Consider a memoryless system with

$ g\left(x\right)=\left\{ \begin{array}{lll} +1 ,x>0\\ -1 ,x\leq0. \end{array}\right. $

Consider $ \mathbf{Y}\left(t\right)=g\left(\mathbf{X}\left(t\right)\right)=\textrm{sgn}\left(\mathbf{X}\left(t\right)\right) $ . Find $ E\left[\mathbf{Y}\left(t\right)\right] $ and $ R_{\mathbf{YY}}\left(t_{1},t_{2}\right) $ given the “statistics” of $ \mathbf{X}\left(t\right) $ .

Solution

$ E\left[\mathbf{Y}\left(t\right)\right]=\left(+1\right)\cdot P\left(\left\{ \mathbf{Y}\left(t\right)=+1\right\} \right)+\left(-1\right)\cdot P\left(\left\{ \mathbf{Y}\left(t\right)=-1\right\} \right) $$ =\left(+1\right)\cdot\left(1-F_{\mathbf{X}\left(t\right)}\left(0\right)\right)+\left(-1\right)\cdot F_{\mathbf{X}\left(t\right)}\left(0\right)=1-2F_{\mathbf{X}\left(t\right)}\left(0\right). $

$ R_{\mathbf{YY}}\left(t_{1},t_{2}\right)=E\left[\mathbf{Y}\left(t_{1}\right)\mathbf{Y}\left(t_{2}\right)\right]=\left(+1\right)\cdot P\left(\left\{ \mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)>0\right\} \right)+\left(-1\right)\cdot P\left(\left\{ \mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)\leq0\right\} \right) $

where $ P\left(\left\{ \mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)>0\right\} \right)=\int_{0}^{\infty}\int_{0}^{\infty}f_{\mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)}\left(x_{1},x_{2}\right)dx_{1}dx_{2}+\int_{-\infty}^{0}\int_{-\infty}^{0}f_{\mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)}\left(x_{1},x_{2}\right)dx_{1}dx_{2} $ .

Example

$ \mathbf{X}\left(t\right)=\mathbf{A}\cdot\cos\left(\omega_{0}t+\mathbf{\Theta}\right) $ where $ \mathbf{A} $ and $ \mathbf{\Theta} $ are independent random variables and $ \mathbf{\Theta}\sim u\left[0,2\pi\right) $ . Assume that $ \mathbf{A} $ has a mean $ \mu_{\mathbf{A}} $ and a variance $ \sigma_{\mathbf{A}}^{2} $ . Is $ \mathbf{X}\left(t\right) $ a WSS random process?

Solution

• Check whether $ E\left[\mathbf{X}\left(t\right)\right] $ is constant or not: $ E\left[\mathbf{X}\left(t\right)\right] $

• Check whether $ R_{\mathbf{XX}}\left(t_{1},t_{2}\right)=R_{\mathbf{X}}\left(\tau\right) : $ $ R_{\mathbf{XX}}\left(t_{1},t_{2}\right) = E\left[\mathbf{X}\left(t_{1}\right)\mathbf{X}\left(t_{2}\right)\right] $$ =E\left[\mathbf{A}\cdot\cos\left(\omega_{0}t_{1}+\mathbf{\Theta}\right)\cdot\mathbf{A}\cdot\cos\left(\omega_{0}t_{2}+\mathbf{\Theta}\right)\right] $$ =E\left[\mathbf{A}^{2}\right]\cdot E\left[\cos\left(\omega_{0}t_{1}+\mathbf{\Theta}\right)\cdot\cos\left(\omega_{0}t_{2}+\mathbf{\Theta}\right)\right] $$ =\left(\sigma_{\mathbf{A}}^{2}+\mu_{\mathbf{A}}^{2}\right)\cdot E\left[\cos\left(\omega_{0}t_{1}+\mathbf{\Theta}\right)\cdot\cos\left(\omega_{0}t_{2}+\mathbf{\Theta}\right)\right] $$ =\left(\sigma_{\mathbf{A}}^{2}+\mu_{\mathbf{A}}^{2}\right)\cdot\left\{ \frac{1}{2}E\left[\cos\left(\omega_{0}\left(t_{1}+t_{2}\right)+\mathbf{2\Theta}\right)\right]+\frac{1}{2}E\left[\cos\left(\omega_{0}\left(t_{1}-t_{2}\right)\right)\right]\right\} $$ =\frac{1}{2}\left(\sigma_{\mathbf{A}}^{2}+\mu_{\mathbf{A}}^{2}\right)\cos\left(\omega_{0}\left(t_{1}-t_{2}\right)\right). $

$ \therefore\mathbf{X}\left(t\right) $ is WSS.

Recall

$ \cos\alpha\cos\beta=\frac{1}{2}\left\{ \cos\left(\alpha+\beta\right)+\cos\left(\alpha-\beta\right)\right\} $ .

3.2.2 LTI (Linear Time-Invariant) system

Linear Systems

A linear system L\left[\cdot\right] is a transformation rule satisfying the following properties.

1. $ L\left[\mathbf{X}_{1}\left(t\right)+\mathbf{X}_{2}\left(t\right)\right]=L\left[\mathbf{X}_{1}\left(t\right)\right]+L\left[\mathbf{X}_{2}\left(t\right)\right] $ .

2. $ L\left[\mathbf{A}\cdot\mathbf{X}\left(t\right)\right]=\mathbf{A}\cdot L\left[\mathbf{X}\left(t\right)\right] $ .

Time-invariant

A (linear) system is time-invariant if, given response $ \mathbf{Y}\left(t\right) $ for an input $ \mathbf{X}\left(t\right) $ , it has response $ \mathbf{Y}\left(t+c\right) $ for input $ \mathbf{X}\left(t+c\right) $ , for all $ c\in\mathbb{R} $ .

LTI

A linear time-invariant system is one that is both linear and time-invariant. A LTI system is characterized by its impulse response $ h\left(t\right) $ :

Pasted39.png

If we put a random process $ \mathbf{X}\left(t\right) $ into a LTI system, we get a random process $ \mathbf{Y}\left(t\right) $ out of the system. $ \mathbf{Y}\left(t\right)=\mathbf{X}\left(t\right)*h\left(t\right)=\int_{-\infty}^{\infty}\mathbf{X}\left(t-\alpha\right)h\left(\alpha\right)d\alpha=\int_{-\infty}^{\infty}\mathbf{X}\left(\alpha\right)h\left(t-\alpha\right)d\alpha. $

Important Facts

1. If the input to a LTI system is a Gaussian random process, the output is a Gaussian random process.

2. If the input to a stable L.T.I. system is S.S.S., so is the output. L.T.I. system is stable if $ \int_{-\infty}^{\infty}\left|h\left(t\right)\right|dt<\infty. $

Fundamental Theorem

• For any linear system we will encounter $ E\left[L\left[\mathbf{X}\left(t\right)\right]\right]=L\left[E\left[\mathbf{X}\left(t\right)\right]\right]. $

• Applying this to a L.T.I. system, we get $ E\left[\mathbf{Y}\left(t\right)\right]=E\left[\int_{-\infty}^{\infty}\mathbf{X}\left(t-\alpha\right)h\left(\alpha\right)d\alpha\right]=\int_{-\infty}^{\infty}E\left[\mathbf{X}\left(t-\alpha\right)h\left(\alpha\right)\right]d\alpha=\int_{-\infty}^{\infty}\eta_{\mathbf{X}}\left(t-\alpha\right)h\left(\alpha\right)d\alpha. $$ \therefore\eta_{\mathbf{Y}}\left(t\right)=E\left[\mathbf{Y}\left(t\right)\right]=\eta_{\mathbf{X}}\left(t\right)*h\left(t\right). $

Output Autocorrelation

$ R_{\mathbf{YY}}\left(t_{1},t_{2}\right)=E\left[\mathbf{Y}\left(t_{1}\right)\mathbf{Y}\left(t_{2}\right)\right]=E\left[\int_{-\infty}^{\infty}\mathbf{X}\left(t_{1}-\alpha\right)h\left(\alpha\right)d\alpha\cdot\int_{-\infty}^{\infty}\mathbf{X}\left(t_{2}-\beta\right)h\left(\beta\right)d\beta\right] $$ =\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}E\left[\mathbf{X}\left(t_{1}-\alpha\right)\mathbf{X}\left(t_{2}-\beta\right)\right]h\left(\alpha\right)h\left(\beta\right)d\alpha d\beta $$ =\int_{-\infty}^{\infty}\int_{-\infty}^{\infty}R_{\mathbf{XX}}\left(t_{1}-\alpha,t_{2}-\beta\right)h\left(\alpha\right)h\left(\beta\right)d\alpha d\beta. $

Theorem

If the input to a stable LTI system is WSS, so is the output.


Back to ECE600

Back to General Concepts of Stochastic Processes

Alumni Liaison

Basic linear algebra uncovers and clarifies very important geometry and algebra.

Dr. Paul Garrett