m |
|||
| (4 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
| − | |||
{ Summary: To generate "colored" samples <math>\tilde{x}\in\mathbb{R}^n \sim \mathcal{N}(\mu,\Sigma)</math> from "white" samples <math>x</math> drawn from <math>\mathcal{N}(\vec{0},I_n)</math>, simply let <math>\tilde{x} = Ax + \mu</math>, where <math>A</math> is the Cholesky decomposition of <math>\Sigma</math>, i.e. <math>\Sigma = AA^T</math>} | { Summary: To generate "colored" samples <math>\tilde{x}\in\mathbb{R}^n \sim \mathcal{N}(\mu,\Sigma)</math> from "white" samples <math>x</math> drawn from <math>\mathcal{N}(\vec{0},I_n)</math>, simply let <math>\tilde{x} = Ax + \mu</math>, where <math>A</math> is the Cholesky decomposition of <math>\Sigma</math>, i.e. <math>\Sigma = AA^T</math>} | ||
Consider generating samples <math>\tilde{x}\in\mathbb{R}^n \sim \mathcal{N}(\mu,\Sigma)</math>. Many platforms (e.g. Matlab) have a random number generator to generate iid samples from (white) Gaussian distribution. If we seek to "color" the noise with an arbitrary covariance matrix <math>\Sigma</math>, we must produce a "coloring matrix" <math>A</math>. Let us consider generating a colored sample <math>\tilde{x} = [\tilde{x}_1,\tilde{x}_2,\ldots,\tilde{x}_n]^T</math> from <math>x = [x_1,x_2,\ldots,x_n]^T</math>, where <math>x_1, x_2, \ldots, x_n</math> are iid samples drawn from <math>\mathcal{N}(0,1)</math>. (Note: Matlab has a function, mvnrnd.m, to sample from <math>\mathcal{N}(\mu,\Sigma)</math>, but I discuss here the theory behind it). Relate <math>\tilde{x}</math> to <math>x</math> as follows: | Consider generating samples <math>\tilde{x}\in\mathbb{R}^n \sim \mathcal{N}(\mu,\Sigma)</math>. Many platforms (e.g. Matlab) have a random number generator to generate iid samples from (white) Gaussian distribution. If we seek to "color" the noise with an arbitrary covariance matrix <math>\Sigma</math>, we must produce a "coloring matrix" <math>A</math>. Let us consider generating a colored sample <math>\tilde{x} = [\tilde{x}_1,\tilde{x}_2,\ldots,\tilde{x}_n]^T</math> from <math>x = [x_1,x_2,\ldots,x_n]^T</math>, where <math>x_1, x_2, \ldots, x_n</math> are iid samples drawn from <math>\mathcal{N}(0,1)</math>. (Note: Matlab has a function, mvnrnd.m, to sample from <math>\mathcal{N}(\mu,\Sigma)</math>, but I discuss here the theory behind it). Relate <math>\tilde{x}</math> to <math>x</math> as follows: | ||
| − | <math>\tilde{x}_1 = a_{11} x_1 | + | <math> |
| − | + | \begin{align} | |
| − | + | \tilde{x}_1 &= a_{11} x_1 \\ | |
| − | + | \tilde{x}_2 &= a_{21} x_1 + a_{22} x_2 \\ | |
| − | + | &... \\ | |
| + | \tilde{x}_n &= \sum_{i=1}^n a_{ni}x_i \\ | ||
| + | \end{align} | ||
| + | </math> | ||
We can rewrite this in matrix form as <math>\tilde{x} = Ax</math>, where matrix <math>A</math> is lower triangular. We have, then, that | We can rewrite this in matrix form as <math>\tilde{x} = Ax</math>, where matrix <math>A</math> is lower triangular. We have, then, that | ||
| − | + | <math>E[\tilde{x}_n] = \sum_{i=1}^n a_{ni}E[x_i] = 0</math>, and | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | <math>Cov[\tilde{x}_n,\tilde{x}_m] = E\left[\left(\sum_{i=1}^na_{ni}x_i\right)\left(\sum_{j=1}^m a_{mj}x_j\right)\right] = \sum_{i=1}^n\sum_{j=1}^m a_{ni}a_{mj}E[x_ix_j] \Rightarrow</math> | |
| − | + | ||
| − | + | <math>Cov(\tilde{x}_n,\tilde{x}_m)</math> = <math>\sum_{i=1}^{\min(m,n)}a_{ni}a_{mi}</math>, since <math>x_i</math>'s are independent, <math>E[x_i] = 0</math> and <math>Var[x_i] = 1</math>. | |
| − | + | ||
| − | .. | + | We are now left with the problem of defining <math>a_{ni}</math>'s so that the form of <math>Cov(\tilde{x}_n,\tilde{x}_m)</math> follows the form of <math>\Sigma_{nm}</math>: i.e. |
| − | + | ||
| − | + | <math>\Sigma_{nm}</math> = <math>Cov(\tilde{x}_n,\tilde{x}_m)</math> = <math>\sum_{i=1}^{\min(m,n)}a_{ni}a_{mi} | |
| − | + | </math> | |
| − | + | <math>\Rightarrow \Sigma = AA^T</math>, where <math>A</math> is lower triangular, and <math>\Sigma</math> is positive definite. Therefore, <math>A</math> follows the form of what is called the Cholesky decomposition of <math>\Sigma</math>. | |
| − | + | ||
Latest revision as of 15:12, 20 March 2008
{ Summary: To generate "colored" samples $ \tilde{x}\in\mathbb{R}^n \sim \mathcal{N}(\mu,\Sigma) $ from "white" samples $ x $ drawn from $ \mathcal{N}(\vec{0},I_n) $, simply let $ \tilde{x} = Ax + \mu $, where $ A $ is the Cholesky decomposition of $ \Sigma $, i.e. $ \Sigma = AA^T $}
Consider generating samples $ \tilde{x}\in\mathbb{R}^n \sim \mathcal{N}(\mu,\Sigma) $. Many platforms (e.g. Matlab) have a random number generator to generate iid samples from (white) Gaussian distribution. If we seek to "color" the noise with an arbitrary covariance matrix $ \Sigma $, we must produce a "coloring matrix" $ A $. Let us consider generating a colored sample $ \tilde{x} = [\tilde{x}_1,\tilde{x}_2,\ldots,\tilde{x}_n]^T $ from $ x = [x_1,x_2,\ldots,x_n]^T $, where $ x_1, x_2, \ldots, x_n $ are iid samples drawn from $ \mathcal{N}(0,1) $. (Note: Matlab has a function, mvnrnd.m, to sample from $ \mathcal{N}(\mu,\Sigma) $, but I discuss here the theory behind it). Relate $ \tilde{x} $ to $ x $ as follows:
$ \begin{align} \tilde{x}_1 &= a_{11} x_1 \\ \tilde{x}_2 &= a_{21} x_1 + a_{22} x_2 \\ &... \\ \tilde{x}_n &= \sum_{i=1}^n a_{ni}x_i \\ \end{align} $
We can rewrite this in matrix form as $ \tilde{x} = Ax $, where matrix $ A $ is lower triangular. We have, then, that
$ E[\tilde{x}_n] = \sum_{i=1}^n a_{ni}E[x_i] = 0 $, and
$ Cov[\tilde{x}_n,\tilde{x}_m] = E\left[\left(\sum_{i=1}^na_{ni}x_i\right)\left(\sum_{j=1}^m a_{mj}x_j\right)\right] = \sum_{i=1}^n\sum_{j=1}^m a_{ni}a_{mj}E[x_ix_j] \Rightarrow $
$ Cov(\tilde{x}_n,\tilde{x}_m) $ = $ \sum_{i=1}^{\min(m,n)}a_{ni}a_{mi} $, since $ x_i $'s are independent, $ E[x_i] = 0 $ and $ Var[x_i] = 1 $.
We are now left with the problem of defining $ a_{ni} $'s so that the form of $ Cov(\tilde{x}_n,\tilde{x}_m) $ follows the form of $ \Sigma_{nm} $: i.e.
$ \Sigma_{nm} $ = $ Cov(\tilde{x}_n,\tilde{x}_m) $ = $ \sum_{i=1}^{\min(m,n)}a_{ni}a_{mi} $
$ \Rightarrow \Sigma = AA^T $, where $ A $ is lower triangular, and $ \Sigma $ is positive definite. Therefore, $ A $ follows the form of what is called the Cholesky decomposition of $ \Sigma $.
