Latest revision as of 12:34, 6 December 2020

Restrictions of Stationary Distribution

In the last section, it is emphasized that steady-state vectors can be derived only with regular matrices. What if these vectors are not regular?

Consider a periodic Markov chain:

We can also write that in matrix form:

Going back to the Python program from before, when the initial state is $ [1, 0, 0] $,

When the initial state is $ [\frac{1}{3}, \frac{1}{3}, \frac{1}{3}] $, while the transition probability matrix stays the same, the result looks like this:

From this example, we know that: for a periodic Markov chain, the long-term distribution is dependent on the initial state vector.

Conducting similar experiments, we can reach a more general conclusion: any aperiodic and irreducible finite Markov chain has precisely one stationary distribution. Otherwise, there might be more than one stationary distribution or no stationary distribution at all.

Now, when referring to the Classification of States section to examine the weather example, we can see that the transition probability matrix for the weather example is indeed aperiodic and irreducible. Therefore we get one stationary distribution, as demonstrated above using python.

Back to Markov Chains