Steady-State Vectors
A transition matrix is considered regular if it has all positive entries. This is reflected in our example, as the transition matrix has only positive entries, so it must be regular. Regular Markov chains will eventually approach a fixed vector state $ q $, which is also known as the steady-state vector of a Markov chain. A steady-state vector is a vector that does not change after multiplying it with the transition matrix, satisfying the equation
This can also be written as
Where $ I $ is the identity matrix. To demonstrate this with our weather problem,
Here, we set up the equation to fit our example.
Next, we simplify the transition matrix to reduced row echelon form (RREF). The reader is encouraged to refer to the additional readings section to learn more about this process. However, simply put, the rows are manipulated so that the transition matrix as a whole will have leading ones in each row, with each subsequent row having a leading one that is to the right of the previous row’s leading one.
The matrix can be viewed as a system of equations, where each row is a separate equation and each column is a separate variable. Then, each equation is solved. In the first two rows, q1 and q2 are solved in terms of q3. However, the third equation does not give any information for the value of q3 because everything is just zero. Therefore, we set q3 to be equal to variable s. The new q is shown above.
To solve for $ s $,
Plugging s back into q, we get
Notice that the values of $ q_{1} $, $ q_{2} $, and $ q_{3} $ are identical to the values in the state vector from the Python demonstration after thirty iterations. The steady-state vector provides the long-run probabilities for the Markov chain.
