Revision as of 13:34, 6 December 2020 by Nlfang (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)


Restrictions of Stationary Distribution

In the last section, it is emphasized that steady-state vectors can be derived only with regular matrices. What if these vectors are not regular?

Consider a periodic Markov chain:

Statediagram.jpg

We can also write that in matrix form:

Transitionmatrix2.jpg

Going back to the Python program from before, when the initial state is $ [1, 0, 0] $,

Pythondemo7.png

When the initial state is $ [\frac{1}{3}, \frac{1}{3}, \frac{1}{3}] $, while the transition probability matrix stays the same, the result looks like this:

Pythondemo8.png

From this example, we know that: for a periodic Markov chain, the long-term distribution is dependent on the initial state vector.

Conducting similar experiments, we can reach a more general conclusion: any aperiodic and irreducible finite Markov chain has precisely one stationary distribution. Otherwise, there might be more than one stationary distribution or no stationary distribution at all.

Now, when referring to the Classification of States section to examine the weather example, we can see that the transition probability matrix for the weather example is indeed aperiodic and irreducible. Therefore we get one stationary distribution, as demonstrated above using python.


Back to Markov Chains

Alumni Liaison

Ph.D. on Applied Mathematics in Aug 2007. Involved on applications of image super-resolution to electron microscopy

Francisco Blanco-Silva