Latest revision as of 01:38, 6 December 2020

Recurrent State and Transient State

The recurrence probability examines whether the Markov chain can return to a state again n-steps after starting from that state. If it’s true, then we call those recurrent states; otherwise, we call them transient states.

Let $ f_{j}^{n} = P(X_{n} = j, X_{k} \neq j, 1 \leq k \leq n | X_{0} = j) \\ f_{j} = \sum_{n=1}^{\infty} f_{j}^{n} $

If $ f_{j} = 1 $, state $ j $ is recurrent; if $ f_{j} < 1 $, state $ j $ is transient.

If we have a Markov chain with limited states; if $ i \leftrightarrow j $, and $ i $ is recurrent, then so is $ j $.

To get more examples and proofs for the aforementioned theorems and properties, readers can check out this document

Back to Markov Chains