Recurrent State and Transient State
The recurrence probability examines whether the Markov chain can return to a state again n-steps after starting from that state. If it’s true, then we call those recurrent states; otherwise, we call them transient states.
Let $ f_{j}^{n} = P(X_{n} = j, X_{k} \neq j, 1 \leq k \leq n | X_{0} = j) \\ f_{j} = \sum_{n=1}^{\infty} f_{j}^{n} $
If $ f_{j} = 1 $, state $ j $ is recurrent; if $ f_{j} < 1 $, state $ j $ is transient.
If we have a Markov chain with limited states; if $ i \leftrightarrow j $, and $ i $ is recurrent, then so is $ j $.
To get more examples and proofs for the aforementioned theorems and properties, readers can check out this document
