(Created page with "Category:Walther MA271 Fall2020 topic2 =Markov Chain Theorems= A transition matrix is considered '''regular''' if it has all positive entries. This is reflected in our e...")
 
Line 1: Line 1:
 
[[Category:Walther MA271 Fall2020 topic2]]
 
[[Category:Walther MA271 Fall2020 topic2]]
  
=Markov Chain Theorems=
+
=Steady-State Vectors=
  
A transition matrix is considered '''regular''' if it has all positive entries. This is reflected in our example, as the transition matrix has only positive entries, so it must be regular.  
+
A transition matrix is considered '''regular''' if it has all positive entries. This is reflected in our example, as the transition matrix has only positive entries, so it must be regular. Regular Markov chains will eventually approach a fixed vector state <math>q</math>, which is also known as the '''steady-state vector''' of a Markov chain. A steady-state vector is a vector that does not change after multiplying it with the transition matrix, satisfying the equation
 +
 
 +
<center><math>Pq = q</math></center>
 +
 
 +
This can also be written as
 +
 
 +
<center><math>q(I - P) = 0</math></center>
 +
 
 +
Where <math>I</math> is the identity matrix. To demonstrate this with our weather problem,
 +
 
 +
<center><math>\left(\begin{array}{c}q_{1}\\q_{2}\\q_{3}\end{array}\right)(\left(\begin{array}{ccc}1&0&0\\0&1&0\\0&0&1\end{array}\right) - \left(\begin{array}{ccc}0.4&0.1&0.3\\0.2&0.7&0.1\\0.4&0.2&0.6\end{array}\right)) = 0</math></center>
  
 
[[ Walther MA271 Fall2020 topic2|Back to Markov Chains]]
 
[[ Walther MA271 Fall2020 topic2|Back to Markov Chains]]
  
 
[[Category:MA271Fall2020Walther]]
 
[[Category:MA271Fall2020Walther]]

Revision as of 02:16, 6 December 2020


Steady-State Vectors

A transition matrix is considered regular if it has all positive entries. This is reflected in our example, as the transition matrix has only positive entries, so it must be regular. Regular Markov chains will eventually approach a fixed vector state $ q $, which is also known as the steady-state vector of a Markov chain. A steady-state vector is a vector that does not change after multiplying it with the transition matrix, satisfying the equation

$ Pq = q $

This can also be written as

$ q(I - P) = 0 $

Where $ I $ is the identity matrix. To demonstrate this with our weather problem,

$ \left(\begin{array}{c}q_{1}\\q_{2}\\q_{3}\end{array}\right)(\left(\begin{array}{ccc}1&0&0\\0&1&0\\0&0&1\end{array}\right) - \left(\begin{array}{ccc}0.4&0.1&0.3\\0.2&0.7&0.1\\0.4&0.2&0.6\end{array}\right)) = 0 $

Back to Markov Chains

Alumni Liaison

To all math majors: "Mathematics is a wonderfully rich subject."

Dr. Paul Garrett