Classification Of States and Periodicity of Markov Chain with Solved Problems

In Markov Chain, there is the concept of accessibility of states from each other. Say, if it is possible to go from state i to state j, we say that the state j is accessible from state i.
Also, two states are said to communicate if they are accessible from each other. Therefore, the states of a Markov Chain can be divided into communication classed such that one member of the same class communicate with each other. i.e, two states i and j belong to the same class if and only if state i and j communicate with each other.

Jump to Probability and Queuing Theory Index Page

Some of the states are:

• Transient State
• Absorbing State
• Recurring State
• Reducible State
• Irreducible State

Transient State is the states which can transit to another state. 

Absorbing State is the state that does not transit and loop continuously on it.

Recurring State is the state which transits to more than 2 states and comes to its initial state. 

Reducible State is the state that does not communicate with other states. 

Irreducible State is the state that does communicate with other states and with itself. 

Condition for Irreducible State:

• Period of all States is the same. 

---

Periodicity of a Markov Chain

Let 0,1 and 2 be different states, then the one step Transition Probability Matrix is given by,

We have the formula to find the Period of state i, 

---

Solved Problems of Periodicity of Markov Chain

Solved Problem 1:

Suppose that a Markov Chain with 3 states has a Transition Probability Matrix given below:
i. Find the period of all states.
ii. Find whether or not the Markov Chain is irreducible.

= Solution:

---

Related Posts:

• Stochastic Process- Definition, Specifications, and Classification
• Markov Chain: Definition and Representation with Solved Problems
• Stationary Distribution | Steady State Distribution with Solved Problems
• n-Step Transition Probability Matrix of a Two-state Markov Chain with Solved problems
• Birth Death Process
• Chapman-Kolmogorov Equation