n-Step Transition Probability Matrix of a Two-state Markov Chain with Solved problems

Jump to Probability and Queuing Theory Index Page

Let {X_n,n ≥ 0} be a Markov Chain (MC) with Transition Probability matrix P with two states (0 and 1) which can represent two different events.

Here in the given figure,
probability of transition from state 0 to 1 is P₀₁  =  a
probability of transition from state 1 to 0 is P₁₀  =  b
so, P₀₀  =  1 - P₀₁ = 1- a
and, P₁₁  =  1- P₁₀  = 1-b


The Transition Probability Matrix is

Such that, |1 - a - b| < 1 

Then the n-step TPM is given by the formula:

For n is very large i.e (n tends to infinity)

---

Solved Problems of n-step TPM

Solved Problem 1:

In a hypothetical market, there are only two brands A and B. One Customer buys brand A with probability 0.7 if his last purchase was A, and buys B with probability 0.4 if his last purchase was B. Assuming Markov Chain model, Obtain:
i. One step Transition Probability Matrix
ii. n-step Transition Probability Matrix
Hence highlight the proportion of customers who would buy brands A and B in the long run.


= Solution:
Let 0 and 1 represent the event of buying Brand A and Brand B respectively.

We have, Transition probability Matrix is:

---

Related Posts:

• Stochastic Process- Definition, Specifications, and Classification
• Markov Chain: Definition and Representation with Solved Problems
• Stationary Distribution | Steady State Distribution with Solved Problems
• Classification Of States and Periodicity of Markov Chain with Solved Problems
• Birth Death Process
• Chapman-Kolmogorov Equation