Markov chain and its use in solving real world problems

Markov chain is one of the most important tests in order to deal with independent trials processes. There are two major principal theorems for these processes. The first one is the ‘Law of Large Numbers’ and the second one is the ‘Central Limit Theorem’. When probability experiments create independent trial processes, the outcomes for all iterations are same. Also, they increase or decrease with the same probability. Markov chain is the process where the outcome of a given experiment can affect the outcome of the future experiments. It consists of finite number of states and some known probabilities, where probability of changing from state j to state i. For instance, there are two sectors; government and private. The probability here is the likelihood of moving from state j to state i in a given time for an individual.

Below are some example situations showing the application of Markov chain.

Markov chain application example 1

RA Howard explained Markov chain with the example of a frog in a pond jumping from lily pad to lily pad with the relative transition probabilities. Lily pads in the pond represent the finite states in the Markov chain and the probability is the odds of frog changing the lily pads.

Markov chain application example 2

Manager in a company state their intention to get a promotion by the end of next month to her employee P. And P forwards the same message to Q, who further gives the news to R and so on. In such scenario there is a probability x that a person will replace the answer from yes to no while conveying the message to the next person and a probability y that the person will change the answer from no to yes.

Markov chain application example 3

Minister X has 55% votes and Minister Y has the rest 45% votes in one state. However, probability of current followers voting Minister X staying with Minister X in future is 70% and switching to Minister Y is 30%. And probability of the present followers voting Minister Y staying with Minister Y is 90% and switching to Minister X is 10%. Even though Minister X has more votes than Y in the current situation, but in the next election Minister Y will have more votes with 57% and X will be left with only 43% votes.

Simple business case for application of Markov chain

The case supposes that there are 2 types of weathers in a particular area, ‘sunny’ and ‘cloudy’. A news channel wants to broadcast their prediction about the next week’s weather.

The news channel hires a weather forecast company to find out the next week’s weather and also for the following weeks. Currently there is ‘sunny’ weather in that area.

  • Probability of the weather staying ‘sunny’ the following week is 80%.
  • Probability of the weather changing from ‘sunny’ to ‘cloudy’ over a week is 20%
  • Probability of the weather staying ‘cloudy’ the following week is 70%
  • Probability of the weather changing from ‘cloudy’ to ‘sunny’ over a week is 30%

Although the weather is predicted to be ‘sunny’ the whole week, one cannot be sure about the next week without making some transition calculations. The below matrix is to explain the transition:

Transition matrix

Transition matrix

Transition diagram

Transition diagram

Transition diagram

The diagram simply shows the transitions and the weather conditions same as the above matrix. The calculations can be done by the following matrix multiplication:

Current State * Transition Matrix = Final State

S=Sunny; C= Cloudy

It can be seen that there is 80% chance that the next whole week will also be ‘sunny’. But, there is 20% chance that the weather might change to ‘cloudy’ next week. This calculation is called Markov chain. If the transition matrix doesn’t change with time, one can also predict weather for the further weeks using the same equation. Calculations for the weather forecast in the two weeks’ time:

Mathematical steady state calculations

Therefore, to predict the weather for further days/weeks/months in the long run, steady state vector helps in calculating the likelihood of the weather staying ‘sunny’ or changing to ‘cloudy’ without depending on the given initial condition of weather.

Mathematical steady state calculations

Mathematical steady state calculations

Equation 1: 0.2x1 + 0.7x2 = 0 = 0.2x1 - 0.7x2
Equation 2: x1 + x2 = 1 (Probability vector)

Therefore, from Equation 1 and 2, about 78% of the weeks in the future will be ‘sunny’, while the rest will be ‘cloudy’.

Probable areas of application of Markov chain

Markov chain has many applications in the field of real world process are followings:-

  • The one of the most popular use of Markov chain is in determining page rank by Google.
  • Markov chain-based methods also used to efficiently compute integrals of high-dimensional functions.
  • This method plays an important role to allow sample from any arbitrary probability distribution.
  • It helps determine the probability of consumer switching from one brand to another.
  • Markov chains are popular in finance and economics to model different phenomena, including market crashes and asset prices.

Software supporting Markov chain

A number of software can be used for Markov chain analysis, such as Ram Commander, SoHaR Reliability and safety, Markov Analysis software and MARCA (Markov Chain Analyzer).

Prateek Sharma

Prateek Sharma

Analyst at Project Guru
Prateek has completed his graduation in commerce with a rich experience in Telecom, Marketing and Banking domains for preparing comprehensive documents and reports while managing internal and external data analysis. He is an adaptable business-minded Data Analyst at Project Guru skilled in recording, interpreting and analysing data with a demonstrated ability to deliver valuable insights via data analytics and advanced data-driven methods. Apart from his strong passion towards data science, he finds extreme sports interesting. He keeps himself updated with the latest tech and always love to learn more about latest gadgets and technology.
Prateek Sharma

Related articles

  • How to perform LASSO regression test? In statistics, to increase the prediction accuracy and interpret-ability of the model, LASSO (Least Absolute Shrinkage and Selection Operator) is extremely popular. It is a regression procedure that involves selection and regularisation and was developed in 1989. Lasso regression is an […]
  • Solving complicated problems with decision tree A decision tree is a graphical representation of possible solutions to a problem based on given conditions. It is called a tree because diagrammatically it starts with a single box (target variable) and ends up in numerous branches and roots (numerous solutions).
  • How to conduct generalized least squares test? In statistics, Generalized Least Squares (GLS) is one of the most popular methods for estimating unknown coefficients of a linear regression model when the independent variable is correlating with the residuals.
  • Getting acquainted with neural network analysis Neural network, popularly known as Artificial Neural Network (ANN) is an information processing system with a large number of nodes and connections as part of a structure which helps in processing complex information.
  • How to perform nonlinear regression? Regression analysis is a statistical tool to study the relationship between variables. These variables are the outcome variable and one or more exposure variables. In other words, regression analysis is an equation which predicts a response from the value of a certain predictor.

Discuss

We are looking for candidates who have completed their master's degree or Ph.D. Click here to know more about our vacancies.