Markov Processes in Management Science

Markov process models are used to study how systems behave over time periods. As an example, assume that we have a group of consumers. Each consumer purchases a product every week. There are only two brands of the product – A and B. We can set up a model to find out the share of the market that brand A has and the share that brand B has, in the long run. We would need to know the relevant probabilities – the probability that a consumer purchasing brand A stays loyal to the product from one week to the next, and the probability that a consumer purchasing brand B stays loyal to it. Whilst these probabilities easily describe what happens in the first week, the amounts purchased of brands A and B in subsequent weeks will change week by week. However, as the weeks roll by, the size of the change reduces. Ultimately we reach a steady state solution. As far as the mathematics is concerned, we can go straight to calculating this steady state solution.