Selecting the Markov Chain item under Stochastic Analysis, provides the opportunity to construct a Markov Chain Model. We describe below the various analyses possible for the example whose state-transition diagram is shown above. The analysis types are listed at the left, and the description of an analysis can be reached directly by clicking on the item.

Example: Light Bulb Replacement

Historical data and probability analysis determine the probabilities of replacement based on the age of a bulb. This information is shown by the matrix below.

Each row of the matrix describes what might occur for a bulb of a particular age. For example, the row labeled New indicates that if a new bulb is inserted in a socket during one of the monthly maintenance inspections, the probability that it will fail during the month and be replaced with a new bulb at the next inspection is 0.5. The probability that it will not fail and survive to an age of one month is also 0.5. This poor quality seems to be rather extreme, but we exaggerate for illustration. The remaining rows indicate similar data for other ages. In every case the bulb is replaced with a new one or ages by one month. For the example, we assume the bulb is always replaced after it is four months old.

The names New, 1-mo, 2-mo., etc. are the states of the system. At a monthly inspection the bulb must be in one of the states. The matrix is called the transition matrix because it shows the probability of transition from each state to every other state. We call this a Markov process when the transition probabilities depend only on the state of the system. The figure appearing at the beginning of this article is called the State Transition Diagram and represents the same information as the transition matrix.