Absorbing Markov chain

When considering only transient states, the probability found in the upper left of Pk, the (i,j)-entry of Qk.

This can be established to be given by the (i, j) entry of so-called fundamental matrix N, obtained by summing Qk for all k (from 0 to ∞).

The computation of this formula is the matrix equivalent of the geometric series of scalars,

With the matrix N in hand, also other properties of the Markov chain are easy to obtain.

Although in reality, the coin flips cease after the string "HTH" is generated, the perspective of the absorbing Markov chain is that the process has transitioned into the absorbing state representing the string "HTH" and, therefore, cannot leave.

For this absorbing Markov chain, the fundamental matrix is The expected number of steps starting from each of the transient states is

Therefore, the expected number of coin flips before observing the sequence (heads, tails, heads) is 10, the entry for the state representing the empty string.Games based entirely on chance can be modeled by an absorbing Markov chain.

To determine the expected number of turns to complete the game, compute the vector t as described above and examine tstart, which is approximately 39.2.

Infectious disease testing, either of blood products or in medical clinics, is often taught as an example of an absorbing Markov chain.

[4] The public U.S. Centers for Disease Control and Prevention (CDC) model for HIV and for hepatitis B, for example,[5] illustrates the property that absorbing Markov chains can lead to the detection of disease, versus the loss of detection through other means.

at which the health system administers tests of the blood product or patients in question.

The subsequent total absolute number of false negative tests—the primary CDC concern—would then be the rate of tests, multiplied by the probability of reaching the infected but undetectable state, times the duration of staying in the infected undetectable state: