Markov Models in Health Economics: A Practical Guide

A Markov model in health economics follows a hypothetical group of patients through a small set of health states, one time step at a time, and adds up the costs and quality-adjusted life years (QALYs) that each treatment option produces. It is one of the most common ways to turn short-term trial results into the long-term estimates that payers need.

Trials usually run for months or a few years, while many treatments change costs and outcomes for life, so bodies that carry out health technology assessment (HTA) rely on decision models to fill the gap. The ISPOR-SMDM modelling task force counts state-transition models, the family Markov models belong to, among the most widely used techniques in HTA.

Beck and Pauker described Markov models for medical prognosis in 1983. Sonnenberg and Beck’s 1993 practical guide explained why they suit clinical problems: risk continues over time, timing matters and some events recur. A decision tree that tried to capture all of that would quickly become unmanageable.

How Markov models work in health economics

A Markov model has three building blocks: health states, a matrix of transition probabilities and a cycle length. Once per cycle, the model applies the probabilities to the cohort and records how many people are in each state.

Building a Markov cohort model: the main steps

1. Define the decision problem (PICO) and the perspective of the analysis.
2. Choose health states that reflect the disease pathway and the main drivers of cost and quality of life.
3. Set the cycle length and a long enough time horizon.
4. Estimate transition probabilities and treatment effects.
5. Attach a cost and a utility to each state.
6. Run the cohort, apply a within-cycle correction and discounting, and sum costs and QALYs.
7. Calculate the incremental cost-effectiveness ratio, test uncertainty and validate the model.

Markov model cost-effectiveness: a worked example

The quickest way to understand a Markov model is to run one by hand. Take a hypothetical chronic disease, modelled with one-year cycles and round, illustrative numbers. Everyone starts Well; each year some people become Sick, anyone can die and nobody recovers. Under standard care, the transition matrix is:

Each row sums to 1 because everyone must end the cycle somewhere. A zero means the move is not allowed.

The cohort trace

Start 1,000 people in Well and apply the matrix once per cycle. In cycle 2, for example, Sick receives 850 × 0.10 = 85 people from Well and keeps 100 × 0.80 = 80 of its own, giving 165.

Fractions are normal: a cohort model tracks expected numbers, not individuals. Checking that every row still totals 1,000 is a simple verification test.

Adding up costs and QALYs

Now attach hypothetical annual values: Well costs €1,000 (utility 0.85), Sick costs €8,000 (utility 0.50) and Dead costs nothing (utility 0). Each cycle’s cost is people × state cost; its QALYs are people × utility × cycle length, as explained in our guide to how QALYs are calculated from utility values.

Here people are counted at the end of each cycle, without discounting; both choices matter, as shown below.

From two arms to the ICER

Suppose a hypothetical new treatment costs an extra €2,000 a year for everyone in Well and halves the yearly risk of becoming Sick, from 0.10 to 0.05, so Well to Well rises to 0.90. The same steps put 900, 810 and 729 people in Well and 50, 85 and 108.5 in Sick over the three cycles.

Whether that is good value depends on the decision-maker’s threshold, as explained in our guide to the incremental cost-effectiveness ratio (ICER). But three cycles is far too short for a chronic disease, as the sections below show.

Transition probabilities: converting rates and applying hazard ratios

Transition probabilities must match the model’s cycle length, and published evidence rarely arrives in that form. The standard fix is to convert between probabilities and rates, assuming a constant rate.

A rate measures how fast events occur per unit of person-time; a probability is the chance that an event happens over a given period. With a constant rate r, the probability of an event over time t is:

The reverse is r = −ln(1 − p) ÷ t. Sonnenberg and Beck warned against dividing an annual probability by 12 to get a monthly one; derive the rate first. For a hypothetical 30% five-year probability of progression:

Hazard ratios apply to rates, not probabilities

As the 2023 Medical Decision Making tutorial by Alarid-Escudero and colleagues sets out, a hazard ratio acts on rates: convert the baseline probability to a rate, multiply by the hazard ratio, then convert back. A hypothetical annual probability of 0.20 with a hazard ratio of 0.5 becomes about 0.106, not 0.10.

Two further cautions. The simple conversion is not exact when people can leave a state for more than one destination in a cycle; Gidwani and Russell (2020) describe the options. And when no head-to-head trial exists, treatment effects often come from a network meta-analysis, whose uncertainty has to be carried into the model.

The Markov assumption and tunnel states

The Markov assumption says that the probability of moving to the next state depends only on the current state, not on how or when a patient got there. The model has no memory.

Someone who became Sick last year therefore faces the same risks as someone Sick for ten years. Sonnenberg and Beck acknowledged that this is not strictly true in medicine, but it keeps the number of states finite. Modellers relax it in three ways:

• Tunnel states: temporary states passed through one cycle at a time (Sick year 1, Sick year 2, Sick year 3+), each with its own probabilities, costs or utilities.
• Time-dependent probabilities: values that change with model time or age, such as background mortality.
• More states or an individual-level model: ISPOR-SMDM suggests adding states to capture relevant history, and individual-level simulation when the states become too numerous.

Half-cycle correction and other within-cycle corrections

Half-cycle correction adjusts for the fact that a Markov model moves people only at cycle boundaries, while real events happen throughout the cycle. It assumes that, on average, transitions happen halfway through each cycle.

Counting the cohort at the end of each cycle underestimates time alive; counting at the start overestimates it. ISPOR-SMDM recommends applying a half-cycle correction to costs and effects in the first cycle, and in the final cycle when the horizon is shorter than a lifetime. NICE’s company evidence submission user guide asks companies to state the cycle length and whether a half-cycle correction was applied.

Applied to both arms of the three-cycle example, the counting convention changes the answer considerably:

The very short horizon exaggerates the gap, but the lesson holds: errors from discrete cycles do not necessarily cancel out between arms.

Time horizon and discounting in a Markov cohort model

The time horizon should be long enough to capture every important difference in costs and outcomes, which for chronic conditions usually means a lifetime. Future costs and QALYs are then discounted cycle by cycle.

NICE says a lifetime horizon is usually appropriate when a technology affects costs and outcomes over patients’ lifetimes. Canada’s guidelines, published by CADTH in 2017 (now Canada’s Drug Agency, CDA-AMC), call a lifetime horizon most appropriate for chronic conditions and when options affect mortality differently.

NICE’s reference case discounts costs and health effects at 3.5% a year, with 1.5% only in narrow circumstances; the Canadian guidelines use 1.5%, with 0% and 3% in sensitivity analyses. At 3.5%, a QALY gained in year 10 is worth about 0.71 of a QALY today.

The horizon can matter even more than the rate. Run the hypothetical example for 60 cycles, by which time almost everyone has died, with a half-cycle correction and 3.5% discounting, and the ICER falls from about €52,200 (the same settings over three cycles) to about €7,500 per QALY, because people kept Well go on gaining QALYs after year 3. The reasoning behind discounting is covered in Module 5: Health Economics Fundamentals.

Decision tree vs Markov model vs partitioned survival model

Decision trees suit short, one-off decisions; Markov models suit diseases that unfold over time with recurring risks; partitioned survival models are common in advanced cancer, where state membership is read straight from survival curves. NICE expects the choice of model type to be justified for each new decision problem.

A partitioned survival model is not a Markov model

In a three-state partitioned survival model, the progression-free share comes from the progression-free survival (PFS) curve, the dead share is one minus overall survival (OS), and the progressed share is the gap between them. No transition probabilities are estimated.

NICE Decision Support Unit Technical Support Document 19 (2017) found the approach in 22 of 30 NICE cancer appraisals completed between May 2013 and February 2016, almost all in advanced or metastatic disease. Only 12 of the 22 company submissions named it correctly; the rest called it a Markov or semi-Markov model. Because PFS and OS are modelled independently, extrapolated survival ignores progression and PFS can even rise above OS. A 2020 follow-up by the same authors concluded that state-transition models should be used alongside partitioned survival models to explore uncertainty in the extrapolation period, such as survival after progression.

Microsimulation and other individual-level models

A microsimulation runs simulated patients one at a time using random draws, so each carries a personal history. ISPOR-SMDM recommends a cohort model when a manageable number of states will do, for its transparency, efficiency and ease of debugging, and an individual-level model when it will not.

What HTA assessors check in a Markov model

Assessors test whether the structure fits the disease, whether inputs are traceable and appropriate, whether the model is coded correctly and whether its conclusions survive uncertainty.

NICE says companies must justify the model type and structure for each new decision problem; noting that a structure has been used before is not enough. It also asks for the methods and results of validation and quality assurance. The ISPOR-SMDM validation report (Eddy and colleagues, 2012) describes five types:

• Face validity: experts judge the structure, data, assumptions and results.
• Verification: the model is coded correctly, for example rows sum to 1 and the cohort is conserved.
• Cross validity: results are compared with other models of the same problem.
• External validity: outputs are compared with real-world data.
• Predictive validity: predictions are compared with events observed later.

The report treats external and predictive validation as the strongest. The 13-item AdViSHE tool gives companies a structured way to report what validation they did.

Uncertainty in Markov model inputs

Every input is an estimate. In a probabilistic sensitivity analysis (PSA), each parameter gets a probability distribution and the model is re-run many times; NICE says the committee’s preferred estimate should come from a probabilistic analysis. See Module 9: Handling Uncertainty.

Common red flags in Markov model submissions

• Matrix rows that do not sum to 1, or a cohort that leaks.
• Probabilities rescaled by division, or hazard ratios applied to probabilities.
• Cycles too long for how often key events occur.
• A horizon that stops before differences between options play out.
• Treatment effects assumed to last indefinitely, with no scenario in which they stop or wane.
• Extrapolation chosen on statistical fit alone, without external data or clinical opinion.
• A within-cycle correction that is missing or inconsistent between arms.
• A partitioned survival model described as a Markov model.

For how these checks feed into funding decisions in England, see our guide to how NICE technology appraisals work.

Why Markov models still matter in HTA

More than four decades after Beck and Pauker’s paper, the Markov cohort model remains a standard tool because it is simple enough to check by hand and flexible enough to represent many chronic diseases. The arithmetic is rarely the hard part; the judgement lies in the structure, inputs and assumptions.