Probabilistic Sensitivity Analysis (PSA) in Health Economics

A cost-effectiveness model can hold dozens of inputs: the probability of disease progression, the cost of a hospital stay, the utility of living with a complication, a hazard ratio from a trial. None of them is known exactly. Probabilistic sensitivity analysis shows what that uncertainty means for the decision.

Instead of running the model once with a best estimate for each input, a PSA gives every uncertain input a probability distribution, samples from all of them at once, runs the model and repeats the process thousands of times.

PSA matters to anyone who builds, reviews or relies on the incremental cost-effectiveness ratio (ICER). NICE in England, Canada’s Drug Agency and the US Institute for Clinical and Economic Review all ask for it in some form.

Why parameter uncertainty matters in health economic models

Parameter uncertainty matters because a funding decision based on single best estimates can be wrong, and because in many models the average result of uncertain inputs differs from the result of average inputs.

The ISPOR-SMDM Modeling Good Research Practices Task Force, in its 2012 report on parameter estimation and uncertainty, separates four kinds of uncertainty. PSA addresses the second.

Why the probabilistic result can differ from the base case

Many health economic models are non-linear: probabilities compound over the cycles of a Markov model and survival curves are extrapolated. In a non-linear model, entering the mean of each input does not, in general, give the mean of the outputs.

This is why NICE’s manual says the committee’s preferred cost-effectiveness estimate should come from a probabilistic analysis when possible, unless the model is linear. Using deterministic results instead must be justified.

Deterministic sensitivity analysis vs probabilistic sensitivity analysis

Deterministic sensitivity analysis changes one input, or a few, to set alternative values and shows which ones drive the result. Probabilistic sensitivity analysis varies all uncertain inputs together and shows how likely the decision is to be wrong. HTA bodies expect both.

• One-way analysis: each parameter moves across a plausible range while the others stay at base-case values.
• Tornado diagram: one-way results as horizontal bars, the most influential parameter at the top.
• Scenario analysis: the model is rerun with alternative assumptions or data, such as another survival extrapolation.
• Threshold analysis: finds the parameter value, often the price, at which the decision would change.

The ISPOR-SMDM task force recommends reporting both in a single evaluation. It warns that arbitrary variations, such as plus or minus 50% on every input, can measure sensitivity but should not be used to represent uncertainty.

Choosing distributions for probabilistic sensitivity analysis

Match each distribution to the nature of the parameter and the evidence behind it: beta for probabilities and most utilities, gamma or lognormal for costs, lognormal for relative risks and hazard ratios, and Dirichlet for sets of probabilities that must sum to one.

NICE states that distributions should represent the available evidence rather than be chosen arbitrarily, and that formal elicitation methods can help when data are lacking.

ISPOR-SMDM also advises favouring continuous distributions that look realistic across the parameter’s range, questioning uniform or triangular shapes, and using wide distributions where evidence is thin. A parameter should never be excluded because it is uncertain; NICE says the same. Confidence intervals and hazard ratios are covered in Module 3: Biostatistics.

Correlated parameters

Sampling inputs independently is only right if they are independent. Parameters estimated together, such as regression coefficients, come with a covariance matrix and can be sampled jointly, typically from a multivariate normal distribution. The parameters of a fitted survival curve and relative effects from a network meta-analysis raise the same issue.

NICE asks analysts to consider evidence on correlation, to avoid unsupported assumptions of perfect correlation and to state their assumptions. Canada’s guidelines note that correlation can change both expected values and their uncertainty.

Monte Carlo simulation in PSA: how many iterations are enough?

Run enough simulations for the results to stop moving. NICE asks for enough runs to minimise Monte Carlo error. Convention is 1,000 to 10,000 runs, but the right number should come from a convergence check.

How a PSA runs, step by step

1. Define a distribution for every uncertain parameter, with any correlations.
2. Draw one value for each parameter.
3. Run the model and record costs and QALYs for every option.
4. Repeat steps 2 and 3 many times.
5. Average costs and QALYs across simulations. The probabilistic ICER is the difference in mean costs divided by the difference in mean QALYs, not the average of individual ICERs.
6. Check convergence, then summarise the results.

Checking convergence

A 2018 tutorial in PharmacoEconomics by Hatswell and colleagues notes that 1,000 to 10,000 simulations are usually run with no assessment of convergence. In their example, twenty separate PSAs of 1,000 simulations all gave mean ICERs within £500 of the “true” value, but only twelve came within £250. At 10,000 simulations, all twenty did.

They suggest running until the 95% confidence interval around mean incremental net monetary benefit excludes zero at the threshold of interest, or until the ICER’s interval falls within an agreed tolerance. NICE similarly suggests watching how the variance of results changes as runs increase.

Reading the cost-effectiveness scatter plot

Each PSA simulation becomes one point on the cost-effectiveness plane, with incremental QALYs on the horizontal axis and incremental costs on the vertical axis. The share of points below the threshold line is the probability that the new option is cost-effective at that threshold.

A cloud that spreads across several quadrants signals real decision uncertainty. The plane itself is explained in Module 7: Cost-Effectiveness and the ICER.

Counting simulations by their ICER is unreliable, because a negative ICER can mean a cheaper, more effective option or a costlier, less effective one. Analysts use incremental net monetary benefit (INMB) instead, which is positive whenever a point lies below the threshold line.

Take a hypothetical new treatment compared with standard care in 1,000 simulations, using round numbers. The mean incremental cost is £20,000 and the mean gain is 0.80 QALYs, so the probabilistic ICER is £20,000 ÷ 0.80 = £25,000 per QALY. At a threshold of £30,000, expected INMB is (£30,000 × 0.80) − £20,000 = £4,000 per patient.

If 720 of the 1,000 points lie below the £30,000 line, the probability that the treatment is cost-effective at that threshold is 720 ÷ 1,000 = 72%. The error probability is 28%.

CEAC and CEAF: the cost-effectiveness acceptability curve and frontier

A cost-effectiveness acceptability curve (CEAC) plots the probability that each option is cost-effective across a range of thresholds. The cost-effectiveness acceptability frontier (CEAF) shows, at each threshold, the probability for the option with the highest expected net benefit, which is the option to choose.

A CEAC repeats the scatter-plot count at every threshold. For the hypothetical treatment above:

Fenwick, Claxton and Sculpher argued in 2001 that the CEAC addresses the decision problem more directly than a confidence interval around the ICER. With more than two options, ISPOR-SMDM recommends plotting every curve on the same graph.

Why the CEAC alone can mislead

The decision rule is to choose the option with the highest expected net benefit; the probability of being cost-effective only measures uncertainty around that choice. Barton, Briggs and Fenwick showed in 2008 that the option with the highest probability is not always optimal, and concluded that the CEAC should not be used to determine the optimal decision.

The CEAF follows the option with the highest expected net benefit, so here it would show A, at 40%. NICE asks submissions to present and explain the frontier, and Canada’s guidelines require both CEACs and CEAFs.

Expected value of perfect information (EVPI): what better evidence is worth

EVPI is the expected cost of decision uncertainty: the expected net benefit of deciding with perfect information minus the expected net benefit of the best decision with current information. It sets an upper limit on the value of further research.

In the A versus B example, current information points to A. With perfect information, a decision-maker would switch to B in the 600 simulations where it is better, gaining £1,500 each time. EVPI per patient is 600 × £1,500 ÷ 1,000 = £900.

The ISPOR Value of Information Task Force recommends scaling EVPI to the population that could benefit over the life of the decision. If 2,000 patients a year were treated for five years, population EVPI would be 10,000 × £900 = £9 million, before discounting. Research costing more cannot be worthwhile; research costing less might be.

Two related measures narrow the question. The expected value of partial perfect information (EVPPI) values information on one parameter or group of parameters; the expected value of sample information (EVSI) values a specific study design. The 2012 ISPOR-SMDM report argued that EVPI, alongside acceptability curves, is the most appropriate way to present decision uncertainty from a PSA, because it combines the probability of a wrong decision with its consequences.

What NICE and other HTA agencies expect from PSA

NICE expects a probabilistic base case for non-linear models, evidence-based distributions, enough simulations and clear presentation of decision uncertainty. Canada and the US Institute for Clinical and Economic Review set similar expectations.

NICE (England)

Section 4.7 of the NICE health technology evaluations manual asks for:

• A preferred estimate from probabilistic analysis where possible, unless the model is linear.
• Distributions that reflect the evidence, with no parameter excluded simply because it is uncertain.
• Scenario analyses that are, in general, also probabilistic.
• Scatter plots and confidence ellipses when two alternatives are compared, acceptability curves and an explained frontier.
• The probability of cost-effectiveness at the maximum acceptable ICERs and the error probability, particularly with more than two alternatives.

Following a government decision, NICE raised the standard range for technology appraisals from £20,000–£30,000 to £25,000–£35,000 per QALY from April 2026. See our guide to NICE technology appraisals.

Other agencies and good-practice guidance

EU joint clinical assessments analyse clinical evidence on relative effects and feed into national HTA, so cost-effectiveness models and their PSA are still prepared nationally. See where this fits in the HTA process, step by step.

Common PSA red flags and key takeaways

Reviewers check whether a probabilistic sensitivity analysis is complete, evidence-based and stable, and whether its outputs are read correctly. Common problems include:

• A deterministic base case for a non-linear model without justification.
• Arbitrary ranges, or uniform and triangular distributions, without justification.
• Parameters fixed because they are uncertain.
• Parameters estimated together but sampled independently.
• A round number of simulations with no convergence check.
• A probabilistic ICER calculated as the average of individual ICERs.
• The option with the highest CEAC probability presented as the best choice.

Spotting these takes a working knowledge of how models are built and validated, the focus of Module 8: Economic Modelling.