Systematic Literature Review in HTA: Steps, Methods and Standards

A systematic literature review (SLR) answers a focused question by finding, appraising and synthesising every study that meets criteria fixed in advance, using methods explicit enough for another team to repeat. In health technology assessment (HTA), it is the standard route from scattered publications to the comparative evidence an agency appraises.

Read together, the Cochrane Handbook and the University of York’s Centre for Reviews and Dissemination (CRD) guidance describe five features: a question and eligibility criteria set out in a protocol before work starts; a search that tries to find all relevant studies; transparent, reproducible selection; an assessment of bias in the included studies; and an impartial synthesis that states how certain the findings are.

HTA needs this discipline because a funding decision compares a technology with the care it would replace, often for several patient groups. Citing only favourable trials would bias that comparison, which is why NICE’s manual says evidence on outcomes “should come from a systematic review”. The question itself is usually framed with PICO (population, intervention, comparator, outcomes); how HTA bodies set those elements is covered in our guide to the PICO framework in HTA.

Systematic review vs literature review: what is the difference?

A literature review summarises studies an author chooses; a systematic review follows a protocol that fixes how studies are found, selected and judged, so its conclusions can be checked. The Cochrane Handbook names reduced bias as the main advantage over ad hoc literature searching.

The boundary is also drawn formally. PROSPERO, the international register of systematic review protocols run by CRD and funded by the UK’s NIHR, does not accept scoping reviews or literature reviews that use a systematic search. A systematic search on its own does not make a review systematic; the protocol, selection, appraisal and synthesis have to follow.

How to do a systematic literature review: 9 steps

A systematic literature review follows a standard sequence: plan, search, select, extract, appraise, synthesise, grade and report. The steps below follow the Cochrane Handbook and the CRD guidance.

1. Define the question. Turn the decision problem into one or more PICO questions and eligibility criteria, including the study designs to be included.
2. Write and register a protocol. Fix the question, criteria and methods before screening. PROSPERO requires submissions before data extraction starts, and PRISMA-P 2015 covers protocol reporting.
3. Design the search. Combine a few concepts, typically population, intervention and study design, each with many synonyms, using free text and controlled vocabulary such as MeSH and Emtree.
4. Run and document the search. Search bibliographic databases and trial registers, record the date and full strategy for each source, and check reference lists and regulatory documents.
5. Screen in two stages. Titles and abstracts first, then full texts, recording a reason for every study excluded at full text.
6. Extract the data. Capture design, population, interventions, outcomes and results on a standard form, linking multiple reports of the same study.
7. Assess risk of bias. Apply a validated tool matched to study design to each result that will be used.
8. Synthesise. Pool results in a meta-analysis where studies are similar enough; otherwise use structured methods without meta-analysis.
9. Grade and report. Rate certainty for each outcome with GRADE and report the review against PRISMA 2020.

A systematic literature review example (hypothetical)

Suppose a manufacturer needs every randomised trial of a new medicine and its comparators in a chronic disease. The numbers below are invented and rounded to show how a PRISMA flow diagram adds up:

• Identified: 2,400 database records plus 300 register records gives 2,700.
• Duplicates removed: 600, leaving 2,100 records to screen on title and abstract.
• Excluded at title and abstract: 1,950, leaving 150 full-text reports.
• Excluded at full text, with reasons: 130, for example wrong population, wrong comparator or not randomised.
• Included: 20 reports describing 12 studies, which then go to extraction and risk of bias assessment.

Systematic literature review methodology: the standards at each step

A small set of reference standards defines good practice: the Cochrane Handbook for conduct, RoB 2 and ROBINS-I for risk of bias, GRADE for certainty and PRISMA 2020 for reporting. HTA guidance draws on the same tools.

Searching

The Cochrane Handbook names CENTRAL, MEDLINE and Embase as the sources for all Cochrane reviews (Embase where the team has access) and states that a search of MEDLINE alone is not considered adequate. ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform should both be searched, because searching the WHO platform alone is not sufficient. Searches should not be restricted by language or publication status, which risks publication and language bias, and should aim to maximise sensitivity while keeping precision reasonable. The Handbook recommends peer review of each strategy by an information specialist before it is run, and reporting in line with PRISMA and its search extension, PRISMA-S.

Risk of bias: RoB 2 and ROBINS-I

RoB 2 is Cochrane’s tool for randomised trials. It judges a specific result, not a whole trial, across five domains: the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome and selection of the reported result. Each domain is rated low risk, some concerns or high risk, and the overall judgement is the least favourable across domains.

Non-randomised studies of interventions are assessed with ROBINS-I, which NICE’s manual names as an example of a validated tool; its developers posted a revised draft of version 2 in November 2025 and note that it is still subject to change. Study designs, bias and validity are taught in Module 2: The Evidence, and the role of non-randomised data is covered in our guide to real-world evidence in HTA.

Synthesis

Where studies are similar enough in population, intervention, comparator and outcome, a meta-analysis pools their results into one estimate. Where it is not possible, the Handbook accepts methods such as summarising effect estimates, combining P values or vote counting by direction of effect, but calls vote counting based on statistical significance unacceptable. When treatments were never compared head to head, a network meta-analysis combines direct and indirect evidence through shared comparators; see our guide to network meta-analysis and indirect comparisons.

Certainty: GRADE

GRADE rates the certainty of a body of evidence for each outcome as high, moderate, low or very low. Randomised trials start at high and non-randomised studies at low; when ROBINS-I is used, non-randomised studies can start at high and are then usually rated down for risk of bias. Five domains can lower certainty: risk of bias, inconsistency, indirectness, imprecision and publication bias. A large effect, a dose-response gradient or plausible confounding that would work against the observed effect can raise it. Cochrane presents the results in a summary of findings table with up to seven critical or important outcomes.

Reporting: PRISMA 2020

The PRISMA 2020 statement, published in the BMJ on 29 March 2021, has 27 items in seven sections and replaces PRISMA 2009. It comes with an abstract checklist and flow diagram templates for original and updated reviews. PRISMA 2020 is a reporting guideline: its authors state that it does not guide how a review is conducted and should not be used to judge a review’s methodological quality.

What HTA bodies expect from a systematic literature review

HTA bodies expect submitted clinical evidence to come from a documented, systematic search, and some name the minimum sources and how recent the searches must be. As of October 2026, the EU joint clinical assessment and Germany’s G-BA both require searches run no more than three months before dossier submission.

In the EU, Implementing Regulation (EU) 2024/1381 of 23 May 2024 sets the dossier template for joint clinical assessments of medicines. Searches for comparator studies are required where relevant, for example for indirect comparisons. The assessment report template then asks assessors to judge the appropriateness of the developer’s sources and search strategies and to list any included studies they excluded. How the procedure runs is explained in our guide to the EU joint clinical assessment.

Where the review fits in England’s process is explained in how a NICE technology appraisal works, and the German dossier in our guide to the G-BA early benefit assessment. A practical approach for companies is one global SLR designed for the widest set of PICOs and the strictest source list, re-run before each deadline.

Clinical, economic and HRQoL SLRs in an HTA dossier

A full HTA submission can rest on several SLRs, each answering a different question. NICE’s manual sets a separate expectation for each type of evidence.

Economic SLRs inform model structure and the choice of analysis, described in our overview of types of economic evaluation. Utility reviews usually centre on EQ-5D; see EQ-5D utility values for how those numbers are produced.

Targeted, rapid and living reviews

Not every evidence question needs a full SLR. Targeted and rapid reviews trade completeness for speed; living reviews do the opposite and keep a full SLR continuously up to date.

Updating matters because reviews age. The Cochrane Handbook notes that there is no consensus on when to update, cites research finding a median of 5.5 years before reviews needed updating, and warns that some were out of date within a year or two. Living reviews avoid this but need considerable ongoing resources. For HTA submissions, the three-month search windows of the JCA and the G-BA in effect set the update schedule.

Common pitfalls in systematic literature reviews for HTA

The standards above warn explicitly against several shortcuts. Each of the following weakens the evidence or invites questions from assessors.

• Searching MEDLINE only. Cochrane does not consider it adequate, and the JCA and G-BA templates name CENTRAL as well.
• Language or publication-status limits. Cochrane advises against them because they risk language and publication bias.
• Single-reviewer screening without justification. The G-BA template asks for a reason if selection was not done by two people independently.
• Changing methods after seeing results. CRD advises against changes driven by knowledge of individual study results, and PRISMA 2020 asks authors to describe amendments to the protocol or registration.
• Vote counting by statistical significance. The Cochrane Handbook calls it unacceptable.
• A scope built around the trial, not the decision. Comparators in the HTA scope may differ from the trial’s control arm.
• Stale searches. A search older than three months must be updated before a JCA or G-BA dossier is submitted.