Systematic review

Preparation

Before you start

Before you begin - answer the questions:

1. Is my question appropriate for a Systematic Review?

2. What type of systematic review is the most appropriate for my question?

3. Is there an existing systematic review on my topic?

4. Do I have enough time to complete a systematic review?

5. Do I have access to necessary resources? (The University of Sydney)

Steps in conducting a Systematic review:

1. Formulate a review question

2. Define inclusion and exclusion criteria

Consider PICO and other criteria:

Participants
Intervention, exposures, tests, or other factors of interest
Comparators
Outcomes

3. Prepare protocol

cover points 1-2 and methods for 4-8 in as much detail as possible
Prespecify potential sources of heterogeneity to be explored

4. Locate studies

Develop search strategy considering the following sources:

Electronic databases
Checking of reference list
Handsearching of key journals
Personal communication with experts in the field

5. Select studies

Have eligibility assessed by >1 observer
Develop strategy to resolve disagreements
Keep log of excluded studies, with reasons for exclusion

6. Assess risk of bias or studies quality

Consider assessment by >1 observer
Use domain-based assessment or simple check numeric scales

7. Collect data

Design and pilot data collection form
Consider data extraction from reports by >1 observe
Consider possibility of collating individual participant data

8. Analyze and present results

Tabulate characteristics and results of individual studies
Examine forest plot
Explore possible sources of heterogeneity
Consider meta-analysis of all studies or subsets of studies
Perform sensitivity analyses, examine funnel plots
Make list of excluded studies available to interested readers

9. Interpret results

Consider limitations, including publication and related biases
Consider strength of evidence, including amount of evidence and quality of studies
Consider consistency of evidence across studies
Consider applicability
Consider meaningful presentation of findings

Egger, M., Smith, G. D. and Altman, D. Eds. 2008. Systematic reviews in health care: meta-analysis in context. John Wiley & Sons.

Recommended readings:

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I. et al. 2021. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 372-n71 doi:10.1136/bmj.n71

Define research question

A focused research question is imperative for the success of your systematic review. This section provides tips on how to make your research question suitable for a systematic review.

There are several different frameworks you can use to help structure your research and ensure you have clear parameters for your search. The most commonly used one used for health-related reviews is the PICO framework:

Population
This could be the general population, or a specific group defined by: age (e.g. infants, children, adolescents, elderly); socioeconomic status (e.g. low-income, homeless); risk status; location (rural or urban) Intervention
Refers to the therapy, test, strategy to be investigated (e.g. drug, behavioral change, environmental factors, counselling) Comparator
A measure you will use to compare results against (e.g. no treatment, alternative treatment/exposure, standard/routine interventions) Outcome
What outcome is significant to your population or issue? This may be different from the outcome measures used in the studies.

PICO example

Review title	The effect of blueberries on cognition and mood: a systematic review of human intervention trials
Population	Individuals of all ages, without regard to gender, race or ethnicity.
Intervention	Supplementation with blueberries, relevant blueberry products or extracts from blueberries. This may include freeze-dried blueberries, blueberry concentrate, or blueberry juice.
Comparator	Placebo or control groups.
Outcome	Changes in cognitive function based on cognitive screening measures (such as Mini mental state examination, Montreal Cognitive Assessment), neuropsychological interview, informant/carer responses to assessment tools or changes in mood. Secondary outcomes include: changes in biochemical levels in biological fluids. Of particular interest are inflammatory markers, cardiovascular disease risk factors, and markers of gastrointestinal health.

This example is extracted from: PROSPERO 2018 CRD42018100888.

Alternative framework:

PECO – Population | Environment | Comparison | Outcome
Very similar to PICO but looking at the effect of exposure to something e.g. smoky atmosphere

SPICE - Setting | Population | Intervention | Comparison | Evaluation
Another variant of PICO but this time including the setting (where? in what context?)

CIMO - Context | Intervention | Mechanisms | Outcome
A variant of PICO suitable for management and organisation studies

SPIDER – Sample | Phenomenon of Interest | Design | Evaluation | Research Type
Developed to create effective search strategies of qualitative and mixed-methods research - more specific than PICO/PECO

Resource: University of Reading

ACU Library

Watch the 4 min. video on how to frame a research question with PICO.

Additional reading:

Chatzopoulos, G. S., Koidou, V. P. and Wolff, L. F. 2022. Expression of Wnt signaling agonists and antagonists in periodontitis and healthy subjects, before and after non‐surgical periodontal treatment: A systematic review. Journal of Periodontal Research. https://doi.org/10.1111/jre.13029

Luijendijk, H. J. 2021. How to create PICO questions about diagnostic tests. BMJ Evidence-Based Medicine. 26(4), 155-157. DOI: 10.1136/bmjebm-2021-111676

Methley, A. M., Campbell, S., Chew-Graham, C., McNally, R. and Cheraghi-Sohi, S. 2014. PICO, PICOS and SPIDER: A comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Services Research. 14(1), 579.

Develop and register protocol

Writing a protocol is an important step in minimizing potential bias in the systematic review process. This section provides tips on how to prepare and register a protocol, and define eligibility criteria.

Why do I need to write a protocol?

A protocol specifies the question a systematic review will attempt to answer and the methods that will be used to answer it. Having a protocol is essential because:

It ensures that the entire team is aware of and in agreement with the question, the methods, the timelines, and the search strategy.
It enables communication with external stakeholders when seeking their contribution.
It keeps you focused and on topic.

Preparing a protocol

There are many reporting standards available which outline the essential information you need to provide in a protocol. Requirements will change depending on where you publish the protocol.

PRISMA for systematic review protocols (PRISMA-P)

Campbell Collaboration publish systematic reviews in social science, including areas such as crime and justice, education, nutrition, & social welfare. They provide a template for protocols which explains each section in detail.

Your search strategy needs to be included as part of your protocol

Refer to Plan search strategy and select databases section for more details.

Why do I need to register a protocol?

You must have a protocol even if you are not planning to register it. Registering a protocol is highly recommended for the following reasons:

To let other researchers know that you are working on a particular topic and help avoid duplication of effort
To avoid publication bias whereby only the research that rendered positive results is published
To help readers assess the quality of the review in terms of bias, through comparison of the protocol and the published systematic review

Registering a protocol

You have a range of platforms to choose from when deciding where to register a protocol. Many of the platforms will have specific conditions and limitations:

PROSPERO accepts registrations for systematic reviews, rapid reviews and umbrella reviews. PROSPERO does not accept scoping reviews or literature scans. Sibling PROSPERO sites registers systematic reviews of human studies and systematic reviews of animal studies.

Cochrane Reviews - Cochrane is a global network of researchers who produce Cochrane Reviews - these reviews are considered the ‘gold standard’ in systematic reviews in the field of health and medicine.

Joanna Briggs Institute
JBI is a not-for-profit research and development centre within the Faculty of Health and Medical Sciences at the University of Adelaide, with more than 70 Collaborating Entities across the world.

Campbell Collaboration
An international research network that produces systematic reviews of the effects of social interventions, Campbell registers protocols for reviews in multiple areas of study but only if you plan on publishing your systematic review through the Campbell Collaboration. Any interested researcher can propose a review title, form an author team, and submit a proposal. There is a rigorous peer assessment process for the title and, if title is accepted, for the protocol.

Information republished from - Library (The University of Sydney)

Plan search strategy and select databases

New accordion content

Conducting review

Perform searches: Accordion content 1.
Screen Results: Accordion content 2.
Data extraction and appraisal: New accordion content
Analyse and interpret: New accordion content

Write and publish

Publish

Finding the best journal to publish your work

Identifying the best journal to submit your research to can be a difficult process. Authors, be aware of predatory journals!

To help you make the choice of where to submit, Think. Check. Submit. has a produced a video that includes checklist for authors to use to find the right journal.