How to Critically Appraise a Systematic Review: Part 1

Summary

This video series teaches how to critically appraise a systematic review. In Part 1, the focus is on determining the validity of the review by examining two key aspects: the exhaustiveness of the search for relevant studies and the unbiased nature of the inclusion criteria. The video emphasizes the importance of a comprehensive search for both published and unpublished studies, including non-English sources, and the need for transparent, unbiased selection of studies. The example used is a systematic review on thrombosis prophylaxis in hospitalized patients, demonstrating the practical application of these principles.

Takeaways

• 😀 A systematic review aims to synthesize the totality of evidence on a topic, including both published and unpublished studies.
• 😀 Narrative reviews cannot be critically appraised because they lack a method section and are often biased toward the author's point of view.
• 😀 Systematic reviews are only as good as the primary studies they are based on. Flaws in primary studies cannot be corrected by the review itself.
• 😀 Critical appraisal of a systematic review involves evaluating if the results are valid, what the results are, and whether they will help in patient care.
• 😀 A valid search for studies in a systematic review should be detailed, exhaustive, and include both published and unpublished sources.
• 😀 To minimize publication bias, systematic reviews should include data from sources like trial registries, expert contacts, and grey literature (e.g., thesis, abstracts).
• 😀 A key to minimizing bias in study selection is using unbiased, clinically relevant inclusion criteria and having multiple reviewers independently assess each study.
• 😀 The search should ideally not be limited to English-language sources to avoid language bias, though this can be time-consuming and challenging.
• 😀 Authors should provide clear, justified reasons for rejecting studies, and multiple reviewers should independently make decisions to reduce bias.
• 😀 The inclusion criteria for studies should be clinically sensible and methodological criteria, like randomized controlled trials, must be appropriate for the review's focus.
• 😀 Flowcharts showing the selection process (i.e., how studies were included or excluded) enhance transparency and help assess if the selection was unbiased.

Q & A

What is the primary goal of critically appraising a systematic review?

The primary goal is to determine whether the results of the systematic review are valid, to understand what the results are, and to assess whether the results will help in patient care.

Why is it difficult to critically appraise narrative reviews?

It is difficult because narrative reviews do not include a method section, making it hard to assess their methodology and bias. They are often seen as biased collections of articles supporting the author's point of view.

What is the key limitation of a systematic review if the primary studies it includes are flawed?

The key limitation is that the review cannot correct the flaws in the primary studies. The quality of a systematic review is ultimately determined by the quality of the studies it includes.

What does an exhaustive search in a systematic review include?

An exhaustive search includes multiple steps: electronic database searches (e.g., Medline, Embase), manual searches of reference lists, contacting experts for unpublished studies, and checking trial registries and non-English language resources.

What is publication bias, and how does it affect systematic reviews?

Publication bias occurs when studies with positive or favorable results are more likely to be published, while those with negative or inconclusive results are often not published. This bias can skew the findings of a systematic review if the authors do not include unpublished or non-English studies.

How did the example review on venous thromboembolism prophylaxis handle publication bias?

The example review did a fairly good job by conducting a multifaceted search, including trial registries and reference lists. However, there was some evidence of publication bias, indicating that some studies may have been missed.

What are the common inclusion criteria used in systematic reviews, and why are they important?

Common inclusion criteria include patient population, intervention, and outcomes that align with the review's research question. They also may include methodological criteria such as randomization for clinical trials. These criteria ensure that the review includes studies that are directly relevant and of sufficient quality.

Why should more than one person independently assess studies for inclusion in a systematic review?

Independent assessment by multiple individuals helps reduce bias in the selection of studies. This ensures that the inclusion of studies is based on the data rather than the authors' personal biases.

What is the role of blinding in the inclusion/exclusion process of a systematic review?

Blinding ensures that the individuals assessing the studies for inclusion or exclusion do not influence each other or introduce bias based on personal opinions. This helps maintain objectivity in the selection process.

Why is it important to provide reasons for excluding studies in a systematic review?

Providing reasons for excluding studies ensures transparency and allows others to assess whether the exclusions were based on objective, well-founded criteria, rather than bias or personal preference.