Summary Measures Used in Systematic Reviews

Summary

This video script offers a comprehensive guide to interpreting outcome measures in systematic reviews and meta-analyses. It explains dichotomous and continuous outcomes, how to summarize them using odds ratios, relative risks, risk differences, mean differences, and standardized mean differences. The script also covers meta-analysis of diagnostic test studies, including sensitivity, specificity, likelihood ratios, and summary ROC curves. It provides examples from systematic reviews to illustrate these concepts, aiming to clarify the significance of different effect sizes.

Takeaways

• 📚 The script discusses the various types of outcome measures used in systematic reviews and meta-analyses.
• 🔍 Outcome measures for treatment studies are categorized into dichotomous (yes/no) and continuous (measured along a continuum like blood pressure).
• 📈 Dichotomous outcomes can be summarized using odds ratios, relative risks, or risk differences.
• 📊 For continuous outcomes measured the same way in each study, the mean difference is used to calculate the average change caused by an intervention.
• 📉 When continuous outcomes are measured differently, a standardized mean difference is calculated to allow for comparison across studies.
• 🔢 The standardized mean difference is expressed in units of standard deviation, which can be difficult to interpret directly.
• 📌 A standardized mean difference of 0.2 or less indicates a small effect, 0.5 a moderate effect, and 0.8 or greater a large effect.
• 🗂 The script provides examples from systematic reviews to illustrate the use of mean difference and standardized mean difference.
• 💊 It explains the use of risk ratios for summarizing studies with dichotomous outcomes, such as death in medical trials.
• 🦴 An example of calcium supplementation's small effect on bone mineral density is given, with a standardized mean difference of 0.14.
• 🩺 The script also covers outcome measures for diagnostic test studies, including sensitivity, specificity, likelihood ratios, and the summary ROC curve.

Q & A

What is the purpose of the video script?

The purpose of the video script is to explain different types of outcome measures commonly used in systematic reviews and meta-analyses.

What are the two types of treatment outcome studies mentioned in the script?

The two types of treatment outcome studies are dichotomous outcomes, which are yes/no outcomes, and continuous outcomes, which occur along a continuum like blood pressure.

How are dichotomous outcomes summarized in meta-analyses?

Dichotomous outcomes are summarized using odds ratios, relative risks, or risk differences.

What is a mean difference and when is it used?

A mean difference is used when the outcome measure is the same in each study and measured the same way. It measures the absolute difference between the mean value in the two groups in a clinical trial.

What is a standardized mean difference and why is it used?

A standardized mean difference is used when the outcome measure is the same but measured differently in individual studies. It standardizes the results to a uniform scale for combination in meta-analyses.

How is a standardized mean difference calculated?

The standardized mean difference is calculated by dividing the difference in mean outcomes between the groups by the standard deviation.

What does a standardized mean difference of 0.5 indicate?

A standardized mean difference of 0.5 indicates that the average effect of treatment across studies is half of a standard deviation unit.

What is the significance of the weighted mean difference in the script's first example?

The weighted mean difference in the first example shows that cardio selective beta blockers significantly reduced FEV1 on average by 2.39 percent more than placebo, but this effect was not significant as the confidence interval crossed the line of no difference.

What is the appropriate measure for summarizing studies with dichotomous outcomes like death?

For studies with dichotomous outcomes like death, it is appropriate to use a risk ratio or relative risk to summarize the individual studies.

How does the script describe the effect of calcium supplementation on bone density in the BMJ study?

The script describes the effect of calcium supplementation on bone density as a small effect with a standardized mean difference of 0.14.

What are some measures used in meta-analyses of diagnostic test studies?

Measures used in meta-analyses of diagnostic test studies include sensitivity, specificity, likelihood ratios, and the summary ROC curve, which looks at the trade-offs between sensitivity and specificity.

Outlines

00:00

📊 Outcome Measures in Systematic Reviews and Meta-Analyses

This paragraph introduces the topic of outcome measures used in systematic reviews and meta-analyses, focusing on treatment studies. It distinguishes between dichotomous outcomes, which are binary (yes/no) like survival status, and continuous outcomes, which vary along a scale such as blood pressure. The paragraph explains how dichotomous outcomes are summarized using odds ratios, relative risks, or risk differences, while continuous outcomes can be summarized through mean differences or standardized mean differences. The mean difference is used when the same outcome measure is used in each study and is measured identically, whereas the standardized mean difference is used when the same outcome is measured differently across studies. The paragraph also discusses the interpretation of standardized mean differences, with values of 0.2 or less indicating a small effect, 0.5 a moderate effect, and 0.8 or greater a large effect. Two examples from systematic reviews are provided to illustrate these concepts: one examining the effect of cardio selective beta blockers on FEV1 and another looking at the impact of calcium supplementation on bone density.

05:01

🔍 Diagnostic Test Studies in Systematic Reviews

The second paragraph extends the discussion to diagnostic test studies within systematic reviews, highlighting measures such as sensitivity, specificity, and likelihood ratios, which are interpreted in the same manner as in the primary studies. It introduces the concept of the summary ROC curve, or receiver operating characteristic curve, which is a graphical representation that illustrates the trade-off between sensitivity and specificity at various cutoff points. The ROC curve is particularly useful for combining diagnostic test summary studies in a meta-analysis. The paragraph concludes by emphasizing the importance of understanding these outcome measures for interpreting systematic reviews effectively and invites viewers to reach out with any questions through the course website or the contact section of the presenter's blog.

Keywords

💡Meta-analysis

A meta-analysis is a statistical technique used to combine the results of multiple studies to identify patterns and trends. It is central to the video's theme as it discusses the process of synthesizing data from various sources. The script refers to meta-analysis when explaining how studies are combined to report a common outcome measure.

💡Outcome Measures

Outcome measures are the tools used to quantify the effects of an intervention or treatment. The video focuses on different types of these measures, such as dichotomous and continuous outcomes, which are essential for understanding the impact of treatments in medical research.

💡Dichotomous Outcomes

Dichotomous outcomes represent binary results, such as yes/no or alive/dead. The script explains that these outcomes can be summarized using odds ratios, relative risks, or risk differences, which are crucial for interpreting the success or failure of a treatment.

💡Continuous Outcomes

Continuous outcomes are those that can vary along a continuum, like blood pressure. The video describes how these outcomes are analyzed, either through mean differences or standardized mean differences, to capture the extent of change due to an intervention.

💡Odds Ratios

Odds ratios are a way to express the likelihood of an event occurring in one group compared to another. The script mentions odds ratios as a method to summarize dichotomous outcomes, indicating the strength of association between an event and a treatment or condition.

💡Relative Risks

Relative risks quantify the risk of an event occurring in an exposed group compared to an unexposed group. The video script uses relative risks as an example of how to interpret dichotomous outcomes in treatment studies.

💡Risk Differences

Risk differences measure the absolute difference in the probability of an event between two groups. The script explains that risk differences are used to summarize dichotomous outcomes, providing a clear numerical difference in outcomes between groups.

💡Mean Difference

Mean difference is the average difference between the mean values of two groups. The video script illustrates this concept by discussing how it can be used when the same outcome measure is assessed in the same way across different studies.

💡Standardized Mean Difference

Standardized mean difference is a measure that allows for the comparison of outcomes that are measured differently across studies. The script explains its calculation and importance in meta-analysis when combining studies with different measurement scales.

💡Effect Size

Effect size is a term used to express the magnitude of a treatment's effect. While the script prefers 'standardized mean difference,' it acknowledges the term 'effect size,' explaining how different values indicate small, moderate, or large effects.

💡Systematic Review

A systematic review is a comprehensive analysis that uses explicit methods to appraise and synthesize evidence about all aspects of a specific research question. The script mentions systematic reviews in the context of analyzing different types of studies, including diagnostic tests.

💡Sensitivity and Specificity

Sensitivity and specificity are measures used in diagnostic test studies to assess the accuracy of a test. Sensitivity refers to the proportion of true positives correctly identified, while specificity is the proportion of true negatives correctly identified. The script explains these terms in the context of summarizing diagnostic test studies.

💡Likelihood Ratios

Likelihood ratios are used to compare the probability of a test result in the presence of a disease to the probability of the same result in the absence of the disease. The script includes likelihood ratios as a measure used in systematic reviews of diagnostic test studies.

💡Summary ROC Curve

A Summary ROC Curve, or Receiver Operating Characteristic curve, is a graphical representation used to show the performance of a diagnostic test at various threshold settings. The script describes how this curve helps to visualize the trade-offs between sensitivity and specificity in diagnostic studies.

Highlights

Introduction to different types of outcome measures used in systematic reviews and meta-analyses.

Explanation of dichotomous outcomes in treatment studies, such as yes/no results like survival status.

Description of continuous outcomes, like blood pressure, measured along a continuum.

Use of odds ratios, relative risks, or risk differences to summarize dichotomous outcomes.

Mean difference for summarizing continuous outcomes when measured the same way across studies.

Standardized mean difference for continuous outcomes measured differently in individual studies.

Formula for calculating standardized mean difference using mean outcomes and standard deviation.

Interpretation challenges of standardized mean difference due to its unit of standard deviation.

Guidance on interpreting the magnitude of standardized mean differences: small, moderate, and large effects.

Example of meta-analysis using mean difference in studies measuring FEV1 the same way.

Illustration of risk ratio application in a meta-analysis of death outcomes in medical patients.

Demonstration of standardized mean difference in a study on calcium supplementation's effect on bone density.

Interpretation of a small standardized mean difference in the calcium supplementation study.

Introduction to outcome measures in meta-analysis of diagnostic test studies, including sensitivity and specificity.

Explanation of likelihood ratios and summary ROC curves in diagnostic test studies.

Overview of how meta-analysis combines different diagnostic test summary studies.

Encouragement for viewers to contact for questions through the course website or blog.

Transcripts

hi Terry Chaney fell for UAB School of

Medicine when a meta-analysis is

undertaken all the studies are combined

and in common outcome measure is

reported in this video I'll describe the

different types of outcome measures that

are commonly used in systematic reviews

and meta-analyses so first let's talk

about the outcomes of treatment type

studies and they can be of two types

that can be dichotomous which is yes/no

outcomes like people were dead or they

were alive or they can be continuous

which are outcomes that can occur along

a continuum something like blood

pressure so first let's look at

dichotomous outcomes and they can be

summarized using odds ratios relative

risks or risk differences and these

measures interpret the same way as it

would be interpreted and used in the

primary studies now continuous outcomes

can be summarized in one of two ways

let's first focus on the mean difference

so if the outcome measure is the same in

each study and it's measured the same

exact way the results can be averaged

and we can calculate a mean difference

and it measures the absolute difference

between the mean value in the two groups

in the clinical trial and it just

estimates the amount by which the

experimental intervention changes the

outcome on average compared with the

control group now over here with the

standardized mean difference if the

outcome measure is the same but it's

measured differently in the individual

studies then we want to calculate

something called a standardized mean

difference and sometimes this is called

an effect size though we prefer a

standardized mean difference and we need

to standardize the results of each of

the individual studies to a uniform

scale so that we can combine them they

can't be all measured using different

scales because then we couldn't combine

them it wouldn't make any sense

so this formula sort of shows us how we

can calculate the standardized mean

difference we'll have a difference in

mean outcomes between the groups and we

divide it by the standard deviation now

this can be a little bit different

- in difficult to interpret because it's

reported in units of standard deviation

not the units of the measurement scales

used in the studies so that's can be a

little bit confusing so for example the

standardized mean difference of 0.5

means that the average effect of

treatment across studies is 1/2 of a

standard deviation unit that's kind of

confusing but what I've shown here is

how we can interpret the importance of

that standardized mean difference so if

the standardized mean difference is 0.2

or less it's really a pretty small

effect that's 0.5 it's a moderate effect

if it's point 8 or greater that's a

pretty large effect now these two four

spots are from two systematic reviews to

demonstrate some of these measurements

so the top plot up here looked at the

effect of cardio selective beta blockers

versus placebo on FEV ones and so in all

these studies it was measure the fev1

was measuring the exact same way

therefore we can use a mean difference

and it's a weighted mean difference here

because most meta-analyses weight

individual studies and so it results in

a way to mean difference but it's the

same thing as that mean difference so

you can see here this is non significant

because the conference interval here

crosses the line of no difference and

the weighted mean differences 2.39 or

the way we'd interpret this is the

cardio selective beta blockers in

significantly reduced fev1 on average by

2.3 9 percent more than placebo now down

here in the bottom is a systematic

review of the effects of low molecular

heparin compared to unfractionated

heparin in medical patients on venous

thromboembolism prophylaxis looking at

the outcome of death and so death is a

dichotomous outcome either a debtor

you're not so it's appropriate to use a

risk ratio or relative risk to summarize

these individual studies and finally

this is a forest pot of the effects of

calcium supplementation on bone density

published in the BMJ a few years ago the

outcome is the same in all these studies

so it's bone density but it was measured

differently so in this case because the

individual measures of the same outcome

or diff

we have to calculate a standardized mean

difference and that's what these authors

did and you can see here at the bottom

the effect of calcium supplementation on

the stand was a standardized mean

difference of 0.14 and if you remember

back to that last slide any standardized

mean difference or effect size of 0.2 or

less is a small effect so calcium have a

very small effect on bone mineral

density and finally to be complete we

can do meta-analysis or systematic

review of diagnostic test studies and

these are some of the measures that will

be used in those types of studies the

things that we commonly know about

sensitivity and specificity and

likelihood ratios again they're

interpreted the same way they would be

in the primary studies and then here's

another newer term a summary ROC curve a

receiver operator characteristic curve

which looks at the trade-offs between

sensitivity and specificity and it plots

sensitivity versus 1 minus specificity

at a variety of cutoff points so these

are different ways diagnostic test

summary studies can be combined with a

meta-analysis and these are the outcome

measures that would be used I hope this

video has helped you understand how to

interpret common outcome measures used

in systematic reviews remember if you

have any questions you can contact me

through the course website or through

the contact me section of my blog have a

great day