Part 3: Assessing risk of bias from the randomization process in RoB 2

Summary

The speaker compares the 2011 Cochrane Risk of Bias (RoB) tool with the updated RoB 2 tool, focusing on significant changes to improve clarity and accuracy in bias assessment. Key updates include eliminating confusing 'selection bias' terminology, combining sequence generation and allocation concealment into one domain, and introducing a signaling question on baseline imbalances. The RoB 2 tool incorporates an algorithm to suggest risk of bias judgments, which can be overridden with explanations. The speaker emphasizes how these changes provide a more precise, user-friendly approach to evaluating randomization and bias in clinical trials.

Takeaways

• 😀 The term 'selection bias' has been removed from the RoB 2 tool because it was confusing and often misinterpreted, as it included issues related to participant selection, which are now considered more an issue of applicability and directness.
• 😀 In the RoB 2 tool, the domains of 'sequence generation' and 'allocation concealment' have been combined into a single domain, as improper implementation of either can lead to bias in group distribution.
• 😀 The removal of 'selection bias' was based on the recognition that the term had different meanings for different people, which led to confusion and misinterpretation in previous versions of the tool.
• 😀 The RoB 2 tool now addresses baseline imbalances by asking whether such imbalances suggest a problem with the randomization process, rather than just identifying them as bias.
• 😀 A key change in RoB 2 is the introduction of signaling questions that help assess whether randomization processes were implemented properly, such as ensuring the concealment of the allocation sequence and its randomness.
• 😀 In RoB 2, baseline imbalances are only considered concerning if they are substantial and provide evidence that the randomization process was problematic, reducing the chance of misinterpreting minor imbalances as bias.
• 😀 The RoB 2 tool includes an algorithm that suggests a default judgment for the risk of bias based on user answers to signaling questions, although users can override these suggestions with a rationale.
• 😀 The algorithm in RoB 2 aims to flag high risks of bias only when issues are serious enough to affect the overall trial result, and users can opt to modify the judgment based on their own expertise.
• 😀 The algorithm does not follow a strict chronological order, but rather arranges the questions and decisions in the most efficient way possible to minimize complexity.
• 😀 The RoB 2 tool offers a clearer approach to assessing bias, ensuring that the tool remains both flexible and user-friendly, with guidance to help users understand when to raise concerns about bias in randomized trials.

Q & A

What is the main difference between the original RoB (2011) tool and the RoB2 tool?

The RoB2 tool merges sequence generation and allocation concealment under one domain, removing the use of the 'selection bias' term, which was often misinterpreted. This change aims to simplify the bias categorization and improve the clarity of the tool.

Why was the 'selection bias' term removed from the RoB2 tool?

'Selection bias' was removed because it was frequently misunderstood and used inappropriately to describe issues unrelated to bias, such as the representativeness of participants. Instead, RoB2 addresses randomization issues more directly, aligning with modern epidemiological standards.

What does 'selection bias' mean in the context of modern epidemiology and the RoB2 tool?

In modern epidemiology, 'selection bias' refers specifically to issues in randomization, such as how participants are assigned to groups. It does not refer to participant choice or the representativeness of the study population, which are now considered issues of applicability or directness.

How does the RoB2 tool address the randomization process differently than the 2011 tool?

RoB2 combines the domains of sequence generation and allocation concealment under one domain, which streamlines the evaluation of the randomization process. It recognizes that issues in either of these areas can lead to unbalanced or biased distribution of patients, affecting the study's outcomes.

What role does baseline imbalance play in the RoB2 tool?

In RoB2, baseline imbalances are only considered problematic when they are substantial and suggest a flaw in the randomization process. It’s not about minor imbalances, which are typically due to chance, but about significant imbalances that may indicate randomization was not properly implemented.

Why was there an emphasis on 'other bias' in the original RoB tool?

In the original RoB tool, 'other bias' was frequently added to the assessment when baseline imbalances were observed. However, this was often misinterpreted as bias, leading to inaccurate classifications. RoB2 clarifies that minor imbalances are not biases and helps prevent this over-interpretation.

How does the RoB2 tool use signaling questions to assess bias?

RoB2 uses signaling questions to assess key aspects of a trial, such as random sequence generation and allocation concealment. These questions help determine whether any issues in these areas indicate a risk of bias, guiding the evaluator toward a final judgment.

What is the role of the algorithm in the RoB2 tool?

The RoB2 tool includes an algorithm that suggests a default risk of bias judgment based on the answers to the signaling questions. This algorithm is designed to identify high-risk biases and can be overridden if the user provides a reasonable explanation.

How are the answers to signaling questions presented in the RoB2 tool?

The answers to signaling questions are presented as 'yes,' 'probably yes,' 'probably no,' or 'no.' Each answer is supported by evidence or comments to explain the reasoning behind the judgment, especially when the answer reflects a negative outcome.

What happens if a user disagrees with the algorithm’s judgment in the RoB2 tool?

If a user disagrees with the algorithm’s suggested judgment, they can override it but must provide an explanation for why the decision was altered. However, most of the time, the expectation is that the user will follow the algorithm's suggestion.