Part 4: A worked example

Summary

This video walkthrough demonstrates how to apply the signaling questions for Domain 1 of ROB 2, focusing on bias arising from the randomization process. The example is based on a trial comparing aerobic, strength, and relaxation training for treating mild to moderate depression. The video explores how randomization was performed, its concealment, and how to assess whether any baseline imbalances in participant characteristics suggest issues with randomization. It guides viewers through analyzing trial details to assess the risk of bias, ultimately concluding with a low risk of bias judgment for the trial’s randomization process.

Takeaways

• 😀 The demo trial compares twice-weekly aerobic, strength, and relaxation training for 4 months to alleviate mild to moderate depression.
• 😀 The trial used randomization to assign participants to one of the three intervention groups, with a centralized, computerized system.
• 😀 Randomization was stratified based on medication status (no antidepressant medication, less than 6 weeks of antidepressant use, or more than 6 weeks).
• 😀 The trial staff had to contact the Copenhagen trial unit (CTU) by phone to determine the allocation sequence, ensuring concealment of allocation.
• 😀 The first two signaling questions focus on whether the allocation sequence was random and whether it was concealed until participants were enrolled.
• 😀 The third signaling question investigates whether substantial baseline imbalances suggest problems with the randomization process.
• 😀 The baseline characteristics table showed no significant imbalances that would indicate a problem with the randomization process, suggesting the randomization was well-executed.
• 😀 Although there were minor baseline differences (e.g., gender and depression scale scores), they were deemed compatible with chance given the small sample size of 55 participants per group.
• 😀 The trial used a block size of 6 for randomization, which was concealed from trial staff, and the potential for block size detection was minimal due to the small sample size.
• 😀 After considering the answers to the signaling questions, the overall risk of bias for randomization was assessed as 'low risk'.
• 😀 The trial’s overall risk of bias judgment was based on well-conducted randomization and the absence of substantial baseline imbalances.

Q & A

What is the purpose of the worked example in the transcript?

The purpose of the worked example is to apply signaling questions from Domain 1 of ROB 2, which addresses bias arising from the randomization process, to a real trial—specifically, a trial comparing different forms of training for alleviating mild to moderate depression.

What types of training were compared in the demo trial mentioned in the transcript?

The demo trial compared twice-weekly aerobic, strength, or relaxation training for 4 months to alleviate mild to moderate depression. The aerobic and strength training arms were the intervention groups, with the relaxation arm acting as the control group.

What are the three signaling questions for Domain 1 of ROB 2?

The three signaling questions for Domain 1 ask: 1) Whether the allocation sequence was random, 2) Whether the allocation sequence was concealed until participants were enrolled, and 3) Whether there were substantial imbalances in baseline characteristics that could indicate issues with the randomization process.

Why were the first two signaling questions emphasized in this presentation?

The first two signaling questions were emphasized because the answers to these questions are often readily available in trial reports, and they address the critical processes of random sequence generation and concealment of the allocation sequence.

How was the randomization process carried out in the demo trial?

In the demo trial, randomization was centralized, stratified by medication status, and performed using computerized restricted randomization with a block size of six. The allocation sequence was unknown to the trial staff.

What was the significance of using a block size of six in the randomization process?

The use of a block size of six was meant to ensure balance between groups, but it was structured so that trial staff did not know the allocation sequence. While concerns might arise due to the fixed block size, it was unlikely to cause a bias due to the small trial size and the randomization occurring within strata.

What concerns might arise regarding randomization when using a fixed block size like six?

The concern with using a fixed block size is that trial staff could potentially deduce the sequence after several participants have been enrolled. However, in this trial, the randomization occurred within strata, and the small sample size further minimized this risk.

What does the baseline characteristics table in the demo trial suggest about the randomization process?

The baseline characteristics table suggests that the randomization process was likely carried out correctly, as the groups were balanced in terms of key characteristics, such as medication use. There were minor differences in depression scores and gender proportions between the groups, but these are deemed compatible with chance given the small sample size.

What is the significance of no p-values being reported in the baseline characteristics table?

The absence of p-values in the baseline characteristics table is not unusual, as it is not expected for review authors to calculate them for the signaling question analysis. Instead, reviewers focus on identifying substantial imbalances, which could suggest issues with the randomization process.

What is the final judgment for the risk of bias in the demo trial's randomization process?

Based on the answers to the signaling questions, the overall judgment for the randomization process in the demo trial was a low risk of bias. This judgment was informed by the fact that the allocation sequence was adequately generated and concealed, with no significant baseline imbalances.