The disadvantages of randomised clinical trials

Summary

The speaker discusses the limitations of clinical trials in cancer treatment development. While randomized trials are the gold standard, they often exclude patients with additional health problems, leading to a disconnect between trial results and real-world effectiveness. The use of surrogate endpoints like progression-free survival is criticized, as they may not reflect true improvements in life expectancy or quality of life. Additionally, the statistical analysis of trials often emphasizes small, marginal effects, sometimes accompanied by increased toxicity. The speaker advocates for focusing on clinically meaningful outcomes, as suggested by recent value scales from ASCO and MO.

Takeaways

• 😀 Randomized clinical trials are considered the gold standard for developing new treatments.
• 😀 Patients in clinical trials are often highly selected, meaning they tend to be healthier with fewer comorbidities, which limits the generalizability of trial results.
• 😀 Cancer patients in clinical trials are typically younger and healthier, which differs from the broader population who may have multiple health issues.
• 😀 Surrogate endpoints like progression-free survival may not accurately reflect the actual benefits of a treatment, such as life expectancy or quality of life.
• 😀 Pharmaceutical companies often choose endpoints that allow for faster approval, even if they don't show significant clinical benefit.
• 😀 Statistically significant results don't always translate to clinically meaningful improvements in patient outcomes.
• 😀 New treatments approved based on small effect sizes may be marketed with higher costs, despite offering limited additional benefits.
• 😀 Large trials detect smaller effects, sometimes with increased toxicity, but the clinical relevance of these findings may be questionable.
• 😀 Value scales introduced by organizations like MO and ASCO aim to assess the clinical significance of trial results beyond just statistical significance.
• 😀 Only about one-third of positive trials meet the ESMA criteria of value, meaning that many treatments are approved without substantial clinical benefit or manageable risks.
• 😀 Future trials need to focus more on clinically meaningful results, rather than just statistically significant differences, to improve patient care.

Q & A

Why are randomized trials considered the gold standard for developing new treatments?

Randomized trials are considered the gold standard because patients are randomly assigned to different treatment groups, which helps isolate the effects of the treatments from other variables. This makes the results more reliable and reduces bias.

What is the problem with the selection of patients for clinical trials?

The issue with patient selection is that people in clinical trials often have good performance status and are typically free from other serious conditions like cardiac, liver, or kidney disease. This selection bias means the results may not apply well to the general population, which often includes patients with multiple health problems.

How does the age of cancer patients impact clinical trials?

Cancer is more common in older individuals, and these patients often have additional health problems that are not considered in clinical trials. This discrepancy can make it difficult to apply trial results to the broader, older population of cancer patients.

Why are the endpoints used in clinical trials a concern?

Many clinical trials, especially those funded by pharmaceutical companies, use endpoints like progression-free survival that focus on delaying disease progression rather than improving overall survival or quality of life. This may lead to approval of treatments that are not truly beneficial for patients.

What is the issue with using progression-free survival as an endpoint?

Progression-free survival may not reflect real clinical benefit if a treatment only delays disease progression without improving life expectancy or quality of life. Additionally, treatments that achieve progression-free survival may come with harmful side effects, making them less valuable in practice.

What does the author think about the cost of new treatments?

New treatments are often very expensive, and their cost does not always correspond to their effectiveness. This leads to a situation where patients may face high costs for treatments with only small improvements in outcomes.

How has the size of clinical trials changed over time?

Clinical trials have become larger in order to detect smaller differences in treatment outcomes. However, this has led to the approval of treatments based on minor, often clinically insignificant, effects, sometimes with added toxicity.

What is the difference between statistical significance and clinical importance in clinical trials?

Statistical significance indicates that a result is unlikely to be due to chance, but it doesn't necessarily mean the result is meaningful in a real-world clinical setting. Clinical importance, on the other hand, refers to whether the treatment effect actually improves patients' quality of life or life expectancy in a meaningful way.

What does the introduction of value scales by MO and ASCO aim to address?

The value scales introduced by MO and ASCO aim to evaluate the clinical importance of treatment effects, not just statistical significance. These scales consider both the benefit and the toxicity of treatments to ensure that the overall value to patients is meaningful.

How successful have clinical trials been in meeting the ESMA criteria of value?

A study found that only about one-third of positive clinical trials met the ESMA criteria of value, meaning that the effect size was clinically significant and the treatment's toxicity didn't outweigh the benefit.