What happens when an RCT stops early, and how can that affect the results and future practice?
The theory behind clinical research trials includes all sorts of concepts that are designed to help us to get an accurate picture of the differences between two interventions. For instance;
- Ideally, both the participant and the researcher should be ‘blind’ to which treatment the person receives, so there is no chance that their perceptions of the treatment’s effectiveness will affect the outcome.
- The method of randomisation should not allow clinicians to influence which group each person enters, in case this biases the results.
- Data from a good number of people needs to be collected in each arm of the trial in order to effectively measure any difference between the groups.
Indeed, it is because of this last point that the methods sections of many papers reporting trials contain ‘the power calculation’: that paragraph of rather inaccessible maths demonstrating how the researchers worked out how many people they would need to recruit in order to get accurate – and potentially significant – results.
There is, however, an increasing trend towards stopping clinical trials earlier than planned – and after fewer people have participated in them than was planned – because the intervention under scrutiny is perceived to be of such great benefit that it is deemed unethical to continue to expose people to the less effective intervention. Generally, when such an announcement is made, the trial receives a larger degree of publicity than normal and results may be more quickly implemented into clinical practice and guidelines – despite the fact that the trial has included fewer participants than was deemed necessary at the outset. Perhaps the most well-known recent example in relation to midwifery is that of the Canadian Term Breech Trial, which was stopped after 2088 out of a planned 2800 women were enrolled, because the Data Safety monitoring Committee decided that the results were clearly in favour of caesarean section (Hannah et al 2000). There are, however, far more blatant examples in areas of industry-funded medicine where large amounts of money can be made from pharmaceutical sales in high-profile areas such as HIV and cancer treatments (Montori et al 2005).
Until recently, it has been rather hard to make an overall assessment of what lies underneath the trend towards stopping trials early, and whether it is a good safety net or something about which we should be concerned. Which is why Montori et al’s (2005) systematic review of this area made such interesting reading for those of us who were not sure whether the trend towards stopping certain trials was a positive reflection of advances in our knowledge and technology or, conversely, something which we should view with scepticism.
The review itself included 143 trials which were stopped early ‘for benefit’.The majority of these trials (64%) were published in one of the “top five” medical journals, thus increasing their impact. Over 94% of the trials omitted to report something the reviewers considered important, for example the planned sample size as well as the actual sample size, or a statistical analysis which was properly adjusted to account for the smaller number of actual participants rather than one based on the intended number of participants. Crucially, the reviewers found that trials which included fewer events (and people) than planned were more likely to show a greater treatment effect. In other words, if you want to prove that a sparkly new proposed treatment is effective, one way of doing this is to get the clinical trial stopped early, as the results are then more likely to favour the new treatment.
Montori et al (2005: 2208) conclude by suggesting that:
“RCTs stopped early for benefit are becomingmore common, often fail to adequately report relevant information about the decision to stop early, and show implausibly large treatment effects, particularly when the number of events issmall. These findings suggest clinicians should view the results of such trials with skepticism.”
In practice terms, of course, scepticism doesn’t mean that we deny the results of such studies entirely. It means that we keep an open mind until we see what the results of further studies say. Keeping an open mind seems, to me, a far better plan than the alternative: adopting new treatments and practices in an area on the basis of one, high-profile study which may or may not be supported by other studies which continue until they recruit the number of participants that they intended to recruit at the outset.
This is an older article. For up-to-the-minute updates on new birth-related research and thinking, jump on my newsletter list!
A version of this article was originally published as Wickham S (2006). RCTs and Cessation Frustration. TPM 9(2):35.
Hannah, M.A., Hannah, W.J., Hewson, S.A. and others(2000) Planned caesarian section versus planned vaginal birth for breech presentation at term: a randomized multicentre trial. The Lancet. Vol. 356.Issue 9239. pp. 1375-1383.
Montori, VM, Devereaux, PJ, Adhikari, NKJ and others (2005) Randomized Trials Stopped Early for Benefit: A Systematic Review JAMA. 2005;294:2203-2209.