Whilst putting together a research proposal I stumbled across a quote from Archie Cochrane, one of the fathers of evidence based medicine. The proposal was about how results of randomised controlled trials generalise (or don’t) to the general public and the quote fitted perfectly.
Between the findings of an RCT and benefit in the community … there is a gulf which has been much underestimated.
Archie Cochrane, 1972 (Effectiveness and Efficiency)
I thought this was a great quote for many reasons, it fit in my proposal incredibly well and it’s always nice for an expert to have had similar ideas. But mostly, I was fascinated that similar concerns to that of my research had been around for 50 years! So I managed to get a copy of Archie Cochrane’s book where this was mentioned and have a read.
Wow, was the book an amazing read! He detailed what he believed the main issues facing the NHS (The UK’s National Health Service) were. What shocked me is how similar many of the issues were to ones we still face today; in Australia and across the world. In this post I’ll outline just one of the issues Cochrane discussed.
Cochrane opened the book with a statement from his time as a medical student in the 1930s where he would protest that “All effective treatments must be free”. This is the premise of the book and is not as straightforward a statement as you might think.
First, what does effective even mean? In order for a treatment to be deemed effective, it must reduce the burden of a condition MORE than the natural history. This is a fascinating point and something often hard to determine, which is why we need randomised controlled trials. Sometimes a condition may improve with time, even without intervention, so if someone does intervene, ie. a doctor giving someone a drug or surgery, and the patient subsequently gets better, it can be easily mistaken for the treatment to have caused this improvement. Without a randomised controlled trial, relying on anecdotal, or observational evidence this can lead to ineffective treatments being used across a country or the world. A well designed randomised controlled trial can show beyond doubt that a treatment CAUSES an effect because there is a control group, who ideally undergoes everything as the intervention group does, without the active treatment. Natural history is often the reason control groups can improve in a randomised controlled trial even though they have had no treatment. This is also why it is crucial to measure BETWEEN-GROUP differences in a randomised controlled trial, because it is the difference which you are looking for, however, this is often not the case because many treatments tested are indeed not effective, making the studies harder to publish. This is a completely separate conversation to be had, but a useful point to note.
Cochrane explains there are many treatments which become embedded into health systems that are not effective. His example from the 70s was that having people with acute ischaemic heart disease admitted to hospital and coronary care units was seen at the time as crucial to improving outcomes, it was also very expensive because almost every hospital had to set up specialist coronary care units. However, later a randomised controlled trial showed that being admitted to a coronary care unit had no better outcomes to treatment at home. If this randomised controlled trial was done PRIOR to embedding coronary care units in hospitals across the country, they likely would have saved millions of dollars because they would not have had to do so.
This seems very abstract and outdated, for all I know this may not be the case anymore, so I’ll discuss an example where a routine procedure proves to be no better than a placebo.
Knee arthroscopy, or getting the knee ‘cleaned out’ is STILL a very common procedure in people with knee issues, meniscus tears, osteoartritis etc. However, way back in 2002 a blinded (meaning the patients did not know what treatment they received) randomised controlled trial compared knee arthroscopy to sham knee arthroscopy. Sham knee arthroscopy? I hear you say, well this seems ridiculous, but the surgeons did EVERYTHING but the actual debridement of the knee joint, meaning the patients underwent surgery where they got cut into, in the same point as the intervention group did, but they were immediately stitched back up having no work done.
Both groups had THE SAME outcome… Meaning, there was no extra benefit of actually doing the surgery. It was all a placebo effect. This is why, randomised controlled trials are IMPERATIVE to do before implementing new treatments into a health service, because things that seem to work may not always.
This could become a much longer blog post but I think just touching on the effectiveness of treatments is enough for now.
Harrison Hansford 