By Peter J Gill
As a junior doctor, the idea of changing the healthcare system seems daunting, even impossible. The delivery of healthcare is not only complex but also frozen with the inertia of a large organization. I felt similarly during my doctoral thesis, one person trying to tackle an ambitious research agenda in a narrow sphere of science.
Archie Cochrane, the late Scottish physician whom the Cochrane Collaboration is named after, probably felt similar doubts as a young doctor. Yet, as a prisoner of war during World War II, he managed to conduct a simple non-randomized controlled trial showing that supplementing prisoner’s diet with yeast prevented the development of severe edema. Cochrane’s first foray into trials ignited a life-long pursuit interrogating the effectiveness and efficacy of routine therapies.
Cochrane’s journey reminds us that important changes start with an idea, often a simple one. Yet, simple ideas can be revolutionary. The most innovative and creative ideas often come from students, trainees and early career researchers. By nature of their status, these individuals are often on the ‘front-line’, seeing the way healthcare is delivered and research is conducted.
Compare Trials is one example of a student-led initiative that is making waves. Many published clinical trials report outcomes that are different from those pre-specified during trial registration. According to the CONSORT Statement, guidelines which outline the minimum reporting standards for clinical trials, outcomes must be pre-specified (i.e. decided before the trial begins recruitment), and trial reporting in peer reviewed journals should be consistent with the protocol.
Unfortunately, this often does not happen. Outcomes that were pre-specified in the protocol are not reported, or new outcomes are reported that were not pre-specified. One might expect this to happen in little known journals, but it happens in prestigious general medical journals like the Lancet, BMJ and NEJM. Research is messy, and unforeseen circumstances frequently arise, but the final paper should clearly flag and explain why outcomes were changed. Ultimately, outcome switching distorts data for front-line clinicians.
Compare Trials, spearheaded by graduate medical students at the University of Oxford and supported by CEBM wants to bring attention to the problem of outcome switching. They are identifying all clinical trials published in major general medical journals (e.g. NEJM, Lancet, BMJ, Annals of Internal Medicine and JAMA) and are looking for discrepancies between the published paper and the trial protocol.
While similar projects have been conducted in the past, Compare Trials is unique for several reasons. First, they are systematically writing to all journals highlighting the issues they identify, requesting responses and tracking progress. Second, they are posting responses from journals and highlighting misunderstandings (which seem to be pervasive in the editorial community). Third, everything is open and transparent including the data extraction table, the letter sent to the editor and the responses received. Compare Trials intends the conversation about outcome switching with medical journals to be a public one.
There are several other examples of successful student led initiatives such as Students 4 Best Evidence and Hacking Health. However, many other ideas stall at the implementation phase. Barriers are erected as new ideas often clash with the conventional mantra. The medical students involved in Compare Trials had help and support from CEBM but such support may not always exist.
At CEBM and Evidence Live, we recognize this dilemma, and we want to help. The Future Leaders in Evidence-Based Medicine stream at Evidence Live targets students, junior doctors and early career researchers. We want to hear about projects, initiatives and new ideas and are offering 5 free places for the top 5 submissions and have 50 discounted places for future leaders.
We also want to create a network of future leaders, identify the challenges that trainees face and come up with creative solutions. Students, junior trainees and early career researchers are vital to the future of healthcare.
This blog was written by Peter Gill, member of the Evidence Live 2016 Steering Committee, paediatric resident at the [Hospital for Sick Children at the University of Toronto and an Honorary Fellow at the Centre for Evidence-Based Medicine.