Manifesto Consultation: What’s on your jobs list?
Aim: To encourage working groups that will identify, take forward and implement solutions for better evidence and healthcare in respective fields. Method: The Manifesto group will facilitate the generation of jobs lists and potential solutions within different stakeholder groups in round table format. Any ideas you already have can submitted through the form below and brought to the table with a plan for discussion.
Why we need an EBM manifesto?
Patients are being let down by serious flaws in the creation, dissemination, and implementation of medical research. Too many research studies are poorly designed or executed. Too much of the resulting research evidence is withheld or disseminated only piecemeal. 1 As the volume of clinical research activity has grown, 2 the quality of evidence has often worsened, 3 which has compromised medicine’s ability to provide affordable, effective, high-value care for patients.”
Why can’t we trust the evidence?
Beyond disquiet, evidence suggests that most published research is more likely to be false than true. 4 Serious systematic bias, 5 error and wastage 6 are also well documented in medicine, and significantly affect the implementation and uptake of evidence into practice. This lack of uptake is compounded by poorly managed commercial and academic vested interests; 7 bias in the research agenda (often due to the failure to take account of the patient perspective in research questions and outcomes); 8 9 poorly designed research trials that suffer from a lack of transparency and independent scrutiny, fails to follow protocols 15 or trials that are stopped too early; 16 10 poor quality dissemination of research too often permits ghost authorship,11 publication 12 and reporting bias (including the under-reporting of harms) 13 14 and publishes results that are over-interpreted or misused, 17 or sometimes have uncorrected errors 4 or undetected fraud 18 19
Bad evidence feeds into poor decisions. And a host of allied organisations and industries have sprung up around evidence to address this problem. Yet, these too are beset with problems such as untrustworthy guideline production, 20 regulatory failings,19 and delays in the withdrawal of harmful drugs.21 Collectively these failings contribute to escalating costs of treatment,22 medical excess (including the related concepts of medicalisation, overdiagnosis, and overtreatment) 23 and avoidable harm. 21
Clinicians and patients increasingly find it difficult to identify and integrate trusted evidence into informed healthcare due to the unmanageable volume of evidence,2 the lack of research answers that matter to patients and the lack of evidence to support shared decision making, 25 3
Steps for developing more trustworthy evidence
The manifesto steps required to develop trustworthy evidence have been refined through a series of stakeholder engagement activities including seminars, round-tables, online consultations and through direct feedback with stakeholders. As a result of these discussions, a set of nine steps has been identified (see Box 1).
1) Expand the role of patients, health professionals and policy makers in research
2) Increase the systematic use of existing evidence
3) Make research evidence relevant, replicable and accessible to end users.
4) Reduce questionable research practices, bias, and conflicts of interests
5) Ensure drug and device regulation is robust, transparent and independent
6) Produce better usable clinical guidelines.
7) Support innovation, quality improvement, and safety through the better use of real world data.
8) Educate professionals, policy makers and the public in evidence-based healthcare to make informed choices.
9) Encourage the next generation of leaders in evidence-based medicine.
Why are these steps important?
1) Expand the role of patients, health professionals and policy makers in research: Research informed by and involving patients, the public, and frontline health professionals is more likely to be relevant and is an essential process by which important research is identified, designed, and disseminated. Explicitly, we need to better understand the role of research done ON patients compared to research done BY or WITH patients and its impact on improving healthcare.
2) Increase the systematic use of existing evidence: The synthesis of evidence has been essential to inform and underpin decisions in healthcare. 26 There are several advantages to using systematic reviews: 27 to supporting new health technologies and demonstrating where interventions cause unnecessary harms or waste resources. Systematically reviews also reduce the risk of wasting valuable resources on new research, when the answer might exist already.6,27,28 However, systematic reviews are not without their flaws. A 2014, landmark Cochrane review on neuraminidase inhibitors, highlighted the need to consider greater sources of evidence – often more complex evidence (such as Clinical Study Reports, regulatory documents and Individual Patient Data). 29 This review showed that informed decisions need to be based on a synthesis of all of the evidence, not just limited sources such as journal publications.
3) Make research evidence relevant, replicable and accessible to end users: Research that matters to patients should be clinically relevant; It therefore makes sense that patients should be core to developing relevant and accessible research. Replication of trial results is important to ensure results are reliable and valid and that the previous results can be applied to new situations determining generalizability. Furthermore, we a considerable amount of research findings are wasted if they are inaccessible or poorly communicated to end users.
4) Reduce questionable research practices, bias and conflicts of interests: Too much research is plagued by biases that are rooted in poor methods, leading to the wrong result and conclusions and preventing uptake into practice. For instance, a trial might be conducted flawlessly, but this is in vain if the results are not fully reported as planned. We do trials to detect modest differences and spend vast amounts of money specifically to exclude bias, yet we allow bias to flood back in through questionable research practice such as selective publication and reporting. This lets down participants, misleads patients and the public, and wastes money.
5) Ensure drug and device regulation is robust, transparent and independent: To permit robust evaluations by regulatory agencies, such as the FDA and the European Medicines Agency (EMA), requires high-quality evidence. However, a substantial number of approved drugs have significant problems that could have been discovered, and dealt with, at the time of approval. To speed the uptake of new drugs into practice there is increasing pressure to lower the burden of evidence required for approval, putting patients at unnecessary risk. But, we need a system that delivers on expediency whilst producing high-quality evidence with outcomes that matter to patients, at the time of approval and in the postmarketing phase, to ensure safe and effective drugs are granted access to the market.
6) Produce better usable clinical guidelines: Clinical practice guidelines have proliferated, yet they often contradict each other and contain little information on implementation for the individual. Clinical guideline development must be a completely transparent process, showing who has made the guideline, why they were involved, with what evidence, and why the recommendations were reached.
7) Support innovation, quality improvement, and safety through better use of real world data: The use of real world data has the potential to improve health but also appreciably worse health if used in the wrong way. True informed decisions require a range of evidence, to inform their uptake into practice. We, therefore, need better use and understanding of the role of qualitative, observational and quantitative research in informing those decisions that matter.
8) Educate professionals, policy makers and the public in evidence-based healthcare to make informed choice: Researchers, editors and journalists all have a role in communicating evidence to the public. Evidence can be misinterpreted, over-hyped and inaccurate. Poor quality or preliminary studies can be highlighted at the expense of research that matters to patients. Such practice may give rise to false hope or harm. Therefore, high quality, important research has to be understandable and informative to, and by, a wide audience. Yet, much of what is currently produced is not directed to a lay audience, is often poorly constructed and is underpinned by a lack of training and guidance in this area. To make fair and informed judgements on the value and relevance of evidence, people must have access to research and have the skills to make informed choices to support their own health.
9) Encourage the next generation of leaders in evidence-based medicine: Clinical expertise is pivotal to effective evidence-based care. Furthermore, dealing with uncertainty when applying evidence to individual patient care requires critical appraisal skills in assessing evidence and recognising poor quality evidence. To therefore deliver great healthcare we need a generation of applied healthcare leaders, clinicians and policymakers with skills in assessing, appraising and applying evidence to patient care.
Taking the first steps to creating trustworthy evidence
To tackle these issues will take time, resources and effort. As an EBM community, we should prioritise initial efforts to tackle the manifesto steps we have outlined. Fixing EBM is a vast project that is being led and will be led, by disparate groups around the globe. We hope to focus global attention on the most effective tools and strategies we can use to deliver that change so that we can all work together and commit to improving healthcare using better quality evidence. These manifesto priorities will change in the light of plans to deliver trusted evidence for better healthcare but hopefully will provide necessary guidance for developing the properties that matter for better, more trustworthy, evidence.
The manifesto has been developed by people engaged at all points in the research ecosystem engaging in fixing the problem, including above all patients and the public who indirectly fund but are directly affected by the outputs of the current system. We would like to thank all those who have provided feedback, all those partners who hosted roundtables and seminars and those who gave feedback and are listed on the www.evidencelive.org site.Editorials
We want to focus on solutions, and help offer a structure for people to coordinate positive action. We want to map what is currently being done. We want to formulate clear goals, so that we can work back from these, to inform the interventions that are deployed. We want your help to achieve Better Evidence for Better Healthcare.