- JoNova - https://www.joannenova.com.au -

85% clinical medical research is false, or not useful, not worth the money – government funded waste

Peer Review, Scientific papers, Cartoon.John Ioannidis paints a picture of a vast hive of researchers all pushed to publish short papers that are mostly a waste of time. The design is bad, the results useless (even when meta-collated with other badly designed studies). Basically, humankind is pouring blood, sweat and tears into spinning wheels in medicine — just paper churn. Most papers will never help a patient.

Ioannidis wants rigor – full registration before the study, full transparency afterwards, fewer studies over all, but with better design. Astonishingly, fully 85% of what is spent on clinical trials is wasted. It’s really a pretty big scandal, given that lives are on the line. I can’t see the media or pollies joining the dots. Imagine how many quality life-years are being burnt at the stake of the self-feeding Science-PR-Industry.

And this is clinical medical research, where standards are higher than in many other scientific areas and where there are easily defined terms of success unlike “blue sky” studies. Ioannidis doesn’t say it directly, but his description of the effect current funding has (which is almost all government based) almost guarantees that researchers will be wasting time in the paper churn — fast, short papers of little importance, that may even be false, but even if true are useless, insignificant. This is what happens when science is controlled by a government monopsony. The aim is the press release, not the patient.

Science can’t be done by an indexed formula or citiation score. Money can’t be spent wisely that way either. Someone needs to be responsible.

How much of climate research is a waste of money? A lot more than in clinical medicine.

Jo

Conclusion

Overall, not only are most research findings false, but, furthermore, most of the true findings are not useful. Medical interventions should and can result in huge human benefit. It makes no sense to perform clinical research without ensuring clinical utility. Reform and improvement are overdue.

The sheer size of the waste and the industry — 85% of a million papers

There are many millions of papers of clinical research—approximately 1 million papers from clinical trials have been published to date, along with tens of thousands of systematic reviews—but most of them are not useful. Waste across medical research (clinical or other types) has been estimated as consuming 85% of the billions spent each year [1]. I have previously written about why most published research is false [2] and how to make more of it true [3].

Clinical research remains extremely expensive, even though an estimated 90% of the present cost of trials could be safely eliminated [26,27]. Reducing costs by streamlining research could do more than simply allow more research to take place. It could help make research better by reducing the pressure to cut corners, which leads to studies lacking sufficient power, precision, duration, and proper outcomes to convincingly change practice.

The problem is the research funding:

Current research funding incentivizes small studies of short duration that can be quickly performed and generate rapidly publishable results, while answering important questions may sometimes require long-term studies whose financial needs exceed the resources of most currently available funding cycles.

One suggestion to improve research funding:

One to two percent of the sales of blockbuster drugs diverted in such a pool [52] could earmark ample funding.

Snippets of interest from the full paper:

Transparency (Trust)

Utility decreases when research is not transparent, when study data, protocols, and other processes are not available for verification or for further use by others. Trust is also eroded when major biases occur in the design, conduct, and reporting of research.

Only 61% of trials published in clinical journals in 2010 had been registered [30], and rates are much lower for nonregulated interventions [31] (e.g., 21% and 29% for trials published in psychological or behavioral [32] and physical therapy [33] journals, respectively). Only 55/200 (28%) of journals that publish clinical trials required trial registration as of 2012 [34]. Few full protocols are registered, analysis plans are almost never prespecified, and the full study data are rarely available [35].

Focusing on major journals.

Some clinicians prefer to read only research published in major general medical journals (The New England Journal of MedicineThe LancetBMJJAMA, and PLOS Medicine). However, these journals cover a tiny minority of published clinical research. Out of the 730,447 articles labeled as “clinical trial” in PubMed as of May 26, 2016, only 18,231 were published in the major medical journals. Most of the articles that inform guidelines and clinical practice are published elsewhere.

 Getting “free” research from students, volunteers and trainees is part of the problem, he says:

Clinical Research Workforce and Physicians

 The clinical research workforce is huge: millions of people have coauthored at least one biomedical paper, and most have done so only once [48]. Students, residents, and clinical fellows are often expected to do some research. This exposure can be interesting, but trainees are judged on their ability to rapidly produce publications, a criterion that lends itself badly to the production of the sort of large, long-term, team-performed studies often needed to inform us about health, disease, and health care. Such researchers can become exploited as low-paid or volunteer personnel [49], and an untrained, noncommitted workforce cannot produce high-quality research. Other perverse recipes in clinical research include universities and other institutions simply asking for more papers (e.g., least publishable units) instead of clinically useful papers and clinical impact not being a formal part of the publication metrics so often used to judge academic performance. Instead of trying to make a prolific researcher of every physician, training physicians in understanding research methods and evidence-based medicine may also help improve the situation by instilling healthy skepticism and critical thinking skills.

h/t Kip Hansen at  Judith Curry

REFERENCES

John P. A. Ioannidis (2016) Why Most Clinical Research Is Not Useful, PLOS Med Published: June 21, 2016 http://dx.doi.org/10.1371 /journal.pmed.1002049

Ioannidis JP. Why most published research findings are false. (2005)  PLoS Med. ;2(8):e124. pmid:16060722 doi: 10.1371/journal.pmed.0020124 ..View Article. PubMed/NCBI

Ioannidis JP. (2014) How to make more published research true. PLoS Med.;11(10):e1001747. doi: 10.1371/journal.pmed.1001747. pmid:25334033.View Article. PubMed/NCBI .

9.2 out of 10 based on 41 ratings