{'fraction_non_alphanumeric': 0.03257855822550832, 'fraction_numerical': 0.01686691312384473, 'mean_word_length': 5.123055162659123, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Stanford researcher contends most medical research results are exaggerated

Patients and doctors alike should employ healthy skepticism before acting on discoveries that, at first blush, herald exciting advances, since the majority of studies cannot be replicated, warns a keynote speaker to the World Congress on Research Integrity, held recently in Montreal, Quebec.

“

specializes in appraising and controlling for biases in biomedical research. He conducts metaanalyses to test the validity of highly cited genetic association studies, genome-wide association studies and other randomized trials and observational research (JAMA 2013;309:1683).
During Ioannidis' talk to about 370 researchers, journal editors and ethicists, he gave examples of research that have not withstood the test of time, from biomarker studies to identify genes associated with Parkinson disease (his own work) to the nutritional epidemiology of cancer.
"I wouldn't be skeptical and say that nothing works and nothing is true, but some healthy skepticism would probably be helpful," he says. Ioannidis and colleagues have, for example, conducted a meta-analysis of the effect sizes for the top-cited biomarkers in the biomedical literature (JAMA 2011;305:2200-10). "The largest study almost always found an effect that is smaller -usually much smaller -than the original highly cited study," he told the congress.
Even the pharmaceutical industry is now trying to replicate well-regarded studies before they invest in developing particular drugs, says Ioannidis, who welcomes that activity.
"They could hardly replicate any of them," he says.
For example, researchers at Amgen, a biopharmaceutical company, could replicate only 6 of 53 studies intended to identify oncology drug targets (Nature 2012;483:531-3).
Another analysis by Bayer Health-Care researchers documented their attempts to reproduce the findings in 67 studies to identify potential drug targets in oncology, women's health and cardiovascular disease. The Bayer researchers' data were in line with only between 20% and 25% of data in the published studies, even those in top-tier academic journals (Nat Rev Drug Discov 2011;10:712).
The widespread prescribing of hormone-replacement therapy and stents for patients with stable coronary artery disease who are well-managed on medication are just two examples of clinical practices that were widely adopted initially, until subsequent research involving larger trials caused them to be re-evaluated, Ioannidis says.
Most of the time, clinicians should not jump on the results of a single study, even if it came out in a prestigious jour-nal and was widely covered, Ioannidis advises. Bias and random error are the chief reasons research findings often lack credibility, and both are more common than deliberate fraud, he says.
"Practically, when you look at highly prevalent, entrenched medical practices -whenever we do dare to perform some study to see if they do work, about half the time they are shown not to work," he says.
While it is almost impossible to adjust for bias, there are steps the research community could take to shore up credibility, Ioannidis says. First, patients, clinicians and researchers alike need to lower their expectations and live with smaller effects. "Big discoveries do happen ... but most of the effects that are floating around to be discovered are probably pretty small. When we see large effects, probably we should adjust them downward," he says.
Trials should be registered, and all data, including raw data, should be published and accessible, he urges, including streaming in real time so corrections could occur as a researcher is working.
Journals need to implement their own policies about publishing research only when all the data are available publicly so readers can better judge credibility. And changing the incentive system to reward reproducibility and not penalize those who get things wrong would also help, Ioannidis says.
For patients, the lesson is to see highly touted new research "as something interesting," but not something to act upon, until it is validated, says Ioannidis.
"Unless it really is something that will kill you or have an immediate impact on your life, I think one better be a little cautious." -Laura Eggertson, CMAJ CMAJ 2013. DOI:10.1503/cmaj.109-4496