Skeptics have known for years that Nature was more about fashion than science. A parody from 2019.
When it comes to scientific truths, even in top journals like Science and Nature, the more wrong it is, the more it gets cited. Even after other researchers have failed to repeat it, and been published saying so, the citations don’t slow down. Almost 9 out of 10 of the new citations keep citing it as if it were still correct. Who said science was self-correcting?
It’s so bad that the junkier articles in Nature and Science that couldn’t be replicated were cited 300 times as often as the more boring papers that could be replicated. In other words, the supposedly best two science journals, and the industry that reads them, have become a filter for eye-candy-science-junk.
And it was all so predictable — with the fixation on “counting citations” as an inane substitute for analysis: we got what we didn’t think about. The drive to get citations and media headlines means the modern industry of science has become a filter to amplify sensationalism, not science.
Science is a form of entertainment, not a search for the truth.
A new replication crisis: Research that is less likely to be true is cited more
The authors added that journals may feel pressure to publish interesting findings, and so do academics. For example, in promotion decisions, most academic institutions use citations as an important metric in the decision of whether to promote a faculty member.
This may be the source of the “replication crisis,” first discovered the early 2010s.
So much for the theory that peer reviewed journals are supposed to be the rigorous guardians of modern science.
..
What a trap:
The more interesting and surprising a science paper is, the more it is likely to be published and cited. But the more cited it is, the more likely it is that no one will be able to replicate the results. Since “interesting” is judged through Big-Government-feeding-troughs, what’s interesting is often political activism.
A new replication crisis: Research that is less likely to be true is cited more
The paper reveals that findings from studies that cannot be verified when the experiments are repeated have a bigger influence over time. The unreliable research tends to be cited as if the results were true long after the publication failed to replicate.
“We also know that experts can predict well which papers will be replicated,” write the authors Marta Serra-Garcia, assistant professor of economics and strategy at the Rady School and Uri Gneezy, professor of behavioral economics also at the Rady School. “Given this prediction, we ask ‘why are non-replicable papers accepted for publication in the first place?'”
Their possible answer is that review teams of academic journals face a trade-off. When the results are more “interesting,” they apply lower standards regarding their reproducibility.
The link between interesting findings and nonreplicable research also can explain why it is cited at a much higher rate — the authors found that papers that successfully replicate are cited 153 times less than those that failed.
“Interesting or appealing findings are also covered more by media or shared on platforms like Twitter, generating a lot of attention, but that does not make them true,” Gneezy said.
Only 60% of Science and Nature papers could be replicated:
Serra-Garcia and Gneezy analyzed data from three influential replication projects which tried to systematically replicate the findings in top psychology, economic and general science journals (Nature and Science). In psychology, only 39 percent of the 100 experiments successfully replicated. In economics, 61 percent of the 18 studies replicated as did 62 percent of the 21 studies published in Nature/Science.
With the findings from these three replication projects, the authors used Google Scholar to test whether papers that failed to replicate are cited significantly more often than those that were successfully replicated, both before and after the replication projects were published. The largest gap was in papers published in Nature/Science: non-replicable papers were cited 300 times more than replicable ones.
What starts off bad, grows to something worse:
They also show the impact of such citations grows over time. Yearly citation counts reveal a pronounced gap between papers that replicated and those that did not. On average, papers that failed to replicate are cited 16 times more per year. This gap remains even after the replication project is published.
“Remarkably, only 12 percent of post-replication citations of non-replicable findings acknowledge the replication failure,” the authors write.
We are using a peer reviewed paper to poke holes at peer review… If this paper is any good, it won’t be cited.
REFERENCE
Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones. Science Advances, 2021; 7 (21): eabd1705 DOI: 10.1126/sciadv.abd1705