May 30, 2024 | “Peer Reviews” And The Shrinking Trust Horizon

It’s now widely understood that science faces a “replication crisis” where attempts to re-do experiments frequently fail to yield the same results. In other words, a lot of cheating and/or incompetence is showing up in peer-reviewed journals these days, making it impossible for, say, doctors to know how to treat their patients or the rest of us to know what supplements to take, foods to eat, and exercise programs to choose.

But apparently,y it’s even worse than that. Check out the mess at a major scientific publisher:

Flood of Fake Science Forces Multiple Journal Closures tainted by fraud

(Wall Street Journal) – Fake studies have flooded the publishers of top scientific journals, leading to thousands of retractions and millions of dollars in lost revenue. The biggest hit has come to Wiley, a 217-year-old publisher based in Hoboken, N.J., which Tuesday will announce that it is closing 19 journals, some of which were infected by large-scale research fraud.

In the past two years, Wiley has retracted more than 11,300 papers that appeared compromised, according to a spokesperson, and closed four journals. It isn’t alone: At least two other publishers have retracted hundreds of suspect papers each. Several others have pulled smaller clusters of bad papers.

Although this large-scale fraud represents a small percentage of submissions to journals, it threatens the legitimacy of the nearly $30 billion academic publishing industry and the credibility of science as a whole.

Scientific papers typically include citations that acknowledge work that informed the research, but the suspect papers included lists of irrelevant references. Multiple papers included technical-sounding passages inserted midway through, what Bishop called an “AI gobbledygook sandwich.” Nearly identical contact emails in one cluster of studies were all registered to a university in China where few if any of the authors were based. It appeared that all came from the same source.

One of those tools, the “Problematic Paper Screener,” run by Guillaume Cabanac, a computer-science researcher who studies scholarly publishing at the Université Toulouse III-Paul Sabatier in France, scans the breadth of the published literature, some 130 million papers, looking for a range of red flags including “tortured phrases.”

Cabanac and his colleagues realized that researchers who wanted to avoid plagiarism detectors had swapped out key scientific terms for synonyms from automatic text generators, leading to comically misfit phrases. “Breast cancer” became “bosom peril”; “fluid dynamics” became “gooey stream”; “artificial intelligence” became “counterfeit consciousness.” The tool is publicly available.

Generative AI has just handed them a winning lottery ticket,” Eggleton of IOP Publishing said. “They can do it really cheap, at scale, and the detection methods are not where we need them to be. I can only see that challenge increasing.”

Science, it seems, has become an early victim of the deep-fake crime wave flowing from generative AI. But let’s not forget the other, potentially more serious problem of “bought and paid for science” where study results are determined not by the data but by the preference of the funding source. How many of today’s widely-prescribed drugs are based on fake results? Possibly many.

The takeaway: Science has joined the list of big systems that can’t be trusted. And we are increasingly on our own.

John Rubino May 30th, 2024

Posted In: John Rubino Substack

Next: Running Bulls Need No Thinking »