What will happen to public debate and our democracy if we can no longer trust research?
Fabricated ‘scientific publications’ could threaten the very foundations of our society.
Illustration: Getty images
It isn’t ‘fake news’ that scares me, it’s fake research. By this, I don’t mean plagiarism and the failure to provide correct citations, but rather completely fabricated ‘scientific publications’ that by their very nature and scale may threaten the foundations for society.
Whenever debates in the news or social media become unhinged and false information is suspected or the contours of a conspiracy theory starts to emerge, the natural inclination of journalists and moderate voices of reason is to refer to research. ‘But yes, the climate crisis is real.’ ‘No, Ukraine is not a fake state.’
What will happen to public debate and our democracy if we can no longer trust research? Unfortunately, this is becoming an increasingly serious problem. And there are (at least) two distinct, but somewhat interrelated causes: artificial intelligence (AI) and the pressure for open access (OA) publication.
To begin with the first. There has been much discussion about predatory ‘pirate’ journals and publishers. There has been far less focus on the so-called ‘paper mills’. While pirate journals and fake publishing houses deceive genuine researchers, the paper mills undermine research as a whole – and in doing so undermine the foundations of our informed society and (by extension) our democratic system. And this is happening on a massive scale.
In the same way as fake news articles are mass-produced and distributed, there are extremely professional actors, motivated by economic gain, who are generating enormous quantities of completely fake ‘scientific papers’. If the price is right, these actors also take on the task of getting the fake papers published in genuine scholarly journals, navigating through the well-established system of strict quality controls. There are cases where these actors have succeeded in establishing s ‘peer-review rings’, and thereby taken control of the entire peer-review process. There are now also many reports of hijacked special issues, where guest editors, authors and reviewers alike are all as fake as the papers they publish.
Until recently, paper mills were not a significant problem for serious journals. There was little likelihood that the articles they produced would get past ‘desktop’ checks at submission. But with AI, we are facing a new reality.
We are seeing consistently ‘better’ articles, complete with citations and references to ongoing debates in the relevant journal. The ‘authors’ have ORCID IDs, some of which belong to genuine researchers who have fallen victim to identity theft, and the cover letters are less and less obviously in standard template format.
The initial quality control for fake research in new submissions, which used to take up only seconds, has now become time-consuming and more difficult for editorial teams.
This is happening while journals simultaneously are under increasing pressure to publish more in order to remain financially viable.
This leads us to the second factor that is contributing to undermining the foundations of our society. The Plan S initiative, launched by cOAlition S in 2018 to encourage Open Access publishing is well intentioned but has resulted in some problems with unintended negative consequences. In an article published in Forskerforum (the journal of Forskerforbundet / the Norwegian Association of Researchers) on 13 February, Göran Erik Nilsson of the University of Oslo shared his fear that journals will either die or be forced to sell out to ‘mega-journal publishers’ in the wake of Plan S.
In response to the economic challenges generated by Plan S, publishers have established mega- journals such as PLOS, ACS Omega and BMJ Open. By speeding up production, limiting the peer-review process, and lowering requirements for authors to revise their manuscripts, these publishers hope that quantity will make up for the revenue lost by high-quality journals operating on the basis of either subscriptions or high author fees. There has been a sharp shift from quality to quantity, including among traditional subscription-based journals.
Editors are being instructed by publishers that they must accept more articles and publish more special issues. Whole editorial teams have resigned in protest including, most recently, the mass resignation from the editorial board of Wiley’s Journal of Economic Surveys. Maintaining quality for such a quantity of articles is impossible in a system that depends largely on voluntary efforts.
Research is based on trust and accountability. In scholarly publishing, peer review is the process by which other experts in the field read, comment on, and assess the quality of a scholarly paper anonymously. Peer review is the most important guarantee of quality, and also the most effective tool for detecting fraud. Until now, the gold standard has been double-anonymous reviewing, in which neither the author, nor the reviewer is aware of the other’s identity. Other peer-review models exist, however, where the process may be partially or completely open, and authors are under varying degrees of obligation to incorporate the feedback they receive.
A shared feature of all peer-review models, however, is that at least one independent expert allocates time to do the review. This work is done on a voluntary basis and provides very little benefit to the person doing it. Increasing the publication volume also requires increased voluntary effort.
There has long been talk of ‘reviewer fatigue’. Ensuring properly quality-assured papers is becoming more difficult by the day.
As an example, twenty years ago, my journal would contact an average of four people in order to obtain three peer-review reports. Back then, we were far down on the journal rankings. Now, we are one of the leading journals internationally within our field and regularly we have to send more than 20 requests in order to obtain two reports. Our situation is far from unique.
For many journals, therefore, more ‘flexible’ solutions, such as using fewer and more random peer reviewers, or publishing articles accompanied by an open invitation for anyone to submit an open assessment, have become a reality.
In open review processes editorial responsibility is being shifted to the readers, who must themselves assess the quality of the whole process before concluding whether the article itself meets scholarly standards. We can perhaps assume that a professor in the relevant field can competently do the job, time allowing, but what about a PhD student, or a postdoc under pressure to publish, or – a journalist?
Can we expect them to separate the wheat from the chaff?
It seems that the door is wide open for the paper mills with their ORCID-registered fake authors and peer reviews. Fake scientific papers about Covid-19, vaccines and the climate as well as fake analyses of security policy and other issues with the potential to stoke social tensions are all produced in vast quantities, and appear to be quality-assured genuine research. When these ‘contributions’ are brought into public discourse as a basis for knowledge, we are only a couple of scandals away from a serious problem of trust for the informed society.
I believe we can take it for granted that fake articles with the potential to cause such problems have already been published. A recent article in The Guardian lends weight to this belief. Both as a member of the research community and as a regular citizen of a democratic society, this keeps me awake at night.
Recently, a new Strategy for Scholarly Publishing post-2024 in Norway was launched by the Norwegian backers of cOAlition S. The goal of OA publication must apparently be achieved, it appears, despite the original plan proving both difficult and costly to implement. Quality costs. I look forward to hearing how cOAlition S envisage addressing the problems of a shortage of peer reviewers, over-worked editorial teams, and the emergence of paper mills and AI. Because we do have a problem with how research is produced and published – that’s one thing we can all agree on.
- Marit Moe-Pryce is the Managing Editor of the journal Security Dialoge
- This text was published in Norwegian by Khrono 23 February 2024
- Translation from Norwegian: Fidotext