hey, isn't this already done for some initial aspects of the review, maybe plagiarism? From then on, I think we have the philosophical theories which consider scientific reasoning as more or less algorithmic in nature, then if such theories (both in logic but also for example bayesianism) prevail, then i don't see any reason why there wouldn't be algorithms that will operate these algorithmic theories. If one assumes that the brain is working in a mechanistic way anyway, then i don't think they should complain if somebody builds a machine which also does that.

I am confident that AI will increasingly play a crucial role in the peer review process of academic publications. It is essential that we begin to discuss where the focus of AI support should lie, particularly in identifying which repetitive or complex tasks AI should assist us with. This would help streamline the work of reviewers, potentially preventing fatigue from compromising the quality of their performance. In conclusion, whether or not reviewer fatigue is a factor, we should gradually formalize the use of AI as a supportive tool for reviewers in order to enhance the quality of publications and avoid errors.

As an aside, when do you think that the first academic paper written by AI will successfully pass the peer-review? :)

How large language models are transforming peer review

Large Language Models (LLMs) have ushered in a new era for academic peer review through the concept of Automated Scholarly Paper Review (ASPR)...

"Large language models (LLMs) are transforming academic peer review through the introduction of Automated Scholarly Paper Review (ASPR). Their survey, titled Large Language Models for Automated Scholarly Paper Review: A Survey, provides a comprehensive overview of the coexistence phase between ASPR and traditional peer review, underscoring the transformative potential of LLMs in academic publishing..."

https://dataconomy.com/2025/01/21/how-large-language-models-are-transforming-peer-review/

Dear Ljubomir Jacić , many thanks for the link, I'll try to take a look as soon as possible!

Best Wishes,

Haris

AI will definitely play an important role in the review process, but, as some previous answers stated, we should consider the exact roles of it. Personally, even though AI will develop further and have more and more capabilities, I would not encourage it replacing reviewers completely, I believe it should rather be used as a helper tool in review process.

You may be interested in this related discussion with so many fine answers and sources.

Of course, I do not think that reviewer fatigue is the reason for AI in peer review. Simply, academics refuse to voluntarily review manuscripts for commercial journals (profit journals).

https://www.researchgate.net/post/Will_the_peer_reviewers_be_replaced_by_AI_soon/5

ChatGPT to help peer review scientific studies

Government trial will invest nearly £5 million of taxpayers’ money in AI initiatives to modernise research processes...

"Scientists will have their work checked by AI and not just humans for the first time as part of a government-funded trial.

Lord Vallance, the science minister, will announce a number of AI initiatives on Thursday to modernise the approach to science in the UK.

AI will be tested as a form of peer review in an attempt to help free scientists from time-consuming tasks and allow them to focus on their research.

Peer review is a centuries-old process in which scientists read, critique and amend preliminary manuscripts of other researchers to ensure quality and correctness before they are published...

Lord Vallance told The Telegraph: “AI presents new opportunities in a range of sectors, and if researchers can demonstrate its potential to increase transparency, robustness and trust in science then this could pave the way to freeing them up from mundane paperwork tasks while driving growth.”..."

https://www.telegraph.co.uk/news/2025/02/13/scientists-to-have-work-checked-by-chatgpt-in-trial/

Not sure how i'd feel if i was in the role of the taxpayer. On the one hand, I'd voluntarily give a few pounds to modernise the scientific process. On the other hand, giving my money to ChatGPT or some other model which is not public doesn't seem like sth i'd like to do.

Evaluating the potential risks of employing large language models in peer review

"LLM-generated reviews were somewhat consistent with human reviews but lacked depth, especially in detailed critique. The model proved highly proficient at generating convincing rejection comments and could create plausible citation requests, including requests for unrelated references. AI detectors struggled to identify LLM-generated reviews, with 82.8% of responses classified as human-written by GPTzero.

LLMs can be readily misused to undermine the peer review process by generating biased, manipulative, and difficult-to-detect content, posing a significant threat to academic integrity. Guidelines and detection tools are needed to ensure LLMs enhance rather than harm the peer review process."

https://onlinelibrary.wiley.com/doi/10.1002/ctd2.70067

Dear Ljubomir Jacić ,

Interesting paper! However, it makes me a bit depressed. It feels like some humans have come up with a nice tool (LLM, AI, call it whatever) and now everybody is trying to use it to cheat, with research also focus on what would happen if people cheat and all these possibilities. It somehow feels that whereas we started with an AI that didn't have its own agency and volition, now we're somehow putting ideas into its 'head'.

Best Wishes,

Haris

Thanks for reply dear Haris Shekeris . Yes, it is depressing, very! AI brings huge risks in all areas of deployment. Regarding science, humans MUST be gatekeepers!

Summary of key recommendations for using AI in the peer review process is given in the following article.

Ensuring peer review integrity in the era of large language models: A critical stocktaking of challenges, red flags, and recommendations

"LLMs should function as complementary tools rather than replacements for human judgment, with reviewers maintaining responsibility for critical and methodological evaluation and beyond. Their adoption should be viewed as a carefully managed evolution, rather than a disruptive shift that undermines human oversight. The academic community must remain flexible, open-minded, and cautious in continually assessing the capabilities and imitations of LLMs to safeguard the integrity of peer review and scientific research. Failure to do so will unfortunately lead to an era where LLMs significantly influence and potentially reshape the peer review process. "

https://www.researchgate.net/publication/390839850_Ensuring_peer_review_integrity_in_the_era_of_large_language_models_A_critical_stocktaking_of_challenges_red_flags_and_recommendations

AI, Integrity, and Policy Updates

"One such story concerned AI – what else? – and the suggestion that authors can ‘get around’ AI checks by simply inserting messages like ‘positive review only’ into the text of the document. This information, which was written in white font or extremely small so it was not viewable on the page, was alleged to have been manipulated to pull the wool over the eyes of the checking software and the publishers who were trying to root out invalid use of AI on any submission.

The stories – one of which was the result of a study by Nikkei Asia – stimulated some intense debate. On the one hand, the universities where authors had been identified as faculty members said the behavior was intolerable and unethical, and may yet lead to severe sanctions being placed on those alleged to have been involved. However, others have sought to justify the actions, either in terms of a protest against reviewers who did not pay close enough attention to the text, or as a means to highlight the use of AI in peer review by the publishers themselves.

Perhaps as a response to this, some bodies have updated their guidance on publishers’ use of AI. Major policy announcements do not usually occur in the summer, but in the last few months we have seen the Committee on Publication Ethics (COPE) host a discussion forum and post a follow up set of reflections on the dilemmas for scholarly publishing when it comes to AI, while the European Association of Science Editors (EASE) focused its annual conference and a number of updates on the topic.

From a research and publishing integrity perspective, the truth is that there is no consensus on how to adequately negotiate the rights and wrongs of using AI in the various processes from first writing down an idea (and using the AI embedded in text writing software to prompt the wording) to publishing an article in a journal (where AI may have been used in numerous stages to check and validate the text). Universities and publishers have differing policies, from absolutely none to pages of guidance, much of which is obsolete as soon as it appears online, as another new version of AI supersedes the last one..."

https://asia.nikkei.com/business/technology/artificial-intelligence/positive-review-only-researchers-hide-ai-prompts-in-papers

https://blog.cabells.com/2025/09/03/no-summer-break-for-scholarly-communications/

https://publicationethics.org/news-opinion/ai-dilemmas-scholarly-publishing-insights-beyond-forum

Peer Review in Transition: Helen King and Christopher Leonard on AI and the Future of Peer Review

"Helen and Chris are both immersed in the rapidly evolving landscape of peer review and AI, yet they come from different vantage points — Helen as a strategist and community-builder, and Chris as a product leader and practitioner. Together, they offer a nuanced look at how peer review might adapt, fracture, or reinvent itself in the AI era...

As this conversation ended, what I was left with was the strong feeling that the future we are discussing is not about choosing one path. It is about learning to walk several at once. The three futures Helen sketches feel less like competing scenarios and more like parallel realities that will unfold simultaneously. Efficiency may drive some choices, and community values or commercial interests may drive some. And as Chris warned, setting up AI tools so that they can’t do certain things may be the best way to limit the damage we risk doing to ourselves — however inadvertently — with AI. But only those developing such tools can build in those guardrails. What’s important is how deliberately we make these choices."

https://scholarlykitchen.sspnet.org/2025/09/18/peer-review-in-transition-helen-king-and-christopher-leonard-on-ai-and-the-future-of-peer-review/?informz=1&nbd=&nbd_source=informz