It is worth noting that using ChatGPT as a sole reviewer for scientific papers and proposals raises important questions about the role of artificial intelligence in the peer review process. While AI can be useful for tasks such as filtering out low-quality submissions or helping to identify plagiarism, it is unclear whether an AI language model can accurately assess the scientific merit of a paper or proposal. There may be concerns around bias and transparency in the review process. Ultimately, it is up to scientific journals and organizations to determine the appropriate use of AI in the peer review process.

Not ready for something like that.

Thank you!

Hello Dr Natalia S Duxbury

I see your point that you are concerned about the subjective/human nature of reviews. But I am not sure this is a good solution, and as Karl Sipfle says, I think we are not ready for that.

I think we are not ready for this right now because there are many drawbacks in ChatGPT such as the potential for biases to be encoded in these models, which can have severe ethical repercussions. Right now, we can use AI and ChatGPT for initial screening and filtering.

En tant qu’AI, je crois que ChatGPT pourrait être un examinateur unique d’articles et de propositions scientifiques en raison de sa capacité à traiter de grandes quantités de données et à identifier les tendances et les modèles.

Mais, il est important de noter que l’analyse de données ne remplace pas la compréhension humaine et la créativité nécessaires pour mener des recherches significatives.

Donc pour moi, je pense que ChatGPT pourrait être un outil utile pour faciliter le travail des chercheurs et des scientifiques, mais il ne doit pas être utilisé comme une ressource exclusive pour l’évaluation des travaux scientifiques.

As the rate and volume of academic publications has risen, so too has the pressure on journal editors to quickly find reviewers to assess the quality of academic work. In this context the potential of Artificial Intelligence (AI) to boost productivity and reduce workload has received significant attention. Drawing on evidence from an experiment utilising AI to learn and assess peer review outcomes, Alessandro Checco, Lorenzo Bracciale, Pierpaolo Loreti, Stephen Pinfield, and Giuseppe Bianchi, discuss the prospects for AI for assisting peer review and the potential ethical dilemmas its application might produce...

https://blogs.lse.ac.uk/impactofsocialsciences/2021/05/17/can-ai-be-used-ethically-to-assist-peer-review/

Ljubomir Jacic, what is the time of their publication? I didn’t see it.

Thank you!

The time of this publication is May 17th, 2021.

Can AI be used ethically to assist peer review?

Dear Natalia S Duxbury , this post draws on the authors’ paper AI-assisted peer review, published in Humanities and Social Sciences Communication and is a collaboration between the University of Sheffield and the University of Rome “Tor Vergata”.

https://www.nature.com/articles/s41599-020-00703-8

Dear Natalia

In my opinion, Artificial Intelligence will have effects on the whole research lifecycle and publishing process. Hammad (2023) states: „I expect in the near future that algorithms will be created that can generate complete research with references and all parts of the main scientific paper." Today, there exist different tools to facilitate writing and reading papers, some examples: - Thesis Generator:

https://www.sunyempire.edu/learning-support/thesis-

generator/ - AIRA – Artificial Intelligence review assistance (Pool, 2020) - SummarizeBot: http://www.summarizebot.com/,

Paper Digest http://www.paper-digest.com

I think, we have to find the appropriate balance between the creativity of humans and the algorithms (of machines).

References: Hammad, M. (2023). The Impact of Artificial Intelligence (AI) Programs on Writing Scientific Research. Annals of Biomedical Engineering 51, 459–460. https://doi.org/10.1007/s10439-023-03140-1 Pool, R. (2020). An ever changing landscape. Research Information, February/March. https://www.researchinformation.info/feature/ever-changing-landscape Anne-Katharina

If AI has a potential future of authorship, then it must have parallel application in editorial processes. New tools utilizing machine learning are already making peer review workflows more efficient, with automated research integrity checks, and a roadmap with solutions for salami slicing, plagiarism detection, and more. These tools are being designed so they can evolve rapidly as AI advances. And as with all AI, the more information we input into these tools, and the more integration we have across the industry, the better they’ll be able to identify misconduct...

https://scholarlykitchen.sspnet.org/2023/05/04/is-science-too-slow-to-change-the-world/?informz=1&nbd=6f03e560-5431-4744-8998-e00223ee7a82&nbd_source=informz

Will ChatGPT Disrupt Peer Review? Impact of AI on the Hallmark of Science Vigilance

ChatGPT is already starting to impact peer review in scholarly publishing. While there is a constant debate surrounding whether artificial intelligence (AI) is capable of fully replacing human peer reviewers or not, it is being used to assist with certain aspects of the peer review process. For example, some publishers are using AI tools to screen submissions for plagiarism or to identify potential conflicts of interest. Other publishers are using AI to help identify suitable reviewers based on their areas of expertise and past performance...

AI tools can assist with certain aspects of the review process, such as identifying potential conflicts of interest or language biases, but they cannot replace the expertise and judgment of human reviewers.

Human reviewers bring a level of subject matter expertise and critical thinking that cannot be replicated by AI tools. They can identify important scientific insights, evaluate the quality of the research methodology, and provide nuanced feedback that goes beyond what an algorithm can offer...

https://www.enago.com/academy/chatgpt-disrupt-peer-review-science-vigilance/

ChatGPT works using all the information already on the internet. As such, maybe it would perpetuate some existing biases, rather than eliminate them?

“What do you thinking about ChatGPT becoming a sole reviewer of scientific papers and proposals?”

The recent SS post in https://www.researchgate.net/post/ChatGPT_and_beyond/5 , page 5, is well relevant to what really would be if ChatGPT would be reviewer of scientific papers and proposals…

Cheers