Recently, some reviews seem to rely solely on AI-generated evaluations, providing little to no meaningful feedback while simply rejecting papers. In some cases, journal editors do not thoroughly verify these AI-generated reviews and merely pass them on to the authors. How can this issue be addressed?
Dear Isaac Obiri
Do you have concrete examples of journals using this practice? As far as I know there is no journal/publisher that implemented this AI peer review approach. As a matter of fact looking at for example Article Artificial intelligence to support publishing and peer revie...
which describes some features that might be useful it is also stated clearly "Artificial Intelligence does not seem yet capable of performing peer review for manuscripts submitted to academic journals."So it would be bad practice if there are journals/publishers that would use AI in the way you described.
Best regards.
El uso de IA en revisiones académicas plantea riesgos cuando no se supervisa adecuadamente, generando evaluaciones superficiales y sin retroalimentación útil para los autores.
Para abordar este problema, es crucial que los editores utilicen controles de calidad, verificando las revisiones generadas por IA antes de enviarlas.
La IA debe ser un complemento y no un sustituto, utilizada para tareas objetivas como detectar errores técnicos, mientras los revisores humanos evalúan aspectos críticos y contextuales.
La transparencia en los procesos es esencial, informando a los autores sobre el uso de IA y diferenciando sus aportes de los revisores humanos.
Además, se deben realizar auditorías periódicas para evaluar las herramientas de IA y promover estándares éticos que prioricen la calidad y la responsabilidad editorial.
Written by AI, reviewed by AI, and published by AI - the human editor as the ultimate gatekeeper in publication ethics
An exercise in AI driven publishing...
"It would be a human editor (and editorial assistants, including AI-based tools, that the editor has available) who could recognize (and as such potentially penalize or minimize) the AI dependency of a manuscript, enable its fair and rigorous peer review (even if the reviewers covertly use AI to write their reports), and make constructive and insightful editorial decisions based on the balance of the reviews (AI- or human-produced). In making an initial editorial assessment before proceeding to engage peer reviewers, the editor could examine the AI-based content of a manuscript and whether declarations made on such by the authors are in line with journal guidelines. The editor could also ensure that reviews that are eventually used in making an editorial decision post-peer review are authentically crafted by human reviewers with true expertise to navigate and interrogate the scientific and technical intricacies of the manuscript content. The exercise above thus reaffirmed the notion of the editor as the ultimate gatekeeper of ethical oversight in academic publishing. While the use of AI in writing is allowed by many journals and publishers (with proper declarations by the authors), there are, to this author’s knowledge, thus far no explicit guidelines for review reports that could also be crafted with AI. However, some reviews are indeed already being produced by AI, and such instances are only going to increase in extent and frequency. If AI-based academic publication is a growing trend that can neither be avoided nor reversed, please at least let the editors be human."
https://www.researchgate.net/publication/383663749_Written_by_AI_reviewed_by_AI_and_published_by_AI_-_the_human_editor_as_the_ultimate_gatekeeper_in_publication_ethics [accessed Mar 14 2025].
Researchers ‘polarised’ over use of AI in peer review
Views becoming more entrenched on both sides, with roughly equal numbers positive and negative about new technologies, poll finds...
"But publishers must fund ways to “reconcile” the two opposing viewpoints, the publisher added.
A solution could be developing tools that can operate within peer review software, it said, which could support reviewers without positing security or integrity risks.
Publishers should also be more explicit and transparent about why chatbots “are not suitable tools for fully authoring peer review reports”, IOP said.
“These findings highlight the need for clearer community standards and transparency around the use of generative AI in scholarly publishing. As the technology continues to evolve, so too must the frameworks that support ethical and trustworthy peer review,” Laura Feetham-Walker, reviewer engagement manager at IOP and lead author of the study, said."
https://www.timeshighereducation.com/news/researchers-polarised-over-use-ai-peer-review
- Badges
- Science topic
- Similar topics
- Engineering
- Materials Engineering
- Powders