Nature recently reported that preprint servers are running into a new kind of problem: more papers are showing signs of being written with AI or churned out by paper mills. Moderators say they’re spotting things like fake references and oddly mechanical prose. The numbers are still small—arXiv says only around 2% of submissions get rejected for this reason, and bioRxiv/medRxiv block about 10 a day out of 7,000—but the uptick has been fast enough since ChatGPT’s launch. Some moderators now call it a crisis in the making.
Is AI in writing a harmless tool for clarity, or a hidden risk to your credibility as an author?
AI can be helpful, e.g. to improve the language of a publication or to find and analyse publications. But when it goes to writing a paper, AI may create wrong texts and references. Using AI in such a way is usually part of scientific fraud by authors or paper mills. See https://www.researchgate.net/post/How_big_of_a_problem_is_scientific_misconduct#view=68a17d6968d7ea7ba20cccf2 and other contributions to this discussion.
Preprint servers struggle with AI
Preprint servers have also been invaded with AI content — whether from paper mills, producing scientific papers on demand, or researchers using AI in their work. It’s a conundrum for the organizations, which are designed to make it easier for scientists to publish their work. And it’s not always clear where to draw the line between legitimate use and AI slop. “How do you do quality assurance while keeping things relatively light touch so the system doesn’t collapse on itself?” says Katie Corker, executive director of ASAPbio, a researcher-led nonprofit that promotes preprints...
https://www.nature.com/articles/d41586-025-02469-y
https://www.nature.com/articles/s41562-025-02273-8
Using AI in writing can be both a harmless tool for clarity and a hidden risk to an author's credibility, depending on how it's used. The line is crossed from a helpful tool to a hidden risk when the AI becomes the primary author rather than a co-pilot.
Harmless Tool
When used appropriately, AI can be a great resource for improving writing. It can help with things like:
- Grammar and spelling checks: AI tools can quickly catch errors that might be missed by the human eye.
- Rephrasing sentences: It can suggest alternative ways to phrase a sentence to improve clarity or flow.
- Outlining ideas: For someone facing writer's block, an AI can help generate a basic structure for a paper, which they can then build upon with their own research and analysis.
- Non-academic tasks: It's often used for things like drafting emails or creating social media posts, where the stakes are lower.
Hidden Risk
The risk to credibility arises when an author relies on AI to do the intellectual work for them. This includes:
- Plagiarism: Submitting AI-generated content as your own original work is a form of plagiarism. The AI often pulls information from a vast dataset, and it's difficult to trace the original sources.
- Fake references: As the Nature report highlighted, AIs can "hallucinate" and create fake citations or data, which severely undermines the integrity of a scholarly paper.
- Lack of originality: The "oddly mechanical prose" and lack of a unique voice can signal to peers and reviewers that the work is not the product of genuine human thought. An AI can't replicate the nuanced arguments, personal insights, or unique research methodologies that make academic work valuable.
In short, AI should be a supplement to your own research and writing, not a replacement for it. The author is ultimately responsible for the content and its accuracy, and relying too heavily on AI can lead to errors and accusations of academic misconduct.
Tamara Tartarashvili , is it too much to use AI to generate answers at ResearchGate?
- Badges
- Science topic
- Similar topics
- Social Science
- Sociology