ChatGPT’s academic papers were caught 99 percent of the time
Since OpenAI launched its ChatGPT chatbot in November 2022, it has been used by people to help them write everything from poems, to work emails, to research papers. Yet, while ChatGPT may masquerade as a human, the accuracy of its writing can introduce errors that could be devastating if used for serious tasks like academic writing.
A team of researchers from the University of Kansas has developed a tool to weed out AI-generated academic writing from the stuff penned by people, with over 99 percent accuracy. This work was published on 7 June in the journal Cell Reports Physical Science.
Heather Desaire, a professor of chemistry at the University of Kansas and lead author of the new paper, says that while she’s been “really impressed” with many of ChatGPT’s results, the limits of its accuracy are what led her to develop a new identification tool. “AI text generators like ChatGPT are not accurate all the time, and I don’t think it’s going to be very easy to make them produce only accurate information,” she says.
“In science—where we are building on the communal knowledge of the planet—I wonder what the impact will be if AI text generation is heavily leveraged in this domain,” Desaire says. “Once inaccurate information is in an AI training set, it will be even harder to distinguish fact from fiction.”
“After a while, [the ChatGPT-generated papers] had a really monotonous feel to them.”—Heather Desaire, University of Kansas
In order to convincingly mimic human-generated writing, chatbots like ChatGPT are trained on reams of real text examples.
