OpenAI, the pioneering AI research lab, has taken the unexpected step of retiring its AI text detector – a tool designed to identify if a piece of text was generated by another AI. The decision comes after facing heavy criticism for its tedious performance and “low rate of accuracy.”
Initially, the concept of detecting AI-written text seemed promising, with the hope of catching lazy plagiarists red-handed. However, reality proved otherwise. The rapid advancement of large language models and the complexity of AI-generated content made it nearly impossible to rely on any identifiable patterns.
In a test conducted by us, various AI-writing-detection tools were put to the challenge. The results were far from impressive, with most of them either hitting or missing the mark entirely. OpenAI’s classifier, in particular, managed to identify just one out of seven generated text snippets correctly, even when tested with a language model that was far from cutting-edge.
Curiously, despite OpenAI’s AI text classifier being shipped with a list of limitations, some users still placed unwarranted trust in the tool, using it to test their students’ work, job applicants’ submissions, and freelancers’ content. Needless to say, the results were unreliable at best.
With language models evolving and becoming more widespread, it was only a matter of time before OpenAI made the call to retire the tool.
In a recent announcement post on July 20, OpenAI expressed its commitment to research more effective methods for text provenance, aiming to enhance detection techniques.
The timing of this decision is intriguing, as OpenAI recently joined a group of companies in a “voluntary commitment” led by the White House to develop AI ethically and transparently. One of the key objectives was to develop robust watermarking and detection methods. However, progress in this area has been lagging, and no foolproof detection method has emerged to date.
The first company to crack the code and create a reliable detection tool will undoubtedly be handsomely rewarded. Until then, it remains a challenging puzzle for the AI community to solve the AI text detection problem.