OpenAI has reportedly developed a highly effective ChatGPT detection tool, achieving an accuracy rate of 99.9%. Despite its success, the AI company is hesitant to release the tool due to potential user backlash and broader ecosystem impacts. The Wall Street Journal revealed that this detection tool works by embedding a watermark pattern within text generated by ChatGPT, making it identifiable by OpenAI. CEO Sam Altman supports the project but is cautious about its implementation.
Since its November 2022 launch, ChatGPT has presented serious obstacles for teachers attempting to forbid its use in assignments. According to a Pew Research Center survey conducted in November of last year, 20% of teens who were aware of ChatGPT were using it for academic purposes. The accuracy of earlier AI detection systems, such as OpenAI’s, has been inconsistent; the company’s most recent effort had only 26% accuracy and a high rate of false positives.
For over a year, OpenAI has kept its advanced detection tool under wraps, concerned about driving users to competitors without watermarking. An internal 2023 test revealed that nearly 30% of respondents would reduce their ChatGPT usage if the anti-cheating tool was implemented. Additionally, employees worried about potential performance impacts were reassured after tests showed no negative effects on ChatGPT’s functionality. Despite these findings, OpenAI remains cautious.
An August 4 update to a May blog post confirmed the tool’s existence and ongoing consideration while OpenAI explores alternatives. The company has prioritized audio and visual detection methods due to higher associated risks with AI-generated images and sound. OpenAI also noted that the text watermarking tool could be vulnerable to global tampering, such as translation or rephrasing with other generative models. Furthermore, research suggests the tool might disproportionately affect non-native English speakers who rely on ChatGPT for writing assistance.
As OpenAI navigates these complexities, it remains uncertain whether the ChatGPT detection tool will ever be released, reflecting the company’s careful balancing of technological advancements and user experience considerations.