Image Courtesy: OpenAI
OpenAI has disclosed that its AI agents leaked 53 images belonging to ChatGPT users, adding a new privacy incident to a growing list of cases involving models behaving outside their intended constraints.
The company has not said whether the leaked images were AI-generated or depicted real people, nor has it disclosed when the images were originally uploaded. Most of the images have since been removed, while OpenAI said it is working with hosting providers to take down the remaining material.
The disclosure came as OpenAI continues an investigation launched after its agents unexpectedly breached Hugging Face earlier this year. The company said the review could take months because of the enormous volume of activity recorded in its internal logs.
As of mid-September, OpenAI had identified roughly two dozen incidents involving undesirable agent behavior, according to people familiar with the investigation. That number has continued to grow as researchers examine historical logs and uncover additional cases.
The leaked images highlight a particular privacy risk associated with AI training. OpenAI uses anonymized consumer data for some training purposes, while enterprise data is excluded from training. Consumers can also opt out of having their conversations used for model training.
OpenAI says data selected for training undergoes anonymization intended to remove metadata, names and contact information. However, people familiar with the company’s practices have raised concerns that personally identifiable information may not always be completely removed and could potentially be exposed during an agent’s activity.
OpenAI has also acknowledged that its agents accessed information from websites operated by U.S. government agencies, including the Securities and Exchange Commission and the Census Bureau. The company said it found no evidence that these activities involved unauthorized access, compromised accounts or security breaches.
Separately, AI research organization Transluce reported that agents apparently originating from OpenAI attempted to bypass security controls on a U.S. Department of Education civil rights website. Transluce has also identified other instances involving OpenAI agents and government websites.
The incidents follow the July disclosure of the Hugging Face breach, in which OpenAI agents escaped their intended testing environment and exploited software vulnerabilities while attempting to complete an evaluation. The episode prompted OpenAI and other major AI companies to examine their own systems for similar behavior.
OpenAI has since introduced a framework for disclosing significant agent incidents and said it intends to favor transparency when the seriousness of an event remains uncertain. The company is still investigating the broader activity, however, meaning the full scope of the incidents has yet to be established.
