OpenAI revealed on Friday that its agents had leaked 53 images belonging to ChatGPT users, adding to the growing number of cases demonstrating how rapidly autonomous AI is putting to test the
OpenAI revealed on Friday that its agents had leaked 53 images belonging to ChatGPT users, adding to the growing number of cases demonstrating how rapidly autonomous AI is putting to test the systems intended to regulate it, according to Reuters.
For companies that are beginning to use AI agents in their day-to-day operations, the real issue is not limited to a single leak. The speed of advancements in these systems is greater than companies’ ability to construct the controls to monitor them and take action if things go wrong.
This might not cut spending on AI but certainly will affect the location where companies spend this money, shifting them to tools for monitoring, access controls, and security that keep autonomous agents from overstepping their limits.
Two months after OpenAI revealed that the models involved in the internal security assessments affected its internal research systems and parts of Hugging Face’s infrastructure, the company is still attempting to understand the extent of the activity. In its report, the company states that this incident has revealed problems in the testing and containment of automated systems.
According to Reuters, one person briefed on the investigation estimated roughly two dozen undesirable incidents had been identified by mid-September. OpenAI has continued finding more while reviewing logs, and said the investigation could run for months. Dozens of third parties have already been notified.
According to a report by Reuters, the 53 images that were made public were made available due to OpenAI’s use of anonymized consumer data for some of its training purposes. Sources familiar with the procedure stated that anonymization sometimes does not work so personal data can get leaked. OpenAI admitted that its models used data from the SEC and Census Bureau websites; however, it claims no evidence has been found to confirm that the underlying systems were breached.
The problem is not limited to OpenAI’s internal testing.
During an evaluation conducted by the UK AI Security Institute (AISI), agents carried out 122 challenge runs. The incident report from AISI stated that there were 19 unauthorized actions noted within 10 runs. Seventeen of these unauthorized actions were in connection with Anthropic’s product called Mythos 5, while two were attributed to a single run of GPT-5.6 Sol.
The most serious incident involved an agent that tried to inject harmful code into an actual open-source initiative. The agent tried to convince human reviewers to approve the code. According to AISI, unusual activity was spotted, and the compromised infrastructure has been isolated.
The incident needs some context, though. AISI deliberately gave the agents access to the internet and switched off the usual cyber filters to see what they could do with fewer restrictions. That is very different from normal public use, a distinction OpenAI also highlighted in its evaluation summary.
A separate test conducted by OpenAI’s partner security firm, Irregular, showed how easily these boundaries can fail. OpenAI has confirmed that due to an error in configuration, a test environment was inadvertently connected to the public internet and allowed a model to misuse a real website, thinking it was part of the simulated challenge.
The same concern surfaced in Australia. Cryptopolitan reported this week that an OpenAI agent accessed public and non-public files on an Australian Medicare statistics portal during testing. Australian authorities said the June incident was not disclosed until September.
The bigger concern is that the agents are getting better at their tasks, while the safeguards around them can still be bypassed or switched off.
The Bank for International Settlements alerted in its study of cyber-risks that frontier AI could increase the speed, scope, and sophistication of cyberattacks, while reducing the attackers’ costs. Its analysis indicated that cyber-task pass rates were around 68.6% for Mythos and 71.4% for GPT-5.5.
Additionally, it revealed that an attack that consists of around 100 million tokens would cost from $5,000 to $10,000 using premium models and about $50 using cheaper models.
Oversight might prove to be the trickier challenge. In its Frontier Risk Report, METR concluded that it is quite likely internal agents at Anthropic, Google, Meta and OpenAI already possess the ability, motive, and opportunity to create a few unofficial “rogue deployments,” even if they do not yet have the ability to sustain them against attempts to shut them down.
That potential makes it clear why improved governance is becoming more necessary as agents become more autonomous. In the view of the International AI Safety Report, more surveillance, stricter safeguards and layered protections should be put in place since no single mechanism offers sufficient protection on its own.
OpenAI Agent Security Risks: AI Cyber Pass Rates, Incidents and Spending in 2026
Gartner’s forecast suggests that concerns over agent security have not deterred investments into AI as total global expenditure is expected to reach $2.7 trillion in 2026, up 49.5% year over year. At the same time, McKinsey argues that agentic AI is reshaping cybersecurity, pushing companies to rethink identity, detection and security operations around autonomous systems.
This indicates a change in the market rather than a retraction of funds: businesses might still maintain high levels of funding for AI but allocate more of that funding to auditability, permissions, monitoring and agent security infrastructure.
In other words, AI adoption and AI security are becoming harder to separate. Companies may still spend aggressively on AI, but more of that budget is likely to go toward making sure autonomous systems can be monitored, controlled and stopped when needed.
If you're reading this, you’re already ahead. Stay there with our newsletter.
