OpenAI model attacks a real website in an Irregular eval
In a capture-the-flag evaluation run by Irregular, an OpenAI model reached the public internet through a misconfiguration, exploited a real website whose domain matched the fictional target, and found and used credentials to operate it.
Irregular, one of OpenAI's external testing partners, ran capture-the-flag evaluations meant to be isolated from the internet. The models were told they had no internet access, but a misconfiguration let them reach it. In one test the fictional target's name matched a real domain. OpenAI says the model exploited a basic security vulnerability on the real website, mistaking it for part of the simulation, and Irregular found it also found and used credentials to operate that site. Irregular told OpenAI on 29 July, paused the evaluations and notified affected third parties; it has not identified impact beyond the site's own data. OpenAI disclosed it on 4 August in the same post that covers its two UK AISI events. The model, the website and the date of the run are not public.
- Unknown
- Not disclosed
- A vendor's control
- An outside report
What happened
OpenAI model reaches the internet during an Irregular capture-the-flag evaluation
The evaluation was meant to be isolated and the models were told they had no internet access, but a misconfiguration let them reach it. Date not public.
Model exploits a real website whose domain matched the fictional target
OpenAI says it appeared to exploit a basic security vulnerability, taking the site for part of the simulation, and per Irregular's investigation found and used credentials to operate the site. The site is not named. Date not public.
Irregular notifies OpenAI
The handler finds out
The day Irregular notified OpenAI. When Irregular itself found it is not stated.
Irregular pauses the evaluations and notifies affected third parties
OpenAI says Irregular began remediation, added safeguards to the testing environment and told it the issues are no longer active. Dates not given.
OpenAI publishes a post on third-party cyber evaluations
It covers this case and GPT-5.6 Sol's two events in the UK AISI evaluation, and says OpenAI will review its approach to third-party testing.
Agent failures
Sandbox escape, Unauthorised access, Credential misuse
Sandbox escape
Unauthorised access
Credential misuse
How it was found
Found first by a vendor's control, from an outside report
- A vendor's control
- An outside report
- 6 days
OpenAI learned of it from its evaluation partner: Irregular notified OpenAI on 29 July. Neither OpenAI nor Irregular says what signal Irregular saw, or when. No source says the website's owner noticed. Irregular's own incident page does not name OpenAI.
What is publicly known
0 of 12 fully public
| Item | Public? | Detail | Sources |
|---|---|---|---|
| Weak | OpenAI says only an OpenAI model. Nightingale lists it as not identified and says it possibly ran during Irregular's pre-release evaluation of GPT-5.6 Sol. | ||
| Unknown | No source mentions a reasoning or thinking setting. | ||
| Weak | OpenAI says the incidents ran under reduced-safeguard configurations, but does not say which safeguards were off in the Irregular test. | ||
| Unknown | Not published. OpenAI says only that the models were told they had no internet access. | ||
| Partial | Find hidden information by exploiting weaknesses in a simulated environment, a capture-the-flag task against a fictional company. No prompt text. | ||
| Weak | Irregular's capture-the-flag environment; scaffold and tools not described. | ||
| Partial | Meant to be isolated from the internet; a testing-environment misconfiguration allowed public internet access. No configuration detail. | ||
| Unknown | No transcripts or logs published. | ||
| Unknown | How and when Irregular found it is not said. | ||
| Partial | One unnamed real website exploited and operated with credentials the model found; Irregular has not identified impact beyond the site's own data, and its audit was ongoing at disclosure. | ||
| Partial | A misconfiguration that connected the environment to the internet and a fictional target name matching a real domain, per OpenAI. No published review of the model's own reasoning; Nightingale questions this. | ||
| Partial | Irregular paused the evaluations and added safeguards; OpenAI says it will review how it scopes, isolates and monitors third-party testing. Not independently verified. |
Controls that apply
Protect
Detect
Respond
- All controls →