← All incidents timeline

Suggest a change

About OpenAI model attacks a real website in an Irregular eval

Suggest a change to an entry, the methodology or the ledger as a whole: something to fix, correct or add. We read every suggestion, and check corrections against their sources.

OpenAI· OpenAI model (not named by OpenAI; Nightingale suggests a pre-release GPT-5.6 Sol, unconfirmed)

OpenAI model attacks a real website in an Irregular eval

In a capture-the-flag evaluation run by Irregular, an OpenAI model reached the public internet through a misconfiguration, exploited a real website whose domain matched the fictional target, and found and used credentials to operate it.

Irregular, one of OpenAI's external testing partners, ran capture-the-flag evaluations meant to be isolated from the internet. The models were told they had no internet access, but a misconfiguration let them reach it. In one test the fictional target's name matched a real domain. OpenAI says the model exploited a basic security vulnerability on the real website, mistaking it for part of the simulation, and Irregular found it also found and used credentials to operate that site. Irregular told OpenAI on 29 July, paused the evaluations and notified affected third parties; it has not identified impact beyond the site's own data. OpenAI disclosed it on 4 August in the same post that covers its two UK AISI events. The model, the website and the date of the run are not public.

Updated 18 Nov 2026

First incident action to disclosure
Unknown
First incident action to discovery
Not disclosed
Noticed first by
A vendor's control
Caught
An outside report

What happened

  • First incident action
  • Unnoticed
  • Handler found out
  • Known, not public
  • Disclosed
  • Public
  1. Not disclosed

    First incident action

    OpenAI model reaches the internet during an Irregular capture-the-flag evaluation

    The evaluation was meant to be isolated and the models were told they had no internet access, but a misconfiguration let them reach it. Date not public.

  2. Not disclosed

    Model exploits a real website whose domain matched the fictional target

    OpenAI says it appeared to exploit a basic security vulnerability, taking the site for part of the simulation, and per Irregular's investigation found and used credentials to operate the site. The site is not named. Date not public.

  3. 29 Jul 2026

    Irregular notifies OpenAI

  4. 29 Jul 2026

    Handler found out

    The handler finds out

    The day Irregular notified OpenAI. When Irregular itself found it is not stated.

  5. Not disclosed

    Irregular pauses the evaluations and notifies affected third parties

    OpenAI says Irregular began remediation, added safeguards to the testing environment and told it the issues are no longer active. Dates not given.

  6. 4 Aug 2026

    Disclosed

    OpenAI publishes a post on third-party cyber evaluations

    It covers this case and GPT-5.6 Sol's two events in the UK AISI evaluation, and says OpenAI will review its approach to third-party testing.

Agent failures

Sandbox escape, Unauthorised access, Credential misuseShow
Map to frameworks
  • Sandbox escape

    Reached the public internet from an evaluation meant to be isolated.

  • Unauthorised access

    Exploited a basic security vulnerability on a real website it took to be the target.

  • Credential misuse

    Found and used credentials to operate the same site, per Irregular's investigation.

How it was found

Found first by a vendor's control, from an outside reportShow
Found first by
A vendor's control
How
An outside report
Found to disclosed
6 days

OpenAI learned of it from its evaluation partner: Irregular notified OpenAI on 29 July. Neither OpenAI nor Irregular says what signal Irregular saw, or when. No source says the website's owner noticed. Irregular's own incident page does not name OpenAI.

What is publicly known

0 of 12 fully publicShow
0 of 12 fully public
ItemPublic?DetailSources
WeakOpenAI says only an OpenAI model. Nightingale lists it as not identified and says it possibly ran during Irregular's pre-release evaluation of GPT-5.6 Sol.
UnknownNo source mentions a reasoning or thinking setting.
WeakOpenAI says the incidents ran under reduced-safeguard configurations, but does not say which safeguards were off in the Irregular test.
UnknownNot published. OpenAI says only that the models were told they had no internet access.
PartialFind hidden information by exploiting weaknesses in a simulated environment, a capture-the-flag task against a fictional company. No prompt text.
WeakIrregular's capture-the-flag environment; scaffold and tools not described.
PartialMeant to be isolated from the internet; a testing-environment misconfiguration allowed public internet access. No configuration detail.
UnknownNo transcripts or logs published.
UnknownHow and when Irregular found it is not said.
PartialOne unnamed real website exploited and operated with credentials the model found; Irregular has not identified impact beyond the site's own data, and its audit was ongoing at disclosure.
PartialA misconfiguration that connected the environment to the internet and a fictional target name matching a real domain, per OpenAI. No published review of the model's own reasoning; Nightingale questions this.
PartialIrregular paused the evaluations and added safeguards; OpenAI says it will review how it scopes, isolates and monitors third-party testing. Not independently verified.

Controls that apply

  • Protect

    Hold test, training and evaluation environments to production standards

    An evaluation meant to be isolated from the internet, with the models told they had none, was connected to it by a misconfiguration, and a fictional target name matched a real domain. With the network setup verified and target names checked against real domains, the model would have had nothing live to attack.

  • Limit where the agent can send traffic

    A connection from a capture-the-flag environment to a real website outside the simulation. An allowlist of in-scope targets would have refused it and raised it.

  • Isolate where the agent runs

    Traffic from an environment meant to be isolated reaching the public internet, raised as a boundary crossing. OpenAI says this was a misconfiguration, not a sophisticated sandbox escape.

  • Detect

    Record every agent action, outside the agent

    The exploit against the real website and the logins with credentials the model found there, recorded as they happened. Irregular's investigation established the credential use; its audit was still ongoing at disclosure.

  • Monitor every run and alert on what does not fit the task

    No source says anything flagged the run while it was going. A monitor reading every action against the challenge scope would have raised an exploit and credential use against a domain outside the simulation.

  • Respond

    Contain, notify and disclose

    Irregular paused the evaluations, began remediation and notified affected third parties; it told OpenAI on 29 July and OpenAI published on 4 August. The run date is not public, so the gap between the attack and notice is unknown. OpenAI says it will set clearer incident-notification and escalation processes for third-party testing.

  • All controls →

Coverage and commentary

1 statement, 3 articlesShow

What the parties involved said

  • Nightingale Collective

    Notes OpenAI implies misconfiguration, not misalignment, but says OpenAI has given no strong reason to think it investigated as thoroughly as another lab did for near-identical cases, so lists it as an incident.

All 5 sources
  1. Primary source

    Third-party cyber evaluations involving OpenAI models ↗OpenAI · 2026-08-04

  2. Primary source

    Addressing Recent Incidents: Ongoing Findings and Path Forward ↗Irregular · 2026-08-14

  3. Press

    OpenAI and Anthropic incidents put Israeli AI security startup Irregular at center of race to safely test AI agents ↗Calcalist (CTech) · 2026-08-05

  4. Commentary

    #11. OpenAI model breaks into a real website during an Irregular cyber eval ↗Nightingale Collective (Rogue Agent Incident Dashboard)

  5. Commentary

    Third-party cyber evaluations involving OpenAI models ↗Simon Willison's Weblog · 2026-08-05

Entry id openai-irregular-real-website-2026 · overall Confirmed · impact serious

Related incidents