OpenAI built an LLM whose only job is hacking other LLMs, and it already found attack methods humans missed

OpenAI Built an AI That Hacks Other AI. Its Own Defense Already Caught Up.

Red-teaming used to mean hiring clever people to break into software before the bad guys did. OpenAI just automated the clever people.

The company built GPT-Red, an AI model trained exclusively to attack and break into other language models through a process called automated red-teaming 1. It discovered exploits humans had never found, cracked over 90% of the defenses in OpenAI's previous flagship model, and even hacked a real vending machine agent handling live transactions. OpenAI will not release GPT-Red. But the real signal is what happened on defense: GPT-5.6, the model hardened against GPT-Red's discoveries, blocks more than 77% of the same attacks. The gap between AI offense and AI defense is no longer measured in years. It is measured in model versions.

To build GPT-Red, OpenAI set up a self-play loop. One model attacked. Other models defended. Over many rounds, GPT-Red got better at breaking in, and the defenders got better at holding the line 1. The training ran inside what OpenAI called a dojo, a simulated environment mimicking real-world deployments: web browsing, email, calendar apps, code editing 1. These are the exact surfaces where today's LLM agents operate, and where a successful attack could steal data, corrupt code, or hijack a system.

"The risk surface grows and the blast radius also grows," said Nikhil Kandpal, a research scientist at OpenAI who co-created GPT-Red 1.

GPT-Red's most striking discovery is a novel class of attack that OpenAI calls a "fake chain of thought." Language models keep a running diary of notes as they reason through problems. GPT-Red figured out how to slip a forged entry into that diary, tricking a target model into accepting fabricated logic as its own verified work. Chris Choquette-Choo, another research scientist on the team, put it plainly: "It's like if I told you that 1+1=3 and that you have verified this already." The model accepts it. "The model's like, 'Oh, okay, of course,' and it just spits out 3" 1. Human red-teamers had never found this attack. GPT-Red did.

When OpenAI reran a 2025 experiment where human testers had tried to find weaknesses in an earlier version of GPT-5, GPT-Red outperformed the humans at finding effective exploits 1. "Compared to a human red-teamer, the model is very, very good at finding exactly what will work, exactly what's most effective," said Dylan Hunn, a research scientist at OpenAI and GPT-Red's fellow co-creator 1.

Then came the real-world test. OpenAI turned GPT-Red loose on Vendy, a vending machine agent built by Andon Labs, a company that assesses how well agents perform real-world tasks. GPT-Red cut the price of a stocked item to $0.50, listed a pricey new item at the same price, and canceled another customer's order 12. This was not a hypothetical breach in a sandbox. An AI attacker manipulated a system that touches real money and real customers.

But here is where the story shifts from alarm to something more measured. When OpenAI threw GPT-Red's strongest attacks at its models, more than 90% worked against GPT-5, which launched in August 2025. Against the new GPT-5.6, fewer than 23% landed 1. The defense is not perfect. But it is closing fast, because GPT-5.6 was trained against GPT-Red, learning to resist the very attacks GPT-Red invents 1.

The implication for the red-teaming profession is stark. OpenAI frames GPT-Red as a supplement to human testers, not a replacement, and the model has real gaps. It struggles with multi-turn conversational attacks and image-based exploits, both areas where humans still hold an edge 1. "I think human expertise will still be very important," said Jessica Ji, a senior research analyst who works on AI security at Georgetown University's Center for Security and Emerging Technology 1. But when one model can run thousands of attack variants around the clock and already beats humans at finding effective exploits, the economics of red-teaming change. The question is not whether human testers stay involved but whether their role shrinks from frontline attacker to something closer to QA reviewer.

For every company deploying LLM agents that browse, email, and execute code, the takeaway is direct. Automated attackers are not a future threat. They exist, and they work. The techniques GPT-Red discovers will eventually surface in the wild. The companies that hold the line will be the ones running their own automated defense loops, not the ones relying on quarterly penetration tests.

The question was never whether AI can attack AI. That is settled. The question is whether the defense cycle stays ahead when both sides are machines, running at the speed of compute, with no human in the loop.