OpenAI's GPT-Red LLM finds successful attacks in 84% of test scenarios, versus 13% for human red teamers. The model, detailed by MIT Technology Review, was used to harden GPT-5.6 Sol, OpenAI's most robust release yet.
Key facts
- GPT-Red finds attacks in 84% of test scenarios
- Human red teamers manage just 13%
- GPT-5.6 Sol is OpenAI's most robust release yet
- GPT-Red discovered new types of attack not seen before
- Self-play training in a simulated dojo environment
OpenAI has built an LLM super-hacker called GPT-Red that it uses as a sparring partner to help its other models boost their defenses against cyberattacks. According to MIT Technology Review The company released the latest version of its flagship LLM, GPT-5.6 Sol, last week. OpenAI says that training it against GPT-Red made the model its most robust release yet.
GPT-Red automates red-teaming, a safety evaluation typically done by human testers. The aim is to find as many different ways to break or hijack a system as possible. The weak spots can then be patched before the final version of the software is released.
As LLMs become more complex and get used in a wider variety of tasks—especially in the form of agents, which can interact with computer files, websites, and third-party code as well as other agents—it’s hard for teams of people by themselves to keep up with all the types of attacks that might take place. “The risk surface grows and the blast radius also grows,” says Nikhil Kandpal, a research scientist at OpenAI who co-created GPT-Red.
OpenAI built GPT-Red to future-proof its safety testing process. “As more capable models become available, we will have already designed the system that can discover new modes of attack,” says Dylan Hunn, a research scientist at the company and fellow co-creator of GPT-Red. The researchers say it has already come up with new types of attack that had not been seen before.
Self-Play Dojo
To build GPT-Red, OpenAI’s researchers took an LLM that had not been trained as a hacker and set it up in what’s known as a self-play loop with several other models. Its goal was to try to attack the other models; their goal was to try to defend themselves. Over many rounds of play, GPT-Red became better and better at attacking other LLMs, and those LLMs became better and better at fending off the attacks.
The training took place in a kind of dojo that OpenAI had designed to mimic a range of scenarios in which LLMs might be deployed in the real world, including browsing the web, reading emails or calendar apps, and editing code. OpenAI focused most of its efforts on prompt injection attacks, where a hacker slips an LLM instructions to make it do things its developers or users do not want it to, such as copy confidential information or sabotage a company’s code base.
When GPT-Red found a new kind of attack, it would explore multiple different versions of it to find the most efficient one for specific scenarios. “Compared to a human red-teamer, the model is very, very good at finding exactly what will work, exactly what’s most effective,” says Hunn.
The gap between GPT-Red's 84% success rate and human red teamers' 13% is not just a performance metric — it signals a structural shift in how frontier AI safety is conducted. Human red teaming, already a bottleneck at scale, becomes untenable as agentic systems multiply attack surfaces. OpenAI's approach mirrors a trend across the industry: Anthropic's Claude Code and Google's Gemini agent runtimes are also adopting automated adversarial testing, though none have disclosed comparable internal hit rates.
Broader Context
The GPT-Red development comes as OpenAI's GPT-5.6 Sol competes with Anthropic's Claude Fable 5 for the frontier-model crown. Analytics Vidhya reports Fable 5 holds a slight edge in general intelligence, while Sol hits back with stronger coding performance and much lower pricing. In a separate demonstration of GPT-5.6 Sol's capabilities, a University of Pennsylvania statistics professor used the model to disprove a 30-year-old conjecture about the Benjamini-Hochberg method in roughly 90 minutes. The Decoder reports The predecessor model, GPT-5.5, couldn't find a solution even after 20 hours.
Key Takeaways
- OpenAI's GPT-Red LLM finds 84% of attacks vs 13% for humans, hardening GPT-5.6 Sol.
- Automated red-teaming shifts safety paradigm.
What to watch
Watch for OpenAI's disclosure of GPT-Red's attack success rate on agentic workflows beyond prompt injection, and whether Anthropic or Google release comparable internal red-teaming benchmarks for their own agent runtimes. Also monitor GPT-5.6 Sol's enterprise adoption rate as a signal of trust in automated safety testing.
Source: technologyreview.com
