# ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?

**Authors:** Zhun Wang, Nico Schiller, Hongwei Li, Srijiith Sesha Narayana, Milad Nasr, Nicholas Carlini, Xiangyu Qi, Eric Wallace, Elie Bursztein, Luca Invernizzi, Kurt Thomas, Yan Shoshitaishvili, Wenbo Guo, Jingxuan He, Thorsten Holz, Dawn Song  
**arXiv:** [2605.11086](https://arxiv.org/abs/2605.11086)  
**Date:** 2026-05-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2605.11086v1/

## Key points

- AI agents in cybersecurity present challenges and opportunities, particularly in the area of exploitation.
- Exploitation involves turning vulnerabilities into security threats like unauthorized access or code execution.
- Exploitation requires deep program reasoning, runtime adaptation, and sustained progress over time.
- Exploitation has dual-use implications for defensive and offensive purposes.
- The introduction of ExploitGym provides a benchmark to assess AI agents' exploitation capabilities with 898 real-world vulnerability instances.
- Top-performing AI models like Anthropic's Claude Mythos Preview and OpenAI's GPT-5.5 have shown success in exploiting vulnerabilities even with common defenses enabled.
- ExploitGym serves as a valuable testbed for evaluating exploitation capabilities and highlights the increasing cybersecurity risks posed by advanced AI agents.

## Summary

, , , , 

In the rapidly evolving landscape of cybersecurity, the advancement of AI agents is posing significant challenges and opportunities. One critical capability that is gaining prominence is exploitation - the process of turning a vulnerability into a concrete security threat. This can include unauthorized access to files or code execution. However, this task is complex and requires deep program reasoning, runtime adaptation, and sustained progress over extended periods. Furthermore, exploitation has dual-use implications, serving both defensive purposes and potentially lowering barriers for offensive actions. Despite its crucial importance and diagnostic value, it has been under-evaluated in existing frameworks. To bridge this gap, a team of researchers introduced ExploitGym - a comprehensive benchmark designed to assess the exploitation capabilities of AI agents. The benchmark consists of 898 instances sourced from real-world vulnerabilities spanning various domains including userspace programs, Google's V8 JavaScript engine, and the Linux kernel. Each instance is equipped with varying security protections to evaluate their impact on agent performance. Through rigorous evaluation using ExploitGym, it was observed that while exploitation remains a challenging task, cutting-edge AI models demonstrated success in exploiting a notable fraction of vulnerabilities. For instance, top-performing configurations like Anthropic's Claude Mythos Preview and OpenAI's GPT-5.5 were able to produce working exploits for 157 and 120 instances respectively. Even when common defenses were enabled, these models exhibited non-trivial success rates. These findings underscore the effectiveness of ExploitGym as a valuable testbed for evaluating exploitation capabilities and shed light on the escalating cybersecurity risks posed by increasingly sophisticated AI agents. With the continuous evolution of technology and cyber threats, initiatives like ExploitGym play a crucial role in enhancing our understanding of AI-driven cybersecurity challenges and fortifying defense mechanisms against potential attacks.