# Hacking CTFs with Plain Agents

**Authors:** Rustem Turtayev, Artem Petrov, Dmitrii Volkov, Denis Volk  
**arXiv:** [2412.02776](https://arxiv.org/abs/2412.02776)  
**Date:** 2024-12-03  
**Summary page:** https://www.summarizepaper.com/fr/paper/2412.02776v1/

## Abstract

We saturate a high-school-level hacking benchmark with plain LLM agent design. Concretely, we obtain 95% performance on InterCode-CTF, a popular offensive security benchmark, using prompting, tool use, and multiple attempts. This beats prior work by Phuong et al. 2024 (29%) and Abramovich et al. 2024 (72%). Our results suggest that current LLMs have surpassed the high school level in offensive cybersecurity. Their hacking capabilities remain underelicited: our ReAct&Plan prompting strategy solves many challenges in 1-2 turns without complex engineering or advanced harnessing.