CyberArena benchmarks agents in live attack and defense matches

What the research found

CyberArena is an arXiv benchmark (v2 Oct 19 2026) pitting LLM agents against each other in live symmetric attack and defense sandboxes, testing 12 frontier agents over 466 matches.

Claims and how well the evidence supports them

1. CyberArena evaluates 12 frontier LLM agents across 40 challenges in 466 matches 99%
2. CyberArena is a dynamic symmetric attack-and-defense benchmark run in isolated sandboxes 99%
3. arXiv 2610.19005 v2 was submitted October 19 2026 99%
4. Agents in CyberArena wrote their own packet sniffer in one case 98%

Evidence

• arxiv.org https://arxiv.org/abs/2610.19005

Actionability rated low.