Brief, mostly facts
SlopCodeBench: 15 coding agents degrade code quality over iterations
The best agent passed only 14.8% of checkpoints. The paper dates from March with a May revision, so the models tested are not current.
What the research found
SlopCodeBench from University of Wisconsin-Madison found that 15 coding agents degrade code quality over iterative tasks, with the best passing only 14.8 percent of checkpoints. The paper dates from March with a May revision, so the tested models are not current.
Claims and how well the evidence supports them
- SlopCodeBench tested 15 coding agents on 36 tasks with 196 checkpoints 98%
- The best agent passed 14.8 percent of checkpoints (GPT 5.5) 99%
- Code erosion increased in 77 percent of trajectories and verbosity in 75.5 percent 99%
- The study was first published in March and revised in May, so models tested may be outdated 97%
Evidence
- pcgameshardware.de https://pcgameshardware.de/Kuenstliche-Intelligenz-Hardware-279517/News/KI-Coding-Agenten-Langzeittest-Code-Qualitaet-ueber-70-Prozent-schlechter-1557263
Actionability rated low.