Sauron verifies agent rubric claims against environment state

What the research found

Sauron is an evidence-first agentic verifier that checks rubric claims against environment state and traces, reporting 0.96 F1 versus Gandalf 0.92 at 23% lower cost and detecting 97% of injected failures with 2% false flags.

Claims and how well the evidence supports them

1. Sauron reaches 0.96 F1 versus Gandalf 0.92 with GPT-5.6 Sol at 23% lower cost 99%
2. Sauron detects 97% of injected failures while flagging 2% of clean controls 98%
3. Evaluation uses 2,586 human-labeled criteria from 100 personal-assistant episodes 99%

Evidence

• huggingface.co https://huggingface.co/papers/2610.17724

Actionability rated low.