Epoch tests six models on real tasks: strong on defined work, weak on open-ended

What the research found

Epoch AI's Automation Reports test six models on 11 real Epoch tasks; frontier models (Fable 5.1, GPT-6 Astra) do well on well-defined tasks but fail open-ended work, style matching, and experiment design.

Claims and how well the evidence supports them

1. Six models were evaluated on 11 real Epoch work tasks across five categories 98%
2. Fable 5.1 and GPT-6 Astra broadly lead but fall short of automating Epoch work 99%
3. Models struggle to match Epoch's visual style and standards 98%
4. Open-weight models lag behind frontier models 98%

Evidence

• epoch.ai https://epoch.ai/publications/can-ai-automate-epoch

Actionability rated low.