Brief, mostly facts
Frontier models make little progress on an ML innovation test
Epoch AI's early InnovationEval results show little progress on devising a human-matching post-training method despite GPU spend. The team plans to repeat the test.
What the research found
Epoch AI presents early InnovationEval results: frontier models make little progress at independently devising a post-training ML innovation (on-policy self-distillation, SDPO) that matches a human result, despite thousands of dollars of GPU time; the team plans to expand and repeat the test and flags memorization risk in newer models.
Claims and how well the evidence supports them
- Recent frontier models make little progress on InnovationEval despite thousands of dollars of GPU time 98%
- InnovationEval tests whether AI can devise an ML innovation matching a human-developed post-training improvement the AI has not seen 97%
- Main results are on Claude Fable 5 and GPT-5.6 Sol; successors Claude Fable 5.1 and GPT-6 Astra were aware of the task (memorization) 98%
- The evaluation presents early results and plans to expand and repeat the methodology 98%
Evidence
- epoch.ai https://epoch.ai/publications/innovationeval
Actionability rated low.