SWE-Marathon leaves agents at 74%

Frontier coding agents topped out at 74% on SWE-Marathon-Ext, a 12-product-clone benchmark that starts from empty workspaces rather than patching mature repositories. The benchmark report says eight models completed three rollouts across 12 SaaS tasks—288 rollouts total—and the hardest categories were UX at 32%, time logic at 43%, and concurrency at 50%; the report says the top model’s 74% still leaves roughly a quarter of behavior failing. For teams using whole-system harness comparisons, the filed-away number is not the leaderboard winner but the unfinished product surface: score concurrency, validation, and user-visible workflows before allowing an agent to ship from a blank repo.