OSWorld 2.0 puts long-horizon agents at 20.6%

OSWorld 2.0 tested 108 long-horizon computer-use workflows and found Claude Opus 4.8 completed only 20.6% outright, despite a 54.8% partial score at the 500-step limit. The OSWorld 2.0 paper says tasks average 318 tool calls and a human median of about 1.6 hours; Snorkel’s benchmark briefing says failures cluster around lost constraints, guessing, and skipped verification. Teams shipping Claude Cowork-style knowledge-work agents should measure multi-run completion and state correctness, not polished transcripts.