AutoTuneBench cuts a 10.6x serving claim to 2.03x

AutoTuneBench found an agent-tuned kernel’s headline 10.6x speedup fell to 2.03x against an honest baseline, while median speedup was 1.0001x across 51% of KernelBench Level-1 tasks, per the arXiv paper. Its protocol keeps provenance checks, anti-cheat tests, and pre-registered readouts outside the agent’s modification surface. That is a useful companion to the archive’s audit of RAG benchmark accuracy: serving teams should demand reproducible baselines before accepting an autonomous optimization claim.