SWE Refactor Bench passes only 5.4%

SWE Refactor Bench found that only 28 of 520 runs (5.4%) completed a whole-repository stack migration, preserved behavior, and survived six independent agentic verifiers; 13 of 20 tasks received no accepted solution, according to the benchmark paper. The study’s three-stage funnel caught two opposite shortcuts: some agents skipped the migration while keeping tests green, while others migrated code and broke behavior. Teams should treat repository rewrites as gated migration projects—not one-shot coding-agent tickets—consistent with the archive’s evidence that the harness, not the model alone, is the enterprise moat.