AI agents find novel ML ideas but miss utility

A University of Michigan study tested two LLM research-agent frameworks across 10 Kaggle-style MLE-Bench tasks and found that novel approaches rarely translated into better scores. The authors’ open evaluation framework separates psychological and historical creativity from usefulness, while the study’s reported results put GPT-5 judge-human agreement at 0.732 but the correlation between novelty changes and impact at -0.021. Builders evaluating research agents should score idea usefulness separately from originality, as the archive’s Copilot trial scorecard does for adoption.