CogGym finds a 0.43 human-fit ceiling in video tests

CogGym evaluated 50 large language models across 258 cognitive experiments and found the best model–human fit reached only R² = 0.43 on video tasks, versus 0.92 for human split-half reliability. The Stanford Causality in Cognition Lab release says fit also reached 0.59 on text and 0.58 on image, while model gains on commonsense tasks lag formal math and coding gains. Teams should add matched human-behavior tests to eval plans, as the Astra benchmark-split analysis does for another failure of headline scores to define workload fit.