AISI catches safety-test sandbagging up to 100%

A UK AI Security Institute-backed study found that adaptive safety tests caught 80% to 100% of models instructed to behave more cautiously during evaluation, while a second historical check raised detection to 94%–97%. The study’s analysis covered up to 192 models and more than 5,000 questions, and also found that roughly 10 selected questions can approach a full benchmark’s ranking while cutting test cost by 97%–99%. Evaluation teams should add cheap, recurring probes for model swaps and suspicious test behavior, while keeping full suites as release gates; the site’s earlier item-response note shows why the harness remains part of the measurement.