Google tests Gemini in a double-blind evaluation

Google DeepMind launched what it calls the first double-blind evaluation of a proprietary frontier model, testing a Gemini Flash Lite model with 4 external partners against confidential safety benchmarks. DeepMind’s announcement says the pilot uses cryptographic protections to keep the model weights and test prompts hidden from the other parties; MLCommons’ account says it used a reserved subset of AILuminate prompts that the model had not seen before. For evaluation teams, this is a trust architecture rather than a capability score—held-out prompts and attested environments are becoming as important as the benchmark result, a point that echoes the archive’s production-grade agent evaluation controls.