Wire
NEEDLE refreshes agent-search tests hourly
Keenable open-sourced NEEDLE, a five-vertical benchmark whose news queries refresh hourly while finance, scholar, deep-tail, and legal suites refresh daily. The live dashboard gives every engine the same query, scores titles and snippets under a 2,000-character evidence cap, and does not fetch pages or rerank them; the launch note makes its code and run artifacts public. Agent teams should add NEEDLE to search-vendor reviews beside the archive’s 12-provider cost benchmark, because stale leaderboards can reward memorization instead of retrieval quality.