AWS benchmark leaves AI security below 10%

AWS’s new Deception Benchmark finds that none of 12 tested model configurations kept both false-positive and false-negative rates below 10%. The benchmark covers 14,822 samples in 16 languages and more than 70 CWE categories; AWS reports direct prompting caught up to 95% of real vulnerabilities while flagging 41–99% of safe code, and proof-of-exploit prompting lowered false positives but missed 7–44% of real flaws, according to AWS’s benchmark release. Security teams should treat model-generated vulnerability findings as triage until they measure both error rates on their own codebase, extending the agent-evaluation governance case rather than shipping an accuracy-only scorecard.