OpenAI's Jalapeño chip cuts inference latency

OpenAI says its Jalapeño inference chip delivered 1.5–1.9x more AI work per watt and 1.7–3.6x lower end-to-end latency than Nvidia GB200 and GB300 systems across GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T, according to The Verge’s report on the InferenceX results. OpenAI’s first-results post says the Broadcom-developed ASIC keeps model state and KV cache local to reduce communication delays; small-volume deployment is planned by the end of 2026, with a larger ramp in 2027. The result is a vendor benchmark against today’s Blackwell, not a general GPU replacement, so inference buyers should keep flexible capacity contracts and treat latency-per-watt as a workload-specific procurement metric, extending the archive’s analysis of OpenAI’s 20% monitoring compute tax.