Kog targets 30x faster GPU inference

French AI-infrastructure startup Kog says its Kog Inference Engine reaches 3,000 output tokens per second per request on eight AMD MI300X GPUs and 2,100 on eight NVIDIA H200s, using a 2-billion-parameter model without speculative decoding. The technical preview says the speed comes from co-designing the model, runtime, and GPU kernels; TechCrunch’s report says the 11-person company has collected 200 business leads while it works toward larger models. That is a latency demonstration, not a frontier-model or cost-per-task result: teams extending the archive’s local-inference hardware analysis should demand the same measurement on their model, context length, and batch shape before treating the headline as a fleet saving.