WIDE speeds LLM decoding 1.55x end to end

WIDE delivered 1.55× end-to-end decode acceleration and 1.68× prefill acceleration by letting each token choose which attention-head and feed-forward groups to run. The authors’ preprint reports that at 50% sparsity the method retained 55.1% more performance than dynamic depth pruning, while custom kernels reached 4.95× decode speedups before whole-system overhead narrowed the gain; its implementation is public. For teams responding to enterprise AI’s shift from token volume to cost per outcome, the useful number is the smaller end-to-end gain, not the kernel headline: dynamic sparsity may lower serving cost, but it still needs workload-specific quality and hardware tests.