NVIDIA reports 10.4× MoE training gain

NVIDIA says JAX and Transformer Engine lifted DeepSeek-V3 MoE training from 103 to 1,068 TFLOPS per GPU—a reported 10.4× gain—and held 97% scaling efficiency across 1,024 GPUs. Its technical benchmark is a vendor result, not a universal production promise; the example depends on dropless MoE kernels and NVIDIA’s stack. Teams comparing training hardware should keep workload-specific measurements beside NVIDIA’s warning that faster routing does not pool GPU memory.