Edge0 runs a 35B model in 3 GiB of active memory

Edge0 serves a 35B mixture-of-experts model at 14.9–17.7 tokens per second while using 2.9 GiB of active memory on a 24GB Mac mini, according to its model card. The open pipeline streams experts from SSD and predicts the next route one token ahead; the accompanying paper reports an average score of 79.2 versus 83.2 for the FP16 base across five public benchmarks, while the model card warns the preview is weak on agentic tasks. Builders can now test a 35B local checkpoint without keeping its full weight set active, but storage latency, long-context KV cache and agent quality still decide whether it is useful; Nvidia’s local-inference routing analysis shows why distributed speed does not pool memory.