Qwen3.8-Next streaming - 150tps prefill, 3.6 tps decode on M5 Air
Heat trend
Collecting trend data
The percentage is based on available heat signal, not comment count or independent people.
A developer successfully adapted their DSv4 streaming stack to run Qwen3.8-Next on a 32GB M5 Air, achieving faster performance than a dense 27b model. Using a 3-bit version of the MoE, Qwen3.8-Flash-Next-MLX-oQ3-MTP, it reached 150 tps prefill and 3.6 tps decode for a 2k token prompt in low power mode. This compares favorably to the 27b-4bit model's 70 tps prefill and 3 tps decode.
Out of curiosity, I thought I'd see if I could adapt my DSv4 streaming stack from a few weeks ago to take Qwen3.8-Next. It worked, better than I thought - it actually runs faster on my 32GB M5 than the dense 27b does (admittedly not apples to apples as I decided to use a 3bit of the MoE, Qwen3.8-Flash-Next-MLX-oQ3-MTP, and the dense was 4bit).
For a 2k token prompt, running on low power mode on my M5, 3.8-Next-3bit gets 150 tps prefill, 3.6 tps decode. 27b-4bit gets 70tps prefill, 3 tps decode