AtomicChat/Qwen3.8-Flash-Next-GGUF is Really Good
Heat trend
Collecting trend data
The percentage is based on available heat signal, not comment count or independent people.
Llama model activity is surfacing — worth tracking for capability changes, ecosystem impact, and availability.
AtomicChat's quant for Qwen3.8-Flash-Next-GGUF significantly reduces memory usage by utilizing llama.cpp mmap and making the PLE table pageable. This innovation shrinks the model's RAM footprint from 106GB to 65GB (starting from 55GB). Additionally, the pageable PLE table improves prefill performance, achieving a cold start of approximately 500 t/s, with ongoing open PRs in oMLX aiming to further enhance size and performance, including nearly tripling SSD cold prefill speed.
Qwen3.8-Flash-Next is a great model I benched in my previous post , but it is very tight, since all n-grams / PLE are loaded along with the experts, taking 106GB, leaving very little room for K/V, context, etc. Offloading PLE to SSD currently slows down prefill from 600 t/s to 180 t/s on oMLX.
u/erikdhoward suggested to try the Atomic Chat quant which I did not know anything about. I tried it, and it is... really good.
AtomicChat's quant uses llama.cpp mmap (through GGUF shard layout vs. in the runtime) and keeps the PLE table (n-grams) pageable backed by a file. Because of this the same model that took 106GB, now takes 65GB (starts from 55GB) in RAM. And since PLE is pageable the prefill is actually not that bad, cold start is about 500 t/s.
Qwen3.8-Flash-Next just came out, and there are many open PRs in oMLX to address the size and performance, including this one that makes PLE offload SSD cold prefill almost 3 times faster 🎉