AtomicChat/Qwen3.8-Flash-Next-GGUF is Really Good
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Llama 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。
AtomicChat针对Qwen3.8-Flash-Next-GGUF的量化技术表现出色，它通过利用llama.cpp mmap并使PLE表可分页，显著降低了内存占用。这项创新将模型的RAM占用从106GB减少到65GB（起始为55GB）。此外，可分页的PLE表改善了预填充性能，冷启动速度约为500 t/s。oMLX中还有许多开放的PR正在解决大小和性能问题，其中一项有望使SSD冷预填充速度提高近三倍。
Qwen3.8-Flash-Next is a great model I benched in my previous post , but it is very tight, since all n-grams / PLE are loaded along with the experts, taking 106GB, leaving very little room for K/V, context, etc. Offloading PLE to SSD currently slows down prefill from 600 t/s to 180 t/s on oMLX.
u/erikdhoward suggested to try the Atomic Chat quant which I did not know anything about. I tried it, and it is... really good.
AtomicChat's quant uses llama.cpp mmap (through GGUF shard layout vs. in the runtime) and keeps the PLE table (n-grams) pageable backed by a file. Because of this the same model that took 106GB, now takes 65GB (starts from 55GB) in RAM. And since PLE is pageable the prefill is actually not that bad, cold start is about 500 t/s.
Qwen3.8-Flash-Next just came out, and there are many open PRs in oMLX to address the size and performance, including this one that makes PLE offload SSD cold prefill almost 3 times faster 🎉