Mac Heads: Is there any point to MLX in September 2026?
热度趋势
百分比基于当前可用热度信号，而非评论数或独立用户人数。
一位Reddit用户正在探讨MLX在2026年9月对Mac用户的意义，特别是在Apple M5 Pro芯片上运行Qwen3.8 27b等模型时。他们想知道自己观察到的性能，即Q8/8位量化下约19t/s的生成速度和300t/s以上的预填充速度是否已是最佳，或者是否有方法能获得更好的结果，或有效利用主流MLX MTP。该用户还在思考，Qwen3.8 27b这一特定模型是否是性能的限制因素。
This may be somewhat specific to Qwen3.8 27b and the Apple M5 series, perhaps, but enough of us are running this combo that it's worth tossing out there.
GGUF models with MTP have been the fastest way to go for some time for token generation, except possibly for a few tweaked MTPLX models running on alpha-stage MLX forks. Prefill, however, was still much faster for M5s under MLX. This has caused me to switch models depending on the expected generation/prefill mix, which is annoying.
While I wasn't looking, it appears that llama.cpp for Metal must have added support for M5 matmul/"neural accelerators" because prefill performance with e.g. Unsloth's Q_8 GGUF is now at least as good (~300-350t/s) as anything I have seen with MLX models--even in oMLX. This was a pleasant surprise! Now I can't think of a reason to use MLX models at all.
Am I missing something? Are my observation bogus? Could I do better than ~19t/s generation and 300t/s+ prefill on a M5 Pro with Qwen3.8 27b in the Q8/8-bit range? Is there a secret handshake to get mainstream MLX MTP working?
Or is this just because of the specific model in question?