Qwen3.8 Flash Next now at 1.2k t/s prefill on Strix Halo
A developer has successfully optimized Qwen3.8 Flash Next on mainline llama.cpp to achieve 1.2k t/s prefill performance on Strix Halo, matching a closed-source solution called Halogen. This improvement involved extensive debugging and optimization, with the developer planning to submit proper PRs to mainline llama.cpp and a community fork. The work also aims to benefit the GLM 5.3 Flash architecture due to similar sparse attention mechanisms.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
PublishedOffset at this time: UTC+0Sep 12, 2026, 21:08 UTC
IngestedOffset at this time: UTC+0Sep 13, 2026, 15:01 UTC
- Published
- Sep 12, 2026, 21:08
- Ingested
- Sep 13, 2026, 15:01
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
As you all know, Qwen3.8 Flash Next on mainline llama.cpp is still in a pretty experimental stage, but a lot of community forks are trying to get it to work better. There's also a closed-source solution called Halogen (https://github.com/peonist-ai/halogen-flash-server) that boasted 1.2k t/s prefill numbers when the community fork barely reached 400. Since I dislike closed source and I like open source, I decided to take the challenge and bring llama.cpp up to the same performance level and I'm happy to report that after burning through a few evenings and a lot of tokens I got there.
The link contains the (Opus-generated) recap of the entire debugging / optimization journey made, as well as of course the links to the branch, the custom HIP runtime (making a glorious return), a probably-not-working-on-the-first-try installation script and the numbers. I'm trying to also explain the ecosystem - how the mainline, the community forks and custom forks/branches like mine work. Now that I've gotten the result, I'll work on cleaning it up and submitting proper PRs to mainline (will also submit a clean PR to the community fork), this should also help the GLM 5.3 Flash architecture since they use similar sparse attention.