Qwen3.8 Flash Next llama.cpp config tuning
A user is discussing configuration tuning for Qwen3.8 Flash Next within llama.cpp, specifically for the llama-server. They provided a detailed command-line signal including parameters like --port, --model, --mmproj, --ctx-size 131072, --flash-attn on, --batch-size 2048, and --ubatch-size 512. The user manually adjusted ngl, ncmoe, and ts to prevent crashes, indicating a focus on optimizing model performance and stability.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
PublishedOffset at this time: UTC+0Sep 12, 2026, 08:18 UTC
IngestedOffset at this time: UTC+0Sep 12, 2026, 22:01 UTC
- Published
- Sep 12, 2026, 08:18
- Ingested
- Sep 12, 2026, 22:01
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
Do you guys mind sharing your LLama.cpp config and system setup details for Qwen3.8 Flash Next?
Model's quite big and tryining many combinations of llama.cpp options takes lots of time, so looking for other people setup details. I've attached my current config at the bottom, so if anyone sees something that could be improved please shout.
- PP within 130...200 tps (limited by cpu?) - TG within 14..22 tps (~15tps on average)
- Dual RTX 3090 (48GB VRAM) - 128GB DDR4 - Some old Xeon 40 core - Proxmox VM, pcie passthrough, numa binding to a single phys cpu
llama-server --port ${PORT} --model /nvme/gguf/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf --mmproj /nvme/gguf/mmproj-Qwen3.8-Flash-Next-F16.gguf --load-mode none --lazy-mode off --parallel 1 --ctx-size 131072 --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --fit off --temp 1.0 --min-p 0.0 --top-p 0.95 --top-k 20 --presence-penalty 0.0 --repeat-penalty 1.0 --batch-size 2048 --ubatch-size 512 --split-mode layer -ts 26,10 -ngl 99 -ncmoe 26 --no-mmproj-offload --override-tensor per_layer_token_embd=CPU --chat-template-kwargs '{"reasoning_effort":"xhigh"}'