Qwen3.8-Flash-Next (UD-Q4_K_XL) on a single RTX 3090 24GB + 128GB DDR4, is this config optimal?
- 发布
- 09/05 12:52
- 收录
- 09/05 20:00
- 来源类型
- 开发者社区
- 档位
- 社区
- 信源状态
- 正常
一位用户正在就其在单个 RTX 3090 24GB 显卡和 128GB DDR4 内存上运行 Qwen3.8-Flash-Next (UD-Q4_K_XL) 的配置寻求优化建议。性能数据显示，处理 4,096 个 token 的速度为 183.6 tok/s，处理 11,895 个 token 的速度为 189.4 tok/s，并能持续生成文本。该用户乐于接受任何改进建议，并愿意分享更多日志以供分析。
Qwen3.8-Flash-Next (UD-Q4_K_XL) on a single RTX 3090 24GB + 128GB DDR4 — is this config optimal?
Hardware
- CPU: Intel Core i5-12600K
- RAM: 128 GB DDR4 @ 3600 MHz
- GPU: NVIDIA RTX 3090, 24 GB VRAM
- OS: Windows 11
- llama.cpp: freshly compiled from today's master (build b10794, Sep 4 2026)
Model
Qwen3.8-Flash-Next, UD-Q4_K_XL
Launch command
C:\llama\llama-server.exe -m "D:\Modelos\Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf" ^ -ngl 99 -ncmoe 42 -fit off -c 204800 -fa on -ctk f16 -ctv f16 ^ -b 2048 -ub 1024 -t -1 --jinja --port 8083 --tools all ^ --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.00 --repeat-penalty 1.0 ^ --reasoning on --reasoning-effort xhigh --reasoning-budget -1 --reasoning-preserve ^ --parallel 1 --metrics --host 0.0.0.0 --load-mode none
Key points of the config:
- -ngl 99 → offload all layers to GPU where possible
- -ncmoe 42 → force 42 MoE (expert) layers to stay on CPU/RAM
- -c 204800 → 200k context window
- -fa on with f16 K/V cache
- -b 2048 / -ub 1024 → batch / micro-batch sizes
- --reasoning-effort xhigh, --reasoning-budget -1, --reasoning-preserve → full reasoning mode, no truncation
State right after loading
- Free system RAM: 52 GB
- Free VRAM: 0.6 GB
So the GPU is essentially maxed out (good VRAM utilization), and there's still a healthy ~52 GB of RAM headroom for the CPU-offloaded experts and context.
Real-world performance (from server logs)
Prompt processing (long context, ~12k tokens):