vllm + p2p driver hack + qwen 3.8 27B vs llamacpp + qwen flash next?
A developer is considering migrating from vllm with Qwen 3.8 27B (fp8+256k kv cache) to llamacpp with Qwen Flash Next iq4xs (8-bit cache, 200k kv cache). However, a new patch enabling p2p between cards on open-source NVIDIA drivers could significantly benefit vllm on their hardware, but not llamacpp. This creates a dilemma, as vllm lacks 4-bit quantization and insufficient VRAM for Qwen Flash Next fp8, leading to a question about whether the increased intelligence of Qwen Flash Next outweighs the higher throughput of vllm with Qwen 3.8 27B.
- Published
- Sep 6, 2026, 23:00
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Times shown in UTC
More details
Hi everyone
I'm running four rtx 4090, 64GB ram, on a threadripper pro motherboard so all PCIe x16 ports, as a homelab machine for coding.
I was migrating from vllm + qwen 3.8 27B (fp8+256k kv cache) to llamacpp + qwen flash next iq4xs + 8 bit cache 200k kv cache... until someone had to ruin my migration with this awesome patch to enable p2p between cards on the open source nvidia drivers. On my hardware configuration apparently this would matter a lot. But only VLLM would benefit from it, not llamacpp - and no 4 bit like quant available on vllm and not enough vram for qwen flash next fp8.
I hereby ask the grand experts, what should I do amd what are your opinions?
- vllm + qwen 3.8 27B dense + p2p driver patch + 256k kv cache
- llamacpp + qwen flash next MoE iq4xs + 200k kv cache
Is qwen flash next really worth the extra intelligence versus higher throughput with vllm and sticking to qwen 3.8 27B ?