vllm + p2p driver hack + qwen 3.8 27B vs llamacpp + qwen flash next?
一位开发者正在考虑从使用 vllm 和 Qwen 3.8 27B（fp8+256k kv 缓存）的配置迁移到使用 llamacpp 和 Qwen Flash Next iq4xs（8 位缓存，200k kv 缓存）的配置。然而，一个针对开源 NVIDIA 驱动的 p2p 显卡间通信补丁可能会显著提升 vllm 在其硬件上的性能，但 llamacpp 无法从中受益。这使得开发者陷入两难，因为 vllm 缺乏 4 位量化支持，并且没有足够的 VRAM 来运行 Qwen Flash Next fp8。因此，开发者正在权衡 Qwen Flash Next 带来的更高智能是否值得放弃 vllm 和 Qwen 3.8 27B 组合所提供的高吞吐量。
- 发布
- 2026年9月6日 23:00
- 来源类型
- 开发者社区
- 档位
- 社区
- 信源状态
- 正常
时间以 UTC 显示
更多信息
Hi everyone
I'm running four rtx 4090, 64GB ram, on a threadripper pro motherboard so all PCIe x16 ports, as a homelab machine for coding.
I was migrating from vllm + qwen 3.8 27B (fp8+256k kv cache) to llamacpp + qwen flash next iq4xs + 8 bit cache 200k kv cache... until someone had to ruin my migration with this awesome patch to enable p2p between cards on the open source nvidia drivers. On my hardware configuration apparently this would matter a lot. But only VLLM would benefit from it, not llamacpp - and no 4 bit like quant available on vllm and not enough vram for qwen flash next fp8.
I hereby ask the grand experts, what should I do amd what are your opinions?
- vllm + qwen 3.8 27B dense + p2p driver patch + 256k kv cache
- llamacpp + qwen flash next MoE iq4xs + 200k kv cache
Is qwen flash next really worth the extra intelligence versus higher throughput with vllm and sticking to qwen 3.8 27B ?