You can offload most of Qwen3.8-Flash-Next's KV cache to RAM with little decode slowdown

Qwen3.8-Flash-Next, and potentially other qwen4exp-based models, can offload most of their KV cache to system RAM, significantly reducing VRAM requirements. This allows for running models at maximum context length without KV cache quantization, even with quantizations that barely fit in VRAM. While a full KV cache row is 2,048 B per token, only about 66 B per token per layer needs to remain on the GPU, leading to minimal decode slowdown despite the PCIe 4.0 x16 slot's 32 GiB/s bandwidth limiting host-resident cache to about 5 tokens per second.