Qwen3.8 Flash on 12GB VRAM - 15 tokens/s

A user achieved 15 tokens/second output and 100-120 prompt processing per second with Qwen3.8-Flash-Next-GSQ-RCO-GGUF on a 12GB GPU (RTX 5070 SFF). The model, at 3 bpw (IQ3_XXS), utilized 47GB sharded across VRAM and RAM, with the rest handled by SSD. Performance metrics included a 4.9s server startup, 31.7s cold launch for a 1K prompt, and generation speeds ranging from 11.3 to 14.6 tokens/s for various prompt sizes.

This report details specific performance metrics for Qwen3.8 Flash on a 12GB GPU, unlike general claims about its efficiency, providing concrete numbers for token generation and prompt processing speeds.