I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8

There's an interactive chart and some extra data in the blog post if you're interested.

There are plenty of KL-divergence benchmarks for GGUF models, but most of them compare one GGUF quant against another. I wanted to know how those quants stack up against other commonly used formats (especially NVFP4).

I tested 16 quantizations of Qwen3.6 27B: GGUF models in llama.cpp and the others in vLLM. At each token in the test set, I compared the quantized model’s next-token probability distribution with that of an unquantized reference. The resulting KL divergence measures how far the quant has drifted from the original model; lower is better.

GGUF results occupy most of the lower envelope of the chart. For almost every size, a GGUF running in llama.cpp has the lowest measured KL divergence among nearby weight sizes. The main factor here is likely the activation quantization - GGUFs don’t quantize activations at all. Several vLLM checkpoints quantize weights, activations, and sometimes the KV cache.

Quantizations of similar size do not preserve the reference distribution equally well. Particularly of note is the Sakamakismile NVFP4 (W4A4) quant, which has substantially higher KLD compared to similarly sized (and even smaller) quants.

The two conventional Q4 GGUFs are consistent with each other. Bartowski Q4_K_L measures 0.2218 and Unsloth UD_Q4_K_XL measures 0.2273, with heavily overlapping intervals. AWQ and NVIDIA’s mixed NVFP4 are also nearly tied at 0.2776 and 0.2807.

Checkpoint Weight quantization Activation quantization KV cache uns_UD_IQ3_XXS Dynamic 2.0, IQ3_XXS base; per-tensor type from calibration none none bart_IQ3_XS IQ3_XS imatrix mix none none nvfp4_MTP_gguf custom tensor mix on NVFP4 weights; RSF scale fitting on the Q_K tensors; MTP tensors NVFP4 none none AutoRound_INT4 INT4, symmetric, group 128 none none uns_UD_Q4_K_XL Dynamic 2.0, Q4_K base; per-tensor type from calibration none none bart_Q4_K_L Q4_K imatrix mix none none NVFP4_Text_MTP NVFP4, group 16, static scales, all LM Linear NVFP4, group 16, static (W4A4) none AWQ_INT4 INT4, asymmetric (int8 zero-point), group 32 none none NVFP4 NVFP4 group 16 on mlp.* + lm_head; FP8 E4M3 on self_attn.* and linear_attn.{in_proj_qkv,in_proj_z,out_proj} static FP8 on the FP8 group (W8A8) static FP8 uns_UD_Q5_K_XL Dynamic 2.0, Q5_K base; per-tensor type from calibration none none uns_NVFP4 NVFP4 group 16 on mlp.{gate,up,down}_proj in layers 0-55; FP8 E4M3 per-channel on self_attn.*, linear_attn.*, lm_head, and mlp.* in layers 56-63 NVFP4 group 16 on the NVFP4 group (W4A4); dynamic per-token FP8 on the FP8 group (W8A8) static FP8 bart_Q6_K_L Q6_K imatrix mix none none uns_UD_Q6_K_XL Dynamic 2.0, Q6_K base; per-tensor type from calibration none none bart_Q8_0 uniform Q8_0 none none qwen_FP8 FP8 E4M3, 128×128 weight blocks dynamic per-token FP8 (W8A8) none uns_UD_Q8_K_XL Dynamic 2.0, Q8_K base; per-tensor type from calibration none none

At every prompt position, the benchmark computes D_KL(P_reference || P_quant): how much the quantized model’s next-token distribution differs from the full-precision distribution. Zero means no measured change; larger values mean more of the reference distribution was displaced.

Both engines compute exact full-vocabulary softmax probabilities, but only the top 200 log probabilities per position are used. The benchmark solves for the minimum KL consistent with the two measured top-200 lists, their remaining probability budgets, and the fact that an unlisted quant token cannot exceed the quant’s smallest reported probability, in order to get a lower bound on full-vocabulary KL.

The mean reference tail mass outside the top 200 was 0.0025 for both engines in this run. Top-1 agreement does not depend on the tail approximation and provides a complementary check.

Top-1 agreement is the fraction of positions at which the quantized model and its reference assign the highest probability to the same token.

I created my own dataset for the KL measurements, which ended up being 100 structured agentic tool-use conversations containing 182,306 tokens. Prompts range from 1,700 to 1,950 tokens.

Quantized checkpoints ran without changes, including any declared compute dtype, activation quantization, or KV-cache scheme, in order to measure the true fidelity of each quant recipe.

The size measurement includes MTP/NextN layers and excludes KV/recurrent caches, activations, workspaces, CUDA graphs, runtime context, and unloaded multimodal components. It is not total serving memory. Take these measurements with a grain of salt, as they’ll vary in actual deployment depending on your configuration.

- Quantization format alone is not enough to predict quality. Look at the quantization recipe to determine if it fits your needs.

- Activation quantization can improve throughput on supported hardware, but this comes at the cost of quality.

- If quality per loaded GiB is the priority, the tested GGUF recipes provide the strongest tradeoffs.

- GGUF Q5 for Qwen3.6 27B seems to be the sweet spot from the results.

KLD benchmarks may be able to show the relative differences in quantization quality, but this doesn’t translate perfectly into real-world performance. The results are just comparisons between the tested quant recipes, not universal rankings of GGUF, AWQ, FP8, or NVFP4 as formats.