Gemma 4 E4B IQ2_XXS: + 140.54% Reasoning Performance From Tensor Level Quantization Allocation
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Gemma 4 E4B IQ2_XXS 模型通过张量级量化分配，推理性能显著提升。在相同的 3.3 GiB 预算下，推理分数从 28.9 提高到 69.5，增长了 140.54%。这种分配结合 imatrix，极大地恢复了量化损伤，上下文分数从 15.625 跃升至 95.833。…
iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget.
I posted my Gemma 4 12B q3 result a couple days ago, where tensor level allocation gave me an +8.55% relative improvement over the category imatrix baseline.
I pushed further down to iq2_xxs territory where the damage is more severe and the result is far more dramatic.
Same idea as before, build imatrix from a category oriented corpus, measure damage, redistribute precision at the tensor level under a fixed byte budget.
The BF16 source scored 71.875, meaning the allocated iq2 model retained 96.74% of the source reasoning performance while reaching only about 24% of the BF16 size.
It was not isolated to reasoning. However, at iq2_xxs with gemma 4 e4b, there is no legal stock model and the imatrix model mostly collapsed.
Compared with the same size imatrix baseline 10 of 11 categories improved by point estimate. The only regression being stability.
There is no post training, lora, pruning or weight updates. The results were achieved solely through precision allocation.
My goal is the same. Feed the pipeline with a bf16 gguf, select the capabilities you want and have it spit out the appropriate quantization level and tensor level allocation that gives you the best recovery available for those capabilities within the byte budget.
TL;DR: At essentially the same ~3.3 GiB IQ2_XXS budget, tensor-level allocation + the same category imatrix improved reasoning from 28.906 to 69.531 (+40.625 points, 95% CI [+24.6, +55.4]). Context went from 15.625 to 95.833. Ten of eleven evaluated categories improved versus the imatrix-only quant. The allocated model retains 96.74% of BF16 reasoning performance at roughly 24% of the source model's size and 2.30× its effective generation throughput. It is not BF16-equivalent across the board, but the amount of quantization damage recovered purely through precision redistribution is pretty wild.