[Release] SOTA GGUFs for Qwen3.8-27B: GSQ-RCO at 2.5 to 3.0 bpw
Heat trend
The percentage is based on available heat signal, not comment count or independent people.
New SOTA GGUFs for Qwen3.8-27B are released, utilizing GSQ + RCO quantization methods. These models offer higher quality at the same file size, with both the search and quantizer now learned. The 3.00 bpw version (10.1 GB) matches the base model on AIME25 (100.00) and performs closely on GPQA-Diamond (88.89 vs 89.90) and LiveCodeBench v6 (84.57 vs 85.71). Feedback and requests for further quantizations are welcomed.
We're releasing Qwen3.8-27B quantized with our newest methods, GSQ + RCO. Higher-quality models, same file size, now with the search and the quantizer both learned.
- GSQ (Gumbel-Softmax Quantization): post-training scalar quantization that jointly learns grid assignments and scales, closing most of the scalar-vs-vector gap at 2 to 3 bits while staying fully GGUF-deployable
- RCO (Riemannian Constrained Optimization): assigns a quantization type to every tensor under a strict size budget by gradient descent directly on the task loss, no per-constraint tuning
- Three GGUFs (2.50 / 2.75 / 3.00 bpw, 8.4 to 10.1 GB) plus the vision projector
Uniform quants give every tensor the same precision. RCO finds where the precision actually matters and GSQ quantizes every tensor at its assigned type. The result runs unmodified in llama.cpp, Ollama, and LM Studio.
Results (vs. the BF16 base and Unsloth Dynamic quants) To the best of our knowledge, these are the best size-to-accuracy GGUFs available for Qwen3.8-27B at these file sizes: at every size we evaluated, they match or beat the strongest published quantizations.
- 3.00 bpw (10.1 GB): matches the base model on AIME25 (100.00), within about one point on GPQA-Diamond (88.89 vs 89.90) and LiveCodeBench v6 (84.57 vs 85.71)
- 2.75 bpw (9.3 GB): AIME25 100.00, and its zero-shot average actually exceeds BF16 (75.70 vs 74.34)
- At matched ~8.4 GB size: +10.0 AIME25, +8.6 GPQA-Diamond, +4.6 LiveCodeBench over UD-IQ2_S
- GSQ: paper https://arxiv.org/abs/2604.18556 | code https://github.com/IST-DASLab/GSQ
- RCO: paper https://arxiv.org/abs/2605.00649 | code https://github.com/IST-DASLab/RCO
This is the first of a series: we plan to release more SOTA GGUFs across model families as a lab (ISTA Deep Algorithms and Systems Lab).