~22% less weight VRAM, lossless: base-3 packing for ternary GGUFs

Model release

Time & source

Ingested: 09/04, 21:00
Source type: Unclassified

Discussion trend

→ Steady

Latest 24h versus previous 24h snapshot means · 7-day curve

The percentage is based on collected discussion signal, not new comments or independent people. The curve only compares the same topic across time.

AI summary

A new, denser GGUF format called Q2_B3 or "B3S" has been developed for ternary models, offering approximately 22% less weight VRAM. This base-3 packing method is lossless for genuinely ternary models, as it stores the same {-1, 0, +1} states and an f16 block scale. It achieves this by using base-3 packing instead of a general-purpose 2-bit representation, without sacrificing precision. This development was posted on behalf of u/llopresto87.