I audited 443 GGUF quants across 25 repos. 64 of them can't be the quant their filename claims.
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
Llama 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。
对 25 个存储库中的 443 个 GGUF 量化模型进行审计后发现，其中 64 个与其声称的量化类型不符。此问题出现的原因是 k-quants 要求张量行可被 256 整除；如果不能，llama-quantize 会默认使用约 4.5 bpw 的类型，同时保留原始的低位文件名。例如，Nemotron-3.5-Lightning 的所有四个 IQ2 级别，尽管标签介于 2.06 和 2.56 bpw 之间，但实际测量值为 4.58 bpw，这表明存在显著差异。
TL;DR: k-quants need tensor rows divisible by 256. When they aren't, llama-quantize quietly swaps in a ~4.5 bpw type and the file keeps its low-bit name. I audited 443 quants across 25 repos; 64 are affected. On Nemotron-3.5-Lightning all four IQ2 rungs are the same 4.58 bpw file under four different names. Tool and full census linked at the bottom.
Your quant's filename tells you what the quantizer was asked to make. It doesn't necessarily tell you what ended up in the file.
K-quants and i-quants need the first tensor dimension divisible by 256. When it isn't, llama-quantize substitutes a compatible 32-block type instead, often IQ4_NL for i-quants or Q4_0 for k-quants. Either way you can end up around 4.5 bits per weight instead of the low-bit type you requested. That's intentional, it's been in llama.cpp since PR #3747 in 2023, and the quantizer does print a warning. The catch is where the warning goes: into the quantize log. If you're downloading the finished GGUF you never see it. The filename still says IQ2_XXS, the model card says IQ2_XXS, and the metadata still describes an IQ2_XXS recipe.
https://preview.redd.it/mx32ahg9c6mh1.png?width=2320&format=png&auto=webp&s=b5183bfe1a1a109e202eb5800a1f4630f1d94e40
Above: every k/i-quant rung in bartowski's Nemotron-3.5-Lightning repo, claimed bpw vs measured. Two other makers uploaded the same model and got the same result, which is the first clue that this is the tooling rather than the uploader.
I wanted to know how far it spreads, so I wrote a tool that reads the tensor table and reports what's actually in the file. Works on a local GGUF or a whole HF repo. For remote repos it uses range requests to pull just the headers, usually a few MB, without downloading tensor data. One Python file, stdlib only, no pip install.
- Nemotron-3.5-Lightning: n_embd is 2688 and the expert widths are 1856 and 3712, so about 99% of the parameters are forced into fallback types. All four IQ2 rungs are labeled between 2.06 and 2.56 bpw and all four measure 4.58. Four names, effectively the same density, across what looks like a 2.2x range.
- Qwen3.8-Flash-Next: 51.9% of parameters forced into fallback types. The file labeled UD-IQ1_S at 1.56 bpw measures 3.28.
- Nemotron-3-Super-120B: 18 of its 23 quant rungs contain fallbacks. That makes four affected repos in the Nemotron-H MoE family.
- byteshape's Qwen3.6 quants: the filenames report measured bpw and my independent measurements match. Best labeling practice I found anywhere.
Every maker with an affected repo in my census also has a clean one using the same pipeline. The model's tensor dimensions decide this, not the maker.
That's why I don't think this is about careless uploaders or misleading model cards. The requested recipe is valid. The quantizer runs successfully. The fallback is intentional. Nothing in the finished file tells you that most of the recipe couldn't be applied.
Practical takeaway: on a fallback-dominated model, the lowest labeled rungs may not buy you the size reduction you think they do. If IQ2_XXS and IQ2_M land at the same measured density, there's not much reason to pick between them by filename, and you may be better off taking the honestly labeled Q4_0 or IQ4_NL and dropping the guesswork.
I'm not the first to notice the behavior. Issue #26616 asked for a --no-fallback option after someone got a 24.5 GB file where they expected about 18. What was missing was the measurement: how often it happens, which architectures it affects, and how much of each model is involved.
Background: the PR that added the fallback https://github.com/ggml-org/llama.cpp/pull/3747 and the open request for a fail-fast flag https://github.com/ggml-org/llama.cpp/issues/26616
I've got a follow-up coming on what can be done about the affected models, because "just use the 4.5 bpw file" isn't a satisfying answer when the whole reason you wanted a low-bit quant was to fit a 16 GB card.
(Reposting this one, the formatting on my first attempt came out mangled and unreadable. Sorry to the two people who commented/saw it.)