llama.cpp Open PRs list - CPU/RAM/Disk/Hybrid Related - Better for CPU-only & Hybrid inference
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
llama.cpp社区正在积极推进改进，目前仅剩50个开放的Pull Request，有望在年底前实现更快的推理速度。其中一项重要进展是为CPU添加了Maple 20B-A1B三元MoE架构，这体现在PR #27000中。这些努力主要针对CPU-only和混合推理场景，对于没有强大GPU的用户来说尤其有利，旨在提升CPU/RAM/Disk/Hybrid相关的推理性能。
Folks! We're just 50 PRs away from more faster inference. Hopefully by end of year.
- [Discussion] RFC: MoE expert cache, VRAM caching of hot CPU-resident experts with hybrid hit/miss execution #24528
- ggml-cpu: add AVX-512 and VNNI paths for Q5_K/Q6_K dot products- #27590
- ggml-cpu: add x86 VNNI Q2_0 dot product -- 3x speed improvement for VNNI-compatible CPUs- #26348
- ggml-cpu: Added RVV VLEN=1024 vector dot product (vec_dot) kernels for quantized types.- #25397
- Improve performance of ggml_gemv_q4_K_8x8_q8_K for +12-23% tok/s on AVX-VNNI systems- #23309
- ggml-cpu: optimize ggml_gemm_q4_K_8x8_q8_K interleaving/staging for AVX-512 (and AVX2)- #22525
- ggml/cpu: skip zero-scale blocks in TQ1_0 and TQ2_0 vec_dot kernels- #23439
- ggml-cpu : fix riscv xtheadvector builds and add a q1_0 vec dot kernel- #23009
- Optimize reduction stage of dot product of q4_L/q5_K to q8_K on AVX2- #22181
- ggml: optimized runtime for x86 cpu backend and Q4_K quantized weights paired with Q8_K activations - #18495
(Just had some extra time, so went through almost entire Open PRs of llama.cpp. For Poor GPU Club mainly)