CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token by ynankani · Pull Request #27621 · ggml-org/llama.cpp

查看原文

Llama模型发布

热度趋势

趋势数据积累中

百分比基于当前可用热度信号，而非评论数或独立用户人数。

推荐理由

Llama 相关模型动态已经出现，适合跟踪能力变化、生态影响和后续可用性。

AI 摘要

A recent pull request, #27621, by ynankani for ggml-org/llama.cpp, aims to extend MOE fusion to specdec. Previously, MOE glu fusion and topk-router fusion were restricted to a single token. This change is expected to significantly speed up MTP for MoE models, particularly with draft widths greater than 1, according to initial observations and benchmarks.