CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token by ynankani · Pull Request #27621 · ggml-org/llama.cpp

View original

LlamaModel release

Heat trend

Collecting trend data

The percentage is based on available heat signal, not comment count or independent people.

Why it matters

Llama model activity is surfacing — worth tracking for capability changes, ecosystem impact, and availability.

AI summary

A recent pull request, #27621, by ynankani for ggml-org/llama.cpp, aims to extend MOE fusion to specdec. Previously, MOE glu fusion and topk-router fusion were restricted to a single token. This change is expected to significantly speed up MTP for MoE models, particularly with draft widths greater than 1, according to initial observations and benchmarks.