Up to 3.2x Faster Inference with LFM2.5-DSpark
热度趋势
百分比基于当前可用热度信号，而非评论数或独立用户人数。
官方发布带来Hugging Face 模型更新信号，适合跟踪能力变化、生态影响和后续落地。
LFM2.5-DSpark草稿模型检查点已发布，适用于LFM2.5-1.2B-Instruct、LFM2.5-2.6B和LFM2.5-8B-A1B模型。这些检查点引入了推测性解码路径，可在不改变输出质量的情况下，将解码速度显著提高高达3.2倍。此改进仅需极少的内存增加，并在各种基准测试中提供了显著的性能提升，平均每秒令牌速度提升了2.10倍和2.54倍。
Today, we release DSpark draft model checkpoints for three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality:
- Faster inference: up to 3.18 throughput improvement on a GPU and up to 2.87x on-device.
- Toward on-device agentic inference: cuts function-calling latency by 57% on average for LFM2.5-2.6B
- Day-one support for llama.cpp and SGLang: LFM-compatible DSpark integration is open-sourced upstream
How does DSpark work
The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them all in a single forward pass, sharing the cost of loading the weights across all tokens we verify.
Over the years, multiple approaches of speculation have been proposed, with the most prominent being EAGLE-3 , DFlash , and, most recently, DSpark , which combines three components:
- DFlash-style parallel backbone conditioned on the target model’s context features, producing hidden states for all draft tokens in a single forward pass.
- A lightweight sequential head, modeled as a Markov chain between neighboring tokens, that adds inter-token dependency, raising the acceptance rate at later positions.
- A confidence-scheduled verifier that predicts each token’s survival probability and prunes low-confidence suffixes when verification would cost more than it saves.
Training and Architecture
We follow the DSpark recipe with a larger and more diverse data mix covering SFT, chat, code, and function-calling data. Based on our ablations, the first versions of the draft models are simplified attention-only draft models, with 5 layers and a block of 9. For each draft model, we ran 15 epochs on the entire dataset and selected the epoch with the highest acceptance rate rather than the lowest loss.
The resulting draft models are relatively small, with each around ~300M parameters.
Component LFM2.5-1.2B-Instruct LFM2.5-8B-A1B LFM2.5-2.6B
Decoder stack (5 layers) 241.2M 241.2M 241.2M
Hidden-state projection 21.0M 21.0M 21.0M
Markov head 33.6M 65.5M 65.5M
Norms + confidence head 27.5k 27.5k 27.5k
Total 295.7M 327.7M 327.7M
Quality parity
Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model's own token takes its place. The emitted sequence is therefore identical to baseline greedy by construction, so benchmark accuracy (pass@1 or exact match) is unchanged.
Inference Speed Up on CPU and GPU
Our DSpark draft models for LFM2.5 ship with day-one support for llama.cpp (implementation builds on top of the official codebase , which we run with experimental metal kernels ) and SGLang (implementation builds on the official SGLang implementation of DSpark ).
We measure on-device throughput with llama.cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens. We measure GPU throughput with SGLang on a single H100 80 GB in BF16. Both configurations use a DSpark block size of 9, a batch size of 1, and a temperature of 0. We evaluate them on five benchmark datasets.