[Model] Support for Spark2_5ForCausalLM implementation by KnightYao · Pull Request #27868 · ggml-org/llama.cpp
- Published
- 09/06, 16:36
- Ingested
- 09/06, 23:00
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Spark-X2.5-4B and Spark-X2.5-1.7B are new compact, general-purpose language models from XHToken, designed for practical, efficient, and accessible AI. They offer strong performance across conversation, writing, translation, reasoning, coding, and agentic workflows, achieving leading results among open-source models of comparable size. These models feature an efficiency-oriented architecture with native context windows up to 1M tokens and support over 200 languages. They were trained on Huawei Ascend clusters using advanced algorithms like MOPD to enhance reasoning, coding, and instruction-following capabilities.
We are introducing Spark-X2.5-4B and Spark-X2.5-1.7B, two compact, general-purpose language models designed to make capable AI more practical, efficient, and accessible. The models deliver strong performance across a broad range of everyday tasks—including conversation, writing, translation, reasoning, coding, tool use, and agentic workflows—achieving leading results among open-source models of comparable size. Spark-X2.5 combines an efficiency-oriented architecture with native context windows of up to 1M tokens, and support for more than 200 languages.
- Efficient Architecture and Native 1M-token Context: The models use a hybrid attention architecture that combines one full-attention layer with three sliding-window attention layers. This design substantially reduces the computational overhead typically associated with long-context models while natively supporting a context window of up to 1M tokens.
- Strong Coding and Agent Capabilities: The models are deeply integrated with popular agent harnesses, including Codex, Claude Code, OpenClaw, and Hermes. They deliver state-of-the-art performance among models of comparable size across everyday coding, agentic workflows, reasoning, and instruction-following tasks.
- Broad Hardware and Software Compatibility: The models support a wide range of hardware platforms, including NVIDIA, Huawei, Hygon, HOUMO.AI , etc. It is compatible with leading inference frameworks such as vLLM, SGLang, llama.cpp, MLX, and can be deployed quickly through platforms including Ollama and LM Studio. The models can also be customized using popular fine-tuning frameworks such as LLaMA-Factory. Across multiple hardware platforms, they deliver superior TTFT, TOPT, and overall inference efficiency compared with similarly sized models.
- Advanced Training Algorithms: The models were trained on Huawei Ascend clusters. Large-scale reinforcement learning and post-training techniques such as MOPD significantly enhance its reasoning, coding, agentic, and instruction-following capabilities.