Granite 4.2 LLMs: How They're Built
热度趋势
百分比基于当前可用热度信号，而非评论数或独立用户人数。
官方发布带来Hugging Face 模型更新信号，适合跟踪能力变化、生态影响和后续落地。
Granite 4.2 推理模型家族的构建方式已通过技术演练详细说明，该家族包含 3B、8B 和 30B 密集模型。这些模型共享超参数的通用主干，例如 SwiGLU MLP 激活和 RoPE 位置嵌入。…
A technical walkthrough of how we built the Granite 4.2 reasoning model family.
Authors: Granite Team, IBM
TL;DR: Granite 4.2 is our first family of dense, decoder-only reasoning LLMs, released in three sizes: 3B, 8B, and 30B. Each model is pre-trained from scratch on roughly 15T tokens with a five-phase strategy that extends the context window to 512K tokens, supervised fine-tuned on chain-of-thought, reasoning, and agentic-trajectory data, then post-trained with a multi-stage reinforcement learning pipeline. That pipeline includes agentic RL, where the 8B and 30B models learn to act with tools inside real sandboxed environments. Every model has a thinking / non-thinking switch, a low-effort thinking mode that spends a short reasoning budget on easy questions, and native tool calling. All Granite 4.2 models are released under the Apache 2.0 license.
Links:
- Granite 4.2 HF Collection
- GitHub Repository
- Granite Docs
Overview
Granite 4.2 is the reasoning-focused release of the Granite language-model family. Earlier Granite releases were strong instruction-following assistants; Granite 4.2 adds explicit reasoning. Every model can produce a chain of thought before its answer and can run in thinking or non-thinking mode depending on how much deliberation a task needs. A low-effort mode falls between the two, spending a short reasoning budget on easy questions.
The three sizes (3B, 8B, and 30B) share the same architectural design and follow the same training pipeline (pre-training from scratch, SFT, then multi-stage RL), each at its own scale. All three are strong reasoners and instruction followers. The clearest capability split shows up in post-training. The 8B and 30B models additionally go through an agentic RL block that teaches them to operate as agents: calling tools, editing and running code, driving a terminal, and searching the web inside real environments. Every model supports native tool calling. Served through an OpenAI-compatible endpoint (for example, with vLLM), it emits tool calls in the OpenAI function-calling format and plugs into agentic harnesses without extra glue. Granite 4.2 is also supported in SGLang, see the SGLang cookbook for a ready-to-serve recipe.
The rest of this post walks through the build: architecture, pre-training, supervised fine-tuning, the multi-stage RL pipeline, and results.
Model Architecture
Granite 4.2 models are built on a decoder-only dense transformer architecture with the following core components:
- Attention: Grouped Query Attention (GQA) with 40 attention heads and 8 KV heads
- Position Embedding: Rotary Position Embedding (RoPE) with θ = 10,000,000
- Feed-Forward: MLP with SwiGLU activation
- Normalization: RMSNorm (ε = 1e-5)
- Embeddings: Separate input/output embeddings (not tied)
- Precision: bfloat16
Component 3B Dense 8B Dense 30B Dense
Embedding size 2560 4096 4096
Number of layers 40 40 64
Attention head size 64 128 128
Number of attention heads 40 32 32