Agens Volundr 32B Preview: our small team's first model on our own hybrid architecture. Only 18 of 72 layers keep a KV cache (Apache-2.0)

Blockway, a small team in Hong Kong, has released Agens Volundr 32B Preview, their first model built on a hybrid architecture. This model, which uses an Apache-2.0 license, only keeps a KV cache for 18 of its 72 layers. Despite being trained on limited compute, it outperforms Qwen3.8-27B on several benchmarks, including LiveCodeBench v6 (+4.2), HumanEval (+4.3), AIME 2025 (+2.9), and MATH-500 (+1.6). The team encourages users to test the model and report any issues.