Liked Muse, so I cut the 30B model in half by width, distilled it back, and it does 57 of 60 tool tasks its parent does 60 of

A developer successfully distilled a smaller model, Ornith-1.0-9B, from the Muse-Glimmer-30B parent model. By reducing the parent's width and using distillation, the 14.44B parameter model achieved 57 out of 60 tool tasks, closely matching the parent's 60 tasks. The distillation process involved 6,000 steps and 98.3M tokens over 162 hours, focusing on tool calling fidelity. This smaller model, available under Apache-2.0, demonstrates effective downscaling while retaining significant functionality, though it's not a general replacement for Muse.

Unlike other distillation efforts that aim for general replacement, this project specifically focused on retaining tool-calling capabilities, achieving 57 of 60 tasks compared to the parent's 60.