Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

This guide is a fully public, inexpensive recipe for making a small model substantially better at structured-output compliance. We fine-tune LFM2.5-350M with Group Relative Policy Optimization (GRPO) using the TRL library and evaluate it on the IFStruct benchmark . The full run takes around 500 samples and 100 training steps, small enough for a free-tier Colab or Kaggle GPU, and is available on GitHub . The results show that even a light fine-tuning procedure improves performance from 22.6% to 29.7% on the IFStruct benchmark.

Structured output is one of the most common real-world tasks for LLMs, yet most benchmarks fold it into broader reasoning or extraction scores rather than measuring it on its own. Whether a model reliably returns valid, parseable output in the requested format and shape — schema compliance — is often what decides whether it can be wired into a downstream system at all.

Note that the training pipeline described here is not the one used to train the RL model described in the IFStruct blog . This notebook doesn't aim to recreate the IFStruct benchmark score, but to show how task-specific fine-tuning of smaller models can improve performance and match that of far larger models.

Prerequisites

This guide has two halves that run in different places:

- Fine-tuning runs on a GPU. The accompanying notebook is sized for a free-tier Colab or Kaggle GPU.

- Evaluation can run locally on a MacBook (here, a MacBook Pro with an Apple M5 Max and 36 GB of unified memory) through llama.cpp, which exposes an OpenAI-compatible server that the IFStruct evaluator talks to.

We will need uv for the Python tooling and llama.cpp for serving. Following the Liquid AI llama.cpp deployment docs , install llama.cpp with Homebrew and verify that llama-server is available:

brew install llama.cpp llama-server --version

IFStruct Evaluation on LFM2.5-350M (Base model)

Before we begin, let's evaluate LFM2.5-350M on the IFStruct benchmark and see whether we can reproduce the reported score of 21.1%.

IFStruct is a benchmark for testing the validity of LLM outputs and schema adherence. The benchmark is open-source in Liquid4All/ifstruct , with the public benchmark dataset available on Hugging Face at LiquidAI/ifstruct-v1.0 .

git clone https://github.com/Liquid4All/ifstruct.git

For the eval comparison, we serve the model locally on the MacBook with llama.cpp. We will use the BF16 GGUF ( LiquidAI/LFM2.5-350M-GGUF ).

Then we start the base-model server with the following command:

llama-server \ -hf LiquidAI/LFM2.5-350M-GGUF:BF16 \ -c 32768 \ -np 4 \ -ngl 99 \ --alias LiquidAI/LFM2.5-350M \ --host 127.0.0.1 \ --port 8080

- --alias: model name IFStruct sends to the OpenAI-compatible endpoint

- -ngl 99: asks llama.cpp to offload all layers to the GPU when available

- -np 4: serves four requests in parallel

- -c 32768: size of the prompt context

Once the server is running, we can run the full benchmark with 2000 samples:

uv run ifstruct-eval \ --model LiquidAI/LFM2.5-350M \ --base-url http://localhost:8080/v1 \ --api-key dummy \ --dataset data/test.jsonl \ --results-file results/lfm2.5-350m-llamacpp-base.json \ --n-threads 4 \ --max-tokens 2048 \ -v

============================================================ Model: LiquidAI/LFM2.5-350M ============================================================ Overall: 452/2000 passed (22.6%) Average latency: 1453ms

By format: JSON: 180/1000 passed (18.0%) YAML: 272/1000 passed (27.2%)