Benchmarking calories evaluation with LLMs
A developer benchmarked LLMs for calorie evaluation from meal pictures and descriptions, noting that model size doesn't directly correlate with performance. The Qwen 3.8 Max achieved 32% within 20% with a -11 kcal mean bias, while Muse Spark 1.3 reached 48% with a -24 kcal bias. Notably, Muse Glimmer 30b outperformed Qwen 3.8 27b, suggesting that the "best" model for consumer hardware depends on the specific task.
- Published
- Sep 7, 2026, 05:56
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Times shown in UTC
More details
I wanted a quick calories counter for myself, using LLMs to evaluate the calories from pictures of meals + descriptions.
I needed to pick a model so I made a quick benchmark.
The setup was:
- Nutrition5k photos for photo + calories: https://github.com/google-research-datasets/Nutrition5k
- A tool with access to calories information from USDA FoodData Central + MEXT
- I evaluated models based on how many of the meals they managed to have under 20% of error
- All on the same randomly picked 25 meals.
Models too big for my machine were run through OpenCode Go/OpenRouter. I've also included Spark 1.3 since it'll supposedly be open weights.
Results
Model % within 20% Mean bias Median Error Qwen 3.8 27b 16% +64 kcal 148 kcal GLM 5.3 Flash 28% +18 kcal 65 kcal Qwen 3.8 Max 32% -11 kcal 48 kcal Muse Glimmer 30b 32% +25 kcal 92 kcal Qwen 3.8 Flash 36% +2 kcal 91 kcal DeepSeek v4 Flash Vision 40% +52 kcal 65 kcal Muse Spark 1.3 48% -24 kcal 45kcal I know it's not the most scientific benchmark, but it's interesting to see that the order is not really linked to model size.
The most interesting for me is how Muse Glimmer 30b trounces Qwen 3.8 27b here. I think it highlights how "the best" model on consumer hardware (~32Gb VRAM) really depends on the task.