Differences Between Fable 5 and Fable 5.1 on MineBench
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
MineBench 上的一位用户指出了 Fable 5 和 Fable 5.1 之间的差异，并表示更喜欢 Fable 5 的某些构建。他们观察到 Fable 5.1 的某些构建，例如 Astronaut，与 Opus 5 的风格相似。这种相似性引发了对 Fable 5.1 编码风格的担忧，该用户表示，如果 Fable 5.1 采用 Opus 5 的“乱码英语”方法，他们会非常失望。
- Average JSON Size: 34.07 MiB (largest 88.76 MiB) - Roughly comparable to Fable's 5 average of 30.65 MiB
Despite no change in API pricing, Fable 5.1 was nearly 3x as expensive as Fable 5 on MineBench. With roughly 2x the inference time, much of that difference appears to come from substantially longer reasoning.
The price increase is quite significant considering Anthropic advertises the same API prices, though it still is massively cheaper than GPT 5.6 Sol Pro (the current top model on the leaderboards). I find that quite interesting as in my personal usage, GPT 5.6 Sol is extremely efficient with my Pro subscription, though MineBench benchmarked 5.6 Sol Pro and not the standard Sol variant ^^
There are some builds/styles I (personally) liked better from Fable 5. To me some of Fable 5.1's builds, like the Astronaut, are much closer to Opus 5's style which makes me curious about what it's like coding with Fable 5.1; I'd be very disappointed if Fable 5.1 adopted the Opus 5 style of gibberish english 😭
Also, it was really interesting to see how Fable 5.1 actually was the first model to create genuinely recognizable interiors! Here's a video showing the interior of Fable 5.1's cottage build (you can see a bed, table, bookshelf, and fireplace) – you can explore any build now on MineBench by clicking the joystick icon in the voxelBox header :)
It's been a while since I've done a full comparison post, so here's some quick highlights of things I've added to the benchmark that were requested:
- Gallery that allows anyone to showcase their generated prompts publicly - You can also regenerate official MineBench prompts to see how the nondeterministic results vary
- API costs were getting expensive, so I thought this would be a great way to account for prompt saturation; anyone can upload any (difficult) prompt and look at all how all the models perform!
- Accounts and sign ins to save your generations and upvotes - Signed-in accounts also have unlimited Gemini 3.7 3.8 Flash generations (thank you DeepMind!)
- A MineCraft like explorer for all builds, allowing you to walk/fly around builds in first person
- **If you enjoy these posts please feel free to help** **fund** **the benchmark** - All funds are currently going directly towards API costs for benchmarking new prompts
- **Alternatively, if you have the API credits, please feel free to add prompts and generations to the gallery and post them around!** - This is actually preferable to donations to me directly, the hosting expenses and whatnot I've always been able to cover out-of-pocket, just the API costs were hard to cover 😓
Essentially it's a benchmark that tests how well a model can create a 3D Minecraft-like structure.
So the models are given a palette of blocks (think of them like legos) and a prompt of what to build, so like the first prompt you see in the post was a fighter jet. Then the models had to build a fighter jet by returning a JSON in which they gave the coordinate of each block/lego (x, y, z). It's interesting to see which model is able to create a better 3D representation of the given prompt.
The smarter models tend to design much more detailed and intricate builds. The repository readme might help give a better understanding.