Explicit Edit Benchmarks: 6 harnesses x 11 models x 226 tasks
A developer created the explicit-edit-benchmark to evaluate text editing models and harnesses across 226 tasks, 6 harnesses, and 11 models. The benchmark aims to determine which combination performs best in precise text editing, a common coding task. Preliminary findings indicate that the harness significantly impacts model performance, especially for models that follow instructions well, such as open-ai models. For instance, gpt-5.6-luna showed a pass score range between 70.2% and 98.9% depending solely on the harness used.
时间与来源
时间显示为 UTC
显示时区：UTC
本地时区尚不可用，暂时显示 UTC。
发布当时偏移：UTC+02026年9月19日 08:39 UTC
收录当时偏移：UTC+02026年9月19日 19:00 UTC
- 发布
- 2026年9月19日 08:39
- 收录
- 2026年9月19日 19:00
- 来源类型
- 开发者社区
- 档位
- 社区
- 信源状态
- 正常
档位是按信源手工设定的编辑判断，不是逐条打分。
Hi! I've created and been maintaining https://github.com/alexshpunt/explicit-edit-benchmark which tries to answer the question: which model is better, which harness is better and which combination is better overall in a very straightforward task - precise text editing. Most of daily coding is text editing. I've seen when a model for multiple turns couldn't figure out how to express the line it wants to change, so it made me ask myself "Why is that so complicated, it's just a text, right?". Seems like not, taking into account how widely different the same model behaves across different harnesses. My preliminary conclusion is: harness and tooling behind it actually matters! But it matters the most with the models which *can* actually follow the instructions well (e.g. open-ai models), there is a wide range between 98.9% and 70.2% of pass score for gpt-5.6-luna simply depending on the harness!
I've tried to run as many combinations as possible, but my resources are limited. I've exhausted all my quotas and even credits, that's why I'm reaching out to the community, as I think it's a pretty interesting topic and I would be happy to gather even more data, because of the stochastic nature of the runs, it’s only possible to make any conclusion when you have enough of runs.
https://preview.redd.it/ik1qy3ihvfqh1.png?width=1882&format=png&auto=webp&s=48849acf8b379d33839bed61a184d0a519cb97f7