Explicit Edit Benchmarks: 6 harnesses x 11 models x 226 tasks
A developer created the explicit-edit-benchmark to evaluate text editing models and harnesses across 226 tasks, 6 harnesses, and 11 models. The benchmark aims to determine which combination performs best in precise text editing, a common coding task. Preliminary findings indicate that the harness significantly impacts model performance, especially for models that follow instructions well, such as open-ai models. For instance, gpt-5.6-luna showed a pass score range between 70.2% and 98.9% depending solely on the harness used.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
PublishedOffset at this time: UTC+0Sep 19, 2026, 08:39 UTC
IngestedOffset at this time: UTC+0Sep 19, 2026, 19:00 UTC
- Published
- Sep 19, 2026, 08:39
- Ingested
- Sep 19, 2026, 19:00
- Source type
- Dev community
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
Hi! I've created and been maintaining https://github.com/alexshpunt/explicit-edit-benchmark which tries to answer the question: which model is better, which harness is better and which combination is better overall in a very straightforward task - precise text editing. Most of daily coding is text editing. I've seen when a model for multiple turns couldn't figure out how to express the line it wants to change, so it made me ask myself "Why is that so complicated, it's just a text, right?". Seems like not, taking into account how widely different the same model behaves across different harnesses. My preliminary conclusion is: harness and tooling behind it actually matters! But it matters the most with the models which *can* actually follow the instructions well (e.g. open-ai models), there is a wide range between 98.9% and 70.2% of pass score for gpt-5.6-luna simply depending on the harness!
I've tried to run as many combinations as possible, but my resources are limited. I've exhausted all my quotas and even credits, that's why I'm reaching out to the community, as I think it's a pretty interesting topic and I would be happy to gather even more data, because of the stochastic nature of the runs, it’s only possible to make any conclusion when you have enough of runs.
https://preview.redd.it/ik1qy3ihvfqh1.png?width=1882&format=png&auto=webp&s=48849acf8b379d33839bed61a184d0a519cb97f7