Real-SWE: Benchmarking AI models on private, real-world, enterprise codebases
A September 2026 benchmark, "Real-SWE," evaluated AI models on enterprise codebases, revealing varying resolution rates and rollout costs. Gemini 3.8 Flash achieved a 31.2% resolution rate at $2.50, while GPT-6 Astra reached 33.8% at $4.67. Fable 5.1 demonstrated the highest resolution at 38.8% with a cost of $6.96. Overall, estimated rollout costs for these models ranged from $2.50 to $6.96.
时间与来源
时间显示为 UTC
显示时区：UTC
本地时区尚不可用，暂时显示 UTC。
发布当时偏移：UTC+02026年9月12日 20:25 UTC
收录当时偏移：UTC+02026年9月13日 02:01 UTC
- 发布
- 2026年9月12日 20:25
- 收录
- 2026年9月13日 02:01
- 来源类型
- 未分类
- 档位
- 社区
- 信源状态
- 正常
档位是按信源手工设定的编辑判断，不是逐条打分。
September 2026
Introducing Real-SWE
Benchmarking frontier AI models on private, real-world, enterprise codebases.
Results Analysis Effort Setup
01 Introduction
Today we are releasing Real-SWE, a benchmark that evaluates frontier AI models on private, real-world, enterprise codebases. Each task comes from a private production codebase that we licensed from a real-world company. These are problems their engineers work on, with all the context and complexity that comes with an existing product.
- Private codebases. Agents must navigate proprietary systems whose code and solutions aren’t available on the public internet.
- Work with business consequences. Getting billing right, calculating taxes, migrating customers. Changes that affect how a business runs, often across multiple services.
- Company-specific complexity. Every company has its own rules and ways of writing code. Agents have to understand those conventions and make changes that work with what’s already there.
Can a coding agent actually do the work of a software engineer in the real world?
- 1
Fable 5.1
Claude Code
Resolution rate: 38.8%
- 2
GPT-6 Astra
Codex CLI
Resolution rate: 33.8%
- 3
Gemini 3.8 Flash
Gemini CLI
Resolution rate: 31.2%
- 4
GLM 5.3