New agentic benchmark: Session-Bench compares what 10 coding harnesses preserve after the work is done
热度趋势
趋势数据积累中
百分比基于当前可用热度信号，而非评论数或独立用户人数。
这条记录涉及编程工具或代码能力更新，适合开发者评估工作流变化和可复用价值。
一项名为 Session-Bench 的新型代理基准测试，旨在评估 10 种编码工具在任务完成后保留了哪些内容，这与仅衡量任务完成度的 SWE-bench 不同。目前，Pi 以 18/19 的分数领先，OpenClaw 以 17/18 紧随其后。Claude Code 和 Codex 并列 12/18。…
SWE-bench measures whether an agent completed the task. Session-Bench measures what the harness preserved afterward.
Every coding run leaves behind a session record: prompts, decisions, tool calls, command output, costs, failed approaches, reasoning summaries, and subagent work. After hundreds of sessions, that record becomes part of the project's history.
I built Session-Bench after encountering these differences while developing a local session browser. It compares 10 CLI session formats across 19 practical gates covering completeness, readability, stability, openness, and tooling.
- The same small probe produced a 1.5 KB session in Pi and roughly 101 KB in Kimi Code.
- Some harnesses preserve readable reasoning or summaries. Others store sealed reasoning, signatures, or no rationale.
- Several formats are readable today but offer no stable contract for tools that need to read them tomorrow.
Pi currently scores 18/19, OpenClaw 17/18, and Claude Code and Codex tie at 12/18.
This is not a coding-quality ranking or a recommendation to choose one harness over another. It is a report card for an overlooked part of coding-agent infrastructure: whether the resulting work history remains useful, inspectable, and portable.
The benchmark has limitations. It covers CLI session stores, not complete desktop or IDE behavior. Observation windows vary, and some measurements could not be completed. The raw probe artifacts are not publicly archived yet, so v0.3 is documented and mechanically scored but not fully independently reproducible. Copilot's documentation verdict is also marked as disputed.
I created and maintain the benchmark. I'd especially like feedback on the standard itself: what should a responsible coding-agent work record preserve, and which of the 19 gates would you change?
https://jazzyalex.github.io/agent-sessions/bench/?campaign=reddit&ref=r-chatgptcoding-discussion-v03
Moderator note: This standalone informational/discussion post was approved in advance by the r/ChatGPTCoding moderation team.