Show HN: Gemini 4 Argon plays StarCraft
StarSkirmish, a platform that tracks AI model performance, has diverged from widely reported evaluations like the Artificial Analysis Intelligence Index (ρ 0.26) and Terminal-Bench 4.0. While GPT-6 Astra ranks highly on the former, it only ties for 4th on StarSkirmish. Similarly, Claude Sonnet 5.5 and Muse Wonder Full perform well on Terminal-Bench 4.0 but poorly on StarSkirmish. This suggests that heavily benchmarked evaluations might be targets for model tuning, whereas less targeted, difficult coding tasks still show correlated performance.
This report highlights a divergence in AI model performance tracking, showing StarSkirmish no longer correlates with widely reported benchmarks like the Artificial Analysis Intelligence Index, unlike its previous alignment with Terminal-Bench 4.0.
时间与来源
时间显示为 UTC
显示时区：UTC
本地时区尚不可用，暂时显示 UTC。
发布当时偏移：UTC+02026年10月16日 14:18 UTC
收录当时偏移：UTC+02026年10月16日 15:00 UTC
- 发布
- 2026年10月16日 14:18
- 收录
- 2026年10月16日 15:00
- 来源类型
- 未分类
- 档位
- 社区
- 信源状态
- 正常
档位是按信源手工设定的编辑判断，不是逐条打分。
讨论趋势
百分比基于采集到的讨论信号，不代表新增评论数或独立参与人数。曲线仅用于同一话题在不同时段的比较。
- 判定依据
- 热度约为该来源近期上榜条目中位水平的 2.2 倍
- 指标对比
- 192 vs 中位 88（20 条基线样本）
- 检出时间
- 10/17 03:00
TL;DR
- StarSkirmish is a benchmark where LLMs write bots for StarCraft: Brood War
- Each LLM gets one hour to write a Protoss bot in C++, which is then evaluated against other LLMs' bots and established human written bots
- StarSkirmish tests long horizon reasoning and agentic coding as LLMs must incorporate learnings from practice games to improve their StarCraft bot
The StarSkirmish Bench evaluates how strong of a StarCraft bot can each LLM write given 1 hour of wall clock time. v0.2 updates StarSkirmish Bench v0.1 with flagship models from OpenAI, Anthropic, Google and Meta, each running in its lab's 1st party CLI harness.
An LLM’s score reflects its win rate against a roster of competitive human written bots and demo bots.
StarSkirmish Bench v0.2
1 hour runs, in its 1st party CLI harness. Scores are average win rates (0–100) of each model's 5 runs against a roster of competitive human written bots.
Gemini 4 Argon runs at high, the highest reasoning effort available to it; every other model runs at max.
GPT-6 Astra is the top scoring StarSkirmish Bench v0.2 LLM, a clear step ahead of GPT-6.1 Sol in second. Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1 are functionally tied behind them, all scoring between 37 and 39.
StarSkirmish Bench vs. Cost
StarSkirmish Bench score against the average API cost of one 1 hour run. Cost is on a log scale.
Muse Wonder Full's pricing is based on Muse Wonder's API pricing.
GPT-6.1 Sol offers uniquely good value: it is the only LLM in the most attractive quadrant, scoring second at just over a tenth of GPT-6 Astra's cost per run.
No Signs of Saturating
The strongest human written bots are still well ahead of every model. Against Stardust, PurpleWave and BananaBrain, the best model, GPT-6 Astra, wins only 7% of its games.
Win rates vs the top human written bots
Average win rate of each model's 5 bots against top human written bots Stardust, PurpleWave, and BananaBrain.
1 hour reasoning limits are a factor, but these matches are played on three 2 player maps. Real human written bot tournament conditions are significantly harder.
New AI Model Highlights
Google and Meta's new flagships replace v0.1's Gemini 3.8 Flash and Muse Spark 1.3: Gemini 4 Argon places 3rd, and Muse Wonder Full 7th. GPT-6 Astra and Claude Opus 5.5 were covered in v0.1 .
AI Model Behaviors
Each run's agent trace records how it spent its hour. The strongest predictor of a model's score is how much it practiced against the 7 strongest bots on the roster.
Win Rates vs. Practice Against Strong Bots