Show HN: Gemini 4 Argon plays StarCraft
StarSkirmish, a platform that tracks AI model performance, has diverged from widely reported evaluations like the Artificial Analysis Intelligence Index (ρ 0.26) and Terminal-Bench 4.0. While GPT-6 Astra ranks highly on the former, it only ties for 4th on StarSkirmish. Similarly, Claude Sonnet 5.5 and Muse Wonder Full perform well on Terminal-Bench 4.0 but poorly on StarSkirmish. This suggests that heavily benchmarked evaluations might be targets for model tuning, whereas less targeted, difficult coding tasks still show correlated performance.
This report highlights a divergence in AI model performance tracking, showing StarSkirmish no longer correlates with widely reported benchmarks like the Artificial Analysis Intelligence Index, unlike its previous alignment with Terminal-Bench 4.0.
Time & source
Times shown in UTC
Display time zone: UTC
Local time zone unavailable; showing UTC.
PublishedOffset at this time: UTC+0Oct 16, 2026, 14:18 UTC
IngestedOffset at this time: UTC+0Oct 16, 2026, 15:00 UTC
- Published
- Oct 16, 2026, 14:18
- Ingested
- Oct 16, 2026, 15:00
- Source type
- Unclassified
- Tier
- Community
- Source status
- Healthy
Tier is a per-source editorial setting, not a per-item score.
Discussion trend
The percentage is based on collected discussion signal, not new comments or independent people. The curve only compares the same topic across time.
- Basis
- Running about 2.2× the median of this source's recent listed items
- Triggering item
- Show HN: Gemini 4 Argon plays StarCraft
- Metric comparison
- 192 vs median 88 (20 baseline samples)
- Detected
- 10/17, 03:00
TL;DR
- StarSkirmish is a benchmark where LLMs write bots for StarCraft: Brood War
- Each LLM gets one hour to write a Protoss bot in C++, which is then evaluated against other LLMs' bots and established human written bots
- StarSkirmish tests long horizon reasoning and agentic coding as LLMs must incorporate learnings from practice games to improve their StarCraft bot
The StarSkirmish Bench evaluates how strong of a StarCraft bot can each LLM write given 1 hour of wall clock time. v0.2 updates StarSkirmish Bench v0.1 with flagship models from OpenAI, Anthropic, Google and Meta, each running in its lab's 1st party CLI harness.
An LLM’s score reflects its win rate against a roster of competitive human written bots and demo bots.
StarSkirmish Bench v0.2
1 hour runs, in its 1st party CLI harness. Scores are average win rates (0–100) of each model's 5 runs against a roster of competitive human written bots.
Gemini 4 Argon runs at high, the highest reasoning effort available to it; every other model runs at max.
GPT-6 Astra is the top scoring StarSkirmish Bench v0.2 LLM, a clear step ahead of GPT-6.1 Sol in second. Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1 are functionally tied behind them, all scoring between 37 and 39.
StarSkirmish Bench vs. Cost
StarSkirmish Bench score against the average API cost of one 1 hour run. Cost is on a log scale.
Muse Wonder Full's pricing is based on Muse Wonder's API pricing.
GPT-6.1 Sol offers uniquely good value: it is the only LLM in the most attractive quadrant, scoring second at just over a tenth of GPT-6 Astra's cost per run.
No Signs of Saturating
The strongest human written bots are still well ahead of every model. Against Stardust, PurpleWave and BananaBrain, the best model, GPT-6 Astra, wins only 7% of its games.
Win rates vs the top human written bots
Average win rate of each model's 5 bots against top human written bots Stardust, PurpleWave, and BananaBrain.
1 hour reasoning limits are a factor, but these matches are played on three 2 player maps. Real human written bot tournament conditions are significantly harder.
New AI Model Highlights
Google and Meta's new flagships replace v0.1's Gemini 3.8 Flash and Muse Spark 1.3: Gemini 4 Argon places 3rd, and Muse Wonder Full 7th. GPT-6 Astra and Claude Opus 5.5 were covered in v0.1 .
AI Model Behaviors
Each run's agent trace records how it spent its hour. The strongest predictor of a model's score is how much it practiced against the 7 strongest bots on the roster.
Win Rates vs. Practice Against Strong Bots