GPT-6 Astra makes a massive leap on ZeroBench (an extremely difficult vision benchmark), surpassing the human baseline across all three metrics

GPT-6 Astra has achieved a significant breakthrough on ZeroBench, a challenging vision benchmark. It surpassed the human baseline across all three metrics: pass@5, pass^5, and "pass@1". The pass@5 metric scores if at least one of five attempts is correct, while pass^5 scores only if all five attempts are correct, serving as a reliability metric. The "pass@1" metric represents the average score across five attempts, not a true single-attempt pass@1.

This marks the first time an AI model has surpassed the human baseline across all three metrics of the ZeroBench vision benchmark, unlike previous models that only achieved partial success.