GPT-5.6 Luna vs GPT-6 Astra: benchmark on 50 real PRs, looking for feedback on the methodology

A benchmark comparing GPT-5.6 Luna and GPT-6 Astra across 50 real PRs from projects like Cal, Sentry, Discourse, Keycloak, and Grafana revealed that Astra found 92 confirmed bugs compared to Luna's 69. However, Luna caught 75% of the bugs at only 3.6% of the cost, with every finding independently verified. The methodology of this benchmark is currently seeking feedback.

Unlike many benchmarks, this report uses 50 real-world pull requests from major open-source projects, offering a more practical comparison of GPT-5.6 Luna and GPT-6 Astra.