OpenAI no longer recommends SWE-Bench Pro

OpenAI has retracted its recommendation for SWE-Bench Pro, a coding evaluation benchmark. This decision follows concerns about the benchmark's reliability and its ability to accurately assess coding capabilities. The company is now advising against its use, suggesting that it may not effectively differentiate between signal and noise in evaluating coding performance. The announcement has generated discussion online, with 56 points and 20 comments on Hacker News.