Some nuance on the mathematical proofs released by OpenAI

I keep seeing people reacting to the backlash from mathematicians by framing it like the only problem is that the humans aren’t getting the credit. Okay, maybe some people have had that reaction, but it is not the reason behind the backlash. The real issue goes much deeper, and ignoring it is robbing us of models that are more helpful. We really do need things to change here, but to get change people first need to see the problem.

Let me explain:

There’s a lot more to math than writing proofs. For the sake of argument, let’s divide the practice of doing math into five domains: deciding what’s worth asking, proposing a mechanism, establishing a result and its correctness, explaining why it works, and making it comprehensible to others.

Right now, the models are particularly good at establishing that certain results are correct, and I worry that the companies will focus on the first two domains before turning to the last two. That sounds good, but it could make things worse if we end up with systems that can autonomously generate and prove results without making those results comprehensible to the mathematical community. I - and many of the mathematicians we’ve been hearing from - would argue that explanation and communication are among the most important things to focus on next.

This might be a tough pill to swallow. It seems a lot of people haven’t even considered it. Heck, I even saw someone post Terence Tao’s response to the situation only to make the same point I’m arguing against here. What I’m reacting to there is that what Tao was really saying is closely related to the point I’m about to make in the next paragraph.

Without making the proofs comprehensible, mathematicians have to work backwards from the proof to figure out what the result actually means. That’s a significant part of the value of the proof: the understanding it can provide. The mathematicians still have to do all that work themselves though, and it’s the most important part of the job. Moreover, as Tao stated in his post, “all of the valuable followup activities that traditionally flow out of that solution” are at risk of being lost (e.g. talks, seminars, workshops, etc.). Either because the person or institution that generated the results hasn’t yet developed an explanation that others can understand, or because they simply have no interest in devoting the extra resources.

Try putting yourself in the shoes of these mathematicians. Imagine yourself back in school working on a math problem you just don’t understand yet. After bashing your head against the wall, you flip to the back of the textbook to look at the answer key. You get the answer, but that’s it. Your understanding of the problem hasn’t necessarily been helped. You don’t suddenly understand the underlying mathematical ideas that will help you solve similar problems in the future or move on to more advanced mathematical ideas.

In case it wasn’t clear, the answer key is analogous to the AI proofs here.

The lack of insight into what these proofs actually mean is a real problem here. It is not about any one individual’s understanding of a mathematical subject, but about our ability to push the entire field of mathematics forward while humans remain able to contribute to and understand that process.

I hope the companies figure out reward signals for comprehensible math. However, if the models become completely autonomous because the companies devote resources to the first two domains before the last two, that’ll look more impressive to investors and customers who are more concerned with capabilities than advancing our mathematical understanding.

Don’t get me wrong these proofs still showcase huge leaps in model capabilities, hence the flair. Although, that doesn’t change the fact that generating proofs doesn’t automatically advance our mathematical understanding in the same ways that humans working on proofs can. If the companies don’t focus on explanation and communication, the production of mathematical results could outpace our ability to understand and build on them. Does that sound like a good future to you?

The fact that OpenAI did not appear to adequately address the warnings of the math community following the previous release of mathematical proofs shows a worrying trend here.

Edit:

Clarifications and concessions:

Proofs can have significant mathematical value even when they are difficult to understand. The issue is not that such proofs are worthless, but that generating results alone does not guarantee progress in our collective mathematical understanding or the ability to build on said understanding. That work is still manual.

AI-generated discoveries can ultimately advance human understanding, even if the original proofs are opaque. The concern is whether the mathematical community can effectively verify, understand, generalize, and build upon those results.

The concern about autonomous AI and societal dependence is a potential long-term risk, not an inevitable consequence of current developments.

Changes to the original post:

Fixed typos and grammatical issues.

Expanded the four domains into five, separating establishing correctness, explaining why a result works, and communicating it to others.

Rephrased claims that overstated the limitations or implications of AI-generated proofs.