AI Anthology

As I noted at this MathOverflow answer (with a concurrence by Bill Thurston), one of the most intellectually satisfying experiences as a research mathematician is interacting at the blackboard with one or more human co-authors who are exactly on the same wavelength as oneself while working collaboratively on the same problem. I do look forward to the day that I can have a similar conversation with an AI attuned to my way of thinking, or (in the more distant future) talking to an attuned AI version of a human colleague when that human colleague is not available for whatever reason. (Though in the latter case there are some non-trivial issues regarding security, privacy, intellectual property, liability, etc. that would likely need to be resolved first before such public AI avatars could be safely deployed.)

I have experimented with prompting GPT-4 to play the role of precisely such a collaborator on a test problem, with the AI instructed to suggest techniques and directions rather than to directly attempt solve the problem (which the current state-of-the-art LLMs are still quite terrible at). Thus far, the results have been only mildly promising; the AI collaborator certainly serves as an enthusiastic sounding board, and can sometimes suggest relevant references or potential things to try, though in most cases these are references and ideas that I was already aware of and could already evaluate, and were also mixed in with some less relevant citations and strategies. But I could see this style of prompting being useful for a more junior researcher, or someone such as myself exploring an area further from my own area of expertise. And there have been a few times now where this tool has suggested to me a concept that was relevant to the problem in a non-obvious fashion, even if it was not able to coherently state why it was in fact relevant. So while it certainly isn’t at the level of a genuinely competent collaborator yet, it does have potential to evolve into one as the technology improves (and is integrated with further tools, as I describe in my article).

In my opinion, the ordering of risks in the near and medium term will be: “Autonomous AI malfunction” << “Humans using AI incorrectly” < "Socioeconomic disruption caused by AI adoption" < "Beneficial uses of AI shut down due to AI panic" < "Malicious humans" < "Malicious humans assisted by AI". There are already examples for instance of AI being used to improve the quality and targetting of spam emails, phishing scams, and the like, and we should definitely be focusing a significant amount of effort to defending against such threats and building community resilience to AI-generated disinformation, as well as cyberresilience to infrastructure attacks that may well be AI-assisted in the near future. The Presidential Council of Advisors on Science and Technology (PCAST) that I serve on is currently working on both these topics and is seeking public input (see here and here respectively). I believe that efforts such as these will also help build defenses, expertise, and “antibodies” that will be able to manage and contain the future risks of malfunctioning autonomous AI in future decades, and may in fact be the most practical and grounded way to arrive at such workable defenses.

I don’t think that unassisted error checking will be feasible any time soon, but one can already use AI tools (such as ChatPDF) to ask other questions about a given paper, such as a summary of the main results and how they are proved. (See also the discussion on this recent blog post.)

One related direction where some progress is likely to be made in the near future is in using LLMs to semi-automate some aspects of formalizing a mathematical proof in a formal language such as Lean; see this recent talk by Jason Rute for a survey of the current state of the art. There are already some isolated examples in which a research paper is submitted in conjunction with a formally verified version of the proofs, and these new tools may make this practice more common. One could imagine journals offering an expedited refereeing process for such certified submissions in the near future, as the referee is freed to focus on other aspects of the paper such as exposition and impact.