Charles London, 07 Jan 2025
I have a confession to make: I want AGI to be achieved, but I want to be able to contribute, and I don’t want it to be better than me. You know, AGI, but AGI that does the annoying things, not the things that I have set so much store by being able to do. I think it is likely that I have invested too much of my self-worth in being smart, and good at maths, and the idea of that not much mattering any more is a little terrifying. The OpenAI o3 announcement sunk me into a bit of an existential crisis, as we now have the benchmarks to prove that there exists an AI better than me at maths. And I understand that when o3 releases, there will be teething troubles, and it will likely be a little underwhelming relative to benchmarks (as I have found through my own use of o1), but there seems a clear path to a point where one of the skills I most value in myself is automated.
The worst part for me is that, as a theorist, previous empirical leaps didn’t impinge upon my work so much. They were welcome, they made the models more useful for bouncing ideas off, and explaining terminology or definitions. Brilliant, I now work faster, but someone still needs to come along and explain why things work, so I can carry on doing my little proofs and theorems. But now, with o3, I can see a future where I am not needed (and that’s much earlier for me than other, better mathematicians). It’s a little ironic that mathematics, the most abstract of the sciences, constructed by humans (for the logical positivists, and at least discovered by humans, for the Platonists), is likely one of the first to be automated. If humans were not on Earth, the forces of physics would still exist, chemical processes would still occur, and there would be other biological organisms, but mathematics and its theorems would not. Higher mathematics is not independent of humanity, and up till now was largely verifiable only by human mental processes. It is this inherent verifiability via thought alone that has led us to this point, however, with RL bringing us to the precipice of my ability.
While the theory of deep learning has often struggled for practical relevance (with notable exceptions, shout-out Greg Yang), there is still something exciting and interesting about prodding at the magic box to see how it works. To misquote a favourite film, “neural networks are like onions, they have layers” (multi-level joke right there), and peeling back the layers to feel like you understand something is a great feeling. Sadly, what you often understand is only valid in the specific situation you have understood it, and it doesn’t scale. And, oh boy, is it all about scale now. Scale that I can’t compete with, or sometimes even run, with ARC’s pitiful computational resources, and scale that opacifies and blurs, that makes the magic box more magical, and even less amenable to prodding. But prod I do, with a weak torch in the thick haze, and I enjoy it. But there’s now another glimmer of light in this darkness, but not another torch, a train; the train of the models being better at mathematically understanding itself than I, and running me over.
And so we come to the analysis paralysis. I am not immune to propaganda, and perhaps I’ve fallen for the OpenAI FUD. But here we find ourselves, and with the following questions: what do I do, what do I work on, and what is my worth in this new age? Again, we find ourself in fog, but this is miasma, oppressive and choking. I want to continue with theoretical work, because it’s what I enjoy, but I can’t shake the feeling that I’m wasting my time. The grass seems greener in empirics, but for how long, with advances in coding ability? I can see few paths forward:
Maybe I’m not “cracked”, and am lacking the “agency” that is currently all the rage on Twitter, but I feel stumped. For now there’s ICML deadlines, but I’m not sure my heart’s in it.