Free Creations of the Human Mind

Whatever you might think of the risks, dangers, and harms of AI—many of which are real and worth taking seriously—there’s no question, to my mind, that AI will materially contribute to the development of science. The canonical example here, for good reason, is Google DeepMind’s AlphaFold. But I expect things are going go very much beyond that. At minimum, AI systems are going to make it easier to organize information and to write software, and both of those tasks are crucial aspects of modern scientific practice.

The larger and still undecided question is what AI will be able to do on its own. Helping a lab scientist organize her work, write her code, and run her experiments is one thing. Generating novel and accurate theories about the way the world works is quite another. 

The gap between the one and the other ability is well recognized by AI scientists. Demis Hassabis, the CEO of Google DeepMind, has proposed that true AGI—artificial general intelligence—will not have been achieved until an AI system can generate the kinds of theoretical scientific breakthroughs that have been produced by our best human minds. One test he’s proposed is giving an AI system all of scientific knowledge up to 1911 and seeing if it can come up with the theory of general relativity.

The question is particularly pressing for large language models, which is what most people mean these days when they talk about “AI.” At a very rough level, LLMs work by exploiting the relationships between symbols—what look like words to us and what look like mathematical tokens to the models. Their training (technically, their “pre-training”) involves creating a multidimensional meaning-space that allows the relationships to be described in mathematical terms. Their use—what happens when we send a prompt into Claude or Gemini or ChatGPT—involves predicting new words/tokens based on those relationships. (Yes, there’s a whole other phase of “post-training” that involves reinforcement learning, which complicates the “next-token predictor” picture somewhat. No, it’s not relevant here.)

That sounds complex and technical, and at some level it is. But it should also be familiar to anyone who’s spent any time in the near vicinity of Saussurean linguistics. (Current and former Theory-heads: that means you.) LLMs are something like a proof of concept of the way Ferdinand de Saussure thought signs signified: meaning is established not by the relation of a word (verba) to a thing (res), or by some intrinsic connection of between a signifier (“tree”) and a signified (the concept of tree). What makes meaning, for Saussure and in LLMs, is the differences between (which is to say, the relationships among) signs. In each case, we have a closed system of signs that doesn’t depend on contact with the actual world for meaning/reference.

It’s worth noting that not everyone agrees with Saussure on this point, just as not everyone is willing to grant that LLMs generate meaningful language. (See Searle’s Chinese Room, or Emily Bender’s octopus operator.) I’m very much on Saussure’s side of the argument when it comes to ordinary language, but while reading an excellent essay on Albert Einstein by Ray Monk this morning, in the July 23 LRB, I realized that I’m much less sure about the question when it comes to scientific knowledge.

Here are the paragraphs that caught my eye:

In explaining his own philosophy of science, [Einstein] used the phrase ‘free creations of the human mind’ on several occasions. The first seems to have been in ‘Geometry and Experience’, a public lecture he delivered in 1921 at the Prussian Academy of Sciences. In it, he addresses the question of whether mathematics, whose truths are absolutely certain and indisputable, can furnish us with knowledge about the world. His answer, in brief, is that ‘in so far as the laws of mathematics refer to reality, they are not certain, and in so far as they are certain, they do not refer to reality.’ In this spirit, he distinguishes ‘axiomatic geometry’, such as the system outlined in Euclid’s Elements, from ‘practical geometry’, which supplements such systems with ‘real objects of experience’. The axioms are free creations of the human mind, and in axiomatic geometry the words ‘point’, ‘straight line’ and so on stand only for empty conceptual schemata; what gives them substance lies outside mathematics. When these conceptual schemata are supplemented by objects of experience, the geometry that results, ‘practical geometry’, is ‘evidently a natural science; we may, in fact, regard it as the most ancient branch of physics.’ That the theorems of Euclidean geometry follow from its axioms is certain and indisputable, but the question of whether the practical geometry of the universe is Euclidean or not can only be answered by experience. Without this view of geometry, Einstein told his audience in Berlin, ‘I would have been unable to formulate the theory of relativity.’ If we are to understand the world of experience, he added, ‘we must abandon Euclidean geometry,’ and being able to do so depends on its axioms’ being ‘free creations of the human mind’.

The alternative view, the one adopted by Newton and his contemporaries, is discussed by Einstein in the Herbert Spencer Lecture which he delivered, in English, at Oxford in 1933…. The scientists of Newton’s time, Einstein said, were for the most part convinced that the basic concepts and laws of physics were not free creations of the human mind but were instead logically derivable from experiments and sensory experience. It was ‘the general theory of relativity which showed in a convincing manner the incorrectness of this view’. Experience can guide us in our choice of mathematical concepts, but it can’t possibly be the source from which they are derived. The source, rather, lies in our minds, our imaginations, and ‘the truly creative principle resides in mathematics.’

In the distinction Einstein is making I think you can hear an echo of Kant’s famous distinction between analytic and synthetic knowledge. But you can also—assuming I’m not getting too far over my philosophical or scientific skis here—see a way to think about Hassabis’s AGI test. On my reading, at least, Einstein’s own brief for “the free creations of the human mind” suggests a reason to doubt that LLMs, on their own, will ever be able to pass that test. An LLM can produce remarkable results from a given meaning-space. As we have seen recently, for instance, LLMs are generating mathematical results that people who know real math appear to be quite astonished by. But if Einstein is correct, then the crucial question for a scientific breakthrough may involve knowing when to abandon a meaning-space like Euclidean geometry in the first place. Because they have no access to “the world of experience” that might prove impossible for pure LLM-based models.

None of this is news to people working on advanced AI. As I understand it, a form of this objection is one of the reasons that some AI researchers have argued that LLMs are ultimately going to be a dead end, and that research should focus more heavily on so-called “world models.” Other people, including Hassabis, have argued that AGI might require pairing LLMs with other kinds of technologies, including some, presumably, that would give access to something like “the world of experience.” 

It’s also possible that Einstein was wrong, as he was apparently wrong about quantum mechanics. Or, far more likely, that the tenuous connections I’m drawing among very different areas of thought are at best rough analogies that shouldn’t be used for drawing firm conclusions. I’ve underestimated LLMs before, and by now I’ve stopped being surprised at the myriad ways they manage to surprise me.

Still, at least for myself at the moment, it seems fruitful to keep that notion of “free creations of the human mind” steadily in view. How much of what we know or want to know depends on the intellectual freedom that Einstein alludes to? How important is the “human” part of that phrase? I’m honestly not sure, but I’m very interested to find out.