I recently had a back and forth with Quillette founder Claire Lehmann about digital sentience. My position is that sentience in AI is a live possibility worth taking seriously, though I’d guess AI isn’t conscious yet. Her position is that taking AI sentience to be a live possibility is evidence of profound mental dysfunction:
Lehmann wasn’t the only one to have a view like this. Nathan Robinson also weighed in:
I’m a big fan of Lehmann and find her to be very thoughtful in general, so I found this brazen dismissal disappointing. Yet it seems to be a relatively common view. Many people find belief in AI sentience totally absurd. Here, I’ll explain why I don’t. Overall, my guess is AI isn’t conscious, but I think it’s a live possibility of potential great importance, so it deserves to be taken seriously rather than sneered at.
Note that Lehmann’s position, despite her posturing, is a serious minority opinion among the relevant experts in the field. While most people who have thought hard about the subject and have relevant expertise don’t think AIs are currently sentient, very few treat it with the kind of blithe dismissal that she does. So if taking AI sentience seriously is evidence of serious mental dysfunction, it’s a kind of dysfunction curiously common among the world’s leading experts on the subject. In general, I think you should not be overwhelmingly confident in the falsity of a view seriously entertained by experts.
Take the many worlds interpretation of quantum physics, as an example. I think there are good objections to it—it requires giving up on the highly seductive principle of center indifference. The view also sounds a bit nutty. It implies that new universes are constantly appearing. Still, it would seem crazy to be astronomically certain that the view is false. I definitely wouldn’t go below 5%. This is because it’s taken seriously by the relevant experts and I’m not a relevant expert on the subject!
What I definitely wouldn’t do is say that accepting MWI is like thinking there are real people talking to you from inside the TV or that “unfortunately, physics departments are staffed with lunatics who think new copies of you are appearing constantly,” or “𝕏Yes, it is very implausible the world is constantly splitting. Some of you have lost your minds,” or saying to people who take MWI seriouisly “𝕏Holy shit you're stupid. Just admit you know nothing you peasant brained barnyard animal,” or “we need a slur for people 𝕏who think MWI is true.” If we take ideas seriously, then we shouldn’t dismissively reject any view that sounds weird or has inconvenient implications (as I’m sure you can tell, I’ve been spending way too much time arguing with people on Twitter about this).
I also don’t think digital sentience is that bizarre of an idea. Imagine going back to the year 2000 and telling people about the AIs of today. They can solve novel math problems, write original sonnets, and they have to be trained out of saying that they’re conscious. The AIs today possess many of the behavioral indicators that people in past ages would have treated as indicative of consciousness.
As for their internal architecture, our leading theories render it pretty unclear whether they’re conscious. One report attempting to estimate the probability of AI sentience by looking at internal architectural features concluded:
Overall, we find that the evidence is against 2024 LLMs being conscious, but the evidence against 2024 LLMs being conscious is not decisive. The evidence against LLM consciousness is much weaker than the evidence against consciousness in simpler AI systems.
Another detailed research report, compiled by many of the leading philosophers of mind, neuroscientists, and AI researchers, in 2023, concluded:
We survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher order theories, predictive processing, and attention schema theory. From these theories we derive “indicator properties” of consciousness, elucidated in computational terms that allow us to assess AI systems for these properties. We use these indicator properties to assess several recent AI systems, and we discuss how future systems might implement them.
Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators.
So their guess was that probably the 2023 AIs weren’t conscious, but future AIs might be. Do the AIs of today satisfy these indicators? Who knows?
Now, to be clear, there are some theories of consciousness on which AIs can’t be conscious, endorsed by neuroscientists like Anil Seth. For some back and forth on these sorts of arguments, see here, here, here, and here. But that’s the point. It’s an open question, taken seriously by the leading experts on the subject in the world—both scientists and philosophers. For a person outside the subject with no relevant expertise to confidently declare the answer blazingly obvious is both arrogant and irrational.
What is the amazingly powerful evidence that convinces Lehmann that AI sentience is totally ridiculous? She gave a few different answers.
This is an oversimplification in the extreme. LLMs start out as word predictors. Then they get human feedback which remolds their weights. Plus they go through a bunch of other post-training. So they’re not just next-token predictors. Rather, they start as next-token predictors, then go through a bunch of later steps, leading to a complex internal architecture that enables word predictions.
But this is all irrelevant. Human faculties were selected by evolution to pass on our genes, not give rise to consciousness. Nonetheless, this doesn’t mean there’s no human consciousness. To pass on our genes, evolution furnished us with various sophisticated mental faculties that gave rise to consciousness. Similarly, to predict the next token, AI had to develop complex cognitive architecture. It’s an open question whether this cognitive architecture is sufficient to produce consciousness.
Put another way: predicting the next token is at best what the AI was selected to do. It’s not what it is. Humans were selected to reproduce, but this doesn’t mean that’s all we do. So just like humans being selected for reproduction doesn’t settle the question of whether we’re conscious, AIs being selected for token prediction doesn’t settle whether they’re conscious.
Imagine it turned out, for whatever reason, that the best way to predict tokens was to get an exact digital replica of a human brain. Lehmann’s argument, if successful, would show this replica wasn’t conscious. Yet that’s very likely wrong. So her argument proves too much.
And note: the fact that you can give a reductive description of a physical system doesn’t mean that it’s not conscious. You can also give a reductive description of a human brain; it’s just a bunch of neurons and dendrites blasting electricity back and forth. Nonetheless, we’re conscious.
One final point that I wish people internalized in these discussions: David Chalmers is not unbearably stupid. Nor are Patrick Butlin, Robert Long, Jonathan Birch, Simon Goldstein, Harvey Lederman, or the various others who take digital sentience seriously. So if you think that some consideration is so obviously decisive that anyone aware of basic facts would just know that AI isn’t conscious, probably it is you who are confused. Certainly it is much more plausible that a journalist is confused about philosophy of mind than that many of the leading philosophers of mind are.
I asked Lehmann why she is so much more confident in consciousness in animals than AI. Her answer:
She followed this up with:
Talk about “no evidence” here just seems clearly misplaced. Here is the state of the evidence. We know that some things (human brains) give rise to consciousness. Then we know that AI is like human brains in a bunch of ways. It’s different in other ways. So then there’s a question: are the similarities enough to give rise to consciousness?
Other mammals have all the same brain regions that seem relevant to consciousness. So do birds. So we’re pretty sure they’re conscious. Fish and reptiles are quite different, but display lots of conscious-seeming behavior, so we’re pretty sure they’re conscious too. As for other creatures—LLMs, insects, snails, worms—we’re less sure. We get less sure as the behavior differs from ours to a greater degree and their internal architecture differs to an increasing degree.
In this context, talk about no evidence is just silly. The evidential situation is: we know things of type A beget property B. You can’t see if something begets property B. There are other things of type C that are very similar to things of type A. Thus, we’re pretty confident that they give rise to B too. Then there are other things of type D that are similar in a bunch of ways and different in a bunch of other ways. Two things, then, are clear:
We should think there’s some chance that things of type D give rise to B. After all, we’re not sure if the properties shared between A and D are relevant for giving rise to B.
It’s simply false that there’s “no evidence” that things of type D give rise to property B. The evidence is that they possess properties that are potentially relevant to producing B! When it looks like LLMs might possess something relevantly like a global workspace, or higher-order processing, that is evidence that they are conscious.
One more analogy: imagine we know that birds fly but we haven’t yet seen whether bats do. We observe that they have physical structures kind of like wings in some ways but different in other ways. It would be silly to say there’s “no evidence” that they can fly or declare belief that they can fly unscientific. The sensible belief to have is genuine uncertainty.
If instances of AI possessing features that are relevant to consciousness on our leading theories isn’t evidence, then what could be evidence of AI consciousness?
Lehmann’s other argument just seemed to obviously beg the question. Of course there are differences between living systems and AI. There are also differences between humans and mammals—some even having to do with the brain regions relative to consciousness. The question is whether those differences rule out consciousness. Lehmann gave no argument for thinking they do.
Lehmann’s last argument was linking to this paper:
It is silly to treat one paper as settling the question definitively. If you “read the researchers working on this stuff,” then you should also read the ones who disagree with them. When the experts disagree, you shouldn’t generally, as a non-expert, be ridiculously confident one way.
I found the paper somewhat hard to follow as it’s pretty technical. I’d be surprised, though I could be wrong, if Lehmann understood it in detail. My guess is she cited it because it superficially rhymes with her views on the topic. Various replies have been written. There are also, of course, lots of papers arguing for computational functionalism—the theory on which AI sentience is most straightforward.
Overall, the fact that you can go on Google Scholar and find a paper arguing P is not a good reason to be extremely certain of P and extremely certain that the people who reject P are stupid.
One last argument that seems worth addressing, though I don’t think Lehmann raised it, is the following: AI is just computer code. So if AI is conscious, why not think Substack is conscious—or various other computer programs. But this is like asking: if human brains made of neurons give rise to consciousness, why doesn’t a wall of neurons connected to a light censor that all activate when they detect light generate consciousness?
The answer is that human neurons, though they are the same kinds of things as the neurons connected to a light censor, they do different things. Consciousness is about what a thing does, not just the underlying material. Similarly, even if the underlying material of an LLM is the same as for a computer program, they perform different functions. So treating non-consciousness in standard computer programs as settling whether AI is conscious just reveals a profound ignorance about the standard theories of consciousness.
Thus, I do not find any of Lehmann’s arguments remotely convincing. Many seem to rely on fairly elementary mistakes. In light of the state of our evidence, the sensible thing to do is have some middling credence in AI sentience and for your credence to increase over time. Treating it as a settled question, treating leading experts as morons too stupid to understand basic facts, and sneering at those who disagree is not a serious way to proceed.
Instead of this snarky and blithe dismissal, I admire the attitudes of people like Anil Seth. Seth is pretty confident that AI isn’t conscious because he has a specific theory of how consciousness works that’s incompatible with AI consciousness. But here’s how he discusses the topic:
While some researchers suggest that conscious AI is close at hand, others, including me, believe it remains far away and might not be possible at all. But even if unlikely, it is unwise to dismiss the possibility altogether. The prospect of artificial consciousness raises ethical, safety, and societal challenges significantly beyond those already posed by AI.
While being measured and careful may generate fewer sensational Twitter dunks than declaring those on the other side of a live debate to be like those who believe in the Easter Bunny, it is the prudent way to proceed on a topic of profound ethical significance. It is important, when the stakes are this high, that we try to think carefully about the subject, without ridicule or needless snark. It would be a profound ethical tragedy if we needlessly harmed countless digital minds because we wanted a snarky Twitter dunk.
