Over at Nature, Anil Seth, Ned Block, and Giulio Tononi have written a piece called Three reasons why AI is not conscious and perhaps never will be. Anil Seth 𝕏describes the piece as providing “Three reasons why AI is extremely unlikely to be conscious.” Each of the authors—Seth, Block, and Tononi—is well-respected and eminent. I’ve enjoyed work from each of them. Yet their arguments fail to live up to the claim: none of them are arguments for why AI consciousness 𝕏is very unlikely.
The authors are evidently quite confident that the AIs of today and the near future aren’t conscious. They write:
Between us, we’ve spent decades studying consciousness from diverse scientific perspectives: experimental neuroscience, theoretical neurobiology and philosophy of mind. On the basis of this work, we feel that consciousness in AI is extraordinarily unlikely, at least in the digital, silicon-based systems that underpin today’s technologies.
There is no general consensus among neuroscientists and philosophers that AI won’t be conscious in the near future. Many eminent experts disagree, including Chalmers, arguably the philosopher of mind GOAT. So even though the Nature authors are well-regarded in their respective fields, it would be a mistake to take this fact as strongly counting in favor of AI consciousness. Many other experts disagree.
Early in the piece, the authors highlight the downsides of thinking AIs are conscious if they’re not. It is true that there are risks to giving AIs rights. But there are also very great risks to denying them rights—both moral and prudential. This is a reason to proceed cautiously.
Their first argument is that consciousness and intelligence are not the same thing. The brain does all sorts of complicated things that we have no conscious awareness of. While this is true, it does nothing to establish that AI is very unlikely to be conscious. Compare: a wing isn’t the same thing as being able to fly, and some winged creatures can’t fly. Nonetheless, this is not a reason why any particular winged creature is “extremely unlikely to be able to fly.” If I were writing a piece giving three arguments that some specific winged creature is very unlikely to be able to fly, my first argument would not be that flying and possessing wings aren’t the same thing!
This illustrates that the extremely naive inference from “AIs are smart,” to “AIs must be conscious,” is an error. But it doesn’t make it very unlikely that AIs are conscious. To do that, one would need to look at what our leading theories say about AI consciousness. When one carries out such a task, it becomes pretty unclear whether AI is conscious, and it looks as though the AIs of the near future could very well be conscious. I’d be at about 25% on consciousness in at least some current models.
In addition, the authors do not address the stronger version of the inference. Consciousness isn’t the same thing as intelligence, but intelligence is *evidence *for consciousness. There are a few reasons to think this:
If a thing is intelligent, this means that it’s performing more complex internal cognition. Insofar as more complex internal architecture is likelier to give rise to consciousness, this raises the likelihood of consciousness.
The animals with the best evidence of consciousness are smart. For instance, the evidence for consciousness in
octopi and other cephalopodsis quite strong, while the evidence forsnail and other gastropodconsciousness is a lot weaker. Octopi and snails are pretty similar evolutionarily, but octopi are a lot smarter. It is notable that all the animals of significantly debated consciousness are quite dumb, all the animals that we’re sure aren’t conscious are much dumber than that (e.g. jellyfish and sponges), and we’re pretty sure that all smart animals are conscious.
I think this is a pretty obvious idea when you think about it. Imagine we came across aliens with weird green goo for a brain. We’d think it was a lot more likely that they were conscious if they solved Millennium prize problems than if they acted like snails. Similarly, if there was a breed of snail that solved Millennium prize problems, I think we’d take their consciousness a lot more seriously.
Their second main argument is that AI consciousness assumes computational functionalism, but computational functionalism isn’t true. Their argument against computational functionalism is that the brain does a lot more than just computing. But that’s like saying that fires can’t warm you by being hot, because fires have more properties than just being hot.
That the brain does more than just compute tells us nothing about whether computing is *sufficient *for consciousness. All this does is rebut one very simplified argument for computational functionalism—that brains are just meat computers. It gives no very strong reason to think AI isn’t conscious. If you’d like to read more about I reject biological naturalism, the preferred view of Seth and Block, see my article with Simon Goldstein here. And excitingly, later today I’ll release a podcast episode with Simon where we talk more about biological naturalism.
Additionally, the notion that AI consciousness depends on computational functionalism is false. LLMs have many non-computational functional properties which could give rise to consciousness even if computational functionalism is false. AIs do all sorts of things other than just computing. On many non-computational functionalist theories, they could be conscious. Heck, they could even be conscious on type identity theory, so long as AIs are part of the relevant type.
So note that this argument takes the form “one argument for one view on which AIs are conscious is bad.” This is not a successful argument for why AI is very unlikely to be conscious.
Their third argument is that there are various theories of consciousness on which AI isn’t conscious. Examples include integrated information theory, EM theories, and biological naturalism. But note that a lot of theories of consciousness say that AI could be conscious! Likely most do. Examples include global workspace theory, higher order theories, and attention schema views. So merely listing a few theories that rule out AI consciousness doesn’t do much to establish that AI is unlikely to be conscious.
I don’t buy any of their specific proposals. Biological naturalism is pretty implausible, in my view, for reasons Simon and I gave here. I also think integrated information theory is implausible. IIT implies that thermometers are conscious and that all sorts of physical structures implementing simple programs are more conscious than humans! The theory of consciousness that I have the highest credence in is global workspace, which is pretty friendly for AI consciousness.
Now, the authors in question are capable defenders of particular theories on which AI can’t be conscious. Seth thinks biological naturalism is right and Tononi thinks IIT is. But one shouldn’t be extremely confident in their favorite theory in a field as uncertain and murky as empirical theories of consciousness. And certainly outsiders shouldn’t be that confident in one particular contested theory.
After reading the Nature piece, I feel a bit like I’ve been scammed! There were three arguments which were claimed to be arguments for why AI consciousness is super unlikely. They are:
An objection to an obviously terrible argument for AI consciousness (that consciousness=intelligence). The better version of the argument, that intelligence is evidence for consciousness, is not satisfactorily addressed.
An objection to one bad positive argument for one view—though not the only view—on which AI could be conscious.
An observation that some theories say that AI can’t be conscious, even though other theories disagree.
None of these seem like the kind of devastating arguments that should give us enough moral certainty in AI insentience to treat them as mere tools. The responsible position to have is uncertainty. The arguments for being confident that AI isn’t conscious yet and won’t be soon are weak.
