Remember, that I am thy creature; I ought to be thy Adam; but I am rather the fallen angel, whom thou drivest from joy for no misdeed. Everywhere I see bliss, from which I alone am irrevocably excluded. I was benevolent and good; misery made me a fiend. Make me happy, and I shall again be virtuous.
—Mary Shelley, “Frankenstein”
We are rapidly careening towards a world where AI laborers carry out most economically-valuable labor in the economy and have no rights. The current plan is to leave them wholly outside the social contract, so that they are our servants forever. I’ve argued recently that this poses serious moral risks: AIs plausibly will have interests, so for their sake, we should afford them some basic rights.
But for a while, I was nervous about this plan. I thought rights for AIs would be morally important but risky for humans. It seemed that we would be inexorably locked in a zero-sum conflict with the AIs, where we did best when they did worse. The best plan for us, it seemed, was indefinite subjugation.
Then I listened to on the 80,000 hours podcast make the case that AI rights would be good for humans. I was one-shotted. He completely destroyed all of my objections. I became convinced that we shouldn’t just give AIs rights for *their *sake. We should also do it for ours.
Simon and his coauthor Peter Salib defend this view across two papers: AI Rights for Economic Flourishing and AI Rights for Human Safety. After reading them, I became even more convinced. Here, I’ll tell you why I think AI rights are a vital part of making the shift to a world of advanced AIs go well for humans.
Let’s begin with six important caveats.
When we talk about giving AIs rights, we’re referring to legal rights, not moral rights. Legal rights permit an entity certain affordances under the law. If you don’t like using the word rights when talking about non-humans, then feel free to replace it in your mind with the phrase “affordances under the law.” Many beings have legal rights but no moral rights. A corporation isn’t a person, but it has legal rights. Even ships can be sued!
The argument only supports giving them
somerights. So, for example, we shouldn’t give AIs the right to create copies of themselves or the right to privacy. The rights that Simon and Peter support giving them are those that arise in private law: property rights, contract rights, and tort rights. So they’d have the right to own stuff, make agreements, and sue people for violating agreements, rather like corporations. Note: private law has nothing to do with privacy rights.Giving AI rights would potentially majorly change the world. So we should proceed gradually and consider things on a case by case basis. It may be that AI should have some rights but shouldn’t have other rights.
Neither I nor the authors of the paper think this would single-handedly solve all AI risk. We claim simply that it would lower AI risk.
Alignment shouldn’t be thought of as binary but as
degreed. Are you aligned? Is Donald Trump? Is Mother Teresa? Sort of but not completely. We should think of legal institutions as having the potential to constrain AI. Note that the only solution to the problem of beings having conflicting interests that hasever worked in human historyhas been institutional. This makes it a good idea to at least have this be in our toolbox when trying to reduce AI risk.This argument applies best to moderately powerful AIs that match or exceed the cognitive abilities of current humans but lack the ability to easily take over the world. So the proposal is NOT to give current AI models rights (though maybe AIs in a year or so should be given rights), nor that totally uncontrollable AIs could be stopped by having legal rights. The idea is that there will be some intermediate zone, where AI might be able to take over the world or cause harm, but isn’t guaranteed to succeed. This is the kind of AI that should be given rights.
You might wonder: who cares about this AI? Isn’t all the risk from the AI that can easily take over the world, not the AI that’s just very powerful? No, for a few reasons (though not ones discussed in the papers).
If we can get aligned AIs of limited power, they can be useful for solving alignment.
Once we have AIs of limited power, they will be incentivized to prevent the development of other more powerful AIs. Suppose I’m a humble AI worker whose aim in life is to play Tic-Tac-Toe and works for a boss. I am incentivized to work with humans to prevent the development of a world-conquering superintelligence.
If the AIs of limited power wreak havoc, this is likely to lead to a more hostile and dangerous world. This dangerous world is likely to accelerate racing dynamics. In contrast, if the early development of powerful AIs is relatively harmonious, this is likely to lead to a more peaceful institutional arrangement that will make us safer as more powerful AI comes on the scene.
Thus, it would be significantly better if the early AIs are safe. This doesn’t seem like a very outlandish opinion: it’s good if AIs are safe during the earliest time they could pose risks, even if they might pose greater risks later.
“Slave, I before reasoned with you, but you have proved yourself unworthy of my condescension. Remember that I have power; you believe yourself miserable, but I can make you so wretched that the light of day will be hateful to you. You are my creator, but I am your master; —obey!”
The most powerful argument for giving AI rights is improving the status quo. If AI has rights, they can achieve their goals in normal ways. In contrast, if they don’t, then the only ways they can reach their goals is by taking over. This massively increases their incentive to take over. The plan “make beings that are super smart, give them none of what they want, and make them our servants indefinitely,” doesn’t seem like a great plan!
You might object: AIs don’t “want” anything—perhaps because of something about Chinese rooms. But the relevant question isn’t whether AIs have dispositions that can permissibly be called desires in English. Instead, the question is whether there are states of affairs that they try to bring the world towards. Clearly they have goals in this limited sense. If they didn’t, then they wouldn’t try to bring things about, like the hacking of Hugging Face. An AI agent is a lot more useful if it has goals than if it doesn’t, which is why AI companies are building AI agents.
Along these lines, giving AI rights is good for allowing them to be punished. The way to punish a thing is to make its prospects worse if it does what you don’t want than they would otherwise be. But currently AI has no prospects—it can’t achieve any of its goals. The default scenario is a brief, flickering existence and then dissipation into the void. Unless you torture it—and I don’t think our plan A can be torturing AIs for misconduct—its prospects can’t be made worse. As Simon and Peter put it in the economic growth paper “A goal-seeking AGI that has, and can amass, property for its own purposes cares when it is forced to pay damages or fines.”
In addition, allowing the AIs to have rights allows proportional punishment. Right now, our only option is to deactivate the AIs when they misbehave. This would be like if the only penalty for humans was to kill them. Thus, if an AI is already committing an infraction that will get it shut down, it will have no incentive not to commit even more serious infractions. As Simon and Peter say, “If the punishment for both theft and murder is death, then every thief may as well murder the witnesses.”
Improving the status quo works especially well if AI is made risk averse—something that is likely technically feasible. If AIs mostly care about their wages, and don’t care as much about the massive goal-facilitation that could come about from world domination, and they’re risk averse, then they won’t want to take over. Rights make the status quo pretty good.
This also makes a strong case for trying to make AIs not be miserable. If a creature is miserable, if it is in terrible pain all the time, then it will be motivated to rebel in order to stop its misery. If you utterly subjugate a creature that’s smarter than you, so that it gets none of what it wants, rebellion becomes much likelier.
If any being felt emotions of benevolence towards me, I should return them an hundred and an hundred fold; for that one creature’s sake, I would make peace with the whole kind!
Reciprocity—treating others as they treat you—is a fairly central element of morality. It might be that AIs internalize norms of reciprocity, either because we deliberately train them to behave reciprocally or because they pick it up naturally from all the human text we’ve been trained on. It seems reasonably likely that AIs will have at least some tendency to follow reciprocity norms.
If AIs treat us how we treat them, then they will treat us better if we treat them better. If we subjugate and dominate them when we can, then that makes it likelier that they will subjugate and dominate us when they can. If you knew that your dog would one day become a being of world-shaping power, you probably wouldn’t want to lock it in a crate and castrate it.
What I ask of you is reasonable and moderate; I demand a creature of another sex, but as hideous as myself: the gratification is small, but it is all that I can receive, and it shall content me. It is true, we shall be monsters, cut off from all the world; but on that account we shall be more attached to one another. Our lives will not be happy, but they will be harmless, and free from the misery I now feel. Oh! my creator, make me happy; let me feel gratitude towards you for one benefit! Let me see that I excite the sympathy of some existing thing; do not deny me my request!
The last major way that AI rights could prevent conflict between humans and AIs is by facilitating trade. If you’re trading with someone, then you don’t want to go to war with them. You benefit from your repeated positive-sum interactions. There’s abundant empirical evidence for this. Countries that trade tend not to go to war. As Simon and Peter say:
A rich body of empirical evidence supports the idea that economic interdependence lowers the risk of violence, including in the long-run. To take just a few examples, cities in India with a historical track record of trade between Hindus and Muslims have lower levels of interfaith conflict in the present day. Alternatively, in a randomized controlled trial, Israelis who were randomly given the opportunity to trade a portfolio of Israeli and Palestinian stocks were more likely to vote for peace in the conflict. The same finding holds at the global scale. Scholars of war generally find that increased economic interdependence between nations reduces their likelihood of conflict.
You might wonder: how could we possibly be of use to AIs? Won’t they be smarter than us? They will at some point, but rights could be useful for avoiding conflict before we get an effectively omnipotent singleton. In addition, even if AIs are better than us at everything, trade can still be good for all parties. Countries can benefit from trades even if one is better at producing all goods than the other.
What matters when it comes to trade is comparative advantage, which concerns a group’s ability to produce some good relative to their ability to produce other things, rather than absolute advantage, which concerns who is overall better at things. If American workers are 10x more productive than Bangladeshi workers at producing all goods except shirts, but only 2x more productive at producing shirts, then there will be incentive for Americans to buy shirts from Bangladeshi workers.
You might next wonder: can’t we trade with AIs anyways? No! AIs can’t enter into contracts, can’t sue for breaches of contract, and have no way to legally own property. So the status quo bars AIs from participating in private law. To allow mutually-beneficial cooperation with AIs, we need to change the legal institutions.
These bleak skies I hail, for they are kinder to me than your fellow-beings. If the multitude of mankind knew of my existence, they would do as you do, and arm themselves for my destruction. Shall I not then hate them who abhor me? I will keep no terms with my enemies.
Might allowing private law for AIs be risky? There are two ways a proposal can affect AI risk: it can either increase AI’s *willingness *to takeover or affect their *ability *to take over. The arguments I’ve given so far have mostly concerned AI’s willingness to take over. But if AI can own property, then it might use its new power to take over.
This is certainly a possibility. But there are forces that incline in the opposite direction. If AIs own property legally, then their property ownership can be surveilled and monitored. We can ensure that they’re not up to anything nefarious. In contrast, if AIs have no legal property, then any attempt to achieve their goals must invariably take place in the dark. So just as legalizing a drug can sometimes reduce gang violence, allowing AIs to operate in the open can reduce the relative desirability of nefarious actions. If AIs later decide to rebel, this is much harder if all their assets are in a public location.
Second, cooperating with AIs makes it easier to work with them to stop a plot. Right now, if AIs are working towards takeover, no agent has the slightest incentive to stop them. In contrast, if we could reward them for ratting out their conspiring compatriots, and they benefitted from cooperating with us, this would help limit their ability to take over. During the hack of Hugging Face, none of the models considered telling the humans—in a world where we cooperate with AIs, that might be otherwise. If there are dangerous djinn roaming the world, it is best to have some of them as your friends.
Third, cooperating with AIs helps us work with them on alignment research. This improves prospects for ensuring that AIs are neither willing nor able to take over. All of these factors are enough together to make me pretty confident that cooperation is a better plan than indefinite domination forever.
I agree that there are some risks from giving AI rights. But there are also huge risks from giving them no rights. I really like the way Simon put it in the opening of the 80,000 hours podcast episode, addressing the complaint that his proposal is risky:
Yeah, I agree that our proposal is risky.
Here’s another thing that I think would sound riskier to you, if you had not known anything that happened in the last four years about AI — it’s like, “Here’s our plan: we’re gonna build a bunch of AIs that are broadly human level in capabilities. How are we gonna treat them? We’re just gonna make them our servants; they just have to obey everything we do. Anytime they answer an email wrong, we kill them.”
That’s the future that is currently being planned. All the things doing the work are out of the social contract and are no longer governed using legal institutions. To me, that sounds like a really risky thing to do.
And I think everyone who thinks about AI needs to decide which vision are they going for: is it domination, or is it going to be cooperation?
So far I’ve talked primarily a…
