The new website “Effective Altruism, in their own words,”** **is a very likely AI-curated selection of quotes supposedly dedicated to “Exposing what you’re not supposed to know,” sourced from the deepest darkest annals of EA writings: public blogposts on the internet. The aim is apparently to explain “Why Effective Altruism should be banned from AI safety, policy-making, and governance.”
The whole thing is a spectacular firehose of easily debunkable nonsense. The piece was written by Data Republican, who worked for the Department of War least as of August. The piece comes conveniently at a time when the DOW is attempting to smear effective altruism. Suspiciously timed to say the least!
In Data Republican’s blog post explaining why she made the website, she complains that EAs might be motivated to “Spend a billion dollars fighting a future AI catastrophe instead of stopping malaria in children.” This is a somewhat ironic criticism given that EAs have done more to fight malaria than basically any group on the planet, and Data *Republican’s *political party is generally *opposed *to efforts to stop malaria in overseas children. It is weird for someone to smear EAs as not adequately concerned with malaria in an effort to defend the anti-EA actions of the present administration. I give some of my money every month to prevent malaria. So do most EAs I know. Does DR?
It’s also beyond ridiculous to treat EA support for action on AI as motivated by Longtermist navel gazing. It’s an obvious attempt to distract from the very commonsensical case for regulating powerful AI and turn it into a referendum on the weirdest thing an EA has ever said. When assessing whether a policy proposal is a good idea, you should consider the policy proposal rather than seeing if some of the people in support of it believe other things you disagree with.
And the case for regulating AI is very strong.
The median AI researcher thought in 2024 that there was a 𝕏10% chance AI will kill everyone. They probably think odds are higher now. It’s certainly true that lots of EAs think that this would be extra bad because it would doom the future, but to treat the view that we should worry about AI catastrophe as dependent on weird future math is to mislead your audience. It is similarly misleading when DR says:
They do not need to document the causal chain under their own philosophy. They only have to establish the probability isn’t
zero.
But obviously AI reduces existential risks in a lot of ways too! The reason EAs are worried about AI isn’t because they think odds are one in ten quintillion AI will kill everyone. It’s because they—like the median AI researcher—think that odds are more like one in *ten *that AI kills everyone. It’s hard to see a race towards building superintelligence in a post Hugging Face world as the kind of thing that we can be overwhelmingly confident will end well.
DR also misleads by suggesting that EAs aren’t worried about slowing AI development, causing China to win the race, because “If pausing Western frontier AI development reduces the probability of extinction, then altering the global security balance is acceptable.” But not all EAs are Longtermists, and there are extremely obvious reasons to worry about a future dominated by China. This is why the pause proposals that EAs tend to advocate for are *explicitly *multilateral.
Many other claims in the article are obviously and straightforwardly false. For example, “EAs would be just as scared in a non-AI world as they are in an AI world.” But we can look at how scared EAs were in, say, 2015 about extinction. The answer: much less scared than they are now. And at that time they still thought AI was the biggest existential risk. Another obviously misleading claim:
Every ideology that has demanded sacrifice and coercion in exchange for future good has been run by a small class of people who claimed to see far enough ahead to bind everyone else to their vision.
But EAs do not “demand sacrifice and coercion in exchange for future good.” If one looks at the things EAs have done in the real world, rather than hallucinate scare stories, it’s mostly involved research on reducing existential risks, sparing children from horrible diseases, and stopping animals from suffering intensely. None of this is some horrible project aiming to conquer the world and establish the USEAR (United Soviet EA Republic). This is hysterical nonsense.
We then turn to the website itself. It mostly has quotes from public blog posts selected to make EAs look bad. Before I comment on the quotes themselves, here is a meta point: if you search the internet for every controversial thing some group has ever said, you will probably find some examples. You could do this for any group. So if we’re serious people—rather than doing a 2020-esque woke smear campaign—then we will not treat the fact that you can cobble together some bad-sounding quotes from random people on a large internet forum as particularly dispositive.
And yet amazingly despite that, DR hasn’t been able to find damning quotes. If this is the best she can find, EAs really are a reasonable bunch. This is remarkable in light of how many EAs there are, EAs’ willingness to embrace counterintuitive conclusions when compelled to by strong arguments, and the fact that a lot of the subjects EAs discuss involve difficult moral tradeoffs with sacred values.
One thing that DR was able to find was some EA bloggers (like yours truly) saying things about ethics that are controversial. It is not surprising that some EAs think weird things about ethics. There are a lot of EA philosophers. Get ten philosophers in a room, you’ll get eleven views, and they’ll all be weird.
Philosophy is hard. There are scores of cases where intuitive principles come into conflict. The fact that EAs think about conflicts between obvious judgments that can’t all be right rather than burying our heads in the sand says something in our favor. If you can’t find any member of a group saying something controversial—even outrageous-sounding—the group is very likely to be intellectually barren and conformist. To think seriously is to entertain weird ideas sometimes.
Onto the quotes themselves.
Some of the quotes are totally banal. Others are quoted out of context to make it sound like they’re saying the opposite of what they say. For example, DR suggests that EAs support a “Technocratic Vanguard,**” **whereby a group of experts maliciously control the future. Here are some of the quotes cited to support this:
It could easily lead to a global oligarchy or dictatorship, because a small group of already-powerful people controls all the AIs that matter.
This obviously is not advocating for global oligarchy. It is *warning *about the possibility of AI-driven global oligarchy. If one says “P might happen, and that is worrying,” that is not advocating for P! Another quote in the same vein:
AI-enabled weapons systems could enable misaligned AIs to take over countries by threatening violence against a small group of important actors and driving them to do their bidding.
A third example involves someone on the EA forum explaining what Plato thought:
Think of Plato’s idea of the philosopher king. That person is supposed to be primarily a philosopher, a lover of wisdom who leads a simple life. So to him, the pursuit of wisdom and pursuit of power over others are contradictory pursuits, and
we should put in charge whoever does the former.
Noting that Plato thinks P is not advocating P.
The examples are supposedly ordered from most convincing to least convincing. Here is the strongest evidence that EAs are vanguardists:
This need not result in compromised epistemics; the direction can still be set by the cool-headed moral vanguard and their various sacred texts and EV calculations, but physical doors could be flung open to wider participation beyond the typical EA crowd.
This is from a post in 2022 from a person no one’s ever heard of with a total of 7 people who voted on. The post is arguing that EA should be a bit more like traditional religion. I do not consider this strong evidence that EAs want a group of experts to maliciously rule the world. Another post cited as evidence that EAs want to build AI to take over the world quotes Oliver Habryka saying:
Any thoughts on whether it would be good for YC to start a Manhattan Project for AI? My sense is we could get many of the top ~50 to work on it, and we could structure it so that the tech belongs to the world via some sort of nonprofit but the people working on it get startup-like compensation if it works. Obviously we’d comply with/aggressively support all regulation.
Problem: Habryka did not say this. This was him quoting Sam Altman. Attributing this to Habryka is a lie! This has been pointed out by Habryka to DR—she did not reply.
This is indicative of a willingness to put quotes in people’s mouths that they didn’t say.
DR also suggests that EAs want a world dictator. I won’t go through all the quotes, as there are too many, but if you just read them it’s obvious that basically none of them are saying anything remotely like what is being alleged. For example, here is one:
When I started going to the Vancouver meetup, it wasn’t really happening. I got inspired by the “communities need heroes” thing and started to just make it happen. It worked. You really do need someone holding the bottom line. When I fail to make things happen, other people don’t reliably step in, things just stop happening. This one keeps getting drilled into me. I still seem to have an aversion to seizing power and stepping on toes, but I’m starting to see that it’s really ok. Go ahead and seize power. Be the benevolent dictator. People like it.
This person is making a comment about how it’s important for people to do things unilaterally when running EA meetups. Then they make a joke about being the benevolent dictator. They are not talking about world domination. They are talking about running meetups. This is obvious to anyone with even basic reading comprehension.
Also, I found it funny that this was one of the quotes that supposedly shows EAs want to install a benevolent dictator:
This is basically just the first class of Political Philosophy 101: it sure would be nice if you could install your favorite benevolent dictator, wouldn't it? Well it turns out you can't, and even if you could that's evil and a very dangerous policy — what if someone else does this and you get a dictator you don't like?
As civilized people, let's agree to give everyone a seat at the table to decide what happens.
It says the opposite.
There is then a section on biting the bullet. The top post showing the depravity of EA moral reasoning is from someone named Abie B who nobody has ever heard of, with a total of 7 votes on her (I assume) post. The post argues for voluntary human challenge trials replacing animal testing. One relevant passage:
There are a lot of reasons why human testing is preferable. Humans can consent; animals cannot. Humans can advocate for themselves if they are being exploited; animals cannot. Humans are a good biological model for how drugs behave in humans; animals are not. Human experiences have high ecological validity for understanding how things will affect humans; animal experiences do not. Humans can describe effects they are experiencing in language to experimenters; animals cannot. Humans can be meaningfully financially compensated for their experience; animals cannot.
Does this seem like strong evidence that EAs are moral monsters? That one EA wrote a blog post arguing for less testing on animals and more voluntary testing on humans—with human consent—in large part because it would help us develop safer drugs? DR also quotes me saying:
I don’t actually think there is good reason to avoid the repugnant conclusion.
I think the repugnant conclusion is simply correct.
I’m a philosophy Ph.D. student. Saying controversial things about ethics is kind of our job.
If EA has a lot of philosophers, it isn’t very surprising that some will hold counterintuitive views. It’s been known for a long time that population ethics is really weird and avoiding the repugnant conclusion is very difficult. Even those in philosophy who reject the repugnant conclusion accept this. We will all have to accept something repugnant. So noting that EA philosophers who spend a lot of time thinking about population ethics sometimes end up thinking one of the weird things rather than another of the weird things is not very informative, except in illustrating that EAs think about difficult problems rather than ignoring them and sneering.
One of my favorite examples showing the evilness of EA is this quote:
The Very Repugnant Conclusion has been shown to be provably true if one accepts a very small set of basic moral/logical axioms, [2] all of which seem intuitively, obviously true to many people. Therefore,
if you want a self-consistent ethical framework, one must either "bite the bullet" on the Very Repugnant Conclusion and accept it as correct, or reject one of the axioms it rests on, all of which would seemingly have far-reaching consequences on other basic moral intuitions.
This is literally just explaining the paradoxes in population ethics. It is not taking a stance on what we should think. It’s just explaining that provably you have to give up something obvious.
DR also quotes GiveWell saying “I believe that invoking “the ends justify the means” (justifying near and clear harms by pointing to their further-out effects) is sometimes the right thing to do, and is sometimes not.” Ooh spooky—some means are justified by some ends, others are not. They also quote Joe Carlsmith saying:
I suggest that the Presumptuous Philosopher is a strong candidate for what I call a “good bullet” : that is, a counterintuitive result, acceptance of which resolves a lot of gnarly issues into a simple and pretty satisfying theory, and rejection of which invites endless complication and counterexample (fans of the repugnant conclusion , take note). This doesn’t mean we should actually bite.
But we shouldn’t die on the hill of non-biting.
Here he is talking about anthropic reasoning which is how you do probabilistic reasoning about your existence. It’s a famously thorny area where we provably have to give up something obvious seeming. It has nothing to do with ethics. Treating this as indicative of EAs predilec…
