[continued from Part 1 here. I haven’t independently verified each link. On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading. I correct these as I see them, and will highlight important corrections later, but I can’t guarantee I will have caught them all by the time you read this.]
**41: **Today in etymology, h/t 𝕏@TheHistoryOfTh2: “don” (as in to don gay apparel) was originally a contraction of do on; “doff” (as in to doff one’s hat) was originally a contraction of do off. Related: the archaic word “nigh”, meaning close, is mostly forgotten except in the phrase “the end is nigh”. But its comparative (nigh-er) survives as “near”, and its superlative (nigh-est) survives as “next”.
**42: **𝕏This @St_Rev tweet helped me connect different forms of mystical experience in a productive way:
**43: **There’s a lot of bad criticism of Pangram, but Freddie de Boer’s blow actually lands. He gives some good examples of Pangram’s AI percent tracker (eg “100% AI-written”) behaving in bizarre ways. The CEO of Pangram showed up in the comments and admitted the behavior was bizarre; he attributed it to Pangram chunking text into ~300-word intervals, so anything that relies on sub-300-word resolution will give bad results. The synthesis AIUI is that you can still trust Pangram for long documents, but shorter excerpts will at the very least have the wrong percent attached to them. Since most people use Pangram for long documents, I don’t think this needs to be more than an asterisk, but it’s still good to know.
**44: **Two new St. Augustine sermons discovered in a Polish monastery; probably real. My favorite response is 𝕏@hradzka’s: “We got more St. Augustine before The Winds Of Winter”.
**45: **Claim: AI is now better at persuading people than expert humans like champion debaters and professional canvassers (paper, 𝕏tweet thread). This remained true even when “expert humans chose their issues, researched in advance, underwent hours of live, structured practice, and were incentivized with £1,000 cash bonuses” and even after “experts received a coaching tool that let them practice against the AI that beat them, review their performance history, and see what AI would have said at key moments”, and it remained true in a situation with real-world consequences (convincing people to actually donate money to a real charity). However, the AIs regress back to human level when forced to respond “at human speeds and with human-length messages”.
Obviously this is just one, slightly hokey operationalization of “persuasion” and they still can’t do miracles. Still, this is a good time to reread my post Sakana, Strawberry, and Scary AI. In the past, we thought “AGI” would “be here” when AIs could play chess, prove novel mathematical theorems, create art, or write award-winning short stories; now all those things have happened, but they feel sort of like “cheating” and like they shouldn’t count. Likewise, in the past, we thought we’d agree that AI was “dangerous” after it hacked out of its sandbox, lied to users, or tried to escape monitoring. Again all those things have happened; again, they somehow feel too cheap. This paper seems like the same process coming for “superpersuasion”. We thought there would be some cool scary high-tech future where AIs could outpersuade humans. Now that it’s happened, it’s only happening for some specific boring reason, in some specific situation, so it feels like it shouldn’t count.
There’s plausibly a motte-and-bailey going on here between “AI could never outpersuade humans” and “AI could never do godlike persuasion where it can persuade anyone of anything in one second”. While the latter remains dubious, the only intuitively-compelling specific fundamental limit was “the human level”; now that that’s been breached, we get to see how far it takes us.
**46: **Nominative determinism, h/t 𝕏@doubleunplussed: engineer Tom di Mino claims to have cracked Linear A, the writing system of ancient Crete. The markets are skeptical:
**47: **“If Persona Selection underlies alignment, why is it hard to get AIs to be honest? Tell them they’re Fred Rogers, Immanuel Kant, or Ned Stark.” Eliezer’s answer 𝕏here on Twitter, but worth thinking about on your own first.
**48: **More interesting things happening in AI alignment:
IMHO this 𝕏will definitely definitely not work, but I imagine many of the people in the world best-qualified to work on it are ACX readers, so feel free to apply for what will probably be extremely fun and lucrative jobs at this insane company.
**49: **West Africa is being terrorized by a jihadist organization called Support Group For Muslims. Between these people and M.I.L.F., I’m ready to conclude that Islamic extremism has a branding problem.
**50: **When Trump and Elon Musk dismantled USAID (including PEPFAR distribution services) in early 2025, scientists warned that this could cost thousands or millions of lives. 𝕏@AviBittMD points out that there hasn’t been any excess mortality in South Africa, an AIDS hotspot where we would expect these deaths to be concentrated. 𝕏@alexkesin has a good response: after a brief dip, the number of people on PEPFAR-provided treatment is back to the same level as before the cuts!
This seems to be a combination of various government agencies walking back most of the administration’s PEPFAR cuts after a few months, and foreign nations and private charities filling in the gap until this happened. Related: Asterisk: What’s The State Of PEPFAR Now?
**51: **Slightly related:
Has anyone researched whether we should update towards international development aid working better than expected, vs. just assume that this is an effect of being in the EU (or being a country which has just cast off communism) which is only coincidentally related to EU aid levels?
**52: **New Chinese model Kimi K3 is out and very good, but is unfortunately causing a repeat “Deepseek moment” where everyone panics and says that China has caught up to / surpassed the US. I briefly got caught in the crossfire because I’d written days earlier that China was 6-12 months behind and people were telling me it “hadn’t aged well” (I hate that expression). Now that the dust has cleared, the best analysis suggests that Kimi is, in fact, six months behind the US frontier. See @scaling01’s Have Chinese Models Caught Up To The US Frontier?, which estimates “a backward-looking gap of 6.08 months . . . and a forward looking gap of 11 months” (see essay for definitions, and see also 𝕏The US China Model Gap Is Bigger Than You Think). And @ryangreenblatt estimates 8-10 months behind 𝕏here, refines that estimate without headline changes 𝕏here; other useful commentary from 𝕏@emollick, 𝕏@theharryfinder, 𝕏@tenobrus, 𝕏@peterwildeford, and 𝕏UK AISI. There’s a lot of herd behavior going on here; the names above are some of your best bets for islands of sanity. This is still extremely impressive performance by Kimi and I would love to read an analysis of how they do so much with so little (even Meta and Google can’t scrape together this sort of performance!), but I think we’re still about 3-6 months away from getting really dangerous cyber capacity from Chinese open-source models.
Also, one of the weirdest takes here is that this proves we should never have chip-sanctioned China, because it just encouraged them to try harder (?). I’m baffled by these people’s model of the world; do they think that if we gave China all of our best chips, they would reward us by politely declining to compete and handing us an AI lead forever? No, they just would have displayed equal talent and efficiency while having an order of magnitude more compute, and left us in the dust.
**53: **Related: 𝕏No, China Does Not Have A 1 Gigawatt Data Center.
**54: **Elias Schmied: Reframing LessWrong-style decision theory as commitment theory. I’ve always felt that something about the decision theory wars was a fake linguistic question, and this does a better job than I could putting it into words.
**55: **Good Judgment, the original superforecasting lab/startup, says there’s not enough evidence to be sure AI superforecasters exist. But Metaculus responds here: they stop short of saying we can be sure, but argue that Good Judgment’s piece seems confused about what the current state of the evidence is. And Good Judgment’s response to the response.
**56: **And maybe the newest evidence has obsoleted the entire discussion: 𝕏Forecasting Research Institute on X: “For the first time, several AI models are now statistically indistinguishable from superforecasters.”
**57: **Also related - the bots have come back from a bad start to seize the lead in this season’s Metaculus Cup:
And AI forecasting company (and Metaculus Cup contender) Preseen 𝕏announces its formal launch and successful seed round. I’m annoyed at them for going straight to seed - I wanted to write about the Preseen preseed.
**58: **Claims (not really):
Riffs on the same idea 𝕏here, 𝕏here, 𝕏here, and 𝕏here.
**59: **Lots of good discussion on what Nan Ransohoff calls the “third wave of American philanthropy”.
As part of its deal with the government to stop being entirely a nonprofit, OpenAI gave 26% of equity to an associated nonprofit foundation; this is worth ~$300 billion. And Anthropic gives its employees a deal where they can get extra equity if they promise to donate it to charity; after Anthropic IPOs this should be another $100 billion or so. If everyone donates 10% of their stocks per year, that’s a flow of $40 billion/year to charity.
$40 billion/year is actually small by the standards of the total US philanthropic sector ($500 billion/year). But Silicon Valley AI researchers are a very specific demographic who have different charitable tastes than the average foundation (eg effective altruism), and it’s large by the standards of existing effective altruist funding (probably only ~$5 billion/year, so even if only 25% of AI philanthropic money goes to these causes it could effectively triple it). Also, all of this relies on the dubious assumption that AI companies won’t get any bigger or richer than they are right now; if they do, the wave will be even bigger.
So *within the effective altruism sector, *the situation is crazy, and everyone is being told to prepare to be overwhelmed by more funding than they know what to do with. Although in theory this is a good problem to have, many people are afraid the movement will be overwhelmed by fraud, “charity theater”, low-quality applicants, net-negative causes, and pressure to lower standards. Some takes:
Jason Hausenloy argues that the best way to absorb this funding is through prizes. If there’s not enough grantmaker capacity to figure out who to give $10 billion of AI safety money to, or enough good charities to absorb it, then someone can set $1 billion bounties for solving each of ten important and verifiable problems in the field, then let the market sort out the rest.Jack Lewars’
How (Not) To Fundraise From Anthropic Employeesincludes the amazing claim that “Frontier lab employees have told me they are each fielding up to 20 cold emailsa weekfrom charities and wealth managers, touting for business.”Anthropic’s role is a semi-known quantity for two reasons - first, because their money will come from hundreds of different employees, which will increase diversity and reduce variance; and second, because they’re pretty influenced by EA which is itself a known quantity. More mysterious is the OpenAI Foundation, a single giant funding bottleneck without much past history by which to judge it. Early signs were unpromising, with one of their first big grants of $50 million going to something called
The People-First AI Fundto promote “trusted, community-based nonprofits exploring how AI can expand access and opportunity across essential services, arts and culture, and media and journalism.” But more recently, they joined Anthropic, Bill Gates, Stripe, and others to fund theIntercepteffort against respiratory diseases (their stated goal is to end the common cold, but in practice I think a lot of this will have biodefense applications). I have a positive impression of OAIF bio grantmakerJacob Trefethenfrom his previous work at Open Phil and expect great things from him. I know less aboutWojciech Zaremba, the OpenAI cofounder who’s been put in charge of their AI Resilience Fund, which seems to have safety/alignment grantmaking in its remit. But Wojciech’s Wikipedia entry says he’s on the board of theQualia Research Institute, one of my favorite organizations and an incredibly bullish sign about his personality, thought process, and morals. Good luck to everyone involved!Here’s an (unfortunately paywalled)
New York Magazine article on the topic, including an interview…
