For daily news, updates, and analysis, be sure to follow us on Instagram. If you are able to support independent journalism, please make a tax-deductible donation to News Not Noise here.
In the span of a week, anxiety over AI’s progress has hardened into a strikingly unified message from some of the biggest names in tech: the companies building the most powerful “frontier” models need to slow down, sit together, and agree on limits because, they warn, the future of humanity is on the line.
We wrote last week about the Anthropic whistleblower whose warning helped set this off. There’s no shortage of theories about why his message hit so hard right now.
One theory: OpenAI was barrelling toward a monster IPO, then postponed going public, citing the need to focus on safety. Skeptics note that “safety” is a convenient explanation for pumping the brakes at a volatile moment for the business.
Another: AI companies want Washington’s blessing to meet with competitors and jointly decide how fast to build the most powerful models. Antitrust law would probably prohibit that. But they’d ask for a waiver, because of “safety.” This would give the tech CEOs permission to set their own rules of the road.
To get clarity, I made some calls. I’ve spoken with people who work with some of the most powerful figures in AI.
Here’s what I’ve gathered from them:
Yes, the companies have self-interested reasons to sound the alarm now. But the alarm is needed and real. The near-term fear isn’t that AI turns on humanity anytime soon. It’s that increasingly powerful models hand bad actors the tools to cause catastrophic harm first.
Washington has largely abdicated its responsibility to lead, whether that’s because our elected officials think regulations will slow the race with China, have conflicts of interest, or because the technology is moving faster than lawmakers can understand, let alone regulate it.
The people I spoke with emphasize that we need creative solutions and institutions capable of meeting it. One pointed to the International Atomic Energy Agency as a good model — it’s an independent international body filled with nuclear experts who inspect, verify, and enforce limits based on international agreements. Can the same be stood up for AI? And Congress should play a role in setting the rules here in the U.S. (One idea is to require companies to vet who in the public has access to those frontier models.)
The good news is we’ve built international systems to manage dangerous technologies before. We can do it again. The technology is moving extraordinarily fast. That’s an argument for governing it with equal urgency.
In this newsletter: AI leaders have a safety conversion. Houthis hit Saudi’s oil lifeline. Trump blames Ukraine for diesel prices. The math problem with Trump’s $5,000 checks. EPA scraps power plant limits. And remembering Cher’s legendary designer, who died at 87.
**Breaking News:**After a three month absence, Sen Mitch McConnell (R-KY)returnedto the U.S. Senate today. He spoke to reporters from a wheelchair on Capitol Hill and said, “I’m here to work on the Farm Bill.” He has been absent since mid-June and says he’s still in physical therapy but will “do my best to be present for tough votes when our Conference needs me.”Brake Check: Over the weekend, AI leaders intensified calls for a coordinated slowdown in AI development, warning the technology could spiral out of control within months if left unchecked. Anthropic CEO Dario Amodei published anessayemphasizing the dangers and proposing three steps to allow safety regulations to catch up: First, put third-party evaluators or auditors into AI companies, something he said Anthropic will do. (OpenAI CEO Sam Altman latersaidhe will, too). Second, get democracies to agree on safety standards and how quickly the most powerful systems should advance. Third, work with China and other autocratic governments to set shared limits. Altman, Elon Musk, head of Google DeepMind Demis Hassabis, and Microsoft CEO Satya Nadellaquicklybackedparts of Amodei’s proposal or echoed his calls for a coordinated slowdown. But not everyone is convinced. Trumpdismissedwarnings about AI, saying his priority is winning the AI race with China and theonly guardrailthe US needs is “A STRONG AND SMART (high IQ)” president. Beijingslammedthe calls for regulation as “fearmongering” and portrayed it as an effort to hold back Chinese development, which a state run paper called a “Cold War playbook.” U.S. House Speaker Mike Johnsonsaidhe doesn’t want to “rush” into AI regulation andriskletting China to pull ahead. He suggested bringing AI leaders together to talk. (They’ve been having versions of that conversation for years.) Oneprofessor compared itto an arms race: “To get de-escalation there needs to be simultaneous, neutral self-restraint,” a tall order when so much money, power and technological advantage is on the line. Other experts were more hopeful. “It is not too late,”one said. “But given that we don’t know just how quickly the AI is going to develop, it is quite urgent that we be prepared.”Context:Experts generally agree that today’s AI models aren’t powerful enough to wipe out humanity, recentsafety incidentsaside. The concern is where this is headed, and how fast. Developers are alreadyusing AIto help build better AI. The fear is that at some point, an AI system could start improving itself with little or no human help. That’s called “recursive self-improvement.” If a rapidly improving system developed goals that conflicted with ours, that would be “misalignment.” (Fancy word for very bad.) The nightmare scenario is that a super-capable, misaligned AI harms people or wipes out humanity by, say, poisoning our water supply, releasing a deadly virus, or launching drone swarms. People in the AI world have a term for estimating the odds of a doomsday scenario like that: P(doom). There’s a lot of noise around this, in part because estimates of how likely catastrophe is vary widely, from𝕏almost zerotoalmost certain. Critics also point out that you don’t need to wait for robot Armageddon to find reasons for regulation. Bad actors can already use AI fordeepfake pornography,cyberattacks,weapons targeting, andbioweapons development.Yes, But: Many critics are skeptical about the calls for regulation coming from AI leaders. Some argue that these doomsday warnings areeffectively marketingpitches that highlight (or exaggerate) how capable commercial AI models are or will soon be. For example, the CEO of AI chip maker Nvidia recentlysuggestedthat AI leaders were drumming up hysteria in order to sell new cybersecurity products. Others aresuspiciousabout allowing AI leaders to dictate the regulations governing their own industry. They argue it preempts stricter regulation, a conceptcalledregulatory capture, and solidifies AI leaders’ existing advantage. The CEO of Hugging Face𝕏saidAI safety issues “won’t be solved behind the closed doors of a handful of frontier labs.” Still others argue that simply slowing the pace of development isn’t enough. “We don’t just set a slower rate of progress for capabilities and then hope that provides enough time to get the safety right,” one AI expertsaid. “We set the safety requirements and further progress occurs only when they are met.”Murky Progress: The sudden chorus ofwarningsis prompting calls for Congress to regulate AI. (Sen Jon Osoff (D-GA) delivered thispowerful messageabout the need for AI leadership during a campaign rally this weekend.) But the threat isn’t exactly new, which raises an obvious question: Why now? And when did all these tech giants start agreeing? Lawmakers have filed dozens of AI-related bills over the past few years, andnonehas made any progress. Oneproposal, called the AI Kill Switch Act, would require developers to maintain the ability to suspend their models, and let the U.S. government order them to do it in an emergency. Another would require AI companies take steps tomitigate“catastrophic risks” and allow the government to block models it considers unsafe. There’s also agrowing pushtobanAI superintelligence — i.e., AI that’s smarter than humans — and pause advanced AI research until safety rules are in place.
FAFO:Fightingbetween the Houthis and Saudi Arabia isintensifying. The Iran-backed group says itstrucka major Saudi military base. Its attacks on Saudi oil infrastructure also forced the kingdom to shut itscritical East-West oil pipeline, which had beencarryingroughly 5% of the world’s oil supplies and lets Saudi Arabia bypass the Strait of Hormuz. Regional officialssay much of the pipelinewill remain out of service for weeks, sendingoil priceshigher. Saudi Arabia’s Crown Prince Mohammed bin Salman repeatedlyurgedTrump to take military action against the Houthis, but Trumprefused. U.S. forces are stretched thin by the war with Iran, which has damaged military bases across the region and burned through supplies of critical weapons. “The Saudis are really frustrated,” a former U.S. ambassadorsaid. “In some sense it’s their worst-case scenario.” Trump doesn’t seem too worried, though. Heinsistedthat “everything is going to work out fine and dandy.”Context:This is essentially areturn to a civil warthat never fully ended. In 2014, the Houthis, an Iran-backed rebel group based in Yemen’s mountainous north, seized the capital. A Saudi-led coalition, usingU.S. weapons, intervened to support Yemen’s internationally recognized government. Years of warkilledover 150,000 people, including tens of thousands of civilians, with many more dying to disease and famine. It was called the world’s greatest humanitarian disaster for 10 years. The UN brokered a ceasefire in 2022, but the Houthis say that Saudi Arabia maintained apunishing blockade, which theywant lifted. The Houthis say theirattacks are aimedat Saudi Arabia, and insist other ships can safely pass through waterways they control. The renewed fighting hasalready displacedover 80,000 people.
Keep reading to find out…
A supposed energy truce between Russia and UkraineJared Kushner’s brilliant solutionWhat’s happening to Trump’s $5,000 checksWhy more than two dozen states and cities collectively sued the governmentHow the EPA plans to let power plants get gassy**Remembering Bob Mackie’s legendary looks
All that and more is available to paid subscribers. Your subscription supports independent journalism. Thank you.
