Dario Amodei says there’s a risk of losing control of AI systems. Sam Altman says OpenAl would “melt” all its GPUs if that’s what it took to keep humanity alive. Elon Musk says Dario is right.
AI executives have been on what seems to be a concerted effort to achieve something. Your guess is as good as mine on what that specially is, but I’ll try my best to navigate you through a scenario, which seen Anthropic’s Dario Amodei suggest a few things only to have OpenAI’s Sam Altman and Terafab’s mostly outspoken Elon Musk almost immediately agree. They mostly don’t agree on anything, so the warning signs are well and truly there. In due course, Google DeepMind’s Sir Demis Hassabis and Microsoft’s Satya Nadella chimed in, with support for the call to pump the brakes on AI progress.
“The details need working through, but the direction is correct for meeting this critical moment,” said Hassabis. “Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing,” says Nadella. At this point, worth noting this is all very rich coming from the same set of AI bros who have prophesied the end of millions of human jobs worldwide to sell their own AI models and agents.
Trump administration doesn’t seem to agree
The US Administration might not entirely be aligned with what the AI companies are hinting at, via this word salad.
President Donald Trump in a typically detailed Truth Social post wrote, “ The only control or “guardrails” that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades! The Trump Administration has stopped AI “people” from doing bad, or potentially bad, “things,“ like Dario (Anthropic!), who is now pretending to be a “perfect little angel” - and we will continue to do so! We already have tremendous CRIMINAL and REGULATORY power over these companies! There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so. Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE! Thank you for your attention to this matter! President DONALD J. TRUMP.”
Vice-president JD Vance didn’t mince words. “I have to say, just personally, I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a trojan horse,” he said in a press statement.
House Speaker Mike Johnson told CNN that that the government rushing to place restrictions on AI development could be a ‘national security threat’, adding, “If Congress just races in and does some sort of emergency session to try and regulate AI, we will lose the race to China.”
Two sides to the same coin.
Key trajectories Amodei seems most worried about
AI systems that have the ability to build the next generation of AI, which Amodei says is a dynamic process called “recursive self-improvement.” It is starting to happen across the industry, he points out. Basically, existing AI is building even smarter AI.
Amodei’s core warning is that if left unchecked, our ability to understand and control these systems will be gone at some point.
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage),” the scary proposition we must envision and be terrified of.
Banks and financial institutions, power grids, government systems, who knows, even biological weapons may be fair game for self-improved AI systems that wouldn’t care much for the humans, or the companies that led to their creation.
What do Amodei, and by extension Altman, Musk etc., want?
There seems to be a sense that when the sordid history is finally written, Amodei seems to want to find himself on the right side of it by being the man who warned of it all. ‘Pacing the frontier’ is his grand plan. “To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this,” he explains.
To that extent, he pitches a three point plan. As a starter, Amodei wants every AI company to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.
For this, he wants governments to step in with regulation AND evaluation. Secondly, that frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. And finally, The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.
To be fair, there is a case that open-source AI models can become an untraceable gun if uncensored and un-guardrailed models are allowed to run locally. It could potentially detail how to make drugs, chemical weapons, hack your bank and so on.
Altman pitched in with more thoughts a few days later. “ First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities,” he wrote in a post on X, earlier this week.
Do consider, if you’re as optimistically pessimist as I am
If these AI companies were on the verge of artificial general intelligence (AGI), would they be calling for a slowdown in pace?
Or, if the unreleased models are scaring you lot, enough to think you should slow down, who asked you to rush down this path all these years? What exactly have Amodei, Altman and others seen in the labs, that has them sweating more than the usual huff and puff of the next billion dollar funding?
No need to go on this pretentious positioning that someone must tell you to regulate and evaluate. The Hugging Face incident a few weeks ago didn’t do much to build confidence in OpenAI’s approach to AI development in general. And when an Anthropic engineer Jacob Coxon (who previously worked at OpenAI) resigns with a warning that AI companies
𝕏aren’t acting responsibly.
Amodei is extremely worried that China still has access to powerful AI chips and infrastructure, and I’m sure there is an element of fear that the Chinese companies will reverse engineer these chips as well. He also wants a crackdown on “unauthorised distillation”, but who defines what is unauthorised when AI companies have been collecting data from all across the web to train their models, without a beg your pardon? Basically, this sounds like ‘please ban them because the rapidly evolving Chinese AI models are bad for business’.
Is this a reaction of model performance having started to plateau, in conjunction with compute costs and infrastructure investments becoming more and more expensive? AI data centres have been massively unpopular (particularly with residents in locations where they are coming up), and the backlash has been vocal.
Being vocally suggesting to ‘slow down’ AI development is a good smokescreen to this slowing AI progress. Doesn’t hurt the big narrative, doesn’t slow down funding, valuations remain largely stable as a result of this humanitarian call by the AI execs, and they can regroup to conquer the workplaces after a pause.
With all of AI’s top voices in chorus, let me make one thing clear — this isn’t altruistic. They have either seen AI do something in the labs that’s beyond their control, or they simply have hit a brick wall in terms of capabilities and smartness. I don’t see what is happening in the labs till AI companies release models, but clearly it is one of these two realities playing out in real time — either the AI is getting too smart, or it simply isn’t.
On September 3, for a while OpenAI’s ChatGPT, Anthropic’s Claude, and xAI’s Grok all suffered overlapping service outage. We got varying explanations. OpenAI said the crash was caused by a routing error. SpaceX AI blamed an infrastructure failure at their Memphis Compute Center. Anthropic logged the event as an infrastructure issue. Coincidence? Perhaps.
It is not at all surprising that global AI stocks slipped in trading after this whole spiel about trying to save humanity from a technology that can go rogue at any time.
If you look closely, this whole team Humanity absurdity is essentially being pegged as a negotiation leverage. Get China to agree with our terms. They will absolutely not. Regulate the pace for building AGI. Why? You’re the ones building it, focus less on chasing market share, revenues and funding, and on the real things that matter for once? AI could become so smart that it may take over the world. This is purely an attempt at insulating against massive legal action and perception backlash if an AI entity is to indeed mount an attack to take over some part of the global internet or government systems.
By the way, even when sounding this so-called alarm, the AI bros have been shapeshifting back to form the moment someone asks them nicely. Just a few days ago, Elon Musk said that people have at the most another 10 years to sell their work for money, after which AI will take over everything.
I mean, none of these boffins have ever given any clarity on what changes, except something on the lines of “people learnt when the horse gave way to the car” sort of nonsense. If Musk and the AI brethren want to have AI humanoid robots make cars, but no one in the economy has any jobs, who will buy those humanoid made cars? Oh, we will find a way the same way we found a way when the internet came along…I’m tuned out.
Neural Dispatch is your weekly guide to the rapidly evolving landscape of artificial intelligence. Each edition delivers curated insights on breakthrough technologies, practical applications, and strategic implications shaping our digital future.
