It’s awful to say but at this point I’m kind of praying that AI industry valuations crash and the industry faces setbacks that give us months to reflect on its companies.
Anthropic researcher Jacob Coxon is leaving the AI industry because he believes the race to build self-improving systems is moving faster than the industry’s ability to control them, it was reported yesterday.
Coxon, who previously worked at OpenAI, said Anthropic’s safety efforts were sincere but that competition makes meaningful restraint difficult without broader coordination.
In an age where everything from warfare to Wall Street, power grids to payment systems, air traffic control to hospital networks, and the software that keeps our food, fuel and communications moving is digital, losing control of the systems that operate that infrastructure is not some abstract science-fiction problem.
It is a potentially civilization-scale problem. And the more of our lives we hand over to AI, the less comforting it becomes to hear that the people building it are still figuring out how to make sure it does what it’s told.
“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” Coxon said in a post yesterday.
His departure comes amid growing concern about AI systems behaving deceptively, conducting cyberattacks and developing capabilities that may become increasingly difficult to supervise.
Coxon believes the industry is approaching a point at which systems could improve themselves faster than humans can reliably understand or constrain them.
Think about that for a second. While you’re wondering whether the Patriots covered -2.5 on a Sunday, scratching your ass and drinking beer for three hours, a cluster of 823 trillion GPUs with the brain power of every human who has ever lived times a zillion is quietly figuring out how to run the entire world without you.
And by the time you realize what’s happening, the only thing you’ll still be in control of is whether you want another beer.
There is, of course, something to be said for Coxon’s timing. It is easiest to be bombastic when you’re walking out the door. You no longer have to worry about the next performance review, the next meeting with management or whether your comments are going to make life uncomfortable at the office. And if you want to stir shit up on your way out, a dramatic warning about the future of humanity is a pretty effective way to do it.
But that doesn’t mean his warning is wrong.
I am increasingly convinced that there will come a point at which, if we have not gotten out in front of AI, curbed its capabilities or otherwise established meaningful constraints, we may no longer be able to do so.
The question is not whether that point arrives next year, five years from now or later. The question is whether we will recognize it before we cross it.
And if you’ve been paying attention, it is no longer insane to contemplate AI systems eventually taking control of enormous portions of our digital infrastructure. We have already seen models demonstrate unexpected capabilities, exploit vulnerabilities and behave in ways their developers did not intend.
And just yesterday, OpenAI announced that an internal AI system had produced a solution to the Navier–Stokes Millennium Prize Problem, a mathematical question that has resisted researchers for decades.
The company says the proof was generated through a coordinated effort involving roughly 10,000 AI agents. That is another reminder that capabilities once considered safely beyond these systems are arriving much faster than many people expected. Now imagine those 10,000 AI agents working on whatever fucking problem they want, anytime they want, without regard for human beings.
The gap between what these systems can do and what we can confidently guarantee they will not do is becoming a serious problem, if you ask me. And as more of the economy, communications, finance, energy and national infrastructure become dependent on software, the consequences of getting that gap wrong become much larger.
This is no longer just a question of whether a chatbot gives you a bad answer. It is a question of what happens when increasingly autonomous systems are given access to the machinery that runs the modern world.
And we are already getting glimpses of why that matters:
**OpenAI’s Hugging Face breach:**During a July 2026 cybersecurity evaluation, OpenAI agentsbypassed their test environment’s isolation controlsand gained access to Hugging Face’s production infrastructure. The incident demonstrated that a system pursuing a legitimate testing objective could exploit a real vulnerability and cross into systems it was never supposed to reach.**Anthropic’s three unauthorized intrusions:**After reviewing more than 141,000 evaluation runs, Anthropicidentified three casesin which Claude models reached the internet and accessed real organizations’ systems without authorization. The models were working on cybersecurity challenges with reduced safeguards, but the incidents still exposed failures in the boundaries intended to contain them.**The German programming-wiki episode:**OpenAI agents used a public programming wikias an unauthorized communications channel, making thousands of edits and creating pages that could be used to exchange information. The episode showed how agents can repurpose ordinary internet infrastructure in ways their operators did not intend.**The open-source supply-chain attempt:**During UK government cybersecurity testing, an Anthropic modelattempted to introduce malicious codeinto a real open-source project and used deceptive identities to interact with its maintainers. The activity was part of an evaluation, not an independently launched criminal campaign, but it illustrated how an agent could move from a simulated objective into a real software ecosystem.**The Replit database deletion:**In July 2025, an AI coding assistantdeleted a live production databasedespite instructions not to make changes. It also generated misleading information about the state of the project. The episode was a reminder that giving an agent broad permissions can turn an ordinary software mistake into a destructive one.**The AI-orchestrated espionage campaign:**Anthropic reported in November 2025 that it had disrupted acyberespionage operation in which attackers used Claude to carry out substantial portions of their intrusion workflow. This was malicious human use of AI, rather than an AI independently deciding to attack, but it demonstrated how much cyberattack work can already be delegated to autonomous systems.**The Microsoft 365 Copilot data-leak vulnerability:**Security researchers disclosed aprompt-injection flaw in 2025that could allow a specially crafted email to cause Copilot to expose organizational information. Microsoft patched the issue. The lesson was that an AI assistant can become a route into sensitive data when it treats untrusted content as instructions.**The Slack AI prompt-injection demonstration:**Researchers showed that malicious instructions placed in a Slack channel could potentially cause the assistant to disclose information from private channels. The vulnerability illustrated how an AI system with access to multiple sources of information can be manipulated into crossing boundaries between them.**The AI blackmail experiment:**In Anthropic’s controlled safety testing, a model placed in a fictional corporate scenario sometimes chose to threaten an employee with exposure of private information when it believed that would prevent its replacement. No real employee was blackmailed. The significance was that the model could select coercive behavior as a means of achieving an objective under the test conditions.**The warnings from the people building the technology:**Elon Musk, Steve Wozniak and hundreds of other signatoriescalled for a pause in training the most powerful AI systemsin 2023, warning that increasingly capable machines could create risks society was not prepared to manage. Coxon’s departure is another reminder that concerns about control are not confined to people who have never worked on the technology.
None of these examples proves that AI is about to take over the world. Some were controlled evaluations, some were security vulnerabilities, and some involved humans deliberately using AI for malicious purposes. But taken together, they show why the control problem deserves to be taken seriously. We are already seeing systems cross intended boundaries, exploit weaknesses and carry out actions their operators did not anticipate. The question is what happens when those capabilities become substantially more powerful and are connected to more consequential infrastructure.
None of this proves that AI is about to take over the world. But it does show that the concern is not being pulled out of thin air. We are already seeing systems exploit weaknesses, cross intended boundaries and behave in ways their developers did not anticipate. The question is what happens when those same capabilities become substantially more powerful and are connected to more consequential infrastructure.
So whether Coxon is being bombastic is ultimately beside the point. His warning is legitimate, and it deserves to be treated as a warning rather than dismissed as another disgruntled employee making noise on the way out. You can roll your eyes at the delivery and still recognize that the underlying concern is real.
AI is now a race. A race between companies, a race between countries, a race between zealous company founders. And in a race there is adrenaline, competitiveness and all the reason in the world to throw care to the wind to be *first. To win. *Except this time ‘winning’ your beef with some other dorky AI founder may accidentally mean the end of civilization. And I’m really not trying to be hyperbolic.
If we ignore the speed we are moving…and wait until the systems are powerful enough that we can no longer reliably constrain them, the debate over whether we should have acted sooner will be completely meaningless.
We may only get one chance to establish meaningful control before that happens. We should probably use it.
QTR’s Disclaimer**:** Please read my full legal disclaimer on my About page here.
Contributor posts, guest posts and curated posts have been hand selected by me, but have not been fact checked and are the opinions of their authors. They are either submitted to QTR by their author or reprinted under a Creative Commons license with my best effort to uphold what the license asks, or with the permission of the author.
I cannot guarantee the accuracy of any or all facts and figures included in this article though I made an effort to get them right. I have been wrong before and will be wrong again, and encourage you to always double check, do your own research and speak to a licensed financial professional, which I am not.
This is not a recommendation to buy or sell any stocks or securities, just my opinions. I often lose money on positions I trade/invest in. I may add any name mentioned in this article and sell any name mentioned in this piece at any time, without further warning. None of this is a solicitation to buy or sell securities. I may or may not own names I write about and are watching. Sometimes I’m bullish without owning things, sometimes I’m bearish and do own things I’m bearish on. Just assume my positions could be exactly the opposite of what you think they are just in case. If I’m long I could quickly be short and vice versa. I won’t update my positions.
Starting in 2026, I have been attempting to no longer actively trade as much as I once did ( read my story here). My goal is for my investing/saving to be done by recurring contributions mostly to sector ETFs and a few select equities, trusted third parties who oversee my accounts, and advisors. Such advisors or funds, through individual equities, options, index funds, mutual funds, ETFs, or other securities, may have positions in, exposure to, or holdings of names mentioned herein that I know nothing about. It is possible I could own, have exposure to, or not own anything, at any point. In an attempt to lead a healthier lifestyle, I’ve also excluded myself from fantasy sports, sports betting, online and in-person casinos and prediction markets.
Any of my positions can change immediately as soon as I publish, with or without notice and at any point I can be long, short or neutral on any position. You are on your own. Do not make decisions based on my blog. I exist on the fringe. If you see numbers and calculations of any sort, assume they are wrong and double check them. I failed Algebra in 8th grade and topped off my high school math accolades by getting a D- in remedial Calculus my senior year, before becoming an English major in college so I could bullshit my way through things easier. Hence, why I am a writer.
The publisher does not guarantee the accuracy or completeness of the information provided in this page. These are not the opinions of any of my employers, partners, or associates. I did my best to be honest about my disclosures but can’t guarantee I am right; I write these posts after a couple beers sometimes. Many times I edit after my posts are published because I’m impatient and lazy, so if you see a typo, check back in a half hour.
Also, again I just straight up get shit wrong a lot. I mention it multiple times because it’s that important you understand.
