BYLINE TIMES IMPACT: Find out about the changes you made happen
There is something deeply strange about the way we talk about artificial intelligence.
We are increasingly being warned that the technology we are rushing to develop could pose an existential threat to humanity. Not by science-fiction, but by the people who actually built these systems.
Engineers and researchers from companies like OpenAI and Anthropic have warned that increasingly capable AI could cause catastrophic harm including environmental catastrophe, biological warfare and active war dangers, as well as the possibility of human extinction by 2030. These aren’t predictions we can dismiss as the ravings of technophobes; they reflect a critical debate within the industry about what happens when system capabilities outstrip our ability to contain them.
These systems are no longer merely sophisticated chatbots. AI agents built by humans are now acting autonomously: navigating computers, writing code, accessing sensitive data, and pursuing goals with minimal human supervision. Researchers have already demonstrated systems exploiting software vulnerabilities and behaving in ways their creators did not anticipate. Anthropic has even faced scrutiny over its refusal to submit a recent model for evaluation by the UK’s AI Security Institute. They aren’t required to, so why would they?
None of this proves AI will wipe us out. But if there is even a meaningful possibility of catastrophic harm, the rational response is to assess that risk before accelerating further. Instead, we are building systems faster, granting them more autonomy, and competing to make them more capable all while the people building them admit we may not understand where this ends.
Which leads to an urgent and quite sophisticated question: what the f**k are we doing?
If we genuinely believe AI could become an existential threat, why are we treating its development as a race?
The world’s most powerful companies are competing to build increasingly capable systems, while governments scramble to ensure their domestic champions don’t fall behind.
The logic is depressingly familiar: if OpenAI slows down, Anthropic moves ahead; if the US hesitates, China takes the lead. Every individual decision is justified in isolation as necessary for competitiveness or responsible development.
And collectively, those decisions produce pure acceleration.
Nobody wants to apply the brakes first. We are repeatedly told AI will transform medicine, boost productivity, and solve intractable problems. Those possibilities might be real, but potential benefits do not make catastrophic risks vanish.
Our current model lets profit-driven companies set their own safety limits. That isn’t a safety system—it’s a commercial free-for-all.
There is a reason this all feels vaguely familiar. We have been here before. The closest parallel may be the climate crisis – not because AI and climate change are remotely the same problem (though the former contributes to the latter), but because the political failure is remarkably similar.
For decades, we knew that burning fossil fuels at an enormous scale was changing the climate. Scientists warned governments. Governments commissioned reports. Researchers refined the models. The evidence accumulated. And still, we kept going.
The problem was not simply that politicians didn’t understand the science. It was that acting on the science required confronting some very powerful interests and accepting some very unpopular short-term costs.
Fossil-fuel companies had enormous financial incentives to keep extracting and selling fossil fuels. Industries depended on cheap energy. Politicians worried about jobs, prices and economic growth. Governments competed with one another. Lobbyists worked to weaken regulation. And every individual decision to approve another project, delay another target or water down another policy could be presented as perfectly reasonable.
We could deal with the problem later. Later, unfortunately, is now. We are living with record temperatures, increasingly severe weather, pressure on food and water systems and ecosystems under extraordinary strain. None of this was unforeseeable. The tragedy of climate change is precisely that so much of it was foreseen.
And this is where we should be extremely wary of repeating the same mistake with AI. We are already hearing the warnings. We are already being told that increasingly capable systems could create risks that are difficult to control. And we already have an industry worth hundreds of billions of dollars whose commercial success depends on developing those systems as quickly as possible. Why would we expect the political incentives to be any different?
We shouldn’t.
This isn’t about assuming Sam Altman, Dario Amodei, or Elon Musk are inherently evil (though they do have a Bond villain vibe). The problem is structural. Companies are built to compete, grow, and create shareholder value – not to make decisions for humanity as a whole. Even well-intentioned executives – if they exist – face pressure from investors, competitors, and governments.
If restraint looks like unilateral disarmament, dangerous systems become normalised because everyone has a plausible excuse to keep going. We saw this dynamic play out with tobacco, financial markets, opioids, and fossil fuels: private profits are concentrated, while eventual costs are distributed across society. We are remarkably bad at regulating powerful industries before their risks become impossible to ignore. AI gives us a rare luxury: a warning before the consequences arrive.
Taking the risk seriously doesn’t mean banning AI or halting progress every time a risk is named. The principle is far simpler: if we don’t know whether something is safe, we shouldn’t make it exponentially more powerful just because a competitor might get there first.
Taking the risk seriously means putting genuinely independent safety assessment ahead of commercial competition. The most powerful models must be tested before deployment, dangerous capabilities disclosed, and systems scrutinised by external researchers without relying on corporate goodwill.
Governments must establish binding, universal limits on AI development and they must do it now.
We cannot build public safety around the assumption that private companies will voluntarily sacrifice profits in the name of humanity. We don’t do this with nuclear power, pharmaceuticals, or aviation. We create rules, regulators, and independent oversight – and crucially, we reserve the right to say no.
Inevitability in tech development is not a law of nature; it is a political choice and a marketing frame to force acquiescence. We can decide that certain risks are unacceptable and that the pace of development must be set by what we can safely govern, not what market competition dictates.
The most uncomfortable truth is that we don’t know where this race ends or why we are running it. AI may well help treat diseases, accelerate discovery, and improve lives. It may also lead to species extinction. Those outcomes are not mutually exclusive but the former don’t really matter if we’re all dead, which is why blindly accelerating toward maximum short-term profit is reckless.
If the goal of technology is human flourishing, then flourishing must be what we optimise for. And if a technology genuinely threatens human survival, then surely survival must be our non-negotiable constraint.
With AI, we have been given an extraordinary chance to intervene before the crisis hits.
We don’t know if the worst-case scenarios will materialise, but if there is even a meaningful chance they will, we must stop asking how fast we can build the future and start asking what kind of future we are actually building.
