Something happened this week that deserves more than another round of arguments about artificial intelligence.
** Jacob Coxon**, a researcher who spent the past three years working on the development of frontier AI systems at OpenAI and Anthropic, resigned from Anthropic and issued an extraordinary warning. He accused both companies of racing toward self-improving superintelligence without having adequately solved the problem of controlling what they are creating. Coxon believes the consequences could include human extinction.
It would be easy to file this under another apocalyptic prediction about technology. We have a long history of imagining that the newest invention will either save civilization or destroy it. The internet was going to democratize knowledge. Social media was going to connect humanity. Nuclear power promised an age of abundance while nuclear weapons placed the human future beneath a mushroom cloud. Our technological imagination has always oscillated between salvation and catastrophe.
What makes this moment different is not that someone somewhere believes artificial intelligence could become dangerous. It is where some of these warnings are coming from.
Coxon worked inside two of the organizations building the most advanced AI systems in the world. His warning was subsequently reinforced by Anthropic researchers still working there. Evan Hubinger, who works on the problem of aligning advanced AI systems with human goals, publicly estimated a greater than 10 percent probability that AI could kill all humans within the next decade. Samuel Marks, another Anthropic researcher, wrote that developers of advanced AI genuinely believe their technology could produce extinction-level outcomes.
These predictions could prove spectacularly wrong. There are serious disagreements among researchers about the likelihood, timing, and mechanisms of catastrophic AI risk. Nobody knows whether the technological trajectory being described will unfold, whether superintelligence will emerge in anything resembling the anticipated form, or whether safety research will solve problems that presently appear formidable.
I am not interested in pretending certainty exists where it does not. But I also think we are in danger of thinking too narrowly.
The obvious question is whether artificial intelligence will become powerful enough to threaten humanity. Beneath it lies a more unsettling one: what does it say about us that we can recognize the possibility of creating something we may not be able to control and still feel compelled to create it?
The AI crisis is not fundamentally a crisis of technology. It is a crisis of human development.
AI is exposing a contradiction that has been accumulating throughout modern civilization. Our ability to alter reality has developed far faster than our capacity to responsibly inhabit the consequences of that power. We have become astonishingly good at extending what human beings can do without developing anything comparable in our ability to determine what should be done, what should not be done, when restraint is necessary, how consequences should be shared, or how to stop when the systems we have created make stopping extraordinarily difficult.
AI did not create this contradiction. It is revealing it.
The same species capable of splitting the atom built weapons capable of destroying cities. We engineered systems capable of connecting billions of people before understanding what the industrial manipulation of attention would do to identity, childhood, belonging, politics, loneliness, and public reality. We possess technologies capable of altering biological and ecological systems while remaining organized by tribalism, competition, domination, fear, status, and short-term interest.
Now intelligence itself is becoming something we can engineer. That changes the stakes.
We are approaching the possibility of creating forms of cognition exceeding our own while the systems developing them remain shaped by vulnerabilities far older than the technologies passing through our hands. Companies fear competitors. Nations fear rivals. Investors fear being left behind. Researchers recognize the risks while believing that someone else will proceed if they do not.
The most dangerous intelligence in the AI story is not artificial intelligence. It is human intelligence without sufficient human capacity.
That is the larger story AI is forcing us to confront. The question is no longer simply how capable our machines might become. It is whether we are developing fast enough to live responsibly with what our ingenuity has made possible.
The term “existential threat” gets thrown around so casually that it has nearly lost its meaning. Elections are described as existential. Businesses face existential threats. Institutions announce existential crises. We have borrowed the language of existence to intensify whatever problem happens to be in front of us.
Human extinction is existential threat in its most literal form.
Our species has always lived under this possibility. An asteroid does not need our permission to strike the earth. A sufficiently devastating pandemic could radically alter civilization. Ecological systems upon which our lives depend are not infinitely resilient. The universe has never offered Homo sapiens a guarantee of permanence.
What has changed is our relationship to the threat. For most of our history, extinction belonged largely to the realm of forces beyond human agency. The development of nuclear weapons altered that relationship. Human intelligence had become powerful enough to create the means of destroying much of civilization. The creature capable of contemplating its mortality had acquired the technological means to participate in bringing it about.
I wrote about this in a previous article: ** Going, Going, Gone: The Sixth Mass Extinction and the Crisis of Being Human**.
Artificial intelligence introduces a different version of the same problem. The concern is not simply that somebody will build a machine capable of catastrophic harm. The deeper possibility is that we could create forms of intelligence whose capacities eventually exceed our ability to reliably understand, constrain, or govern them. In that scenario, the threat would arise from the combination of extraordinary capability, autonomy, speed, and goals that do not remain sufficiently aligned with human flourishing.
The argument deserves neither automatic belief nor casual dismissal. Its significance lies partly in what it exposes about the human situation. We have become creatures whose power to alter reality is approaching domains our psychological, political, ethical, and cultural development was never designed to manage. That is an existential health problem.
One of the distinctions I make in my work in existential health is between capability and capacity. Capability concerns what we are able to do. Capacity concerns what we are able to hold.
Modern civilization has become extraordinarily effective at expanding capability. We split atoms, manipulate genomes, engineer global communication systems, alter ecosystems, automate cognitive labor, and build machines capable of performing tasks that only recently seemed inseparable from human intelligence. Every technological threshold enlarges the territory of what is possible.
Capacity concerns our ability to live responsibly in relationship with what our power makes possible. It involves tolerating uncertainty without escaping into certainty, exercising power without becoming possessed by it, and recognizing limits before catastrophe imposes them. It also requires coordinating across differences, practicing restraint when restraint carries a competitive cost, and remaining in contact with consequences that extend beyond our immediate interests and lifetimes.
The problem is not simply that capability and capacity are different. It is that they are developing at radically different speeds. I call the distance between them the Capacity Gap.
The Capacity Gap is the growing distance between what a civilization is technologically capable of doing and what it is psychologically, ethically, and politically capable of responsibly holding.
Modern history offers increasingly consequential examples of this gap.
The splitting of the atom gave humanity access to extraordinary energy and, almost simultaneously, the capacity for destruction on a scale previously unimaginable. Nuclear weapons did more than introduce a new instrument of war. They created a permanent civilizational condition in which decisions made by a relatively small number of human beings could affect the future of millions, perhaps billions. Our technological capability crossed a threshold before our systems of governance, tribal loyalties, and capacities for international cooperation had developed anything comparable.
A similar pattern appears in our relationship with the planet itself. Industrial civilization acquired the power to extract, produce, consume, and transform natural systems on a planetary scale long before we developed adequate ways of remaining in contact with the cumulative consequences. Climate disruption is not simply evidence that humans possess powerful technologies. It exposes the difficulty of exercising restraint when consequences are distributed across borders, generations, economic systems, and futures that no single actor controls.
The digital revolution brought the pattern inside human consciousness. We learned how to capture, measure, predict, and manipulate attention before understanding what the industrialization of attention would do to identity, childhood, belonging, loneliness, politics, and shared reality. Technologies built to connect people became infrastructures capable of shaping what billions of people see, desire, fear, believe, and experience as socially real.
Biotechnology carries the same tension into life itself. Genetic engineering and synthetic biology increasingly give us the ability not merely to observe biological processes but to intervene in and redesign them. The scientific possibilities are extraordinary, including possibilities for reducing suffering and treating disease. Yet every expansion of biological capability also raises questions about unintended consequences, access, ownership, inequality, consent, limits, and who should possess the authority to alter living systems.
Autonomous weapons sharpen the dilemma further. The technological possibility of delegating increasingly consequential decisions to machines forces questions that engineering alone cannot answer. Some decisions should never be automated, and determining where human judgment must remain is an ethical and existential responsibility. As agency becomes distributed across programmers, commanders, institutions, algorithms, and machines, responsibility becomes increasingly difficult to locate.
These developments are usually discussed as separate problems. Nuclear proliferation belongs to national security. Climate disruption belongs to environmental policy. Attention engineering belongs to technology and media. Genetic engineering and synthetic biology belong to bioethics. Autonomous weapons belong to military policy. Artificial intelligence belongs to computer science, economics, governance, and increasingly almost everything else.
But beneath those institutional categories, another pattern becomes visible. Human capability repeatedly outruns human capacity. Our capabilities advance before we have developed the psychological, ethical, and political maturity necessary to responsibly hold their consequences. The technology changes. The underlying developmental problem remains.
The greater our technological power becomes, the more consequential that distance becomes. A civilization with relatively limited capability can survive substantial failures of judgment. One capable of altering planetary systems, engineering life, manipulating the attention of billions of people, automating warfare, and potentially creating intelligence exceeding its own has considerably less room for existential immaturity.
The problem is not simply that our technologies are becoming too powerful. It is that the consequences of our power increasingly depend upon forms of restraint, discernment, responsibility, cooperation, self-governance, and wisdom that technological development does not automatically produce.
Greater intelligence does not necessarily produce greater wisdom. Greater power does not necessarily bring greater capacity to govern it. Technological advancement does not guarantee human development.
The Capacity Gap may be one of the defining dangers of advanced civilization.
AI makes that gap unusually visible because intelligence itself has now entered the equation. We are no longer merely using human ingenuity to create more powerful tools. We are beginning to create increasingly powerful forms of intelligence.
That is why the AI question cannot finally be reduced to what these systems become. It also forces us to confront what we have become in relationship to our own power.
The question confronting us is larger than whether increasingly powerful systems can be aligned with human purposes. It is whether we are developing sufficient capacity to govern what our own intelligence has placed within reach.
The Capacity Gap points toward an uncomfortable possibility about modern civilization. We have mistaken technological development for human development. They are not the same thing.
The sophistication of our tools can create the impression that the people using them have undergone a corresponding transformation. We look at laboratories, satellites, global communication networks, advanced medicine, enormous computational systems, and machines capable of generating language and solving complex problems, and it becomes easy to tell ourselves a story of human advancement.
In many important respects that story is true. Knowledge has expanded. Lifespans have increased. Scientific understanding has transformed our relationship with disease, matter, energy, biology, and the cosmos. Human ingenuity has accomplished things that would have appeared supernatural to most of our ancestors.
But technological sophistication does not demonstrate existential maturity. The human being operating these extraordinary systems remains vulnerable to fear, tribalism, domination, status competition, certainty-seeking, imitation, resentment, greed, narcissism, and the desire to belong.
We remain susceptible to charismatic authority and collective panic. We still organize ourselves into groups that exaggerate the virtues of insiders and the dangers of outsiders. We can understand consequences intellectually while arranging our lives so that someone else will bear them. None of this disappears because the tools in our hands become more powerful.
I think of this as *the *Great Developmental Lag: the widening discrepancy between the extraordinary development of human capability and the comparatively slow development of the capacities required to govern it wisely. Our technological evolution is occurring at a speed that biological evolution, cultural adaptation, political institutions, and human meaning-making were never required to match before. We are becoming enormously powerful without becoming proportionately capable of holding that power.
The twenty-first-century human being inhabits a peculiar contradiction. We carry technologies capable of altering planetary systems while possessing nervous systems shaped for threats that were once immediate, local, and embodied. We participate in global networks while remaining deeply responsive to tribal belonging.
We make decisions whose consequences can extend across continents and generations while our attention remains naturally drawn toward what is nearby, urgent, emotionally vivid, and personally consequential. Our reac…
