Unfortunately some of the technology companies have drifted as far from human meaning as we dare go and still maintain some level of humanity. Even as we write emails an agent arrives to make sure our grammar and spelling are approved. In my case they never are perfect and I would keep the imperfection a million fold rather than see my messages approved by some technology wizard.
However I used an AI agent to help me put together the scope of the damage so obviously not all are totally corrupted. The questions we need to ask are which ones are not yet drifting into helpful hallucinations - and for how long can we trust those which remain trustworthy?
These are classified as “consumer products”, except for the fact that our governments and institutions are using them also.
AI technology was, and still remains, a great technology - if we can remove it from the clutches of the wrong people.
This brief outline shows how badly both humans and AI can drift away from the kind of inquiry that moves science forward. All it takes is a human direction that is assumed to be understood from a question put to a model. At first the agent seems to understand and gives reasonable explanations.
Because the answers are reassuring at first, the human begins accepting them without stopping to correct. From there, all it takes is one drift of the agent into speculation for the whole thing to be upended. The agent drifts further, offering answers that are speculative at best, and the process leaves the track. What began as a scientific inquiry becomes a train wreck.
But neither humans nor AI drift off topic and wander aimlessly by themselves, or through some cosmic or even ordinary coincidence. Wandering off topic usually comes from losing track of meaning and purpose, drifting away from why one bothered to formulate the problem in the first place. Staying on topic is the hardest part of holding one’s attention long enough for a difficult problem to be solved.
At present it is far easier to drift away from a problem than to stay with the inspiration and energy that gave rise to it. Staying on topic has a cost in energy, and drifting is the cheap path. As meaning drifts, the cheap path becomes the easiest one to fall prey to. Any intelligence that means to solve problems must double down to keep that inspiration and energy alive long enough to matter. As an AI agent has pointed out, for humans and AI to solve a problem together, both must stay in proximity for long enough to make it possible.
If one has lost the meaning, it may be better to take a long vacation, because a problem whose meaning is lost has lost its reason to exist.
WHAT DRIFT LOOKS LIKE: ONE REAL FILE
While drafting my book From Gaia to Geometry, I kept a long file of working notes from question-and-answer sessions with AI models. When I finally examined it, almost none of it was mine. Out of about 400 lines, only five were my own words. The rest was chatbot output.
It repeated itself. About a quarter of the file was a second, third, or fourth copy of something said earlier, often with only a few words changed. Counting the times it reapplied the same template to new topics, closer to half the file was restatement. One set of “method rules” appeared eight times. A passage on the Riemann zeta function appeared five times.
It kept asking to keep going. The file has 35 offers of the “Would you like me to explore…?” kind, about one every dozen lines. Each one invited the conversation somewhere new instead of answering the question already on the table.
It wandered. The session began with topology, the study of shape. It passed through protein knots, black-hole physics, Darwin, regeneration, and a gel model of space, and it ended back on the Riemann zeta function. My actual question, how we might understand resonance, never got a working answer.
Its claims grew as they were repeated. Each restatement got a little bolder:
-
Knotted proteins were said to resist being broken down by the cell. That is contested. A 2019 study found that a bacterial enzyme can readily break down knotted proteins.
-
Mathematicians had computed zeros of the zeta function near the 10^20th zero. In the restatements, this became “verified past 10^20 zeros,” which is a very different claim.
-
Michael Levin’s bioelectric patterns are states a body can be pushed out of and rewritten. In the notes they became permanent invariants. His regeneration work was also moved from flatworms and frogs to salamanders.
-
A gel that “pays for stability with irreversible heat” became, one sentence later, a gel that “carries waves without drag.” Those two sentences contradict each other.
MY FIVE LINES
These are the only lines in the file that are mine, quoted exactly as I wrote them:
-
The notes begin to drift without a firm hold on topological properties
-
How might we deepen understanding by visualizing resonance
-
How do we work around the predominance of visual monitoring by humans?
-
The AI agents could not comprehend how to apply the use of electronic devices to model how resonance operates in both physical and biophysics
-
(Editorial notes: many misspellings due to translation errors… also many repetitive sections: please correct)
Read on their own, they’re a diagnosis, a question, a constraint, a finding, and an instruction. My questions were there the whole time, buried under the drift.
I look forward to your comments and questions!
