For, today, these are indeed days of miracles and wonders:
I have gotten “AI” working well enough that for just electricity costs it saves me an hour a day dealing with the every day firehose of things I ought to read from strangers and friends to do my job well and be my proper self. What does it miss? Perhaps 5%. Plus it saves me half an hour each day in writing boilerplate and reminding me of what I said and thought about issues and questions one, five, or twenty-five years ago. Five years ago I would not have thought such an info-butler would be possible in my lifetime. 2 1/3 years ago I would have thought it might be possible in five to ten years, but would be quite expensive.
I really did not expect open-weight LLMs on consumer hardware (admittedly, high-end consumer hardware) to be an effectively free working alternative.
But can it think for me?
I would still say no. But I will not say never, for that would require that I better grok what “thinking” is. And I don’t think that it will, not in my lifetime, except in restricted domains. Noah Smith persuaded Ramez Naam to write something about where we are headed that struck me as smart:
:
Ramez Naam: Where’s the “Intelligence Explosion”? <[https://www.noahpinion.blog/p/wheres-the-intelligence-explosion]>; ‘A skeptic’s take on Recursive Self-Improvement…. I do expectnarrowsuperintelligencein highly verifiable domains.Think chess, Go, formal math, parts of computer science and coding. Highly verifiable domains are largely formal and structured types of work where machines can generate unlimited training data, with perfect or near-perfect verification of correct vs incorrect, and do so entirely in software without waiting on the physical world or humans. That’s an ideal setting for AI learning.In fact, we already have narrow superintelligence in game plang. We’re seeing it happen now in the most formal parts of math, in particular in proofs and in finding counter-examples…. Parts of software development are also extremely verifiable, while others are a bit less crisp (such as understanding what humans want). That isn’t the same as
broadsuperintelligence….Enormous increase[s] in AI output… [have] accompanied a much smaller increase in experiments run…. Anthropic reports that their engineers are now
[producing 8x as many lines of code]per person as they did in 2024… [but with] significant diminishing returns between productivity and AI progress. Here’s a direct quote from its[Mythos Preview system card]: “Productivity uplift does not translate one-for-one to capabilities progress…. We estimate that reaching 2× on overall progress via this channel would require uplift roughly an order of magnitude larger than what we observe.” Translation: To double the pace of AI progress, Anthropic estimates that AI would need to increase the productivity of their employees by roughly a factor of 40 relative to no AI assistance…
But what do I think?
Start with the premise that the bulk of language is redundancy, error correction, and social connection. Most of the rest is the presentation of concepts, ideas, and facts that have been thought by humans before: moving from storage into processing memory the background knowledges of the real ASI that is the Anthology Super-Intelligence of the collective space- and time-binding collective human Mind. Beyond and above that, extra information is carried by the un- or less-expected word: what is not error-correction, boilerplate, redundancy, or background conventional wisdom, but which carries textual meaning appropriate to the situation. Where does this meaning come from? From three places:
Finely and productively slicing the data storage of the real-ASI to dredge up information that can be carried by that slice’s expected word that is a near-exact match to the particular situation.
The unexpected word that comes from evaluation of an accurate world model, driven by the truth that there is a great deal outside the pure text.
The unexpected word that springs from thoughts that have not previously been put into words. My teacher Jeff Weintraub taught me when I was nineteen to look at key words in an author’s work that they are trying to redefine: like “virtú”, manliness, for Machiavelli; like “natural liberty” for Smith, like “democracy” for de Tocqueville; like “charisma” for Weber. That creation and transmission of meaning through the warping of language into new forms is not something an LLM, for which all there is is the text, can do.
LLMs can do boilerplate, redundancy, error-correction, conventional wisdom. They can also do, sometimes, if the stars align, if the vector embedding self-constructed the right similarity metric, if they are properly and skillfully harnessed, the fine and productive slicing of the data storage of the real-ASI to dredge up information that can be carried by that slice’s expected word that is a near-exact match to the particular situation.
That is what they can do. But there is much that they cannot and will not, not in my lifetime, be able to do.
