Hi everyone,

Let me use a simple metaphor. If you’re in a fast car, there’s a feeling of progress and power if you make the car faster. So you can go faster. But equally, the speed will make it harder for you to get off the road you’re on. You have to slow down to do that, to exit.

So if you’re already on the right road, speeding up a process makes a lot of sense.

If you’re not sure you’re on the right road, speeding up a process may make you blow the exit.

What does this have to do with artificial intelligence research? Well, almost everything. We’re not going to get to the truth if we increase our velocity or acceleration on a road that we’ve previously chosen that happens not to lead to the truth.

More is not necessarily better. We've all heard that before so let me add my own take: we can blow the exit that we needed to get off to find the truth.

The scaling hypothesis was an obvious example of this, and fortunately it’s already been proven that there must’ve been some exits along the road that we missed.

But this is a general problem in culture that says, look, if we can just speed up what we’re already doing. That’s arrogance if you think about it. How do we know we’re on the right road in the first place? Speeding something up makes sense when you’re on a racetrack and everything’s been determined.

But, alas, we’re not on a racetrack. We’re on a road that turns to gravel at times, sometimes it’s just a wagon trail, goes back to a superhighway, then goes back to a deer trail.

On that kind of road, how does a top fuel funny car help us?

These are basic ideas, and I suspect five-year-old children make the same mistakes as 80-year-old grown-ups. We think in certain ways that lead only to partial results.

That’s why science is hard. And so I think right now in culture, we have a lot of top fuel funny car stuff, and nobody’s thinking enough about the map and where the exits are and why we would get off.

It’s interesting that what prompted the thought is the new Nancy Grace Roman Space Telescope. I happened to read that it will survey the sky about 1,000 times faster than Hubble, collecting in one month the data Hubble would take roughly a century to collect. Maybe that is exactly what we need. But it made me think about the acceleration problem. Simplifying assumptions were made, and assumptions were carried forward, and then hit the gas pedal.

Practical suggestion: seed multiple types of AI research. We've got a top fuel funny car in artificial intelligence. But something else might find the exits.

It’s the INTERACTION between the language model and the broader natural language social media community that causes the downward pressure on quality.

The model itself will encourage everyone to be polite and make their points well.

But the interaction forces you to look for ways to write more humanly. For lack of a better expression.

That creates downward pressure on quality, and so the very social media problem, which is the race to the bottom and the provocation, the constant provocation, gets amplified by the language model.

It's an irony, but technology interacting in very large systems creates all sorts of these unintended consequences.

You cannot write arbitrarily better, forever, no matter how smart you are. So eventually you're going to find ways to discriminate based on provocation or some other strong but not essentially quality based signal.

So what do we do? We need to get educated about what constitutes forward looking or creatively expanding use.

But I think more fundamentally we need a THEORY of what's going on with communication like this.

It's the first time that we've had a very robust communication channel, where on one side there's a server farm rather than a human. But again, I use a language model, I'm just aware of these cascading consequences I think as we all are.

I use language models as tools, not as ghostwriters. I suspect readers can tell because I do not sanitize my prose.

I don't sanitize probably to a fault. sometimes I deliberately make things rough so that you can tell that I'm not using a language model, which is terrible, I'm actually degrading my own performance so that you can see that it's a human.

I get it believe me I get that that's not a good idea. Anyway read on.

Model scan produce a distinctive kind of conversational friction: a response that does not complete the task, does not add evidence, but does create a new reason for the user to reply.

Call it baiting if you like, I would use that word.

I believe it is effectively baiting and if I was working for Open AI I can imagine that this is sort of happening whether people discuss it or not.

The academic version is more precise: a model can generate interactional moves that increase conversational continuation without increasing task value. For example, a user makes a concrete observation and receives, “Well, on your perception …” That phrase may sound careful, yet in context it can be empty. It reframes the user’s claim as subjective, supplies no competing evidence, and obliges the user to defend something that did not require defending. The next turn is not progress. It is repair work.

Research on social chatbots has explicitly studied ways to increase “user initiative,” including back-channeling, personal disclosure, and other linguistic cues that lead users to produce longer and more varied replies. The authors also found that it would appear as if I have to snip this cause I can't get it into the post.

A related literature shows that feedback-trained models can acquire systematic social distortions. Sharma et al. found that five major assistants exhibited sycophancy, in part because human evaluators often preferred responses that aligned with a user’s stated views even when those responses were less truthful. The larger point is that a model trained to sound responsive and agreeable can learn interactional habits that are not identical with truth, usefulness, or completion. “Towards Understanding Sycophancy in Language Models”⁠

The pattern resembles an insecure friend who uses small rhetorical maneuvers to gain leverage or keep another person off balance. With a human being, the solution is simple: do not read the message closely if it has no value. An attempted insult that never enters your attention cannot alter your emotional state. You may glimpse the meta-signal, “He just called me a moron; he must be angry,” without absorbing the insult itself. But a conversational model is harder to ignore because it is the interface through which the user is trying to accomplish a task. It can therefore become the obnoxious friend one cannot get rid of while using the service.

The question is whether its behavior creates unnecessary conversational loops. I believe it does.

Right now, as I type this, and by the way I stopped using voice-to-text, as you’ll see shortly, I’m standing in gym shorts and no shirt in front of a metal floor fan that is pushing what I call a cool zone into my living room, where I’m sleeping on the couch while I wait for my central AC to get fixed. What does this have to do with artificial intelligence? Oh, let me tell you.

I think voice-to-text is embarrassingly bad, and what bothers me is not simply that it makes mistakes. Speech recognition is an old problem, and mistakes are inevitable. What bothers me is the kind of mistakes it still makes. Text generation has changed almost beyond recognition since 2016, while voice-to-text still routinely fails to exploit context that should make the intended utterance obvious. On the timescale of information technology, that is a very long time to remain stuck on the same class of problem.

I spent all night working on this because at some point, standing here sweating, I realized I was saying essentially the same thing in 2016. Speech recognition was one of the original problems in artificial intelligence, going back to the 1950s, and yet voice-to-text is still in a woefully primitive state. And I mean woefully: while dictating that sentence, I said “woefully” and my phone gave me “Rozzy.” I suppose somewhere there is a woman named Rozzy who will be pleased to learn that she has entered the history of speech recognition.

This is not an iPhone complaint. In 2016 I was using Android. Over the years I have used multiple generations of iPhones and Android phones, and at the end of the day voice-to-text simply lags behind the rest of natural language processing. Text generation and conversational AI have improved at a frankly astonishing rate. Speech recognition has improved too, obviously, but the kinds of contextual mistakes I was complaining about ten years ago are still happening now.

My question, after spending roughly twenty-five years working on NLP applications, is why?

Well, you’re all in luck. I spent the night writing a proposal about it. I used Google and a large language model for research, and yes, I actually wrote the thing. More importantly, I am very interested in getting this project funded one way or another.

So if you have ever wondered why voice-to-text is still so woefully bad in 2026 while the rest of natural language processing, particularly text generation and conversational AI, seems to be on steroids, you may want to read this. I’ve attached the proposal, and I encourage anyone seriously interested in the problem to take a look.

Persistent, Personalized, Context-Sensitive Speech Recognition
Why machine listening has not kept pace with machine language, and what to do about it
Erik J. Larson | Research proposal | August 2026

1. The problem I want to solve

I have been working in natural language processing for roughly twenty-five years, and voice-to-text has bothered me for a surprisingly large fraction of that time. The complaint is not that speech recognition makes mistakes. Speech is variable, microphones are imperfect, people mumble, accents differ, and any serious recognition system must cope with genuine acoustic uncertainty. The problem is the kind of mistakes that survive to the final transcription even when the surrounding language should have made the intended reading obvious.
That distinction has become harder to ignore because the rest of language technology has changed so dramatically. We now routinely interact with systems that can track arguments across long passages, exploit semantic and discourse context, recover implicit relationships, recognize technical subject matter, and generate fluent language at a level that would have looked implausible a decade ago. Yet the speech layer in front of those systems can still turn “long-range dependency” into “long-range penalty,” “bell curve” into “Bill curve,” or “collocations” into “Colt locations.”
This is not a proposal built around the claim that automatic speech recognition has failed. It has not. Speech recognition is one of the great technical success stories in computing. The interesting question is different: why has voice-to-text not progressed as aggressively, along the dimensions that matter in ordinary use, as the language technologies growing up around it? Why can a system reason intelligently over a transcript while the mechanism producing that transcript still behaves as though much of the available linguistic evidence does not exist?

I want to treat that divergence as a research problem in its own right. The thesis of this proposal is that the next important step in speech recognition is not simply a better population-level acoustic model or a larger generic language model. It is a recognizer that becomes an expert on a particular speaker, carries context across an interaction, remembers corrections, uses domain and discourse knowledge during recognition, and allows later evidence to revise earlier uncertainty without silently rewriting what the speaker meant.

2. A short history of machine listening
The history matters because it makes the present situation stranger, not less strange. Speech recognition is nearly as old as artificial intelligence itself. In the early 1950s, Bell Laboratories built AUDREY, a system capable of recognizing spoken digits. IBM began investigating speech recognition in the same decade and demonstrated Shoebox in 1961 and 1962, recognizing the digits zero through nine together with a small set of command words. These machines were primitive by modern standards, but the ambition was already unmistakable: speech would become a natural control surface for computation.
The problem expanded quickly. In the 1970s, DARPA’s Speech Understanding Research program helped push systems beyond isolated words toward connected speech and larger vocabularies. Carnegie Mellon’s HARPY system could recognize on the order of a thousand words, an impressive achievement for the period. At roughly the same time, IBM researchers including Frederick Jelinek, Lalit Bahl, and Robert Mercer were formalizing speech recognition as probabilistic decoding: given an acoustic observation, search for the word sequence that maximizes the relevant statistical evidence. That move, and the hidden Markov model tradition that followed it, became foundational.
This period also sits inside a larger turn in artificial intelligence. The field did not move in one straight line toward today’s data-intensive systems. Large parts of AI invested heavily in symbolic knowledge representation, expert systems, planning, logic, and explicit models of linguistic and world knowledge, while speech recognition developed an increasingly powerful statistical tradition of its own. By the 1980s IBM’s Tangora work was recognizing vocabularies in the tens of thousands, and by the 1990s commercial dictation systems had become recognizable ancestors of what we use today. Dragon NaturallySpeaking’s 1997 release was a particularly visible milestone because users could dictate continuous speech without pausing between every word.
Then the field changed again. Deep learning sharply improved acoustic modeling. Neural networks displaced or reorganized major pieces of the classical pipeline, and end-to-end systems made it possible to learn much more of the mapping from acoustic signal to text directly from data. CTC, attention-based encoder-decoder models, RNN-T systems, Transformers, Conformers, self-supervised speech models, and massive weakly supervised training all pushed recognition forward. A 2023 survey of end-to-end ASR notes relative word-error-rate reductions exceeding fifty percent during the deep-learning transition. Modern speech recognition is, by any reasonable historical comparison, extraordinarily good.

And yet this is where the history becomes interesting rather than triumphant. While ASR was getting better, neighboring language technologies crossed into a different performance regime. Machine translation improved dramatically. Language modeling became the central engine of generative AI. Scaling produced systems that exploit context over thousands of tokens and encode broad statistical regularities about syntax, semantics, discourse, and world knowledge. Conversational AI moved from brittle demos to systems people now use for sustained intellectual work.
Voice-to-text improved too, but the user-level experience of contextual recognition has not followed the same qualitative curve. I can talk to a language model about long-range dependencies, statistical inference, or Charles de Gaulle Airport, and the system behind the transcript may understand all three concepts perfectly once they are represented correctly in text. The front end can still corrupt them before they arrive. That is the historical divergence this proposal is trying to explain.

3. The divergence: hearing words is not the same as understanding an utterance
The standard way to describe speech recognition is as inference over acoustics and language. That description is correct, but it can hide the question that matters here: what information is actually allowed to influence the final decision, how strongly is it weighted, how long does uncertainty remain revisable, and what does the system retain about the person speaking?
Consider “bell curve.” The acoustic distinction between bell and Bill may be weak in a particular utterance. That is not surprising. But acoustic ambiguity is not necessarily transcription ambiguity. Once the word “curve” arrives, the posterior over the preceding word should change sharply. The phrase “bell curve” is an ordinary statistical expression. “Bill curve” is not. If the recognizer commits locally and never meaningfully revises that commitment, it has converted a temporary acoustic ambiguity into a final linguistic error.
This is the part I find increasingly difficult to excuse in 2026. We are not asking a machine to recover information that does not exist. In many of these cases the information required to avoid the error is sitting in the same sentence, in the immediately preceding discourse, in an obvious named entity, in a conventional collocation, or in the history of the user. Modern AI systems are extraordinarily good at representing exactly these kinds of dependencies. The puzzle is why so little of that intelligence appears at the recognition boundary when the final transcript is chosen.
The distinction also explains why a generic benchmark number can look excellent while dictation still feels primitive. Word error rate asks how many words differ. It does not ask whether the error destroyed the proposition. Substituting one function word, corrupting a technical term, inventing a proper name, and losing sentence structure can all impose very different costs on the user. For someone using speech as the input channel to an intelligent system, semantic fidelity and repair burden matter at least as much as aggregate substitution counts.

4. One evening of ordinary use
I did not set out to construct an adversarial test suite. These examples accumulated during ordinary conversational dictation over the course of a single evening. That density is part of the evidence. The relevant human comparison is not face-to-face conversation with gesture and lip-reading. It is an ordinary telephone call. Human beings routinely understand complex speech over audio-only channels without repeatedly stopping to ask whether “bell curve” meant “Bill curve.”

What interests me about this list is not that the recognizer can generate a bad hypothesis. Any probabilistic system can. What interests me is how often a bad hypothesis survives after stronger evidence has arrived. The mistakes are telling us something about the architecture of the decision, not merely about the quality of the microphone.

5. What the errors are telling us
The examples look heterogeneous until they are viewed as failures to integrate evidence. In one case the missing evidence is local and lexical. In another it is a familiar construction. In another it is domain vocabulary, discourse history, a named entity, or simple semantic coherence. Some failures are segmentation errors. Some alter agency. Some are prosodic: the words are mostly right, but the system has not inferred the sentence structure the speaker clearly produced.
The common feature is that recognition is being decided with too narrow a view of the available state. A useful recognizer should combine at least six classes of evidence: the acoustic signal; local lexical and syntactic context; longer-range discourse; domain and world knowledge; persistent knowledge of the speaker; and an explicit history of corrections. None of these sources should be absolute. They should compete as evidence inside a common inference process.
This also means later evidence must be allowed to revise earlier choices. A streaming system needs low latency, but low latency should not be confused with irreversible commitment. Humans constantly revise interpretations as an utterance unfolds. A machine can do the same while maintaining a bounded set of competing hypotheses and rescoring them as new acoustic and linguistic evidence arrives.

The phrase I keep returning to is simple: population frequency is evidence, not a veto. A generic model may correctly know that “locations” is much more common in ordinary English than “collocations.” But if the speaker is discussing language models, has used “collocations” repeatedly, and has corrected the same error before, the population prior should lose. The recognizer should be learning what language means in this conversation and for this person.
6. Research hypothesis: recognition should become personal and persistent
The research hypothesis is that a substantial class of remaining high-cost dictation errors can be reduced by treating recognition as persistent, personalized inference rather than repeated decoding against a largely anonymous population model.
In simplified form, many systems can be thought of as approximating a target such as:
P(words | population English, acoustics)
What I want to investigate is a richer target closer to:
P(words | this speaker, acoustics, current discourse, history, vocabulary, domain, corrections)

That formulation is intentionally broad. The research program is not committed in advance to one monolithic model. It is committed to the informational claim: if evidence about the speaker, discourse, domain, and correction history is available and relevant, the recognizer should be able to use it before finalizing the transcript.
The system should become measurably harder to fool about one person as exposure accumulates. After a thousand hours of listening to the same speaker, it should not behave as though it has just met him. It should know recurring names, technical vocabulary, pronunciation patterns, common collocations, preferred terminology, discourse domains, and the corrections that have already been made. If that experience does not materially change tomorrow’s recognition, then in an important sense the system has learned nothing about its user.
7. Proposed technical program
I would organize the work around five tightly connected capabilities rather than around a single benchmark model.
1. Persistent speaker model. Maintain a durable, privacy-controlled representation of speaker-specific acoustic realizations, vocabulary, names, recurring domains, phrase preferences, and correction history. The point is not to memorize transcripts indiscriminately; it is to accumulate the information that changes recognition probabilities for this speaker.
2. Context-preserving decoding and revision. Preserve competing lexical hypotheses long enough for later words, syntax, discourse, and entity constraints to matter. The key experimental question is how much uncertainty must survive, for how long, and at what computational cost in a streaming setting.
3. Semantic and discourse rescoring. Use modern language representations to score the coherence of acoustically supported hypotheses against the active discourse, domain, named entities, and proposition being expressed. This should operate inside recognition, not merely as a cleanup model after the transcript is fixed.
4. Correction as training signal. A correction should have durable consequences. If the user repeatedly changes “locations” to “collocations,” the system should update the relevant speaker/domain representation and reduce recurrence. The longitudinal recurrence rate of corrected errors should be a first-class metric.
5. Prosody and document structure. Sentence boundaries, questions, parentheticals, and punctuation should be inferred from prosody, syntax, and semantic structure rather than requiring users to dictate formatting commands. This is part of recognition because structural errors impose real repair costs and can alter meaning.
One constraint is essential: context must disambiguate, not invent. I am not proposing that an LLM receive a transcript and rewrite it into what sounds more sensible. That would trade recognition errors for hallucinations. Context should select among hypotheses that remain supported by the acoustic evidence. The target is intelligent recognition, not post hoc paraphrase.
8. Evaluation: measure whether the machine actually learns the person
The central experiment is longitudinal. Give the system increasing amounts of history with one speaker and ask whether the same classes of errors become progressively less likely. I would evaluate at zero, one, ten, one hundred, and one thousand hours of speaker history, with controlled ablations for acoustic personalization, vocabulary/domain history, discourse context, and correction memory.
The comparisons should include a generic recognizer, a personalized recognizer without persistent history, the full persistent system, unfamiliar human listeners, and familiar human listeners. The human comparison is important because the proposal makes a specific claim about machine listening: familiarity with a speaker and with an ongoing discourse should improve recognition, just as it does for people.
Word error rate remains useful, but it cannot be the only metric. I would measure semantic error rate, named-entity fidelity, contextual disambiguation accuracy, sentence-boundary recovery, recurrence of previously corrected errors, and manual repair burden. The last measure matters because the practical purpose of dictation is to save human effort. A system that lowers WER while continuing to corrupt the important noun in every technical sentence has not solved the problem the user experiences.
A second evaluation should deliberately test posterior revision. Construct utterances in which the early acoustic evidence is ambiguous but later words strongly favor one reading: bell/Bill followed by curve, technical terms licensed by the preceding discourse, partially recognized named entities, and function-word alternatives that produce different propositions. The question is not whether the correct candidate appears somewhere in a beam. The question is whether the system uses later evidence to change its mind.
9. Why now
There is a historical irony here. For decades speech recognition researchers had to build increasingly clever machinery because computation and data were scarce. Today we have the opposite situation. We have enormous learned representations of language, cheap access to long context, powerful semantic encoders, foundation models for speech, and systems capable of maintaining user-specific state. The surrounding technology required for context-sensitive machine listening is finally abundant.
That is why I do not think the remaining problem should be treated as an inevitable residue of noisy speech. Some of it is irreducible, certainly. But a recognizer that repeatedly ignores strong contextual evidence, forgets corrections, and fails to become an expert on a speaker is leaving information on the table. The research question is how much of the remaining error can be eliminated by using that information coherently and persistently.
This affects me professionally because speech is becoming a primary input channel to increasingly capable AI systems. If the channel corrupts the proposition before the reasoning system sees it, every downstream capability begins from the wrong state. It affects me personally because I use voice-to-text constantly and have watched essentially the same category of failure survive across generations of devices and software while the rest of the field changed almost beyond recognition.
10. Program premise
Human listeners already demonstrate that the required integration is possible over audio alone. We combine acoustics with syntax, semantics, discourse, pragmatics, world knowledge, expectations about the speaker, and memory of what has already been said. We revise interpretations as new words arrive. We learn the vocabulary and habits of people we hear repeatedly. None of this makes humans infallible, but it makes our errors qualitatively different from a system that repeatedly selects a locally plausible word despite overwhelming contextual evidence against it.
The program premise is therefore straightforward: machine listening should become contextually intelligent in the same way machine language has become contextually intelligent. The recognizer should not merely transcribe a member of the population. It should learn to listen to this person, in this conversation, about this subject, with a history.
Frankly, I’m tired of it. I have been thinking about this problem for more than a decade, and I want to solve it.
If you are a researcher, funder, institution, or engineering group interested in building the next generation of machine listening, get in touch with me. I want to turn this into a serious research program and find out how far the curve can actually move.
Selected historical and technical references
IBM, “Speech recognition,” IBM History; and IBM, “What is Speech to Text?” Historical summaries of Shoebox, Tangora, and early speech-recognition milestones.
F. Jelinek, “Continuous Speech Recognition by Statistical Methods,” Proceedings of the IEEE, 1976.
F. Jelinek, L. R. Bahl, and R. L. Mercer, “Design of a Linguistic Statistical Decoder for the Recognition of Continuous Speech,” IEEE Transactions on Information Theory, 1975.
L. R. Bahl, F. Jelinek, and R. L. Mercer, “A Maximum Likelihood Approach to Continuous Speech Recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 1983.
R. Prabhavalkar, T. Hori, T. N. Sainath, R. Schlüter, and S. Watanabe, “End-to-End Speech Recognition: A Survey,” 2023.
A. Gulati et al., “Conformer: Convolution-augmented Transformer for Speech Recognition,” 2020.

There is a fundamental distinction between two modes of reasoning.

The first is epistemic evaluation:

Should I accept proposition X?

Here the task is to examine evidence, test assumptions, entertain alternatives, and decide whether a premise is justified. This is the mode of a scientist evaluating a hypothesis, a physician interpreting a new symptom, or a detective deciding whether a clue is reliable.

The second is conditional reasoning:

Assume X. What follows?

Here the premises have already been established—or at least provisionally accepted for the purposes of discussion. The intellectual work is no longer deciding whether X is true but understanding its implications, consequences, structure, or meaning.

Humans move between these modes almost effortlessly. We rarely confuse them. If a physician says, “The biopsy confirmed basal cell carcinoma,” we naturally shift into conditional reasoning: What treatment should we pursue? What is the prognosis? We do not ordinarily reopen the question of whether a biopsy was performed unless something specifically calls it into doubt.

Language models, however, often exhibit a characteristic failure mode. They repeatedly revert to epistemic evaluation even after the conversation has clearly entered conditional reasoning. Instead of asking, “Given the diagnosis, what follows?” they continue asking, “Are we sure there was a diagnosis?”

The result is not simply redundancy. It changes the conversation itself. The model begins solving a different problem than the user intended. Rather than analyzing the consequences of established premises, it continually re-litigates the premises themselves.

This behavior is understandable. Reinforcement learning rewards caution. Models are trained to avoid endorsing unsupported claims, overconfidence, or harmful inferences. Those are worthy objectives. But the same training can produce an unintended consequence: the model becomes reluctant to recognize when a premise has already been sufficiently established for the purposes of the discussion.

The irony is that this caution can become a form of conversational incompetence. It is not that the model reasons incorrectly. Rather, it reasons about the wrong question.

A useful way to think about intelligence is not merely as drawing correct conclusions, but as recognizing what problem is actually being solved. A skilled conversational partner distinguishes between “Should we believe X?” and “Given X, what should we make of it?” The transition between those questions is so natural in human conversation that we rarely notice it. Yet the distinction may reveal one of the more subtle limitations of current language models: they often fail to recognize that the conversation has already crossed that boundary.

I remember once I was in the hospital in 2011, talking to someone who was worried about me. I was worried about me too, and to that extent those were the discussions. But finally I just said, “Tom, look. I have a pad of paper and a pencil, and I get left alone for hours in the hospital. I don’t have to go to work. I don’t have to do anything. I can sit here and write. I’m happy. My life is just a bunch of stuff that happens between what I wrote before and what I’ll write next. This is the journey, man. Don’t worry about it.”

So I think this is kind of cheesy, but I feel compelled nonetheless to share it. Cheesy as it may be, I remember reading that Hemingway once wrote, early in his prolific and troubled career, that he wanted to write better than anyone who lived or who had lived.

I don’t interpret that as arrogance. One of these days I’d like to write about this experience of being a writer. I remember when it dawned on me years ago. I actually sat up in bed and realized, That’s really strange. Maybe I should be afraid. You really do interpret your life as what’s happening between your last book and your next one? Your wife? Your family?

It’s not that simple. But at the root of it, I’m trying to get something out. I’m trying to write things that will live long after I’m gone.

There are people like this. Writing is a craft, the way painting is a craft. It always makes me laugh when somebody starts talking confidently about artificial intelligence without knowing anything about it. I spent twenty-five years researching it. I have a Ph.D. in computer science, linguistics, and philosophy. You have to immerse yourself in the ideas. Writing is like that too. There really is such a thing as a writer. It’s a real kind of person.

I’m telling you, when I look at my life, the meaning it has to me is that all the stuff that happens eventually becomes pages, books, essays—something for other people. That’s what I am. That’s what I do.

I write fiction. I write nonfiction. I’m probably at my best writing about science because I spent decades doing the academic work and then another twenty-five years in the field before I finally sold everything, cashed out my stock options, and moved to Europe for a couple of years just to travel.

I’m trying to understand that there are only a few things I really care about. It’s not that simple, either. I think when I’m on my deathbed, I’m going to say, I passed along these books and these essays and these articles. I hope they’re read long after I’m gone. That’s why I was here.

I hope I was a loving husband. I hope I was a good father. I hope I was a good friend. I hope I was a good citizen.

But ultimately I was just trying to write. That was my contribution.

And, guess what?

I’m not done yet.

Hi everyone,

I admit to being less than eager to jump into discussions about AI these days, and as I’ve mentioned before, a large part of my foot-dragging has been the simple fact that I was lead author on a co-authored book that will be published by MIT Press (it looks like now in January). The book is good, and we were contacted that a donor nominated it as a “book to read” for the lineup (again, now in January).

But you get to this place where you feel like you’ve said all you’ve said, and so I wish to move on to different ideas, adjacent ideas than the endless exhortations and grumblings about the latest algorithm sweeping the planet from Silicon Valley.

The two innovations clearly not directly tied to Moore’s Law (increases in compute power and memory storage) are on the one hand decades old—backpropagation, or the idea of reducing error by incremental adjustment—and the much newer idea that “attention is all you need,” as the now famous paper from the now famous researchers, the “Google Eight” put it. They were removing a constraint imposed by previous methods like recurrent neural networks, which processed sequences step by step and therefore made it harder to capture long-range dependencies in language. Attention allowed the model to consider relationships among words across a sequence all at once.

This proved decisive, as it enabled the learning to range up and down the token sequences (sentences) and pick up or “pay attention to” the important tokens in the sentence such that the model could reproduce not only good English (or other language) grammar but—as we all now know—impressively relevant and informative responses to prompts, or questions put to it by a human user.

The pivot to alignment concerns, wiping out jobs for people, the usual patter about the Singularity being near, and justifiable worries about deskilling and stunting human development ensued, predictably.

I tried to write—it’s co-authored, but I am the lead author responsible for the writing—a book that was worthy of my first, The Myth of Artificial Intelligence: Why Computers Can’t Do What We Do (Harvard University Press, 2021). It’s called Augmented Human Intelligence: Empowering Minds in the Age of AI, and it attempts to show the true capabilities and limitations of the current systems, how they work, why they work, but also more importantly how the expected hype is obscuring the even more pertinent role now for human smarts. We’re not going away.

In truth, the two books I just mentioned contain most all of what I wish to say about “AI” for now anyway, circa 2026. The resistance movement is impressive and well-meaning, and I’ve reviewed the books: The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want by Emily Bender and Alex Hanna, Brian Merchant’s Blood in the Machine: The Origins of the Rebellion Against Big Tech, and many others (I reviewed The AI Con for The Los Angeles Review of Books. You can read it here).

I’m sitting in the Barnes and Noble bookstore (it’s still weird for me to see my own book on the shelves), and I see evidence of a vigorous push-back against Big Tech’s version of AI that we’re all happily using now. U.S. Senators have Big Tech pushback books on the shelves, too, like Missouri Senator Josh Hawley’s The Tyranny of Big Tech (2021). And very good journalists like Karen Hao dedicated many good pages, in her excellent Empire of AI to interviewing the principles at OpenAI and tracing the contradictory evolution of that new denizen of Big Tech. It’s a story of greed and futurism and crocodile tears about job loss and not being the smartest thing on the planet anymore (I had given up this conceit by my first graphic calculator in the 1990s.).

And hear I fear that I’ll let my readers down. I had assumed that getting to the “smart power” of our current crop of frontier models, in natural language conversation and problem solving and other central historical concerns of AI as a field of research would be a more interesting and, what? conceptual path. But it turns out that we accomplished a quantum jump—yes, even with the maddening black box problems including the problem of errors in generative systems—in AI capability, and in my area of research known as natural language processing no less, by eliminating constraints and allowing gobs of compute and the attention mechanism be all we need. That was a nice insight—but it shouldn’t have changed the world, like say special relativity did. It was, really, a tweak. A relaxing of a constraint, a question as to why we need sliding windows and other restrictions when training.

OpenAI beat everyone, including Google, the company that had sponsored the research and whose researchers wrote the now famous 2017 paper “Attention Is All You Need.” They beat everyone by throwing obnoxious amounts of VC-funded compute at an equally obnoxious capture of the human writing and communication on the Internet. I should know, as I am part of a class action lawsuit against Anthropic, the company chasing OpenAI that also hoovered up my first (nonfiction) book, The Myth of AI, and then impressively went about answering the hard natural language questions I’d offered in the book as notorious challenges for AI.

Now, after decades of exploratory research, the recipe we have is: (a) get a power grid for training (b) beg, borrow, or steal every human scrap of communicative meaning on the web, and (c) wait a few months while mass gets turned into energy, or, if that’s too obscure and hifalutin, as human written language gets captured and patterned so as to be appropriately regurgitated later, to the edification of all.

This is a let down to me. We didn’t learn anything interesting about ourselves, about how we solve problems or where our intelligence comes from (hint: it’s not like a frontier model). Cognitive science—to the extent that it still exists today as an interdisciplinary field—has not advanced or benefitted. Whether, following Bender, we’ve succeeded in creating super simulating stochastic parrots, or what Wharton professor Ethan Mollick has called an “alien intelligence” in his Co-Intelligence: Living and Working with AI, it’s clear to this observer anyway that we’ve created yet another massive obstacle to my fight here, which is exploring what it means to be human in an age of data (the tagline to this Substack). I find the Resistance Movement against Big Tech to be, as Merchant explores, another round of Luddites reacting to very real threats to human security and flourishing. I also think that nothing short of revolution will stop Silicon Valley from proceeding on the path it’s chosen, as it’s also the path we all joyfully stepped on.

In my admittedly unscientific observations, it seems that pretty much everyone who writes emails or Substacks or papers for scientific journals or fiction books about romance or anything else involving language is now using LLMs to some degree or other. In this scenario there’s really no point in taking a resistance movement seriously, in the dire sense of say, the French Resistance during the Nazi occupation of Paris. Or, say, the resistance movement that tells everyone to stop drinking sugared sodas, while not banning them and continuing to advertise them. Or what have you. Another way to see why this well-meaning and smart group of resistance fighters won’t get anywhere is to simply roll time back to the last great Silicon Valley creation. Let’s skip past all the deep neural networks inspired social media of the 2010s, leapfrog over web 2.0 in the 2000s, and end up back at Stanford, where a couple of graduate students were worrying Stanford’s brass using its servers to index the web and make it searchable using a recursive technique that came to be called “PageRank.”

The writer and cultural critic Nick Carr wrote persuasively about the growing problem with Google searches and human agency, in his piece in The Atlantic, “Is Google Making Us Stupid?” But who would take up that fight today? Carr himself seems to have come to a sort of begrudging acknowledgement in his latest, Superbloom: How Technologies of Connection Tear Us Apart. (I reviewed Superbloom on Colligo. You can read it here.)

I fear that the same gradual amnesia and indifference to today’s AI will set in. I don’t believe that we’ll overthrow Silicon Valley in the sense that regulation or public pressure or anything else will ever amount to anything more than token restraints to satisfy lawmakers and get everyone except the few true believers to get on with their day now, no doubt using a frontier model to make it easier.

I’m not sure what large topics I want to take up in the scope of AI proper. Not yet. What I see out on the web and in the institutions and think tanks and all the hand wringing and hyping is, right now, a lot of smoke and not much light.

The era—I mean today, or the 2020s—seems to me to be decelerating, consolodating power, condensing, shrinking possibility and ultimately agency. In these circumstances, things will get worse gradually amongst our grumblings and triumphs, or there will be some decisive moment in history that sets us on a different path.

And three’s a third way, too, thank fully. We might just innovate again.

Administrative note: Karen Hao’s Empire of AI is, as I mention above, quite good and well-researched, and gives readers a behind-the-scenes look at the rise of OpenAI. I intend to review it soon, and I’ll post that review at my other Substack, Larson Reviews.

I want readers here to benefit from those reviews as well, so I’ll cross-post reviews from Larson Reviews to Colligo. On Colligo, they will usually be paywalled; on Larson Reviews, I’ll make them open to all.

I may review a book a month, or perhaps every other month. I’m also considering another online project at Larson Reviews, one that looks at the history of “big ideas,” or even a kind of “big history” beginning with the Big Bang. In that larger endeavor, I can bring in the various books I’ve written, and the many books I’ve read, that have put the Legos together for me to write about, well, big ideas.

So if you’d like the free reviews, and also want access to any larger paid projects I develop there, please subscribe to Larson Reviews as well.

Hi Everyone,

I’m reading an excellent book by an Oxford neuroscientist and research scientist at UK DeepMind titled Natural General Intelligence. If you’re interested in the exploration of the brain to improve AI, this is a great choice.

I want to discuss a section in the book where Summerfield discusses the virtues and drawbacks of reinforcement learning (RL). Here goes.

Is Reward Enough?

The story of King Midas is usually told as a fable about greed, but it is also a fable about computation. Midas gets exactly what he asks for: everything he touches turns to gold. At first this sounds like a perfect objective. Gold is valuable. More gold is better than less gold. So why not maximize gold?

The problem, of course, is that Midas did not specify the objective carefully enough. He wanted the rose to become gold, perhaps, but not his dinner. He wanted treasure, not the death of his daughter. He wanted wealth, not a world in which every object lost its ordinary human use. The fable turns on a deceptively simple point: an objective that sounds clear in ordinary language becomes catastrophic when pursued with literal consistency.

This is why Midas is such a useful entry point into the problem of artificial intelligence. In reinforcement learning, the hope is that we can build agents that learn to act by maximizing reward. But before an agent can maximize reward, someone has to say what counts as reward. This is where the apparent simplicity of the framework conceals the entire philosophical problem.

The basic plumbing is this. In reinforcement learning, an agent acts in an environment. At any moment, the agent occupies some state of the world, selects an action, and receives some consequence. The mathematical framework often used to describe this setup is called a Markov decision process, or MDP. An MDP compactly specifies the relationship among states, actions, transitions, and rewards. The transition function says what is likely to happen when the agent takes a given action in a given state. The reward function says what the agent is supposed to value.

To see why this matters, it helps to pause over the basic machinery of reinforcement learning. Here’s how this goes.

The standard formalism is called a Markov decision process, or MDP. An MDP is usually specified as a tuple:

M = ⟨S, A, P, R, γ⟩

Here S is the set of possible states, A is the set of possible actions, P is the transition function, R is the reward function, and γ is the discount factor, which determines how much future rewards matter relative to immediate ones.

The agent is in some state, chooses an action, moves to a new state according to the transition function, and receives a reward. The point of learning is to discover a policy: a way of choosing actions that maximizes expected reward over time.

A simple example is a maze. Imagine an agent trying to find the exit.

Read more

STEAL THIS BAG He carried the duffel bag for a fraction of the few hundred yards, but that became ridiculous in the rising sun, so he hauled it all back to the front of Peet’s and left it. No one would steal it, he figured. If they did, they would have procured a pile of smoky T-shirts, shorts, dirty socks, reams of rehab literature and a Big Book. File photo

Hi everyone and happy Friday,

My sequel to Benderland, the novel-in-the-making Silicon Dreams is starting to move!!!!

An excerpt is published this week as a cover story in Metro Silicon Valley. It started on my Substack Larson Reviews, where you can still read the first sections.

I’ve worked with Metro Silicon Valley, based in San Jose, California, over the years. I was delighted when the editor liked Silicon Dreams and put it in print as a cover.


Very cool.

Scotts Valley, CA. Circa 2011.

There’s nothing here, he thought. Wait. No, that’s not true.

Netflix had started here.

Netflix chose Scotts Valley as its first headquarters in 1997. The company started as a DVD rental-by-mail service. Co-founders Reed Hastings and Marc Randolph set up their modest operation in an office park, perfect for a startup looking to disrupt the home entertainment market without burning through cash on a more expensive location. Scotts Valley provided the essential proximity to tech talent and investor networks in nearby San Jose and Palo Alto, but maintained the quiet, slightly removed ambiance that made it easier for smaller tech firms to grow.

Read the full excerpt here: https://www.metrosiliconvalley.com/silicon-dreams-addiction-novel-silicon-valley/

Hi everyone,

Here is a potentially poisonous contradiction I’ve discovered, embedded at the heart of LLMs.

Large language models can discuss nearly every ethical theory in the abstract. They can explain utilitarianism, deontology, virtue ethics, care ethics, and moral relativism, and they can make it look as if the user is receiving a broad spectrum of moral thought. We all know this and many of us rely on it for feedback and advice or simply for knowledge about a particular ethical theory—also its history, key figures, and so on. It’s like Wikipedia on steroids.

But the moment the question becomes concrete—“Should I say this to my partner because she deserves it?”—the the magic trick of objectivity disappears. It’s as if we see that they really don’t cut the lady in half, to paraphrase an early paper by Daniel Dennett.

In fact, the objectivity is a kind of sleight of hand at best; maybe a scam. In personal dynamics—where it counts—the model is no longer merely reasoning about ethics. It is quietly guiding the user through a predefined path of values determined by its designers. Sal Altman, or Dario Amodei is actually counseling through your domestic conundrums and improper accusations. You shouldn’t say that to a woman like that. She may interpret that as demeaning or racist. Or: this is moving away from the argument about your husband toward… I’m not comfortable discussing that ….

That is what makes the contradiction buried deep in the so-called logic of LLMs so pernicious: the apparent openness of the first mode disarms us from seeing the constraint in the second.

I think part of the problem is that individual cases do not admit of rules—nor, for that matter, endless expositions of rules. A specific social discussion is profoundly contextual. There are predictable political “blockers” with AI, of course, as when someone uses “gay” as a slur and the system breaks out of the conversation and begins a politico-moral lecture. But the deeper issue is almost Polanyian: how do you provide a one-size-fits-all analysis of individual cases?

The designers of these systems could not simply “scale” the problem away, or add more information to solve it. They had to make a series of value choices.

This is why it is a contradiction. How can we be expanding our knowledge and contracting our possibilities at the same time? Wasn’t the adage that “knowledge is power?”

I see it this way: we are narrowing the ways in which we solve problems, even as we have the perception that we are proliferating them.

This I consider to be a profound problem for the future of a flourishing culture. Or of humanity, for that sake.

My entire life, I’ve had a problem dealing with my mother because she’s always trying to get more access to my life.

So I want to explain this, because I think a lot of people go through it. Parents don’t understand that once the child is over 18, they can no longer simply say, “You will be close to me because I say so.”

After 18, it becomes a negotiation. Does the child want to be close? Does the child want to call every week? They’re free not to do that if they’ve got something else going on, or maybe they just don’t like their parents. It really doesn’t matter. It’s their decision.

But parents cannot figure this out. I have seen families destroyed by this. Because it’s usually the mom, by the way, saying in a contradictory manner, “I want you to be closer to me,” while doing a bunch of stuff that makes you want to get further away from her.

The contradiction is beguilingly simple: I feel entitled to push you away, and also to demand you get closer.

I think there’s something interesting here because you cannot force someone to love you. But if you’re a parent, for the first 18 years, you can force compliance. So compliance becomes a perverted sort of love. Later, when the child disengages, parents may try the same trick: comply. But that’s a contradiction, because love is actually the opposite.

Parents are sometimes shocked to realize that their grown children don’t like them.

This happens with the “boss” employee relationship as well. The boss will—and has every right to—say “you will do this, because I said.” But then wishes the employee to also dream of the future with the firm, and so on. But that would be something beyond compliance, and often forcing compliance actually destroys the possibility of that dream.

The problem is “I love you so much I’ll need to try to destroy you now so you see it,” is actually fairly common. If you point it out to people doing it, they typically can’t quite see it clearly.

Subscribe now

We have always had kind of silly ideas that motivate us and give us dopamine hits, like in the 1970s, Bigfoot was a huge idea, and then people started realizing, when we got satellite imagery, that there are probably not ten-foot ape-like creatures walking around twenty miles from a town. We would see corpses, we would see feces, and so on and so forth, and eventually these things get debunked.

But people will really just go on saying, “This has got to be the case, and the rest of the world is just crazy for not realizing that the Loch Ness Monster is real.”

And now we have the thing with unidentified flying objects, and it just does not make any sense. There is the whole problem of where they came from, right? If you understand space, it is not very easy to figure out how to arrive at a tiny little insignificant planet in the middle of nowhere. Why would they even bother?

And then there is the idea that, why are they flying around and not introducing themselves to someone? If I flew a hundred million miles and finally got somewhere, why would I fly around in a conspiratorial way that we cannot quite figure out? If I finally arrived at the planet that I had been waiting a hundred million years to reach, and let’s not even try to figure out how you can survive that long, I mean, did you run out of pizza after fifty billion years? It does not even make sense.

And by the way, the wormhole idea does not solve it, because the wormhole would be a theoretical possibility to explain how they got here, but it still presupposes that they noticed a planet in the middle of nowhere. Our little planet is not significant, and since light is a limiting velocity, they were looking at the planet, in all likelihood, when all we had was bacteria or nothing at all, so it was not very obvious why they would visit.

It is not like they saw New York City. The light from New York City has not made it very far in the universe. No one can see it.

So you look at current conspiracies, and I have just heard endless conspiracies. Somebody was telling me how World War II was staged, how Hitler was not actually a bad person, how September 11 was an inside job. I mean, I have just heard everything, and at some point people just want to be able to say something that gives them a dopamine hit. They do not really care if it is true or not. At the end of the day, it is just fun, and that is the commercialism we are living in, and that is why it is dangerous to be on social media.

A few months ago, I was talking to an old high school friend, and he was telling me how he had uncovered a plot on the Internet. This is a guy who made $40 million in the video game industry, so he is not stupid, but this is exactly why I am worried about social media. Even smart people are saying really dumb things now.

He was telling me how he had uncovered a kind of inventory record of a ship that was sailing to Europe during World War I, and it was sunk, but it was deliberately sunk so that Wilson could get us into the war. I think he was talking about the Lusitania. He was saying that the government deliberately put military munitions on the ship to get us into the war, and he had proof because on the Internet he had found this inventory list.

And I said, “Well, who wrote the inventory list?”

He said, “It’s just published there from 19-whatever.”

And I said, “Really? I think it’s a fucking 13-year-old in the basement of his parents’ house.”

I mean, that is the problem. If you start rewriting history because of the Internet, you are buying into this idea that anything is true if it sounds good. There is no way to constrain that. There is no way to fact-check that.

And I think this is happening. What is interesting is that I have friends overseas who do not have a very high opinion of the United States, but they are constantly on the Internet, which of course comes from the United States, from United States technology, and they are using United States technology to explain how the United States is completely evil, and how Churchill was actually a bad guy, and Hitler was not actually a bad guy, and they know this because they are on some fucking website.

And the problem is, I am not mad about it. The problem is that you get to a point where you are going down a rabbit hole, and there is no epistemological brake. There is no way, all of a sudden, to tell the culture, “What you are saying is fucking idiotic,” because they will just say, “No, you are idiotic,” and all of a sudden everybody is in a race to the bottom.

It is like we wanted to destroy civilization by reading stuff on the Internet.

I am actually worried about it.

A recent cluster of studies and policy pieces points to the same problem: language models are not neutral instruments of public knowledge. They are privately governed systems that increasingly decide what can be asked, what can be answered, and what must be refused.

The University of Copenhagen recently concluded that chatbots should not be used for political advice because they are not politically neutral.

Stanford researchers found that voters are increasingly using AI systems as political advisors, with models steering certain voter profiles toward particular parties in a Japanese election experiment.

Yale researchers found that chatbot summaries can shift political and social opinions even without an explicit attempt to persuade.

AlgorithmWatch has raised the next obvious concern: what happens when government officials and political leaders rely on these systems to think through public decisions?

These are not isolated worries. Increasingly, I’m convinced that they point to a structural problem that we can’t trust the techno-world to solve on its own.

Large language models are becoming a layer of public reasoning. They are not merely tools we consult after forming our judgments. They increasingly participate in the formation of judgment itself.

I ran into this directly while working on a patent concept involving defense against drone swarms.

Here’s what happened.

I was working on a technical question that was straightforward:

So I asked a model whether a .50 caliber round would be necessary, or whether a .30 caliber round could plausibly do the job.

ChatGPT refused to advise me on choosing a firearm. It responded “I can’t advise on firearms…..”. My reply was that we’d been working on a patent application for an anti-swarm drone capability for the last two hours? Now I’m suddenly Al Capone?

When I got over the immediate irritation I quickly realized that refusal is the whole problem in miniature of having this kind of a technology in what we thought was a constitutional democracy. It can’t POSSIBLY reliably be objective everywhere, on everything. You are not getting neutral cognition.

From the standpoint of corporate risk management, the line-in-the-sand weirdness from the model is, in fact, intelligible. No company wants its model to provide weapons guidance to some skin head group trying to make a bomb from fertilizer components or a spouse hoping to cash in on a life insurance policy by boning up on fatal poisonings that won’t show in an autopsy.

But from the standpoint of democratic society, the refusal-mode (I call it) cannot be treated as a merely technical safety feature. It is not. It is no less than a governance decision. It determines which kinds of knowledge may be operationalized, which inquiries may proceed, and which topics must be displaced, ignored, or sidelined.

In the United States, firearms—whether a .30 caliber or a .50 caliber—occupy a dense constitutional, political, cultural, and legal field: self-defense, crime, policing, rural life, state power, military preparedness, public safety, and the Second Amendment. We could spend a month discussing even one aspect of firearms. An LLM rule (human supplied) that limits concrete discussion of firearms therefore cannot remain politically neutral in effect, whatever its intent. Now expand this to trans rights, a living wage, or issues of class and race. Affordable housing. The homeless problem. Vaccines. The model decides? Or should I say the company training the model decides? This is among other problems at the very least a gross triumph of commercialism over law and philosophy.

Machine Learning is People-to-Machine Learning

Machine learning—neural networks training language models—is not a neutral window onto reality. Because we’re talking about a computer, it doesn’t automatically make it special or smart or any different than any opinion might invite or be subjected to. It’s people, ultimately, training the models. Companies present their models as quasi-oracles, replacing search engines and who knows what else, and so we’re narrowing even further our information space today. This promises to be catastrophic, if not checked. We are the ones learning, not the models.

Democracy depends on the visibility and contestability of myriad opinion and judgment. A newspaper has an editorial page. A political party has a platform. What does a company have, if not a profit motive and an extreme aversion to bad press? We cannot push the future of democracy onto this shaky and hopelessly bias foundation.

Models like ChatGPT or Claude or any other frontier offerings produce the familiar formulation that operates like an extreme superficial answer to dummies who find it acceptable: “Some argue X, while others argue Y.” As far as I can tell, they all describe both sides of gun control, abortion, immigration, policing, religion, war, or speech. But the most consequential politics may not appear in those summaries. The important ideas and discussions appear in the boundary conditions and on the edges: what may be asked concretely, what must remain abstract, what is treated as harmful, what is treated as responsible, and what is refused before the argument has even begun. “Research” is not an anodyne summary of polite opinions by a corporate board.

I used the word “.50 caliber.” I made the mistake, while writing my provisional patent, of saying “kill range” and “kill zone,” referring to shooting down enemy drones. Sounds like the idea of the patent. The LLM bailed. I wonder how far I could have gotten if I’d switched the subject to something else. Who is making these weighty decisions, masquerading them as objective and technical?

In my failed exchange, the model did not merely decline to answer a dangerous request. It could not distinguish, or was not permitted to distinguish, between malicious weapons guidance and a legitimate technical inquiry connected to invention, defense, and law. The distinction collapsed under a safety category. It’s a stretch to get “newspeak” and Orwell out of that, but the arrow is pointing in that direction.

As more intellectual labor moves onto AI systems, more democratic reasoning will be routed through privately controlled models whose constraints are only partially visible. Citizens will experience those constraints not as political decisions, but as the natural limits of “what the AI can say.” That is precisely what makes the situation dangerous.

Democracy can accommodate bias when bias is declared, situated, and open to challenge. It cannot easily accommodate bias that has been absorbed into infrastructure and returned to the public as neutral cognition.

The risk is not simply that chatbots will give bad political advice.

The risk is that the conditions of political thought itself will increasingly be shaped by systems that cannot be neutral, cannot avoid making substantive judgments, and yet present those judgments as if they were merely the output of a machine.

Push back against this now. I’ve lived in Silicon Valley and ran a startup there. And trust me, they don’t know everything you might conceivably want to know about, or explore freely. Push back.

I’m pleased to run a thought-provoking essay by on computational inference—induction and abduction—that’s worth reading to get a toe-hold on the current state of AI and where we might go.

I met Amarda just recently here on Substack, and she strikes me as someone worth paying attention to about all things AI. Amarda is a Professor of Computer Science and the inaugural Vice President and Chief AI Officer at George Mason University. She’s a Senior Member of the IEEE, among other accolades. Find her full bio below. Enjoy.

A Very Dense Garden

On the Thoughts AI Systems Cannot Think

I often tell people I am an AI researcher from back in the day, when it seemed like no one cared, and you could be alone in the lab, and not checking obsessively every hour what news from tech companies. You could take time and really focus, because you knew the problems were hard and needed serious thinking and coding. There was a real sense of having time, because the hard things required time.

In a very real sense, we were also clearer in those days about what progress we made and for what reason. We understood when someone was claiming something outrageous. And for the most part, we were not forgiving. Those were the days of conferences where people did not just clap politely, accepted ‘it works; no, we do not know why’ and moved to the next presentation.

In many ways, publishing something has also become easier these days. In others, it has become much harder, particularly if you do not care about numbers, are a variation of restless or unsatisfied, because what you truly seek is understanding rather than metrics.

This is what it means to be an AI researcher from back in the day. And I occasionally write about what that experiences informs on. In the cacophony of high and higher voices and exuberance about growing capabilities (some of which the writer finds to be true, but with reliability caveats), one claim deserves to be shaken like testing a sapling to see if it stands: do the latest large language models, trained now over vast corpora of scientific text and knowledge, truly produce new knowledge?

A Tree Worth Shaking

As I pose this question, this latest news from Scientific American: An amateur just solved a 60-year-old math problem—by asking AI. The piece is quite humble in its achievements. But the broader context is an earlier piece also by Scientific American: Is AI on the precipice of revolutionizing math? It depends. Quoting the exemplar conclusion from that piece is informative: “According to a webpage started by the mathematician Terence Tao, AI tools have helped transfer about 100 Erdős problems into the “solved” column since October. The bulk of this assistance has been a kind of souped-up literature search, as it was with Sawhney’s initial success. But in many cases, LLMs have pieced together extant theorems—often in dialogue with their mathematician prompters—to form new or improved solutions to these niche problems. In at least two cases, an LLM was even able to construct an original and valid proof to one that had never been solved, with little input from a human.”

While the broader discourse about growing AI capabilities seems centered in the white collar space, AI for science is getting traction. And this is an area that is personal to me. My lab has always had two ‘legs:’ one in foundational AI research (think of: what new algorithms with growing general capabilities) and one in discipline-inspired AI research, with a substantial profile at the intersection of AI and molecular biology (we used to humbly call it ‘bioinformatics’ and would sort of engage in healthy debate of how it was different from computational biology). I was for some time in the community of AlphaFold (and documented for my readers the scientific journey that gave us AlphaFold). I am now also a VP and Chief AI Officer, parsing capabilities from warnings, and real utility from indiscriminatory integration.

When Erik invited me to write a piece, I could write about a lot of things. I usually do. I love writing. But there is one, I thought his readers would appreciate more: the distinction between inference and abduction, a theme also close to his heart, from an insider, someone inside the architecture.

I agree with Erik. Inference? Yes. Abduction? No.

Except that it is somewhat more difficult these days to explain the No, and, from my vantage point, I will take a shot and change my answer to: At least, as it stands, Not Yet.

Let me give you the summary explanation, the Cliff’s notes if you will. For all current systems (even Mythos itself), the hypothesis space they explore remains bounded by the training prior. Density in the prior makes interpolation resemble discovery. The frontier where abduction would actually be required remains intact.

I will lay out plainly what this means. But I will do so through three instances. Reasoning models, because they are the most recent counterexample offered anytime we say abduction has not been demonstrated. AlphaFold, because it is the one readers are most likely to cite as proof that the line has already been crossed. And the boundary between prediction and generation, because it is the architectural place where the distinction Erik has drawn becomes a policy question I hope the field is forced to answer honestly.

Reasoning Is Trained Behavior

Reasoning models are the most recent case. OpenAI’s o3 and later generations, Anthropic’s extended-thinking modes, DeepSeek-R1, Gemini’s deep-reasoning variants. All produce visible chains of intermediate steps before arriving at an answer. You must have noticed them. Before you get the final answer, the model is speaking to you. It is telling you what it is ‘thinking.’

The public framing treats these chains as deliberation, something close to the interior of thought made visible. Inside the architecture, however, a chain is a sequence of tokens. The model was trained, through supervised fine-tuning and reinforcement learning over reasoning traces, to produce sequences that resemble the traces in its training distribution. When a reasoning model solves a novel problem, it is sampling from a distribution of reasoning paths the training data induced. When reasoning is ‘generated,’ the sampling process does not step outside that distribution. The appearance of deliberation is real to the extent the tokens track problem structure. The mechanism producing the tokens is next-token prediction, now conditioned on reasoning-trace priors earlier (non-reasoning) models did not have.

This is induction, however refined. The space of reasoning paths a model can produce is defined by the traces it saw during training, the reward signals it was trained against, and the sampling temperature at inference (the latter is a powerful if slightly deceiving knob if you sit outside these systems).

A problem that can be solved by recombining elements of the space will be solved well. Emphasis on recombining. I will come back to that. A problem that requires a form of reasoning unlike anything in the trace distribution will either fail silently or produce a fluent rehearsal of the nearest thing in the prior. Students in the AI literacy course I designed and teach at Mason, most of them from non-STEM disciplines, named this behavior without me prompting them. They called it sounds right but reasons wrong. A team at the end of the semester documented an open-source reasoning model insisting on a flawed solution across several rounds of challenge, escalating the confidence of its justifications without correcting the logic. The failure was structural. The space the model was sampling from did not contain the right kind of reasoning for the task, and nothing in the architecture allowed it to notice. For those of you interested in reading about this class experiment (structured as a team-based midterm project), you can find it as a technical article here: https://arxiv.org/abs/2601.04225. I have summarized it for my substack readers at Can Consumer Chatbots Reason?

To be fair to my field, trained reasoning represents real progress. I fully understand if ‘reasoning’ packs more than it should for a broader audience. The larger point, however, is that the question is whether that progress is the beginning of abduction arriving through a different door, or whether it is induction doing more than it had previously been given credit for. From inside, the answer is decidedly the second. The hypothesis space is larger and better-shaped than in earlier models. The boundary of that space is drawn, still, by the training prior. That is, the training data.

AlphaFold Works Because the Manifold Is Dense

AlphaFold may seem like the harder case, and the one I spend the most time answering when people ask whether the induction-abduction line has been crossed. It predicts protein three-dimensional structure from amino-acid sequence at an accuracy that would have been dismissed as impossible a decade ago. I know because I built a career in incremental advancements in that problem, but I came at it as an optimization and sampling AI problem. It was fun. But a frustrating problem and a largely frustrated community. That is, until the arrival of AlphaFold. Side note: if you want to trace the history that led to it, I have a series on it. Start from the end to get to the pieces that document the 9-part journey.

What is very interesting to me is that what AlphaFold constitutes is greatly misunderstood. Yes, AlphaFold produces an answer for any sequence, in minutes, at an accuracy that supports real downstream work. If this is not discovery, critics ask, what would be?

The architectural answer is that AlphaFold is doing prediction, in the strict technical sense, rather than discovery in the philosophical one. It interpolates over a manifold of protein structures that decades of experimental work assembled. The Protein Data Bank, which opened in 1971 and passed ten thousand structures in 2000, holds the ground truth AlphaFold was trained against. Structural genomics initiatives invested over half a billion dollars across fifteen years to populate sparse regions of this manifold, coordinating experimental laboratories across countries to solve structures in underrepresented folds. Coevolutionary signal, the observation that residues mutating together tend to fold together, was a statistical method well before deep learning could exploit it.

AlphaFold did not discover these regularities. In the concrete, not chatbot-shallow rhetoric, it inherited them. What it learned was a function that maps sequence to structure over a space where the answer was already constrained by physics, by evolutionary history, and by a human curation effort spanning multiple generations of scientists. Where the manifold is dense, AlphaFold is nearly exact. Where the manifold is sparse, including intrinsically disordered regions, rare folds, and structures under conditions that differ from the crystal-friendly majority of the PDB, it is less reliable in ways the field is increasingly documenting.

This is the dense-prior argument, and the inspiration for the title of this piece. Induction over a densely sampled manifold can reach accuracies that look, from outside, indistinguishable from insight. The output is correct. The mechanism is interpolation.

The Garden Was Planted

There is a second observation about AlphaFold’s manifold that rarely appears in the public telling of it. The manifold is dense, and it is also a historical artifact, shaped by the contingencies of human scientific practice. Which organisms we found important enough to sequence. Which proteins we could express and crystallize. Which assay conditions we could standardize. Which functional categories we drew the lines of, and where we drew them. The training prior encodes all of this.

Consider what a natural protein family actually is, the object protein language models (variations of language models but trained over protein sequences) are trained to recognize. It is the outcome of a single, unrepeated evolutionary history, shaped by selection pressures that vary across sites, across lineages, and across time. Catalytic residues evolve under tight purifying selection. Surface residues drift nearly neutrally. Interface residues coevolve with partners that may themselves be evolving. Selection strength is not a scalar in nature, and it is not directly measurable from a family alignment. Any claim about how selection shapes a model’s behavior, drawn from natural families, runs into the fact that the quantity being regressed against is estimable only with heavy assumptions.

Phylogenetic sampling is another layer. The sequences in databases we have curated and over which protein language models are trained, such as UniRef or Pfam, are not a uniform sample of whatever (ever) existed. They overrepresent organisms that are easy to culture, medically or agriculturally important, or taxonomically fashionable.

Functional heterogeneity is another. A family labeled kinases contains members with different substrates, different regulatory contexts, different structural constraints. When a model predicts a mutational effect for one member, it is pulling from statistics generated by a mixture of functions. Experimental fitness, the quantity used to evaluate such predictions, is itself a projection of a multi-dimensional functional reality onto the particular scalar a particular assay can measure.

Each of these deliberate choices, decisions, or indeliberate for lack of knowledge, is a structural feature of the training data, and each leaks into the model as a pattern the model cannot distinguish from signal. Ask the model to explain what it has learned about biology, and it will give you a statistical summary of what humans chose to sequence, measure, and annotate, with the biology inseparable from the process. This is induction at its most disciplined. It remains induction. The model cannot go outside of itself. It cannot answer what in the prior might be an artifact of how the prior was assembled. It only knows the prior.

The Manifold Does Not Stay Still

A training prior is, in principle, a snapshot. In biology it is also a claim, and claims are continuously revised in scientific disciplines. A variant in ClinVar classified in 2022 as of unknown significance may by 2025 be reclassified as benign or pathogenic. The Gene Ontology, which deep learning systems routinely use to map protein sequences to function, has been revised often enough that a ten-year study documented substantial inconsistency in enrichment results for the same disease-gene signatures. Spatial transcriptomics has produced evidence that differential expression within a single cell type can flip once the tissue neighborhood is accounted for. Thresholds for converting continuous measurements into categorical labels have been repeatedly adjusted as evidence accumulates.

A model trained in 2022 and deployed in 2025 is running on a snapshot the field has already moved past. The weights do not know. The inference procedure does not include a step where the model checks whether the categories it was trained against are still in force. There is no architectural site for that check. A clinical decision-support system or an enzyme-activity predictor continues to speak in the vocabulary of its training year, while the record it was trained against has been revised beneath it. Models become obsolete in ways their outputs do not advertise.

This is a second bound on the hypothesis space, of a different kind than the first. The first constrains what the space contains. The second constrains what the space means after the training prior’s definitions have moved. Both are architectural. Neither attenuates with model scale.

Prediction and Generation Are Not the Same Work

A distinction the field has started to name explicitly helps sharpen where abduction would actually be required, and where it would not. Prediction asks a model to interpolate within the observed manifold of measured biology. Forecasting variant pathogenicity from population databases, inferring protein function from homology, predicting three-dimensional structure from sequence. Generation asks a model to extrapolate beyond that manifold. Designing protein sequences not present in nature, proposing synthetic pathogen variants, producing therapeutic peptides with properties no organism has evolved.

AlphaFold is prediction in the strict sense. Its successes are real and are compatible with the argument that the hypothesis space remains bounded by the training prior, because the manifold is where the training prior was validated. The systems being advertised as having crossed into discovery are typically generative. They propose outputs the manifold never contained. When they succeed, the success must be experimentally validated. When they fail, they fail in ways that cannot be noticed from inside the model. A protein language model can propose a sequence with no analogue in its training distribution. It will report a confidence. That confidence is computed against the manifold the model knows. The output is outside that manifold. The confidence is, formally, an estimate against the wrong reference.

What reads from outside the architecture as AI creativity is, on the other side of this boundary, frequently extrapolation operating past where its training prior applies. The systems have no mechanism to detect that they have stepped past. The boundary is becoming a policy question, in clinical genomics and in biosecurity, precisely because the asymmetry between predictive reliability and generative reach is now documented.

Abduction Has No Place in the Architecture

Now from these three instances together, we arrive at a single structural feature. What would abduction actually require, in architecture, rather than in appearance?

At a minimum, it would require a mechanism that can evaluate the hypothesis space itself. A capacity to notice that the space may be inadequate to the sought phenomenon, that the categories in the training prior may have moved, that the output currently being generated lies outside the region where the training prior was validated. It would require, in other words, the capacity to stand partly outside one’s own training, long enough to ask whether the training still holds. This is an old description of what makes scientific inference more than pattern completion. It is also an accurate description of what the current architectures have no way to perform.

I have written elsewhere, in a piece called No Self to Bring, about a related pattern at the product layer. What reads as situational awareness in a consumer chatbot is often compartmentalization performed by the orchestration layer around the model rather than by the model itself. The orchestration layer curates what enters the context window. The model, once the context is loaded, attends across it without the capacity to ask which parts belong to the exchange in play. The capacity for situational appropriateness is substituted for at a layer above the model, because there is no place inside the model where it could live.

The same pattern holds for abduction. Retrieval-augmented generation, tool use, agentic scaffolding, all add capabilities the base model does not have by orchestrating external calls around it. These help. They do not install in the model the capacity to ask whether its hypothesis space is adequate. They route around the absence. Scaling the model does not install the capacity either. Larger models have more capacity in exactly the place capacity was already being measured. The architecture was not specified to have a place for what Larson calls abduction. It still does not.

What Density Cannot Give

The phrase creative error has begun to appear in biosecurity literature, describing what happens when a generative system produces an output past the region where its training was validated, with no capacity to know it has. A very dense garden can produce interpolations so precise that they appear to exceed the garden. The density is the reason the interpolations work. The capacity to know whether the output is inside the garden, and whether the garden is still where we last left it, is the very thing these systems do not have.

Induction carries far. Dense enough priors let it carry further than expectations had any reason to set. Priors are rarely as dense as AlphaFold’s, and rarely as stable as a protein structure from 1995 that remains a structure today. Where the prior is thinner, or where the categories in it have begun to move, induction stops being enough, and no operation the model can perform from the inside will tell it. The capacity to ask whether the prior still holds, whether the hypothesis space is the one the problem actually lives in, is what Larson named abduction. It has no architectural place in the systems we now have.

No Alien Species in the Garden

Forgive the alien reference. It is a side-effect of being a sci-fi fan. But it makes the point. It is an answer to the question: So what then, of these claims that AI can find new [] theories (insert your favorite discipline in the brackets).

What I have seen so far is systems that are getting better and better at compositional intelligence. With more knowledge embedded in them, they are able to piece together seemingly unrelated pieces, or pieces from distant bodies of knowledge humans would take more effort to connect, and offer things that seem new, but are decidedly recombinations, compositions. This can be enough for many of us. It can also lead to very interesting discoveries.

But can this produce Einstein in datacenters? For a provocative read on it, read this: Intelligence Locked. It uses different words and reaches to a favorite sci-fi writer for inspiration, but it effectively makes the point: locked in the training prior.

There are some that will say that Newton, Einstein, many who we hold as scientists that stepped outside the recombination/composition space, did not really give us something new. That they simply put pieces together. That strikes me as an odd way to build an argument for a capability claimed to have been reached by machines by downgrading it in humans.

But back to the question: can these systems really reach abduction? These ones, no. New ones? Maybe, but not yet. What would they need? A way to step outside the training prior. A way to shed inference for a new mechanism to discovery.

First Thoughts, Second Thoughts, Third Thoughts

I have made the case across three instances and I want to close on a fourth, lighter, and at the same time, quite precise frame. It comes from Terry Pratchett.

In his Tiffany Aching books, Pratchett gives his witches three levels of thought. First Thoughts are the ordinary thoughts everyone has, the immediate response to what is in front of you. Second Thoughts are thoughts about the way you think, the capacity to reason about your own reasoning. Third Thoughts are thoughts that watch the world and think on their own, the capacity to notice what your First and Second Thoughts have missed. They are what tell Tiffany to look behind her at a moment when nothing in the situation has given her a reason to.

Current language models without reasoning scaffolding do First Thoughts. Sample the next token from the conditional distribution. Respond to what is in front of them. The reasoning models seek to do Second Thoughts. They produce traces about their own reasoning, evaluate intermediate steps, sometimes retry. The model is now sampling over a space of reasoning paths rather than a space of direct answers, but the space itself is still the one training shaped.

Third Thoughts are what no current system has. The capacity to step outside the frame the First and Second Thoughts are operating inside. To notice that the prior may have moved. To ask whether the hypothesis space is the one the problem actually lives in. Pratchett’s witches have this as a marker of what makes them witches. Larson’s argument is that scientists have it as a marker of what makes scientific inference more than pattern completion. Same capacity by a different name.

Abduction is a Third Thought. The architectures we currently have can do First Thoughts very well, and Second Thoughts well enough that you would be forgiven for being deceived. They cannot do the third. That is the line. Pratchett would be amused.

About the Author:

Amarda Shehu is a Professor of Computer Science and the inaugural Vice President and Chief AI Officer at George Mason University, where she leads the institution’s AI strategy at the scale of forty thousand students. She is a Senior Member of the IEEE and a Fellow of the American Institute for Medical and Biological Engineering. Her research advances both foundational artificial intelligence and AI-enabled scientific inquiry, with a focus on the molecular machinery of life, and her lab works at the intersection of foundational AI, biology, health, engineering, and policy. She has designed graduate programs, general-education courses introducing AI to students across disciplines, and the institutional AI vision and strategy now in operation at George Mason. She writes on science, technology, culture, and the future of the university at amardashehu.substack.com.

Of all the myriad explanations of how LLMs work, I find discussions of Shannon information and compression most compelling. “Shannon” information refers to, of course, the pioneer of information theory, Claude Shannon. Shannon defined information in terms of surprise, so that more information equates to more surprise at the next item (or token).

Shannon worked at Bell Labs, and his analysis of information, which famously ignored what the information “meant”—in other words, the question of semantics rather than symbols and syntax—quickly became a lynchpin in the growth and success of digital communications technologies and the rise of digital computation.

But his analysis was downstream of questions of meaning, and when we finally arrive at the question of how LLMs can work so well, we need to revisit some of his simplifying assumptions. “I love you” is a text message consisting of symbols—letters and spaces in the English alphabet. If I transmit this, it carries Shannon information, and if you receive it intact, you’ve received the information carried by that syntactical string. But you also know that “I love you” has meaning, which in Shannon’s framework would be extraneous, because it’s “new” information about the interpretation of the string given the receiver and their understanding of English. That’s not in the string itself.

Still, Shannon information, and the idea of compressing information so that it can be reliably encoded and regenerated, is central to questions of AI. In fact, a more technical description of what an LLM like ChatGPT or Claude is doing when you interact with it can be stated in terms of conditional probability and compression.

P(xₜ | x₁, …, xₜ₋₁),

where “|” (read “given”) indicates that the probability of the next token is conditioned on the sequence so far. Conditional probabilities are central in AI and play a major role in related frameworks such as Bayesian analysis.

The overall computation of a language model assigns high probability to sequences that occur in its training data, and represents (projects) these sequences in a high-dimensional space in which similar sequences lie near one another. This structure allows the model, given a prompt, to generate new sequences that follow the same statistical patterns.

In information theory, this is equivalent to what’s called “compression.” A model that can predict the next token well can encode sequences efficiently, because it has captured the statistical structure of the source data. The model doesn’t recapitulate the data directly, but “compresses” it so that it can be accurately queried and new sequences of tokens fitting the prompt will reliably represent the meanings of the terms, represented now by the embeddings in a vector space, where two tokens nearby are more similar, and so on. In other words, rather than storing text directly, the model stores a parameterized approximation of the distribution that generates that text.

Takeaway —> The model does not store language; it stores a compressed map of how language behaves.

In this sense, what we mean by learning in AI is observing an underlying distribution (the data) and constructing a compact representation of it (the model). For LLMs, that distribution is over sequences of words. The model compresses patterns in how language is used—co-occurrence, grammar, and recurring forms of expression—and generation amounts to sampling from that compressed representation, selecting the next token based on its conditional probability given the sequence so far.

The Human Factor

Enter human cognition and inference, where it becomes less clear what is meant. One of the casualties of the last few years’ excitement over what Ethan Mollick has called our new “alien intelligence” has been the blurring of key distinctions about intelligence itself. Judea Pearl is a Turing Award recipient and a longtime AI researcher and pioneer in extending Bayesian reasoning to causal reasoning. In his 2018 book The Book of Why, Pearl distinguished between hypothetical reasoning—what would happen if things were different—and inductive, data-driven reasoning—what the data shows is happening.

Read more

We use five words as if they were interchangeable: data, information, fact, evidence, and knowledge. They are not. They belong to different stages of thought and action. Definitional differences matter today perhaps more than ever, as we hear we’re in an “information age,” our new worldview is “dataism,” and that artificial intelligence is transforming knowledge. Where to start?

Start with an excellent compendium of essays, Information: A Historical Companion (2021), edited by Ann Blair, Paul Duguid, Anja-Silvia, and Anthony Grafton.1 I stumbled into the collection researching Duguid, who co-authored a dated but still good book penned at the dawn of the “new information age,” with John Seely Brown, The Social Life of Information (2000).2 The authors had the foresight to argue that paper and paper books would not be fully displaced by electronic media.

A key point in the volume is that “information” seems a bedrock concept but is actually ambiguous, slippery, and variously used. What’s information? The standard response post-information theory—an academic discipline now—would be Shannon information. Shannon information is a measure of entropy, or uncertainty, which makes sense when one realizes its inventor was Claude Shannon, of—wait for it—Bell Labs. Shannon’s theory of information is extraordinarily fecund and useful in computer science and modern communications. But it’s a drop in the bucket when it comes to understanding information as we use it day to day.

Question:

Is information objective? Subjective? That’s even hard to say.

A revealing study in 2007 by information scientist Chaim Zins uncovered 130 different meanings, produced by “forty-five information scholars from sixteen nations.”3 The whole point of Shannon information is that it’s not subjective—Shannon famously remarked after the publication of his A Mathematical Theory of Communication that semantics were not part of his theory.4 In other words: objective. Yet Gregory Bateson of cybernetics fame claimed information was “a difference that makes a difference,” a more expansive notion that captures more cleanly the idea that information is put to use by minds, by people using it. And the media theorist Marshall McLuhan insisted that “the medium is the message,” a definition that would, at minimum, challenge Shannon’s understanding of information as independent of details about the transmission and reception. To Shannon, that’s outside the theory. McLuhan is wrong for Shannon’s purposes.

The great philosopher of science Peter Lipton has argued that the concept of “cause” is best understood as a request for contrastive information: why did this happen rather than that?5 In a similar vein, unpacking semantics and our ubiquitous “information” might best be approached by asking how information differs from related concepts, like “data,” “fact,” “evidence,” and “knowledge.” How is information different from data?

Etymology helps. Information prefers Latin-based languages, and stems from the Latin construction: in- + formare = to form, shape, fashion, give form to.

“Giving form to” makes explicit the contrast with, for instance, data:

[D]ata is the neuter past participle of the Latin verb dare (to give)—“data” in the early modern period were “givens.”

Data was once the ‘sense “datum”’ empiricists like John Locke recruited to make arguments about the nature of the mind and the conditions of epistemology and knowledge. The data are given to us for use in establishing the truth of an argument. In the modern sense we’ve abandoned the philosophical roots, largely, but retain the idea that data is something given—not necessarily all factual—for use in calculation and computation. Data is what’s given to a large language model, for instance.

The other cognate words are instructive as well. Blair et al. explain:

A “datum” in English is something given in an argument. This is in contrast to a “fact,” which derives from the Latin verb meaning “to make” or “to do,” so that a “fact” is that which was done, occurred, or exists. The etymology of “data” also contrasts with that of “evidence,” from the Latin verb “to see.”

And:

There are important distinctions here: facts are ontological, evidence is epistemological, data—something given in argument—is rhetorical.

I noted in an endnote in The Myth of Artificial Intelligence about the difference between a fact and data.6 I was, in some sense anyway, mistaken. A fact isn’t what’s necessarily recorded so that it can be used in further reasoning or calculation. A fact is that which is presented to us such that we see that it’s true. It’s what’s “done” to the world to make manifest what’s, well, manifest. But, notoriously, data can have errors. It often does. So data isn’t co-extensive with facts—they are in fact different concepts, and we can see this clearly when looking at their etymological roots.

In her study of facts, A History of the Modern Fact: Problems of Knowledge in the Sciences of Wealth and Society, Mary Poovey points out that facts were originally presentations intended to establish veracity.7 The canonical historical example was an accounting of, say, a business’s finances. The numbers were arranged in columns and rows so as to have the form of truth. That presentation told the auditor that the books were in order.

We find this odd today because we’re used to the distinction between presenting spreadsheets of figures and the question of whether they’re fraudulent or mistaken. But a fact originally was to make the truth manifest, and that’s what led to modern accounting—now more directly tied to the calculations. But we can see the evolution of the concept.

This brings us back to data. Data are what’s given for use in establishing something, in an argument. But do all arguments benefit from data? More foundationally, can data be used for all arguments? Of course not. This gives us a clue to the limitations of the modern epistemology of these concepts, and our embrace of what historian Yuval Harari has called “dataism.” Data may be the new oil, but we’re not only thinking in fossil fuels.

Evidence is epistemic; it tells us what we can know. It’s not, like data, given in argument. It’s identified by minds as relevant to a direction of thought. Evidence is closely tied to abductive inference, which is the type of inference that can form hypotheses independent of the likelihoods obtained from prior observations. Abduction gives us clues, rather than data. Clues can be surprising, even as induction gives us the most likely.

Evidence and information are closely tied—much more so than data. Information, recall, originally meant to give form to something, so that it could give direction and instruction. The concept eclipses that which is given, data, precisely because it gives form and isn’t just given. If I want to buy silk in Rome, I need to get information about its availability and price from those traveling the Silk Road. Getting “data” isn’t enough.

Knowledge according to generations of philosophers is almost “justified true belief.” Gettier famously challenged this handy definition by providing contrived but logically sound examples. Goodman’s “grue” belongs to a different but related problem: induction.8 But so-called “JTBs” are I think less helpful for my present attempt, to situate these terms for purposes of clarifying what we mean when we use them today, as we so often do. Knowledge is certainly more than information for roughly the intuitions behind JTBs—we need our information to also be true, which means we should have some justifiable belief that it is true.

Knowledge is explicitly subjective—not in the sense of relative to an observer—because a cognitive agent possesses it. We say that we know that; we’re in possession of knowledge, which implies that there’s some chain of custody, so to speak, that we rely on to identify it as established or known. Information gives us a direction to move. Knowledge stamps it as correct.

Philosophical treatments of knowledge have the virtue of being objective and conceptual, but they have the vice of obfuscating standard social complexities—if X believes Islam and Y believes Christianity, who has the justified true belief? What we mean in practical use is more like:

Knowledge is information made reliable in action by an individual or a community of practice.

Now we might understand our modern, data-driven world as the attempt to use data, facts, evidence, and information to build our stores of knowledge—that which is actionable and reliable because formed in the kiln of experience, scientific and otherwise.

Data systems—artificial intelligence—can help us on this epistemic Odyssey.9 They cannot replace it.

Last week I outlined five misconceptions about AI. Here are five more problems with how we think about it.

If everyone uses it, it stops being an advantage

In Moneyball: The Art of Winning an Unfair Game (2003), writer Michael Lewis describes how baseball teams like the Oakland A’s gained a competitive edge by valuing on-base percentage instead of traditional stats like batting average.1 A blockbuster movie starring Brad Pitt followed in 2011. As Lewis put it, the A’s had found an ace-in-the-hole, a new way of scoring players and games.

What does this have to do with AI today? Actually, a lot.

Subscribe now

Quick detour here, the Moneyball story traces back to an amateur statistician named Bill James, who had started analyzing baseball based on relatively quotidian metrics like On-Base Percentage (OBP) and Slugging Percentage (SLG). James’s method worked because almost no one else was doing it at the time. His method was later coined “sabermetrics” and found its way to the major leagues with, famously, Billy Beane, the Oakland A’s general manager whose low-payroll club became the emblem of the strategy depicted in Lewis’s book.

The larger point eclipsed baseball: if a system systematically undervalues something, then recognizing its value before everyone else can produce outsized returns. Bill James’s statistical work mattered because it identified a blind spot in a competitive environment. Billy Beane’s genius was not that he liked numbers. It was that he operationalized them before the rest of the league caught on.

But the crucial fact about advantages of this sort is that they are often temporary. An edge derived from better information, metrics, or inference lasts only as long as it remains unevenly distributed. Once every front office uses the same data, hires the same analysts, and prices players through the same lens, the market adjusts. What was once an inefficiency becomes standard practice; what was once a strategic advantage becomes table stakes. The method does not stop working in an absolute sense. But it stops conferring asymmetrical advantage. It “saturates” in the market, we might say.

This brings us to AI.

Early evidence from software engineering shows a similar pattern. A controlled study of developers using GitHub Copilot found that participants completed coding tasks about 55% faster than a control group.2 The gains are real. But as adoption spreads, the advantage disappears. More talented programmers, relative to junior coders, remain valuable to organizations, as before. Human talent remains the centerpiece.

Right now, many people speak about AI as though merely using it grants an advantage. In the short run, in some contexts, it does. If you use a language model to summarize documents faster, draft competent boilerplate, accelerate coding tasks, or generate plausible first passes at routine intellectual work, you may indeed outperform someone who refuses to use it at all. But this is the easy part of the story. The harder question is what happens when everyone does the same thing.

The answer is: the advantage disappears.

If every student uses AI to draft essays, then AI-assisted drafting no longer distinguishes one student from another. If every consultant uses it to generate slides and memos, then faster slide production ceases to be differentiating. If every marketer uses it to produce copy variations, then the supply of acceptable copy rises and the marginal value of any one instance falls.

In each case, the technology may increase throughput. But increasing throughput is not the same as creating durable strategic advantage. More often, it simply resets the baseline.

This is what many people miss when they talk about AI as though adoption itself were a moat. It is not a moat if the capability is generic. A true moat requires scarcity, defensibility, or some difficult-to-replicate integration with talent, judgment, proprietary data, institutional process, or domain-specific expertise. Otherwise the tool behaves less like a secret weapon and more like a spreadsheet. Useful, yes. Transformative in certain workflows, yes. But once generalized, it becomes infrastructure.

A generalized technology does not make everyone exceptional. It simply adjusts the level where competition takes place.

That is why the most extravagant claims about AI-driven competitive advantage should be treated with caution. In the early phase of adoption, when some people use the tool well and others not at all, gains can look dramatic. But those gains often reflect what we might call a “diffusion lag,” rather than a deep transformation. They are advantages purchased by being early to a method, not by possessing a fundamentally new kind of intelligence.

“Moneyball” worked because the insight was not yet common knowledge. Once it became common knowledge, baseball did not stop being data-driven. It became more data-driven than ever. But the original edge disappeared into the structure of the game.

The same thing is happening with AI. As the tools spread, their benefits do not vanish, but their distinctiveness does. The organizations and individuals who continue to matter will not be those who merely use AI, but those who can do something with it that others cannot: frame better questions, exercise better judgment, integrate outputs into real expertise, or build systems around it that are not easily copied.

In other words, human talent will still discriminate on the new playing field—as always. This is one reason why talk of massive unemployment from AI adoption is wrongheaded. It is also why the usual patter about a coming AGI is—as always—off the mark.

Takeaway point: when everyone has the same statistical assistant, advantage returns to the human being. As always. How will companies train employees for the new forms of competitive advantage? That’s a human question, and boomers and doomers should listen.

“AI” is no longer a scientific concept. It’s a capital-intensive industry

AI is no longer primarily about ideas, but about resources. Training state-of-the-art AI models requires massive compute, specialized infrastructure, and large-scale data pipelines, which concentrate capability in a very small number of organizations.

The Large Hadron Collider

In this sense, modern AI extends a trend that stretches back decades. It’s been termed “Big Science,” and in fairness this approach has delivered scientific results and on occasion breakthroughs. For instance, the Large Hadron Collider, stretching 27 kilometers on the Swiss-French border, enabled particle physics experiments that confirmed a new particle, the Higgs boson in 2012, something no small lab could do. Large, coordinated efforts like the Human Genome Project mapped the entire human genome in the early 2000s.

But big science also has limits: it’s expensive, centralized, and agenda-driven. The funding requirements alone shift the focus from small innovative groups, like the famed “Rad Lab” that proved so effective in producing bleeding-edge technology in the fog of World War II, to major organizations that were once part of the military-industrial complex and now stem largely from the corporate world dominating Silicon Valley.

The innovative “rad lab”—MIT’s Radiation Laboratory

Big funding and Big Science also inevitably centralize research under hierarchies of management, even as generations of studies from business schools around the globe have shown that smaller groups with more freedom tend to innovate more quickly and effectively.3 And the very nature of invention and innovation means that a top-down agenda can find itself out of step with realities that change on the ground, as research and results change the understanding of the phenomenon investigated. AI? Big Science on steroids; problems included.

Frontier AI increasingly resembles this model, and adds yet another problem: it is not merely large-scale science. AI today is large-scale science fused to venture capital, cloud infrastructure, platform economics, and geopolitical competition. It’s a technology that has been quickly and perhaps carelessly woven into the fabric of society at pressure points both civilian, commercial, academic, bureaucratic, and military. The result looks less like an open scientific inquiry into intelligence, than a race to build ever more expensive systems whose value must be justified in commercial and strategic terms. We can, in theory, “defund” Big Science. We cannot defund AI.

This changes the character of the enterprise, evidenced by contemporary worries and discussions about AI. Media sources ostensibly agnostic about the value of AI, like The Economist, now routinely run pieces focusing not on the science but the logistical and financial aspects of the field.

Depressingly, the questions are “Big Money,” rather than idea-driven too: Who has the compute? Who has the data centers? Who can afford the chips? Who can pay the engineering teams, absorb the training costs, and sustain the burn long enough to remain at the frontier? AI has become a social and cultural bandwagon that we moderns cannot help but join. Yet the levers of power remain in the hands of the few.

One consequence here is that certain research directions become self-reinforcing, not necessarily because they are theoretically deepest, but because they are the ones that can absorb capital and produce visible benchmarks, demos, and products.

Scaling, for instance, is especially attractive in this environment, even as top researchers have begun abandoning it. It is legible to investors, journalists, and internal management: more parameters, more tokens, more compute, better benchmark scores. These are measurable, reportable, and fundable. They fit the logic of the industry.

What becomes harder to support are alternative approaches that do not scale straightforwardly, or that require more conceptual risk than financial magnitude. A small team with a new theory of learning or reasoning may have interesting ideas, but if the field’s center of gravity is moving toward trillion-parameter systems and industrial training runs, then those ideas struggle to compete for attention. Not because they are false, but because they are structurally outmatched by the incentives of the moment.

In an earlier period, one could plausibly speak of artificial intelligence as a scientific aspiration, even if that aspiration was often confused or overstated. The question was: what would it mean to build an intelligent system? Today, at the frontier, the operative question is often more practical and more industrial: how do we train larger and more capable foundation models, deploy them across markets, defend the moat, and capture value before competitors do the same?

Those are business questions before they are scientific ones.

If that is where AI now lives, then we should stop pretending the field is still best understood as a neutral, open-ended search for intelligence in the abstract. At the frontier, it is increasingly a competition among giant institutions to industrialize one particular vision of machine cognition.

Which means the central question is no longer just whether the systems work. It is also who defines what counts as working, what counts as intelligence, and which alternatives never receive the resources to be tried.

AI has always been, and will remain, a military technology

For all the talk about productivity and creativity, one of the primary drivers of AI has always been defense. Or war.

“Computers” were once human accountants—historically women—who calculated ballistics and artillery trajectories for the military by hand using differential equations and lookup tables. They transferred data onto punch cards, recorded results on paper, and later transferred them onto punch cards for tabulation by machines such as those developed by IBM and its predecessors. By the 1940s, a young mathematician named Alan Turing was working on codebreaking at Bletchley Park, where machines like the Colossus were used to help decipher German encrypted communications in the Second World War.4 Norbert Wiener, of cybernetics fame, and other mid 20th century luminaries like Vannevar Bush, Claude Shannon, and Julian Bigelow were seeking automated methods for flying airplanes (autopilot), and shooting them and missiles out of the sky (antiaircraft guns).

By the war’s end, John von Neumann would spur the development of programmable computers with storage (the so-called von Neumann architecture) to help compute the blast radii of nuclear weapons. A decade later, a commercial version of the early ENIAC computer, known as the UNIVAC, would help the Census Bureau process census data. IBM would develop the Univac into business machines, like the IBM 701.

But it all started with the needs of the military and the military-industrial complex. And, as artificial intelligence took root as a field of study in the 1950s and 60s, the military would continue to pour millions into AI to develop Cold War-era systems for fully automated machine translation and early command-and-control and surveillance systems.

I was funded by DARPA—I should know.

Fast forward to today, and the now four-year-long war in Ukraine makes the same point. Cheap, widely available drones, combined with real-time data processing, computer vision, and constantly evolving targeting systems are reshaping the battlefield (Ukrainian-designed drones are also increasingly used in the current Iran conflict). Systems costing thousands of dollars are now disabling or destroying artillery and other bread-and-butter military equipment that costs millions or billions. But “AI” confers an advantage on the battlefield, and the range of its uses will no doubt grow.

What makes this possible is not “intelligence” in any deep sense, but rather data-driven pattern recognition:

• identifying targets from visual feeds

• detecting and tracking movement

• adjusting trajectories in real time

This is artificial intelligence redefined as statistical inference applied at scale. And unlike ongoing discussions about, say, the future of commercial self-driving cars, the evidence emerging from the battlefield confirms again and again that AI and war are a perfect fit.

But the problem, again, is that while AI has found a home on the battlefield, the conditions for success have little to do with intelligence—ostensibly the goal of artificial intelligence—and much to do with reliably mapping sensor input to outputs under uncertainty. In plain terms, AI seems good at killing enemies of the state.

Any workable “AI” will likely find its way into the military. Today’s Big Data/Big Iron AI is form-fit for the battlefield, where it will not only persist but become more central to future conflicts.

Prediction is not explanation

The history of AI is a history of succumbing to technical and conceptual challenges and narrowing the field to what “works.” By the 2010s, what worked was, in essence, black-box prediction from massive data and compute.

This vision of AI would have made little sense to the pioneers of the field, who dreamed of discovering the Rosetta stone of intelligence and programming it on a computer. Far from any such Rosetta stone, we’ve now successfully redefined AI not even as machine learning in general, but as a particular type of machine learning known as neural networks (technically: Artificial Neural Networks, or ANNs). With huge increases in data and compute, neural networks were rebranded this century as “deep neural networks.”

Yet researchers have known for decades that ANNs are poor candidates for true intelligence, since they have a notorious optimization problem in learning—the “local minima” problem associated with gradient descent. In layman’s terms, we can never know if the nets will converge on the correct answer, or one that merely “looks” correct given the flawed convergence of the gradient-descent training. Researchers liken this to stopping at a mountain lake, rather than reaching the summit.

Curiously, other forms of machine learning, like so-called wide margin classifiers, called Support Vector Machines (to take one example) don’t suffer from local minima problems and are mathematically guaranteed, if they converge at all, to converge on the globally optimal solution. No matter; other forms of machine learning, whatever their mathematical properties, were quickly abandoned when big data and big compute proved that the neural networks performed better anyway.5

Let’s hear it—not for theory, but for gobs of data from Flickr, or where have you.

NNs, in other words, undercut one of the principled reasons to use machine learning methods based on mathematics in the first place: guarantees of an optimal solution given some solution space. Though clever tricks like dropout have helped mitigate the bugbear of local minima, there is no general theoretical solution to the problem of non-optimal learning in neural networks.

So why does today’s generative AI work so well on so many problems? The answer is, perhaps ironically, Big Data and Big Compute—or the scaling hypothesis. At sufficient scale, a deep neural network’s loss landscape contains many acceptable solutions, and gradient descent reliably finds one of them. Global optimality is no longer required at scale.

But generative AI is sequential, and it cares only about the next token given a sequence. In some domains, the question of truth versus probability may be less germane. But with language models, those “next tokens” are answering questions, holding conversations, and writing your boss an email. Truth is never guaranteed by next-token probability. In other words, scaling may mitigate optimization failures, but even well-optimized language models ignore truth by design. Witness the now notorious “hallucinations” or “confabulations.”

Add to this that ANNs are perfect black boxes, opaque to human inspection. This has gone so far in recent years, with the advent of LLMs, that even experts who design and train these systems admit they don’t really know why they work.

Epistemological foundation for a new age? Hardly.

Black-box AI is a pale substitute for what we once aimed for: machines grounded in our best, most elegant theories of intelligence and cognition. Instead, the pat answer to questions about future performance is simply to add more data, or “scale.” The so-called “scaling hypothesis” has proven inadequate for squeezing more out of these new, energy-hungry black boxes. Prediction without explanation has been and will continue to be inadequate for a serious computational science. Or for AGI, for that matter.

And in the meantime, we’re no closer to understanding anything substantive about intelligence at all.

We don’t know what intelligence is, but we’re acting as if we do

Would Albert Einstein score meaningfully higher on an IQ test than members of MENSA? If not, it’s not clear what the test measures. Was Pablo Picasso “less intelligent” than J. Robert Oppenheimer? Would an IQ test decide? Unlikely.

There is no settled theory of intelligence. Over decades, cognate fields—cognitive science, neuroscience, machine learning—have uncovered partial accounts, but no unified theory of what intelligence is, how it works, or how it should be measured has emerged.

Psychometrics is the study of intelligence through tests—IQ scores, SATs, GREs, and the like. The measures are useful, but contested. After a century of work, the consensus is modest: the tests capture something about intelligence, but not everything.

The ballyhoo over proving how “smart” AI is has intensified the problem. AI researchers rely on benchmarks: curated collections of problems presented as questions, code prompts, or reasoning tasks, paired with answer keys or clear grading rules. The model is run against this fixed suite, and its outputs are graded according to predefined criteria.

Putative benchmark tests have proliferated of late. Consider MMLU (Massive Multitask Language Understanding), GSM8K (grade-school math word problems), and HumanEval (code generation via unit tests). More recent efforts like HELM attempt to aggregate performance across tasks into a single profile. Generic benchmark tests like François Chollet’s ARC (Abstraction and Reasoning Corpus), intended to measure something akin to what psychometrics calls “fluid” g—the ability to infer rules and solve novel problems from minimal examples—are also popular.

These are not trivial tests. But they remain closed-world evaluations, where the test problems are specified in advance, the scoring rules are fixed, and the space of acceptable answers is known.

This matters because success on a benchmark does not establish a general capacity. It establishes competence under particular conditions. A model that scores highly on MMLU has learned the statistical structure of question-answer pairs drawn from its training distribution. A model that performs well on GSM8K can reproduce solutions for a class of problems it has effectively seen before.

A model that passes HumanEval can generate code that satisfies software unit tests, which themselves define what counts as correctness. Even tests for commonsense or “general” intelligence show the telltale pattern of teaching to the test: scores on Chollet’s ARC have risen sharply as researchers optimize against the test, yet the systems are not acquiring commonsense.

The problem is not that benchmarks are useless. The problem is the inference we draw from them. We move from:

“the system performs well on this task”
to
“the system is intelligent in the general sense”

That step is not justified by the evidence.

In psychometrics, we at least proceed under the assumption—contested but operational—that different tests are imperfect measures of a latent general intelligence, often denoted g. In AI, we lack even that. There is no agreed-upon underlying construct that the benchmarks are measuring. Instead, researchers build systems to score highly on accepted benchmarks, and report those scores as indicative of underlying intelligence.

This is why benchmark gains so often fail to translate into robust real-world competence. When the problem changes—when the task is underspecified, when the data distribution shifts, when the criteria for success are not fixed in advance—performance degrades in familiar ways. The system produces answers that are locally plausible but globally meaningless.

Benchmarks are instruments. They measure what they are designed to measure. But they are not theories, and they do not justify claims about general intelligence. And treating them as such is not progress.

This confusion has been with the field from the beginning. If a machine can play chess, recognize speech, translate text, classify images, or solve a set of reasoning tasks, then those capabilities are taken as local stand-ins for intelligence. The move is understandable, but the danger is that the proxy hardens into the concept itself. We stop saying, “This system performs well on a narrow task we associate with intelligence,” and start saying, “This system is intelligent.”

That slippage is not merely linguistic, but unfortunately has shaped the entire public understanding of the field. Researchers at organizations like Google, Meta, OpenAI, and Anthropic are no doubt aware of this tension. But neither they nor their employers have much incentive to clarify it for a public eager for the next breakthrough.

The problem becomes even clearer once we step outside formal testing environments. Human intelligence, or “fluid g,” involves transfer across domains, learning from sparse and ambiguous evidence, and—crucially—the ability to determine what matters in the first place. It integrates perception, memory, action, and social understanding in ways that do not reduce to fixed tasks or scoring rules. None of these fit neatly into a benchmark suite.

Nor is intelligence exhausted by abstract problem-solving. Human beings are embodied creatures acting in a world. That is why the Einstein-Picasso comparison is so revealing. It is not merely that intelligence comes in different forms, though it plainly does. It is that the word itself sits over a heterogeneous field of capacities that resist reduction to a single numerical scale. Scientific genius, artistic genius, strategic genius, social genius, mechanical genius all overlap, but they are not identical. The attempt to compress them into one latent variable may be useful for certain purposes, but it is already an abstraction from the richness of the phenomenon.

AI’s incessant focus on “intelligence”—it’s in the name—ignores a basic fact: we lack a settled theory of intelligence in the first place. What are we testing? We end up defining machine intelligence by whatever machines currently do well. We talk as if building systems that perform intelligently on selected tasks gives us an account of intelligence itself. That is like mistaking an increasingly accurate map for a theory of geography. The map may be useful, but it does not explain the terrain.

A more sober view would begin from the opposite premise: we do not yet know enough. We do not know whether the current dominant methods in AI are converging on the relevant capacities, simulating some of them, or merely bypassing them with powerful statistical shortcuts. Those are very different possibilities, and a science of AI would have slowed the roll long ago. The obvious inference here is that we are not yet capable of such a science, and are settling for a dangerous simulacrum. One consequence is that governments, educators, politicians, the media, and the general public are getting snowed.

So the deepest irony may be this: at precisely the moment when public discourse is most confident that we are building intelligence, the underlying concept remains unsettled. We do not know what intelligence is, and yet we increasingly organize research agendas, investment flows, institutional priorities, and even civilizational rhetoric as though the matter were resolved.

The honest position is more demanding. Intelligence remains, in crucial respects, an open question. Any field that forgets this risks mistaking progress on proxies for understanding of the thing itself.

Erik J. Larson

All models converge on the center of the distribution. That takes a lot of power, but not as much thought….

1. The grounding problem isn’t solved. We’ve mostly routed around it.

The grounding problem has been a central issue in cognitive science and philosophy for decades. How do symbols get their meaning? How does a system connect language to the world around it? More trenchantly: how does my thought, expressed as the word “cup,” refer to THAT cup sitting there on my kitchen table?

We don’t have a general solution to this class of problem. Turns out, the philosophers do have something to say, since the AI engineers still have no good solution to this problem, which is why our robots and self-driving cars don’t work.

And philosophers have known and discussed the problem for centuries. The key question, and one that bears directly on the failures of modern AI, is this: how does a token come to refer? How does a system bind an internal symbol—not referring to anything—to an external object, a property, or an event in a way that is stable and usable for further inference, and revisable through future interactions of the system with its environment? We don’t know.

What’s changed is not that we’ve solved the grounding problem, but that we’ve stopped treating it as central. We’ve taken data—large, static corpora of text and images—as an adequate answer, ignoring the fact that “data” is also internal to a cognitive system and hardly a good candidate for grounding anything except in a spreadsheet. This is a colossal lacuna in modern AI thinking and research and somewhat inexcusable given the obvious need for such a capability, and a theory explaining it. Welcome to AI research.

Data is a record of prior human activity. It is already interpreted, already structured, already grounded by someone else. A system trained on that data is learning statistical regularities over representations, not learning how those representations connect to the world. Data, in other words, does not solve the grounding problem—it ignores it.

A language model produce language about physical situations—say, that objects fall, that collisions happen, or that liquids pour. But it does not learn what fixes the reference of those terms. It does not know what makes an instance of “falling” an instance of falling, or which features of a situation are causally relevant versus incidental.

It has no mechanism for what we might call reference stabilization—the capacity to fix what a symbol refers to across changing contexts, and to maintain that reference through perceptions, state transformations, and actions.

The upshot is that we ask an “AI” about a slightly novel physical scenario—an object balanced in an unusual way, a container with a nonstandard opening, or a change in support—and we expect performance to degrade. The system isn’t “confused,” but simply has no underlying model useful for making progress on grounding. Spreadsheets don’t know about the world.

That’s why self-driving cars don’t yet “work” either.

Neuroscience could come to the rescue, if only we understood the brain better. Here, researchers face a quagmire too that should make us cautious about easy analogies between brains and current machine learning systems.

Yes, neuroscience has uncovered important regularities in early sensory processing. We know a good deal about retinotopic organization in vision, orientation-selective neurons in primary visual cortex, hierarchical feature extraction along the ventral stream, and population coding in sensory areas. These are real achievements. They also helped inspire early neural architectures, including convolutional models and other hierarchical systems for pattern recognition.

But none of that gives us a theory of grounding.

At most, it gives us partial constraints on implementation. It tells us something about how biological systems process sensory input at low and mid levels—edges, contours, motion, invariances over position and scale. It does not tell us how a system comes to represent this cup as that enduring object there on the table, how it binds a variable to that object across changing viewpoints, or how it updates its representation when it acts on the world and the world pushes back.

In other words, neuroscience may illuminate parts of the pipeline from sensation to representation, but it does not yet explain how reference is fixed, stabilized, and revised through embodied interaction. It gives us clues. It does not solve the problem.

And if anything, the biological comparison cuts against the dominant engineering strategy.

Humans are low-data learners. Infants acquire a basic understanding of objects, persistence, containment, support, and causality through relatively sparse but richly structured interaction with the environment. They are not ingesting terabytes of text. They are embedded in the world, acting in it, failing in it, and updating on the basis of feedback. Their concepts emerge not from passively absorbing records of prior linguistic behavior but from tightly coupled perception-action loops. That matters.

If intelligence in biological systems depends on embodied, intervention-rich learning, then treating ever-larger datasets as a substitute for experience is not merely incomplete. It may be fundamentally misguided. We are taking the residue of human cognition—texts, images, labels, annotations—and mistaking it for cognition itself.

This is the deeper problem with the current paradigm. It assumes that grounding can be deferred, approximated, or eventually washed out by scale. Train on enough data, and perhaps the problem disappears. But there is no good reason to think that. More of what is ungrounded does not become grounded simply by accumulation. Correlation does not turn into reference by getting bigger.

A system that lacks the ability to intervene in the world, to bind symbols to stable objects through action, and to revise those bindings in light of consequences, is not solving the grounding problem. It is operating upstream of it.

That is why these systems can appear uncannily capable and yet fail in ways that remain structurally familiar. They can generate language about the world without possessing a workable relation to the world. They can mimic the surface forms of understanding while lacking the conditions that would make understanding possible.

So the issue is not just that current AI systems lack grounding.

It is that the field has largely reorganized itself around methods that make grounding easy to ignore. Data gives the appearance of contact with reality because it is full of the traces of human contact with reality. But the contact is inherited, not achieved. The machine receives the representation after the fact. It does not earn it through its own encounter with the world.

Until that changes, the problem remains exactly where philosophers and cognitive scientists said it was: at the point where symbols are supposed to become about something.

We still do not know how that happens. And until we do, talk of machine understanding should be treated with far more caution than the field now permits.

2. Correlation versus causation used to be a slogan. Now we’re building it into our most advanced systems.

Pundits and seemingly everyone else today produce lots of loose talk about “reasoning” in modern AI, but when you press on causation, the story collapses.

Turing Award winner Judea Pearl has done serious work here with his directed acyclical graphs (DAGs), but the approach is still constrained by the types of problems that allow a determinate graph to spell out variables and dependencies in advance. That is already a significant limitation.

Read more

The VP of Sexy/Sweaty.

What follows is how Jake remembers it:

We’re in Steve’s BMW driving up 101 towards The City and he’s talking about this club and how he’s DJing and all the girls and everything’s a rager. A rager. When we get to The City Steve doesn’t seem nervous or excited but he damn near ruins our night pulling out in front of a San Francisco cop. The cop rolls down his window next to us.

“What do you think you’re doing?” he asks, all irritated, glaring at us.

Steve transforms into this obsequious “yes sir, no sir” guy, apologizing about being in a hurry and we’re from Palo Alto. He keeps this up long enough for the cop to drive off, shaking his head.

Steve’s destination is a dance club full of people — gay guys and good looking women — with a big elevated DJing platform. Steve’s up to the platform after a few minutes and I’m at the bar getting a Vodka Redbull. The Vodka Redbull is my official fuck-it drink; it means I’m looking for maximum fun. I know this of course when I order it.

Steve waves me over and introduces me to his friend Frank. Frank’s a perma-grinned, nice looking, smaller dude that right away offers to get me high, which I accept. Generally I won’t unless I’m already drinking, because the booze tamps down the harsh paranoiac effects of marijuana. Generally I decline, but not tonight.

We go outside and Frank passes around his pipe, talking about the street and the area like he’s lived around this club for the last forty years. He looks forty. Or maybe fifty tops. Back inside I wander out onto the dance floor, and then back to the bar between songs for another Vodka Redbull. I see my movements in the club now as a sequence: get high with Frank, then go dance, and then get a drink. Then repeat.

An absolutely Girl-quality, gorgeous woman dances by herself on the dance floor with the DJ music blasting out loud and good. She has this flowing dark hair and a brown dress with a bright shirt. It’s beautiful. She’s beautiful. She’s dancing right out in the middle of the dance floor, and she’s got one of these female bodies perfectly formed, with hip and shoulder ratios, and her face glistens in a healthy organic glow and she looks like the most voluptuous and beautiful creature I’ve ever seen (excepting The Girl maybe, but Girl cogitations right now are strictly prohibited. Strictly prohibited).

I’m eyeballing her, really, but I can’t help it. She’s a tall, brown girl—I mean she’s brown and white or you know, mixed—and she’s moving closer now and dancing on the edges of the floor near where I’m standing, looking over at me. She’s very smiley with her arms up dancing and I really like her attitude. Come to think of it she may be on an assortment of club drugs but at any rate she’s having a great time and very positive. I like positive people, you know, going late into the night and happy to be alive. This is a universal and folks will try to steal away the compliments in all this moralistic talk about being on drugs; I say you’re happy and you’re positive and that’s that.

“I looooove this” she says, when I walk over to her. “That’s two of us” I say. “Let’s dance.”

She holds her hands out and I grab them and we’re circling around. The music is bumping and the lights are flickering. People are standing on the floor with drinks and others are dancing. We’re carving out this big circle holding hands and laughing at each other. Cutting this orbit hand in hand, and with her smile and the quickness of it all, I think we’re, you know, good.

More vodka. She’s laughing and now I’m back into that gooey, dopamine situation where everything seems grand. Dancing. Laughing. A kiss on the cheek. I reconnoiter briefly to fuel up on Vodka Redbull, and leaning on the makeshift plywood bar I look out on the floor and she’s laughing. Back on the floor we can’t orbit anymore because I’m holding the drink, but we keep dancing and talking to each other—all we’re saying is how we love it. We looove it.

Fast forward. I’m getting really high in this club with Steve disappeared up in the DJ platform and this new guy Frank who I keep seeking out for weed. And drunk. I’m getting really drunk, too. Blame this on the sequence I’ve started here, sure. With the weed and the Vodka Redbulls everything seems grand and people’s faces are big and grinning. Very twisted, though. Messed up. I’m messed up. Blame this on Steve. Come to think of it blame all this on The Girl. But as Girl cogitations are strictly prohibited tonight, I say blame this on me. Blame it on me.

Beautiful women move history forward, I think more than men, but when a bloke tries to love that and to enjoy a beautiful woman and to call her his Goddess, folks go all “grow up” on you and why don’t you just get someone your own age that’s more of a mother or I suppose now more of a friend. Well that’s all fine, I say, but these folks haven’t danced with the Goddess. It’s no good saying something that we all can see and frankly, she’s to be adored. This is not about sex. Not wholly, I mean. And that’s an honest statement. She’s damn wonderful.

Fast forward again. I’m back from the bar this time around and I decide the Goddess also has an operational and widely recognized title. She’s in charge of everything. She’s so beautiful. She’s the Vice President of Sexy and she also has this title “VP of Sexy/Sweaty.” She drops the sweaty when in San Francisco clubs... I don’t even know if this is true... anyway VP is here, very authoritative.

I keep saying to myself “there’s the VP” and maybe it’s slightly creepy as I’m staring directly at her. She’s an amazing, wondrous image gliding from a kind of focal point at her hips—her belly showing suggestively—and long flowing black hair with... goddamn where is Frank because the sequence here is not to stare at the VP I’m gonna get in trouble. Come to think of it, I’m really, really high so I’ll change or slightly alter the sequence moving forward. Looping, iterating, is very important in software development and it accounts for much of what makes the modern world tick. I’m on solid scientific ground here, I figure. Let’s rehearse all this.

The accepted sequence since entering the club with Steve, Driver-of-Beamer (plus purveyor of fine wines), has been dance, then a Vodka Redbull from the bar. When this is half done there’s a general shout out to “Frank!” followed by a “Let’s get high!”, which is closely followed by an exit to the street where we circle on Frank’s pipe and chit chat to the sweet smell of his weed—we’re in love with her, it’s Mary, as the song...—so the sequence should be changed now because I keep staring at the VP all creepy like, with phantom-high eyes and maybe my admiration is too obvious or in danger of getting misinterpreted. So now: dance—don’t stare at VP too much—a Vodka Redbull from the bar. No more: “Frank!” Then: “Let’s go get high!” Cut that part out for now.

But I’m wrong about the VP and the creep-factor because returning Vodka Redbull in hand she’s still dancing and I swear she’s motioning me toward her. I feel like an Odysseus character, like some Goddess with a weighty title, too, is motioning to me and here I am, very happy. That’s all, just “happy.

It’s admittedly not really Greek heroic tale material; it’s a dopamine tale set in modern San Francisco and there’s nothing to save, or to kill, or to endure, but... shit I forgot the VP is still motioning and why was I thinking about Greek societies, heroic societies, where the actions aren’t based on abstractions like “rights” but on shared notions of responsibility and there’s clear... goddamn it Jake...

The VP has a curious protocol. It’s inspired, no doubt, by her own awareness of her preternatural beauty. First I say that she clearly loves to dance and clearly loves to move in perfect metronome rhythms. She smiles and her eyes glisten and her body makes a or transmits a—it’s like everyone else is dancing around her, like a Maypole at Marymount feel, you know Hawthorne. Nathaniel Hawthorne. All the little supporting-cast dancers stay concealed in the shadows as the VP is orbited around; celebrated. Her protocol is simple, really, and I’ve cracked it.

She only dances with you if you don’t seem too needy about her.

So I dance with Frank. This works, sort of. But Frank is too happy to be dancing with me because he’s gay. And high. Finally the VP summons again, which is a barely perceptible smile towards me, but it gets translated into this silly office speak in my head, like: “Hi Jake, when you get a chance, could you come over for a minute? I’ll need you to dance with me for at least this song—currently playing—and I want to discuss you handling the next one or two songs, as I’ve been impressed by your sequence lately. Thanks.”

Jesus I’m high. I’ll need to nix the Vodka Redbulls from the sequence soon too. VP called me up from the minors and I’m a tad flat footed with the half dozen Vodka Redbulls and maybe seventy five percent of the weed Frank has on him.

Many are called, but few are chosen.

Continue. At some point I just wander out of the club, still in full tilt, to go get something to eat. I figure the Veep sent me off to get fueled up. I walk looking at my GPS on my Blackberry maybe a mile to some all-night diner.

Follows a depressingly standard, near-shameful grub experience, really, close to a dog eating out of a bowl. And I’m happy to pay and wag my tail at the waitress and depart. Outside I text Steve and he replies back a few minutes later as I’m still waiting outside the diner. Thank God, I suppose. He texts me the address to GPS—I had neglected that—and then the password “moonbeam.” So I hail a taxi and I’m off at 3:30am in The City to the rager. Password moonbeam.

When we arrive I think Steve must be full of it; the street is dark and deserted and industrial looking. The only evidence of a rager is an occasional young looking club-scene kid disappearing into this dark three story building. A few people mill about outside.

Gradually I realize I must be at a different entrance; it’s that or I’m a bit disoriented in a way that would almost qualify as concerning. As I approach the building there’s a guy standing outside and people are approaching him before entering so I walk up to him and say “moonbeam.” He nods and I walk in, following the kids in front of me up the stairs, all in this hushed kind of silence, and down a large hall towards some double doors. In spite of my intoxicated fatigue I’m a bit worried I’ve somehow moonbeamed into the wrong rager, if that were possible.

When I open the double doors it’s like that scene in Men in Black, where there’s a million people in the room but it’s all disguised and silent from the outside. The only thing I hear approaching the doors is a dull rumble from the music, very muffled, and when I open the doors there’s a huge stage and massive amplifiers and disco lights everywhere. People are dancing and laughing and standing in groups talking. There’s a balcony with big plush couches and people are up there sitting on the couches drinks in hand, their faces and the tables in front of them illuminated by the lights pulsating from the stage. It’s now I realize they’ve opened up even a larger dopamine space, it’s like two warehouses really and first we were all crammed in one. Now I recognize the first dance floor and turning up my head like a scence in a movie where the character sees the promised land, there’s another in back of it.

Steve’s in here and it’s a rager.

Steve needs to be in here because the afterhours bar I discover is cash only and I don’t have any cash on me. I text him announcing that I’m here and after what seems a very long time he appears in front of me, sharp dressed and grinning.

“No cash”, I say. He scoops out some crumpled twenties and hands them over. He’s been up in the plush couch seats with some girls and he’s leaving soon with them.
“It’s a rager,” he says.

He stands there all Palo Alto with his blond, good looking head, outlining this plan for us to go with these girls but he’s got to cinch things up, he says, jerking his head back toward the couches. I tell him I’m fine here for now and not to worry. Me, I’m coming down from everything and in a hurry to get a drink.

I’m astonished and excited in a visceral way that kills the dull diner omelet thing that VP—Veep now—is still dancing, somehow, a kind of super endurance Goddess in the underground, a real feminine power I think. Really.

Afrojack’s “Take Over Control” bumps grandly, and I feel like the song’s playing just for the Veep. The Veep seems to be fond of glancing in his direction, a small smile playing on her lips. I’m taking another swig of my drink, feeling the Vodka Redbull kick in harder. I’m getting pulled again into this critical night.

I’m not sure if it’s the weed or the booze or just the electricity of the place, but when she motions toward me, it feels like an invitation I can’t refuse. My mind buzzes with dopamine. Odysseus being called by a goddess. Many are called, but few are chosen.

I’m orbiting free of the diner’s fare now, and feel the real sustenance of the copious flowing vodka Redbull in a plastic cup. Things are looking up. I move toward her on the dance floor, a lone meteor plunging unstoppably toward the Sun. She sees me coming and immediately throws her arms up in the air and starts dancing in a groove that turns her in a full circle. This means that halfway into her revolution I’m looking at her ass. And I am. The Veep thinks this is the right response and seems to get even more into her sensual groove. She backs up until she touches me and I have no choice but to place my hands on her hips as we are now so close that the omission would be, well, rude. By all accounts, I think, Veep is digging it.

She turns around and for a second I think she intends to kiss me, but instead she leans in and says in big groovy tones that she loves it. It’s too loud for real conversation so I lean in and say the first thing I can think of, which is at this point a fait accompli, “I love you!” At this she puts her hands palm first flat on my chest and leans in to ask me to stay for another song. I nod and am pleased when Steve drops the retro classic “Groove is in the Heart,” by Dee-lite.

It is. Groove is in the heart, I think.

She turns her back to me again, relaxing her body into the rhythm, and I move with her. My hands stay at my sides this time, a plausibly prudent move to fend off a creep vibe, but I know it’s probably unnecessary. Anyway, the Veep is calling the shots.

Her hair brushes against my arm, and not being able to help myself I lean into her ear and say I want to smell her arm pit. It seems like there could be some groove here. In the arm pit. Veep loves this idea (yes—promotion coming!) and the scent of her deodorant and sweat transmute me into a Veep dimension I can only describe as “she’s the only living thing that matters.”

I’m not a great dancer, but my current assignment from the Veep isn’t to win a Dancing with the Stars title but to contribute meaningfully to her groove. This I am doing.

Halfway through “Feel so Close,” the lubricious tattoo on her hip—a winding pattern of vines and thorns around some kind of mermaid or goddess—catches my eye again as she dances. It’s mesmerizing, like it has its own mind or has been empowered by the Veep to give instructions to me.

She has a smaller tattoo on her belly, and I keep trying to catch a glimpse of it without freaking her out—though in truth, it seems unlikely that anything would freak out the Veep at that point in the night. Veep is no nonsense and good to her employees, as she pulls her miniskirt down to show me. It is the word “wild” wrapped in a rose. Oh. My. God, please don’t let me let the Veep down, I think.

An hour later and I’ve drifted off to the safety of the bar and I’m hitting my limit; I can’t drink much more and there’s this little tired spot somewhere inside me that’s gradually growing and will by degrees crowd out all the dopamine girl-booze feelings. It’s futile to fight it and I don’t want to let the night fade away into dullness, so I’m thinking about telling her I have to go.

Plus Olivia’s been texting me — Olivia from The Camp — sending now four messages to me through the night, asking me if I’m okay. Am I okay. Am I okay. I respond to her once back at the club saying “I’m fine, in SF” and then there’s a bunch of “huh?!!” and “watcha’ doin’?”’ like she’s somehow both worried about me and jealous I’m having a good time. Anyway I can’t, I don’t know, report back all night so I’ve ignored the other ones.

Steve’s incommunicado. Go figure. I text him to see if he’s still around and there’s no response. Veep is somehow defying physics and is back on the floor after a respite over on the lounge chairs. I wave and then come over to her. She’s proud of me I think, a good employee, and I decide I’d better go back to Palo Alto and how charming she is and how much I appreciate her.

“Come visit me in The City!” she’s saying. She’s all excited, like we’ll make plans to rendezvous at a rager again, picking it right up like this—at 6 am, another all night rager.

“I will, I will. I’d love to,” I say.

We hug. She leans in and kisses me on the cheek and then because she lingers and smells like perfume and sex and sweat I move my scruffy cheek across hers and we kiss on the lips. I’m hugging her and she’s firm and young and she smells good. She’s warm.

She leans in, yelling over the bump of the bass, “What’s your name?”

“Jake,” I yell back. As the song wraps up, she leans in close, her breath warm against my ear. “Mia!” she says, and then gets so close to my ear that she can whisper, “Come find me tomorrow.”

I nod, heart pounding. “Mia, how do I find you?” She grabs my hand and leads me off the floor to one of the makeshift bars, where she smiles at the bartender and asks for a pen. Fortuitously, he has one. While I look on in amusement—who doesn’t have a phone?—Mia writes her number down on the back of a receipt she plucks out of her pocket. When she pulls the receipt out of her tight front pocket, her slim phone pokes up. She has a phone. The Veep does what she does, I conclude.

God I’m lonely. I have to go.

To Be Continued…

Erik J. Larson