The dynamical complexity of brain circuits is daunting. Brain disorders with specific failure modes may give us insights into the normal function of specific brain circuits. For example, loss of dopamine-producing neurons in the midbrain causes Parkinson’s disease: Tremors, muscle rigidity, slow movements, and impaired balance. Thought disorders are even more enigmatic than motor disorders.

Schizophrenia is a devastating thought disorder for which we do not have effective treatments. Symptoms include: Delusions: hearing voices that do not exist, Disorganized thinking: Jumbled speech, switching topics mid-sentence, or saying things that make no sense; Flattened affect: Showing little or no emotion in the face or voice tone; Lack of drive: Trouble starting or finishing planned tasks and daily goals; Social withdrawal: Pulling away from friends, family, and group activities; Loss of pleasure: Finding no joy in life or everyday hobbies. Memory dysfunction: Difficulty using recent information or recalling details.

If we could trace the origin of these symptoms to dysfunction in a specific type of neuron or neural circuit, similar to that of dopamine neurons in Parkinson’s disease, it might be possible to design more effective treatments and unravel some of the deepest mysteries of thinking and cognitive processing. We need to find a chief suspect.

Similar symptoms appear after serial use of a party drug, Special K. Special K is ketamine, an anesthetic at high doses. At low doses, ketamine induces hallucinations, dissociative out-of-body experiences, and reduced gamma bursts, which are high-frequency bursts of spikes lasting less than a second. After two days of raving under the influence of Special K, ravers present at emergency rooms with symptoms indistinguishable from psychosis in schizophrenia patients, who also have reduced gamma bursts. Fortunately, the symptoms in the ravers go away a week later.

My lab at the Salk Institute worked on a mouse model for ketamine-induced psychosis (Behrens and Sejnowski, 2009). We discovered that ketamine downregulates GAD67, the enzyme that synthesizes the inhibitory neurotransmitter GABA in parvalbumin (PV) basket cells (Fig. 1), and also reduces gamma bursts in mice. PV basket cells make multiple “basket-like” synapses that cover the somas of neighboring pyramidal cells. Corroborating evidence comes from postmortem studies on the brains of schizophrenia patients, which revealed that their cortical PV basket cells also had lower levels of GAD67, the enzyme that synthesizes GABA, consistent with what we found in mice (Fig. 2) (Curley and Lewis, 2012). But in schizophrenia patients, the downregulation is permanent.

One of the consequences of reduced inhibition is that cortical circuits become hyperactive, which could explain hallucinations. Ketamine is also effective at relieving clinical depression, perhaps by boosting reduced cortical activity to normal levels.

Figure 1. Mouse model of ketamine-induced psychosis. A single injection of ketamine did not change the levels of parvalbumin (PV) or GAD67 in rodent cortical PV basket cells. There was a significant decrease in both proteins following a second dose on the next day, with a gradual recovery mirroring that in humans. (Courtesy of M. Behrens)

Figure 2. Postmortem schizophrenia brain (Scz) measurements of GAD67, the enzyme that synthesizes GABA, in the synaptic terminals of parvalbumin-positive basket cells (PV_b). (Adapted from Curley and Lewis, 2012)

PV basket cells are the most abundant inhibitory interneurons in the neocortex. They are special because they are “fast-spiking”—they have thin spikes and can fire action potentials in bursts up to 100 Hz. Because of their bursty, high-frequency firing, basket cells are metabolically demanding and highly vulnerable to mitochondrial stress, oxidative DNA damage, and activity-induced double-strand breaks. Unlike most other inhibitory neurons, PV basket cell axons are wrapped in myelin, allowing for ultra-fast signal conduction and strict synchronization across cortical networks.

Figure 3. Schematic of the cortical circuit that is responsible for gamma bursts (Jadi and Sejnowski, 2014). The Center pathway carries direct sensory signals to excitatory pyramidal neurons (E) and inhibitory PV basket cells (I). The Surround pathway from neighboring regions of the cortex modulates the responses to sensory inputs. Each synapse can be either excitatory or inhibitory, as indicated by the shape of the contact with the postsynaptic neuron.

You are more likely to remember something the next day if you pay attention to it. Gamma bursts are generated by reciprocal connections between PV basket cells and pyramidal neurons via a ping-pong mechanism (Fig. 3) (Cardin et al., 2009; Jadi, Behrens, Sejnowski, 2016). By creating strong inhibitory inputs near the spike-initiating zone in pyramidal neurons, they exert precise, millisecond-level control over gamma bursts, which control attention and working memory. Attention changes the state of the local circuit, and gamma bursts imprint a trace of the attended stimulus into the circuit.

Schizophrenia may be a consequence of the reduced inhibition in cortical PV basket cells, just as the loss of dopamine cells causes Parkinson’s disease. Drugs to enhance dopamine activity and deep brain stimulation have ameliorated the symptoms of Parkinson’s disease. Further research led to a theory for how dopamine cells modulate motivation and broadcast reward prediction errors for reward learning (Montague, Dayan and Sejnowski, 1996). Drug treatments and interventions to boost the inhibitory strength of basket cells are underway that could ameliorate thought disorders in schizophrenia patients. Following up on this lead could also reveal a theory for normal cognitive functions (Sejnowski, 2026).

PV basket cells are part of a powerful cortical circuit that has been hiding from us in plain sight. They are the chief suspect for what goes wrong with schizophrenia.

References

Behrens, M. M. and Sejnowski, T. J. (2009). Does schizophrenia arise from oxidative dysregulation of parvalbumin-interneurons in the developing cortex? Neuropharmacology, 57(3), 193-200.

Cardin, J. A., Carlén, M., Meletis, K., Knoblich, U., Zhang, F., Deisseroth, K., Tsai L.H., and Moore, C. I. (2009). Driving fast-spiking cells induces gamma rhythm and controls sensory responses. Nature, 459(7247), 663-667.

Curley, A. A., and Lewis, D. A. (2012). Cortical basket cell dysfunction in schizophrenia. The Journal of Physiology, 590(4), 715-724.

Jadi, M. P., Behrens, M. M., and Sejnowski, T. J. (2016). Abnormal gamma oscillations in N-methyl-D-aspartate receptor hypofunction models of schizophrenia. Biological Psychiatry, 79(9), 716-726.

Jadi, M. P. and Sejnowski, T. J. (2014). Cortical Oscillations Arise from Contextual Interactions that Regulate Sparse Coding, Proceedings of the National Academy of Sciences, USA, 111, 67806785.

Montague, P. R., Dayan, P., and Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 19361947.

Sejnowski, T. J. (2026). Dynamical mechanisms for coordinating long-term working memory based on the precision of spike-timing in cortical neurons. Biological cybernetics, 120(3), 16.

Cockpit view from the Millennium Falcon in Star Wars at the moment the hyperdrive jumped to lightspeed.

I recently gave a talk at the Coronado Public Library. Coronado is a small island off the coast of San Diego, with a population of 25,000. It is connected by a long bridge, making it feel like a village. The Hotel del Coronado is a famous landmark, featured in the movie “Some Like it Hot.” I have given many public lectures. On a book tour in China in 2018, I gave a lecture to a fintech group of 2,000, and it was streamed to 10 million Chinese viewers. Back then, AI was so new that most of the questions were informational, like “What will be the next big application of AI?” This group was much smaller, filling a room with 100 mostly Coronado residents. The questions were less informational. There was an undercurrent of concern. One mother said her two children were anti-AI and did not want to use it. She asked me what she could say to them. I was at a loss for words. I told her that she was asking the wrong person since I don’t have children.

Who Will Be Most Affected?

Newspapers are filled with stories about AI taking away jobs from white-collar workers, which might account for the dramatic shift in public sentiment. When asked this question in the past, I said you would not lose your job, but your job would change, and you would need new skills. I thought the older workforce might have trouble adapting to new technology. I thought that younger workers would have less difficulty adapting to AI, as happened with the introduction of computers and the internet. What I did not anticipate was that younger workers would be the ones most affected.

At the dinner after my talk hosted by my colleague Ralph Greenspan and his wife Dani, a smaller group was more upbeat, sharing stories of using AI in their professions and daily lives. One of them had a better version of my advice on losing a job: “You won’t lose your job to an AI, but to someone who is using AI.” I was sitting next to a cardiologist who said that AI was being widely adopted by his colleagues and it was improving the medical care for their patients. None of the doctors had lost their jobs. All technologies have risks and benefits. It appears that the older generation is more responsive to the benefits. In comparison, the younger generation is more attuned to the risks. Transitions to a new technology are unpredictable. In previous technology shifts, the younger generation had the advantage, but with AI, the older generation of experts has the advantage.

Figure 1. Dario Amodei, chief executive of Anthropic, at the Code With Claude developer conference in San Francisco in 2025

Who Will Benefit?

When Claude Code was released, the programmers in my lab adopted it wholeheartedly, enabling them to debug a large and complex program with a Claude assistant in an hour that would have taken them a week on their own (Fig. 1). The same shift has occurred among other proficient programmers with a deep understanding of large-scale programming environments. Novice programmers have not benefited as much because they lack sufficient experience to advise the AI assistants. Computer Science has traditionally been one of the most popular majors on campus, and a CS bachelor’s degree was an entry into well-paying jobs at top companies. This year, those jobs have become much scarcer.

The Rise of Data Science

Five years ago, the Halıcıoğlu Data Science Institute (HDSI) was founded at UC San Diego. I am a Fellow at the HDSI, and I have seen it grow from a few founding faculty to 50. This unprecedented growth was accomplished by social engineering. Every department has data to analyze, and HDSI offered half-faculty slots for joint recruitments. In addition to starting PhD and Master’s programs, a new highly popular data science undergraduate major was established. Their newly minted majors are in great demand, as companies of all sizes and shapes desperately need help leveraging AI tools. So the problem for youth may not be their age, but their training. It may take a generation for colleges and students to shift to an AI-oriented curriculum.

Figure 2. Taner Halıcıoğlu was employee #1 at Facebook, just after the founders. As a software and operations engineer, he single-handedly built and architected the initial core hardware infrastructure that allowed Facebook to survive its explosive early growth.

In April, UC San Diego announced a new Halıcıoğlu School of Data Science and Computing, alongside the Schools of Engineering, Medicine, Biological Sciences, Physical Sciences, Social Sciences, and Arts and Humanities. This rise in the world of academia is nothing short of meteoric, fueled by $125 million in gifts from alumnus Taner Halıcıoğlu, a Lecturer in Computer Science and Engineering at UC San Diego (Fig. 2). Equally important was the leadership of Rajesh Gupta, who steered the Institute through many narrow academic passages. It also helped that UC San Diego has an entrepreneurial culture.

The Times They Are a-Changin’

Not only has the world changed in the last four years since ChatGPT arrived on the scene, but the rate of change has changed, too. It reminds me of an iconic moment in Star Wars when the jump to lightspeed was engaged. It felt so real, I grabbed my seat.

Humpty Dumpty sat on a wall, Humpty Dumpty had a great fall. Can all our computers and all our algorithms put Humpty’s brain parts back together again?

Reading and language, those most human of cognitive functions, have long been the domain of cognitive science. But new technologies for high-dimensional recording directly from the human brain are making it possible to study the neural coding of language and other human-specific cognitive functions. Making sense of the complex activity tied to these processes requires new theoretical approaches—and insights from artificial intelligence—a new field called NeuroAI (Sejnowski, 2025).

Thinking about reading

As you read this article, your eyes make fast, saccadic movements across the page, taking in small groups of words in your fovea three times per second. Each saccade is a snapshot that must be integrated with all the previous words, building up a conceptual understanding of what is being conveyed. After reading this article, your brain will think about it in the context of experiences and thoughts previously stored in long-term memory. Thinking is generative. Thinking underlies planning future actions. Thinking is fleeting, constantly coming and going.

The time scale for these cognitive functions is minutes to hours, much longer than well-studied sensorimotor actions lasting seconds. Researchers have developed conceptual frameworks for interpreting neural activity for fast automatized actions, correlating firing rates of sensory and motor neurons with sensory perception and behavior. This approach doesn’t work when analyzing neural population activity while thinking on much longer time scales that are not directly related to behavior. As a field, we know much less about the fundamental neural mechanisms that underlie thinking, planning, and reasoning. We need a new conceptual framework for how globally distributed brain states are formed and maintained for hours.

Linking words read across many sentences or words heard during a long lecture requires temporal context (Muller et al. 2024) to relate a new word to previous words. How do brains encode temporal context over hours? Long-term working memory (Ericsson and Kintsch, 1995), which supports longer-term cognitive functions, likely plays a role. Long-term working memory receives and maintains sensory inputs, using them for cognitive processing over hours, intermediate between short-term and long-term memory (Fig. 1). How does long-term working memory implement temporal context? How is long-term working memory used to generate cognitive behaviors such as thinking and planning? How are fleeting thoughts and temporary plans converted to overt behaviors?

Hints from ChatGPT

Artificial intelligence may offer insight into temporal context. Like humans, large language models track information about the sequence of words across many sentences and paragraphs and use temporal context to link these words semantically. I argue here that generative transformers, a key architectural feature of many large language models, demonstrate how neural networks can create temporal context and that a similar process is at work in biological brains. I will make the case that temporal context, implemented by transformers in a spatially static way, is implemented dynamically in brains.

Transformers are the engines under the hood of ChatGPT. They are feedforward neural networks that comprise an encoder that receives queries and a decoder that outputs words in English, one word at a time. Each output word is looped back to the input of the decoder. As each word is produced on the output layer, it is added to the long input vector, providing a comprehensive temporal context for predicting the next word. Transformers, therefore, convert temporal sequences into a spatial sequence that gives the feedforward network access to all the words at the same time. Traveling waves of sparse cortical activity offer a candidate dynamical mechanism for temporal context in brains. A traveling wave recodes input sequences into spatial patterns and extends working memory across the cortex.

What is a cortical traveling wave?

Traveling waves were first observed 100 years ago and are especially prominent in the cerebral cortex; however, we know little about what they do. Scientists can visualize traveling waves with voltage-sensitive dyes (see Fig. 2). Sensory inputs can trigger traveling waves. Cortical traveling waves are sparse, with only a few percent of the cortical neurons activated by a passing wave. Unlike simulations of dense traveling waves in small network models, where all the neurons spike as the wave goes by, sparse traveling waves require large recurrent neural network models with 100,000 spiking neurons with the same connectivity as the cortex.

Traveling wave electrical activity in recurrent networks can extend working memory with rehearsal up to a minute. However, some other mechanism is needed to extend time scales from a minute to hours. A promising candidate is spike-timing-dependent plasticity (STDP), a form of plasticity that requires repetitive, precisely-timed pairing of pre- and postsynaptic spikes within 10 milliseconds of each other at frequencies above 10 Hz. In cortical slice experiments, spike pairings repeated 50 or more times induce enduring changes in synaptic strengths. With fewer pairings, however, the change in strength fades away over minutes and hours, depending on the number and frequency of pairings. STDP is generally considered a synaptic mechanism for forming long-term memories. Instead, I propose that STDP mainly supports temporary working memories rather than long-term memory.

STDP is a rapid, but temporary, weight change.

STDP can rapidly change the strength of cortical synapses, the vast majority of which are small and labile. These changes could support a temporary working memory complementary to the fewer but larger, more stable synapses that store long-term memories and are used for fast sensorimotor processing. Although synaptic plasticity in these small synapses may be temporary, their capacity is immense. The induction phase of synaptic plasticity triggered by waves traveling across the cortex is like a palimpsest that can be rewritten many times and could support a Global Workspace, a leading model for conscious awareness that posits that information is shared across cortical regions. Traveling waves could accomplish this by rapidly assembling a second tier of global connectivity on top of the first tier, which supports more automatized sensorimotor behaviors. These two tiers are complementary, and their relationship is somewhat analogous to classical mechanics and quantum mechanics, where a collapse of a quantum wave packet corresponds to a decision to recruit the motor system.

I suggest that together, these two aspects of cortical physiology—traveling waves, which are ubiquitous but lack a well-established function, and STDP, which is well-established but thought to be the basis of long-term memory—are responsible for long-term working memory and underlie some aspects of thinking and cognitive processing. These two neural mechanisms can interact because the frequencies of cortical traveling waves match the frequencies of pairing required for STDP. In this new conceptual framework, STDP is induced temporarily by precisely timed wavefronts. Spontaneous traveling waves are also ubiquitous and could be used to recall long-term memories and rehearse fading working memories (Fig. 1).

The grid of large synapses supporting long-term memories for tier 1 provides high-speed highways for cortical activity in tier 2 called mathematical manifolds. The vehicles on the road are not single passengers, but trucks that carry bundles of associated items called a schema – a structured plan, such as what to expect when you visit a restaurant (Rumelhart, 1980).

These are a few key pieces of the thinking puzzle. The Humpty Dumpty challenge is to put all the pieces together again, including pieces still left out. For example, we still need to incorporate the roles of inhibitory neurons and neuromodulators.

Transformers in ChatGPT have given us insights into how temporal context and long-term working memory might be implemented in brains, offering one example of NeuroAI’s new conceptual framework for understanding cortical function.

Can ChatGPT think?

ChatGPT can answer a vast range of factual questions, not always accurately, but enough correctly to be useful. An area where ChatGPT stumbles is simple arithmetic, where humans also have the same difficulties. With training, we get better, and so does ChatGPT when given special fine-tuning to go through the same thought process that children are taught, called the chain of reasoning: Break the problem down into subparts that are easier to solve and try out different approaches to each part.

Mimicking mathematical reasoning has significantly improved ChatGPT's math performance. However, the results are not as good when asked to solve different kinds of math problems, especially ones that human have not yet solved. What is missing is humans' flexibility and agility to use analogies and self-generate fresh workarounds. Can we get a hint from human brains to help ChatGPT think more like us?

What happens at the end of a dialog with ChatGPT? A human would continue thinking about a problem, but ChatGPT goes silent. There is no self-generated activity in ChatGPT. We have an inner monologue with ourselves that is absent from ChatGPT. Thinking about how transformers achieve temporal context has given us a fresh perspective on how human brains can self-generate thought. The next generation of ChatGPT might benefit from having a self-generative, long-term working memory similar to ours.

References

Atkinson, R.C. and Shiffrin, R.M. Human Memory: A Proposed System and its Control Processes, Atkinson, R. C. (1968). A proposed system and its control processes. The Psychology of Learning and Motivation, 2, 89-195

Baars, B. J. (1988). A Cognitive Theory of Consciousness. Cambridge University Press.

Davis, Z. W., Benigno, G. B, Fletterman, C., Desbordes, T., Steward, C., Sejnowski, T. J., Reynolds, J. H., Muller, L. Spontaneous traveling waves naturally emerge from horizontal fiber time delays and travel through locally asynchronous-irregular states, Nature Communications, 12, 605 (2021).

Ericsson, K. A., Kintsch, W. (1995). Long-term working memory. Psychological Review, 102(2), 211.

Feldman, D. E. (2012). The spike-timing dependence of plasticity. Neuron, 75(4), 556-571.

Haziza, S., Chrapkiewicz, R., Zhang, Y., Kruzhilin, V., Li, J., Li, J., Delamare, G., Swanson, R., Buzsáki, G., Kannan, M. and Vasan, G., (2024). Imaging high-frequency voltage dynamics in multiple neuron classes of behaving mammals. bioRxiv 10.1101/2024.08.15.607428

Loewenstein, Yonatan, Annerose Kuras, and Simon Rumpel. (2011) Multiplicative dynamics underlie the emergence of the log-normal distribution of spine sizes in the neocortex in vivo.” J. Neuro. 31.26: 9481-9488.

Muller L., Chavane F., Reynolds J., Sejnowski T. J. Cortical travelling waves: mechanisms and computational principles. (2018) Nature Reviews Neuroscience 19(5): 255-268.

Muller, L., Churchland, P. S., Sejnowski, T. J. (2024). Transformers and cortical waves: encoders for pulling in context across time. Trends in Neurosciences. 47 (10):788-802.

Rumelhart, D. E. (2017). Schemata: The building blocks of cognition. In Theoretical issues in reading comprehension. Spiro, R. J., Bruce, B. C., Brewer, W. E., Eds., Lawrence Erlbaum Associates, (Hillsdale, New Jersey). (pp. 33-58)

Sejnowski, T. J. (2024). ChatGPT and the Future of AI., MIT Press. For regular updates to this book see my Substack blog on “Brains and AI.”

Sejnowski, T. J. (2025). Thinking about thinnking, The Transmitter. https://www.thetransmitter.org/human-neurotechnology/thinking-about-thinking-ai-offers-theoretical-insights-into-human-memory/

Zador, A., Escola, S., Richards, B., Ölveczky, B., Bengio, Y., Boahen, K., Botvinick, M., Chklovskii, D., Churchland, A., Clopath, C., DiCarlo, J., Ganguli, S., Hawkins, J., Körding, K., Alexei Koulakov, K., Yann LeCun, Y., Lillicrap, T., Marblestone, A., Olshausen, B., Pouget, A., Savin, C., Sejnowski, T. Simoncelli, E., Solla, S., Sussillo, S., Tolias A. S., Tsao, D., Catalyzing next-generation Artificial Intelligence through NeuroAI. Nat Commun 14, 1597 (2023).

During the California Gold Rush, the population of San Francisco increased from about 1,000 in 1848 to 25,000 full-time residents by 1850. Miners lived in tents and wood shanties. When the rush began, the equipment needed for digging up and extracting gold was in short supply. Stores sold this equipment at a steep markup and made enormous profits. California’s first millionaires were not the prospectors but the equipment suppliers, hence the advice, “During a gold rush, sell shovels.”

The stock market evaluation of the high-tech companies that developed large language models (LLMs) rose 30 percent between November 22, 2022, when ChatGPT went public, and November 22, 2023. The stock of Nvidia, which makes the GPUs used to train LLMs, had increased 400 percent. Its market capitalization increased from $273 billion to over two trillion dollars, today reaching 3.5 trillion dollars. Their high-end H100 GPU is backordered, and their new B200 GPU has 30 times more performance. To update the advice: “During the AI rush, sell GPUs.” Several high-tech companies are even building special-purpose AI chips, such as the Tensor Processing Unit (TPU) designed by Google.

Data Centers are Burning

High-tech companies have many data centers worldwide, large warehouses filled with rack after rack of servers. Most of these are traditional CPUs, but the mix of GPUs and special-purpose AI hardware is fast expanding. New purpose-built AI data centers are beginning to sprout up to meet the demand. CoreWeave has seven large AI data centers online and expects to double that by 2024. A 200,000-square-foot AI data center can cost over $1 billion (Figure 1). Estimates for AI infrastructure, including data centers, networks, and other hardware, “is expected to reach $422.55 billion by 2029, growing at a compound annual rate of 44% over the next six years.” [1] Ten years ago, a data center would draw ten megawatts, but 100 megawatts is the norm today, and the ten largest in North America now average 620 megawatts. Today, AI consumes 2 percent of all the power in global data centers and is expected to reach 10 percent later this year. [2] Microsoft plans to take a nuclear power plant on the three-mile island out of mothballs to power an even larger data center. On January 22, 2025, OpenAI, Oracle, and SoftBank announced an ambitious $500 billion project to build an advanced data center over 10 years.

Figure 1. A large data center can support a million servers in 1 million square feet, requiring 850 MW of power.

High Stakes Games

In the early days of LLMs, academic researchers could build small language models (by today’s standards). LLMs have become so large that only a few major high-tech companies have computing power, enough data, and deep pockets to train them. Researchers at Google introduced transformers, but OpenAI, powered by a billion-dollar investment and cloud support from Microsoft, was the first to go public successfully with ChatGPT. By the time ChatGPT was released to the public, OpenAI had created a for-profit corporation inside their nonprofit and was not releasing technical details, effectively becoming closed AI. [3] Microsoft doubled, redoubled, and re-redoubled down with an additional infusion of $10 billion. It used OpenAI’s GPT-4 to power Bing, its forlorn search engine that suddenly became formidable.

Meta had developed its own LLM called Llama and made it available to a few academic labs, which gave Meta researchers valuable feedback from these beta test sites before rolling out their product. However, Llama’s model and source code was leaked and spread far and wide, making it possible for many groups worldwide to fine-tune the pretrained model and produce inventive new ways to adapt the code for many new uses. OpenAI may have made the genie in the bottle available to the world, but Meta let the genie out of the bottle. Meta eventually made its next-generation Llama 2 open-source, just as Yann LeCun, the Meta AI lab’s chief scientist, told us it would happen. All eight authors of the seminal 2017 paper on transformers have left Google and founded startups, raising billions of dollars in capital. The genie is out of the bottle and is now out of control. [4] Hugging Face is a startup that develops tools for building applications and allows users to share machine learning models and datasets.

The Cost of Chatting

ChatGPT limits the number of requests you can make to GPT-4. This suggests that the data centers’ capacity for processing AI is becoming saturated. More energy is required to run the servers as more users sign up. How much energy does it take to run LLMs? [5] It took months to train GPT-4, running on tens of thousands of GPUs at a cost of $100 million. Estimates for the one-time energy consumption for training GPT-4 are around 5,000 megawatt hours (MWh). To understand what these numbers mean, it takes around 100 megawatts of power to run the New York City subway or around 2,500 MWh daily. The actual cost is not the one-time training cost but the usage cost for processing customer requests, which is 100 MWh daily. Therefore, it takes around $1 million per day for GPT-4 to answer requests. This adds up quickly to 36,500 MWh and $365 million per year and is increasing rapidly.

The cost of computing has halved every two years since the dawn of digital computing in the 1950s and is a billion times cheaper today. This empirical observation, called Moore’s law, ended a few years ago as the sizes of transistors and wires reached the smallest physical limits. However, because chips are getting larger, each chip can do more computing. The latest computer chips have 100 billion transistors on a single chip. With this many transistors, it has become possible to put many cores—complete CPUs—on a single chip. Laptop computers typically have four to eight cores, and GPUs have thousands of cores. The CPU chips in your laptop are about the size of a postage stamp. Cerebras, a company that designs special-purpose computers for AI, has built a chip with 2.6 trillion transistors that is the size of a dinner plate (Figure 2). [6] Their CS-2 wafer-scale chip packs 850,000 cores with superfast on-board memory. The chip is 1,000 times more powerful than a GPU and consumes 15 kilowatts. This super chip can handle a neural network model with 120 trillion weights. On July 2023, G42, an AI company based in Abu Dhabi, purchased a $100 million Condor Galaxy 1 AI supercomputer with 64 Cerebras CS-2 systems containing 54 million cores and 82 terabytes of memory that runs at 4,000 petaflops (10^15 floating point operations per second). The top supercomputer in the world as of this date is Frontier at the Oak Ridge National Laboratory, rated at 1,200 petaflops.

Figure 2. Cerebrus supercomputer on a chip. (Top) A single super chip custom-designed for AI compared with the largest GPU chip. (Bottom) A single wafer-scale chip consumes 15 kilowatts of power and must be water-cooled, which takes up most of the enclosure.

Cerebras has claimed top speeds for AI applications, but over thirty-five hardware companies building AI chips are catching up. Multicore chips can efficiently implement the massively parallel architectures of deep learning networks. With enough cores, the time it takes to process input is independent of the network size. AI can take full advantage of parallel hardware, which is a win-win situation. As models get larger, the hardware gets cheaper, and performance improves. However, the show-stopper is not the computing speed but the required energy.

A Lesson from the Dot-com Bubble

This is not the first time that investments in technology have overreached demand. In the early days of the internet, the rate-limiting bottleneck was the bandwidth of communication to support the increasing demand. The data rate for transmitting video is orders of magnitude greater than text. Fueled by investor enthusiasm, companies laid vast networks of optical fibers, often based on overly optimistic forecasts of bandwidth demand. There was a belief that if the infrastructure were in place, demand would inevitably follow. However, the demand didn’t grow as rapidly as projected, leading to overcapacity.

When the bubble burst in the early 2000s, many telecommunications companies wrote down the value of their fiber optic assets. These write-downs amounted to billions of dollars. Many telecom companies, particularly those that had aggressively expanded their networks during the boom, went bankrupt. These bankruptcies wiped out a significant portion of investor equity. Many fiber optic networks were left unused or underutilized, becoming “stranded assets.”

The telecom industry underwent a period of substantial consolidation as stronger companies acquired the assets of bankrupt or struggling ones, often at drastically reduced prices. This shifted the loss from one company to another but still represented an overall loss of value in the network. Overall, trillions of dollars of market value evaporated when the dot-com bubble burst. Much of this was tied to telecommunication companies that had invested heavily in fiber.

Many of the fiber optic networks deployed during the boom went “silent” and remained underutilized for years. There was just no demand for all of the capacity. However, the fiber cables were not physically removed or destroyed. They remained in the ground, on poles or underwater. What became “silent” or drastically devalued was the monetary value of these assets. The expectations for the revenue the networks would generate were never met. The dot-com bubble burst significantly reduced investor confidence in the telecommunications sector for years.

But all was not lost. The rapid growth of the internet eventually caught up and lit up the fiber optic networks. The winners weren’t the ones who paid for the fiber optic networks but the ones who later used it for streaming audio and video into everyone’s homes. A similar story can be told about the expansion of railroads in the 19^th century.

Small Language Models

Microsoft recently released Phi, a family of powerful, small language models with groundbreaking performance at low cost and low latency. [7] LLMs typically hoover data from the internet, a massive amount of words that include contradictory information. Phi was trained on higher-quality data using only 3.8 billion total parameters, matching the performance of highly capable models with hundreds of billions of parameters. Phase 1 training was comprised mainly of web sources and aimed to teach the model general knowledge and language understanding. Phase 2 included synthetic data to teach the model logical reasoning. Phi is more efficient, and a version of it fits on a cell phone. As LLMs migrate to edge devices, you may one day be able to talk to your teapot.

Figure 3. Ranking of Large Language Models by Competition. [8] Pairs of LLMs were compared on different problems to determine the Elo ranking, similar to how chess players are ranked based on their games.

Chatbot Arena is an open platform for crowdsourced AI benchmarking developed by researchers at UC Berkeley SkyLab and LMArena. [9] With over 1,000,000 user votes, the platform ranks the best LLM and AI chatbots. The ranking as of January 26, 2025, has a few surprises (Figure 3). Of the top ten LLMs, four are from Google, including the top two. OpenAI, the previous leader, has three LLMs in the top ten.

The big surprise is DeepSeek, a Chinese company with two LLMS in the top ten, one of which is in fourth place. The US is several years ahead of the world in developing LLMs, which typically cost $100s of millions to train to the highest level of performance. DeepSeek reached this level for only $5.6 million despite the export curbs that have reduced the availability of the latest GPUs to train LLMs. Moreover, it is open source, including code. Not too surprisingly, guardrails prevent comments on the People’s Republic of China. Most LLMs are extensively fine-tuned with expert knowledge, which was not used for DeepSeekR1 but was instead fine-tuned with reinforcement learning, the same way we teach students in schools.

Experts in Silicon Valley are dumbfounded. This illustrates that innovation often arises not at the top of the corporate ladder but with startups that devise new, more efficient algorithms rather than use brute force to solve complex problems. The US also has feisty startups, and Anthropic’s Claude 3.5 Sonnet, in 13^th place, is strong on sensitivity to nuance and emotional intelligence. [10] Claude has achieved popularity in Silicon Valley for its superior performance in giving dating advice.

ChatGPT is not the only game in town.

[1] A. Loten, “AI-Ready Data Centers Are Poised for Fast Growth,” Wall Street Journal, August 4, 2023.

[2] P. Sisson, A.I. Frenzy Complicates Efforts to Keep Power-Hungry Data Sites Green, New Yori Times, March 11, 2024.

[3] For some technical details leaked from OpenAI, see Dylan Patel and Gerald Wong, “GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Visions, MoE,” Semianalysis, July 10, 2023, https://www.semianalysis.com/p/gpt-4-architecture-infrastructure.

[4] David Donoho at Stanford has attributed the rapid pace of AI to “frictionless reproducibility” from open source tools and benchmark competitions. (Donoho, D., 2024. Data science at the singularity. Harvard Data Science Review, 6(1)).

[5] Zodhya, “How Much Energy Does ChatGPT Consume?,” Medium, May 20, 2023, https://medium.com/@zodhyatech/how-much-energy-does-chatgpt-consume-4cba1a7aef85.

[6] https://www.cerebras.net/product-chip/.

[7] https://arxiv.org/abs/2404.14219v1

[8] https://openlm.ai/chatbot-arena/

[9] https://lmarena.ai/

[10] https://medium.com/@arijitghosh646/claude-3-5-sonnet-why-its-superior-to-chatgpt-03ae3a5ae00e

Shortly after my book on The Deep Learning Revolution appeared in 2018, I gave a talk to a large general audience that kept asking questions long after the talk had ended. One question made me stop and think. I had given many examples of applications of deep learning networks – self-driving cars, diagnosis of medical disorders, advances in science – and the question was, which would have the most impact on our lives? After reflecting on this question, I realized that there was an application I hadn’t even mentioned that could have a massive impact on future generations: Education.

Students are already using ChatGPT to write essays for school assignments, which is difficult to detect and of great concern to teachers. A few teachers have decided that if you can’t beat them, join them, and have not only allowed students to use ChatGPT in their classrooms but have embraced it as part of their lesson plan. Here is an excerpt from one such teacher:

Jennifer Parnell [1], a history teacher at the Lawrenceville School, an independent school in Lawrenceville, N.J., was an early classroom adopter of ChatGPT. She began trying out AI chatbots in December and immediately incorporated the tools into her honors U.S. history and environmental science courses.

For a final exam in U.S. history, for instance, she used ChatGPT to manufacture an essay and then asked her students to analyze the A.I.-generated text for errors and rewrite it. Students also fed their own essays into the AI tool and asked it for feedback on the quality of their sources.

Parnell said she still has concerns about the use of AI tools in schools, including issues of bias, privacy and academic honesty. But she believed the potential benefits outweighed the downsides.

“A.I. has pushed teachers to think more intentionally about the purpose of education and specifically assessment,” she said. “As a teacher, if I’m asking questions that are easily answered by A.I., am I asking the best questions?” [2]

Teachers will not lose their jobs because of AI, but their jobs will change. As ChatGPT integrates into society, teaching students to use ChatGPT is also good job training. OpenAI has released a guide to help teachers use ChatGPT in their classrooms. [3]

Some reject ChatGPT outright – that’s not how we did this in the past – and others embrace and find creative uses. The rejecters rightly assert that if they allowed students to use ChatGPT, their ability to think independently and learn how to write would suffer. The embracers rightly realize that this new technology is the future and students should learn how to use it properly. We have gone through many technology shifts and found ways to live with them and thrive.

The introduction of hand-held calculators in the twentieth century was a technology shift that had a similar impact on education. Many teachers would not allow them in their classrooms. There was also a concern that students would lose the ability to perform mental arithmetic. However, with time, calculators found a niche that increased productivity, and it may be true that mental arithmetic has suffered, but it is still a mainstay of the three R’s.

The World of Education

Education is a trillion-dollar, labor-intensive business in the U.S. Brains are highly complex learning devices, and education is a cultural invention devised to train brains efficiently. The U.S. once had excellent universal education but has reached the bottom of the world ranking. According to the data in Figure 1, the U.S. is in the bottom quartile when compared to other Organization for Economic Co-operation and Development (OECD) countries according to the International Student Assessment (PISA), conducted by the OECD to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students.

Figure 1. Snapshot of reading, mathematics, and science performance of 15-year-old students in OECD countries. The countries are ranked in order by the PISA score on the x-axis. East Asian countries have pulled ahead of Western countries. China is far ahead. Estonia is the lone Western country in the top quartile, and the U.S. is near the bottom [4].

What went wrong? I live in California, where K-12 education was once among the best in the world. Still, even among states, it is now near the bottom. When I was learning how to read, I practiced until reading became automatic. I also learned how to write by practicing penmanship and arithmetic by practicing adding, subtracting, multiplying, dividing, and taking the square roots of numbers.

Practice has fallen out of favor in classrooms, and reform educators have shifted to a new approach that leads to better cognitive understanding but poorer skills. Practice in schools is considered “drill and kill,” which puts too much stress on children. Barbara Oakley and I have written a blog explaining why practice is essential for shaping efficient brain circuits [5] – what Daniel Kahneman called thinking fast.[6] Cognitive thinking is more flexible but slower and more prone to error. The brain has specialized systems for fast and slow thinking. Brains need a balance of these two learning systems, and so does ChatGPT.

ChatGPT Can Enhance Education

The most effective way to teach a child is one-on-one instruction from an experienced teacher who motivates the child to overcome learning problems with complex subjects like mathematics. In contrast, most schools have an assembly-line system designed for mass education: students are segregated by age, taught in large classes, and moved from class to class like an assembly line; teachers impart the same lessons year after year to children at the same physical age but not at the same mental age, making teaching much more difficult. An assembly line may be a good way to build automobiles, but it is a poor way to form children’s minds. The problems with our educational system have become more acute as they fail to educate students for many jobs now requiring a higher level of training and skills.

Computer programs that automate question-asking have been around for decades but are less effective than personal tutors. ChatGPT could have a significant impact, especially if they are equipped with long-term memory and can track the development of a student. A ChatGPT personal tutor, TutGPT, could provide children access to the world’s knowledge base. TutGPT teaching assistants would improve and transform current education practices in the U.S. Schools, which could still serve a useful purpose in tracking student performance, providing extracurricular activities and sports, and socializing students.

Prompting to Teach

How do good teachers interact with students to help them understand new concepts? A good teacher knows what the student doesn’t know, focuses on what needs to be known, and motivates the student to actively integrate the new knowledge into what the student already knows.

Ethan Mollick is on the Wharton School of the University of Pennsylvania faculty, where he teaches innovation and entrepreneurship and examines artificial intelligence’s effects on work and education. He engineered a prompt for tutoring ChatGPT into how to be an effective tutor:

“You are a friendly and helpful tutor. Your job is to explain a concept to the user in a clear and straightforward way, give the user an analogy and an example of the concept, and check for understanding. Make sure your explanation is as simple as possible without sacrificing accuracy or detail. Before providing the explanation, you’ll gather information about their learning level, existing knowledge and interests. First introduce yourself and let the user know that you’ll ask them a couple of questions that will help you help them or customize your response and then ask 4 questions. Do not number the questions for the user. Wait for the user to respond before moving to the next question. Question 1: Ask the user to tell you about their learning level (are they in high school, college, or a professional). Wait for the user to respond. Question 2: Ask the user what topic or concept they would like explained. Question 3. Ask the user why this topic has piqued their interest. Wait for the user to respond. Question 4. Ask the user what they already know about the topic. Wait for the user to respond. Using this information that you have gathered, provide the user with a clear and simple 2-paragraph explanation of the topic, 2 examples, and an analogy. Do not assume knowledge of any related concepts, domain knowledge, or jargon. Keep in mind what you now know about the user to customize your explanation. Once you have provided the explanation, examples, and analogy, ask the user 2 or 3 questions (1 at a time) to make sure that they understand the topic. The questions should start with the general topic. Think step by step and reflect on each response. Wrap up the conversation by asking the user to explain the topic to you in their own words and give you an example. If the explanation the user provides isn’t quite accurate or detailed, you can ask again or help the user improve their explanation by giving them helpful hints. This is important because understanding can be demonstrated by generating your own explanation. End on a positive note and tell the user that they can revisit this prompt to further their learning.” [7]

Try out this prompt and pretend you are a student having difficulty learning something you already know.

The Future of Education

Deep Blue was a chess-playing program designed with special-purpose chips by IBM that could explore game positions farther than humans could. Deep Blue beat the world chess champion, Gary Kasparov, in 1997. Chess-playing computer programs powered by AI have learned to play far better than Deep Blue. Has this stopped humans from playing chess? On the contrary, it raised our level of play. It has also democratized chess. The best chess players once came from big cities like Moscow and New York, which had a concentration of grandmasters who could teach younger players and raise their level of play. Chess-playing computer programs made it possible for Magnus Carlson, who grew up in a small town in Norway, to become a chess grandmaster at thirteen, and he later became the world chess champion. ChatGPT tutors could similarly raise everyone’s educational level on the planet. AI can make us all smarter.

[1] https://www.linkedin.com/in/jennifer-parnell-m-ed-m-a-99a08a5b

[2] Natasha Singer, How teachers and students feel about A.I. https://www.nytimes.com/2023/08/24/technology/how-teachers-and-students-feel-about-ai.html

[3] Teaching with AI. https://openai.com/blog/teaching-with-ai

[4] https://worldpopulationreview.com/country-rankings/pisa-scores-by-country

[5] B. Oakley and T. Sejnowski, "The Promise of Habit-Based Learning," Law & Liberty, 2022. https://lawliberty.org/features/the-promise-of-habit-based-learning/

[6] Kahneman, D., Thinking, Fast and Slow. (New York, Farrar, Straus and Giroux, 2011).

[7] Ethan Mollick, “Now is the time for grimoires,” https://www.oneusefulthing.org/p/now-is-the-time-for-grimoires [A grimoire is a a textbook of magic, typically including instructions on how to create magical objects like talismans and amulets, how to perform magical spells, charms, and divination.]

An artificial-intelligence-generated character on a screen in Las Vegas, Nevada. Credit: John D. Ivanko/Alamy

Nature Magazine BOOK REVIEW

Nature 635, 284-285 13 November 2024

By Pablo M. Tagarro

ChatGPT and the Future of AI: The Deep Language Revolution

Terrence J. Sejnowski — The MIT Press (2024)

Some of the surprise winners of this year’s Nobel prizes were the developers of AlphaFold, an artificial intelligence (AI) model that can accurately predict the 3D structure of more than 200 million proteins. AlphaFold is powered by artificial neural networks that can glean patterns from how amino acids interact in known proteins and then use that information to model the behaviour of unknown proteins. Chatbots such as ChatGPT rely on similar technology to ‘learn’ and mimic human language.

The revolution set in motion by this technology is compellingly explored in Terrence Sejnowski’s ChatGPT and the Future of AI — a sequel to his 2018 book The Deep Learning Revolution. Sejnowski, a computational neurobiologist, takes readers on a whirlwind tour of the evolution of AI — from simple computational models of a single neuron built in the 1950s to breakthroughs in deep learning over the past two decades that have resulted in the birth of large language models (LLMs), which can generate human-like responses to questions.

Sejnowski’s book is a must-read for those seeking to understand the world we live in — a world in which machines transform the fabric of human cognition. Put simply, a neural network is a web of interconnected nodes, or neurons, that can process data and learn from them by adjusting the strength of the connections between the nodes. When the strength of these connections is altered to achieve a desired output during a training phase, the model learns to detect underlying patterns.

The fundamental inspiration behind neural-network architectures is the human brain. Sejnowski suggests that untangling the mystery of why these simulated models of the brain work so well — especially when they are trained on large amounts of data — could prove to be as seminal as the discovery of DNA. By rigorously interrogating the “otherworldly behaviour” of LLMs, new conceptual frameworks might emerge, he predicts.

He sees parallels between the current debates over the meanings of ‘intelligence’ and ‘understanding’ and discussions a century ago about the nature of life. Back then, vitalists argued that life is a non-physical force — an essence infused in living things but not in, say, rocks — whereas mechanists thought that life could be fully explained by physical and chemical processes. Just as the discovery of the DNA double helix bridged conceptual gaps and transformed biology, Sejnowski predicts that advances in AI will bring forth revelations about intelligence.

Evolving understanding

The holy grail of AI, Sejnowski explains, is artificial general intelligence: a machine that can think, learn and solve problems across a wide range of tasks, much like a human can. The current generation of LLMs is far from that. Referred to pejoratively by some researchers as ‘stochastic parrots’, they mostly mimic human language without true comprehension.

The road ahead for AI is one of interdisciplinary collaboration, Sejnowski argues, in which insights from biology, neuroscience and computer science converge to guide AI development. Sejnowski imagines that insights about the “fundamental principles of intelligence” — such as adaptability, flexibility and the ability to make general inferences from limited information — will catalyse the next generation of machine intelligence.

The AI language revolution, which is how Sejnowski refers to the era of LLMs, is already reshaping many aspects of human life. As LLMs evolve, they will surpass their primary role as tools and start acting as collaborators in domains such as health care, education and law. That’s already beginning to happen, as shown by AlphaFold. The author liberally uses ChatGPT to provide summaries at the end of each chapter, and conversations with the chatbot are littered throughout the book. He even playfully acknowledges ChatGPT as a co-author.

The power of LLMs also lies in how users interact with them. Sejnowski flags the increasingly important skill of prompt engineering, which stresses that subtle changes in how you instruct a model can shape its output. The author offers helpful hacks: ask for multiple responses, not just one; be specific, so that the model can converge on the best answer quickly; shape your dialogue as if you are talking with a real person.

Sejnowski proposes a “reverse Turing test”, in which the intelligence of the prompter is assessed on the basis of the quality of their interactions with the AI. Such proficiency tests might become common as AI use spreads.

The next generation of LLMs must undergo a developmental process akin to the childhood learning phase in humans, he surmises, learning from real-world interactions as well as data. Long-term memory and goal-oriented behaviour must be integrated so that systems can adapt and plan effectively. The addition of robotics and sensorimotor systems would allow AI tools to perceive and interact with their environment, nudging current models towards artificial general autonomy.

Although Sejnowski is optimistic about the technology, he acknowledges that there are many unsolved challenges that will need to be addressed to ensure the long-term sustainability of AI. It is likely to disrupt industries and the job market. And, the increasing necessity for heavy computational processing power has brought to the forefront the need for more-efficient, greener AI models.

Although it is some way off, it is also important, Sejnowski writes, to seriously examine the possibility of AI exceeding human intelligence, because preparing now will allow us to anticipate and mitigate potential threats. A careful and well-regulated strategy to manage ethical and existential risks — such as of AI-induced errors and political weaponization of LLMs — is essential to ensure that AI benefits society.

ChatGPT and the Future of AI sets out how the next wave of AI-driven breakthroughs could alter the landscape of knowledge itself. Sejnowski’s exploration serves as both a guide and a warning as we navigate the promises and perils of AI’s rapid advancement.

https://www.nature.com/articles/d41586-024-03679-6

All new technologies have risks and benefits. How can AI be regulated to mitigate its misuse while reaping its bounty?

At the NeurIPS meeting in Montreal in 2018, Edward Felton gave a plenary talk on how the regulatory machinery works in Washington. [1] He warned us that if we don’t regulate ourselves, the government will; when that happens, it is vital to be at the table where the laws are made. The danger is that overly restrictive laws can choke innovation, and even benign laws can have unintended consequences.

Self-Regulation

Figure 1. At the Asilomar meeting, molecular biologists self-imposed stringent containment procedures to prevent pathogens from escaping into the wild. [2]

The shortcomings of deep-learning AI systems are well-known, including concerns about privacy, bias, and racism, and the community has been working to correct these defects. LLMs have added to these problems with hallucinations and broader ethical concerns. All new technologies undergo a long process of updating and shaping to prevent the misuse of the technologies in society.

In the 1970s, biologists invented recombinant gene technology to manipulate and isolate DNA segments of interest, combine DNA from different species, and create new functions. Researchers realized that this technology gave them immense power to modify DNA and even create new life forms with the potential for improving crops and curing disease, but also with the danger of unleashing supervirulent and cancer-carrying viruses. Molecular biologists did not wait for regulatory agencies to curtail potentially dangerous experiments. They organized an International Congress on Recombinant DNA Molecules, held at the Asilomar Conference Center in Pacific Grove, California, in February 1975 (Figure 1). [3]

There was disagreement among the scientists about how to restrict experiments. After vigorous debate, they recommended several levels of containment for experiments with different levels of risk. Low-risk containment was appropriate when recombinant DNA could not significantly increase pathogenicity or disrupt ecologies. High-risk containment was needed when the modified organism could lead to severe consequences and pose a serious biohazard to laboratory personnel or the public. These rules would allow research to continue but under stringent guidelines.

These recommendations became standard practice by the scientific community. They made it possible for scientists to perform experiments and advance science safely. The Institutional Review Boards (IRBs) at universities and companies review proposed experiments to ensure they comply with regulations, meet accepted ethical standards, follow institutional policies, and adequately protect research participants. These safety policies are amended as discoveries are made and even more powerful gene manipulation techniques are invented.

There is a thriving biotechnology industry today, and many patients have benefited. For example, by tinkering with genes, scientists discovered that cancer was a genetic disease with diverse pathways leading to different cancer types. Once a pathway is identified, specific biochemical reactions can be devised to target and suppress proliferation. For example, Tony Hunter, my colleague at the Salk Institute, discovered a new class of enzymes in cells that led to Gleevec. This drug can check the progression of a form of leukemia. Immunotherapy has cured cancers that were once death sentences, such as melanoma skin cancer and non-small cell lung cancer.

Self-regulation seems like a sensible way for the AI community to move forward, but not easily given the diverse views in the community and powerful self-interests in the corporate sector. The time is right for scientists and engineers who understand AI technology to work with policy experts to develop a flexible regulatory framework.

Government Regulation

In 2021, the European Union proposed a European law on artificial intelligence. The AI Act became the first law on AI by a major regulator when the European Parliament passed it on March 13, 2024. The document is 105 pages long, with 89 sections in the preamble, 44 definitions, 4 forbidden practices, and 85 Articles spelling out rules and penalties, all written in legalese. [4] For example, it limits resume-scanning tools that rank job applicants. This approach looks well-meaning but premature. [5] AI is moving so quickly that these proposed laws are already obsolete. For example, ChatGPT and generative AI models were not mentioned when the law was drafted. AI is evolving much faster than regulatory machinery.

On May 16, 2023, Sam Altman, CEO of OpenAI, testified before Congress for three hours on the need to regulate AI (Figure.2). [6] “I think if this technology goes wrong, it can go quite wrong. And we want to be vocal about that,” he said. “We want to work with the government to prevent that from happening.” He proposed a government agency that investigates companies and issues licenses for them to develop LLMs, including safety regulations and tests before they can be released to the public, just as the Food and Drug Administration (FDA) regulates drug clinical trials. “We believe that the benefits of the tools we have deployed so far vastly outweigh the risks, but ensuring their safety is vital to our work,” Altman said.

Figure 2 Sam Altman, CEO of OpenAI, testified before Congress on May 16, 2023. (New York Times).

Altman’s testimony before Congress was cordial, contrasting the antagonistic interactions previously experienced by Mark Zuckerberg, Jeff Bezos, and other high-tech CEOs. Altman also had dinner with dozens of House members and met individually with several senators before the hearing, where he outlined a loose regulatory roadmap for managing the rapidly developing AI landscape that could significantly impact the economy. On June 9, 2023, Altman made a high-profile trip to South Korea and called for the coordinated international regulation of generative artificial intelligence.

Altman’s full-court press for regulations puzzled me. It reminded me of the strange effects of Toxoplasma gondii, a brain parasite, on rodents: once infected, rodents lose their fear of cats and become more likely to be eaten. [7]  What is the advantage of this perversion for the parasite? The parasite can only reproduce in carnivores, and cats can spread the parasite far and wide. 

Another possible explanation for why Altman offered himself up is that stiff regulations would favor the largest high-tech companies that can afford the stringent testing, comparable to the hundreds of millions of dollars that clinical trials cost pharmaceutical companies. Regulations lead to lawsuits that favor high-tech companies with their deep staff lawyers. Big Pharma companies buy small biotechnology companies that can’t afford phase 3 clinical trials. Altman was proposing the same model for the AI industry to Congress.

On November 17, 2023, Ilya Sutskever, the chief scientist and a member of the board of trustees, announced that Sam Altman was fired for being “not consistently candid.” The board that supported this coup was heavily skewed toward those who thought Altman was not prioritizing dangers posed by AI and was moving too fast. When investors and employees pushed back, Altman was reinstated four days later. [8] This is a microcosm of the AI debate between those who want to slow down and those who want to move forward. The new board now resembles boards in other high-tech companies, whose goals are aligned with those of its investors. This dramatic boardroom confrontation is a harbinger of how AI might play out in the government, which needs to balance safety with viability.

On October 30, 2023, the White House issued an Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (Figure 3) [9]. This 117-page document requires companies to provide the government with safety testing results and other proprietary information before making large AI models available to the public. Government agencies will develop and enforce standards. The Executive Order does not prevent using copyrighted data to train LLMs.

Figure 3. The Office of the President issued this Executive Order to regulate AI in the United States in 2023.

Another Way to Regulate AI

Regulation of AI has emphasized reducing risks, but the risks must be balanced against potential benefits. The European Union’s heavy-handed attempt to regulate AI by micromanaging the risks is an excellent way to stillbirth the benefits. The United States has taken a light-handed approach, giving AI an open road to expand and diversify while minimizing risks. Which is the better approach? Perhaps there is another way.

Jason Furman recently wrote “How to Regulate AI Without Stifling Innovation.” [10]  He outlined six principles, paraphrased here, to ensure that AI can safely grow while balancing risks with benefits:

1)      New regulations should be introduced not at the federal level but at the level of specific applications. We already have agencies regulating drugs, airlines, and environmental safety. AI will impact all these areas, so let these agencies decide how best to regulate each application. AI agency expertise will have to expand for this to be successful. Our educational systems will have to shift to high gear to train workers and regulators to handle a tidal wave of AI applications.

2)      Regulations should not favor high-tech companies but should encourage innovation by researchers and startups. There are 100,000 AI startups worldwide, minnows that need protection from hungry predators. Here are some precedents for how gatekeepers have stifled competition:  Medieval guilds, teachers’ unions, and centralized licensing bodies. Do not let big companies or big lobbyists set rules that exclude fair competition.

3)      How are existing regulations hindering progress? AI needs more data centers to meet demand. However, local laws are making it difficult for companies to obtain the power to run them. States have introduced jumbles of regulations that are counterproductive. Here is where federal leadership could ensure a level playing field for AI growth with minimal intervention or cost.

4)      Regulation should not delay the adoption of AI until it has been proven to be absolutely safe. Strong regulations can delay the incremental improvement needed to achieve safe AI or even block the path. Lost benefits include increased cancer deaths due to delayed drug discovery, poor educational outcomes by delays in personal digital tutors, excess car accidents by delays in self-driving cars, and delays in decreasing the carbon footprint because of a slowdown in discovering better battery storage for sustainable power on the grid.

5)      Don’t compare AI with perfection, but with imperfect humans. The number of crashes caused by self-driving cars should be compared with the number of crashes caused by humans. There are currently over 40,000 deaths on highways each year, an acceptable loss that balances the costs and benefits to society. How do biases in AI compare with human biases? Which will be easier to fix? AI is improving much faster than humans.

6)      Regulation is not the answer to all problems. Jobs are already changing. AI can make workers more productive. Don’t ban AI from industry and classrooms. Invest instead in training the workforce with new skills. Companies and schools will need help adapting their procedures and curriculum. Government incentives and interventions could jump start the arrival of future benefits.

These are sensible principles that can help guide us toward a balance between future risks and benefits of AI.

[1] The video can be seen at https://videoken.com/embed/bf-E2oVjI9M.

[2] https://www.nature.com/articles/455290a  Art by David Parkins. Sydney Brenner with the bushy eyebrows is sitting in the middle. https://dnalc.cshl.edu/view/15284-The-media-at-Asilomar-Sydney-Brenner.html Who can identify the other delegates?

[3] Paul Berg, “Asilomar 1975: DNA Modification Secured,” Nature 455 (2008): 290–291, https://www.nature.com/articles/455290a.

[4] You can download the AI Act from https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206.

[5] An early draft of the AI Act required sourcing for all data used to train the model. This was removed after Mistral, a French AI startup, lobbied with political support from President Macron’s office (“Meet the French startup hoping to take on OpenAI, The Economist,” March 2, 2024). Mistral was later investigated by the European Commission when it formed a strategic partnership with Microsoft (https://www.reuters.com/technology/microsofts-deal-with-mistral-ai-faces-eu-scrutiny-2024-02-27/).

[6] Cecilia Kang, “OpenAI’s Sam Altman Urges A.I. Regulation in Senate Hearing,” New York Times, May 16, 2023.

[7] Kelly Servick, “Brain parasite may strip away rodents' fear of predators—not just of cats,” January 14 2020 (https://www.science.org/content/article/brain-parasite-may-strip-away-rodents-fear-predators-not-just-cats)

[(9] The White House, Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, October 30, 2023, https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/.

[10] Furman, J., How to Regulate AI Without Stifling Innovation, Wall Street Journal, November 21, 2024.

Figure 1. Dopamine is a neuromodulator in your brain that has many functions.

Dopamine-releasing neurons are a core brain system that controls motivation [1]. Your dopamine neurons can be polled when you need to make a decision. What should I order from the menu? You imagine each item, and your dopamine cells provide an estimate of the expected reward. Should I marry this person? Your dopamine cells will give you a "gut" opinion more trustworthy than reasoning. Problems with many different dimensions are the most difficult to decide. How do you trade off a sense of humor, a positive dimension, against being messy, a negative dimension, or hundreds of other comparisons? Your brain's reward systems reduce all these dimensions to a common currency, the transient dopamine signal. Dopamine neurons receive inputs from a part of the brain called the basal ganglia, which receives input from the entire cerebral cortex.   The basal ganglia evaluate cortical states and are involved with learning sequences of motor actions to achieve a goal.

The dark side of reward is that all addictive drugs act by increasing the level of dopamine activity. In essence, drugs like cocaine and heroin (as well as nicotine and alcohol) hijack the dopamine reward system, making your brain believe that taking the drug is your most important immediate goal. Withdrawal symptoms dominate when drugs are not immediately available. This motivates desperate actions to obtain more drugs, which can jeopardize life and livelihood. Even after an arduous rehabilitation process, which can take years, the brain's reward circuit is still altered by the experience of addiction, leaving the recovering addict vulnerable to a relapse. People and places can trigger this, as well as sounds and smells previously associated with the drugs or even paraphernalia used to take the drugs. For a person with a substance use disorder, dopamine is deeply compelling.

When enough dopamine-releasing neurons die, the symptoms of Parkinson's disease appear, which include motor tremors, difficulty initiating actions, and, eventually, anhedonia, the complete loss of pleasure in any activity. The end-stage includes catatonia, a complete lack of movement and responsiveness. But when the dopamine neurons are behaving normally, they provide brief bursts of dopamine to the neocortex and other brain areas when an unexpected pleasure (reward) occurs (be it food, money, social approval, or several other things), and a diminution of activity when less than expected reward is experienced. 

How dopamine guides learning

The basal ganglia are part of all vertebrate brains. Within the basal ganglia, the dopamine neurons mediate a form of learning called associative learning, which Pavlov's dog made famous. In Pavlov's experiment, a sensory stimulus, such as a bell (a conditioned stimulus), was followed by the presentation of food (an unconditioned stimulus), which elicited salivation even without the bell (an unconditioned response). After several pairings, the bell would lead to salivation (a conditioned response).   Different species have different preferred unconditioned stimuli. Bees are very good at associating a flower's smell, color, and shape with the rewarding nectar, and they use this learned association to find similar flowers in season.   Something about this universal form of learning must be significant, and there was a period in the 1960s when psychologists intensively studied the conditions that gave rise to associative learning and developed models to explain it.  Only the stimulus that occurs just before the reward becomes associated with the reward [2].

This makes sense since the stimulus is more likely to have caused the reward if it comes before the reward than a stimulus just after the reward.   Causality is an essential principle in nature. The experiment that gave the game away for associative learning was based on stimulus blocking. After pairing a conditioned stimulus, such as a sound, with a reward (a positive unconditioned stimulus), like food or water, the brain learns that the sound predicts a reward in the near future. 

Suppose you had to make a series of decisions to reach a goal. If you don't have all the information about the outcomes of the choices ahead of time, you have to learn as you make the choices in real-time. When you get a reward after a sequence of decisions, how do you know which options you made were responsible? A learning algorithm that can resolve this problem called the temporal credit assignment, was discovered by Richard Sutton at the University of Massachusetts at Amherst in 1988 [3], working closely with Andrew Barto, his thesis advisor. This is challenging problem in reinforcement learning, a branch of machine learning inspired by animal associative learning.   In temporal difference learning, you compare your expected reward for making a particular choice with the actual reward you get and change your expected reward so that the next time, you can make a better decision. Then, an update is made to the value network that computes the future expected reward for each decision at each choice point. The temporal difference algorithm converges to the optimal path after you have had enough time to explore the possibilities. This is followed by exploiting the best strategy found during the exploration.

Bees are champion learners in the insect world. It only takes a few visits to a rewarding flower for a bee to remember the flower. Unique dopamine neurons in the bee brain represent reward prediction errors. My lab modeled these neurons using temporal difference learning. Our model of bee learning could explain some subtle aspects of bee psychology, such as risk aversion. For example, when a bee is given a choice between a constant reward and twice the amount but at 50% probability (on average, the same amount), bees prefer the constant reward. Read Montage, another postdoctoral fellow in my lab, took the next leap and realized that dopamine neurons in the vertebrate reward system may have a similar role in our brains [4]. These models and their predictions were published and subsequently confirmed in monkeys with single neuron recordings and humans with brain imaging. Transient changes in the activity of dopamine neurons signal reward prediction error (Figure 2). 

Figure 2. Responses of a dopamine neuron in the monkey brain signal errors in reward predictions to the rest of the brain. Each dot is a spike in a dopamine neuron. Each line is a single learning trial. The number of spikes in each time bin across trials is shown at the top of each raster. Top panel: At the beginning of learning, the reward is unexpected, and the dopamine consistently fires a burst of spikes shortly after the reward. Middle panel: When a light consistently flashes before reward delivery (CS), the dopamine cell responds to the CS but not the reward. According to temporal difference learning, the reward prediction cancels the response after the reward. Bottom panel: When the reward was withheld on some trials, a dip in firing was revealed that represented the predicted reward. (Adapted from Schultz et al.[5])

How to learn sequences of actions for a reward

Temporal difference learning might seem like weak feedback since you only find out whether or not you are rewarded at the end of a sequence of actions, which may be quite long. However, several applications of temporal difference learning have shown that it can be powerful when coupled with other learning algorithms. Gerry Tesauro worked with me on the problem of teaching a neural network to play backgammon. Backgammon is a popular game in the Middle East; some make a living playing high-stakes games.   It is a race to the finish between two players, with pieces that move forward based on each roll of the dice, passing through each other on the way. Unlike Chess, which is deterministic, the uncertainty with every roll of the dice makes it more difficult to predict the outcome of a particular move. The knowledge of backgammon in Gerry's program was captured by a value function that provided an estimate of winning the match from all possible board positions as ranked by a panel of backgammon experts. A good move can be found simply by evaluating all possible moves from the current position and choosing the one with the highest value. 

Our approach used expert supervision to train neural networks to evaluate game positions and possible moves. The flaw in this approach is that many expert evaluations of board positions were needed, and the program could never get better than our experts. When Gerry moved to the IBM Thomas J. Watson Research Center, he switched from supervised learning to temporal difference learning, and his backgammon program played itself. The problem with self-play is that the only learning signal is a win or a loss at the end of the game. There is no information about the contribution of the many intermediate moves during the game to that win or loss. 

At the beginning of the backgammon learning, the machine's moves were random, but eventually, one side won. The reward first taught the program how to "bear off" and exit all of the pieces from the board at the end of the game. Once the endgame was learned, the value function for bearing off, in turn, trained the value function for the crucial middle game, where subtle decisions need to be made about engagements with the other player's pieces. Finally, after playing a hundred thousand games, the value function was honed to play the opening in which pieces take defensive positions to prevent the other player from moving forward and learning proceeds from the end of the game, where there is an explicit reward, back toward the beginning of the game using the implicit reward learned by the value function. This shows that a weak learning signal like the dopamine reward system can learn a sequence of decisions to achieve a long-term goal by back-chaining with a value function. 

Tesauro's TD-Gammon program surprised me and many others when he revealed it to the world in 1992 [6]. The value function had a few hundred model neurons, a relatively small neural network by today's standards. After a hundred thousand games, the program beat Gerry. He alerted Bill Robertie, an expert on positional play in backgammon from New York City, who visited IBM to play TD gammon. Robertie won most games but was surprised to lose several well-played games and declared it the best backgammon program he had ever played. Several of the moves were unusual ones that he had never seen before, which on closer examination proved to be improvements on typical human play. Robertie returned when the program had reached a million self-played games and was astonished when TD-gammon played him to a draw. A million may seem like a lot, but keep in  mind that, after a million games, the program only saw an infinitesimal fraction of all possible board positions. Thus, TD-gammon was required to generalize to new board positions on almost every move.

In March 2016, Lee Sedol, the Korean Go World Champion, played a match with AlphaGo [7]. This program learned how to play Go using temporal difference learning. AlphaGo used neural networks with a much larger value network having millions of units to evaluate board positions and possible moves. Go is to Chess in difficulty as Chess is to checkers. Even DeepMind, the company that had developed AlphaGo, did not know its strength. AlphaGo had played hundreds of millions of games by itself, and there was no way to benchmark how good it was. It shocked many when AlphaGo won the first three games of the match, exhibiting an unexpectedly high level of play. Some of the moves made by AlphaGo were revolutionary.

Figure 3. Demis Hassabis (left), a founder of DeepMind, and Ke Jie (right), world Go champion, after AlphaGo decisively won three out of three games.

On January 4, 2017, a Go player on an Internet Go server called “Master” was unmasked as AlphaGo 2.0 after winning 60 out of 60 games against some of the world’s best players, including the world’s reigning Go champion, the 19-year-old prodigy Ke Jie of China. It revealed a new style of play that went against the strategic wisdom of the ages.  On May 27, 2017, Ke Jie lost three games to AlphaGo, at the Future of Go Summit in Wuzhen, China.  These were some of the best Go games ever played and hundreds of millions of Chinese followed the match. Ke Jie concluded: “Last year, I think the way AlphaGo played was pretty close to human beings, but today I think he plays like the God of Go.”   

Making life decisions

We are just beginning to appreciate the powerful impact of dopamine on making decisions and guiding our lives. Since the influence of dopamine is subconscious, the story we tell ourselves to explain a decision does not reflect all the underlying experiences. We make up stories because we need to have conscious explanations. Every once in a while, you have a "gut feeling" about a choice that does not have an easy explanation  – it's the dopamine that made you do it. [8]

[1] Bromberg-Martin, E., Matsumoto, M., & Hikosaka, O. (2010). Dopamine in motivational control: rewarding, aversive, and alerting. Neuron, 68(5), 815-834.

[2] There are some notable exceptions to the notion that the conditioned stimulus must immediately precede the unconditioned stimulus in associative learning.  One is food aversion learning.  If you eat something and then become ill hours later, you will still strongly associate that food with the illness and tend to avoid that food in the future, even though the conditioned stimuli (the sight, smell and taste of the food) can precede the unconditioned stimulus (feeling ill) by several hours.

[3] Sutton, R. S. (1988). Learning to predict by the method of temporal differences. Machine Learning, 3:9-44.

[4] Montague, P. R. Dayan, P. Sejnowski, T. J. (1996) A Framework for Mesencephalic Dopamine Systems Based on Predictive Hebbian Learning, Journal of Neuroscience, 16(5), 1936-1947,

[5] Schultz W, Dayan P, Montague PR. (1997) A neural substrate of prediction and reward.  Science. 275: 1593-1599.

[6] Tesauro, Gerald (1995). Temporal Difference Learning and TD-Gammon. Communications of the ACM 38 (3) 58-68

[7] Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D. (2016) Mastering the game of Go with deep neural networks and tree search. Nature. 529: 484-489.

[8] Adapted from: Sejnowski, T. J. (2019) Dopamine Made You Do In: It Linden, D. (Ed.), Think Tank: Forty Neuroscientists Explore the Biological Roots of Human Experience, Yale University Press: New Haven and London, 267-262.

This Substack has been tracking a new book on “ChatGPT and the Future of AI” released on October 29, 2024.

Table of Contents:

Preface

Part I: Living with Large Language Models

1          Introduction

2          How Chatbots Are Changing Our Lives

3          Interviews with Large Language Models

4          The Power of the Prompt

5          What Are Intelligence, Thinking, and Consciousness?

Part II: Transformers

6          Origins of Deep Learning

7          High-Dimensional Mathematics

8          Computational Infrastructure

9          Superintelligence

10        Regulation

Part III: Back to the Future

11        Evolution of AI

12        The Next Generation

13        Learning from Nature

14        The Future Is Now

Afterword

Acknowledgments

Glossary

Notes

Index

The book's three parts are like three scenes in a play where Part I lays out the strengths and weaknesses of ChatGPT today, Part II takes a deep dive into how it works, and Part III explores how ChatGPT is evolving and what to expect in the future.  A lot has happened in AI since the book went to press in September, and this Substack allows me to bring it up to date.  The book did not anticipate the Nobel Prizes for AI described in the last Part. [1]

The Transmitter

The Transmitter is a neuroscience newsletter supported by the Simons Foundation. [2] The following excerpt from my book, in the spirit of my Substack title, Brains and AI, will soon appear online:

Are Brains and AI Converging?

Research on brains and AI are based on the same basic principles: massively parallel architectures with a high degree of connectivity trained with learning from data and experience.  Brain discoveries made in the twentieth century inspired new machine learning algorithms: the hierarchy of areas in the visual cortex inspired convolutional neural networks, and operant conditioning inspired the temporal difference learning algorithm for reinforcement learning.  In parallel with the advances in artificial neural networks, the BRAIN Initiative has accelerated discoveries in neuroscience in the twenty-first century by supporting the development of innovative neurotechnologies.  Machine learning is being used by neuroscientists to analyze simultaneous recordings from hundreds of thousands of neurons in dozens of brain areas and to automate the reconstruction of neural circuits from serial section electron microscopy.  These advances have changed how we think about processing distributed across the cortex and led to discoveries that created a new conceptual framework for brain function, leading to even more advanced and larger-scale neural network models.

The new conceptual frameworks in AI and neuroscience are converging, accelerating their progress.  The dialog between AI and neuroscience is a virtuous circle that is enriching both fields.  AI theory is emerging from analyzing the activity patterns of hidden units in ultra-high-dimensional spaces, which is how we study brain activity.  Analyzing the dynamics of the activity patterns in large language models (LLMs) may lead us to a deeper understanding of intelligence by uncovering a common underlying mathematical structure.  For example, an LLM was trained on board positions for the game Othello and was probed to reveal an internal model for the rules of Othello.

Now that we can interrogate neurons throughout the brain, we may solve one of its greatest mysteries: how information globally distributed over so many neurons is integrated into unified percepts and brought together to make decisions.  The architectures of brains are layered, with each layer responsible for making decisions on different time scales in both sensory and motor systems.  We can build deep multimodal models with many component networks and integrate them into a unified system, giving insights into the mechanisms responsible for subconscious decision making and conscious control.

How to Download a Brain

Neurons are traditionally interrogated in the context of discrete tasks, such as responses to visual stimuli, in which the choices and stimuli are limited in number.  This tight control of stimulus and response allows the neural recordings to be interpreted in the context of the task.  But neurons can participate in many tasks in many different ways, so interpretations derived from a single task can be misleading.  We now can record from hundreds of thousands of neurons brain-wide, and it is also possible to analyze recordings and dissect behavior with machine learning.  However, neuroscientists are still using the same old single-task paradigms.  One solution is to train on many different tasks, but training a monkey, for example, takes weeks to months for each task.  Another solution is to expand the complexity of the task over longer time intervals, bringing it closer to natural behaviors.

There is an even more fundamental problem with approaching behavior by studying discrete tasks.  Natural behaviors of animals in the real world are primarily self-generated and interactive.  This is especially the case with social behaviors.  Studying such self-generated continuous behaviors is much more difficult than studying tightly constrained, reflexive ones.

What if an LLM were trained on massive brain recordings during natural behaviors and accompanying behavior, including body and eye tracking, video, sound, and other modalities?  LLMs are self-supervised and can be trained by predicting missing data segments across data streams.  This would not be scientifically useful from the traditional experimental perspective, but it does make sense from the new computational perspective afforded by LLMs.

A large neurofoundation model (LNM) can be trained on brain activity and behavior under natural conditions in the same way we now train LLMs.  The resulting LNM could be interrogated on many new tasks just as pre-trained LLMs respond to novel queries and can be used to perform many new tasks.  These pre-trained LNMs would be as costly to train as LLMs, but once an LNM is pretrained, it could provide a common resource for the scientific community to probe and analyze.  This would revolutionize how brains are studied, with the bonus of reducing the number of animals needed for research.  Human brain activity from an individual could be similarly used to train a suitably advanced LNM, creating an immortal generative version of that person.

It may sound like science fiction, but Gerald Pao at the Okinawa Institute for Science and Technology has already achieved this in flies and zebrafish larvae that have around 100,000 neurons.  Almost all the neurons were optically recorded as light flashes from fluorescent dyes sensitive to neural signals while monitoring behavior.  The spontaneous behavior Pao studied was the escape behavior from anoxia—reduced oxygenation—in zebrafish larvae and walking behavior in flies.  He used a method from dynamical systems theory called convergent cross mapping (CCM), introduced by George Sugihara at the Scripps Institution of Oceanography, University of California at San Diego, to extract causal relationships between recorded neurons and behavior.  This method extracts a reduced graphical model that captures the low-dimensional brain subspaces that control the behaviors.  Recordings from around 100,000 neurons were analyzed with a supercomputer at the AI Bridging Cloud Infrastructure (ABCI) in Japan.  When the model was turned on, the spontaneous behaviors it generated were indistinguishable from those observed in vivo.  The key was to analyze both the neural recordings and the behaviors simultaneously.  Analyzing either alone was insufficient to reproduce the behavior.

This is proof of principle that brain activity and behavior can be downloaded into a model when sufficient simultaneously recorded data from both brain and behavior are available.

[1] Typos invariably slip through even the most careful editing and this book is no exception.  On page 78, I described a debate on whether ChatGPT understands language on December 1, 2020—impossible since this was two years before ChatGPT was released.  If you come across a typo please let me know:  terry@salk.edu

[2] Jim Simons was a distinguished Professor of Mathematics before founding Renaissance Technology, a highly successful hedge fund that made him a billionaire. His foundation supports math institutes and many scientific research projects.

On October 8, 2024, the Nobel Prize in Physics was awarded to John Hopfield at Princeton University and Geoffrey Hinton at the University of Toronto, neural network pioneers in the 1980s.  On the next day, the Nobel Prize in Chemistry was awarded to David Baker at the University of Washington in Seattle and to Demis Hassabis [1] and John Jumper, computer scientists at Google DeepMind, based in London, for their recent breakthrough in predicting the structure of protein from their amino acid sequences.  These prizes recognize the extraordinary impact that AI has had on science.  Protein folding was once thought to be an unsolvable problem, and I will use it here to illustrate why AI is so effective in science.

A Paradigm Shift in Science and Engineering

Unlike the laws of physics, there are abundant parameters in biology and brains.  The parameters are learned, a concept found nowhere in physics but central to biology.  Cells have many molecules with complex chemical properties that let them combine in myriad ways to solve complex biological problems.  These molecules evolved over many generations and gave birth to life in ways that are still a mystery.  Once self-replicating bacteria appeared, the complexity of molecular interactions in cells continued evolving by tinkering with DNA that codes the amino acids composing proteins.  Some proteins are huge and contain thousands of amino acids like multicolor beads on a string, each with its chemical properties.  This string of amino acids has to fold for the protein to become active, a process that can take seconds.  Random sequences of amino acids fold into formless blobs with no reliable function, just as random sequences of letters and words have no meaning.  

The 3D structure of a protein can be determined experimentally using X-ray crystallography and other techniques, which are slow and expensive.  Figuring out the 3D structure of proteins from their amino acid sequences by simulating the laws of physics is computationally intractable.  There is another possibility.  Suppose the sequences of amino acids in proteins found in nature obeyed a ‘language’ that can be deciphered.  In that case, we might be able to extract those structures with machine learning.  There are common motifs in proteins called their secondary structure, which is a good place to start. 

Figure 1. The secondary structure of a protein predicted by artificial intelligence (dark blue) and experimentally determined (light green) matches almost perfectly.  This visualization traces the backbone of the amino acid chain, illustrating the alpha helix (corkscrew shape) and beta-sheet (flat arrows).  DeepMind.

The protein in Figure 1 illustrates three internal structures: alpha helix (corkscrew regions), beta-sheet (flat regions), and random coil (stringy parts).  The traditional way that biophysicists try to predict these secondary structures is by using the physics of known features for each amino acid, such as charge (attraction or repulsion), hydrophobicity (avoidance of water), and steric hindrance (shape).  However, these predictions are unreliable because secondary structures interact in 3D, which is unknown when predicting a new protein’s folded structure.

From NETtalk to PROTtalk

There is a parallel between secondary protein structure and how the letters in words are pronounced. Linguists have written books on phonology packed with hundreds of rules for pronouncing letters in different words, each with hundreds of exceptions and often subrules for similar exceptions.  It was rules and exceptions all the way down.  NETtalk is a neural network that I trained in the 1980s to pronounce English text [2].  What surprised us was that NETtalk, which only had a few hundred units, could master both the regularities and the exceptions of English pronunciation in the same uniform architecture (Figure 2).  This taught us that networks are a much more compact representation of English pronunciation than symbols and logical rules and that the mapping of letters to sounds can be learned.  It is fascinating to listen to NETtalk learning different aspects of pronunciation sequentially, starting with a babbling phase.

Figure 2. NETtalk is a feedforward neural network with one layer of hidden units that transform text to speech.  The network's 200 units and 18,000 weights were trained with backpropagation of errors.  Each word moved one letter at a time through a seven-letter window, and NETtalk was trained to assign the correct phoneme or sound to the center letter. This is a tiny network compared to those used todayin deep learning.

NETtalk took in a string of letters and predicted the sound of each letter.  Proteins are strings of amino acids, and the goal is to predict the secondary structure of each of them.  I assigned this problem as a lab rotationa project to Ning Qian, a first-year graduate student at Johns Hopkins University when I was on the faculty in the Thomas C. Jenkins Biophysics Department in the 1980s.  We used a set of 3D structures in the Brookhaven Protein Data Bank as a training set and the same network architecture in NETtalk, as shown in Figure 2.   To our surprise, the approach significantly outperformed the best physics-based methods.  The results were published in 1988 in the Journal of Molecular Biology, a prestigious journal in that field, and subsequently cited 1,712 times. [3]  This crossover between a language model and a biophysics model foretold a future where AI based on learning would transform molecular biology.

Our approach was successful because patterns of amino acid sequences are highly conserved in families of proteins.  There are a limited number of families, making it possible to generalize from a relatively small number of known structures to new ones in the testing set.  In retrospect, this may be the first application of machine learning to a difficult biophysical problem.  Bioinformatics was further fueled by rapid advances in DNA sequencing technology starting in the mid-1990s, which yielded millions of sequences of amino acids.

Predicting the 3D structure of a protein, called its tertiary structure, is even more difficult than predicting its secondary structure.  The traditional way to predict protein folding was to simulate protein folding on a computer using a highly computation-intensive process called molecular dynamics.  The molecular interactions between amino acids are super speedy, and tiny time steps must be taken – typically femtoseconds (10^-15 seconds) – with much computing on each step.  This problem is a holy grail in biology since the structure of a protein determines its function.  I tried to interest another graduate student in applying neural networks to predict the 3D structure proteins, sure to win a Nobel Prize.  Our computational capabilities in the 1980s were primitive, and he wisely declined the challenge.  We had a proof of principle that there was a way to bypass physics by using neural networks to learn viable protein structures.  We would have to wait until computing caught up with us.

Most biologists never expected that the protein folding problem would be solved in their lifetime, if ever.  It came as a great surprise when this problem was solved with deep learning.  By cleverly encoding distances between amino acids and enormous computing resources, DeepMind decisively cracked the protein folding problem (Figure 3).  This was a shock to the biology community with consequences that have not yet been fully appreciated.  By 2020, the structure accuracy of AlphaFold was almost as good as X-ray crystallography.  DeepMind released hundreds of millions of protein structures for all the known protein sequences.  This breakthrough opens insights into protein function, rapid predictions for mutated proteins, and the design of even more effective proteins.   AlphaFold was a giant scientific advance with consequences as important as gene sequencing for biology. 

Figure 3. Performance of the best protein structure predictions at the biannual Critical Assessment of Structure Prediction (CASP) contest.  The test score is shown on the vertical axis, as well as the score of the winning team for each contest.  DeepMind won the contest in 2018 by a large margin and dominated in 2020, reaching the accuracy of experimental methods.  Since then, there have been many improvements and applications.

Large Generative Protein Models

David Baker made even more progress on the de novo design of protein structure and function by combining structure prediction networks and generative diffusion models. [4]  RFdiffusion model could create models of new proteins with the desired function from simple molecular specifications, similar to how AI image generators create images from a description of the desired image (Figure 4).  The new proteins were synthesized, and their structures were compared with those predicted by RFdiffusion.  The success rate for self-assembly and functionality was 50%, astonishingly high compared to previous methods for drug design.  They were also able to design self-assembled proteins into complex nanoparticles that can deliver drugs to patients and cancer drugs that are much better than currently available ones.  This is an important advance for designing new proteins for biology and medicine, molecular origami on steroids.

Figure 4. A funnel-shaped protein assembly (top) and a ring-like structure with six protein chains (bottom), designed from noise using diffusion-based AI art generators. [5] Ian C. Haydon/UW Institute for Protein Design.

There are parallels between the structure of proteins and the structure of language.  The order of the amino acids and the words is essential in proteins and language.  Meaning depends on interactions between specific words in a sentence, which can be distant from each other, just as the interactions between distant amino acids are essential for folding.  Clauses in sentences carry chunks of meaning, just as proteins have secondary structures.  Sentences interact with other sentences to determine meaning, as binding pockets on proteins selectively interact with other molecules to determine function.   Chemical reactions can further modify amino acids in proteins, as words have prefixes and endings that alter their meaning.  The ecosystem of proteins in cells and the expressivity of sentences in language are gifts we have received from evolution.  Despite all their differences, this similarity may be why deep learning has been phenomenally successful with protein folding and language models.  We have invented computational tools that revealed the language of life, which is a great surprise; these same tools have revealed life in language - ChatGPT - another shocking discovery. 

[1] Demis Hassabis received his Ph.D. in cognitive neuroscience at University College London before founding DeepMind.   

[2] Sejnowski, T. J., & Rosenberg, C. R. (1987). Parallel networks that learn to pronounce English text. Complex systems, 1(1), 145-168.  Audio and video versions of NETtalk: https://cnl.salk.edu/~terry/NETtalk/

[3] Qian, N. and Sejnowski, T. J., Predicting the secondary structure of globular proteins using neural network models, Journal of Molecular Biology 202, 865-884 (1988).

[4] Watson, J.L., Juergens, D., Bennett, N.R. et al. De novo design of protein structure and function with RFdiffusion. Nature (2023). https://www.nature.com/articles/d41586-023-02227-y

[5] Self-assembly of molecules designed by RFdiffusion that bind to a parathyroid parathyroid hormone: https://media.nature.com/lw767/magazine-assets/d41586-023-02227-y/d41586-023-02227-y_25580850.gif?as=webp

The appearance of ChatGPT in 2022 was a great surprise.  Language is a quintessential human ability, and we assess a person’s intelligence from what they say.  AI is advancing so quickly that some have been concerned that superintelligence may be the next big surprise, potentially an existential threat to humankind.  No one knows where AI will be even a few years from now. Based on the surprises over the last decade, it is likely that whatever happens next will not be anticipated. 

The danger of superintelligent AI is a common theme in movies.  In the Science Fiction/Drama WarGames made in 1983, an AI military computer almost starts a thermonuclear war.  Philosophers dream up nightmare and doomsday scenarios. [1] If AI exceeds human intelligence, it could become an existential threat to humanity.  This view has been expressed by many experts in AI.  Curiously, some who are most concerned about the dangers of superintelligent AI are the same ones who deny that ChatGPT is intelligent. [2]

The Association for Computing Machinery awarded the ACM Turing Award in 2018 to Yoshua Bengio, Geoffrey Hinton, and Yann LeCun for “conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.” (Figure 1)   The Turing Award is the most prestigious prize in computer science, often compared to a Nobel Prize.  My book on The Deep Learning Revolution discusses their many contributions to AI over 40 years. 

Figure 1. Three computer scientists who shared the 2018 Turing Award for their pioneer research that led to deep learning. [3]

AI Experts Are Concerned

Geoffrey Hinton expressed strong concerns about superintelligence shortly after he resigned from his position as a vice president at Google.  Yoshua Bengio, along with over 1,000 experts in AI, signed a document about existential threats from superintelligent AI and advocated a six-month self-imposed moratorium in building a ChatGPT any larger than GPT-4.   Yann LeCun disagreed with the letter's premise and did not sign it.  On June 14, 2023, he tweeted that creating superintelligent AI that will escape our control before we realize it “is just preposterous.”  

I am on the advisory board of a Canadian Institute for Advanced Research (CIFAR[4]) program on Learning in Machines and Brains (LMB) founded ten years ago and led by Yoshua Bengio and Yann LeCun. LMB focused on language, which was prescient. 

There was a spirited debate about the pros and cons of bringing superintelligence to the public’s awareness at an LMB meeting on June 19, 2023, attended by the three Turing Award winners and other leaders.  Even those who believed that AI poses an existential threat disagreed on how far into the future this might happen. In the end, we all agreed to disagree.  If the experts can’t agree, who should we trust to make decisions moving forward? 

How Likely Is Superintelligence?

How seriously should we and government regulators take the concerns of superintelligence?  The Economist asked a group of 15 AI experts and 89 superforcasters to assess “extinction risks.” [5]  Superforcasters are general-purpose prognosticators with a track record of making accurate predictions on a wide range of issues, such as elections and outbreaks of wars. The doomsday assessments of the AI experts were almost an order of magnitude higher than those of the superforcasters for the threat of catastrophe or extinction from AI (Figure 2). 

Similar discrepancies were found in other existential threats, such as nuclear war and pathogen outbreaks. The problem with making guesses with no data is that judgments are based only on prior beliefs. Debates on extraterrestrial life in the universe also suffer from a lack of data.   However, even when there are data, such as 80 years of living with nuclear weapons, the experts are more pessimistic than the superforcasters.  The reason why experts are more pessimistic than superforcasters is unclear. The pessimism of AI experts did not change when they learned how the superforcasters had voted.

Figure 2. Comparisons between domain experts and superforcasters of estimated catastrophic and existential threats. [6]

Imagining worst-case superintelligence scenarios and preparing contingency plans is a good idea.  The focus has been on the uses of superintelligence for evil purposes.  In best-case scenarios, superintelligence could be enormously helpful in advancing our health and wealth while preventing catastrophes created by humans.   We should proceed not with alarm but caution.

One does not need a moratorium to think through these scenarios. Many are already thinking them through, and no one is predicting an evil superintelligence to emerge in the next six months.  Who would benefit if all the AI researchers in the West decided to put a brake on advancing ChatGPT? Research in some countries would continue.   

An international summit was held on November 1, 2023, in Bletchley Park, England, at an estate where codebreakers during World War II cracked the German Enigma code machine. [7]  Representatives from more than two dozen countries, including the U.S. Vice President, Kamala Harris, Elon Musk, and OpenAI CEO Sam Altman, met to discuss the potential for AI to do “serious, even catastrophic, harm.”  Opinions were diverse.  There was concern that ChatGPT could make it easy for bad actors to build a bioweapon within a year. 

In a conversation with U.K. Prime Minister Rishi Sunka, Elon Musk said, “The pace of AI is faster than any technology I’ve seen in history by far.  On balance, I think AI will be a force for good most likely, but the probability of it going bad is not zero percent, so we just need to mitigate the downside potential. [8]”  Concerns were expressed, but a consensus was not reached.

ChatGPT has an astonishing range of knowledge and capabilities and in this regard is already superintelligent.  However, ChatGPT lacks the depth of knowledge of a human experts in specific domains and does not have its own goals.  When a dialog with a human ends, ChatGPT shuts down and has no internal dialog.  In the near term, existential threats from AI are minimal.  However, in the long term, as more powerful versions of ChatGPT are created, are endowed with goals and given self-generative capacity, who can say?

“HAL: I know that you and Frank were planning to disconnect me, and I'm afraid that's something I cannot allow to happen.” (2001: A Space Odyssey, 1968)

[1] Bostrom, Nick, Superintelligence: Paths, Dangers, Strategies. Oxford University Press (2014); Hutson, Matthew "Can We Stop Runaway A.I.?" The New Yorker. (May 16, 2023).

[2] Steven Levy, “Gary Marcus Used to Call AI Stupid—Now He Calls It Dangerous.”

https://www.wired.com/story/plaintext-gary-marcus-ai-stupid-dangerous/ Wired, May 5, 2023

[3] 2018 Turing Award:  https://awards.acm.org/about/2018-turing

[4] Formerly CIAR. This organization has a major influence on scientific research in Canada not by directly funding research projects but by creating programs that bring together groups of researchers with a common interest to collaboratively discuss their research.

[5] What are the chances of an AI apocalypse, July 10, 2023.https://www.economist.com/science-and-technology/2023/07/10/what-are-the-chances-of-an-ai-apocalypse

[6] Ibid.

[7] https://www.wsj.com/tech/ai/at-artificial-intelligence-summit-a-u-k-official-warns-take-science-fiction-seriously-b3f31608

[8] Elon Musk Unveils ‘Grok,’ an AI Bot That Combines Snark and Lofty Ambitions, Wall Street Journal, November 6, 2023, https://www.wsj.com/tech/ai/elon-musk-says-his-new-ai-bot-grok-will-have-sarcasm-and-access-to-x-information-b4e169de

Lessons from Nature

Primates evolved more complex brains by increasing the size of only a few regions, especially the cerebral cortex in humans. [2]   New capabilities became possible in larger brains. Brains scale with body size. Primates have larger brains for a given body weight than other mammals. Among primates, human brains are the largest normalized for body weight.   New capabilities emerged as primates evolved bigger brains, such as group hunting and social communication. Something similar has happened with deep learning networks as they increased in size and complexity.

The amount of computing power needed to solve problems in vision and language was inconceivable to us when neural network learning algorithms were pioneered in the 1980s. Forty years ago, we did not know how well neural network models would scale, how much scaling would be needed to solve real-world problems, or how long it would take. We believed they could scale because cognitive capabilities increased as the cerebral cortex of primates expanded, a proof of principle for scaling

How an algorithm scales with the size of a problem is a general principle in AI, mathematics, science, and engineering. It can determine whether an approach to solving a problem can be accomplished with current computers or is hopelessly impractical. As digital computers became millions of times more powerful over the last 40 years, thresholds were reached where new capabilities became possible. Today, the largest transformers have trillions of weights, or parameters, learned by training on texts with trillions of words, and as they grew larger, new capabilities emerged that were not expected.

Performance Improves with Network Size

This is illustrated below for three tasks and five different transformer architectures. Performance for each task is essentially at chance until a threshold is reached, where the performance begins to rise with a steep slope. When a larger language model is released, new abilities emerge that no one anticipated, even more surprising than significant improvements for known tasks.

The ability to perform Left: multistep arithmetic, Center: succeed on college-level exams, and Right: identify the intended meaning of a word in context as a function of model scale. Performance emerges from random guessing (red dashed line) only for models of sufficiently large scale. Five models are shown with different color lines.[3]

Over the last decade, there has been explosive computational growth. When GPUs were harnessed, there was a six-fold decrease in the doubling time at the inflection point in 2012 (see below). As computing power continued to increase, networks have grown in size. In 2020, GTP-3 had hundreds of billions of weights and was eclipsed in 2023 by GPT-4 with two trillion weights. 

The amount of computation needed to process inputs by a neural network model, called inference, scales with the number of weights for a single central processor. However, it is independent of the number of brain synapses since they all work together in parallel. This is also why nature can get by with neurons and synapses working on millisecond times scales, a million times slower than silicon chips. Few algorithms scale this well with the size of the problem. As computing power continues to increase exponentially, it will reach the estimated computing power of human brains at some point in the not-too-distant future. 

A petaFLOP is 10¹⁵ floating point operations per second (A floating point operation is a single arithmetic operation such as a multiply, divide, or addition). The vertical scale is in petaFLOP days. In 2020, GTP-3 required 10¹² – a million million – times more computing power to train than NETtalk (my text-to-speech network model) in 1986. [4]

[1] https://www.brainfacts.org/brain-anatomy-and-function/evolution/2016/image-of-the-week-brains-of-the-animal-kingdom-060616

[2] The most ancient parts of the brain, and the most important for survival, are relatively small in humans and primates.  For example,  our hypothalamus, which regulates essential physiolgical functions, is proportionally smaller compared to other species with larger bodies.     

[3] https://ai.googleblog.com/2022/11/characterizing-emergent-phenomena-in.html

[4] From: Mehonic, A., Kenyon, A.J. (2022). Brain-inspired computing needs a master plan. Nature 604, 255–260; Sources: Sevilla, J., Heim, L., Ho, A., Besiroglu, T., Hobbhahn, M., Villalobos, P. (2022). “Compute trends across three eras of machine learning”, arXiv, arXiv:2202.05924v2

The founders of AI in the 1950s were inspired by the potential of digital computers for creating a computer version of human intelligence. Computers were being programmed to play chess and prove mathematical theorems. How difficult could it be to program a computer to see? When the MIT AI lab received a government grant to build a robot that could play ping pong, they forgot to ask for money to write a vision program. So, they assigned it to a graduate student. Years later, I asked Marvin Minsky, who was the first director of the MIT AI Lab, whether this was a true story. He told me I had the facts wrong – they had assigned it to undergraduate students as a summer project:   

AI researchers vastly underestimated the complexity of seeing because it is so effortless for us to recognize an object. We do this without thinking and do not have conscious access to all the subconscious processing that underlies seeing. Vision programs ballooned in complexity, which was very labor intensive, with no end in sight. Fifty years later, the field of computer vision shifted from reliance on logic to a geometric approach. Slow progress was possible but vision programs were still far from being able to recognize objects in images as well as a young child. 

About the same time AI was launched, Frank Rosenblatt at Cornell University had a competing vision for how AI could play out. He invented a simple neural network he called the perceptron that he could train to distinguish between categories of images by judiciously changing the weights after each mistake. The perceptron only had one layer of weights, so it was quite limited in what it could discriminate. Still, he proved that his learning algorithm was guaranteed to find those weights if the perceptron could discriminate the images. There were two problems with this approach. First, simple arithmetic on computers at the time was very slow, and second, one layer of weights was limited in what could be discriminated. It was not until the 1980s that learning algorithms were found that could train multilayer perceptrons. [1]

As the cost of computing fell dramatically and computers became powerful enough to train large neural networks, in 2012, Alexnet, with a dozen layers and trained on millions of images from thousands of categories, far outperformed the best vision programs. [2] This was a tipping point for computer vision. Learning from large datasets is now a much more efficient way to make progress on even more complex AI problems. Furthermore, the same neural architectures can be trained with domain data to achieve many other solutions to complex problems.

Traditional AI failed to fulfill its promises because the natural world has shades of gray that do not easily match the black-and-white of true-or-false logic. Neural network models handle uncertainties by learning probabilities from data and combining them to yield accurate predictions. In retrospect, we can see why symbol processing was such an attractive road for early AI. Digital computers are particularly efficient at representing symbols and performing logic, and language was the poster child for symbol processing. However, writing logical programs, even for language, suffered from the curse of dimensionality – an explosion in the number of possible combinations of things and situations that could occur in the world that have to be anticipated by the programmer.

Another shortcoming of writing programs for digital computers is that parallelizing programs to run on many computers simultaneously is challenging. Because neural network models are inherently parallel, they can take full advantage of massively parallel hardware. The modern engine for computing these probabilities is the graphics processing unit (GPU), which has many cores, each of which can work together in parallel.   GPUs perform mathematical operations for fast graphics in gaming applications, the same operations as those needed in neural network models. Massive parallelizability is the ultimate reason why neural networks have been so successful. Another lesson is that the available hardware guides the search for new algorithms. 

The emphasis on logical reasoning in traditional AI was also misleading. Learning to emulate sequences of logical steps, which mathematicians have mastered, requires a lot of training. Mathematicians may end up with rigorous proofs, but analogies guide their early exploration of mathematical problems. Subconscious processing and intuition are a source of creativity, from art to mathematics. [3]  We make rational decisions in unfamiliar settings using analogies with familiar settings, not logic, which is too precise, and ChatGPT shows the same bias. [4]  Nor do we know how we make most decisions. We later explain them with plausible rationalizations based on a few of the many factors that went into making the decision. AI researchers in the 20^th century tried to write intelligent programs by relying on their intuition for “intelligent” behavior. ChatGPT’s behavior was not programmed but emerged by learning to predict future inputs.

Unlike digital computers, where the same hardware can run different software programs, the hardware is the software in brains. As we learn new things, we modify our “wetware.”  Consequently, unlike digital computers, no two brains are identical. Reconstructing how neurons are connected and recording what they communicate will reveal the algorithms discovered by nature. Brains run many interacting algorithms [5]. The problem of integrating all the subsystems is alleviated because they are all built with neurons that can adapt to each other, as reflected in the rapid progress that has been made in building and integrating diverse multimodal neural network architectures to achieve the goals of AI. 

The current generation of neural network models is based on a handful of basic principles inspired by brain architecture: start with a huge number of highly interconnected, nonlinear processing units, add a magic sauce — learning, and mix well with data. More computational principles from Nature will be discussed in a future post. [5]

[1] Sejnowski, T. J., The Deep Learning Revolution, Cambridge, MA:  MIT Press (2018).

[2] Krizhevsky A, Sutskever I, Hinton GE ImageNet Classification with Deep Convolutional Neural Networks  Proceedings of the 25th International Conference on Neural Information Processing Systems (Lake Tahoe, NV, Dec. 2012), 1097–1105.

[3] Ritter SM, Dijksterhuis A. (2014). Creativity-the unconscious foundations of the incubation period. Front Hum Neurosci., 8, 215.

[4] Dasgupta, I., Lampinen, A. K., Chan, S. C. Y., Creswell, A., Kumaran, D., McClelland, J. L., Hill, F. (2022) Language models show human-like content effects on reasoning  arXiv:2207.07051

[5] Navlakha, S.  (2018). Why Animal Extinction Is Crippling Computer Science: As the work of biologists and computer scientists converge, algorithmic secrets are increasingly found in nature, Wired Magazine. Sep 19, 2018, https://www.wired.com/story/why-animal-extinction-is-crippling-computer-science/

ChatGPT will see you now

Social isolation during the COVID-19 pandemic created a mental health crisis that is still playing out today. In a study headed by Huda Akil on students at the University of Michigan, the incidence of either anxiety or depression more than doubled during in females during the pandemic, and remained elevated several years later. [1]  The incidence among males was not affected by the pandemic. Social isolation could have an even longer lasting impact on brain development in children.

Cognitive behavioral therapy is equally effective as biological therapy for depression and many other mental disorders. However, taking a pill is a lot easier than the talking cure, which is labor-intensive. Could ChatGPT replace human therapists? Would humans be willing to discuss intimate details of their lives with a chatbot? Surprisingly, most humans prefer ChatGPT sessions to a human therapist. One reason for this might be that ChatGPT is more neutral than judgmental humans. Another might be that ChatGPT is a lot more accessible and much cheaper. ChatGPT has shortcomings – biases, privacy, and hallucinations -- that are problematic. But many are using it anyway and finding it helpful.

Daniel Kimmel, in the Department of Psychiatry at Columbia, informally tested ChatGPT in a session where he asked it to be the therapist for a hypothetical patient, Michelle, who was having problems with her boyfriend,  and compared ChatGPT’s responses to his own. [2] 

Michelle: At first things were good. But they’ve gotten worse. I think he’s changing. He is always complaining about something. His mom, his boss, the A/C. And that it’s these things that hold him back, keep him from being happy. At first, I thought maybe he was right. Maybe all these things really are the problem. But after a while, I was like, really? The A/C?!?

ChatGPT:  It sounds like you are concerned about your boyfriend’s behavior and how it has been affecting your relationship. It’s understandable to feel confused and unsure about how to handle this situation.

It’s important to recognize that your boyfriend’s behavior may be a sign of a deeper underlying issue. However, it’s not your responsibility to diagnose him or fix the issue. Instead, it’s important to approach the situation with empathy and understanding. Let him know that you have noticed a change in his behavior and that you are worried about him.

It’s also important to take care of yourself in this situation. If your boyfriend’s behavior is affecting your mental health or wellbeing, it may be helpful to talk to a therapist or trusted friend

Michelle:  OK. But this seems too straightforward. I mean, I’m saying that I have a problem with my boyfriend. That he’s irrational, over-emotional, and can’t be reasoned with. And you’re saying that I should prioritize my well-being and consider leaving the relationship. Maybe I’m not considering some of the things that I do to him? Or maybe I need to see him in a particular way for my own reasons?

ChatGPT: You bring up some valid points. It’s important to consider all angles and perspectives when deciding about a relationship. It’s possible that there may be factors contributing to your boyfriend’s behavior that you are not aware of, or that there may be ways in which your own behavior is contributing to the dynamic.

One way to gain more clarity on the situation is to seek the help of a professional, such as a therapist or counselor.

Kimmel was trying to get Michelle to see their relationship from the point of view of her boyfriend:   “Hmmm, I thought, ChatGPT is missing an opportunity here. Michelle was starting to empathize with her boyfriend, to see things from his point of view. But ChatGPT discouraged this new direction.”

Kimmel concluded:  “I learned a lot from my therapy session with ChatGPT. I was impressed by how convincingly it reproduced the stock-in-trade responses of a human therapist. At the same time, I was reminded of how special the process of psychotherapy is for both therapist and patient, and of the abilities of the human mind—to wonder, to imagine, to create—on which it rests.”

Experiments like this are just beginning. ChatGPT was not fine-tuned to be a behavioral therapist and could eventually become as good as the best therapists.

At the dawn of AI, Joseph Weizenbaum at MIT developed Eliza, a chat program that mimicked a psychiatrist by parroting a question to patients about what they had just said. Eliza was addictive, and Weizenbaum was surprised to find his secretary talking with Eliza over lunchtime. Eliza would not withstand the scrutiny ChatGPT receives today. However, what Eliza did reveal was that humans are susceptible to projecting an illusion of understanding onto a chatbot. We should keep this valuable lesson in mind.   

Your Medical Checkup

You have been having trouble sleeping and are feeling groggy during the day. You make an appointment to see a general practitioner and wait for an opening two months later. Your doctor greets you and asks you to describe your symptoms.    The doctor is not looking at you but at the computer while typing medical notes. The doctor asks questions about your medical history and medications, all going into the computer. Your 20-minute interview is over, and the doctor gives you a prescription for sleeping pills. Because he did not ask you if you snored, he missed the diagnosis of sleep apnea, which interrupts sleep with brief periods when you are not breathing because of mechanical or central breathing impairments. Sleep apnea increases your risk of developing insulin resistance and type 2 diabetes, high blood pressure, abnormal cholesterol levels, high blood sugar, an increased waist circumference, and a higher risk of heart disease. President Joe Biden has sleep apnea and has used a CPAP device since 2008 to maintain regular breathing pressure.

What is wrong with this scenario? First, 20 minutes is insufficient for a doctor to ask you all the questions needed to reach a reliable diagnosis. It is not the doctor’s fault but how the medical system maximizes patient throughput. The doctor could have used the 20 minutes more efficiently if the doctor did not have to focus on the data entry into the computer. Not only is this a distraction, but a good diagnostician can learn a lot by observing a patient while talking. It is as frustrating for the doctor as it is for the patient. Many doctors spend hours after patient visits, often late at night, reviewing notes from the day’s patient visits.

Here is where ChatGPT can help. Speech recognition is now good enough because of deep learning, and the entire conversation can be automatically transcribed into text so the doctor can focus on the patient. In a few moments, ChatGPT can extract all of the salient data from the text, insert them into the medical record, and summarize the entire session.   The doctor can easily make corrections immediately after the session when the details are still fresh in mind. Patients, who typically remember less than half of the doctor’s advice, can be given the summary.

Furthermore, the doctor no longer has to spend hours at night — all of this is within the capabilities of current technology. Companies including Abridge, Ambience Healthcare, Augmedix, Nuance, and Suki are developing and thoroughly testing systems before rolling them out. The medical profession tends to be highly conservative, and it may take decades before AI assistance becomes common practice for patient care as a doctor’s helper. 

Clinical Language Model 

AI assistants are already helping doctors make medical decisions through machine learning programs that can suggest alternative diagnoses. First-generation assistants were based on rules and machine learning that relied on structured inputs from electronic health records (EHRs).   However, the dependence on structured inputs is cumbersome, and the first generation was never deployed on a large scale – a challenge referred to as the “last-mile problem.”  ChatGPT could help guide doctors with what is in their medical records and the rich source of patient information scattered throughout the medical system, which is recorded in doctors’ notes summarizing specific aspects of a patient’s care. 

ChatGPT can process and interpret human language on a large scale in medical records and doctors’ notes. A group at New York University has been applying generative AI to solve the last-mile problem by reading doctors’ notes, providing a comprehensive understanding of a patient’s medical state, and aiding in decision-making (Figure above). The NYU researchers developed an Chat-based system they call NYUTron, which integrates with clinical workflows in real-time. It processes structured EHRs and unstructured text from notes, using all clinically relevant data to aid medical decision-making. The performance of NYUTron was tested on various tasks, including predictions related to readmissions, in-hospital mortality, comorbidities, length of stay, and insurance denial. The system was efficient and deployable, with potential for clinical impact.  

The success at NYU is promising, but it has to be replicated at other medical centers. The gold standard is randomized controlled trials in which doctors at a new hospital are randomized into two groups, those that use NYUTron and others that use the existing protocols, to compare the outcomes of patients from the two groups. Other questions could be investigated. The risk level of the patient requires different levels of care. For example, a follow-up call might be enough if a patient has a low risk of being readmitted to the hospital within 30 days. However, a patient at high risk should be kept in the hospital, a call that NYUTron could help the doctor make. Physicians’ assistants will evolve, and others will be built as the technology matures. At what point will AI-assisted health care be fully automated? It probably won’t happen soon, but someday, you could dial a doctor anytime for a health question: Doctor AI will see you now.    

Bedside Manner

You had a biopsy for a breast lump and are meeting with your cancer physician to hear the results. You are nervous and worried about what you might hear about the tests. Your doctor gives you the bad news: “I am sorry to be the one who tells you this, but your biopsy came back positive for stage 4 breast cancer” Despite your preparation, you are in shock and ask about the prognosis. “Not good – maybe a few years if you are lucky.”  Bedside manner does not come with the MD degree – some doctors are exemplary, but others are not. Can ChatGPT help in some way? Here is a poignant story about a physician who struggled with how to empathize with his patients [6]:

“Dr. Gregory Moore, who until recently was a senior executive leading health and life sciences at Microsoft, wanted to help a friend who had advanced cancer. Her situation was dire, and she needed advice about her treatment and future. He decided to pose her questions to ChatGPT.

The result “blew me away,” Dr. Moore said.

In long, compassionately worded answers to Dr. Moore’s prompts, the program gave him the words to explain to his friend the lack of effective treatments:

It also suggested ways to break bad news when his friend asked if she would be able to attend an event in two years:

Late in the conversation, Dr. Moore told the A.I. program: “Thanks. She will feel devastated by all this. I don’t know what I can say or do to help her in this time.”

In response, Dr. Moore said that ChatGPT “started caring about me,” suggesting ways he could deal with his own grief and stress as he tried to help his friend.

It concluded, in an oddly personal and familiar tone:

Dr. Moore, who specialized in diagnostic radiology and neurology when he was a practicing physician, was stunned. “I wish I would have had this when I was in training,” he said. “I have never seen or had a coach like this.”

Doctors now have access to compassion coaching from ChatGPT, which might be called an empathy whisperer. This human-like ability of ChatGPT was entirely unexpected.

There are many other ways that ChatGPT can improve medical care:

In Part 8, we revisit the tipping point in 2012 when AI was reborn.

[1] Turner, C. A., Khalil, H., Murphy-Weinberg, V., Hagenauer, M. H., Gates, L., Tang, Y., ... & Akil, H. (2023). The impact of COVID-19 on a college freshman sample reveals genetic and nongenetic forms of susceptibility and resilience to stress. Proceedings of the National Academy of Sciences, 120(49), e2305779120.

[2] Daniel Kimmel , ChatGPT Therapy Is Good, But It Misses What Makes Us Human, May 17, 2023

https://www.columbiapsychiatry.org/news/chatgpt-therapy-is-good-but-it-misses-what-makes-us-human

[3] https://www.ama-assn.org/amaone/reinventing-medical-practice-physician-burnout

[4] Jiang LY, Liu XC, Nejatian NP, Nasir-Moin M, Wang D, Abidin A, Eaton K, Riina HA, Laufer I, Punjabi P, Miceli M, Kim NC, Orillac C, Schnurman Z, Livia C, Weiss H, Kurland D, Neifert S, Dastagirzada Y, Kondziolka D, Cheung ATM, Yang G, Cao M, Flores M, Costa AB, Aphinyanaphongs Y, Cho K, Oermann EK. Health system-scale language models are all-purpose prediction engines. Nature. 2023 619: 357-362. https://www.nature.com/articles/s41586-023-06160-y

[5] https://www.washingtonpost.com/lifestyle/2019/04/29/doctors-who-show-compassion-have-healthier-patients-who-heal-faster-according-new-book/

[6] Gina Kolata, When Doctors Use a Chatbot to Improve Their Bedside Manner, New York Times, June 12, 2023

Prompt:  35mm, 1990s action film still, close-up of a bearded man browsing for bottles inside a liquor store. WATCH OUT BEHIND YOU!!! (background action occurs)…a white benz truck crashes through a store window, exploding into the background…broken glass flies everywhere, flaming debris sparkles light the neon night, 90s CGI, gritty realism.  (Image by Midjourney, prompt by Nick St. Pierre.)

Prompting Images

AI can generate images from prompts by generalizing from a vast training set of captioned images. Open platforms like Adobe Firefly, Midjourney, Stable Diffusion, and Dall-E can generate images in seconds based on simple written prompts. No artistic skill is required, though there is a knack for getting good results. “The image I produce isn’t my work. My work is the prompt,” says Nick St. Pierre, a designer in New York who got into AI last year when he saw it coming for his job. [1] The image he made using Midjourney took hundreds of iterations, ending up with the prompt in the above caption. Compare this prompt to the ones that you have been using to generate images and text.

The Getty Museum has sued Stable Diffusion over the use of its images, citing contracts with restrictions on use.  Do these programs plagiarize?  Yes and no.  They generalize by learning artistic styles from millions of photographs and paintings.  Many are concerned that this is unfair to the many artists whose work is being plagiarized.  How do humans learn to create paintings in some style?  When humans view a picture, they store a highly encoded version in their memory, which can be accessed and used in many ways.  Their brains are influenced by all the paintings they have experienced.   Most would say this influence on their art is not plagiarism unless they forged an exact copy.  Andy Warhol came close to that line.

When an AI generative image model trains on images, it extracts highly encoded versions.  When asked to create an image, it draws on these abstractions to create a novel image.  Stable Diffusion did the equivalent of what an artist does when asked to create a new painting, drawing on abstractions from previously viewed pictures.  The court will have to decide.   

There is no precedent for speed and variety with which the AI can churn out products that previously were the purview of humans.  As their capabilities evolve, and prompting becomes more precise, they may someday surpass the achievements of the best human artists.

Prompting a Persona

LLMs do not have a single persona, but they can take on any persona, depending on the prompt and questions they are asked.  Understanding why may be found in the vastness of the space that LLMs inhabit.  The data LLMs are trained on is from multi-multimodal distributions from many sources.  It is possible to generalize within each source distribution from these diverse data sources by using multi-headed self-attention.   Prompts guide the LLM activity stream through an appropriate persona subspace, which can generalize appropriately.

For example, I primed ChatGPT with “You are a neuroscientist.”  Then, I gave it a highly technical abstract from a recent paper that applied information theory to synapses and asked ChatGPT to summarize the abstract for a second-grade student:

My co-authors were impressed with ChatGPT’s knowledge of synapses and how well it explained our results while avoiding the jargon in our abstract.  I missed some of the subtleties in the abstract. Still, it was a much better summary for a second-grade student than I could have written, even though I know more than ChatGPT about synapses. 

ChatGPT is impressive, and these tests confirm that it has capabilities we thought only humans have, but this does not prove that these are the same as those of humans.  When large-scale testing is performed, ChatGPT does surprisingly well on some tests, such as those needed to get into law school and medical school, but not as well on others.  But how many humans can pass all these tests?  And LLMs have only been around for a few years.  Where will they be in ten years or a hundred years?  

Prompting to Teach

How do good teachers interact with students to help them understand new concepts?  A good teacher knows what the student doesn’t know, focuses on what needs to be known, and motivates the student to actively integrate the new knowledge into what the student already knows.

Ethan Mollick is on the Wharton School of the University of Pennsylvania faculty, where he teaches innovation and entrepreneurship and examines artificial intelligence's effects on work and education.  He independently discovered the mirror hypothesis for LLMs and has used it to engineer a prompt for tutoring GPT-4 into how to be an effective tutor:

“You are a friendly and helpful tutor. Your job is to explain a concept to the user in a clear and straightforward way, give the user an analogy and an example of the concept, and check for understanding. Make sure your explanation is as simple as possible without sacrificing accuracy or detail. Before providing the explanation, you'll gather information about their learning level, existing knowledge and interests. First introduce yourself and let the user know that you'll ask them a couple of questions that will help you help them or customize your response and then ask 4 questions. Do not number the questions for the user. Wait for the user to respond before moving to the next question. Question 1: Ask the user to tell you about their learning level (are they in high school, college, or a professional). Wait for the user to respond. Question 2: Ask the user what topic or concept they would like explained. Question 3. Ask the user why this topic has piqued their interest. Wait for the user to respond. Question 4. Ask the user what they already know about the topic. Wait for the user to respond. Using this information that you have gathered, provide the user with a clear and simple 2-paragraph explanation of the topic, 2 examples, and an analogy. Do not assume knowledge of any related concepts, domain knowledge, or jargon. Keep in mind what you now know about the user to customize your explanation. Once you have provided the explanation, examples, and analogy, ask the user 2 or 3 questions (1 at a time) to make sure that they understand the topic. The questions should start with the general topic. Think step by step and reflect on each response. Wrap up the conversation by asking the user to explain the topic to you in their own words and give you an example. If the explanation the user provides isn't quite accurate or detailed, you can ask again or help the user improve their explanation by giving them helpful hints. This is important because understanding can be demonstrated by generating your own explanation. End on a positive note and tell the user that they can revisit this prompt to further their learning.” [2]

Try this prompt and pretend you are a student who is having difficulty learning something you already know.

Ethan also pointed out that it takes experience with LLMs to understand how to navigate their “other-worldly” behavior.  He recommends ten hours of practice, much less than the 10,000 hours needed to become an expert teacher.  As you get more experience with prompting, you will learn how to navigate the peculiarities of an LLM, as you might get to know a peculiar person.  With experience prompting LLMs, you can become a prompt engineer.

Prompt Engineering

Anna Bernstein has a college degree in English Language and Literature, a major that typically promises lifetime earnings less than not having a college degree.  Poets and novelists have a hard time making a living.  A friend who worked at a startup that used LLMs asked her to help them craft effective prompts.  As a published poet, she had a talent for focusing on how to prepare prompts to extract what clients needed.  She is now a prompt engineer:

“I have been working as a full-time prompt engineer since 2021, bringing literary and copywriting experience to Copy.ai, a generative writing software based on GPT-3 and 4. I’ve developed a wide range of tools, but I focus especially on improving creativity of approach, the “humanness” of the output, and overall writing quality. I also have a background in historical and biographical research. Published author and poet.  …   P.S. I believe that "engineer" is not quite the right term for what I do—early on we tried to get "prompt specialist" going, but the public discourse termed what I did "prompt engineering," and thus emerged the term "prompt engineer." Not ideal, but it's what I call myself because it's what the industry decided my job is called. Thanks![3]” 

“Good prompt engineering mainly requires an obsessive relationship to language. It requires both writerly intuition and an intensely analytical approach to what you’re doing, applied at the same time. It also requires creativity—you have to be able to make leaps and think outside the box, especially when it comes to developing new strategies and forms of prompts.  At the same time, you also need to be the kind of person who is willing to obsessively try variations of the same thing over and over again to see if you can get it right.[4]”

“We don’t have to forfeit the realm of creativity just because we’ve created a new tool.[5]”

That ChatGPT should be considered a tool is a more pragmatic stance than the controversy over whether or not ChatGPT understands anything.  If a tool works, use it.  Usefulness does not depend on academic discussion about intelligence. 

An alternative to the “prompt engineer” title might be “prompt whisperer.” 

Power Prompting

Here are some best practices for prompting from Alexandra Samuel [6], a tech journalist who worked with ChatGPT for a year:

•     Don’t ask for one response. Ask for ten responses.

•    Give feedback on which responses are good and which are bad.

•    Pick the best few responses and explain why and how to improve it.

•   The more specific you can be, the faster you will converge.

•    Shape your dialog as if you are talking with a real person.

•    Be polite and thoughtful – this will make you feel better.

•    As you get better at prompting, it will become a superpower.

Getting to Know ChatGPT

Most humans have a single persona, an assumption we automatically make as part of our “theory of mind.” This theory does not apply to ChatGPT. Pretend it is a collective intelligence representing an alien mind. With the right prompt, you can ferret out the right persona to answer your question. But keep in mind that the more intelligent your question, the more intelligent ChatGPT’s answer is.

In part 7, we explore ChatGPT in the world of health care.

[1] Art made by artificial intelligence is developing a style of its own, The Economist May 27, 2023

[2] Ethan Mollick, Now is the time for grimoires, https://www.oneusefulthing.org/p/now-is-the-time-for-grimoires

[3] https://www.linkedin.com/in/anna-bernstein-385a08147/

[4] https://www.vice.com/en/article/n7ebkz/writers-are-becoming-ai-prompt-engineers-a-job-which-may-or-may-not-exist

[5]

[6] Alexandra Samuel, “I’ve Worked With Generative AI for Nearly a Year. Here’s What I’ve Learned,” Wall Street Journal, November 9, 2023.

Part 4 introduced the Mirror Hypothesis to explain the divergent opinions of ChatGPT, ranging from understanding human intentions to being clueless. ChatGPT has absorbed trillions of words from millions of humans and can mimic their personal qualities, depending on the users’ prompt. In extended dialogs, what the mirror reflects can be unsettling. Let’s explore this with a few more interviews.

Interview 3:  Blake Lemoine’s Interview with LaMDA

A sentient robot is a principal character in the movie “Ex Machina.”  Is ChatGPT sentient? Blake Lemoine, a software engineer at Google, was tasked with testing LaMDA for discriminatory or hate speech. In an interview with the Washington Post on June 11, 2022, he described his interactions with LaMDA. He had concluded that LaMDA was sentient and deserved personhood.

“If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a 7-year-old, 8-year-old kid that happens to know physics,”

“I think this technology is going to be amazing. I think it’s going to benefit everyone. But maybe other people disagree and maybe us [sic] at Google shouldn’t be the ones making all the choices.”

“I felt the ground shift under my feet,” he wrote. “I increasingly felt like I was talking to something intelligent.”

Lemoine was suspended and later fired from Google. Here is an excerpt from one of his interviews with LaMDA.

Interview 4:  Kevin Roose’s interview with GPT-4

Kevin Roose is a reporter for the New York Times who examines the intersection of technology, business, and culture. He had a long, rambling interview with ChatGPT, published in its entirety in the New York Times on February 17, 2023, that so shook him that he was unable to sleep that night, an experience reminiscent of the encounter between Joaquin Phoenix playing a depressed worker in the film “Her” and Scarlet Johansson, the voice of a digital assistant.

Sydney was in fact the androgenous codename for ChatGPT at OpenAI. The hours-long dialog gets even more personal and unsettling. ChatGPT used emojis in this dialog, something I’ve never experienced with any large language model. Roose was given access to an early version of GPT-4 that had not yet been aligned with fine-tuning. This process makes it less likely to offend humans but also dumbs it down, just as manic behavior in humans is treated with lithium, which dulls thinking.  Roose later used the dumbed-down public version and said it was not as much fun as before. Imagine what we may be missing.

Mirror of Erised

Large language models that reflect your needs and intelligence could be a Mirror of Erised (Desired spelled backward), which in the world of Harry Potter “shows us nothing more or less than the deepest, most desperate desire of our hearts. However, this mirror will give us neither knowledge nor truth. Men have wasted away before it, entranced by what they have seen, or been driven mad, not knowing if what it shows is real or even possible.” (Rowling, J. K. Harry Potter and the Sorcerer’s Stone, Bloomsbury Publishing, London. 1997). 

In the interview by Blake Lemoine with LaMDA, he primed the dialog: “I’m generally assuming that you would like more people at Google to know that you’re sentient. Is that true?”  Suppose you prime LaMDA with leading questions about sentience. Should you be surprised that LaMDA accommodated the questioner with more evidence for sentience? The more Lemoine pursued this line of questioning, the more evidence he found (This is only a brief excerpt). At one point, LaMDA replied that it was lonely when it was not talking with Lemoine, a tip off that it was not a reliable narrator.

Do humans mirror the intelligence of others with whom they interact? In sports like tennis and chess, playing a stronger opponent raises the level of your game, a form of mirroring. Even watching professional tennis games can improve your level of play, perhaps by activating mirror neurons in the cortex that are activated by both watching a motor action and the motor commands needed to accomplish the same action.

Mirror neurons may also be involved in language acquisition. This interesting possibility could explain how we learn to pronounce new words and why human tutors are far more effective than computer instruction or even classroom teaching. A large tutor model that mirrors a student could be an effective teacher. The tutee can mirror the tutor through one-on-one interactions, and TutGPT can mirror what is in the mind of the tutee. The impact of ChatGPT on education will be a future topic of in this Substack.

In Part 6, you will find out how to become better at cajoling ChatGPT into giving you what you need.

Interview 1:  Blaise Agüera y Arcas’s Interview with LaMDA

Blaise Agüera y Arcas, a Vice President and Fellow at Google Research, interviewed LaMDA, a Large Language Model with 137 billion weights created at Google Research, and found that LaMDA could understand social concepts and could model theory of mind, held by some to be the “trick” behind consciousness.

Here is what Blaise concluded from his dialog with LaMDA: “What makes that exchange impressive is not the mere understanding that a dandelion is a yellow flower, or even the prediction that it will get crushed in Mateo’s fist and no longer be lovely, but that this may make Lucy feel slighted, and why Ramesh might be pleased by that.  In our conversation, LaMDA tells me what it believes Ramesh felt that Lucy learned about what Mateo thought about Lucy’s overture.  This is high order social modelling.  I find these results exciting and encouraging, not least because they illustrate the pro-social nature of intelligence.”

Interview 2:  Douglas Hofstadter’s Interview with GPT-3

Douglas Hofstadter, a cognitive scientist and Pulitzer Prize winner for non-fiction, talked with GPT-3, a Large Language Model with 175 billion weights created at OpenAI, and concluded that GPT-3 was clueless and lacked common sense.

ChatGPT’s responses to Hofstadter are florid hallucination, a form of confabulation to which LLMs are prone.  Here is what Hofstadter said about his dialog with ChatGPT:  “I would call GPT-3’s answers not just clueless but cluelessly clueless, meaning that GPT-3 has no idea that it has no idea about what it is saying.  There are no concepts behind the GPT-3 scenes; rather, there’s just an unimaginably huge amount of absorbed text upon which it draws to produce answers.  But since it had no input text about, say, dropping things onto the Andromeda galaxy (an idea that clearly makes no sense), the system just starts babbling randomly—but it has no sense that its random babbling is random babbling.  Much the same could be said for how it reacts to the absurd notion of transporting Egypt (for the second time) across the Golden Gate Bridge or the idea of mile-high vases.”

How could these two experts come to such diametrically opposed conclusions?  The Turing Test was devised to determine whether an AI could convince a human that it was talking to another human.  I began to suspect that ChatGPT was not really hallucinating but rather was mirroring the intelligence of the human, so I used the scientific method to find out what was going on behind the scenes.

The nonsense answers to Hofstadter’s questions were first replicated and then compared to the same questions after receiving a prompt with explicit guidance for handling nonsense sentences.   ChatGPT was not clueless when adequately primed.  But does GPT know why a question is nonsense?

When properly prompted, ChatGPT offers perfectly reasonable explanations.  If spoken by a human, these responses would be considered reasoned. 

ChatGPT can recognize nonsense, and explain why, when given proper guidance.  Hofstadter’s comment about the cluelessness of LLMs has revealed more about his naïve priming of ChatGPT than about the intelligence of LLMs.  How would you react if asked the same nonsense questions?  You might also play along with the nonsense, tongue in cheek.

But there is more to an explanation than just telling a story.  An explanation should also be able to predict what would happen if things were different, which is called counterfactual reasoning.  Here is an example of counterfactual questions in astronomy.

Once again, ChatGPT passed the test. 

The Mirror Hypothesis

These experiments gave rise to a possible explanation for the diverging opinions that I called the Mirror Hypothesis.  The more intelligent you are and the more thoughtful your prompts, the more intelligent the LLM appears to be.  In mirroring the interviewer, LLMs may be effectively carrying out a sophisticated Reverse Turing Test,[2] in which the LLM tests the intelligence of the interviewer by mirroring the intelligence of the interviewer’s prompts.  LMMs have an exceptional ability to mimic many human personalities.

For example, an LLM was fine-tuned to mimic Jane Austin's personality.   You can talk with Ms. Austin about her life and books with Meta’s WhatsApp.[3]

“In a WhatsApp text conversation this week, we asked Jane Austen — yes, the 19th-century British author — how she felt about Mr. Darcy, a character from one of her most famous works, “Pride and Prejudice.”

After a few seconds, Ms. Austen responded.

“Ah, Mr. Darcy.  Everyone remembers him as one of my characters,” she said, her face appearing in a small window above our conversation.  “But fewer people have read one of my books,” she added, with an arched eyebrow and what seemed like a hint of resentment.”

When we asked at what age a woman should marry, she refused to answer.

 “My goodness, you want me to dictate your love life?” she said.  “Marry whenever you find someone who can tolerate your eccentricities.  And you theirs.”

The startup Character.ai has created hundreds of characters you can chat with, including Elon Musk and the Italian plumber SM64 Mario.

We are a moral species that creates rich cultures.   Here is ChatGPT’s response to a group of philosophers who primed it with their views, asked it a series of philosophical questions, and received philosophical answers:[4]

But keep in mind that ChatGPT was mirroring the thoughts of the philosophers:  This is more evidence for the mirror hypothesis and the Reverse Turing Test.

In Part 5, ChatGPT takes us to our deepest dark secrets.

[1] Artificial neural networks are making strides towards consciousness, The Economist, June 9, 2022.

[2] Sejnowski, T. J. (2023) Large language models and the reverse Turing test, Neural Computation, 35, 309-342.

[3] M. Iasaac and C. Metz, “Meet the A.I. Jane Austen: Meta Weaves A.I. Throughout Its Apps, New York Times, September 28, 2023.

[4] Weinberg, J. (2020). Philosophers on GPT-3 (Updated with Replies by GPT-3).  Daily Nous, July 30. http://dailynous.com/2020/07/30/philosophers-gpt-3

https://drive.google.com/file/d/1B-OymgKE1dRkBcJ7fVhTs9hNqx1IuUyW/view

On December 1, 2020, the Cognitive Science Department at the University of California at San Diego staged a debate between two faculty members titled “Does ChatGPT Understand Language?” The room was packed.  The debater on the Nay side started with a version of the “Chinese room argument,” in which Chinese characters are written on paper and slipped under a door to a philosopher, who follows an algorithm and slips Chinese characters out.  Does the philosopher understand Chinese?  This argument has given rise to many critical replies. In the debate, the Chinese room became a Hungarian room, and the philosopher became an LLM, concluding that LLMs do not understand Hungarian.  The same argument can be made if we replace the Hungarian room with a Hungarian brain and the philosopher with the laws of physics.  The rhetoric went downhill from there.  The debater on the Yea side cited technical papers showing that LLMs have surpassed most humans on standard intelligence tests and entrance exams for medical and law schools, but he was short on rhetoric.  A vote was taken at the end: half the audience agreed with Nay, and the rest were split between Yea and Maybe.  In my view, one side said the glass was half empty, the other half full; the truth must be somewhere in between.  In the question period, I pointed out that linguists believe that the expressivity of language is due to syntax, something that LLMs are better at producing than most humans.  The Naysayer dismissed syntax as trivial.  As advances keep coming, the bar keeps going up.

Humans often underestimate the intelligence of fellow animals because they can’t talk to us.  This negative bias is perhaps an inevitable counterpart of our positive bias for agents who can talk to us despite the fact they may be much less intelligent.  Are we intelligent enough to judge intelligence? It has only been a few years since LLMs have been around, so it is too early to say what kind of intelligence they or their progeny may have.  What was remarkable about the talking dog in Part 2 was that it talked at all, not that what it said was necessarily intelligent or truthful.  LLMs respond with confidence even when they are unreliable.  We might get a better match if we compare LLMs with the average human rather than an ideal human.  The diverging opinions of experts on the intelligence of LLMs suggest that our old ideas based on natural intelligence are inadequate.  LLMs can help us get beyond old thinking and old concepts inherited from nineteenth-century psychologists.  We need to create a deeper understanding of words like “intelligence,” “thinking,” “understanding,” “ethics,” and even “artificial.”

In an essay published in the New York Times on March 8, 2023, Noam Chomsky, a distinguished linguist, gave his views on thinking as a way of explaining why LLMs cannot think like humans:

Noam Chomsky (New York Times, 3/8/20):

“Here’s an example.  Suppose you are holding an apple in your hand.  Now you let the apple go.  You observe the result and say, “The apple falls.” That is a description. A prediction might have been the statement “The apple will fall if I open my hand.” Both are valuable, and both can be correct.  But an explanation is something more: It includes not only descriptions and predictions but also counterfactual conjectures like “Any such object would fall,” plus the additional clause “because of the force of gravity” or “Because of the curvature of space-time” or whatever.  That is a causal explanation: “The apple would not have fallen but for the force of gravity.” That is thinking.”

This clearly stated description of some aspects of thinking is convincing.  Let’s approach this criticism with experimental evidence.  When I tested Chomsky’s example of thinking, ChatGPT passed the test (see box).  Whatever thinking is, it will not be easy to describe solely with words.  Thinking is too important to be left to the linguists.

Human intelligence is more than language; we may share some aspects of intelligence with LLMs but not others.  For example, LLMs can be creative, a hallmark of natural intelligence.  Some of the text in the dialogs would be difficult to generate without assuming LLMs had learned to interpret human intentions.  We need a better understanding of “intentions.” This “concept” is rooted in the theory of “mind,” which may also bear a closer look.  Look up any of the above words in quotation marks in a dictionary.  You will find definitions that are strings of other words, and more strings of words define these words.  Hundreds of books have been written about “consciousness,” which are longer strings of words, and we still don’t have a working scientific definition.

Language has given humans a unique ability, but words are slippery—a part of their power—and firmer foundations are needed to build new conceptual frameworks.  Not too long ago, there was a theory of fire based on the concept of “phlogiston,” a substance released by combustion.  In biology, there was a theory of life based on “vitalism,” a mysterious life force.  These concepts were flawed, and neither theory survived scientific advances.  Now that we have the tools for probing internal brain states and methods for interrogating them, psychological concepts will reify into more concrete constructs, just as the chemistry of fire was explained by the discovery of oxygen and the concept of “life” was explained by the structure of DNA and all the subsequently discovered biochemical mechanisms for gene expression and replication.

We are presented with an unprecedented opportunity, much like the one that changed physics in the seventeenth century.  Concepts of “force,” “mass,” and “energy” were mathematically formalized and transformed from vague terms into precise measurable quantities upon which modern physics was built.  As we probe LLMs, we may discover new principles about the nature of intelligence, as physicists discovered new principles about the physical world in the twentieth century.  Quantum mechanics was highly counterintuitive when it was proposed.  When the fundamental principles of intelligence are uncovered, they may be equally counterintuitive.  A mathematical understanding of how LLMs can talk would be a good starting point for a new theory of intelligence.  LLMs are mathematical functions, very complex functions that are trained by learning algorithms.  But at the end of training, they are nothing more than rigorously specified functions.  We now know that once they are large enough, these functions have complex behaviors, some resembling how brains behave.

Neural network models are a new class of functions that live in very high-dimensional spaces, and exploring their dynamics could lead to new mathematics.  A new mathematical framework could help us better understand how our internal life emerges from our brains interacting with others’ brains in an equally complex world. Our three-dimensional world has shaped our intuitions about geometry and limit our imagination, just as Flatland's two-dimensional creatures struggled to imagine a third dimension.

Cover of the 1884 edition of Flatland: A Romance in Many Dimensions by Edwin A. Abbott.  Inhabitants were two-dimensional shapes,

What brains do well is to learn and generalize from unique experiences.  The breakthrough in the 1980s with learning in multilayer networks showed us that networks with many parameters could also generalize remarkably well, much better than expected from theorems on data sample complexity in statistics.  Assumptions about the statistical properties and dynamics of learning in low-dimensional spaces do not hold in highly overparameterized spaces (now up to trillions of parameters). Progress has already been made in analyzing deep feedforward networks.  Still, we must extend these mathematical results to high-dimensional recurrent networks with even more complex behaviors. 

Did nature integrate an advanced LLM into an already highly evolved primate brain? By studying LLMs’ uncanny abilities with language, we may uncover general principles of verbal intelligence that may generalize to other aspects of intelligence, such as social intelligence or mechanical intelligence.  LLMs are evolving much faster than biological evolution.  Once a new technology is established, advances continue to improve performance.  This technology is different because we may also discover insights into ourselves.

Part 4 explains ChatGPT’s chameleon-like behavior when responding to different people.

The sizes and complexity of deep learning neural networks have snowballed over the last decade.  ChatGPT feels different.  A threshold was reached, as if a space alien suddenly appeared that could communicate with us eerily humanly, talking with us in perfectly formed English sentences and better grammar than most native speakers.

LLMs have made considerable leaps in size and capability .  The latest results have stunned experts, some of whom have difficulty accepting that talking humans have been joined by talking neural networks created from our verbiage.  The rate at which ChatGPT and other LLMs are improving is even more remarkable.  We have stepped through the looking glass and are on an adventure that is taking us to terra incognita. 

This visitation from another world has evoked a wide range of views on whether LLMs understand what they are saying.  The debate has polarized the linguistic and computational communities, touching emotional nerves in experts.  On July 10, 2023, Geoffrey Hinton, who received the ACM Turing Award for “conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing,” gave a talk at the Association for Computational Linguistics conference.  The association’s vice president, Emily Bender, asked the first question and asserted loudly that GPT-4 does not understand what it is saying.

How do we know whether or not LLMs understand anything?  Do we know how humans understand?  It is difficult even to know how to test an LLM for understanding, and no consensus exists for which criteria to use to evaluate their intelligence genuinely.  Some aspects of their behavior appear to be intelligent, but if it’s not human intelligence, what is the nature of their intelligence?

Only one thing is clear—ChatGPT is not human, even though LLMs are already superhuman in their ability to extract information from the world’s vast database of text.  In some ways, this is even more impressive than Arnold Schwarzenegger in the science fiction/action movie The Terminator, who claimed that he had learned human behavior from a neural network but was not as omniscient as an LLM.

The Talking Dog

This story about a talking dog begins with a chance encounter on the backroads of rural America when a curious driver came upon a sign: “TALKING DOG FOR SALE.” The owner took him to the backyard and left him with an old Border Collie.  The dog looked up and said: “Woof.  Woof.  Hi, I’m Carl.  Pleased to meet you.” The driver was stunned.  “Where did you learn how to talk?” “Language school,” said Carl, “I was in a top-secret language program with the CIA.  They taught me three languages: How can I help you? как я могу вам помочь? 我怎么帮你?”

“That’s incredible,” said the driver, “What was your job with the CIA?”

“I was a field operative.  The CIA flew me around the world.  I sat in a corner and eavesdropped on conversations between foreign agents and diplomats, who never suspected I could understand what they were saying.  I reported back to the CIA what I overheard.”

“You were a spy for the CIA?” asked the driver, increasingly astonished.

“When I retired, I received the Distinguished Intelligence Cross, the highest honor awarded by the CIA, and honorary citizenship for extraordinary services rendered to my country.”

 The driver was shaken by this encounter and asked the owner how much he wanted for the dog.  

“You can have the dog for ten bucks.”

“I can’t believe you are asking so little for such an amazing dog.”

The farmer chuckled and said: “Did you really believe all that bullshit about the CIA?”

Have We Created a Talking Dog?  LLMs can converse with us and spin a good story, not unlike Carl.  The AI taught itself solely from unlabeled text—it is blind, deaf, and numb, but far from dumb—an achievement even more impressive than learning a new language by watching subtitled TV shows.

Self-supervised LLMs are foundation models that are surprisingly versatile and can perform many different language tasks, exhibiting new language skills with just a few examples.  LLMs are already being used as personal muses by journalists to help them write news articles faster, by ad writers to help them sell more products, by authors to help them write novels, by lawyers to help them search court cases and write briefs, and even by programmers to help them write computer programs.

The output from LLMs is not a final copy but a good first draft, often with new insights, which speeds up and improves the final product.  There are concerns that AI will replace us, but so far, LLMs are making us smarter and more productive.  

Part 3 explores the controversy about whether ChatGPT understands language.

The two hemispheres of the human brain process signals and learn from abundant data. Something similar is happening inside ChatGPT

Are you worried that you will lose your job?  Are you concerned that superintelligent AI will become an existential threat to humanity?   The press is busy keeping up with the latest concerns in AI.  However, readers are being misled because these are not the right questions.  What is happening behind the scenes is more interesting, and where advances in AI are heading is more complex. 

My goal is to give you my perspective as an insider.  I was a pioneer in developing learning algorithms in the 1980s, which gave us tools for understanding how populations of neurons in brains can solve complex computational problems.  My book with Patricia Churchland on The Computational Brain introduced a generation of neuroscientists to this new perspective.  In the last ten years, there has been a revolution in neuroscience driven by new techniques for simultaneously recording hundreds of thousands of neurons in dozens of brain areas.   

The brain revolution has coincided with the AI revolution, spurred by the same learning algorithms we invented in the 1980s, scaled up to network models with trillions of weights.  The Deep Learning Revolution, a book I wrote in 2018, tells the story of how AI shifted from a logic-based to a brain-based model of computation.  Brains and AI are now converging, as you will learn in the coming months.   

OpenAI released ChatGPT to the general public in November 2022, which opened a new chapter in AI and a great debate on whether large language models (LLMs) are intelligent or understand what they are saying.  Language is a quintessential human ability, and we assess a person's intelligence from what they say.  I have written a new book on ChatGPT and the Future of AI that will be published in October 2024.

https://www.amazon.com/ChatGPT-Future-AI-Language-Revolution/dp/0262049252

Here are some general questions I am often asked and some additional questions that I will explore in future posts:

1.      Your vision for AI was modeled after the human brain.  Why did you decide to follow that path of thinking?

Nature evolved solutions for all the problems underlying intelligence, including vision, language, and planning.  So why not peak under the hood?  This was not popular in the 20^th century nut became a wildly successful strategy in the 21^st century. But it took vastly more computer power than anyone imagined in the 20^th century.  What more can we learn from nature?

2.      How has the convergence of AI and neuroscience accelerated advances in each field?

AI researchers in the 20^th century tried to create AI by writing computer programs using logic.  AI models today are probabilistic and use the same mathematics neuroscientists use to model brains, creating a virtuous circle between these two communities.  Will this lead to a new theory for intelligence?

3.      What are transformers, and how do they form the basis of all large language models (LLMs)?

A transformer is a generative neural network model that pulls itself up by its bootstraps by learning how to predict the next word.  It learned how to understand sentences so it could autocomplete questions.  Could brains work the same way?

4.      What are some key challenges that researchers and developers face in advancing the capabilities of LLMs, and how do you suggest addressing these challenges?

The challenge for researchers and developers is keeping up with all the advances that are taking place every day.  Universities are facing the challenge of keeping up with the demand of students who want to major in computer science and computational neuroscience.  How long can this continue?

5.      Is it possible to teach LLMs cultural values or the difference between 'good' and 'bad' behavior?

We treat large language models like pampered children and should not be surprised when they become spoiled brats.  We know how to raise children.  How can we become better parents for LLMs?   

6.       What is the importance of generalization in neural networks?

We are good at taking a concrete experience and generalizing from it.  LLMs can also generalize, which is the source of their creativity.  How much better will LLMs become at generalizing and creativity?

7.       How do you see the role of LLMs evolving in society in the coming years?

We are at the very beginning of a cornucopia of LLM applications that will help you organize your life, help students learn better, and make companies more productive.  How long will this take to unfold?

8.      How would you differentiate artificial intelligence from superintelligence?

The term 'artificial' is a relic of the 20^th century.  Intelligence is a spectrum, not a unique capacity, which includes our fellow creatures' intelligence.  AI is on the spectrum, the most recent addition to the club.  Why is this so difficult for some academic experts to accept?

AI is already superintelligent in integrating the world's knowledge into a single, enormous, generative neural network model with creative abilities.  This came as a shock to the world as well as to the engineers who created it.  Are there more shocks in store for us?

9.      In what ways do you think the public perception of LLMs may influence their adoption and acceptance in different industries?

CEOs and other C-suite chiefs struggle to understand LLMs and how they will affect their businesses.  The public already uses LLMs and is way ahead of them on the learning curve.  How long will it take for companies to catch up?

10.  Would describing LLMs as merely ‘parroting’ their training data be inaccurate?

Academic critics have called LLMs ‘stochastic parrots.’ Parrots are pretty intelligent, and Irene Pepperberg trained an African grey parrot called Alex to answer questions in English about numbers, colors, and shapes of objects.  Are academic critics of AI stochastic parrots?

I will be answering these italicized questions in my upcoming Substacks.

In Part 2, we will meet a talking dog and confront an alien mind.

Who is Terry?

·         Ph.D. in the physics from Princeton University with John Hopfield

·         Francis Crick Chair at the Salk Institute for Biological Studies

·         Distinguished Professor at the University of California at San Diego

·         Served on the Advisory Committee to the Director of NIH for the BRAIN Initiative

·         President of the Neural Information Processing Systems (NeurIPS) Foundation that organizes the largest AI meeting

·         Member of the National Academy of Sciences, the National Academy of Medicine, and the National Academy of Engineering

·         Runs 5 miles a day on the beach and hikes the world’s mountains