An LLM begins life as an complex statistical function that takes in text and spits out predictions for the next likely sequence of words. Inside this function are billions or trillions of parameters, values that determine how input data is transformed into predictions. At the start, these parameters are randomly generated. 

This random function goes through pre-training. The pre-training process starts with scraping billions of words of text from the internet, which is then fed through the function. The LLM predicts what word should come next, and then compares its prediction against the data, using math to figure out how to change its algorithm to better match the correct answer. 

As the LLMs initial parameters are random, so too are its initial answers. But over time, the LLM automatically and incrementally adjusts its parameters (or “weights”) to reduce the discrepancy between its predictions and the actual text. By repeating this process with billions of samples, it iterates the function to one that can produce reasonable text. This gradual improvement is referred to as gradient descent or, more simply, learning. The result is the base model, an LLM that can produce walls of webpage-like text. 

Fine-tuning adapts that base model into the helpful chatbot assistant offering to draft our emails. Companies hire people to collect and label large datasets of higher-quality examples covering the particular tasks the LLM will perform. These task-specific data sets refine the LLM, specializing it for particular purposes. Additional fine-tuning steps have humans rate the LLM’s answers for additional feedback to improve its responses, a process known as reinforcement learning from human feedback (or RLHF). This stage also trains the LLM to refuse answering certain types of questions – such as “How do I build a bomb?” 

However, as LLMs get better, they are increasingly being used to create their own training data. This evolving process could herald a breakthrough in LLM capabilities, similarly to how Alpha Go raced past human players after it started playing itself. Such a leap would lead to groundbreaking advancements, as well as new challenges in oversight and control. 

If you want to go deeper on this topic, check out these additional resources on how LLMs work:

Trio of explainer videos from 3Blue1Brown (2017) on how neural networks (the structure underlying LLMs) work technically.

• But what is a neural network? 

• Gradient descent, how neural networks learn 

• What is backpropagation really doing? 

Intro to Large Language Models by Andrej Karpathy (2023) - A practical explanation of how LLMs are trained, such as what output from a pre-trained model looks like. 

How ChatGPT Works Technically by ByteByteGo (2023) - A high-level explanation of how ChatGPT was trained.

The bill, introduced by state Sen. Scott Wiener, aims to rein in AI models that pose an "unreasonable risk" of causing more than $500 million in damages. Now open-source advocates claim the proposed legislation would stifle innovation and competition. 

The controversy erupted after the accelerationist organization Alliance for the Future alleged the bill was being “fast tracked,” and Fast.ai co-founder Jeremy Howard published a post on the bill’s harms to the AI industry. Howard’s assertion that SB-1047 would "stifle open-source AI and decrease safety" was widely quoted on social media. Alliance director Brian Chau summarized the bill as “Close source your models or else.” 

The opposition is unsurprising coming from Alliance for the Future, an effective accelerationist organization. A recent addition to the political landscape around AI, the effective accelerationist movement wants to accelerate technological progress through the unrestricted development of artificial intelligence -- especially open-source AI. The accelerationists have generally gotten support from Silicon Valley VCs like Andreessen Horowitz who want to see their investments in AI pay off. The AI Alliance, founded by Meta and IBM, has also opposed the bill.

What is surprising is the social media frenzy that converged around the claims that the bill was “fast tracked” and that it would at best stifle (and at worst criminalize) open-source models. 

Supporters of the bill have pushed back fiercely. Senator Scott Wiener, who introduced the bill, accused  Chau of “outright fear mongering” and insisted that it is “absolutely false” to say the bill criminalizes open-source. Meanwhile, Dan Hendrycks, director of one of the advocacy groups sponsoring the bill, said the goal was to prevent open-source models where the safety training could be removed “extremely easily and foreseeably.” 

SB 1047 is supported by two of the “Godfathers of AI,” Geoffrey Hinton and Yoshua Bengio. Professor Hinton said, “Powerful AI systems bring incredible promise, but the risks are also very real and should be taken extremely seriously. SB 1047 takes a very sensible approach to balance those concerns.” 

A former California legislative staffer dismissed claims the bill is on a "fast track," noting it must clear multiple committee reviews, one of which prepared this 38 page analysis. 

The bill's text requires developers to refrain from open-sourcing models that pose an “unreasonable risk” of causing critical infrastructure attacks or autonomous actions causing over $500 million in harm. 

But advocates for tighter AI regulation argue it actually creates a loophole by limiting shutdown requirements only to models "within custody, control, or possession" of the developer. So an open-source developer would be required to shut down the copies of the model they were running, but not the copies that were being run by other people. 

The bill is still being revised based on feedback, and it is likely that the open-source provisions will be workshopped further as the bill advances. Senator Wiener has already announced that the bill will limit coverage to models costing at least $100 million to train. 

As of May 21, SB 1047 has passed the California Senate 32-1 with bipartisan support and now advances to the Assembly. There is will go through the committee process all over again. 

As the legislation advances through the state capitol, both sides are digging in for what could be a defining battle. The future of open-source AI — and public safety — may hinge on it.