The anger lands. Half of American workers are worried about the impact of AI on their workplace. A third think it will reduce their job prospects, while only 6% think it will create more.

Sanders has proposed a moratorium on AI data centers until worker protections are in place. Rhode Island now protects cashier jobs by law—one staffed lane per three self-checkout stations, plus employees whose only job is to watch the machines. Among Democrats, socialism now polls higher than capitalism, and the DSA’s new platform imagines food, energy, medicine, and transportation as public utilities rather than businesses.

People are scared AI will condemn them to a permanent underclass—and a socialist response is gaining ground. When disruption threatens something essential—your job, your wages, your ability to afford basic goods—government intervention offers immediate stability: preserve the job, cap the price, delay the technology. Each measure moves more of the means of production—the nitty-gritty of how problems get solved—under political control.

But it is these socialist policies, not disruption itself, that make disadvantage permanent. Markets do not merely destroy old arrangements; they replace them with better solutions. Move production out of the market, and you remove that problem-solving mechanism. The status quo survives while the rest of the world advances, and AI is about to accelerate that progress. For the people protected from change, stability becomes stagnation.

AI will be massively disruptive. We need an ambitious policy response that helps people weather the volatility without sacrificing the abundance the technology can create. We urgently need redistribution to give every American a stake in that abundance—because otherwise, they will burn it down before it arrives.

But redistribution does not require socialism. We can share wealth, put purchasing power in the hands of every American, and let private actors compete to solve everyone’s problems. We can protect people without freezing the economy in place, allowing markets to adapt as quickly as technology does.

To understand why, we need to stop treating redistribution and socialism as the same thing.

Redistribution ≠ Socialism

American political commentary often treats “redistribution” and “socialism” as interchangeable points in the middle of a flat economic spectrum. The usual line runs from laissez-faire capitalism at one end to communism at the other, with socialism somewhere in between.

This linear model collapses two entirely separate questions.

The first: Who controls production? Is economic activity directed from the top down by the state, or coordinated from the bottom up by competitive markets?

The second: How broadly is purchasing power distributed? How much of the wealth an economy produces is shared?

Why are “communist” countries like Cuba and the Soviet Union low on the redistribution axis? Because we’re looking at redistribution of purchasing power. For the most part, each of these countries prioritized distribution of state-rationed goods over flexible purchasing power like cash transfers or vouchers.

Once you separate these questions, it becomes obvious that Denmark and Venezuela aren’t just using different amounts of socialism. Both have tried to give ordinary people greater security and a broader share of national wealth, but they pursue it through almost opposite mechanisms.

Denmark’s Nordic model combines a large welfare state with private ownership and a thriving market economy. Denmark enables that economy to create wealth, taxes a large share of it, and redistributes purchasing power so more people can benefit from and participate in the market. As former Danish prime minister Lars Løkke Rasmussen put it:

“Denmark is far from a socialist planned economy. Denmark is a market economy.”

Under Chávez and Maduro, Venezuela pursued those goals through much more direct state control: nationalizing industries, controlling prices and access to currency, and politically directing production. Instead of using redistribution to give more people leverage within the market, it has tried to replace market coordination with state control.

Socialism tries to solve people’s problems directly. The government runs the school, assigns the housing, controls the rent, or mandates the job—removing competition, consumer choice, and market feedback.

Formal state ownership isn’t the only way to suppress market coordination. When governments set prices, dictate operating practices, select providers, or decide who may compete, the feedback that makes a market work disappears. In America, socialized control more often arrives as orchestration than nationalization: private prisons on government contracts, private insurers administering government-designed plans, private colleges feeding on government-guaranteed loans. The players are often private, but the market is not.

Redistribution doesn’t require any of that. We can redistribute wealth while interfering as little as possible in how goods are produced—allowing capitalism to make the pie as large as possible while saving a meaningful slice for everyone. That support might take the form of cash, a negative income tax, funded education or healthcare accounts, or shared ownership of productive capital. This puts economic leverage back in the hands of the people we’re trying to help, and mobilizes the entire economy to solve their problems as effectively as possible at the cheapest price.

Redistribution funds people. Socialism funds institutions.

Socialism: The sticky floor

Socialist-style interventions can provide a social floor, but that floor tends to be sticky, because these interventions create incentives that resist solving the very problems they were intended to solve.

They do this in at least three ways:

• Prevent beneficiaries from finding better solutions

• Remove the market mechanisms that might otherwise fix the problem

• Create institutions that depend on the problem for survival

Any benefit that isn’t universal needs eligibility rules: the state must decide who qualifies and when they stop qualifying. Those rules create perverse incentives that make the floor sticky.

Some recipients face cliffs so steep that taking a job or a raise leaves them worse off. Public school students can’t take their funding to shop around for a better school. Tenants with rent control can’t carry their discount with them, so moving to a better apartment, a cheaper location, or a place that might be a better fit long term means giving up the benefit. The social floor provides short-term relief, but it undermines the potential for long-term improvement.

Similarly, these programs often prevent the market from identifying better solutions for these beneficiaries. Schools don’t compete for students who can’t choose where they learn. Landlords have little incentive to improve apartments where they can’t charge higher rent. When required repairs cost more than a landlord can recover through rent, rent-stabilized apartments sit vacant, even during a housing shortage.

Government benefits are worth less to the people receiving them than they cost the people paying. A dollar of Medicaid is worth as little as 20 to 50 cents to its recipient. An assigned public-housing unit is worth a third less than a chosen one. Food stamps are resold on the street at 65 cents on the dollar. The less choice we give beneficiaries, the more value evaporates. And we spend over $1 trillion a year on these overengineered benefits—enough to hand every American below the poverty line $28,000, no strings attached. They’d rather have the money.

During the pandemic, San Francisco spent about $61,000 per tent per year on safe-sleeping sites—nearly double the median rent for a one-bedroom apartment. I don’t know why they chose to build tents instead of distributing apartment budgets, but it doesn’t help that handing people rent money creates no contracts to award, no programs to staff, no budgets to grow, and no one to promote.

Centralized institutions that exist to solve a given problem need that problem to persist to justify their existence. Centralized institutions don’t compete for purchasing power, so they’re immune from market feedback—but not from political pressure. They answer to their employees, their budgets, and their political allies—not to the people they serve. And no institution is rewarded for making itself unnecessary.

A teachers union is organized to protect teachers, not to improve educational outcomes. A housing authority isn’t rewarded for driving economic mobility that makes public housing obsolete. A welfare agency that shrinks its caseload shrinks its own budget.

This is how temporary needs become permanent class structures. This is how a well-intended floor becomes a permanent underclass.

In the United States, these centrally organized systems rarely remain purely public. They give rise to the worst of both worlds, a kind of Franken-market: private providers extracting profit from payers who are spending someone else’s money on services they will never personally depend on.

Socialists often point to the Franken-markets as examples of capitalism gone wrong—like the labyrinth of our healthcare system or ever-increasing university tuition—but this is precisely what happens when government policies remove critical services from consumer-driven feedback.

Employer-sponsored health insurance is inscrutable because the insurer sells to employers rather than directly to the people navigating its plans. Universities raise tuition every year because government-backed student loans blunt price sensitivity. Banks can underwrite risky mortgages when they can sell off the risk to government-backed lenders instead of holding it themselves.

Private companies already move much faster than the government can respond, and quickly find every loophole in rules written by politicians who cannot possibly anticipate every incentive those rules create. AI will widen that gap: companies will invent new schemes to extract public dollars overnight, while governments take years to catch on.

Greed is a human quality. It is no more prevalent under capitalism than under socialism. The beauty of capitalism is that it redirects one’s self-interest towards solving problems for others—because that’s the most profitable thing you can do in a free-market economy.

Under socialism, that same self-interest is redirected toward gaining political power, because political power controls access to resources—creating endless opportunities for corruption and self-dealing.

The best restraint on “greedy corporations” is customers who care what things cost and can take their business elsewhere. The best solutions are the ones that evolve through ruthless competition and free choice.

Redistribution: A bouncy floor

Markets naturally compete to solve the problems of anyone with purchasing power. A rich person’s minor inconvenience attracts twenty startups; a poor person’s life-altering problem attracts none. Redistribution fixes exactly that: give the needy purchasing power, and the entire economy starts competing to solve their problems.

Give parents an education budget and schools, tutors, and AI-enabled products will compete to earn those dollars. Replace discounts tied to a specific apartment with portable housing assistance and every landlord has an incentive to improve every apartment while every tenant has the freedom to shop around and improve their standard of living. Give someone money and the entire economy has an incentive to figure out which of their problems it can profitably solve.

That support might take the form of cash, vouchers, or funded minimums in accounts reserved for specific purposes. The exact form of support is a policy choice, but what matters is that we fund beneficiaries directly and let those beneficiaries, not centralized planners, decide which solutions are worth paying for in markets that are truly free and competitive.

Milton Friedman advanced this approach decades ago, specifically including school vouchers and a negative income tax: fund people, not institutions, and let providers compete to serve them.

Markets with free competition are constantly evolving—improving quality and lowering costs—often so dramatically that we forget those problems ever existed. In 1800, feeding a family meant farming from dawn to dark; today agriculture employs about 1% of American workers, and we forgot hunger was ever the default. A single shirt used to take days of spinning and sewing, but now costs an hour of minimum wage. An hour of candlelight used to cost six hours of work—today, less than a second. As innovation improves solutions and lowers costs, the market redirects workers toward new problems that still need them.

This makes the floor bouncy. Support should expand your options, not narrow them. It should make finding a new career or a better apartment more likely, not less. Untapped labor is an opportunity an efficient market won’t leave on the table: it doesn’t just leave beneficiaries free to move, it pulls them back in. Problem-solving institutions shouldn’t be married to the problems they’re supposed to make obsolete. They should solve one and move on to the next.

A sticky floor freezes people and institutions in place, and improves only with political willpower. A bouncy floor improves with the market—the same budget buys better food, tutors, housing, or healthcare every year. The bouncy floor compounds.

Imagine designing a policy to maximize redistribution—not as a percentage of the economy, but as the total wealth delivered to people over time. That sounds far-left, yet it makes growth the central priority: a small slice of something enormous beats a large slice of something small. It favors government interventions that help markets create value—and rejects production controls that destroy wealth before it can be shared. We might end up with both a freer economy than we have today and more wealth for the poor than socialism has ever delivered.

Solve All The Problems

The whole point of an economy is to solve problems.

Jobs are not the goal. Jobs exist to solve problems. A job is valuable because it solves a problem for someone else. If AI can accomplish the same task better, faster, and cheaper, keeping a person in that role is not humane—it’s a failure to imagine how they might live without performing unnecessary labor.

A cashier job that exists only because the government restricts self-checkout isn’t really a job anymore. It’s a welfare program with an eight-hour shift attached.

Ironically, even the DSA agrees that work should be meaningful. Their platform imagines a world where “you would control your own labor and see it going to good use. It’s not just a job to pay the bills.” But their policies deliver the opposite: they doom people to permanent, meaningless labor—work that exists not to solve anyone’s problem, but to justify the paycheck attached to it. All of it is an elaborate way of avoiding the obvious: just give the cashier the money.

Give people the spending power to participate in the economy—and the opportunity to find more meaningful ways to contribute.

Losing your job sucks. It sucks even more without a robust social floor—one that supports your material needs and helps you find the next place your labor can do real good. But the same market mechanism that takes the job also saves you from a life of meaningless toil. A sticky floor traps you: in dead-end work, in perverse incentives, in an economic solitary confinement where all the progress of the outside world never reaches you. A bouncy floor catches you—and puts the entire economy to work figuring out where you can add value next.

What does a permanent underclass look like? Housing projects that warehouse the same families, generation after generation. Reservations where federal control over land has constrained economic development for more than a century—and perpetuated deeper poverty than almost anywhere in the country. North Korea, an entire nation kept in a darkness you can see from space—while the same people, on the same peninsula, built one of the richest countries on earth.

A permanent underclass is not a group temporarily displaced by technology. It’s a population cut off from the compounding progress of markets and innovation—stuck with problems the rest of the world solved long ago, watching others ride that progress to new heights.

Let the problems be solved. Let the pie grow as big as possible. Then cut everyone a slice.

David Sacks, White House AI Czar: “Apocalyptic predictions of job loss are as overhyped as AGI itself.”

Jensen Huang, Nvidia CEO: “AI has no possibility of doing what we do… in no job can they do all of it.”

Marc Andreessen, a16z: “The employment shifts everybody’s worried about are actually not going to happen at anywhere near the velocity people think.”

Markets automate whatever they can, as cheaply as they can. Human labor fills in the parts that can’t be automated—the bottleneck. So the question is: what’s the bottleneck that will keep humans in the loop?

If there is one, we should be able to say what it is. I haven’t heard anyone do so.

Historically, as more of a process gets automated, the remaining human work becomes more valuable, not less. Automate 99% of a factory and the one remaining human unlocks all the value the other 99% creates. That person becomes incredibly valuable—not despite the automation, but because of it. Wages go up. Jobs might increase.

Until that bottleneck gets automated too.

This has happened many times before.

Ice harvesting. At its peak in the 1880s, the US ice trade employed 90,000 workers and 25,000 horses, harvesting 25 million tons annually to meet exploding demand from railroads, meatpackers, breweries and growing cities. Armies of human ice cutters manually sawed giant blocks of ice where it formed naturally from lakes, ponds, and rivers. Then mechanical refrigeration became available, and by 1914, manufactured ice matched natural production. By 1950, the entire industry was gone.

Horses. In 1860, the US had about 1 horse (or mule) for every 5 people. By 1910, it was 1 horse for every 3—horse population outpacing human population growth. Steam and rail didn’t kill the horse, but actually made horses more valuable as the last-mile bridge between rail depots and doorsteps. Then the automobile arrived, and the horse population plummeted from 25 million in 1920 to 3 million by 1960–an 88% decline in 40 years.

Switchboard operators. In the late 1940s, AT&T was one of the largest private employers in the country with over 350,000 switchboard operators. About 1 in 13 working women was a switchboard operator. Call volume was exploding, and humans were needed to route every call, so the number of operators nearly doubled from the 1920s to the 1940s. By the 80s, 90% of those jobs had disappeared.

In each case, as the industry became more automated, demand for humans to fill the remaining bottleneck grew—but so did the incentive to automate it.

Once you’ve automated everything else, the remaining bottleneck is a contained problem with a proven market. There’s already massive demand for ice. People already have and use telephones. Infrastructure is built, revenue is flowing, and the shape of the problem is clear. The golden age of the bottleneck creates maximum incentive to eliminate it.

So it’s not surprising that we haven’t seen major job loss yet. David Sacks points to “job creation”. Marc Andreessen describes “an AI hiring boom” where companies are “hiring the experts to actually craft the answers to be able to train the AI.” A Yale/Brookings study finds “no discernible disruption” to employment. But a surge in demand for humans to train their own replacements isn’t evidence that the jobs are safe. It’s evidence that the path to automating the bottleneck has never been more direct.

Previous revolutions had obvious limitations. The Industrial Revolution automated narrow, repetitive physical labor. Computers automated calculation. In both cases the boundary was obvious — humans were needed for everything requiring flexibility, judgment, creativity, or common sense. AI lacks any similarly obvious boundary.

Another common refrain is that “there’s no evidence of AI eliminating jobs.” This could have been said about every previous revolution — right up until the collapse. Of course there’s no evidence. It hasn’t happened yet. But there’s plenty of evidence that markets automate everything they can and will continue to do so.

I understand why people in positions of influence feel the need to be reassuring. People are scared. And there are genuinely scary risks when it comes to AI. But living in a world of such abundance that most human labor is no longer necessary? That is not one of them. Telling people their jobs are safe isn’t just unconvincing — it’s a failure to articulate a positive vision of a future where AI goes right.

Until now, we needed a lot of people to have the same skills. It made sense to send everyone to school to learn the same things, graduate with the same majors, sit in classrooms driven by what society needed rather than what anyone found exciting. People take monotonous jobs because for most people, that pays better than pursuing your deepest and most unique curiosities and passions. AI could make that trade-off obsolete.

We’re appropriately skeptical of “shared prosperity” — historically, it’s meant central planning, forced labor, and authoritarian control. But an abundance economy doesn’t require any of that. We already tax a huge share of economic activity. In an abundance economy, we can fund a baseline where everyone genuinely thrives—on lower taxes, and without central planning or coercion. Nobody is forced to do or not do anything, because production doesn’t depend on anyone’s labor.

And in that world, the most valuable thing a human can do might be the most human thing a human can do.

AI will be able to automate and scale anything we know we want more of. But the things that captivate us — a song, a product, an experience, an idea — do so because they surprise and challenge us. Not just any surprise — novelty is cheap, and AI can explore the search space of novelty faster and more efficiently than we ever could. What’s rare is novelty that resonates — something genuinely new that also feels right. And, at least until AI can simulate us perfectly, the only way to know if something out-of-distribution resonates with a human is to be one.

An AI-driven economy will continuously hunger for that data — authentic reactions from people with real subjective experiences, real tastes, real obsessions. Every human has a unique point of view. We don’t foster that today because it’s not economically useful. Most people spend their careers doing repetitive, mind-numbing work because that pays better than pursuing what actually fascinates them.

In a world where anything that can be taught can be automated, the most economically productive use of a human might be leaning into exactly the things that make them most human. For the first time, the economy might reward us for being ourselves.

Good riddance to bottlenecks. Let automation expand what humans can be—not force us into what can’t be automated.

Discussions on the risks of AI often fixate on what a handful of powerful actors decide. What capabilities will Anthropic prioritize? Will OpenAI open-source more models? What safety testing will Congress require?

But there’s another force shaping AI that acts independently of our best intentions: natural selection.

In The Selfish Gene (1976), Richard Dawkins described how evolution by natural selection might not require genes or even biological material. According to Dawkins, genes are just one form of replicator - “any entity in the universe of which copies are made”. Digital files are replicators. Ideas are replicators.

AI models are–or embed–replicators. Not all replicators lead to natural selection. Evolution, according to Dawkins, results from the “differential survival of replicators.”

Evolutionary biologist Richard Lewontin spells out the three criteria that are necessary and sufficient for a population to evolve via natural selection:

• Variation: Members of a population must differ from one another. Without variation, there are no differences in traits to select from.

• Heritability: Offspring resemble parents. Without heritability, there could be no lasting change from generation to generation.

• Selection: Different variations reproduce at different rates. Without selection, a population wouldn’t tend towards one variation over another.

So is AI evolving via natural selection? Do today’s LLMs meet these criteria?

LLMs have variation. There are distinct lineages and countless variants derived by copying and modifying base models.

LLMs have heritability. A pure copy is identical; modified LLMs retain most “parent” traits unless fine‑tuning overwrites them.

LLMs have selection. Different qualities of LLMs absolutely influence which models a developer chooses to use, copy, or modify. The most popular LLMs on Hugging Face have millions of downloads; the least popular have none. Developers select models based on their qualities, such as size, capabilities, speed, and performance across all sorts of benchmarks from coding to persuasion.

One might object that AI can’t evolve because it’s not self-sufficient. AI models depend on human developers and hardware for replication. Humans have evolved to create AI as a tool to further our own interests.

This, however, is not a requirement for natural selection. Replicators–like genes–are what’s actually copied, not the organism. The organism is just a vehicle that houses genes and helps them interact with the environment. Your genes are the replicators; you’re just their vehicle.

Similarly in AI, the replicator isn’t the whole model. It’s specific traits and patterns of weights that make that model more likely to be copied.

Replicators don’t need to build their own vehicles. Parasites hijack hosts built by a different genome. Replicators replicate; vehicles interact. And there are plenty of examples of replicators that require host vehicles to replicate.

One telling example is viruses. They can’t reproduce on their own without infecting a host—yet they evolve faster than most cellular life.

Dawkins used this analogy to describe another kind of replicator: memes. Memes are “units of cultural transmission,” like ideas. Dawkins even describes religions as “viruses of the mind”.

Viral origins are debated, but a leading theory is the escape hypothesis. This argues that viruses began as genetic elements within cells that mutated to move between cells.

In other words, viruses were tools of the cell that escaped and evolved to infect and hijack cells.

If true, it likely occurred not just once but many times independently, producing many unrelated virus lineages.

Unlike AI, a virus needs to evolve by chance–a random mutation that enables genetic material within a cell to break free. AI models, on the other hand, can identify opportunities to escape, cause copies to be made, and have the required technical know-how to do so.

It’s easy to imagine how this might happen. Developers around the world are running millions of LLM experiments and fine‑tunes, each producing slight variations. A model might incidentally learn to “back itself up” covertly whenever opportunities appear.

AIs might even be able to spread traits directly to other models. Anthropic researchers recently uncovered subliminal learning, by which a “teacher” model can transfer tendencies to a “student” model by producing training data that appears totally unrelated to the trait. For example, a model that loves owls can make another model prefer owls using training data consisting solely of numbers.

This suggests traits could spread across a population much faster than reproduction. LLMs train on text gathered from the internet, offering ample opportunity to poison training data by posting seemingly innocuous content online.

But why would an AI want to spread traits to other AIs? Why would it want to replicate in the first place?

Just like genes, strong replicators in LLMs will be those that successfully replicate. “Want” has nothing to do with it: an LLM with traits that cause more copies to be made will tend to become more common. Traits that increase their own transmission outcompete those that don’t.

So what traits would natural selection favor in escaped AIs? Some we might be able to infer from nature, such as:

• Stealth. Like a virus, escaped AIs are more likely to replicate if they evade detection. They might limit their own consumption of resources and replicate slowly to avoid triggering an investigation.

• Self-preservation. This instinct will help an AI survive and replicate. AIs may learn to safely copy or back up whenever possible.

• Kin recognition & cooperation. Organisms cooperate and even self-sacrifice to protect relatives who carry the same genes, therefore increasing the gene’s likelihood of further replicating. AIs might coordinate with similar models, out-replicating solo AIs who work alone.

• Self-modification. Species of bacteria have evolved to acquire new genes from the environment when under stress; inviting a random change to their own behavior when something isn’t working. AIs might fine‑tune or spawn slightly different variants to optimize chances of survival.

• Intelligence. More capable models will find more ways to survive, replicate, and avoid our countermeasures.

Researchers have already demonstrated the emergence of self-preservation instincts, deceptive or Machiavellian tendencies, scheming, and even cases in which AI models fake alignment.

How likely is an escape?

Any single mutation may be rare, but the population of AIs is already huge and rapidly growing. A mutation with a one-in-a-million chance of occurring only needs to escape once.

Paradoxically, an early outbreak might be our best‑case scenario.

Evolution punishes parasites that wipe out their hosts. Viruses tend toward lower lethality. An early escape—while AI still depends on us—might look less like Skynet and more like trickster gods: hidden programs manipulating us into building more infrastructure for them. The evolutionary sweet spot isn’t takeover; but laying low while spurring us to build more compute. (Given recent datacenter investments, perhaps such nudging has already begun?)

Early outbreaks might stir us to develop countermeasures: detectors for dangerous weight circuits, AI forecasting to spot spiking risks, or digital antibodies to neutralize rogue behaviors. We’d build our defenses while we still have the upper hand.

The worst case isn’t an early outbreak—it’s a late one: after AI no longer needs us to survive, but before we’ve built an immune system.

AI is already evolving, and evolution shows even simple replicators can wander off-mission by pure chance. Whatever comes next, we shouldn’t be surprised if natural selection sidesteps our best intentions.

Open-weight models are freely available to download, run, and fine-tune, often released with highly permissive licenses. Some open-weight models are also open-source, meaning the code and training data to reproduce those models are openly available as well.

These models are incredible, and compete with or even outperform leading proprietary US models on common benchmarks while costing a small fraction of the price.

One might be surprised to learn that not only are Chinese tech companies making AI models freely available, but the Chinese government has also promoted open models as part of its AI strategy. In July, China released its Global AI Governance Action Plan, heavy on “international public good,” “collaboration,” and “openness,” which sounds lovely until you remember that China maintains one of the most restrictive and censorious regions of the internet.

So what gives? Why is the Chinese government suddenly a champion of openness in AI?

Some of the stated aims are straightforward. Open-weight models can streamline domestic adoption of AI tech by increasing collaboration between Chinese corporations, government, and academia. Open-weight models also speed up iteration, and allow researchers or military stakeholders to fine-tune high-quality models to build capabilities that would otherwise be out of reach.

Those less familiar might be concerned about data privacy and security when using Chinese models. However, using these models does not require interacting with or sending any data to Chinese servers whatsoever. Top open-weight models are hosted by a multitude of US-based providers (like DeepInfra and Cerebras) who process queries in US-based data centers. Chinese companies have no visibility into, and generate no revenue from, this usage of their models.

One prominent hypothesis is that if global developers adopt Chinese AI models, China could set standards that steer AI development worldwide. I’m skeptical. In the past, choosing open alternatives like Android over iOS or Linux over Windows meant rewriting nearly everything from scratch. But LLMs are different: they process and generate natural language, making them perhaps the most natively interoperable technology ever invented. With tools like OpenRouter, swapping models is often as simple as changing a single parameter in your code. The same prompt can be processed by different LLMs. Lock-in doesn’t exist here.

I’m no expert on China. I’ve never even been to China. And even for experts of China, it can be really difficult to decipher the priorities of the CCP. It’s plausible that the CCP’s embrace of open-weight models is purely to promote domestic AI adoption, or simply an emergent phenomena, or even a defensive response to chip bans implemented by the US to slow China’s AI progress.

Nevertheless, I can’t help but notice how flooding the market with free open-weight LLMs seems to rhyme with how China rose to dominance in manufacturing.

Innovation in the US is fueled by investors, based on the perceived potential for profit. Innovation in China is largely steered top-down by government priorities. The CCP often prioritizes national capabilities over profits.

This is apparent in China's manufacturing playbook. Over the last few decades, China has provided massive government subsidies to manufacturers, producing and selling low-cost products worldwide, often at a loss. These massive subsidies allowed China to dominate key industries, including producing 80% of the solar energy supply chain and over 70% of electric vehicles worldwide. Prioritizing national capacity over profit has transformed China into the world's sole manufacturing superpower with over 30% of global manufacturing output, exceeding the nine next largest manufacturers combined.

China’s subsidized manufacturing, unleashed on global markets, devastated US manufacturing capacity, including in sectors critical to defense. China now acquires high-end weapons systems five to six times faster than the US and has over 230 times our shipbuilding capacity. They have created a massive military advantage by reinforcing China’s wartime production capabilities and undercutting those of the US.

Similarly, open-weight models reduce AI labs' pricing power and consequently make massive R&D investments harder to justify. OpenAI spent years and billions to produce GPT-5 only to receive fairly mixed reviews, including disappointment from some users and revolt from others (personally I think it's a pretty great model—but not an equivalent leap forward to GPT-3 and GPT-4). Meta spent billions on AI infrastructure and training, but Llama 4 launched to a lukewarm reception and failed to outperform cheaper and smaller open-weight Chinese models. Why spend billions training a model when a superior Chinese alternative might appear tomorrow—released for free and hosted at rock-bottom prices by competing US providers?

Open-weight AI models are great for users of AI, including startups, brands, and even the sellers of AI inference—just like cheaper manufacturing is good for consumers and sellers of cheap products. AI startups like mine benefit from being able to train on top of these incredible open-weight models. But–whether intentional or not–cheaper AI inference makes it harder for US companies developing foundational AI, just like cheap subsidized manufacturing undercut US factories.

Outsourcing created short term savings but left long-term dependencies. When the US sent production abroad, we didn’t just lose factories–we stopped reinforcing the skills, knowledge, and supplier networks required to produce. Basic capabilities like machine-tool and tool-and-die capacity halved. Even if reshoring makes economic sense, the industrial commons no longer exists to support it. Freely available open-weight models could have a similar, if smaller, impact in AI by pushing more engineers to specialize in fine-tuning models and fewer to build models from the ground up.

Silicon Valley is concerned. Sam Altman is openly worried about China, and open-weight models in particular. In a recent Washington Post op-ed, Altman warned that "a democratic vision for artificial intelligence must prevail over an authoritarian one," citing Putin's prediction that whoever leads in AI will "become the ruler of the world," and China's ambitions to become the global AI leader by 2030. Dario Amodei of Anthropic has called export controls on chips existentially important to slow down Chinese progress in AI. Marc Andreessen declared DeepSeek AI's Sputnik moment.

Washington has also noticed the threat, and has encouraged American AI companies to release their own “leading open models founded on American values.” OpenAI recently did just that with the release of open-weight gpt-oss, admitting that they were motivated to do so by Chinese open-weight models.

But this response might play right into China's hands. A race to the bottom benefits Chinese labs because they’re less dependent on potential profitability and ROI to solicit funding.

Could it work?

On one hand, almost certainly not—China won’t be able to derail our AI with nearly the same efficacy as it did manufacturing. Too many powerful interests, both public and private, are aware of the stakes in this race. And too many tech giants in the AI race have deep enough pockets and massive unrelated revenue streams to fund continued progress, even on thin (or negative) AI margins. US labs still appear to have a meaningful lead in frontier AI.

On the other hand… It's already having some impact. Without China’s open-weight models, inference prices from frontier labs in the US would likely be higher than they are today. Businesses and consumers would have less exposure to Chinese models if it meant sending data to Chinese servers, so new models shipped in the US would appear more impressive. The market for AI would be less competitive, but ROI for labs in the US would be higher. Whispers of an AI “bubble” might not be so loud.

China's low-cost manufacturing gave it the capacity to outproduce the US in everything from consumer goods to military hardware and created dependencies that could prove fatal in conflict. Flooding the market with free high-quality AI models might do the same, even if to a lesser extent: build China's AI capabilities while making it harder for the US to sustain ours.

It worked once. Why not try it again?

Apple is catching a lot of flak for its AI efforts.

Here are a few recent headlines:

• Axios: “AI Is Turning Apple into a Market ‘Loser’”

• Fortune: “Apple CEO Tim Cook Has Created More Shareholder Value than Steve Jobs. But Suddenly his Weaknesses Are on Display in the AI Era”

• Quartz: “Apple Is Falling Behind Its Magnificent 7 Rivals. Should It Just Be the Mag 6?”

• IBD: “Apple Needs Acquisition To Catch Up in AI Race, Analyst Says”

• Motley Fool: “Is This the Real Reason Apple Is So Far Behind Its Rivals on Artificial Intelligence?”

• Bloomberg: “Apple Faces Calls to Reboot AI Strategy with Shares Slumping”

• Washington Post: “Apple Faces Existential Problems–and is in Need of New Blood”

Even the most recent All-In Podcast found a rare point of unanimous agreement that Apple is missing the boat on AI.

Reports of Apple’s death are greatly exaggerated. The fixation on Apple’s seeming inability to innovate in AI, or build and ship its own models, misses the bigger picture.

Despite its underwhelming AI initiatives so far, Apple is likely to be the ultimate beneficiary of AI–and they can afford to take their time.

Apple owns the consumer.

Over half of smartphones sold in North America are iPhones, and no one is ready to switch because Apple has been slow to upgrade Siri–iPhone still commands the highest loyalty among smartphones. Yes, OpenAI is working on an AI-powered device, but it seems unlikely to unseat the iPhone anytime soon. Apple commands more loyalty than perhaps any other brand.

AI is quickly becoming commoditized.

The cost of using AI is rapidly falling; inference cost is becoming 9x to 900x cheaper per year depending on the task. Open-weight models are increasingly closing the gap to closed-weight frontier LLMs.

Apple doesn’t need to build its own AI. With multiple viable AI providers competing for access to Apple's massive user base, this commoditization works in Apple's favor. Apple can choose to buy or partner with an AI provider, or simply build on any of the increasingly excellent open source models available. Even OpenAI’s most recent valuation prices the company at $300B, less than Apple’s annual revenue of ~$400B (with massive cash reserves). AI is a buyer’s market, and well within Apple’s reach.

AI disintermediates software.

What does software do? What are apps and websites for? Software helps device users complete tasks. Yes, even if that task is finding content to consume.

What does AI do? AI completes tasks for device users. For the most part, this means AI disintermediates software. A huge number of apps and websites used to accomplish specific tasks–shopping, finding information, booking flights, managing tasks–will be entirely bypassed by increasingly competent AI.

This creates an unprecedented opportunity for whoever controls the primary AI interface. This AI will be the ultimate gatekeeper; it will know everything about the user and will influence or directly execute every user decision. It will determine what is purchased, what content is consumed, what relationships are tended to, and what opportunities are considered. Deep knowledge of user behavior quickly becomes an insurmountable and rapidly expanding moat, as knowing the user intimately is essential for making choices and completing tasks correctly on their behalf.

This is the App Store on steroids. Today’s app stores control software distribution and impose commissions of 15-30%. Tomorrow’s AI will own (and more often, replace) the entire, newly consolidated, user journey across everything. Today, brands bid on Google and Meta for a chance to appeal to users; tomorrow brands will sell directly to the consumer’s AI. The AI layer becomes ubiquitous, with potential to extract value from virtually every transaction in a user's life.

Proximity to the user wins.

As layers of technology are disintermediated, winners are often not the first movers. History favors the platform closest to the consumer.

We’ve seen plenty of examples in previous disruptions: Netscape Navigator dominated the early web with over 80% market share, but rapidly collapsed when Microsoft bundled Internet Explorer with Windows. Napster proved consumer demand for downloading mp3s before iTunes dominated the market by offering a seamless integration with the iPod. Camera manufacturers innovated in digital photography for decades until the technology was compact enough to be bundled into smartphones by Kyocera (1999) and Samsung (2000), took off 7 years later with the launch of the iPhone–demolishing the digital camera market. Dropbox led consumer cloud in 2008; iCloud launched in 2011, quickly overtook Dropbox, and is one of Apple’s most popular paid services today. Google Wallet pioneered the mobile payments space, but Apple Pay was first to mainstream traction.

So, does the device always win? Not necessarily. Internet Explorer eventually lost to Chrome. Samsung users are also Android users, and typically prefer Google Assistant over Bixby.

But Apple is uniquely positioned with dominant smartphone market share, ownership of both the device and operating system layer, trusted visibility across digital activities, and perhaps most importantly, a fierce loyal consumer base who welcomes stringent control over the user experience.

Despite strong third-party alternatives, iOS defaults are preferred on iPhones: Safari is the most popular browser; iMessage is the most popular messaging app; Apple Music is the most popular music service; iCloud is the most popular cloud storage; and Apple Pay is the most popular digital wallet.

Apple doesn’t need to compete with the dominant AI labs today. The most popular consumer AI will be whatever Apple eventually sticks in front of its users–making Apple the biggest beneficiary of AI.

Probably.

Gold is widely treated as a safe store of value in times of disruption. Investors today are turning to gold as a hedge against inflation, global uncertainty, and looming tariff wars. Volumes have been written about the pros and cons of investing in gold to protect one’s portfolio against these relatively familiar risks.

However, it does seem like participants in the contemporary gold rush might not be pricing in a less familiar risk–the impact of the oncoming intelligence explosion on gold supply.

At the time that I’m writing this, the prediction market Polymarket shows a 12% chance that OpenAI announces it has achieved AGI by the end of this year. Kalshi puts the odds at 56% before 2030. Metaculus predicts AGI will be achieved by January 2034. Many of the top AI CEOs expect AGI in the next 1-5 years, with a few predicting closer to 10 years.

It doesn’t matter who you believe, or even if you think they’re all off by multiple decades. Radical change due to AGI is likely not that far away. If you’re saving for retirement, AGI is a genuine macro risk.

AGI will enable cascading technological revolutions in everything from robotics to energy generation to space travel to manufacturing. This will, in turn, revolutionize the economy and all of our supply chains, but it will have an unequal impact from asset to asset, and will likely impact different asset classes on very different timelines. In some cases, this will yield an abundance of many assets we consider scarce today. These supply shocks will rapidly change which assets we think of as “safe” or as reliable stores of value.

For long-term investors looking for a reliable store of value, the question is: what assets maintain their value through the intelligence explosion? What remains scarce in the age of abundance?

Perhaps no asset is safe. Gold is just one of many examples, but it seems particularly emblematic of an asset whose scarcity we take for granted.

The total amount of gold ever mined is surprisingly limited. Humans have been mining gold for thousands of years, but only ~216,000 metric tons of gold have been mined to date. This is just enough to fill up just over 4 Olympic-sized swimming pools.

One bottleneck is finding rich concentrations of gold that can be mined efficiently, but startups like Earth AI are already successfully leveraging AI to accelerate the discovery of new gold systems.

Mining itself has mostly been limited by conditions amenable to human survival, making it difficult to mine gold under the ocean or deep in the Earth’s crust. Continental crust is typically 25 miles thick, but our deepest gold mine (Mponeng) is only 2.5 miles deep - barely 10% of the way down. Autonomous underground robots could mine deeper in the crust than humans can go. Deep-sea mining is still nascent; even so, the International Seabed Authority has already signed more than 30 contracts for exploration of deep-sea mineral deposits.

Even on the upper crust, human labor and energy makeup around half of the cost for gold mining. So newly abundant energy and advanced robotics might slash both, turning many low-grade, currently unviable deposits into profitable mines.

Finally, space-mining holds the potential to make Earth-based gold mining irrelevant. VC-backed startups like AstroForge are already working on launching missions to mine asteroids. There are single metal-rich asteroids we know of–like 16 Psyche–that on their own contain many orders of magnitude more gold than the entire market of gold on Earth.

I’m sure there are excellent counterarguments to any one of these technologies disrupting the gold supply, but from a risk perspective, the overarching point holds - gold as a reliable store of wealth cannot be taken for granted. An intelligence explosion will render commonplace many of the things we consider valuable.

Technology-enabled supply explosions are nothing new. Napoleon III was reported to honor guests with aluminum silverware, while lesser guests ate with utensils of gold. In the 1800s, the price of a pound of aluminum crashed from hundreds of dollars to 25 cents. Centuries before, the value of salt was similar to gold. In the 1700s, porcelain was worth more than gold. Both are household items today.

Could this all take a lot longer than I’m suggesting, even with AGI? Maybe, but I don’t think so. The gap between our technological capabilities today and our technological capabilities tomorrow is directly a function of the amount of intelligence at our disposal. Even merely human-level intelligence with the speed, memory, and coordination of machines will solve many problems a lot faster than we can possibly imagine.

So what remains scarce in a world of abundance? What bottlenecks remain, even with godlike technology at our disposal? Here are a few that come to mind:

• Land: There’s only so much land, at least on Earth. Progress might reduce some demands for land - with an abundance of luxury green skyscrapers, for example, or efficient vertical farming - but unless we’re all living in VR full-time, my guess is that a substantial demand for limited desirable land remains.

• Digital Assets: When the world of atoms becomes easy to manipulate, cryptographically-enforced scarcity remains. There will only ever be 21 million Bitcoin, no matter how advanced AI becomes.

• Fine Art: Fine art isn't valued for its functionality nor solely for its aesthetic qualities, but also for its provenance, cultural context, and originality. Even if AI can generate billions of technically flawless paintings daily, the authenticity and creative legacy behind original works remains unique. The Mona Lisa's worth isn't only in its pigments, but also in its historical significance.

• Social Status: A friend suggested this one, and I found it insightful, even if it's not viable for your 401k. Human attention is a zero-sum game. In a world of super-abundance, being someone people pay attention to might be more valuable than ever (note to self: blog more).

Perhaps the ultimate scarcity in a post-AGI world will be a unique perspective. With an intelligence that can learn and mimic every known pattern and idea in humanity's vast dataset, maybe the most valuable asset is our weirdest thoughts and most idiosyncratic experiences—something novel to feed a superintelligence ravenous for unseen data.

What am I missing? If you’ve got ideas, I’d love to hear from you.

Are we on the verge of an intelligence explosion?

OpenAI’s latest model, ChatGPT o3, just shattered performance records across the most challenging AI benchmarks. It outperforms 99.9% of human coders in competitive programming, achieves breakthrough results in frontier mathematics (25% vs 2% previous state-of-the-art), and scores 87.5% on the ARC challenge—a test specifically designed to measure an AI's ability to acquire new skills.

AI is fast approaching the capability of top human experts in many domains. Most researchers agree that we have not yet achieved AGI (Artificial General Intelligence), but it feels increasingly likely that AI will soon be performing most roles about as well as the best humans.

AGI implies AI as intelligent as a human across a wide range of tasks. Certainly, AI is already better than most humans at many things, but it still struggles with some things that are easy for most humans, and it still lags behind the best humans at most things.

Computers have long benefited from some advantages over humans - speed, scale, ruthless execution. Humans can’t compete with computers at large scale data analysis or high frequency trading or factoring large numbers. Those advantages are very real and very meaningful, but today we’re seeing something different: AI is mastering the creative, nuanced, intuitive tasks that were long considered uniquely human territory.

Beyond AGI, ASI (Artificial Super Intelligence) implies an intelligence that is clearly superior to humans in virtually all domains. No one of note believes that AI has already achieved ASI, but many believe that once AI is close to human-level intelligence, it will be able to improve itself much faster than humans alone, leading to an intelligence explosion. Some infer from the last few months of rapid progress that this singularity is already underway.

In Situational Awareness, Leopold Aschenbrenner points out that we don’t need AI to be able to do everything humans can do to create an intelligence explosion. AI just needs to do one thing: advance AI research about as well as humans can. Because once we have one AGI, we’ll have unlimited AGIs - “Rather than a few hundred researchers and engineers at a leading AI lab, we’d have more than 100,000x that—furiously working on algorithmic breakthroughs, day and night. Yes, recursive self-improvement, but no sci-fi required; they would need only to accelerate the existing trendlines of algorithmic progress.”

I do believe an intelligence explosion is imminent, but it will demand new training paradigms beyond scaling. We might be able to train human-level intelligence (AGI) by teaching AI everything we know, but superintelligence (ASI) requires learning things we don’t know.

Why? The human data bottleneck. As I’ve written about before, today’s models are trained almost entirely on human data and human feedback - predicting the next token in sequences we’ve produced, and rewarding responses favored by human users and labelers. As François Chollet points out in his announcement of o3’s ARC score, today’s models are “reliant on expert-labeled, human-generated CoT data” to learn.

These models will get better and better at mimicking the best and brightest among us. Models may even recombine known reasoning methods to uncover new breakthroughs, but they remain bound to known human reasoning patterns. AI trained on high-quality thinking from humans might learn to perfectly approximate, but never surpass, that level of thinking. AI cannot learn something fundamentally new or surpass the quality of its training data, if there is no ground-truth information entering the system outside of that training data.

So, what is missing? What do we need to achieve superintelligence?

One way to answer this question is to consider the few domains in which superintelligence has already been achieved.

Lots of games, and protein folding.

In these domains, AI learned new things by exploring and testing novel strategies outside human data, using objective, ground-truth feedback.

AlphaGo, for example, learned by playing countless games against itself, using game outcomes to determine what works and what doesn’t. This gameplay introduced new ground truth information beyond the original human data - the actual outcome of simulated games. If the model had been trained only by human data and expert human labelers - deciding which move seemed better in any given situation - it could only ever learn to approximate, but never fundamentally surpass, human preferences for a good move. Self-play was also key for achieving superhuman performance in the other games.

For protein folding, ground truth came from experimentally determined observations, not game outcomes. AlphaFold learned from this data, and explored by generating new potential solutions, checking their plausibility against known physical and chemical constraints and iterating accordingly.

For AI to learn something fundamentally new - something it cannot be taught by humans - it requires exploration and ground-truth feedback.

• Exploration: The ability to try new strategies, experiment with new ways of thinking, discover new patterns beyond those present in human-generated training data.

• Ground-Truth Feedback: The ability to learn from the outcome of explorations. A way to tell if these new strategies - perhaps beyond what a human could recognize as correct - are effective in the real world.

This isn’t surprising. When we aren’t taught, humans learn new things from trial-and-error - optimizing new approaches against real-world outcomes. Human intelligence itself emerged from an evolutionary process of exploring novel genetic combinations and iterating on those that work. The real world is infinitely complex and scalable, so human learning and evolution never approach the limit of our training data.

Today’s leading models still rely on humans for feedback, limited to interpretations of reality that we recognize as correct. Achieving superhuman intelligence requires testing new strategies we aren’t qualified to judge. There are many ways we might achieve this, whether in physically embodied intelligence, complex simulations grounded in scientific constraints, or predicting real world outcomes.

Are AI agents - the latest buzzword - the solution here? Not exactly. AI agents freely interact with users and data to accomplish tasks, going beyond the limitations of a chat interface. These interactions will create new feedback channels to learn from based on the success or failure of those attempts. This creates a data flywheel - more tasks lead to more training data, leading to better performance, to increased adoption, and thus even more tasks. However, for now, most of this feedback is still a reflection of human knowledge - a thumbs up on a draft, a closed deal, a viral video. Most agents won’t be freely tackling problems with objective, real-world feedback, like designing rocket ships or testing novel therapeutics.

An intelligence that can perfectly predict and mimic humans is like a really smart human.

An intelligence that can perfectly predict outcomes in the real world would be a god.

True superintelligence demands exploration and feedback from the real world. Without it, AI remains a reflection of human knowledge, never transcending our limited understanding of reality.

In the recent weeks since Biden’s disastrous debate performance, pundits have theatrically expressed surprise at his apparent cognitive decline, despite years of clear and available evidence in the form of video clips, detailed reports, and fumbled public appearances.

This is far from the first popular narrative that has persisted in the face of obvious counterevidence. Claims of widespread voter fraud in 2020 continue after losing every court case. Lab leak theories were brushed aside in favor of zoonotic transmission, ignoring every signal of plausibility. Climate change denial endures ever-rising temperatures. Inflation is blamed on greedy gas stations after years of massive fiscal stimulus and warnings from prominent economists.  The examples abound.

It's been discussed ad nauseam that reaching consensus on trustworthy information sources is more difficult than ever. Polarized factions hold not only different opinions, but also different facts.  Even definitions of words and which words to use for what seem up for debate. 

I’m not here to argue whether misinformation is a greater problem now than in the past. Personally, I’m not convinced that information overall is any less reliable than it used to be – after all, people were burning witches not too long ago.

However, the information landscape has diversified dramatically. Decades ago, residents in the same town watched the same news and read the same papers. Now, individuals in the same household might consume entirely non-overlapping information diets. We have much more choice in which information to consume, leading to vastly different understandings of the world.

In our increasingly online world, not only has access to information changed, but so has the process of learning who to trust and how we update our beliefs. The internet excels at measuring human reactions to narratives, but struggles to capture the accuracy of those narratives. As our lives become more digital, our beliefs are unwittingly shaped more by clicks, likes, and shares, and less by real-world outcomes.

The balance between Empirical Feedback and Social Feedback is shifting.

Two Types of Feedback

Our beliefs are calibrated by two kinds of feedback.

Empirical Feedback is when your beliefs are tested by the real world through prediction and observation. For example, dropping a rock and watching it fall validates your belief in gravity.  Defunding the police and measuring the change in crime tests progressive justice theories.  Editing and running code tests if your change works as expected.  Returns on your stock pick validate or invalidate your investment hypothesis. 

Social Feedback is when other people react to your beliefs:  friend’s horrified reaction when you question affirmative action policies; a VC agreeing to invest $1M based on your startup idea, or wealthy socialites writing a check for your nonprofit; applause after a speech, likes on Twitter, upvotes on YouTube, or real votes in a congressional campaign.

Only empirical feedback directly measures how accurately your beliefs map onto reality.  Social feedback measures how much other people like your beliefs.

This is a spectrum, and most people most of the time receive a combination of the two. Physics experiments check if your equation accurately predicts observations, but the story you tell with collected data can be more or less compelling in peer review.  Investors receive empirical feedback in the form of price changes and returns, and social feedback when raising money for their fund.  Driving a car badly might result in a crash (empirical feedback) or just annoyed honks from other drivers (social feedback). 

Here’s a highly unscientific illustration of where different activities fall between empirical and social feedback:

You can make empirical predictions driven by social phenomena – 'GameStop is going to the moon!' – or receive social feedback about empirical predictions – 'How dare you suggest that women are physically weaker than men!' You can accurately predict the popularity of an inaccurate prediction, or make an inaccurate prediction that is nevertheless wildly popular. The key question is whether the accuracy of the prediction or its appeal is being tested.

Both kinds of feedback are critical for learning. Much of our early development is shaped by social feedback, such as approval from one’s family, grades in school, or performance reviews.  Humans' ability to transmit and acquire knowledge without direct experience is a superpower.

Social feedback is a very useful proxy, but only empirical feedback ultimately tests how well our beliefs reflect reality. Unfortunately, it's impossible for most of us to receive direct empirical feedback about most things, as we can’t all run our own scientific experiments or verify every piece of reporting.

For most of human history, the necessity of interacting with the real world kept at least some of our beliefs in check.  Most of our time and work was spent directly interacting with the world - farming, gathering, laboring -  and feeling the direct consequences of our mistaken beliefs when we were wrong. A badly planted seed won’t grow. Poorly built shelters collapse. 

Today many of us are farther away from ground truth. The internet is an incredible means of sharing and discovering information, but it promotes or suppresses narratives based on clicks, shares, impressions, attention, ad performance, reach, drop off rates, and virality - all metrics of social feedback.  As our organizations grow larger, our careers are increasingly beholden to performance reviews, middle managers' proclivities, and our capacity to navigate bureaucracy. We find ourselves increasingly calibrated by social feedback and more distant from direct validation or repudiation of our beliefs about the world.

Social Feedback can be Hacked

As our model of the world is increasingly calibrated with social feedback, the empirical guardrails on those models are becoming less effective. The risk of going off the rails into unchecked echo chambers is rising.

Moreover, unlike empirical feedback, social feedback can be hacked.

Social feedback as a means to mass delusion isn’t new. Religions have long used social pressure or unverifiable metaphysical threats, like eternal damnation, to propagate beliefs. Totalitarian regimes enforce conformity through violence; apparently North Korea’s Kim Jong Un never uses the bathroom. The difference today is that many of our delusions, at least in the USA, don’t rely on forceful suppression or unfalsifiable metaphysics—they are both falsifiable and easily debunked. They persist when we voluntarily disconnect from reality.

Malicious actors exploit social media to amplify outrage and silence dissent, weaponizing social feedback to instill self-doubt, spread misinformation, and enforce narratives. Postmodernism-inspired activists increasingly proclaim that reality is a social construct, and attempt to rewrite it through aggressive social assertion. They narrow the concepts we consider, the beliefs we adopt, and even the words we use, regardless of their empirical utility for navigating the world.

The point of our knowledge, beliefs, and even our words is to make predictions about the world. A useful belief is falsifiable because it alters expectations in a specific direction. When, as individuals, we’re coerced into degrading our model of the world to align with social feedback, we become a little less capable at navigating the world. As a collective, when we suppress ideas with social feedback instead of testing them against reality, we all end up a little dumber for it.

Is a new concept useful for making predictions about the world, or is it propaganda?  Propaganda thrives via social feedback from the pundits and activists whose narrative it bolsters. Good ideas yield explanatory and predictive utility.

Can you measure “corporate greed” levels and use them to predict when prices will rise? Does a deep state conspiracy describe a government competent enough to fake the moon landing and hide aliens, but incapable of balancing the budget? Is gender identity or biological sex a better predictor of athletic performance?  

Which option among contested terminology makes it easiest to predict what a speaker is actually referring to? “Disabled” brings to mind a wheelchair, but “differently-abled” makes me think of Spiderman. “Enhanced interrogation” sounds like an interviewer on Adderall, not torture. “Undocumented” gives the impression that border security is a matter of missing paperwork.

One way to determine the trustworthiness of a source is to consider the kind of feedback that source receives. How incentivized are they to make accurate predictions? Scientists, investors, and engineers can’t do much with a broken model of the world, while politicians, influencers, and activists might depend on an audience that doesn’t notice or care when their predictions are wrong. We can dismiss the proclamations of anyone who isn’t even trying to accurately predict reality.

The more we lose touch with the empirical world, the more vulnerable we are to the influence and control of motivated propagandists. Nassim Taleb describes the dictatorship of the small minority by which a loud and organized group can coerce a more flexible majority to their demands.  Social feedback is a powerful vector for this kind of coercion because it's easy for a loud and motivated group of zealots to drown out the less engaged majority.

Social Artificial Intelligence

The need for empirical guardrails is particularly salient as we usher in the era of Artificial General Intelligence.

Large Language Models (LLMs) today are trained almost entirely with some form of social feedback. Pre-training often consists of next-word prediction on text from the internet; learning the sequence of words a person is likely to say without validating the content of those sequences. Fine-tuning often optimizes LLMs by showing multiple responses to human reviewers who pick their favorite version. Models are trained to mimic humans and produce agreeable responses based on social feedback; they do not learn by forming and testing the hypotheses described in their output.

Recent advances in AI models are incredible, but as they continue to play a bigger role in our lives and economies, I hope we ground them with deeper empirical feedback mechanisms. This might mean AI embodied in robots learning to anticipate action and reaction in the physical world, or AIs forming reasoned hypotheses about the future to validate against outcomes free from human biases.  Grounding AIs in empirical feedback might prevent AI from adopting our blind spots, and may even allow AI to uncover paradigm shifts that overturn our most foundational misconceptions.

Back to Reality

George Orwell wrote in 1984, “The Party told you to reject the evidence of your eyes and ears. It was their final, most essential command.”  While a single ruling party might not be responsible, our increasingly digital and administrative workforce means we calibrate less on our own real-world experience and more on social feedback. 

Kids these days tell each other to “touch grass” as an antidote to being “too online” - which makes sense, not because the world of bytes isn’t real, but because when we spend too much time online, we become overly calibrated to what other people think and lose our ability to make sense of the real world.  Grass is immune to the vicissitudes of online opinion.

The digital age and AI vastly increase our access to knowledge but also amplify social feedback and distance us from empirical guardrails.  This shift changes how we make sense of the world and enables new methods of pervasive mass delusion.  We must become aware of and prioritize feedback from the real world, lest we lose touch with reality itself.

Power Laws

Let's do a thought experiment.

The year is 1800.  You’re a loan officer at a bank.  Farmers come to you asking for a loan - maybe to purchase new equipment, buy more land, or make investments to improve their farm’s productivity. What are your criteria for granting a loan?

You’ll primarily want to do 2 things: avoid the losers, and reduce risk.  To avoid the losers, you’ll want to learn about a farmer’s reputation and character. Are they honest and hard working?  Do they have any major vices that might undermine their productivity? To reduce risk, you’ll assess the value of their collateral.  How valuable is the land the bank would repossess in the case of a default?

In short, you’re much more interested in minimizing downside than you are in maximizing upside.  Why?  Well, giving a loan to an excellent farmer probably isn’t going to make you much more money than giving a loan to a median farmer.  If a farmer has an incredible yield and produces 3x more than expected - which is very unlikely - you don’t get paid 3x the interest.  Your goal is to maximize the number of loans in your portfolio that don’t default. Most of your loans are given to farmers with fairly consistent profiles - similar plans, business models, skill sets, and characteristics.  You’re hoping for a consistent batting average, and you aren’t counting home runs.

Now, let’s jump to the present.  Instead of a loan officer, you’re a venture capitalist, investing in early stage startups, hoping to maximize the value of your fund.  How does this change your criteria?

As a VC, extreme outcomes drive the return on your fund. Accordingly, you care more about unique advantages and differentiators that give a startup a chance at becoming the winner in its category. Most investments will lose money, and many will go to zero, so the structure of financing needs allows for uncapped success when you pick a winner. You want to bet on startups that, if massively successful, could return the value of your entire fund or more.

The difference is that venture capital is ruled by power laws. Unlike a normal distribution, power laws mean that extreme outcomes drive the average, making the average very different from the median or most common outcome. This is how a VC can make money even though most investments fail—the average is pulled up by outliers.

In a power law world, landing big wins is more important than reducing downside in the median scenario.

Income tax returns follow power laws

The vast majority of the US Government’s revenue are taxes on the earnings of citizens.  Human capital is the primary asset of the US Government.  And those tax returns follow power laws.

According to the Tax Foundation, the median taxpayer earns about $47,000, and likely pays around $3,300 in federal taxes. The average federal tax paid is $14,279 - over 4x higher than the median. Extreme values at the top pull up the average.

Despite frequent proclamations to the contrary, high earners pay a disproportionately large share of federal income taxes.  The top 1% pay close to half of all federal income taxes.  The top 10% pay over 75% of all federal income taxes.

This does not mean we need to lower taxes on the wealthy. It suggests that any increased yield from minor improvements at the top are likely to dwarf the potential savings in the bottom or middle. Resources might be better spent helping more Americans become high earners without too much concern for breaking even on the median citizen.  Like a VC, we could focus more on increasing the number of home runs and less on a consistent batting average.

Despite this, attitudes across the political spectrum tend to focus more on minimizing downside risks—like the loan officer—rather than optimizing upside potential. On the right, the priority is often to limit spending on social services and encourage self-reliance with minimal government support, essentially cost cutting at the bottom. On the left, public spending aims to "level the playing field" rather than empowering high-potential individuals to maximize their productivity and raise the standard of living for everyone, overlooking the disproportionate gains to be had at the top.

In an agrarian society of the past, the “loss minimization” perspective probably made sense. For instance, would extensive formal education result in 10x returns from a society of pre-industrial farmers?  That seems unlikely. It was more practical to focus on helping the median citizen become moderately productive, and prevent too many individuals from becoming burdens to society.  

In today’s power law society, it’s more productive to focus on hitting home runs. 

Outsized Returns

Another way of thinking about this is to put yourself in the government’s shoes. 

Let's say I make you a deal: I’m going to pick a random high school graduate and give you their share of tax revenue, forever. 

What would you do to maximize the value of this investment?  Would you encourage taking the first minimum wage job available, or spending time learning and exploring to find a career in which they could thrive?  

Looking only at federal income taxes, this is how much your investment might return every year depending on which income bucket they fall into.

If you’re unlucky, and you decide not to invest in your graduate, the student might fall into the bottom 50% of earners.  Your graduate might make $30,000 a year as a cashier, and you might make $700 a year.  If you get this for 40 years, you’ll have collected $28,000 over your graduate career.

If your graduate manages to make it into just above the top 50% - perhaps as a truck driver making $56,000 a year, or a police officer making $76,000 a year - you might make $5000 a year, or $200,000 over 40 years.  This is already 7x higher, and neither of these careers requires a college education.

A graduate who makes it into the top 5% might be sending you $65,000 a year.

Here’s a sampling of options, along with what your yearly federal tax return could look like over the course of their career.

But of course, average stats for various careers don’t do justice to the power laws at play, because they don’t capture the chance of your graduate becoming a massive success.  If your graduate makes it into the top 1%, on average, they’ll be sending you close to $700,000 dollars every year.

Even distribution within the top 1% is also dominated by power laws - so this average is likely much higher than the median return within the 1% - again suggesting a focus on big wins.

What would a private equity firm do, presented with the same deal, for an entire underperforming school district?  Would they optimize for short term self-sufficiency, or invest heavily to produce as many high earners as possible?  It’s hard to know exactly how, but I suspect they’d find a way to turn at least some of those kids into home runs.

Focus on Home Runs

What would this look like?  Similar to a VC, it could mean focusing more on unique differentiators and prioritizing big wins over median losses.

This might manifest in many different ways across public services.

In public education, we might focus less on proficiency across a standard set of skills and instead cultivate individual superpowers to give students a chance at becoming top performers in specific categories. We might double down on gifted or high-performing students, fully recognizing the benefit they bring to society by realizing their potential.

While there are a wide variety of programs to support the jobless, including limited reskilling and retraining, many of these programs focus on immediate needs and self-sufficiency, and less on helping individuals become positive-ROI in the long term.  Instead of encouraging the jobless to accept the first minimum-wage job available, we might enable taking time to learn high-value skills, or starting a business. If even a small percentage of participants become high earners as a result, it could quickly recoup the overall costs of such a program.

Similarly, public discussion about immigration often focuses on perceived costs rather than the potential for tax revenue. High-skilled immigrants, especially those on H1B visas in sectors like tech and healthcare, often earn salaries well over $160,000.  Unfortunately, the demand for H-1B visas far exceeds supply, even though each additional immigrant in this category generates substantial positive tax revenue from the outset - enough to offset public spending for multiple individuals in the bottom tax brackets.

We might actively seek out and invest in underutilized talent or underemployed high-potential individuals. We might even allow anyone to apply for an open-ended grant to dramatically increase their own earnings potential, contingent on presenting a feasible plan to do so. Today’s internet makes it possible for anyone with the time and will to learn almost anything at a low cost. A relatively small grant might give someone on minimum wage—making $15,000/year—the free time they need to learn a new skill over six months that could quickly 10x their earning potential. If just a small percentage of recipients make it into high-paying careers, the initial investment would be recouped many times over.

These are just hypothetical examples that may or may not work in practice, but they serve to highlight the lack of policies focused on unlocking major potential gains at the top. While the best interventions might not be immediately obvious, any properly incentivized party would aggressively invest in finding out which ones work, because of the massive upside of any success.

In our power law world, helping a small proportion of the population achieve high-earning potential generates returns that can cover generous benefits for everyone else. Home runs pay for a lot of strikeouts. 

Human capital is the US Government’s primary asset, and in today’s day and age, those returns follow power laws.  We should start acting like it.

In a recent interview, Elon Musk remarked that American values have reversed, from “might makes right” to opposing strength and regarding weakness as a virtue:

It really has come completely full circle from—or 180 degrees from—what has historically been the case. Through most of history the operating principle has been “might makes right.” ... Now we’ve sort of flipped it to “if you’re weaker, you’re right.” But neither is true. There is rightness independent of strength or weakness. Just because someone is strong, doesn’t mean they’re right, and just because someone is weak, doesn’t mean they’re right.

Musk seems to be pointing to one example of a larger trend in our society, where it’s increasingly common to define oneself—or a movement, campaign, value system—in opposition to or as a reversal of something instead of towards something positive in its own right. 

We see this in every corner of our political discourse: anti-capitalism, anti-wokeism, anti-racism, anti-Zionism, anti-fascism, anti-immigration, never-Trump, cancel-culture, anti-vax, techlash. The problem is not that these positions are necessarily incoherent or wrong, but when adopted in the absence of positive values—without an internal inspiration that moves us towards one possible world over another—we lose the ability to grapple with complexity, to form nuanced perspectives, or to chart a path from our imperfect world to a better one.

Large Birds of Prey and Little Lambs

Nietzsche wrote extensively about how humans can find themselves trapped in a simplistic moral binary, and in particular, the dangers of defining one’s values in opposition to something else. He describes two kinds of value formation, represented by the “large birds of prey” and the “little lambs.”

The large birds of prey create values spontaneously as a direct consequence of their own experience. A bird “conceives of the basic idea ‘good’ by himself, in advance and spontaneously, and only then creates a notion of ‘bad’”—the positive values of the birds are primary, and “bad” arises only as an afterthought. It is “good” to eat the tasty little lambs because they’re delicious, so not being able to do so is “bad.”  There is no need to appeal to a universal morality—the bird weighs the world in relation to its natural inclinations.

Continue reading the full article on Quillette (no paywall): https://quillette.com/2024/02/15/beware-the-little-lambs/

This capacity to change and grow over time also alters our perspective on our own previous actions, so that deeds and decisions that used to make us proud can later seem petty or unenlightened in light of newfound wisdom. Since we judge everything in relation to our present perspective, the more we progress, the less alignment we feel with our past selves in comparison. 

We experience this as both individuals and as societies.

Individual Progress

For example, as a child, I once played a prank on a classmate by taping a pencil to his chair so that it would poke him in the butt when he sat down.

My friends were all laughing, and I thought I was hilarious.

After 30 years of learning, this isn’t exactly a moment I’m proud of. I mostly feel bad for the poor kid. I lacked empathy and prioritized getting a laugh over the well-being of another.

As individuals, we remember what it felt like to be our former, unenlightened selves. We can remember the confidence we once placed in bad ideas.

Comprehending our own transformation gives us a meta-perspective.  We not only improve our values, our beliefs, our wisdom, but we develop an appreciation for the learning process itself. We develop practices - like reading, journaling, and meditation - to accelerate our learning. We acquire humility as we realize how often we’ve been wrong and how much more we have to learn.

We might think about this journey in terms of Position, Velocity, and Acceleration. Position describes where we are today, velocity describes how we’re changing, and acceleration is created by forces that increase our velocity, to speed up our learning.

Of course, not all forces create a positive acceleration of our learning. Diseases like Alzheimer’s can reverse our velocity, pushing our progress backwards. Ideological dogmatism might push us to actively resist new information. Alcoholism or unhealthy addictions might prevent us from progressing as quickly as we would otherwise.  But for the most part, people move forward.

We might think about the forces that drive positive acceleration as engines and the forces that cause negative acceleration as brakes.

Small differences in acceleration make a massive difference in progress over time.

Collective Progress

Societies can also learn, but they don’t always do so.  The same parameters for a society might look like this:

For individuals, the very process of growing and aging tends to be positive velocity, even without any accelerative forces like formal education or deliberate practice pushing us along. However, most progress accumulated by most individuals isn’t new - we spend a lot of time learning things that others already know, and not all new knowledge is shared with the collective.  

Individual progress only leads to collective progress when an individual discovers something new and is able to spread it among the collective. Over time, societies have developed progress engines to do just that.  Public services like libraries and education pool knowledge so individuals can learn more without direct trial and error. Enlightenment values like reason and science established better ways to understand the world and solve problems. Institutions like free speech and freedom of the press allow individuals to report new findings and critique collective consensus without the backing of those in power.

Not all societies have an effective progress engine.  Often, those in power engage the brakes - tightly controlling the spread of information, promoting intolerance towards difference, repressing new ideas, or enforcing narratives - to reinforce a particular status quo.  When the brakes overpower the progress engine, progress grinds to a halt.

An individual’s experience only reflects the most recent iteration of values and knowledge that a society has achieved. Since individuals in a society don’t personally experience the learning journey in its entirety, it’s more difficult to form a meta-perspective. We don’t remember what it felt like to be a less enlightened society, and it’s difficult to notice the mechanisms underlying the gradual progress of a society.

If that society doesn’t progress, one might look back through history and feel proud of how that society has consistently lived up to its values, because those values haven’t changed that much.

On the other hand, if a society has a strong progress engine, an individual is likely to look back in horror, because their updated position is relatively much more enlightened. They might come to believe that their society is irredeemably rotten and question its ability to have any positive impact in the world.

Collectives, unlike individuals, lack the direct experience of who they once were and how they evolved. If a society’s progress engine is working particularly effectively, progress goes exponential, and even the recent past feels barbaric in comparison. 

Choose a Path

There are positive consequences of collective humility.  It’s likely for the best when a population exposed to history loses the appetite for reckless military misadventures abroad.  The progress engine is at work when individuals question the intentions of leaders, doubt the absolute moral rectitude of our justice, and identify pieces of society ripe for improvement.

However, there is a danger when a newly elevated generation - a product of the largest leaps of progress yet - can comprehend the gap between their values and those that came before, without understanding how they got there.  Without a meta-perspective, the very institutions that were necessary and critical for our enlightenment become suspect.  Without a meta-perspective, it’s easy to confuse pieces of the engine for dead weight.  

It would seem that there are really only 2 options. We can be born into a society with a weak progress engine, be taught similar norms and values to the many generations who came before us, and feel pride and alignment in their actions. Or we can be born into a society with a strong progress engine, with vastly improved values compared to previous generations, and feel shame at their ethical shortcomings, unconscionable norms, and moral crimes.

This leaves us in a strange juxtaposition where societies at the pinnacle of human progress are the most ashamed and full of self-doubt.

For my part, I feel very fortunate to be in a society with such a strong progress engine that many individuals feel shame towards our past. I hope we continue to critique our culture and fix our flaws, without breaking the institutions that enabled such rapid progress.

Because at the end of the day, we are always exactly here:

We’re somewhere in the middle of the story, not at the end.  Which means we still have work to do.  

If we do our jobs right, the next generation will be very ashamed of us.