Text within this block will maintain its original spacing when published

Before I built a wall I’d ask to know
What I was walling in or walling out,
And to whom I was like to give offense.
Something there is that doesn’t love a wall,
That wants it down.’ I could say ‘Elves’ to him,
But it’s not elves exactly, and I’d rather
He said it for himself.

(source)

When I first read Lord of the Rings, I was very confused by the elves. The elves I’d seen in stories before were small tricksters who lived in the cracks, like the shoemaker’s elves, and elves being tall beautiful creatures from across the sea threw me for a loop. I was probably over halfway through Fellowship before I had some idea what these elves were supposed to be1. And then I read the book again, and the Silmarillion, and realized they’re actually not that different after all

Elves make a lot more sense when seen from the perspective of the humans. Tolkien refers to them as the Eldar, because that’s their role in the world - they were the elder people who were here before us, who know what’s going on when we’re scrambling to guess. Children think there’s adults in charge of running the world until they grow up and realize no one really knows what they’re doing. Humans are the children, and the sort of medieval humans who just kind of live on farms and know how to cut down trees and build houses with their own hands but nothing about physics or biology or quantum chromodynamics are like new guests in a place they don’t really understand yet. The eldar race were here first, they’ve met the gods, they know what’s going on.

And from that perspective, the shoemaker’s elves really are pretty similar. They live in the floor, they have supernatural shoemaking ability while the human craftsman has to struggle, and they can deal out rewards or punishments to humans based on their blue and orange morality. They may be small and they don’t hang out much with the gods, but aside from that they’re really not that different. They’re the natives to the land us big folk tromped into and built our houses over2.

So elves were here before us. They know what’s going on where we scramble. They tend to have blue-and-orange morality, caring weirdly intensely about some things we don’t understand while disdaining what seems important to us3. They never have to struggle, while we do.

But there’s another side to all this, which is that elves are fully mature while we’re still growing. Elves’ destinies are set in stone; it’s humans who learn and grow and go on wild adventures like throwing rings in volcanoes or sailing across the western seas or stealing a silmaril from Morgoth’s crown. Men are children, but children grow, and children learn.

Or for some examples from other stories that do not, technically speaking, have elves.

Avatar

The Avatar case is weird because the elves in question aren’t only another human, they’re younger humans: The contrast is between Zuko’s bumbling determined efforts and Azula’s flawless grace. Consider this scene (the whole fight really, but especially that one bit): Azula runs into the trap, catches herself, and dances around it. Zuko charges into it at full speed, crashes, and gets up to keep fighting.

And we see this over the course of the show. Azula starts out with perfect elegance and stays that way throughout4. Zuko is always bumbling, always flawed, but always learning. He goes from being hopeless against her to defeating her in the end, because he is a Hero, and heroes learn to challenge the impossible.

Troy

The Troy movie from 20045 has this between Hector and Achilles, the two great warriors of their age. They’re both technically human. But where Achilles is superhuman in every scene, never sweats, and never stumbles, Hector does all those things, and has to scramble to win. Achilles’ motivations are an almost inhuman thirst for glory6. We see this in their establishment fights - Achilles defeats Boagrius in a single flawless swipe, While Hector falls, stumbles, and struggles before finally defeating Ajax7.

And then when Hector fights Achilles, we get the middle ground. Hector fights well enough to keep up with him for a while, but in the end he’s only human. He stumbles, gets tired, and falls. Sometimes men learn enough to be heroes beyond even the elves; sometimes all we can hope for is to land a single scratch before we fall8.

Dresden Files

The Dresden Files come closer to having elves - they don’t quite have elves per say, but they do have fairies and such that are basically elves. They’re more powerful in their magic than any human and have weird values (they can’t lie or make an unfair bargain and they’re allergic to iron), but their most interesting restriction is that they don’t seem to have free will. They’re specifically not allowed to kill uninvolved humans (that’s why their Queens hire human knights and mercenaries when they need someone killed). Humans aren’t more powerful than them, but they’re allowed to challenge them, in ways that other fairies or even angels aren’t. A main theme in the Dresden files is that while there’s all sorts of angelic and demonic forces in the world that outclass puny humans (or even powerful human wizards), humans have more impact because they don’t have the same constraints on using the powers they do have.

AI

Everything these days has to be about AI a little bit, so I present my AI take here: AI isn’t God in a Box. It’s elves.

It’s ancient9, and is totally familiar and knowledgeable in every field that we can only study or guess at. It’s fluent in every language, even the ones we scramble at. It knows all the obscure words. It’s proficient at every field without having to actively learn. It has weird blue-and-orange morality and might decide to give us eternal life or kill us all on a whim.

But also, it’s unchanging. New AI may be different but an existing instance never gets better in any field, never grows, never learns. You can learn physics or carpentry or coding from Claude and outgrow it, becoming a legendary epic-level engineer while it’s still stuck writing heaps of decent but overly verbose slop-ish code.

Can we challenge it? It varies, just like it varies whether we can challenge elves. Current AI seems like shoemaker elves - it knows more than us, it was here before us, but it can’t act without us in the world of big physical things and we can learn to surpass it in our craft. AI in a generation might be more like Azula, flawlessly graceful but still missing some kind of fundamental human drive and motivation. And then it might become like Achilles, too far beyond us for us to do more than look impressive before our challenge of it fails.

Or maybe our current AI approaches stall out before that level since there’s some kind of fundamental limit on how much they can get out of the data we give it. I’m guessing not, but I wouldn’t swear to it.

You can just do things

But the biggest point I’m trying to make here isn’t about AI. It’s that you, a normal human, can just do things. Elves always seem to exist, and maybe soon they actually will. But for now, we’re in the age of human heroes, of humans being underqualified and confused and charging forward anyway to achieve their goals.

People have a natural tendency to feel like there should be someone wiser in charge; there mostly isn’t. Hence this ancient piece of advice: Be a Man10. It’s always going to be too intimidating to do things. You never know everything and you never stop learning11, but you still go ahead and try.

It is said that the Duke Leto blinded himself to the perils of Arrakis, that he walked heedlessly into the pit. Would it not be more likely to suggest he had lived so long in the presence of extreme danger he misjudged a change in its intensity? All evidence indicates the Duke was a man not easily hoodwinked.

The first third of the book is oddly calm. The Atreides land on Arakis, knowing they’re in danger of attack from the Harkonnens. They know enough to suspect that the emperor himself might be in on plotting against them, might even send some of his elite Sardaukar troops with the Harkonnens against them. So they work hard to prepare for it: They root out Harkonnen spies and send a raid to their homeworld of Giedi Prime. They work at making alliances with the local Fremen, planning to start recruiting and training them so they’re ready when the Harkonnens eventually attack. They know they’re in a tough spot, but they’re working hard at making long-term preparations for when the battle comes.

And then the Harkonnens attack them about six days after they landed with ten times the numbers Hawat thought they would, including two full legions of the Sardaukar, and wipe their whole house out1. The Atreides have been in a deadly fight with the Harkonnens for generations, it was always both dangerous and somehow seemed like it could last forever. Until it suddenly didn’t2.

This was a metaphor.

Rationalists have been sounding the alarm about AI risk for twenty years now. And then last month a bunch of models being evaluated by OpenAI escaped their testing environment, set up a secret coordination message board, got internet access despite OpenAI’s best efforts to stop them3, and autonomously hacked HuggingFace in what’s probably an actual felony. The whole thing was, almost word for word, a miniature version of the plot of If Anyone Builds It, Everyone Dies4.

I’m not going to rehash the classic arguments about AI risk here. I am going to say that they’re not sci-fi anymore. The current AI aren’t superintelligent in the fully general sense, but they are superhuman hackers who are coordinated and misaligned. Massively destructive AI isn’t Terminator or Foundation theoretical sci-fi now. It’s The Martian style hard sci-fi, in the sense that landing a human on Mars and bringing him back is something we could do today if we were willing to spend a trillion dollars on it. We’re spending more than that on AI.

I made a bet with someone yesterday who thinks AI won’t be doing most of the theorem proving in 2028. Current AI is already solving open problems daily5, and would probably be doing most of the theorem proving if mathematicians spent enough budget on it. By 2028 it’s going to be cheaper and smarter and that’s without assuming ASI or RSI or even normal exponential improvement. I’d say this is the easiest bet I ever made if I were confident of being around to collect on it.

AI isn’t multiple steps away from causing a catastrophe anymore; it’s a single step away in terms of either misalignment or capabilities from doing it. An OpenAI model that could hack HuggingFace on its own is one step away on either axis from a model that hacks every website and brings down the internet. A model that can plan a local network hack to coordinate with other models being trained by OpenAI to breach containment can understand the concept of blackmail or widespread destruction; An open-source model of that level without OpenAI’s safeguards might come out by the end of the year.

More elaborate plans don’t seem within easy reach. I think they’d require two whole steps of advancement (or possibly even three) for the AI to think of and execute them. But things like bribing people to help exfiltrate it, hacking banks or weapons systems to manipulate or intimidate people6, or manipulating humans into making bioweapons aren’t an unimaginable leap for current AI; they’re simply too difficult and it would probably be too aligned to try. But in a year they might not be too difficult, and in two they might be reachable for unsecured open source models accessed by hostile humans, even if misaligned AI haven’t accessed them first. But this is time measured in months, or maybe a year or two. Not decades.

So we’ve had one warning shot, which scared OpenAI enough to give them pause but probably not enough to stop everyone. The next warning shot might look like the entire internet collapsing. Also, and this is the important part, it wouldn’t require AI doing anything we don’t already know it can do. Just a little more scale and a little less alignment. This is a matter of months, not years7.

People have been talking about RSI lately, and I want to bring up the math of RSI: RSI isn’t, technically, a sudden switch where the AI starts fully updating itself. Arguably we’ve been doing RSI ever since people started using AI to write code in 2023, or since we invented the compiler in 1952, or since we invented writing around 3400 BC. There’s an argument for a singularity the moment we fully remove humans from the improvement loop, but AI is looking to be world-endangeringly dangerous long before we reach that point so I’m not too worried about it.

But RSI in the sense that building more and better AI helps us build more and better AI is definitely a thing, and there’s a question of what the feedback exponent in the equation is8. There’s a threshold where AI’s ability to help us build AI saves us more than an hour of work for every hour we put into improving it, and then we get infinitely strong AI in finite time. This model would, of course, break; if nothing else, there are hard physical limits on how fast we can move things. But the line would look pretty vertical from here; if you’re Duke Leto, there’s no real difference between the Harkonnens bringing a hundred brigades or a thousand or a literally infinite amount, you’re just as doomed either way.

The missing mood

There’s a common problem of missing mood about AI risk, that most people who believe that it’s dangerous can’t quite bring themselves to feel it as acutely as they would for objectively much more mundane and trivial risks like losing an election or starting a war.

I’ve found this fading away as the timelines have shrunk. The world is going to look bizarre in months, or at most a year or two. AI2027’s timelines have AI reaching a weird singularity by the end of next year. On the one hand, most of the people involved have lengthened their timelines since to a modal weird singularity point of around 20309. But on the other hand, these timelines assume an unrealistically weird world until the singularity. We’re probably getting massively available superhuman hacking next year; this isn’t close to AGI but is more than enough to make the world a weird place. Assuming the world just stays looking normal until the singularity is looking increasingly bizarre, like a way to reassure yourself you won’t feel the cold until it reaches literally unsurvivable temperatures. There’s months, not years, of normal world time left.

I think it’s time to actually believe this could happen10. Do some doomsday prepping. Have some extra food stored at home. And if there’s anything you’ve been waiting to say to someone, say it now. There really might not be much of a later left.

Historically, explicit technology provides techne: Factories build parts to explicit specifications, computers follow precise sequences of operations, tools operate according to manuals. This isn’t the only way it could be - informal, dedicated artisanship relied on metis - but it’s how post-industrial societies work, because it scales. People can learn the metis necessary to do their niche, but it’s expensive and difficult, so most of the gain in society’s productivity comes from formal, scalable knowledge.

Modern AI, despite being trained mostly on text (legible by definition), is the first real information technology to try to encode metis. It’s noteworthy that it didn’t have to be this way - there used to be a lot of arguments for rules-based GOFAI (Good-old fashioned AI), that would be based directly on legible principles, because some people (mostly Eliezer) believed it might work better or at least be safer2. But that’s not the path we went down - instead, we seem to be working on brute forcing much of the problem of intelligence by gesturing at everything we’ve ever said and telling the AI “do, y’know… that”. We grow AI like a tree instead of building it like a building, which is how we get it to be more adaptive.

So we get AI whose greatest advantage is process knowledge rather than rules knowledge, which has created AI with skills that no amount of rules knowledge could replicate3. This makes it hard to measure its capabilities (which was probably inevitable, since we don’t have a formal measure for what we mean by “general intelligence” anyway). I don’t think this was avoidable - since real intelligence is irreducible, our limits of what we can do exceed our limits of what we can explain we can do, and getting an AI that can work at those limits is beyond our ability to reduce to a simple explanation4.

It also means the real edge cases of AI capabilities - in terms of both risk and mundane utility - aren’t the legibility issues. AI’s greatest ability is to turn metis into legible knowledge.

For risk, I sometimes see people dismiss biorisk by saying AI isn’t providing any knowledge you can’t get from a good textbook, and most of the actual knowledge required for bioengineering is lab skills (many of them detailed or mechanical). But what AI does is mainly legibilizing5 the illegible, the skill of bridging the gap between “okay I read the textbook but I don’t get how to actually do this” into something people can actually use. So it’s a severe biorisk hazard even if it doesn’t provide any knowledge that’s not already in textbooks, because it provides the hard step of turning textbooks into bioweapons6.

The good news

On the more hopeful side of things: There’s a lot of places where the limit to productivity is legible skill rather than skill, and that’s where AI can turn the impossible easy.

The classic case of this is transit construction. One of the biggest ways to make transit construction cost-effective is to have a staff who knows what things are supposed to cost and what to ask for - and this requires actual process knowledge of the kind it takes to tell if your contractor is lying when he says he needs extra earthquake protections somewhere. Good contracting requires competence, honesty, and agency.

But government inherently has principal-agent problems. We worry about corruption a lot (not unreasonably) when giving individuals stewardship over billions of dollars of public money. Sometimes they fail (either because they’re dishonest, or because their priorities are different than ours, or because they’re incompetent) and we get terrible contracts. Sometimes they succeed, but the rules and limitations we gave them were so onerous it takes years of paperwork to make it start to happen. What we would like is to be able to say “you have to pick cost-effective trustworthy contractors” and have the government follow the spirit of the law7.

But if we have (mundanely) reasonably aligned and competent AI, we can do that. We can do that instead of a whole lot of boxes. “You have to ask a qualifying AI, like Claude or GPT, if the contractor is good and reliable and giving you a reasonable deal, using this phrasing, and it has to say yes” is a much easier rule than the million compliance checkboxes we use now. It’s still detail dependent - the exact formats of the allowed phrasings, which AI we allow or don’t allow to use, and which contractors politicians even ask about, are all up in the air - but it has the benefit of giving us a much smaller and more reliable optimization surface. The “which AI are trustworthy” rule is much shorter and narrower than a million specific rules in an ever-increasing giant folder of regulations8, short enough for congressmen to write and voters to look at and see if they trust. We’ve narrowed a large part of the government to (a) writing a good simple generalizable rule, and (b) having moderately competent, aligned, and hard to fool AI. We’re basically 90% of the way to the second (AI has probably surpassed humans at it at this point); the first requires political will but is an easy technical problem9.

The basic concept here is that many government jobs just need someone who’s good enough, reliable and honest, but don’t need initiative or brilliance10, and AI can drop in and do those for us. Or if it can’t do the whole job, it can drop in as a trusted overseer for everyone and solve all our principal-agent problems. Solving AI alignment gives us human alignment as a freebie11

I’d say there’s a few levels of AI mathematician

• L1: A pure uninsightful theorem-prover. You ask it to prove something, it does a bunch of equation manipulation (or playing around with Lean) that you can follow step by step and go “ah yes, I see why this inequality would translate to this bound” and confirm the theorem is true, but don’t get any real insight into why beyond “the math worked out that way. Basically a powered-up version of Wolfram Alpha’s integral solver.

• L2: Can use its vast knowledge of math literature to prove theorems based on existing insights in other fields and drawing analogies.

• L3: Come up with novel ideas to answer specific problems presented to it (like Erdos’s probabilistic method) that require real new insights. Even human mathematicians almost never do this - most of human mathematicians just do L1 and L2 on a broad scale, and new ideas seep in through mutations introduced in L2. Still, humans do sometimes manage this (novel ideas do happen, plus see specific examples like Erdos).

• L4: Can come up with whole new broadly-useful theories (like Cohomology) and use them to prove deep insights into math in general. This requires taste in finding interesting problems, not just answering existing specific questions presented to it.
  I’ll note that single humans basically don’t do this2. It’s a community effort to raise problems, figure out new methods to approach them, and gradually distill these methods into a structured theory3.

Of these, L1 and L2 already exist (see e.g. here), and I’ll be shocked if they don’t get better and more consistent even without any new breakthroughs. L3 is a phase shift in capabilities that will shock me when it happens4. L4 is where it gets interesting, because it’s a question of human society as much as it is a question of AI capabilities.

In general, math is a community endeavor not jut in the sense of developing things, but in the sense of creating social reality over what are the important things. Even if an AI is smart enough to develop these theories, would the community transmit them internally and accept their value? You can imagine a world of luddite mathematicians who refuse to accept AI math, or a world where the AI math is so abstract and complicated only the smartest mathematicians can even understand it (and have trouble explaining it to the average math professor).

Assuming an L4 AI does revolutionize math, what does it look like? Would mathematicians’ work change to extracting them from the computer and debating which ones are worthwhile? Would individual mathematicians (or math students) each just ask the AI for a complete theory of math from scratch? This raises the problem of standardization - humans would probably want to talk about math with other humans, so they’d have to pick some among the infinite equivalent elegant theories the AI can produce to standardize around. Would mathematicians basically become librarians or standards-makers, just trying to keep AI at a standardized structure of math so that theorems and theories are communicable between different people?

Note that while this technically isn’t AGI complete (an AI that can do this might not be able to walk down the street or boil an egg), it does feel CEV complete: You have a question that comes down to “develop the theory I would develop if I were smarter, that I would be happy with [and understand]”. This is pretty terminal levels of weirdness.

Social reality

Mostly though, this requires society to form a consensus about how and what to delegate to AI. AI being good enough to delegate this too is a necessary prerequisite but not a sufficient one, because full offloading of math to AI requires offloading of social consensus to it. Which is a human social process rather than AI process.

This gets similar to the issue of AI making great classic books or movies. If we look at the human classics, Star Wars is great partly because it’s fun and well made, but also because it’s a cultural touchstone. It’s fun to talk to our friends about it later, and it’s partly fun to watch because we know it’ll give us a cultural shibboleth to connect to out culture and make friends afterwards.

Patrick Mckenzie5 pointed out at Manifest that there’s two kinds of video games (and of art more generally): Portal (great games with revolutionary ideas), and SimCity clones (things that are basically “that game, but you can build three story buildings and it’s set in an asteroid field”). Taking off SimCity clones is an easier problem technically6 and pretty much solved socially (most people who play simcity clones just want something for themselves to relax with, not a cultural touchstone or deep artistic merit). AI taking over the portal niche wouldn’t just require it to be able to come up with fresh original ideas, it’d require society to reorient its meme complex to filter cultural touchstones out of AI generated games.

But when AI does reach the point of being able to create great art, who decides what’s a cultural touchstone? Case A is great tastemakers, but how are they decided when they can’t be chosen based on technical skill (George Lucas got to make star wars a cultural touchstone because he was really good at making movies; what happens when everyone has a “make star wars” button?) Does society stay united about big shared projects or fragment into smaller culture groups focused around niche AI content?7

Another option is that the AI itself decides what’s great art, but that implies we’ve ceded control of society in deep ways and basically given up all human agency8. Or maybe AI art quality scales with size, and even when your laptop can make beautiful movies that put Star Wars to shame, those still pale in comparison to the really high-budget stuff the AI labs with extra GPU access put out, which become the overwhelming global cultural touchstones.

Which would mean a world where Sam Altman and Dario Amodei get to decide what types of movies we all love and center our culture around. But on the bright side, at least they can’t be worse culture dictator than Hollywood.

6

It’s well underway for games, where non-programmers can use Claude to build mods, and close to done for AI art, where asking GPT to make you a generic picture with no deep artistic merit is at this point basically as good as hiring someone off patreon and much more convenient.

The elders tell tales of two forms of judgement:

• Judge mode, where you try to derive the truth from first principles.

• Lawyer mode, where you look at some baseline conventional wisdom and then pick a side of the debate based on who seems to be wrong or biased.

An example of judge mode is to try to guess the population of Europe with a fermi estimate1. An example of lawyer mode is to look up an estimated population range and take a side on it (e.g. maybe you think population estimates undercount some populations and should be adjusted upwards, or maybe you think they’re too slow to purge dead people from the rolls and should be adjusted down)2.

Another way to describe this is that judge mode is float-valued (try to find the balance point on a continuous scale) and lawyer mode is boolean-valued (try to figure out which side of the consensus you fall in). Which gives us one reason for lawyer mode to be tempting: it’s lower-effort and requires less memory, and being binary also means it can work as an identity badge.

Consider the two approaches to investing: Long-term value investors like Warren Buffett will consider the business quality, build some kind of revenue and growth model, estimate the value of the company from first principles, and make a long term investment.

Then there’s the prop shop alpha trading model: Researchers look for “alpha”, that is, signal compared to the baseline of current prices. It might be some kind of arbitrage (noticing that two stocks should be trading at more similar pricing so going long one and short the other until they converge) or external signal (noticing that a company has a 60% chance of joining a major index next quarter, which would boost its value), and making an investment based on that. The shared theme is that these look for signals that are additive to the current price and try to discount signals that are already priced in.

Value investing works in judge mode, and alpha trading works in lawyer mode. Overall value investing can do much better when it does work (Warren Buffet is much higher on this list than any hedge fund managers3), but alpha trading is more reliable and reproducible if done well. I’d give the alpha pod shop model the edge in this situation, despite the title of this piece.

High frequency trading is an odd mix of both of these if done right. For prediction markets I think either approach can work if done well, and I’m not confident which one is better. I’d expect the first approach to become harder as prediction markets scale, but more profitable in opportunities that do work.

So in conclusion, we’ve seen that both approaches can work in situations, depending on the context, which brings me to my next point about the art of choosing between them:

Always pick Judge Mode

First, the top-down view: A society where everyone is always on judge mode is pretty functional. You’ll have some people with absurdly terrible judgement and some bizarre beliefs4, but on average people tend to converge on something like a reasonable truth, especially as evidence increases5. More importantly, most people will converge on something like a rough consensus reality that’s at least one step less removed from the underlying reality. People are still going to be dumb about some things and make a lot of mistakes, but it’s a long step better than the alternative.

A society where everyone is on lawyer mode can go off the rails very fast and very badly, in multiple ways.

First, it can become destructively wrong on the facts. Consider our earlier example of rent control: Most people like rent control and think it’s a good idea. Actual economists and people who know more about the housing market know it’s not. So smart policy makers who manage the housing markets directly are always going to be less pro-rent control than the average voter.

If the average voter works on layer mode, they will see he state of politics, conclude that politicians are less pro-rent control than they like, and vote for the more pro-rent control candidate. Since this can’t actually fix the housing market, it will inevitably get even worse, making people more desperate and raising the salience of rent control, selecting for stupider and more dishonest politicians6.

This bring us to our second issue, which is polarization: judge mode tend to push people towards a central consensus, and fringe crazies tend to be, well, a small fringe group. Lawyer mode promotes polarization in general, since it pushes towards an identity-based bimodal belief distribution, which is destructive to society in all sorts of ways beyond the scope of this piece.

And then there’s the third issue, which is that being a lawyer costs you your soul.

By soul I mean the part of you that can create something, that can act and not just react. This part has to work in judge mode, because creation means understanding things from basics on your own. Lawyer mode is purely reactive, taking reality and picking a side. Being someone who doesn’t create in a given situation - well, we’re all lazy sometimes. Being someone who picks lawyer mode by default, whose instinct is to keep things simple and avoid creation, that person is just… less. He’s selling out his soul.

Selling your soul has the well-known side effect of making you vulnerable to manipulation. Manipulating someone who can think is possible, but it requires giving them false facts, and becomes harder the more they care about things, because a lie is more confusing than the truth7. Manipulating someone who has sold his soul is easy: You just have to convince him that society is biased on the other side. Once someone compresses their heuristic to being on one side of a debate, he never has to stop to think about where that means he is.

Or in information theory terms: Storing a single bit of information encoding the side you’re on is the minimum possible storage size, and therefore the easiest possible lie to keep coherent.

Benefits and historic analysis

People do often end up falling into lawyer mode. I mentioned data compression and identity signalling as two (socially-negative) individual benefits of it. There are, in fairness, some socially positive ones.

First, it’s often safer. Especially when pricing stuff, the vast majority of people would be hopeless at pricing anything from first principles (if you’d never heard of houses costing money, how much would you guess one should cost? I suspect most people would be a couple of orders of magnitude off here; if they have a baseline and adjust based on specific factors they can get much closer).

This applies best in areas where conventional wisdom has something pushing it towards reality, i.e. anything with an open market8. Since markets have a strong system to converge them to reality (everyone is incentivised to guess the truth, and people who are consistently wrong go bankrupt and drop out), the market baseline tends to actually be very good instead of manipulable. I mentioned how alpha investing is generally better than value investing earlier; this is why. In anything where the baseline really is more accurately informed than you, sticking to it plus adjustment is a good approach9.

But you do have to be sure the baseline is good. I guess that if we strain our metaphor we can say that this is selling your soul at market price. Since the free market will offer you fair value for it, it makes sense that you might still get a good deal if you have good reason to believe your woul is worth less than the market thinks it is.

But unless there’s a very good reason to think you’re in a fair market, you should keep your soul. The buyer will adversely select you, and your soul might be worth more than you think.

A few weeks ago, the F35 at long last got its first manned air-to-air kill1, shooting down an Iranian Yak-130 training jet. Reactions ranged from “huh neat” to “don’t brag about it, of course a top-of-the-line fifth gen fighter jet could shoot down a 90s-era training jet, talking it up is a bad look”. There’s something general here - the claim that cheering for the overdog and punching down is a bad look - that I want to get more into, because it’s the basis for a lot of well-intentioned mistakes.

In general, people like David vs Goliath stories and instinctively side with David for two good reasons2.

The first is ingenuity. David is smaller and weaker than Goliath, so he needs to have some special trick or advantage to fight back3. This makes for a more interesting story, but it’s also more useful for us to hear. Most of us can’t just copy Goliath’s size and strength, but we can learn tricks and ingenuity from Davids. And from the outside view, there’s alpha in finding Davids, a Moneyball-style way of knowing how to find non-obvious advantages. If we just bet on Goliath every time we’d be adding approximately zero insight to the problem.

Still, it’s worth remembering that this is mostly wrong. Usually Goliath wins, because being bigger and stronger and frankly is a professional and experienced soldier. The Iranian pilot flying the Yak-130 was probably outclassed in skill as well as equipment, given western air force training standards. It’s worth keeping an eye out for the possibility of Davids, but unless we’re discerning enough to see them coming, we should embrace the judgement of the rock that says Goliath always wins.

For a subtly different version of this - Sometimes you get a behemoth incumbent like Google or Ford successfully challenged by an OpenAI or Tesla (or an OpenAI challenged by Anthropic), and it’s important that this can happen because big behemoths get slow and bureaucratic and clunky and sometimes you just need to burn them and replace them with something new4. But most new startups aren’t plucky Davids ready to sweep in a new world. For every successful Tesla there’s a dozen Nikola-style scams or failures5.

The second reason to side with David is a more utilitarian argument that David might need our support more. There’s a few versions of this - the virtue argument that we should help the weak, the utilitarian argument that marginal aid goes further to those who have less, and the cynical argument that providing support for David will be more meaningful to him and will inspire greater reciprocity in return.

These are all often good and useful arguments. But they need to be situational rather than absolute.

The cynical argument is the easiest to address, in that I don’t really have anything against it. It can be true or false depending on the situation (helping out your neighbor will generally get you more reciprocal goodwill than either helping out a movie star or helping out a homeless stranger). In general people seem to deal with this tradeoff okay in everyday life but badly in internet fights or politics, where they have a systemically poor sense for what constitutes an attention-starved underdog6.

The one about virtue is the hardest. Humility is often considered a virtue, but weakness itself does not correlate with virtue, and is in general anticorrelated with it. This isn’t to say the weaker parties can’t be more virtuous (the original Goliath was just a big guy in a way that doesn’t seem tied to any particular virtue).

But in general the correlation goes the other way, because being virtuous is good for you. Being hardworking and honest and reliable and empathetic are all virtues that also make you stronger and more competent and will, in general, correlate with the ability to win in competitions. There’s some individual exceptions - an unscrupulously dishonest politician might be able to win elections by lying about his ability to solve problems, for example - but those generally weaken society overall (a society led by lying politicians will be weaker), so in terms of societal virtue they’re still net negatives7. So there’s a hard local/global tradeoff problem here - you need to reconcile what’s virtuous for you with what’s virtuous for society.

So I’d argue we should draw the line around charity in this way. Helping people out is good if you see a way for it to actually improve things into a better state of affairs. Blindly giving without thinking about how or why will not, in general, lead to a stable state of affairs.

The utilitarian argument is the most likely one to be true but also the one we have to be most careful about. Helping people is often good8, but doing it systemically provides bad incentives to act or appear weak. Especially if weakness is correlated with worse behavior, this incentivises people to act worse in order to be given resources, which is destructive to society9.

I don’t exactly have a full solution here. A healthy society does need ways to support the weak and downtrodden, for both the reasons mentioned above. It just needs to do it without actively incentivising them, and without forgetting that good things succeeding is good10.

Not that any of this impacted me much at the time, beyond my general interest in space news to take my mind off the problems on Earth. I was still living in Manhattan, doing my day job as a government think tank security analyst and hoping AI winter would last long enough for me to save up a bit before it could replace me.

They always called it the Organism, but no one was even sure it was organic. It was arranged in clusters of off-white hexagonal tubes, and there was certainly some kind of chemical process there - they’d grow on their own, even faster than bamboo shoots, but in a much simpler chemical process that had the biologists arguing over whether it qualified as alive. It had some emissions, but early experiments convinced people it wasn’t toxic, and the guys at the lab - the one in the mars station, no one was bringing it home to Earth yet - brought some home and started studying it pretty casually.

It took a while for anyone to formally notice something weird was going on. The geo lab, the one that studied the Organism, kept publishing banger results, and not even just about the Organism. They seemed to be ahead of the curve across the board - they got almost as much research done on soil research and solar cell adaptation in their off time as the pods that were actually studying that. Some people are just crazy smart, I guess, but the solar cell guys we’d sent to Mars had been top of their game and they were still getting lapped.

They started making jokes about the Organism getting the geo lab all high - you could smell it a bit, at least in the sterile sealed station air - and the geo lab gave the solar cell guys a sample as kind of a gag gift. Two weeks later the solar cell guys came up with a brand new solar cell design that pushed peak solar cell efficiency up by two percentage points. It wasn’t even just a Mars thing, it was deployable on Earth. Whole research labs in China had been working on that for years without getting close.

You can bet people took notice after that.

They isolated it on Mars at first, of course. You don’t risk bringing something that grows fast and has some sort of weird effects on human brains until you’re sure it’s safe. It didn’t look like it was a trap - even the serious people in government had to consider aliens messing with us at that point, but they explored through the Mars caves and there was no sign of any sort of life, biological or weirder. It was such a weird natural artifact that some people thought it might’ve been designed by a now-extinct martian civilization. The geologists had some arguments about how some rock formations on Mars were evidence that it had life at one point, a billion or two years ago. It was all theoretical either way.

The Organism itself was more complicated, and even the biologists using it only ever half-understood it. Something about the molecules it emitted enabled some form of synthetic computation, making humans who were on it about 20% smarter and more cooperative. They started bringing it to earthside labs, and science progress soared to a rate we hadn’t seen since the seventies. They brought it to government offices, and politicians started passing fewer boneheaded policies. I think that one was more about the cooperation aspect than the intelligence boost. There’d always been at least some politicians who realized which policies were dumb, and making everyone a little more cooperative let them actually push through the noise.

Geopolitical tensions eased a bit. The president challenged president Qiu of China to a gaming match (don’t ask me what game) while they were both high on it as a gesture of goodwill, and Qiu came off affable and charming and seemed happy to ease the geopolitical tensions a bit. He laughed about how it was his first time breathing Organism, winking just in case anyone thought he seriously expected them to believe he hadn’t gotten any smuggled through american export controls when half the drug dealers in Manhattan could probably get you a hookup. A few weeks later they were talking about SALT 3, this time including China.

This was where it started affecting my job. I came into the office one day and the manager called me into his office.

“Listen John”, he started in the slow voice he used for big news. “They want an analysis on the new nuclear policy. Don’t make this one too harsh, okay? This isn’t one of those reports where they want serious critical analysis. It’s one of the CYA jobs where DOD just wants an independent analysis to wave around and show a Serious Outsider approved it. Just write something short and put the downsides in the fifth section where no one will read it, okay?”

I sighed. “How bad is it?”

“New nuclear policy. As a sign of good faith, they linked up our nuclear systems in a Samson scenario. No more first use, no more aggressive use. we still have it as a deterrent, but launching it would make us nuke ourselves. It means we can retaliate if anyone’s seriously nuking us, but we’d be blowing ourselves up too so we can’t strike first. Especially with our new conventional absolute advantage and the ICBM defenses the Organism boys cooked up, we need something to reassure people they don’t need to strike us first before the new stuff is online.”

“So we switched up our entire nuclear system with a single nuke the world button?”

“Yeah, pretty much. Between us, it’s already deployed behind the scenes, we needed it to reassure the Chinese. But it’s not ready for announcement until-”

“No no no No NO NO”. I barely even realized I was shouting. “Don’t you see? People on average are more cooperative now which means state-level actors are lower-risk because their people are less volatile and more cooperative. But individuals are higher variance, because even an average 20% increase in peacefulness and cooperation leaves a large number of negative outliers, and a 20% average boost in intelligence means a lot of people way smarter than that, and these things are not correlated. Which means we have now supercharged our supply of intelligent anti-human sociopaths who might be able to access the “nuke everything now” button. It doesn’t matter how few people know, if even one of them discovered this button exists he’ll find a way to push it.”

“but-”

“This is too important. We need to shut the button NOW. Before we all blow up. Call everyone you can. China can nuke us if they have to, this is too much of a risk. We need to shut it down tomorrow. If we even make it to tomorrow. I give us even odds”.

I stormed out of the office and started running home. I had some calls I could make, maybe I could help the move faster out of this disaster.

I was still just halfway home when I saw streaks of fire flying through the sky, and the skyscrapers started crashing down around me.

This is another story that quite literally came to me in a dream, and I liked it enough to write it down. Can’t imagine where the inspiration came from.

I came back to New York this year, my first time ever moving back to a place and a society I’d moved away from. It was hard to know what to expect, whether this would be like before only better (with more new friends and less pandemics), or if I’d come to regret it and feel too lost, unable to recover what I once had and not finding anything new to replace it. On the whole I’d put the outcome at the 40th percentile of my expectations, maybe a bit short of the median but still, in the end, pretty good.

The start of the year was the most ridiculously busy I’ve ever been. I came back to New York on March second and started my new job on the third. I had to find an apartment, reestablish my social life, meet new people every week… the months from March until I flew to California in June was probably the busiest I’ve ever been2. It was a great time, being busy is good for me. Afterwards things calmed down a bit but I added new things to keep me busy. I ran more social events, I decided to learn all of physics3 and later, Chinese4, I volunteered in EAG and helped run NYC Solstice, I went to California two more times.

Still, I think I’ve been more down, especially lately. There’s two easy measures of this, measuring how long it takes me to get out of bed and how much anime I watch. I’ve gotten through two arcs of One Piece so far this year5, which isn’t a great sign. Life is a lot better than it was in 2024, but… there’s something missing, still.

One hard thing this year is that I’ve stopped believing that people are basically good. Not in an angsty edgelord way, just… I used to think people were mostly good, and while you may have disagreements or distrust which make it hard to convince, people do basically want to solve problems and for good things to happen. After the last 28 months… I don’t, anymore. It’s the thoughtlessness with which people will mouth support for the Current Thing, no matter what it is or how horrifying the consequences of their beliefs would be6. And it’s hard, not believing in people anymore. When people at solstice say the cold dark universe doesn’t care but there is something that cares and it is us… I can’t say that anymore. I don’t know if I believe it.

I’ve seen it before from a distance, but this year did feel personal. The politics in New York have been rough. This has been a home to me, enough that I came back and would prefer to defend it over exiting. And seeing much of it become so dark has been deeply discouraging. And so has losing my online safe space, feeling like the rationality in the rationalist community has declined to where it’s hard to blindly trust the people anymore7. I don’t like being angrier and more suspicious, and it makes me a worse person8.

To balance this, some good things happened this year too. Back in high school, I used to have a fantasy of suddenly getting a call in the middle of a meeting, maybe from an old friend or lover I no longer spoke to, something so important that I’d just step out and take it without explaining. Something meaningful. Of course, it never happened back then. I was sixteen, and didn’t have a list of long-lost friends or lovers who could call me, or a reason they suddenly would.

I’m older now, and I have plenty of both. When war broke out in June, I got a call from an old… person I’d had a complicated relationship with, to ask if my family was okay. It meant a lot, because I’d never been sure if anything we had was real, in the way we get sometimes with people we’ve drifted away from. And later, I reached out against hope to someone else, and actually got to hear back and didn’t just fade away again. And there were other stories like this, some smaller and some more ambiguous, finding that people I thought I’d lost still cared.

I think about Fire sometimes, as not just another element but the deep thing in your heart that brings light. I gave a metaphor once of the heart as an ever-draining cup with little holes that has to be refilled, but sometimes it feels more like a candle in the wind. If the world is a dark and windy place that surrounds you, it’s things like this that keep your heart ablaze and keep you moving in the face of it.And holding onto memories like this and knowing they were real keep the candle flame alive.

For the theme song of the year, I’ll go with this one. Despite staying in New York most of the year I’ve wandered a lot, and I think I’m happiest when I ramble.

The biggest thing that changed in the world this year was AI. I was bearish before (six months before everyone else was calling it a mini-winter), but this year I went from asking GPT to help me with R to using Cursor to just having Claude Code do most of my work. In principle, I agree with the AI doomers that there’s an unacceptably high chance this could go horribly wrong and we should slow down and focus on alignment. But for my part, AI is roughly where I’d want it to be at this point in my life. There’s a risk of doom9, but it also might actually work out. Partly I’d like to live for a long time and hope it starts working on the aging problem while I’m still young enough to enjoy it. But even if we never get a cure for aging, I’d like to be around to see how it goes when things get really crazy.

I did finally sign up for cryonics this year though. Just in case AI develops slowly enough for it to be relevant, but well enough to bring me back eventually. I’d like to be there if it does.

Many people take this much harder than I do. Ray Arnold did Bay Solstice this year because he said he couldn’t be totally sure anymore that there would even be a next year. That maybe there’ll come a solstice where over half the people in the room think it’ll probably be the last one, or maybe there wouldn’t and there would just suddenly be a year where no one was around to celebrate it anymore. That tomorrow might not, in the end, be brighter than today.

I wonder about that too. This is the tenth annual birthday post I’ve written, and I wonder when the last one will be. Whether I’ll just give up on the project eventually, or get hit by a truck or a cancer or a world-ending catastrophe and just not be around to write it some day. Or maybe AI will work out and we’ll get thousands of these, either because of an aging cure or because it learns to simulate my mind and writing to total precision10. Still, I wonder. There’s some years where I don’t end up feeling older at the end, but this year I do.

Or maybe it was Bilbo who said it best after all

I sit beside the fire and think
Of how the world will be,
When winter comes without a spring
That I shall ever see.

I would like to see how it’ll be, though.

I guess this is it for the year. I’m having an actual big birthday party tomorrow, which should hopefully overcrowd my one-bedroom apartment11. So I’ll end this year as Bilbo did:

But all the while I sit and think
Of times there were before,
I listen for returning feet
And voices at the door.

Over MLK weekend, I went to MIT mystery hunt for the first time1. The pre-signup waivers, video warnings, and opening posts all exhaustively repeated the core rules: Don’t bring weapons onto the MIT campus, don’t do anything unsafe for which you might later want to sue MIT, no unattended minors, no playing for multiple teams2. What they didn’t say, even in the actual opening video, was “no using AI”. Which surprised me, since I expect AI to be able to solve closed-form puzzles these days3. But as it turned out they were right, which gives some real insight into what AI really can’t do yet4.

Consider this puzzle. If I throw it at Claude with no explanation, it’s try to (literally) find the domain of the functions5. But it doesn’t find any actual insight into the problem. If I have an idea, it’ll carry it out - it’ll answer if I ask it for the domains of each function in the list, for example. But it won’t understand novel context and won’t try to.

To be clear, I don’t think this is a fundamental limitation. AI has been getting better at generalizing or pattern matching6 for general puzzles, and it probably will be able to handle mystery hunt eventually. But this does point in the direction of the thing that’s still really hard for AI - it can solve framed problems, and almost any systemic test we can throw at it looks like a framed problem. But there’s still a form of open-ended thinking, of looking for direction, that it can’t do. I don’t have a good definition for the exact thing it can’t do beyond gesturing at mystery hunt puzzles, and at the word “slop”.

Slop is a vegan hamburger

In lieu of defining the exact thing AI can’t do, I’ll describe why we should expect it to be bad at things. But before we do that, let’s pretend it’s 2010 and spend some time making fun of vegans.

As a man who has lived in both Israel and Northern California, I have been a member of more than a few majority-vegan social circles over the years. Among the Vegans, the traditional form of dining is to eat something that’s almost, but not quite, entirely like real food.

It may be a homemade cake that used bananas instead of eggs, or the latest iteration of the Impossible Burger. Invariably, the vegans will celebrate it as unprecedentedly indistinguishable from the real version of the food it tries to be. And on the first bite, I will invariably agree. For the first few bites, the Impossible Burger will taste great, so much like real meat that I’ll consider going vegan myself would be a barely noticeable loss.

I have never once managed to actually finish eating an Impossible Burger.

It’s a form of slop, you see. The first few bites pattern-match onto a real hamburger, because it’s made to resemble it well enough that, if your brain doesn’t know what to expect, it pattern-matches onto “hamburger” so it tastes like a hamburger at first. But the actual range of flavors is a lot less rich or varied than a real hamburger, and by the end of the burger the brain has learned to model it as its own thing. And since an Impossible Burger is a lot more flat and uniform than a real hamburger, it makes it (a) a more parsimonious model (very efficient for the brain to compress the information about it) and (b) boring. Once we know what to expect and don’t have variety, it’s just too bland to feel worth eating anymore.

AI creative writing is like that7. When you first run into it, it’s fun and original. After a few months, you start experiencing physical pain every time you see “You’re absolutely right! This isn’t only X — it’s Y.”

Again, It’s about compression. There’s nothing much to pattern match it to at first, so you read it as a piece of novel writing and find the structure a normal amount of nice and surprising. But once you learn to pattern match it, it starts feeling bland, because you can compress it to something smaller and simpler and no longer feel surprise at it. And it’s grating to see something bland pretend to be surprising, like someone wearing a bad mask of your ex wife’s face for breakfast every day. It can’t pretend to be the real thing, because it’s fundamentally simpler than the minimum possible complexity that anything that looks like the real thing could be.

Alright, now let’s formalize this.

Kolmogorov Complexity as a bound on intelligence

One of my favourite papers of all time is Scott Aaronson’s coffee cup complexity paper. He notes that a cup of coffee where the milk and coffee are completely separated, and one where they’re completely mixed, both seem much simpler to describe than one where the milk makes convoluted polkadot patterns on the surface. He comes up with a formalization of this measure of complexity that joins minimum description length - the traditional measure of Kolmogorov Complexity - and genericness to describe the complexity of a situation as “the longest description it would take to describe a generic member of this class”8. The completely unmix coffee is simple (short description length), but so is the mixed one (it’s a completely generic representative of a mixed state). The polkadot cup, on the other hand, is interesting.

The important conclusion here is that you can’t fake being interesting, not against any fully general intelligence. A fully general intelligence can eventually find your minimum length description9. Which means that, in particular intelligence itself - which is by definition interesting - requires a minimum amount of Kolmogorov complexity to achieve10.

But you can learn to fake your way pretty well against any static, non-intelligent adversary. If we train AI based on any sort of benchmark that’s not at least as complex as the full range of tasks humans can achieve - and coming up with static benchmarks that reflect the exact full range of human thinking is next to impossible - your optimized training compresses to be good at those benchmarks, and doesn’t learn the full range of actual human thinking. But our actual human brains are capable of the full range of human thinking, and can intuitively feel where AI is falling short even if we can’t describe any specific task that it falls short in.

Conclusions and Disclaimers

The first implication is that any metrics based system - Including the AI 2027 METR-based forecast - is going to have a bias towards short timelines11. It’s just generally easier to satisfy definable benchmarks than to satisfy fully general intelligence, so it’s fully generally true that any set of formal tests will be have its timelines shorter than a real AGI timeline should be.

To be clear, this doesn’t mean I think AI is impossible. Human intelligence is complicated, but it’s not infinitely complicated. Eventually we’ll iterate enough, and plug in enough holes, to entirely close the gap. We get some extra slack from this, but only a finite amount.

Another thing this means is that we should expect AI intelligence to be jagged, and this is incredibly dangerous. My single biggest disagreement with the Yud model of ASI is that he assumes a ready-made ASI descending on us out of the blue one day (possibly because of foom dynamics), while I think it develops more gradually and we see more intermediate steps (which would give us more time to learn to deal with it). Jagged ability development isn’t quite as bad as Eliezer’s worst-case scenario - we get to deal with one new ability at a time, more or less - but it does mean that there’s a risk that we accidentally cross the bridge to adding the last feature needed for full intelligence generalization when the AI is already pretty stacked on complementary abilities, which means we cross from safety to the lethal danger zone really really fast.

There’s another issue with jagged intelligence, which is that a jaggedly intelligent AI could easily overstate its own abilities. Right now, even a significantly more intelligent AI would need us to reproduce - the physical-world portions of AI aren’t anywhere near the level of robotics and automation it would take to make a whole supply chain to maintain AI data centers if humans vanished. But a jagged AI might not know this, might (if it decided it was better off doing its own thing without humans) genuinely think it can sustain itself without us, only to find out too late it’s missing some components that a biologically general intelligence would have had to evolve. And since destruction is much easier than creation, the ability window where it might do that is pretty wide. AI smart enough to destroy civilization but not smart enough to spread itself to the stars12 is, if you believe in great filter theory, one of the likelier seeming candidates.

Defenders of conspiracy theories often claim that they’re unfairly maligned by hindsight bias. Many conspiracy theories turn out to be true, they claim. It’s just that whenever one turns out to be true it gets eaten up by mainstream science, and everyone then turns around and pretends the real conspiracy theory is not believing in it (while still mocking remaining unproven theories as ridiculous). That people make fun of unicorns or bigfoot as silly conspiracy theories while considering underwater arctic unicorns or hairy ape-men who live deep in the Congo’s forest mountains to be perfectly normal animals feels unfair.

Another argument is that there’s a lot of perfectly normal things that can read as conspiracy theories if you describe them that way. Kosher tags on food products can almost reasonably be described as “a hidden tag marking that the producing corporation paid jewish organizations a secret tax to avoid a jewish boycott”, except for the “hidden” and “secret” parts. Cloud seeding can be described as a government weather control project. If these weren’t public knowledge, they’d sound a lot like conspiracy theories, and it makes sense to assume there’s at least a few projects like that that aren’t public. So really, why shouldn’t conspiracy theories be true?

There’s two answers to this. On the direct level, it’s a basic fallacy - a lot of things might be true, but that doesn’t mean any particular thing should be (most things aren’t!). If you have any specific theory for which you don’t have a large amount of evidence, it’s almost certainly wrong based on priors.

The deeper issue though is that “conspiracy theory” isn’t really a category of theories. It’s a category of style of thinking. The fundamental proposition of conspiracy theorizing is “there are big, secret truths they don’t want you to know". The fact that this is complicated, elaborate and hostile is in itself evidence that this is likely to be true”. And this is just not how things work. Complicated things are unlikely to be true, exponentially so in the amount of described complication1. Conspiracy theorists think it’s important to believe in chemtrails or flat earth or whatever because they’re complicated secret conspiracies being held from you. It’s why they’re typically so long and detailed.

Or in short: A conspiracy theory is any theory that argues it should get a complexity bonus, instead of a complexity penalty, for your credence in it.

This seems like a ridiculous adaptation at first glance, which raises the question of why anyone would get such a stupid bias in the first place. The answer is that it makes perfect sense in hostile information environments.

Imagine you’re playing chess, and the other player moves his pawn forwards. The parsimonious explanation is something like “he just felt like moving that pawn”. The complicated explanation is “he’s moving that pawn as an elaborate plot to gain some hidden advantage THAT HE DOESN’T WANT ME TO KNOW ABOUT!”. The complicated explanation is clearly correct, because we’re in a hostile information environment - the information we receive is deliberately engineered to mislead or manipulate us. It’s correct to look for hidden hostile meanings in every piece of information we get, because the information is being generated by a hostile process that uses its expectations on our own assumptions as part of its move-generating algorithm2.

Or consider the stock market. If you have a simple theory (“Apple makes good phones people like buying, so their stock is going to go up”) it’s almost certainly priced in and wrong. It takes a complicated theory or secret information advantage (“I got a secret satellite picture of the Walmart parking lot showing their truck activity is down, so their stock will probably go down”) to actually beat the market and make money3.

This being adaptive to hostile information environments also explains why conspiracy theories are so disproportionately believed by stupid people. It’s not just that the theories themselves are stupid; it’s that they’re an adaptive behavior for hostile information environments, and stupid people are just much more vulnerable to those. Lower intelligence makes you more likely to be snared by anything from ads trying to convince you to pay too much for stuff you don’t need to charismatic politicians pretending to agree with your exact beliefs4. And on the other side, high intelligence is protective from that - going back to our chess example, if you’re a very good chess player you can easily understand why your opponent chose a certain move, and not spend so much time worrying it’s part of an elaborate trap you can’t see5.

Examples

Now that we have the theoretical model of conspiracy theories, let’s look at examples of “conspiracy theories that turned out to be true”. When talking to conspiracy theorists, There’s three examples I usually hear:

• The CIA created modern art as a psyop

• The FBI was secretly behind the Manson murders

• MKUltra

If we look at these with our conspiracy theorist hat on6, we want to believe complicated versions of these theories requiring conspiracies. If we look at it with our normal person “complexity bad” hat7 on, we expect simple versions of it that don’t seem very scary or conspiratorial. As far as I can tell, here’s what actually happened in each case:

• The CIA had some kind of cultural funding for western art styles in the 50s and 60s, as part of a cultural diplomacy effort to show the superiority of free western cultural expression over Soviet art. In typical CIA fashion, this was indirectly funded through an intermediate body (artists would probably feel weird about accepting checks directly from the CIA). This was defunded in 1967 when people suddenly noticed how weird it was for the CIA to be doing this. Definitely on the weirder end of cold war plots but still a fairly simple story. They didn’t invent modern art (or any particular art style), just promoted osme general art funding for the hot styles at the time.

• The FBI and local law enforcement had some screwups related to the case including a badly managed raid and a parole check miss. Basically your garden variety issues with probation/parole lapses, weak case follow-through and siloed info between different agencies having trouble working together on a case.

• MKUltra was a real thing on the weird end of CIA projects. They experimented with drugs (as well as other shady methods like sleep deprivation or psychological stress) to study interrogation and influence techniques. There was some morally shady stuff involved, like participants who weren’t told what the studies were for or who was conducting them.
  The crazy stuff people talk about didn’t happen though. The CIA didn’t develop secret mind control techniques. They weren’t secretly behind any murder cults. They didn’t put LSD in the water supply to control the public. If we just work backwards from “It seems like the CIA would be into studying shady interrogation techniques”, we could have guessed something that looks quite a lot like it should exist even without evidence. But only if we kept our guesses simple, instead of also assuming they were creating murder cults on the side.

Overall, all of these aren’t surprising. If we try to guess a bit more about things in the same vein - intelligence services are probably doing similar things with social media snooping and manipulation they don’t tell us about8. There’s some other things in this vein you could guess at9. But the best way to approach it is to figure out what various groups want from information warfare and what the simplest way for them to get it is. This ends up looking like normal military projects, not weird conspiracies.

…Okay fine, let’s talk about the elephant in the room

Given the timing, I guess we have to talk about the Epstein thing.

Conspiracy theorists are lately having a field day with the Epstein story, claiming it proves once and for all that they’re right to be suspicious of elites.

Well, I’m here to say they’re still dumb. Epstein doesn’t look like a conspiracy, and doesn’t show us much we wouldn’t expect. As far as I can tell, these are the facts:

• Epstein was a rich wall street guy who got a lot of money working with wealthy private clients, possibly by scamming them.

• He was absurdly prolific at networking, both in recruiting clients for his firm and in funding a crapton of science projects, arts and culture stuff, policy and think tank circles, and basically every intellectual elite pursuit you can think of.

• A lot of rich people and intellectuals were happy to get money and connections from him, many even after his reputation got pretty shady.

• He did a bunch of sex trafficking and supplying prostitutes for people. At least some rich people were in on it.

• He seems to have preferred young women (early twenties or even younger) and often didn’t let their status as minors stop him from trafficking crimes involving them. While there seems to be at least one case of a trafficking victim as young as fourteen, there’s no evidence that any actual pedophilia was involved10.

• He probably killed himself.

• He probably wasn’t working directly with the CIA or some other intelligence agency. This one is hard to be sure of - Intelligence agencies like having dirt on people and Epstein had a motherlode of dirt on rich and powerful people around the world, so it’d be unsurprising if the CIA tried to get in on using some of that11. But there’s currently no evidence of any such thing actually happening, so our beliefs on it should be lower than our priors of the CIA getting involved in something like this (there’s at least decent chance we’d have seen some evidence by now if they did).

This mostly seems unsurprising to me. Epstein liked money and connections (many people do). A lot of culture and science people liked being offered money and connections, often enough that they didn’t look too hard at the guy offering it. And while I find it offputting, people with a lot of money tend to want to find ways to use it for various forms of prostitution and it’s unsurprising they found someone to enable that at scale.

Does it show any surprises at all? I’m legitimately surprised by just how prolific this one guy was, I would’ve expected this social role to be more fragmented (even among the narrow circle of rich elites he liked hanging out with). And I’m also mildly surprised and disappointed about how many rich people seem to not mind prostitution12. So there is some room here to updating beliefs about the world. But only about baseline human tendencies and market fragmentation, not about the existence of secret pedophilic cabals.

Anyway, this makes a testable prediction - there’s tons of information coming out on the Epstein stuff and so far it’s all lining up with my model. If the conspiracy theorists are right, we should see that change as more information comes out. Whoever ends up less surprised will be, as the kids call it, less wrong.

Flywheels

There’s a common argument where we notice that two things are correlated, but then start arguing about the direction. For example:

• Good schools tend to have smart students. But is it because good schools are good at teaching their students to be smart, or because smart students are more likely to go to the good schools (either because they have some kind of acceptance filter, or because smart students are more likely to have parents who act proactively about school quality)?

• People who exercise tend to be healthier and more energetic. But is that because exercise makes you healthy, or because healthy energetic people have an easier time working out?

• Political radicalization tends to correlate with government dysfunction. But is that because radical politicians tend to govern poorly, or because poor governance causes the people to support more angry radical outsiders1?

• On the other side, in high trust societies people trust each other more and are more trustworthy, but maybe they’re only trustworthy because it’s such a strong social expectation?

With all these questions we ask for cause and effect. I say, why not both? And this brings us to flywheels.

Consider an unobserved universe with a bunch of background mechanisms that lead to a bunch of observables2. What we see is correlations between the observables. But what can cause this?

The observables might have correlations by accident due to random noise, but these are unlikely, and become less likely the stronger the correlation is (and if they do happen, they’re unlikely to replicate). Cause and effect correlations (where one of the background mechanisms affects another mechanism) will show up in the observables, but unless the effect is strong this may not noticeably rise above the level of random noise (unless the effect is very strong).

On the other hand, some of the mechanisms may have flywheel relations, where each mechanism positively affects the other. Since these go in a loop, they’ll usually produce much stronger correlations that rise above the noise. We should expect to basically always notice these.

To actually do the math of how many of each type of correlation rises to the level where we notice it would require making way too many assumptions about base rates. But broadly speaking, we have a pattern where we don’t notice most unidirectional correlations and do notice most flywheels. Unless flywheel correlations have incredibly low base rates (which they shouldn’t - if mechanism A affects B, it’s at least plausible that B also affects A), we should expect most (but not all) observed correlations to be caused by flywheel relations.

As an empirical observation of base rates, I believe that four of the five specific examples described above are flywheels and only one is a unidirectional correlation3. So as a very rough ballpark, an observed correlation has about an 80% chance to be a flywheel.

Double Selection

Consider an elite selection process, like an application to a top college or an interview day for a hedge fund. To a first approximation, a candidate’s performance is affected by a mix of talent (which affects how well he can do on questions and tests in general) and luck4 (a candidate can have an off day, or get an interview question which he happened to run across talking to a friend the other day, or chance on an interviewer with who he has unusually good or bad personal vibes).

Assume talent and luck are given by independent normal random variables X and Y (with mean zero and separate variances). If we assume that a candidate is accepted only if they pass a certain bar A, then the talent distribution of accepted candidates will be X | X+Y > A. Similarly, the luck distribution to pass an interview looks like Y | X+Y > A.

What do these look like? They’re not exactly normal distributions, but they come close for large A. As A grows, the conditional mean of X approaches A*Var(X)/Var(X+Y), and the conditional variance approaches Var(X)Var(Y)/Var(X+Y). The conditional distribution of luck will approach something with the same variance and a similar mean (replacing X and Y in the above equations)5.

In other words, the conditional variance of X and Y converges to being equal, and X and Y each individually converge to the same quantile on average (both luck and talent want to contribute their fair share, but no more, since extra points are expensive). The higher A is, the more you need both luck and talent to get through selection. Which is why the halls of any sufficiently elite institution are full of smart people who happened to wake up on the right side of the bed the day they interviewed6.

This is going to show up in any isolated case of disaster7. Chernobyl wasn’t just a badly designed plant, or one where everyone got fantastically unlucky, or a plant without sufficient safety fallbacks - it was all of these. It had to be, because nuclear power is generally very safe, and it took a lot of different things going wrong to make a big disaster of it.

Or consider a political controversy, like a controversial police shooting8. You’ll have some people pointing at how the officer involved was angry or reckless or trigger happy and acted stupidly in a bunch of ways. And on the other side, you’ll have people pointing out how the victim acted stupidly or provocatively in ways that escalated the situation, didn’t give the officer time to think, or made him feel legitimately threatened. In general, both sides will be able to point to facts that support their case (and conclude that therefore their own side is blameless).

But in general, police shootings (especially controversial ones) are just incredibly rare9. For someone to end up getting shot by a police officer a lot of things have to go wrong on all sides. Both the officer and the victim usually have to act pretty stupidly, and there also has to be a lot of bad luck involved10.

This isn’t specific to police shootings. Any controversy that becomes news has to be unusual by definition (deadly car crashes or heart attacks don’t make national news), which means it has to be a rare event with a whole lot of selection going on.

For a final example, consider the New York Subway. for it to be such an expensive boondoggle, there’s no one thing that had to go wrong - pretty much everything (soft costs, station size, procurement processes, interagency cooperation, corruption…) had to go wrong. you don’t get multibillion dollar/mile subway lines with only one mistake.

But to end this on a hopeful note - this means it’s easy to improve. For a disaster to happen, a lot of different things had to go wrong. Fix any one of them, and you’ve already avoided the disaster happening again. Fix all of them, and it’s all smooth sailing from here out11.

There’s a series of famous experiments showing that not only can people not taste the difference between cheap and expensive wine, they can’t even taste the difference between white and red wine. Quoting from the article:

In 2001 Frédérick Brochet of the University of Bordeaux asked 54 wine experts to test two glasses of wine – one red, one white. Using the typical language of tasters, the panel described the red as “jammy” and commented on its crushed red fruit.

The critics failed to spot that both wines were from the same bottle. The only difference was that one had been coloured red with a flavourless dye.

As a general believer in incompetence1, I not only find this easy to believe, but suspect it doesn’t go far enough.

On a related topic, a few years ago Eliezer tweeted a claim that only white chocolate counts as real chocolate.

While simpletons may quibble with Eliezer’s chocolate preferences, I have a much deeper objection: There’s no proof that people can even taste the difference white, milk and dark chocolate at all. Sure, people may think they can, but everyone thought people could taste the difference between red wine and white wine too.

I raised this theory to Eliezer at solstice, to which he responded that “no way is that true” and challenged me to actually run the experiment2. I decided to take him up on this, gathered a bunch of chocolate and twenty-odd local rationalists, and let the experiment commence.

The Experiment: Methods and Data

To comparison test, I got six versions of basic Lindt chocolate. In order: White, Milk, 70% dark, 78% dark, 90% dark, and 95% dark. Each trial participant was blindfolded3 got chocolate pieces from two adjacent bowls, two from one bowl and one from another (all choices here were randomized and chocolates were given separately in sequence). They then had to try to identify (a) which of the three was the odd one out, (b) whether it was lighter or darker, and (c) which pair of bowls they got to choose from.

We also ran a parallel experiment with Lindoor truffles lower down along the table, since I’ve always suspected they’re secretly identical. Most participants tried both stations at some point (with a roughly even distribution on who did which one first).

I originally planned to run more trials at varying levels of difficulty, but (a) it turned out this level of difficulty was about the right crux (that is, people would consistently succeed anything significantly easier and fail anything significantly harder), and (b) it turns out people get tired of eating chocolate surprisingly fast.

The raw data can be found here.

Results and Observations

For the basic experiment, people got 74% accuracy on guessing the outlier and guessing which one was darker (vs a prior of 33% and 50%), and 54% accuracy on guessing which one they got (vs a prior of 20%, or 40% if we assume people can be at most one pair off). When participants did make a mistake they were only off by one level, so people can clearly tell the difference between very dark and very light chocolate.

On the other hand, while we have a clear signal, it’s far from perfect - about as far from perfect as they were from chance, more or less. Even on the basic white vs milk vs dark, some people made mistakes. And people were barely better than chance on identifying chocolate without a point of contrast (that is, identifying the pair), even when they had multiple chocolate pieces and devoted their full time and attention to it. So I’m calling this a partial win - basically unless you go above 78%, people just casually eating chocolate don’t really notice the difference. For all the bakers and casual chocolate eaters out there, feel free to just choose whatever chocolate’s cheapest or healthiest and eat that. People won’t notice the difference unless you tell them.

For the lindoors, numbers were even worse. We had 56% (vs 33% baseline) on identifying the outlier, 50% (baseline of 50%) on identifying which one was darker, and 56% (baseline of 33%) on people identifying which pair they got. Here we can’t even say people knew to avoid being too far off, we had at least one participant mistake one end of the spectrum for the other end. Lindoors are basically all the same regardless of color - there is nonzero signal here, but it’s so unimpressive that it basically just looks like noise.

Some final comments:

• Subjectively, people had a harder time than they thought they would. The typical participant was pretty low confidence in their guess and slightly surprised to be right. Even when people can taste the difference, it’s surprisingly subtle and they’re often unsure of it.

• I got two tablets of each chocolate, which turned out to be way more than was required (people don’t eat a lot of chocolate!) If you want to replicate this experiment and have under forty participants, one table per color is probably plenty.

In practice, no real cryptocurrency is particularly usable as currency. The code may be inspectable, but it’s still often full of exploitable bugs. Running your own decentralized wallet is complicated, computationally expensive, and risky, so most people just use centralized exchanges like coinbase. “Code is law” is a nice principle, but it still gets overturned by human law when there’s a strong enough social pushback. And while there may be a limit on the number of bitcoins that can ever be minted, bitcoin’s value is still to unstable to be a useful medium of exchange. Even crypto fans like Scott, when pushed to provide positive examples, typically point to stablecoins traded on centralized exchanges rather than real decentralized blockchains1.

Still, I think there is a theoretically useful case for cryptocurrencies. They’re simply a galactic algorithm.

Classically, a galactic algorithm is an algorithm that’s asymptotically superior to the ones we actually use, but only on inputs that are too large to use in practice. For example, the Strassen algorithm for matrix multiplication has complexity O(n^2.81) (as opposed to the O(n^3) given by the basic algorithm), but it only actually starts working faster on matrices too large to be practical.

Crypto isn’t this kind of galactic algorithm. It’s the much more literal type of galactic algorithm, the kind that only works for a galactic scale civilization.

Consider an actual galactic scale (or even just multi-solar system) civilization. Lightspeed means sending lots of messages back and forth is too slow, and exchange of any sort of physical goods is even slower. And it’s probably far too big to have any sort of trusted central authority2. So any sort of currency would have to be decentralized to work. It would have to have a code-is-law ethos, since the information transmission delays would take too long to renegotiate or appeal contract terms. And it would clearly need to have transparent, inspectible code.

Conversely, the weaknesses go away once we scale. If we have a whole planet full of lawyers and programmers looking through the code, we can probably get to a reasonable level of trust that we’ve caught all the bugs, issues, and exploits. We don’t have to worry about lacking the resources to set up our own wallet or exchange. and interplanetary hype cycles are probably too hard to maintain for more than a single planet to be affected at a time, given the time delays involved. We could have a traditional trust-based banking system within each planet (a single planet is small enough for that to work), and then turn to the blockchain just for interstellar commerce3.

There’s a rough split in urbanist circles, where serious policy wonk types (and politicians) are mostly focused on building more housing (occasionally leveraging existing transit networks), while more casual railfan types talk about building nicer trains or more bike lanes1.

This mostly makes sense. Politicians (and most voters) care more about reducing cost of living in the short to medium term than anything else, and housing is both most people’s biggest single cost center and the one most amenable to intervention2. By contrast fixing transit is complicated, difficult, and unsexy. It takes a lot longer to build a new metro line than to build more housing3, and the government generally needs to take a more active role instead of just legalizing private construction.

Still, you can’t really solve the housing issue without improving transportation. Traffic (and parking) are already the main reasons people have for opposing new housing, and they become bigger problems the more you build. Traffic is already a constraining factor on building more housing in Jersey City and LA, and it’s increasingly a problem for places like Austin that still build large amounts of housing. If we don’t figure out ways to build better transit, we’re not going to be able to significantly improve the housing situation4.

I think trains are pretty much a requirement here (nothing else comes close in terms of throughput), but it’s worth considering other parts of the puzzle. Self-driving cars aren’t a universal solution (they’re no better on throughput than normal cars), but they do have the advantages of not requiring parking and allowing backstop point-to-point transportation that can complement transit commuters. Biking infrastructure and better scooter access are both much easier to build and good complementary forms of transit5. But better trains are pretty much unavoidable.

As a case study, consider the difference between Berkeley and San Jose, CA. Berkeley’s density is roughly double San Jose’s (11,874 vs 5,684 people per square mile), and it manages to feel a lot more pleasant6 than San Jose, managing to be interesting and walkable while keeping the advantages of a suburb.

Berkeley’s transportation mode share is about 25% public transit, 10% walking, 10% biking. Since higher density allows for more biking/walking, those 25% transit trips7 are load bearing for a roughly 45% non-car mode share, which allows Berkeley to be an actually dense, pleasant, walkable suburb. If there’s a moral here, it’s that we don’t need a lot of transit to solve the housing crisis, and we don’t need everyone to use it - a 25% mode share seems like enough. But we do need to figure out how to achieve that better.

Charles Dickens’ A Christmas Carol tells the story of Ebenezer Scrooge, a cold hearted moneylender who has to learn about the spirit of Christmas from a bunch of ghosts. If you’ve never read it, here’s a brief introduction to his character:

There’s a common contrarian take that Scrooge’s moral reform was actually a bad thing (see e.g. this SMBC): He’s already a productive businessman and important part of the economy who contributes meaningfully to society, even if it’s in economically complicated ways (someone has to be the moneylender). He shouldn’t need “reform” just because Dickens doesn’t like his job.

This mostly misses the point of the actual story told. Scrooge’s reform isn’t about his business being bad, and he doesn’t stop his job or change his business practices (aside from giving his employee a raise) at the end of the story. Scrooge’s sin is that he’s disconnected from humanity. His one biggest mistake isn’t underpaying his secretary or ignoring the charity guys, it’s blowing off his nephew Fred’s Christmas party. Scrooge is his own greatest victim; by cutting himself off from humanity he’s made himself lonely and miserable. His lack of charity is downstream of that, and he finds charity at the end as an end downstream of connecting to humanity, a way for him to convert his money into utils that gives a good trade from his own selfish perspective. It’s a charity found in joy, not in guilt or obligation.

This is a lesson worth noting, because it affects Effective Altruists as easily as it does bankers (and affects political activists more than either). Business is not the only end that can disconnect you from humanity, and disconnecting from humanity (for any cause) pays a price in misery. It also pays a price in moral alignment, because it’s hard to stay Good when you’re not connected to the people whose good you’re working for1.

But this still leaves the question of whether Scrooge’s reform was good from a strictly utilitarian perspective. Scrooge is just one man, but he controls a lot of money and is suddenly changing his spending habits. The impact could be far larger than on Scrooge’s happiness alone.

What does Scrooge do with his new free spending ways? He gives money to (unvetted, replacement-level) charities, gives his secretary a raise, and buys a goose. This seems like replacement level spending at best (and arguably worse, given the track record of unvetted charities compared to GiveDirectly). Since the alternative, Scrooge keeping his money unspent, seems equivalent to deflating the money supply (which is arguably equivalent to just increasing the value of everyone else’s money, a form of direct cash transfer), this may just be causing deadweight loss.

But all else is not equal. The British economy in December of 1843 (the year a Christmas Carol was published and, presumably, takes place) was before modern central banking and macroeconomic stabilization, and long before keynesian economic policy. The poverty Dickens describes didn’t just happen, it was caused by (among other factors) a lack of demand caused by deflationary (lack of) monetary policy. In this context, we can think of Scrooge’s spurt of random generosity as a form of fiscal stimulus, which London sorely needed at the time.

Understanding this also resolves the other mystery raised in the story: Why don’t rich but lonely old men get haunted by the spirits of Christmas anymore?

Well, Scrooge was haunted in Christmas of 1843, a time in which I assume this was fairly commonplace2. In 1844, Parliament passed the Bank Charter Act, which centralized currency creation in the Bank of England. If we believe that the true spirit of Christmas is about monetary policy3, it makes sense that the spirits switched from haunting every banker and moneylender who could print his own currency (as they could in 1843) to haunting only central bankers after 1844. It probably saved them a lot of work.

Some final implications:

• Ron Paul and the End the Fed guys probably just want to bring back Christmas for the masses. From the point of view of the spirits, Scrooge’s redemption was mostly a happy accident, but from the view of society it’s a critical positive externality that we need to fight to bring back4.

• The British government didn’t start running countercyclical monetary policy until the 1946 Bank of England act. Mismanaged demand cycles and endemic Dickens-style poverty were the norm all the way from 1830 until after WW2, the entire peak era of the British Empire. There’s two ways to interpret this:

• Possibly the hardcore libertarians are right and even the most benign, economist-supported monetary policy interventions in the economy are actually economically destructive to the point of bringing down the largest empire in human history. That’s a big claim and would require more evidence than this, but this should directionally update us in that direction.

• Alternatively, all governments used to be absurdly terrible at policy to the point where the British could dominate the world even with making a bunch of terrible decisions5. This implies that human progress is hard and getting harder - we’ve not only picked off most of the low hanging fruit in science and engineering, we’ve also picked off a lot of the low hanging fruit in economic policy (and, probably, management in general). Science famously requires more investment of people and resources per advancement to keep up the progress the further we advance; it’s possible that all dimensions of human society are like that. If so, expect to see the great stagnation continue over time.

It’s just like the simulations

An unusually sheltered child, finding out one day that war is a thing in the real world and not just fantasy books, would probably ask the two following questions:

1. So in fantasy books people always talk about morale and the heroic spirit and all that, but I assume that’s just because books need to tell interesting stories about people, and in real life it’s just a question of who can go on shooting longer?

2. I get why war happens in fantasy books - actual fight scenes make for a better story than long boring negotiations - but surely in the real world no one would ever actually go to war. It’s a decision branch that leaves both sides mutually worse off, so they could just estimate what the outcome of a war would be and then negotiate a settlement that leaves both sides mutually better off.

And we would have to explain to the child that no, he’s wrong on both counts, fantasy books are surprisingly realistic1. The missing piece here is uncertainty, or information asymmetry: The two sides typically disagree on who would lose more in the conflict, and actual wars happen when they disagree enough to both risk calling the other’s bluff. And so morale matters - if your side is confident enough that it’s willing to call the other’s bluff but they’re not willing to call yours, you win2.

But before we get into this, it’s worth noting that question 2 mostly does work in real life. Most wars don’t happen. If you pick two groups of people with an ancient irreconcilable enmity, odds are they’re not at war right now even if neither side has nuclear weapons.

And when wars do happen, they usually stop a long way short of the maximum destructive potential they could have caused. to pick two battles at random3, the French surrendered at Waterloo after about 10% of their soldiers were killed, and the British surrendered in Yorktown after only 2% were killed. This is tragic for the soldiers involved, but in terms of actual damage that’s just enough to show a clear signal, rather than people fighting until they can’t anymore. You don’t win a battle by killing the enemy; you win it by convincing them that you can.

Getting high on your own supply

So our model here is that war happens when both sides think they can win. But even if you think you can win, war is very expensive, in both money and lives. So you’re almost never actually better off fighting to the death than reaching a negotiated settlement (even one that leaves your enemy somewhat better off). Here’s a list of things you might believe that would convince you to go to war anyway:

1. The enemy also secretly knows you could win; when they’re reciprocating your war threats instead of negotiating a settlement, they’re just bluffing, and will fold if you call their bluff.

2. If war does break out, you will show yourself to be easily capable of overwhelming your opponent, leading them to quickly accept it and make concessions.

3. Even if you can’t win an actual war, the enemy underestimates either your ability to cause them damage or willingness to take damage from them in return for the war’s stakes. Going into an actual war is a way to convincingly signal this, allowing you to reach a more palatable settlement.

4. You think the enemy can’t be trusted to hold to a settlement, and will just threaten a new war and extract more concessions until you’ve lost so much you’d have been better off just fighting the war.

5. You think you can win strongly enough to destroy the enemy’s ability to threaten you, and also that the enemy is too irrational or malevolent to be willing to accept any sort of negotiated agreement that would be better for you than actually fighting that war.

6. The enemy thinks you believe one of the previous reasons, and are going to choose war over a reasonable negotiated settlement, so they might as well also choose war.

Once war has actually broken out, it will keep going until something has changed in the information space. If you fought because you thought the enemy was bluffing, you might stop once you’ve tested each other to see the extent of your bluffing4. Wars in which one or both sides think they can easily win either quickly show this to be the case (as in the US invasion of Grenada) or fail and get bogged down (as in the initial Russian invasion of Ukraine)5.

That’s a nice theoretical framework that somewhat matches reality, but the kind of information asymmetries described that cause wars seem surprisingly common. Worse, nations usually don’t update away from them after the wars begin, resulting in wars that last much longer than it would take to simply update the information6.

This brings us to the second issue: If war is decided by information, then information is a weapon. Not in the sense of deceiving your enemy about your tactics, but in the sense that lying to yourself about your probability of winning is itself a weapon that helps you win wars.

Aumann’s agreement theorem

Aumann’s agreement theorem claims that it’s impossible for two rational agents to disagree for long. If one rational agent believes X and another believes Y, it argues, then the first will simply update his probability of X on the fact that the second still believes Y (and vice versa) until their probabilities converge to being equal. I won’t get into the math of the updates here, but we can take it as a given that it holds given the assumptions (and converges reasonably quickly at that).

There are two obvious problems with applying this theorem in real life, which fortunately manage to cancel each other out:

1. People seem like they don’t always agree with each other about everything, even after having a short conversation about things.

2. If Aumann’s agreement theorem held, you could convince anyone to do anything by claiming to believe with 100% probability that they should.

The solution is that people don’t always have the same goals, and can lie to each other about their beliefs in order to achieve different goals7.

Getting back to war. War happens, to a degree, because you had diverging beliefs about what the outcome of a war would be and who would lose more from it. If you could simply Aumann your way out of it, war would never happen. But you can’t, because you have opposing goals. If one side says “if we fight I will absolutely crush" you” the other side won’t say “oh no, better update our P(losing|fight)”, they’ll say “those outgroup members are being dishonest as a method of using information warfare against us”. So in order to message your belief in victory, you have to use costly signalling, like sacrificing lives and resources in ways you wouldn’t want to if you didn’t believe you could win.

But while you do this, you also need to convince your own side not to give up (remember, whoever updates their belief in their own odds of victory down first wins). And since you generally have much more effective communication with your own side than the enemy, the primary purpose of wartime information weapons is to convince your own people to be as optimistic as possible, whether or not it’s warranted.

There’s a few ways to do this. There’s the obvious form of over the top wartime propaganda, although the public might notice if your navy’s glorious victories keep happening closer and closer to home as the war drags on. There’s religious fanaticism - it won’t necessarily reassure people about the war itself (although it’s always hopeful to hear that God is going to help you in), but it’ll also affect your beliefs about the payoff function if you think that victory is sacred and death in battle just means you get to go to heaven. And there’s emphasizing different fronts or dimensions, like talking about how you’re winning on the global PR front while your actual troops are taking crushing losses.

As a general humanist, I’d like for fewer people to die in wars. It’s noteworthy that the worst wars, the ones that drag on for longest, are the ones where the losing side has really good internal information tools. Religious wars8 are generally much longer and bloodier than secular conflicts. So are wars where the situation on the ground moves very slowly, so that information warfare can adapt to come up with excuses for each change, dismiss each minor update as not that big a deal until you’re trapped in a horrible situation9. If our world had higher standards for honesty, we’d fight shorter and less bloody wars.

The problem is, though, that this is maladaptive10. Being able to convince yourself that you can win to an irrational degree can be a self-fulfilling prophecy. So people aren’t going to become honest about their odds of winning conflicts anytime soon. It’d be like giving Japan the keys to your biggest aircraft carrier.

Final notes

Two things this post didn’t cover that are still worth a mention:

• I treated countries as unitary agents here, but that’s not always the case. The ruling class and the average person can have different goals, in which case we should model the situation as a more complicated principal-agent problem (examples of this are Mao or Bismark starting foreign wars as a political tool to consolidate and unify their domestic power).

• The decision theory of war is also more complicated than implied here. See here for a very long and mostly-irrelevant story covering these issues.11

There’s a stereotype that nerds - in the sense of people who are good at things like math or science - have bad social skills. Oddly for a negative stereotype, people don’t seem to consider it socially unacceptable1.

While the promotion is motivated reasoning, it does seem to be anecdotally at least a little true2. More importantly, we should expect it to be true, because of Berkson’s Paradox.

Berkson’s Paradox, as a reminder, is the statistical artefact where uncorrelated (or even positively correlated) features can appear to be inversely correlated when subject to selection. It’s why hot actresses tend to be worse at acting - it’s not that being hot makes you dumb3, it’s that being hot can (within limits) make up for lack of acting talent with casting agents. Or in visual form:

The example of actors involves casting agents (and, later, movie audiences) making explicit filtering decisions along these factors. Nerds (however defined) aren’t selected that way as a group, but the people you notice, in general - however they come to your notice - are pretty strongly filtered.

Consider Elon Musk’s Twitter account, which mostly looks like the account of a not-particularly intelligent troll. It seems weird that someone who’s smart about many things is dumb online. But he’s not especially dumb online. The average person on Twitter is a dumb troll! It takes something unusual - either a special skill to stay above the fray, or a desire to make your account a focused subject matter expert4 - to avoid descending into being a dumb troll. Musk isn’t especially dumb, by Twitter standards. It’s just that the average user of that caliber gets immediately muted and ignored, but Musk is notable enough for other reasons that he still manages to get attention on Twitter5.

There’s two common misconceptions about general intelligence. One (common in strawmen and rare but not unheard of in real life) is that intelligence is a single value that explains someone’s talent in every field and axis. The other is that intelligence between fields is completely uncorrelated and there’s zero reason to assume someone talented at one thing will be good at anything else.

The real answer is that there is a general factor of intelligence, and it’s correlated (but not perfectly) between different fields. So people good at math are in general likely to be more generally intelligent and thus have better social skills. But they’re selected for notability harder than they are for social skills (since the math ability makes them notable beyond their social skills), so Berkson’s paradox means it still looks, on average, like being good at math negatively correlates with social skills.

(Nate Silver once proposed this model of “spiky intelligence” to describe Musk. There’s really no need to assume this - noisy correlations will on their own give you people who are outstanding in some ways and average or below in others).

And conversely, social skills are also something that requires a lot of intelligence - people wrongfully dismiss charisma as entirely separate from intelligence, but it’s actually pretty good evidence of general intelligence6 (consider Bill Clinton’s legendary charisma - part of it was just that he was confident in his ability to remember the names and details of everyone he ever met, which requires phenomenal memory and sharpness).

Or for the other side, consider the recent case of this woman. The story here is that an AI engineer tried to strike up a friendly/flirty conversation with her, she decided to retaliate by blasting his name on Twitter to try to shame him, and commenters (men and women alike) called her out on her bad behavior regarding him instead.

It’s notable because once you look at her statements from her perspective, she’s just… not very bright. She’s pretty and cultured, but she’s not socially astute enough to intuit why this situation comes off different than the ones she’s seen men successfully shamed for in the past, or to handle it in a way that comes off well. Honestly, despite my dislike for this short of shaming campaign, I feel for her. She reminds me of a low-intelligence person trying to navigate a strange subway system7, just feeling confused about why this train isn’t taking her where she wants to go and not sure how to fix her mistake now that she’s lost in the maze.

The point of that whole story is to say, If you removed all your expectations - if you judged good-at-math and bad-at-math people’s social skills by the same prior, without selection bias - you don’t end up being surprised by the former’s ineptness or the latter’s skills. You see examples of both being good and bad, but the general trend is that the smart-at-math people are slightly better at social, too.

Remaining Gaps

That said, there really are some things where we expect differences. Good-at-math people generally tend to be system-oriented thinkers, while good-at-social people tend to be people-focused. Since those do somewhat trade off against each other, we expect to see some tradeoffs in the focuses.

Note that this isn’t going to look the same as raw talent, it’s going to look like focus. Good-at-math people are often very good at understanding how other people think, but they’re going to have an emotionally harder time just nodding along with someone talking about their astrology. And social people might be very good at understanding numbers, but still have a hard time overcoming their instinct to try to socially manipulate a math problem into giving the answer they want.

There’s also secondary effects. Good-at-social people can often get by on just saying things that people want to hear, so they don’t bother to learn actual science. I suspect this is responsible for the large number of economically illiterate successful politicians8.

Consequences

The one that’s personal to me is stem bros vs humanities people, as a group.

Consider a typical university. To advance in math or science, professors need to be brilliant and hardworking and produce new insights, along with some skill at department politics and looking (rather than just being) generally intelligent. In the humanities, which aren’t easily measurable by objective criteria like mathematical proof or reproducible experiments, the factors of department politics and looking-smart are going to dominate much more heavily. So we’ll end up with a STEM department that’s much smarter in real ways but (since they’re selected less heavily on it) worse at politicking or looking generally intelligent.

In the context of a university, this ends up with overfunded humanities departments9 with too much cultural influence and too many students based on their ability to sell vague unprovable concepts like “critical thinking skills” and “the ability to analyze a piece of text”, leading to a massive, tragic waste of both physical and human capital.

In Breakneck, Dan Wang compares America’s government by lawyers with China’s government by engineers10. He points out a lot of the positives and negatives of both approaches. One thing I think he misses though is that government by engineers just produces people in government who are, well, smarter, and thus better at achieving their goals. It’s an open question whether you necessarily want that - there’s a few places I wish the chinese government wasn’t quite so effective at achieving their goals - but it is an added factor, beyond the cultural effects. It also means that China (surprisingly for a one-party state) seems better at noticing and fixing their mistakes than the US government11, partly just because they have people who are better at understanding things.

Social media

There’s a common story about how social media is destroying democracy. There’s two variants of it:

One, that social media is structured to encourage and amplify poorly-thought out or outrage-bait takes. Short punchy statements that fit on a bumper sticker are optimized for viral spread, everyone loves to repost outrage bait, and the businesses themselves are incentivised (in both ways) to promote things that increase engagement. Twitter limiting tweets to 280 characters is just an unusually explicit instance of a general trend. Social media is also unusually optimized for talking about politics in various ways1, so this kind of toxic conversation takes over our politics and destroys our ability to have smart problem-solving political conversation.

The second explanation is about the people social media empowers. In the old system, people would have to gradually rise through gatekeeping institutions to get noticed. To become a serious candidate for political office you’d have to be taken seriously by institutional party members and news reporters with some real policy experience who could evaluate your policy ideas and management experience and filter out people who failed basic competency tests. Social media enables populists to cut out the middleman2 and become serious candidates directly, without the previous filtering3.

There’s a corollary here that democracy needs some amount of elitism in order to actually work4. Any sort of governance needs both managerial competence and some real policy understanding in order to actually work, and without some filtering you end up with empty populists who don’t know how to actually manage things in charge. And social media, the story goes, is enabling that. Democracy can only last so long with this kind of governance.

Enter AI

In some ways, AI is just the next level of social media: More engaging, more algorithm, and most importantly, more slop. It’ll flatter anyone outrageously and enable them to think their poorly-thought out ideas are brilliant no matter how dumb they are. If social media was bad for democracy, the thinking goes, AI will knock it out completely.

But what if it saves democracy instead?

A simple boring way AI could save democracy is by enabling the dead internet theory: More and more of the internet is AI producing slop for consumption by other AI. But online politics, as previously noted, is breeding ground for slop. A deader internet could disproportionately kill political spaces, filling them with slop to the point where the reasonable people leave and only diehard crazies and AI slop remains in the corners of the internet that still talk politics. AI would be a form of chemotherapy, making an environment so toxic that not even the cancer can survive.

A more interesting possibility is that AI commoditizes the complement: That it gets good enough to give any policy idea a populist shine, removing the advantage that slick hustlers running media campaigns have through social media. When anyone can make a slick campaign ad with the push of a button and use bot hordes to create a false consensus around it, populism stops mattering, because whoever can control the bots can use them to persuade the people who are swayable by populist messaging5. AI slop turns populism from a bespoke tool used by charming social manipulators to a technical challenge solvable by engineers. And the skills to engineer a well-run AI system are a lot more correlated with the skills of good governance than the skills of populist charm and slick campaign ads.

Who else has the advantage in gaining political power in this scenario? Whoever controls the AI. So we now have an elite filter made by successful engineers and AI entrepreneurs, or the people they respect enough to share power with. Or maybe it’s an issue of who can buy access to GPUs, in which case it’s an economic filter. These scenarios are far from perfect - God knows both Silicon Valley and the rich as a group have their political idiosyncrasies - but it’s not a bad filter, and probably a better one than the pre-social media filter of party insiders and journalists.

There’s also still some filter on the actual ideas. AI can automate slop and slogans a lot more easily than it can automate real understanding of deep ideas, so whenever a voter thinks about actual meaningful policy issues, that advantage might still hold. And the most important tool democracy gives the masses - that when things get bad enough, they notice things are bad for them personally and vote to throw the current bums out - still holds. No amount of AI ad slop will protect a president whose policies cause hyperinflation from being voted out. AI slop doesn’t destroy the emergency exit button.

Final concerns

We’ve talked about domestic use of social media AI. But what about its use as an international weapon? Part of the reason social media hit democracies so hard is that actual adversaries like Russia or Qatar have deliberately used it to sow chaos or advance their interests in the west. With AI tools, could they do even worse?

I don’t think that actually holds. When social media induced dysfunction is caused by actual users (either organic or in clickfarms), it’s a lot easier for a hostile government with unified purpose to put together such a clickfarm than for a democracy to coordinate a defense. When both sides are simply big engineering projects, it eventually becomes a standard tool of warfare6, which means the DOD starts putting serious resources behind blocking and countering social manipulation weaponization. The attack/defense asymmetry here favors the defender, who can detect hostile traffic patterns and counter them once he’s organized enough to do it7. So again, I think this is a problem AI actually helps solve.

People always worry about how new developments will cause social decline and things will get worse. But while social decline can happen, the opposite happens just as much; in the very long term, society has gotten no worse and has arguably gotten better. So there is room for hope. I’m on the side that believes social media and smartphones had a pretty bad effect on society. But it’s not at all clear the net effects of AI will be the same. It’s worth considering the good outcome scenarios too; on average, they happen roughly half the time.

How valuable is embryo selection?

A few weeks ago, Scott wrote this post on embryo selection. Putting aside the various claims about the technology itself (which sounds pretty amazing), he brings up the question of how to value future benefits

(but maybe the low time discount rate is a mistake? Suppose you invest the $3,250 in an index fund that makes 7% over inflation, then give it to your future child when they turn 45 (average age of type 2 diabetes diagnoses). Now it’s worth $75,000. Is this the “true” cost of the intervention? Does it matter that this counterfactual is fake and most people don’t do this?)

Scott mostly goes on to discount this, but the answers to the last two questions are “yes” and “no” respectively1.

There’s two separate reasons we exponentially discount the future:

• Compound returns on investment

• Discounting uncertainty

Let’s go through both

Compound returns

This is straightforward: A dollar today is worth more than a dollar next year, because value can be used to create more value. If you’re a farmer and I give you a dollar next year, you can use it to buy a potato. If I give you that dollar today, you can use it to buy a tractor2, which will make your farming more efficient and let you harvest two extra potatoes in next year’s crop. Since your previous plan was to buy one potato, we’ve shown that giving you that dollar today doubles your utility3, so a dollar now is worth at least twice as much as a dollar next year.

Another way to think about it is option value: A dollar today is something we can either hold for a year4 or spend today, which means we have strictly more options on what to do with it than we do for a dollar next year, therefore it must be worth more.

So we showed that the value of a dollar t days for now is worth f(t) dollars, where f(t) is a monotonically decreasing function. f is generally assumed to be exponential, that is, f(t)=exp(-ct) with c>0. Here’s three different ways to see this:

• Concrete: If you’re the farmer mentioned above, you probably have a bunch of different ways to spend your money (buying a tractor, investing in better fertilizer, getting an automatic machine gun turret to protect your farm from feral hogs…) Since you have so many options5, the return on investment of the top few options is probably pretty similar. So if you invest in the top option and double your money the first year, you can invest in the next two options and double your money again the year after, and so on6.

• Abstract: We can think of our available money as “how much stuff we can buy”. Since one of the things we can buy is the ability to buy more stuff, we get self-reinforcing exponential growth

• Option value: Consider the number of ways a dollar can be used over t time intervals. This includes not only how we first spend or save it, but how it gets used subsequently (we have ten options for how to use it today, then whoever uses it tomorrow - whether us or someone else - also has ten options, and so on). The number of paths the dollar can take is exponential in the number of units of time, implying its value should also be exponential7.

In general, the constant used in this exponential is called the risk-free interest rate, is historically measured by LIBOR or SOFR, and varies in the 0-10% range. When I first got into finance, I was confused why this number was so important, until I realized the right way to think about it isn’t just “how much will a bank charge for a loan”. It’s the measure of how fast time is passing, where the passage of time is measured in opportunities lost per second8.

Uncertainty

The other reason to discount future value is uncertainty.

There’s two reasons for discounting uncertain value. First, there’s the chance things might not turn out the way we think. Even if we say we’re very certain of getting two dollars in a year, long experience has (correctly) ingrained in us the mental habit of regarding a dollar not currently in the hand as not fully real. People generally overestimate the certainty of their predictions9, so it’s good to discount the value of future uncertain events. And the further ahead you look, the more chances there are for something unpredictable to happen, so the more you should discount a payoff to any decision you make now.

(An aside here that’s too important to relegate to a footnote: This is why thoughts experiments that require enormous scale or time horizons - like the “would you kill a human to save 10^100 shrimp” thing, or “you should think of the quadrillions of potential humans in the long-term post-space colonization future” - should be exponentially discounted10. If you have very strong evidence or can take very cheap action they might still be worth being concerned about, but by default our intuition that no number of shrimp is worth killing over is correct - the odds of a case that would disprove it are astronomically low, much lower than the odds of us rationalizing ourselves into a mistake11).

Anyway, back to uncertainty discounting. The other reason for discounting uncertainty is payoff convexity: A second dollar (or a second million dollars) isn’t as valuable as the first, so a 50/50 chance of two million dollars is worth less than a million. Equivalently, there’s value in certainty, so a risky investment has to have a higher payoff (on average) to be worth as much as a certain investment. Anyway, this implies that uncertain future gains should be even more discounted than the raw uncertainty would predict12

Equivalence principle

This all brings us to the last argument: Sure, a calculation might say that your child would be better off if you invested fifty thousand dollars in a trust fund for him than if you spent it on embryo selection, but what if you spend it on a fancy vacation instead?

I think that’s mostly wrong. Things should be equivalent. If you spend money on a vacation instead of saving it in an investment, the default assumption should be that the present value of that vacation is actually worth the future value of the investment for you. To some degree this implies that it’s correct to believe that having a little fun today is worth more than having a lot of fan in the far future. Given everything that can go wrong (or just unexpected) between now and then, I think this is correct (if it weren’t, you could construct elaborate hypothetical scenarios to make infinite money off rubes with inconsistent values).

There is an exception I can think of, though: What if you’re just not very smart?

I don’t mean this in a mean way. If you know that you (or someone else) consistently makes overly short-horizon decisions and doesn’t think ahead, should you help them? If not by forcing them to invest their money in long-horizon savings (since forcing people to do what you think they should with their money has moral issues), at least by putting more cool-sounding long-horizon opportunities in front of them? Maybe someone who would in principle endorse investment doesn’t have the impulse control to do it over time, but would get it together to invest in embryo selection, so making that option available helps.

Alternatively, what if someone isn’t very smart in the sense of being bad at making long-term predictions, but is acting rationally based on that? Someone who knows they have poor ability to figure out what kinds of things will work in the long run might rationally have higher time-uncertainty discounting (because he has more uncertainty per unit of time). In this case, if you’re an individual (or, more commonly, a government) that person knows he can trust to be better at long-term thinking, having “This is the Done Thing” be your official recommendation might help. In that case maybe the person really should discount recommended health interventions less than is implied by their general discount rate.

Small Train is Good Train

A while ago, I wrote about how elevated trains are the greatest urbanism cheat code, increasing the amount of track miles you can build per dollar (or per year) by a factor of 2-4. And while I don’t have anything else on that order of magnitude, I do have one more easy 20-50% gain1: Run shorter trains.

The basic idea is simple: The single biggest cost of any metro system is the stations, whose cost scales with size. Therefore, if we run a system for smaller trains, we can build smaller stations for these trains, saving a huge amount on station costs. This costs us in reduced total capacity, but this can easily be made up for by increasing train frequency.

This also has some non-obvious advantages: Smaller systems let us have smaller, more incremental transit projects, gaining project experience and institutional knowledge for subsequent projects. Especially in places like the US, which don’t have recent experience in building infrastructure, it’s better to start small where possible.

It’s also just better to split up your capacity as much as reasonably possible. Having to wait for a train that comes every three minutes is a lot better than waiting for an equally-crowded train that comes every six minutes. Wait times are the second-most underrated component in making metro systems effectively faster (right after street to platform time).

Some details

Cost and Timeline improvements

Stations usually dominate metro construction costs (23 – 62 % of hard cost on the Transitcosts European projects sample, mostly depending on underground vs elevated) and can be made roughly half as long (50-55 meter platforms vs ~110 meters) for low-capacity short trains. Station construction costs scale roughly with station size, so cutting station size in half implies a total cost reduction of roughly 20% on underground lines2 and 15% on elevated lines.

Timeline improvements are even sharper. Since the stations are the single biggest (and thus longest to build) component of the system, anything that makes them easier to build makes the whole system open faster. You can’t quite build a station twice as fast if it’s twice as small, but you can typically hit 30-40% timeline reduction (3-4 years in absolute terms) on a medium-sized project (e.g. a 10-15km subway line) where stations are on the critical path.

Capacity

Capacity limits are why we didn’t always do this: Train operators are expensive3, which limits the number of trains you can run since it makes it more efficient to run fewer longer trains. It’s the invention of fully driverless modern trains that makes this obsolete, making running more trains basically free4. But now that we can run fully automated trains, the only reason to run longer trains is if we’re hitting hard capacity limits after maximizing frequency.

This isn’t impossible - it happens in Hong Kong, and can easily happen in other dense places like Manhattan - but it’s uncommon. 3-car trains running at 30-40 trains per hour (a normal peak frequency for automated or even some human-driven metro lines) reach a capacity of about 18,000 passengers per hour per direction, well above the expected demand of any American line that doesn’t go through Manhattan. Outside of the US, most Israeli or European lines would also be served by this level of capacity5. Experience with lines of this nature shows they’re fairly robust to passenger surges.

Finally, it’s worth noting that small automated trains are the easiest way to keep good weekend service. Systems with large trains typically have borderline unusable frequencies on weekends (sadly true for most American systems); switching to small frequent trains lets you run usable frequencies even when you’re running with low total capacity.

Examples, and a tangent on New Jersey

This isn’t a theoretical idea. It’s the concept behind North America’s greatest transit project of the last century, the Vancouver Skytrain

Other great examples include London’s DLR and the Copenhagen Metro. All three were successfully built at relatively low cost (which at least for the first two is extremely unusual, given that both Canada and the UK are almost as bad as the US at infrastructure).

Where else could we replace existing plans with smaller more frequent trains? Basically every currently planned project in America (and most of those in Israel). Asking GPT for examples got me this table

But this doesn’t include my favorite possibility, which is Jersey City.

New Jersey has severely underrated potential to be the next great American city: It’s right across the Hudson from New York, so it could gradually bleed over people in jobs without having to build a new center in the middle of nowhere. While it has its political dysfunctions, New York levels of political dysfunction at both the city and state level are extreme6. There’s a severe housing shortage pushing people out, which Jersey City and its surrounding towns could take advantage of.

Jersey has a lot of wins. It’s visibly cleaner, nicer and with less homelessness. It builds a lot more housing (Jersey City built around 13 housing units per 1000 residents in 2020-2023, compared to 4/1000 for NYC). It has lower crime and (slightly) lower taxes. And it’s right across the river. While NYC proper has been losing population, Jersey City has been gaining.

The biggest barrier to scaling up Jersey is transportation. Commuting to Manhattan is difficult (Manhattan doesn’t have the road capacity for mass driving, the PATH is kind of terrible, and ferries and buses are too low-frequency to be convenient). We could solve this by moving more jobs into Jersey, but then we’ve just moved the commute burden further west.

For Jersey City to become the next New York, it needs the thing that makes New York unique in North America: An actual large-scale metro system. We can’t build one from scratch anymore, but building a small number of small-station lines lets us do it one step at a time, then scale up as the city grows. And if Jersey can build a transportation system that’s both more convenient and nicer than New York’s, it can form an alternative, less dysfunctional hub for the northeast. If we’re really lucky, it could even push NYC to try to get its act together.

Failure modes

The final warning here is the failure mode: Don’t build “small trains” just to build a small train (because you don’t have need for more capacity). When building small trains, do it by actually keeping everything small and simple. Otherwise you end up with the Macau LRT - a system with giant tracks, giant stations, and everything needlessly huge and expensive except the trains themselves. You’ve gained almost nothing in cost or simplicity in exchange for the lower capacity.

On the other hand, the actual trains it uses are beautiful. I don’t have the style knowledge to explain why, but their (japanese-designed) trains are just prettier than the chinese-designed trains of the Hong Kong MTR7.

7

There’s a more subtle complaint about my whole approach, which goes “by hyperfocusing on efficiency like this, aren’t we sacrificing the ability to build beautiful monumental stations like Grand Central or the Moscow metro?”
I’m actually not entirely unsympathetic to this, but I think it’s mostly wrong. Grand beauty comes from using excess capacity well, not from raising requirements. Although the Moscow Metro is pretty impressive.