Clifford’s central example concerns a shipowner who believes on the basis of wishful thinking that his ship is seaworthy, sends it out for a voyage, and ends up responsible for the deaths of the passengers when it sinks. At first glance, the case is almost embarrassingly flat-footed: when we form beliefs irresponsibly, we make ourselves more likely to be wrong; we then act on those beliefs, and things go poorly. Moreover, even when our beliefs don’t have immediate practical implications, or when we luckily get things right, believing on insufficient evidence is a bad habit, as it makes us credulous and thus likely to form false beliefs on other matters that do have practical implications.

It’s perhaps a bit ironic that while the piece is in large part an implicit (and sometimes explicit) critique of religion, it’s written like a fire-and-brimstone sermon:

If a man, holding a belief which he was taught in childhood or persuaded of afterwards, keeps down and pushes away any doubts which arise about it in his mind, purposely avoids the reading of books and the company of men that call into question or discuss it, and regards as impious those questions which cannot easily be asked without disturbing it—the life of that man is one long sin against mankind.

In “The Will to Believe,” ” James, by contrast, argues that when we face what he calls a “genuine option”—one that is live, forced, momentous, and not decidable on intellectual grounds—our “passional nature” may legitimately decide between the alternatives. Our hopes, commitments, and practical needs may therefore properly help determine where we end up believing, even when they do not constitute evidence for the resulting belief.

While I’m pretty sure most philosophers today would agree that James was both the more subtle thinker and the better stylist, that’s not at all to say they’d all agree he had the better of this particular debate. Rather, you still have contemporary “evidentialists,” who think beliefs should be based only on evidence, and a variety of opponents of evidentialism. One difference, though, is that while some contemporary opponents of evidentialism are, like James, interested in defending the rationality of religious faith, much contemporary opposition to evidentialism is motivated by other considerations.

For example, you’ll find people defending the idea that friendship (or marriage) can require trusting our friends beyond what the evidence supports. As Sarah Stroud puts it, “friendship involves not just affective or motivational partiality but epistemic partiality. Friendship places demands not just on our feelings or our motivations, but on our beliefs.“ And while it’s not clear from that quote, the general thrust of her work on this topic is that it’s more important to be a good friend than it is to have accurate beliefs about our friends.

A related line of resistance to evidentialism concerns the idea, not that we should believe things that we lack evidence for, but that we should refrain from believing things for which we have strong evidence, for broadly moral reasons. For instance, statistical generalizations about demographic groups—or inferences from those generalizations to beliefs about particular people—can wrong people, so that we sometimes ought to refrain from believing something even when, if we were only concerned with the truth, we’d go ahead and believe it. Here’s an example from Sarah Moss:

Suppose you know that most canaries are yellow. Intuitively, when you hear a canary in the forest, there is nothing wrong with believing on the basis of your statistical evidence that it is probably yellow, and there is nothing wrong with acting as if it is probably yellow. By contrast, suppose you know that most women who work in a certain office building are administrative assistants. Intuitively, when you see a woman working in that office building, is there something wrong with believing that she is probably an administrative assistant?

She answers “yes”, and lots of contemporary philosophers have followed her.

To put my cards on the table, my sympathies have always been with evidentialism. In the course of steelmanning his opponent avant la lettre, James sums up my sensibilities quite nicely:

The talk of believing by our volition seems, then, from one point of view, simply silly. From another point of view it is worse than silly, it is vile. When one turns to the magnificent edifice of the physical sciences, and sees how it was reared; what thousands of disinterested moral lives of men lie buried in its mere foundations; what patience and postponement, what choking down of preference, what submission to the icy laws of outer fact are wrought into its very stones and mortar; how absolutely impersonal it stands in its vast augustness,—then how besotted and contemptible seems every little sentimentalist who comes blowing his voluntary smoke-wreaths, and pretending to decide things from out of his private dream! Can we wonder if those bred in the rugged and manly school of science should feel like spewing such subjectivism out of their mouths?

But rather than directly argue for evidentialism in this post, I want to complain about how contemporary evidentialists have lost their nerve; we need more of Clifford’s fire and brimstone.

New Evidentialism

Contemporary evidentialists tend to think that Clifford arrived at the right place, but for the wrong reasons. His mistake, they say, was giving broadly moral arguments for evidentialism, rather than recognizing that moral considerations are just beside the point when it comes to the question of what to believe.

Try to believe—right now, just by deciding to—that you’re reading this post on the surface of the moon. You can’t. What if I sweeten the deal? I’ll pay you $10,000 to believe you’re on the moon. Any takers? I expect not. You might think—for what it’s worth, James seemed to think this—that the problem is that you’re obviously not on the surface of the moon; your power to believe at will doesn’t extend to claims that are obviously false. I don’t think this what’s going on. Try to believe—again, right now, just by deciding to—that there is an even number of stars in the Milky Way. That’s not obviously false. It might well be true. But we still seem powerless to believe it at will.2 By contrast, try to believe, just on the basis of my say-so, that I was born in New York City. I bet you can believe it. I bet you do already. What’s the difference? My saying that I was born in New York City is good evidence that I was, but my being willing to pay you $10,000 to believe you’re on the moon is not good evidence that you’re on the moon.

So what? Here’s how contemporary evidentialists develop a view about reasons for belief out of the observations above. In a very small nutshell, they argue that examples like the ones above show that we can’t believe for moral or practical reasons, and then they appeal to a version of the familiar “ought implies can” principle to argue that it’s not the case that we ought to believe for moral or practical reasons.3

Way back in the 17th century, Blaise Pascal worried about this sort of objection to his famous “wager” argument for believing in God. Here’s what he said: “I know you can’t directly believe in God just because you recognize that it’s a good bet. But once you’re convinced it’s a good bet, you should want to believe in God, and once you want to believe in God, there’s stuff you can do. Go to church, hang out with Christians, and there’s a good chance that eventually their beliefs will rub off on you; people tend to adopt the beliefs that predominate in their social circles.” Empirically, I think Pascal was on reasonably firm ground. If you have non-evidential grounds for wanting to believe something, you can’t directly choose to believe it on that basis, but you typically can exert at least some level of indirect control over your beliefs, taking actions that influence which beliefs you’ll hold in the medium to long run.

What do contemporary evidentialists have to say about this sort of response? They fully concede it. Painting with a broad brush, contemporary evidentialists put a lot of weight on the distinction between reasons to believe something, and reasons to bring it about that you believe something: only evidence can be a reason to believe something, but all sorts of non-evidential considerations can be reasons to bring it about that you believe things. Here’s Nishi Shah:

If [what beliefs to bring about] were the question at issue for the ethics of belief, then the pragmatist would win hands down. I know of no compelling argument that truth is always more desirable than other values such as happiness or virtue, or even that attaining true belief never conflicts with attaining other values. But the question what belief to bring about is distinct from the question what to believe.

I have mixed feelings about this response. On the one hand, I do think the distinction here is a genuine and clarificatory one. But on the other, I think an evidentialism that is only concerned with reasons for belief, and not at all with reasons to bring it about that one believes, amounts to conceding all the interesting and important ground to James and his fellow travelers.

After all, opponents of evidentialism can simply accept the distinction and state their positions in terms of what we have reason to bring it about that we believe. Perhaps friendship doesn’t give me a reason to believe, in this technical philosopher’s sense, that my friend is being truthful. It merely gives me reason to make myself into the sort of person who tends to believe the best about his friends—the sort of person who doesn’t ask the sort of awkward questions that might lead to concluding that my friend was actually being kind of a dick, in this particular instance.

Or they can concede that the evils of gender discrimination and stereotyping are not, strictly speaking, evidence against this or that woman’s being an administrative assistant. Rather, they are reasons to cultivate habits of attention and interpretation—a disposition to be unusually attuned to the possibility that individuals are atypical members of the groups to which they belong—that will de facto lead to my being very slow to form even tentative, probabilistic judgments on the basis of demographic base rate information.

Any version of evidentialism that concedes all this ground is barely worth defending, if you ask me. It abandons the ethos that James describes in the passage I quoted above: the ideal of following the evidence where it leads, even when it leads to unwelcome places. That’s the evidentialism I want to defend. (And for what it’s worth, that’s the evidentialism that seems to engage students in intro classes.) I want to say that someone who conscientiously follows the evidence to an unflattering conclusion about a friend has not thereby failed them as a friend, and is not thereby open to moral criticism. And while I’ll happily concede that it would often be rude to flaunt one’s priors—”oh, you’re in the engineering department? I just assumed you were a secretary, given that most of the women in this building are secretaries and I didn’t have any further discriminating information about you in particular”—I want to insist that there’s nothing wrong with being the sort of person who has such priors, and who rationally interprets evidence about other people in light of them. More generally, I want to defend an ethics of belief where following the evidence where it leads should be praised, rather than criticized.

To be fair, I don’t want to go quite as far as Clifford. If my spouse, dying of premature Alzheimer’s, is chatting about plans for a vacation next year that I know she won’t be alive to see, I’m not going to berate her for failing to base her beliefs on realistic prognoses.4 I’m not willing to say it’s wrong always, everywhere, and for anyone to believe anything upon insufficient evidence. There are few moral absolutes, and just as I’m willing to say that stealing is (generally) wrong without condemning Jean Valjean, the sort of evidentialism I want to defend is a tempered one.

Why have contemporary evidentialists have been so timid? One reason, I suspect, is that to the extent that defending evidentialism is associated with defending Clifford’s absolutist, uncompromising version of it, the project looks hopeless. A tactical retreat will seem the wisest course. And I agree! I just think they retreated too far. Here’s another possibility. The new evidentialist arguments I gestured at are largely exercises in a priori conceptual analysis, with concepts like “reason”, “belief”, and “doxastic deliberation” as the targets. Whatever their limitations, they at least hold out the promise of providing secure answers from the armchair. As I interpret the Clifford/James debate, it’s messier, and turns on empirical social and psychological questions about the likely effects of adopting different sets of norms concerning what habits of attention to cultivate, when to criticize people, when to reject criticism as unfair, and so on. Knock-down a priori arguments are not in the offing. For philosophers who place a priority on avoiding epistemic trespassing, the Cliffordian project can look downright disreputable.

Well, I’m willing to risk my reputation. Whether “submission to the icy laws of outer fact” is something we should cultivate and admire remains a pressing question, even if—sadly—it can’t be resolved just by drawing careful distinctions. While Clifford’s answer was oversimplified, it’s worthwhile for philosophers to explore answers in a similar spirit, even if mounting a convincing defense of one will not be a purely armchair endeavor. There’s a more general philosophical failure mode here. It involves beginning with a messy question we care about, discovering a nearby question that our preferred methods let us answer more cleanly, and then treating that simpler question as the one we were interested in all along. In this case, the result is an “ethics of belief” from which the ethics has been analyzed away.

____________________________________________________________________________________________

Over at least the last few months, a number of artificial-intelligence companies have been paying academic philosophers to train their models to be better at doing philosophy. I know this because I’ve been doing it myself. While the details vary, in general the work involves prompting AI models with questions designed to test the limits of their philosophical reasoning abilities and then grading their responses, in the hopes of generating data that will be useful in training the next generation of philosobots.

Why would we do this? The money is fine, but very far from life-changing. (I’m not talking about the numerous philosophy Ph.D.s employed on a full-time basis by AI companies, but about professional academics who retain their jobs while working for AI companies on the side.) As Chronicle readers know, one of the main ways generative AI is already affecting the academy is that it’s making it easier for students to cheat. Why would we willingly accelerate that process, making our day jobs harder?

This concern — that if AI gets too good at philosophy, then philosophy becomes harder to grade, and thereby harder to teach effectively — presupposes that there’s something good about teaching philosophy effectively. Of course, I think there is — otherwise why become a philosophy professor? But once you say it out loud, it’s hard not to wonder whether some of the very same values advanced by teaching philosophy to people might not also be advanced by teaching philosophy to AI. That is, there’s at least the hint of a tension between the following two thoughts:

1. It’s good to spread the ability to reason philosophically to our students, via teaching.

2. It would be bad if access to high-quality philosophical reasoning became extremely widespread, not just to our students but to the rest of the world as well, via AI.

When I say “the hint of a tension,” I’m not just being cute; this isn’t a polite way of saying the two thoughts are inconsistent. There are coherent ways to hold them both. But when you see them juxtaposed together, it’s hard not to ask the follow-up question: What is good about teaching people philosophy, and is that threatened by AI being good at it?

There are two kinds of answer one might give to the question about the value of philosophy. The first is instrumental: Philosophy is valuable because the skills it cultivates are useful for doing things other than philosophy. The second is intrinsic: The activity of grappling with philosophical questions and achieving some measure of understanding — even if it’s only understanding just how thorny the questions are — is valuable in itself. I think both answers are right; one and the same activity can have both instrumental and intrinsic value. But neither the instrumental nor the intrinsic value of philosophy need be threatened by AI getting good at philosophy.

It’s worth pausing here to ask whether the prospect of AI that’s actually good at philosophical reasoning is imminent enough to motivate this discussion. I have colleagues who think we’re overreacting when we discuss how to radically modify our assessment practices in light of student AI access. As they see it, large language models aren’t writing papers that would earn an A in an advanced seminar, let alone producing strong writing samples for graduate-school applications; we can chill out, at least for now.

I held a version of that view as recently as November 2025. In a Substack post I wrote then on grading and AI, I claimed that LLM-generated philosophy was glib and superficial, the kind of thing that read fluently at the sentence level but couldn’t sustain an argument for more than a paragraph or two. For that reason, I was comfortable continuing to rely on take-home papers as a major portion of the grade in my classes. While I threw in hedges about how I didn’t think there was any in-principle barrier to LLMs getting much better at philosophy, I thought we still had some breathing room.

I don’t think that anymore. I’m not sure if I wasn’t working with the right models back then, or if things really have changed a lot in just half a year. But one of my main takeaways from gig work over the last few months is just how hard it is to catch the most sophisticated frontier LLMs in philosophical blunders. When ChatGPT first emerged, a common pastime among my philosopher friends was posting screenshots on social media of its more egregious mistakes; prompts involving logical complexity would tend to produce answers that were both confident and comically incoherent. Now, finding errors in the responses of frontier models takes hours of expert paid labor, and the errors you do find — which typically require misleading prompts — are much less blatant than they used to be.

More striking, though, is what happens when you stop trying to trip the models up. When I straightforwardly prompt a frontier LLM with a technically sophisticated and deeply unobvious question — “What challenges are there to simultaneously holding Ramsey-style analytic functionalism about mental categories and endorsing a theistic argument from design?” — the model typically sees what I’m getting at and answers better than I’d expect from my graduate students, of whom I think very highly. I picked that example because it involves two ideas that are not typically discussed in the same context, and so you know the model isn’t just straightforwardly summarizing information from its training data: It has to reason its way to a novel answer. So when I ask how we should feel about LLMs being good and quickly getting better at philosophical reasoning, I’m talking about the situation as it stands today, not some hypothetical future.

Back to our central question: Does the existence of LLMs that are good at philosophy threaten either the instrumental or intrinsic value of philosophy? Let’s start with instrumental value. You might think the idea that philosophy has substantial instrumental value is a bit goofy; the stereotype of the philosopher with his head in the clouds who can’t navigate practical affairs goes back to the ancient Greeks. Departmental advisers are fond of rebutting this stereotype with data on things like LSAT scores and starting salaries; and while they’re right that philosophy majors do just fine in the labor market, I want to talk about a very different sort of instrumental value philosophy can provide.

I was listening to a podcast the other day in which one psychologist remarked to another that a research collaboration he was involved in had effectively stalled for a year because he and his collaborator — who came from a different part of the discipline — hadn’t realized that they were using a crucial concept differently. As a result, they were talking past one another and thus finding themselves unable to agree on a promising strategy for investigating their target phenomenon.

My first reaction was guild pride: “This is why research teams should include philosophers! Identifying subtle conceptual variation and thereby facilitating communication between people with different background assumptions is something we excel at!” My second reaction was a bit deflating. “While having some conversations with a good philosopher would likely have saved those psychologists a lot of time, they could have saved even more time if they’d typed up their perspectives as a prompt to ChatGPT or Claude, and then asked the LLM to play philosopher-counselor.”

A world in which people can get this sort of conceptual clarification without having to work with a philosopher — who’d need a lot of area-specific knowledge anyway, since mere philosophical ability typically won’t be enough to diagnose confusions whose substance you’re clueless about — is a world in which all sorts of projects move forward more smoothly. Misunderstandings get caught faster. Verbal disputes get resolved before they consume a year. It might sound naïve, but it’s precisely because I believe in the instrumental value of philosophy that I’m bullish on LLM-philosophy. If the sort of conceptual plumbing we provide is genuinely useful, as I think it is, then widespread access to it is good news.

Of course, there’s something bittersweet about contemplating a world in which the distinctive value I’m able to provide is widely and cheaply available and doesn’t require people like me to provide it. I’m reminded of the story of John Henry, the legendary steel-driver who raced a steam-powered drill to tunnel through a mountain, won, and died with his hammer in his hand. But while John Henry is a romantic figure and his story evokes a kind of wistful nostalgia, the broader arc of history it represents should be celebrated, not regretted. We don’t actually wish, on reflection, that tunnels were still being driven through mountains by men with hammers. Why does the story have the poignant effect it does, then?

In standard tellings, steel-driving isn’t just a means to an end for John Henry: It’s not just the way he pays the bills. Rather, it’s something he excels at, and which he takes pride in excelling at. If we think that achievement and excellence are intrinsically valuable, then there is something lost when technological developments render certain sorts of activities obsolete. Not the instrumental value of the work — that’s provided better by the new technology — but the intrinsic value the activity had for those who excelled at it.

So should we worry about philosopher-LLMs displacing the activity of philosophy as done by humans, and depriving us of the intrinsic value we derive from doing philosophy? I don’t think so. There are plenty of other activities that humans derive intrinsic value from in which we’ve already been surpassed by machines, but we keep plugging along. Chess is an obvious example. It didn’t take contemporary machine-learning methods to beat us at chess; Deep Blue defeated Garry Kasparov way back in 1997. But people didn’t react by giving up chess. Quite the opposite — in the last three decades global participation in chess has substantially increased.

To be sure, learning and playing chess today is different from what it was like before computers excelled at chess. For one, there are a lot more worries about cheating in competition (especially online). But I think that downside pales in significance to the way in which AI chess has broadened access to expertise. In 1990, if you wanted to know where you’d gone wrong in a game you’d lost, you’d need a human coach. Today, you’ll probably run your game through an engine like Stockfish, which will analyze your mistakes and suggest alternative lines you might have played, and it will do a better job than any human alive could ever have hoped to.

What might it look like if philosophy became more like chess? I can imagine a world where people who want to think hard about philosophical questions form salons where they read and discuss together, except now there’s no professor at the front of the room charging tuition. Instead, Claude or something like it sits in the background, ready to clarify a passage, push back on a confusion, or suggest connections to other literatures that might not have occurred to the participants.

I have no idea if that’s where we’ll end up. But when I try to picture it, it doesn’t strike me as a worse world than the one we currently have. More people doing philosophy, throughout their lives, with the help of expert interlocutors who weren’t available to them before. Today, most neighborhood book groups wouldn’t dare tackle Kant. Maybe in the future they will.

I imagine many current philosophy instructors will think I’m being Pollyannaish. The rosy picture I’m painting depends on there being people who want to engage with philosophy and who’d benefit from access to expert interlocutors. But where do those people come from? In the current world, a fair number of them are people whose taste for philosophy was cultivated in undergraduate classes, often against their own initial expectations. If AI is making it harder to cultivate that taste — by making it too tempting to outsource the assignments that help build one’s philosophical muscles — then the net effect of capable AI on people’s engagement with philosophy may still be regrettable.

I take that worry seriously. Here’s why I’m looking on the bright side. For the malign use cases — undergraduates outsourcing their term papers to chatbots, mainly — AI is already capable enough. In my judgment, a student who wants to avoid the work of writing a philosophy paper can already get a high-quality one out of a chatbot, and further improvements at the frontier of AI technology don’t really change that.

In other words, the ceiling for what we can demand in the sorts of introductory classes that whet a student’s appetite for philosophy is just not that high. The ceiling for the beneficial use cases of philosobots is much higher. The interdisciplinary research team doesn’t just want the philosophical acumen of an impressive undergraduate, they want access to the most philosophically sophisticated intelligence possible. For the neighborhood reading group, the better the AI’s ability to clarify the conceptual questions they’re discussing, the more they get out of the experience.

Will professional academic philosophers like me still exist in this brave new world? I’m not eager to go the way of John Henry; I like my job and I want to keep it. Even once LLMs are better at philosophical reasoning than we are, there may still be a place for us; there are still professional chess players, after all. Moreover, universities are slow institutions: slow to adopt new technologies, slow to restructure curricula, slow to make any major changes. That inertia means that being technologically replaceable and actually being replaced are very different things, and the gap between them is likely to be measured in decades rather than years.

But the important question for most people isn’t whether current members of the academic philosophy guild get to keep their jobs. It’s whether the world is made better or worse by AI getting good at philosophy. And if I’m right that both the instrumental and intrinsic value provided by philosophy would be more plentiful and more broadly distributed in a world of expert philosopher-AIs, then such a world should not be feared, but welcomed.

____________________________________________________________________________________________

Postscript 1: On Grading

Back in November I thought I could stick with take home essays, while hedging by using in-class quizzes and final exams. I now think take-home essays are a lot more compromised as assessment tools than I did then. But I still have the same reasons I had back then for not wanting to give up on them entirely. As a refresher, I want students to get practice building arguments that they sit with, think about, and develop over longer periods of time than we can test in an in-class exam. If the only essays my students have to write are in class, I worry they won’t get a chance to practice the sort of disciplined thinking that I think philosophy classes ought to train.

So here’s my plan for the fall. I’ll assign take home essays, but all students—not just those I suspect of over-relying on LLMs—will be interviewed about their essays. That might involve raising objections not discussed in the paper, or asking why they formulated some crucial claim the way they did, or what role some paragraph is playing in the overall argumentative structure of the paper. The idea won’t be to distinguish those who used LLMs from those who didn’t. I’m still not confident I can do that.1 Rather, the hope will be that even among students who do use LLMs, doing well on the interview will require the sort of work that papers were designed to test all along. My hope is that a student who produces a paper in one shot, reads it to get an idea of what it says, and then submits it, will do poorly.

This approach isn’t original—it’s basically what they’ve been doing at Oxbridge for a long time, as far as I understand.2 That’s a point in favor; better to use an established method than create something out of whole cloth. It’s certainly not a perfect substitute for what papers used to test. For one, quickness and cleverness will matter in a way they didn’t before. I’ve had excellent instructors who weren’t all that quick on their feet, and I worry that students who really do understand the material, but who just aren’t as verbally fluent—especially ESL students, perhaps?—will do worse in this new regime, even if they are developing the skills paper-writing was meant to test in the before-times. The approach also doesn’t scale all that well. It’s certainly feasible in the seminar that I’ll teach in the fall. I think it’s also feasible in large lecture courses at a school like Yale, given the numbers of students per graduate teaching fellow. But I know there are schools where each instructor—whether faculty or grad student—is responsible for too many students for an approach like this to be realistic. But it’s the best I’ve got for now.

Postscript 2: Criticisms from Hu and Rothfeld, and my replies

Lily Hu and Becca Rothfeld both wrote letters to the editor at CHE in response to my piece, and I replied. That exchange can be read here. But I want to add some further thoughts on the Rothfeld letter that I couldn’t fit into my reply.

As I see it, Rothfeld is describing an ideal of philosophical education—not just being taught by a human rather than a machine, but from an inspiring human with whom the student has a deep, personal relationship—that I’m happy to grant LLMs cannot meet. I’m mostly happy to recognize what she describes as an ideal—more on that “mostly” later. But I think not much follows about whether we should welcome improvements in the abilities of philosobots from the fact that they cannot achieve this ideal.

The ideal she describes is necessarily scarce—there are only so many philosophy PhDs, and even in that small group, most of us don’t have the charisma and magnetism to inspire followings like Socrates (I know I don’t). So given that most people will never get the sort of philosophical education Rothfeld thinks is ideal, what should we want for them vis a vis their engagement with philosophy? I imagine Rothfeld will say: “If they’re interested in philosophy they should read books. But not talk to LLMs.” Well, I agree they should read books. But I don’t see why they shouldn’t also talk to LLMs! After publishing my piece I got an email out of the blue from a colleague of mine, a Sterling Professor (very fancy) telling me about his wonderful conversation with Claude about Kant’s political theory and ethics. I’m sure he has no shortage of books on his shelves! But skilled readers of books will find that the books raise questions, and they may want to discuss those questions. I can imagine the reply: “Organize a reading group!” But this is to walk right back into the point about scarcity. To quote my illustrious colleague: “who else would talk with me for two hours about political theory at 11pm?”

Basically, I think Rothfeld’s argument is a bit like criticizing people who are trying to improve the sound quality of phonographs on the grounds that recorded music can never live up to the experience of hearing Maria Callas live at La Scala. I bet it can’t! But I’d still rather hear her through speakers than not at all.

Back to that “mostly.” Of course, personal relationships with excellent teachers are wonderful. I’ve had many wonderful teachers and mentors over the years to whom I am immensely indebted, and very grateful. Still, I’m uneasy with the ideal of understanding philosophy through a deep personal attachment to an inspiring teacher. To my ears, it is too closely tied up with a tradition of discipleship in philosophy. There’s a stereotype among analytic philosophers that continental philosophy is particularly prone to this failure mode—to being overly deferential to guru figures—but I don’t know enough about continental philosophers or their social dynamics to know if this is fair. I do know of plenty of examples of philosophers who inspired cult-like followings outside the continental tradition. Ludwig Wittgenstein, Karl Popper, Ayn Rand, Leo Strauss... In each case, I hear stories about slavish students who’d leap down the throats of outsiders who think they’ve hit on some powerful objection to the master’s teachings. I think it’s a good feature of the analytic tradition that we tend to at least give lip service to the ideal of separating ideas from the people who defend them. And I think that ideal sits awkwardly with the one Rothfeld was describing. If we value the practice of evaluating arguments independently of the people making them, perhaps it’s not such a leap to valuing the practice of evaluating arguments independently of whether it’s people making them at all.

Postscript 3:

One more round of back and forth, from Rothfeld and me. I don’t know if that will be the end on her end but it will on mine.

Today’s post will be a few of my own reflections on the report itself, as well as the broader issues it treats.

In a nutshell, my point is going to be that there are lots of decisions that, in an ideal world, would be made in a flexible, holistic, discretionary way, but which cannot be made that way by institutions that have lost the public’s trust. If your decisions need to be trusted, and people don’t trust you not to abuse your discretion, you will be better off tying your hands and relying on decisionmaking procedures that are manifestly imperfect, but publicly legible. (At the end of the post I’ll turn to the question of why trust is so important for higher education.)

Here are a few places in the report where I had thoughts along these lines. First, concerning undergraduate admissions. The report has much to say, but the general thrust is skepticism about how the “holistic” admissions process has been used, such as in favoring varsity athletes, or the children of alumni, faculty, and donors.1 In their words, “The top priority in admissions decisions should be academic achievement.” The most concrete recommendation in the section is the adoption of an explicit minimum SAT score required for admission. Criticisms of standardized tests are by now familiar, and I don’t intend to rehash them here.2 Let’s grant for the sake of argument that in an ideal world, admissions officers would use their best judgment to admit students based on a wide range of factors that interact in complex enough ways that their decisions could not be described by any simple rule like a minimum SAT score.

However, in a world where the discretionary judgments of admissions officers are treated with suspicion rather than trust, this sort of process is much harder to defend. To the extent that the public suspects that an opaque, discretionary admissions process will be abused, we should be looking to rely more heavily on transparent criteria like standardized test scores, because decisions made on the basis of such criteria can be justified to a skeptical public. When universities strongly weight performance on standardized tests in admission, they broadcast openness to students from a broad range of class, cultural, ethnic, and religious backgrounds, so long as they can meet an objective standard.

Another place where I had similar thoughts was in the report’s discussion of grading. The report recommends adopting “a 3.0 mean, or some other college-wide standard, so that letter grades can once again be used in a reliable and comparable way.” Here too I think we have a conflict between the ideal policies that could be used in a high-trust world, and the policies that make sense for the world we’re actually in. Of course some classes have stronger students than others. So to insist that each class have the same average grade can seem absurd—isn’t it best for students to receive grades that reflect the strength of the work they actually did, rather than to grade under the manifestly false assumption that students in each class produce a similar distribution of excellent, good, average, and below-average work?3

In a world where professors could be trusted to award grades that reflect the strength of students’ work, yes, that would be best—I would oppose the imposition of any rigid constraints on grading in such a world. But we’re not in that world. As the report nicely explains, grade inflation is a collective action problem. Faculty members face various incentives to inflate their grades—most obviously, it makes students happy, which helps with course evaluations—and once other faculty are inflating grades, few faculty are willing to risk scaring students away by developing reputations for harsh grading.4 Against this background, “my class was composed of very strong students, all of whom did A-level work” will tend not to inspire much confidence. So, to the extent that we hope that a strong transcript from a university like Yale will be taken as a signal of excellent work—an admittedly non-trivial assumption, to which I’ll return—we should favor grading reform along the lines discussed in the report.

The section also has a report entitled “Politics and Intellectual Pluralism.” They write:

“Our committee spoke with hundreds of Yale students, faculty, alumni, and administrators about the issue of ideological conformity. No subject we took up was more contested.”

From the way the section is written—especially the lack of concrete recommendations—I suspect the issue was as contested on the committee as it was among their interviewees. So this section of my post will be less indebted to their report than the previous two.

A common objection to the idea that universities should explicitly pursue intellectual diversity—especially political diversity—is that intellectual diversity as such is not of value in the search for truth. Rather, what’s important is diversity of good ideas. It’s no problem if the geology department doesn’t have any flat-earthers, if everybody in the medical school accepts the germ theory of disease, etc. etc. (I won’t belabor the point as I’m sure you’ve heard it before.) But which ideas are good is a matter for informed expert judgment. So while everybody can agree we need some debate for intellectual progress, which questions should be debated and which can be safely treated as settled is generally not something we should expect will be transparent to people without specialized expertise.5

In a world very different from ours, where highly trusted universities were having internal debates about whether to explicitly pursue political diversity, an argument along the lines of the one in the previous paragraph might well deserve to carry the day. “Let’s just trust the individual departments/disciplines to determine which perspectives on questions in their areas ought to be represented in their research and teaching.” But in a world where universities face external skepticism from a public who suspects that decisions about research and teaching are systematically ideologically biased, “just trust us” won’t cut it. On my facebook page and in blogs I’ve mostly seen scorn for Harvard’s viewpoint diversity initiative, which is reported to aim to bring more right-leaning scholars to their campus. But the criticisms I’ve seen—this one is typical—strike me as missing the point. They move far too quickly from describing how hiring decisions would ideally be made—of course in an ideal world, a program like Harvard’s would be unnecessary at best and malign at worst—to conclusions about how they ought actually be made. But it’s a familiar point that in a non-ideal world, the best policy isn’t always to approximate what you’d do in an ideal world.

Throughout this post I’ve been presupposing that it’s important for institutions like Yale that they have some measure of public trust. In closing, I want to say a bit about why that is. Of course, there are obvious financial reasons. It costs a lot of money to run a modern university, and that money comes from tuition-paying students, federal grants, donors, and (for public universities) state budgets. If students, congress, philanthropists, and state governments don’t trust universities, then these sources of funds could dry up. If these financial reasons were the only ones trust was important, then federal grants could in principle be replaced by private funding. And for some of what universities do—much scientific research—trust plays this merely instrumental, enabling role. The function itself doesn’t depend on trust, but trust is required to get the resources necessary to carry out the function. One could imagine a world where research functions that are currently carried out in universities are instead carried out in very different institutions, which don’t depend on public trust. Remember Bell Labs? They supported lots of groundbreaking scientific and engineering research in the mid 20th century, and while they needed money to do so, they didn’t need a public reputation.

Some of what universities do, however, is more intimately tied up with trust. I have in mind the function of elite selection and formation, which is part of the social role of undergraduate colleges like Yale. This function isn’t foregrounded in the committee’s report,6 but I think it’s important one which we shouldn’t shy away from. Graduates of elite colleges wield outsize influence in society—at least for now!—so such colleges should aim to select and shape their students so that this influence is wielded well.

The idea that one important function of institutions like Yale is in the formation of an elite is an awkward one, apt to be unpopular on both sides of contemporary culture wars. On the left, there’s a longstanding egalitarian impulse that is uncomfortable with the idea of hierarchy and power differentials of any sort. To the extent that institutions like Yale reproduce divisions of class, power, and privilege, they are part of what’s wrong with society, and can be tolerated only insofar as they help—via the development and propagation of egalitarian ideas—to bring about their own obsolescence. While the right is more comfortable with hierarchy, they don’t like the idea that universities ought to have any special role in picking who occupies the top spots in any hierarchies of importance. Both left and right will tend to see the idea that universities should continue to play their historic role in elite formation as objectionably antidemocratic, even if they’ll flesh out that complaint in very different ways.

But I am a knee-jerk centrist, so I reject both the utopian egalitarianism of the left, and the anti-intellectualism of the right. Nobody has yet figured out how to organize large complex societies—the sorts of societies in which famine is unheard of and people have the leisure to read and write Substack posts for fun—without substantial de facto inequalities of wealth and power. So the question of what sorts of institutions will determine who ends up wielding outsize wealth and power are unavoidable.7 And against the (contemporary, populist) right, to the extent that universities are doing a bad job at that task, there isn’t some alternative set of institutions ready to pick up the slack. It’s no accident, I suspect, that right-wing populist governments—which are typically brought into power by movements whose raison d’etre is distrust of traditional elites—are so characteristically plagued by corruption, nepotism, and cronyism. When people don’t trust credentialing institutions such as universities, they’re more likely to form professional networks based on family connections and personal loyalty, with all that entails.8 While I admit there’s lots not to like about our current elite, I think we should aim to do a better job of elite formation rather than trying to live without it.

This elite-formation role is intrinsically—not merely instrumentally—bound up with trust. Institutions like Yale shape an elite only because other institutions—law firms, investment banks, consultancies, technology companies, governments, etc.—are disproportionately likely to hire graduates of such colleges. Why do they do that? In large part because they take admission to and graduation from such colleges as a signal—of qualities such as intelligence, discipline, judgment, and conscientiousness. But to treat an elite college degree as that sort of signal that just is a kind of trust. If outside the ivory tower it’s widely thought that admissions decisions are arbitrary and grades don’t distinguish levels of achievement, then degrees can’t signal what they traditionally have, and colleges can’t play their historical role in elite formation.

The relevance of intellectual diversity to the signaling function of universities is less obvious—and there are reasons to care about intellectual diversity that have nothing to do with the signaling/elite formation function of universities—but I still think it’s part of the story. Lots of the institutions for which university degrees have historically served as a relevant credential have internal cultures quite different from what’s typical in the Ivy League. If employers think someone can coast through a four-year degree at an elite institution without encountering or having to sympathetically engage with conservative ideas, then they might be worried about a graduate’s likelihood of integrating smoothly into institutions where those ideas are much more common than on campus. Even many left-leaning legal institutions might be concerned about hiring aspiring lawyers who have developed a habit of responding to arguments from right-wing lawyers not with reasoned engagement—which is necessary for a legal system to function—but with shoutdowns and heckling.9

There’s plenty of evidence that a variety of employers think the strength of the signal carried by an elite degree is weakening, in part for reasons I’ve discussed in this post. Is that something to lament? That’s a big question, and while exploring it in depth would be a topic for another post—probably not one I’ll ever write—my money is on “yes.”

Over the past few years, robotaxis have expanded across the country. You can now hail a Waymo in Atlanta, Austin, Dallas, Houston, Los Angeles, Miami, Nashville, Orlando, Phoenix, San Antonio, and San Francisco. But robotaxis have faced opposition in Boston, DC, and New York.

Since last summer, I’ve been worried that robotaxi policy is moving towards a bipolar equilibrium—laissez-faire in some red states and NIMBYism in some blue states. I’m not the only one with this worry. Kelsey Piper calls this possibility: Red states get Waymos. Blue states get studies.

I’m also worried that Congress will respond to blue state opposition by passing a bill that preempts state and local regulation of automated driving. In fact, a preemption bill passed an initial vote House committee in February. The bill would severely limit efforts to steer robotaxi deployment in a more socially desirable direction.

So for the past several months, I’ve been trying to persuade anyone who will listen that neither of these outcomes is inevitable. States and cities can craft regulation that curbs externalities, protects consumers, and ensures that robotaxis have a positive impact on our transportation system.

I put these arguments into a paper I wrote together with Bryant Walker Smith, the godfather of automated driving law. It’s called Regulating Robotaxis, and it was just published in the Southern California Law Review.

In the paper, we offer specific proposals for how to address congestion, emissions, wear-and-tear on infrastructure, and passive surveillance. We argue that regulation can encourage competition by banning lock-in contracts and enabling one-stop access to multiple networks. And if the robotaxi market turns out to be a natural monopoly, we can still protect consumer autonomy by restricting personalized pricing and planning for emergencies.

We also show how policymakers can take advantage of the opportunity robotaxis create to redirect transit investments to high-throughput routes, expand mobility for people with low incomes and people with disabilities, and abolish on-street parking, liberating urban land for higher value uses.

You can read it all here.

But since reading a 65 page law review article can be a slog, I also have a video!

Earlier this month, Hunter College in New York City held a conference on cities and robotaxis. The auditorium was packed, and the audience asked tough questions.

I was on a panel with three women who’ve held important roles in NYC government. We talked about how to regulate robotaxis. You can watch it here!

In the interest of fairness, I want to share other perspectives from that conference. The first panel had speakers from a range of backgrounds and included a rep from Waymo. The second panel had speakers who were more critical of automated driving. Taken together, these panels show the wide range of opinions on this issue.

Last week, we got some good news. Charles Allen, a DC Councilmember who had been blocking robotaxis, proposed legislation that would allow their deployment. I have a lot of questions about his bill, but it’s an improvement from a de facto ban.

I think this public debate about robotaxis is just beginning. And I hope to post more on this topic—as long as the many philosophers among our subscribers can resist the temptation to torture me with trolley problems!

My very first publication in philosophy was on statistical significance testing. In one sense, I still accept the core argument of the paper. But I explicitly rested it on an assumption that I saw back then as relatively innocent, and which I now think is where pretty much all the action is.

Here’s how I came to be thinking about significance testing. I was broadly sympathetic to Bayesian views in the philosophy of science.1 But I was aware that there were foundational disagreements between Bayesians and another camp, “frequentists”, concerning statistics. Moreover, I was aware that most actual empirical research used frequentist-inspired methods for analyzing its data. This seemed to me to create a puzzle. Should Bayesians be skeptical of empirical research that uses frequentist methods? That seemed like a big pill to swallow! I started from the presumption that empirical science couldn’t be in all that bad shape, and wanted to understand for myself why the theoretical disagreement between Bayesians and frequentists did not mean that Bayesians should adopt a wholesale distrust of scientific practice.

The answer I gave turned on a simple idea. Researchers won’t normally bother collecting and testing data unless they think there’s some real chance of finding something interesting. If that’s right, then the mere fact that a researcher chose to run this test already tells you something—in the lingo, it tells you something about the researcher’s prior probability judgments. Under those conditions, a statistically significant result can end up being more informative than the formal theory of significance testing, taken in isolation, would suggest.

That was the core of the argument. (If you didn’t follow, don’t worry, I’ll explain in more detail below.)

Even at the time, I was explicit that the argument depended on the assumption that researchers follow norms of “predesignation”; you’re supposed to decide what tests you’ll run on your data before you collect it. In the absence of that assumption, the reasoning breaks down. But I was able to cite various places where frequentists endorsed those norms, and didn’t think much about actual scientific practice. It wasn’t remotely on my radar that there might be myriad ways to obey the letter but not the spirit of such norms, let alone that people might frequently violate the letter of the norms too.

Significance Testing

To explain in a bit more detail, I’ll need to say a bit about how significance testing works, and how it can sound unsatisfying to someone with Bayesian sympathies.

A null hypothesis is just a kind of default or baseline claim—nothing interesting is going on. If you’re testing whether a drug works, the null hypothesis says it doesn’t. If you’re testing whether two groups differ, the null says they don’t.

Significance testing is built around trying to rule out that default. You ask: if the null hypothesis were true, how surprising would this data be? If the answer is “very surprising,” you reject the null. There’s a connection here to Karl Popper’s falsificationist philosophy of science: instead of trying to directly confirm a hypothesis, you test it by trying to rule out alternatives that are inconsistent with it.2 Significance testing is sometimes billed as the minimal modification of Popper’s approach that it needs to make to handle the fact that oftentimes the theories we want to “falsify” don’t make deterministic predictions, but only stochastic ones. Want to falsify the hypothesis that a coin is fair? Good luck! No matter what sequence of flips you observe, that’s consistent with the hypothesis that it’s fair. But you can run a significance test where you’ll reject the hypothesis that the coin is fair if the proportion of heads is far enough from 50% in a long enough series of tosses.

The basic Bayesian complaint about significance testing is that it answers the wrong question. We want to know: “in light of the evidence, is the null hypothesis improbable?” Significance testing aims to answer the converse question: “in light of the null hypothesis, is the evidence improbable?” These questions are related, but they’re not identical.3 The following toy example will bring out why.

Imagine you flip a coin twenty times, and obtain a random-looking sequence:

H T H H H T H H T H T H H H T H T T T T

How likely was it that we’d obtain exactly that sequence, if the null hypothesis that the coin is fair is true? Pretty unlikely! One half to the 20th power. But that’s no evidence that the null is false. Why not? Well, it’s not like there’s some alternative hypothesis on which a sequence like the one above was exactly what we should have expected to see. From a Bayesian point of view, if you’re going to decrease your confidence that the coin is fair, you’ve got to increase your confidence that the coin is biased. And if the coin were biased, it’s not as if you’d expect it to produce a sequence like the one above.

By contrast, suppose instead we’d flipped the coin and had gotten 20 heads:

H H H H H H H H H H H H H H H H H H H H

How likely was it that we’d obtain just that sequence, if the null hypothesis that the coin is fair is true? Exactly as unlikely as in the previous case: one half to the 20th power. But this sequence would be evidence, by Bayesian lights, that the null is false. What’s the difference? In this case, the sequence is the sort of sequence you’d expect to see if the coin were not fair. It’s not just that the evidence is unlikely on the null but also that it is likely on a pretty salient, not too crazy alternative hypothesis: the coin is (extremely) biased towards heads. By Bayesian lights, what matters is not just how (un)likely the evidence was on the null hypothesis, but also how (un)likely it was on alternatives to the null, as well as how antecedently likely those alternatives are.

How would a frequentist think about cases like this? First, the frequentist would already be uncomfortable with the framing: “here’s the evidence, does it let you reject the null?” Rather, before even generating the evidence, the frequentist would want to have a plan for what test statistic to analyze. A pretty normal way of testing whether a coin is fair would involve looking at the number of heads. So if we decided to test whether a coin was fair by flipping it 20 times, and counting the number of heads, then we’d handle these cases pretty sensibly. In the first case, where we got 11 heads, we’d be able to calculate that a value for our test statistic at least as extreme as that is not at all unusual; we wouldn’t be able to reject the null.

By contrast, it would be very rare to obtain a result as extreme as 20 heads. That outcome would license rejecting the null, given the ex ante plan of counting heads.

In principle, someone could propose to test whether a coin is fair by planning to flip it 20 times and instead of counting heads, they could check for how closely the outcome matches that first “random-looking” sequence. If they did that, and then obtained exactly that sequence, they’d be entitled—by frequentist lights—to reject the null hypothesis.

Is that a bad result? I don’t particularly think so. If someone successfully predicted in advance that a coin I assumed was fair would produce that particular sequence, I might reasonably think they knew something I didn’t. Otherwise, their success would seem too coincidental. Maybe the coin is being remotely controlled? Maybe there are magnets under the table? You might have to go to some weird places to explain how someone could successfully predict such a random looking sequence, but at the same time, that success is so unlikely on the null that going to those weird places might be reasonable.

To unpack this a bit, I think what’s going on is that if a hypothesis being rejected in a significance test can be interpreted as a successful prediction on the part of the researcher, then Bayesians have a lot less reason to worry about spurious rejections of the null. Even evidence that ordinarily wouldn’t count against the null—our “random-looking” sequence that seems like exactly what a fair coin would produce—can reasonably count as evidence against the null when somebody ex ante predicted some highly specific features it would have. And that’s because it suggests there’s some not-too-implausible alternative to the null—the one the researcher had in mind—that makes the evidence unlikely in just the respects it is.

Is Running a Test a Prediction?

That’s more or less where I left things in the paper.4 The idea was that significance testing can look much better in practice than it does in theory, because the decision to run a test itself carries information—because it can be interpreted as a kind of successful prediction.

Back then I wasn’t worried about the assumption that a hypothesis being rejected in a significance test can be interpreted as a successful prediction. I flagged it as an assumption, but I thought it was basically reasonable, and I didn’t really dwell on it.

I wrote the paper in 2008–9, and it was published in 2010. While the opening shots of “replication crisis” discourse had already been fired—John Ioannidis’ “Why Most Published Research Findings are False” had already appeared—the idea that lots of science was producing spurious, non-replicable findings definitely wasn’t on my radar yet. And in looking around for this post, it seems that most people date the onset of a collective awareness of a replication crisis to the mid-2010s, maybe around 2014.

If I were writing it today, I’d frame it very differently. Rather than trying to vindicate significance testing, I’d present it as a way of understanding how it can go wrong.

The basic idea would be this: you can get a unified picture of what’s wrong with many of the practices now associated with the replication crisis by seeing that they all break the same link—the link between a hypothesis being rejected in a significance test, and the researcher having made a costly, successful, ex ante prediction.

Some cases are obvious—classic “fishing expeditions,” where researchers collect lots of data, run many tests, and only report the ones that come out significant. In this case, if they reject some null hypothesis H in a significance test, you can’t interpret them as having predicted ex ante that H was false. Rather, you can only interpret them as having predicted that some hypothesis in a large set including H is false. And that’s compatible with their having no special reason to think H in particular was false, and so with thinking that its being rejected in a significance test is not some surprising coincidence.

Some are less obvious. Even without running lots of statistical tests, researchers face lots of questions about exactly how to analyze their data, and when they can make these decisions post-hoc after the data are in—when they’re not constrained to be making genuine predictions—it’s a lot easier to run just a few tests and nevertheless have a very high chance of obtaining significant results even if the null is true.5

Or we can put the point positively by seeing how it helps make sense of the push for preregistration. Very roughly, preregistration means that before collecting or analyzing data, researchers publicly specify what hypotheses they’re going to test, what outcomes they’ll measure, and how they’ll analyze the results. Then after the fact, third parties can check to see whether the tests they actually ran are the ones they said they’d run.6 If the evidential force of a statistically significant result depends on the decision to perform a test functioning like a genuine prediction, then it makes sense to try to enforce that condition explicitly. While the idea of “predesignation” is old, actually existing preregistration databases are a post-replication-crisis phenomenon. Such databases, along with professional norms requiring their use, can be seen as a way of restoring the link—forcing researchers to commit in advance to what they’re going to test, so that a significant result really does reflect something like having stuck one’s neck out and gotten it right.

In a way, this is a slightly lame example of something I changed my mind about. The replication crisis blindsided a lot of people. I might as well write that I changed my mind about how hard it was to hijack planes after 9/11. But in my case, at least, I do think there’s an interesting underlying mistake. My argument for why Bayesians can trust significance testing was, if not a priori, at least not sensitive to the sociological realities of scientific practice. I thought that because statistics textbooks discussed norms of predesignation, you could assume they were being followed in letter and in spirit, and I could get on with the argument treating that as an assumption.

And that strikes me, in retrospect, as a familiar kind of error to which philosophers in particular may be prone: thinking that you can understand a complex social practice just by looking at the abstract, formal rules that describe it, without paying enough attention to how those rules are concretely implemented.

Subscribe now

Today’s post will be something of a sequel to that last one, in which I zoom in on what I think is a mistaken way of drawing a particular social science/humanities boundary in thinking about the study of history. Here’s the thought I’ll be targeting. While one can study history in a social scientific way, as cliometricians do—constructing quantitative models that relate things like Nile flooding to ancient Egyptian tax revenue, and thereby making various predictions about how the course of Egyptian history would have been different had the weather been different—there’s also a distinctively humanistic approach to history, which is not concerned with identifying general laws of history, but instead involves constructing narratives about what actually happened in particular times and places of interest, and without any commitments about how things might have gone differently. The humanistic historian will tell us a detailed story about how World War One in fact unfolded, without any commitments whatsoever about how it would have unfolded differently had Franz Ferdinand never been shot.

This view—that proper history concerns what actually happened, and not what might have happened—has long had influential defenders. For example E.H. Carr, in What is History? remarks dismissively that “one can always play a parlour game with the might-have-beens of history”, but goes on to tell us that these might-have-beens have nothing to do with proper history. He tells us about readers whose impatience with history “takes the form, when they read history, of letting their imagination run riot on all the more agreeable things that might have happened, and of being indignant with the historian who goes on quietly with his job of explaining what did happen…”

Or Michael Oakeshott, in Experience and its Modes, tells us that “the distinction…between essential and incidental events does not belong to historical thought at all; it is a monstrous incursion of science into the world of history.”2

Carr and Oakeshott aren’t just drawing a distinction between a more social scientific and a more humanistic approach to history, but are also disparaging the social scientific, counterfactual-concerned approach to history as misguided.3 While one can draw a distinction similar to the one they’re getting at while thinking both approaches are valuable, I think it’s helpful to dwell a bit on why one might take the hostile stance they do towards counterfactual-involving history.

One might think that counterfactual history can never be adequately grounded in evidence. I think this is a mistake, though if your stereotype of counterfactual history is that it inevitably ends up as a genre of speculative fiction—and some certainly does—I can see why this thought would be tempting, and I’ll return to it at the end of this post. But why should this be? Why couldn’t we have good evidence for historical counterfactuals?

On one way of drawing the science/humanities distinction, science is concerned with general laws, while the humanities are concerned with the particular.4 There’s certainly some sociological truth in the neighborhood. I’m currently serving on the humanities tenure committee at my university, and it’s my first exposure to a wide range of academic writing in the humanities outside of philosophy. In my experience so far, humanists love to “problematize” this or that general principle: to show that things are more complicated, more resistant to compact description, than people have thought. By contrast, scientists, whether social or natural, are more eager to find informative generalizations, even if they involve smoothing over some matters of detail. “Fuck nuance,” they say. So if you think there are no laws of history—it’s just one damn thing after another—then you might, with Carr and Oakeshott, think that it’s a mistake to approach history as a (social) scientist.

While I’ll admit there’s some good distinction in the neighborhood, both as it concerns humanities versus science in general, and as it concerns approaches to the study of history in particular—after all, it’s undeniably true that some historians analyze quantitative data sets, while others travel to remote archives and pore over dusty tomes—I deny that it’s fruitful to understand it as turning on whether they concern themselves with counterfactuals. All historians, at least implicitly, must take on commitments concerning counterfactuals. Or at least, that’s what I’ll argue here.

Counterfactuals and Causation

I’ve been using the word “counterfactual” a lot, but haven’t yet said what I mean by it. I mean to be following standard usage in philosophy, on which counterfactuals are “if, then” statements where the “if” part is understood to be false.5 “If my son doesn’t nap this afternoon, he’ll be cranky” is an “if, then” statement, but not a counterfactual, since the “if” part concerns a future scenario that is not (yet) false.6 “If my son had napped this afternoon, he wouldn’t be so cranky” is a counterfactual; imagine it uttered in a scenario where my son did not nap, and where the statement concerns what would have happened if, counter to fact, my son had napped.

Philosophers have had a ton to say about counterfactuals. They’ve both been extensively discussed in their own right—what logic is appropriate to reasoning with counterfactuals? what makes them true or false?—but also used as a tool in thinking about other topics. One of the most important topics counterfactuals are used to think about is causation. The tradition of connecting counterfactuals and causation goes back at least as far as David Hume, who wrote the following in his Enquiry Concerning Human Understanding:

we may define a cause to be an object, followed by another…where, if the first object had not been, the second never had existed.

Suppose my son Ben hits a baseball at my neighbors window, and it breaks the window (true story). On the simple Humean picture, we get to say that the baseball was the cause of the broken window because if it hadn’t hit the window, the window wouldn’t have broken. Or, my grandmother’s smoking was the cause of her contracting emphysema because, if she hadn’t smoked so much, she wouldn’t have gotten emphysema.

Sadly, this simple definition can’t be quite right. Suppose Brutus stabs Caesar, killing him, with his co-conspirators looking on. But if Brutus had been sick that morning and hadn’t shown up at the agreed upon time, his co-conspirators would’ve done the deed without him. What caused Caesar’s death? Being stabbed by Brutus. But it’s not true that, if not for being stabbed by Brutus, Caesar wouldn’t have died. His dying on that day was “overdetermined”, in the lingo.

There’s a rich tradition not just in philosophy but also in the social sciences of trying to iron out the wrinkles in this definition of causation.7 For example, maybe we should say that the proper counterfactual to look at, when thinking about whether Brutus’ stabbing caused Caesar’s death, is one where we imagine that Brutus didn’t stab Caesar, and nobody else did. Will that work? If so, how can we earn our right to say it? That is, can we derive it from a more general story that tells us how to translate causal talk into counterfactual talk? These are thorny questions, but what’s significant for my purposes is that the dominant reaction to noticing that the simple counterfactual definition doesn’t quite capture what we mean by “cause” is to hold on to the idea that there’s an important connection between causation and counterfactuals, and to look for other, more subtle ways of capturing it. Roughly, the idea is that whatever we want to say about causation can be expressed somehow in terms of counterfactuals, even if it’s hard to provide a crisp, general definition of the one concept in terms of the other.

Causation in History

What’s the relevance of these general remarks about causation for the practice of history? If the humanistic historian wants to talk about causation, then she’s already taking on some counterfactual commitments. But it’s very hard to imagine doing history without talking about causation.

Even if our humanistic historian conceives of what they’re doing as just crafting a particular narrative—A happened, and then B, and then C—and not talking about anything that didn’t happen, if A, B, and C are to cohere as a historical narrative, then there had better be some causal connections between them. But that’s to take on commitments to the effect that A’s having played out very differently, or not at all, would’ve mattered to if and how B and C unfolded.

That was rather abstract, so let’s consider a concrete, paradigmatically humanistic historical question—why did Truman decide to drop the atomic bombs on Hiroshima and Nagasaki? Whatever insight we’re going to have here, it’s not going to come from analyzing quantitative data. Rather, it will involve close attention to and interpretation of texts—diary entries, diplomatic correspondences, internal memoranda, and the like. The major interpretive fault line concerns whether he was mainly motivated by a desire to avoid the loss of life that an invasion would have entailed, or instead was largely concerned with geopolitical strategy vis a vis the USSR. For example, perhaps he wanted to intimidate the Soviets, and/or to guarantee that the USA would play the primary role in dictating the post-war settlement in Japan. Depending on which of these positions you hold, you’ll inevitably have different commitments about how his decision might have played out differently, if various factors had been different.

The US dropped the first bomb on August 6, the USSR declared war on Japan on August 8, and the US dropped the second bomb on August 9, all of 1945. If you think geopolitical considerations were not playing a major role in Truman’s thinking, then you should probably think that even if the USSR hadn’t declared war—and if they hadn’t already agreed at Yalta to enter the war against Japan soon after Germany’s surrender—the US still would have dropped the second bomb around the same time it did. Alternatively, if you think Truman was largely concerned with ensuring that the US, rather than the USSR, would determine what post-war Japan would look like, then you should think that if the USSR hadn’t declared war when it did, and hadn’t been planning to invade Japan, Truman likely wouldn’t have dropped the second bomb when he did, and maybe not even the first.

It’s hard to see how one could come to a view about what factors motivated Truman without thereby taking on some commitments about what would have made a difference to his decision. I picked the question of Truman’s motivations as it seems like an ideal case for the idea of history as a humanistic discipline concerned with particular events, rather than with general laws. I certainly grant that there are no historical laws that we can appeal to in answering why he decided as he did—Truman’s predicament was singular, and nobody in history before or since has faced a choice quite like the one he did. But even in that best case scenario for the humanistic conception of history, we find that we’re asking causal questions—what moved Truman to make the decision he did? What influenced his thinking?—and that the competing answers to those questions contrast in their counterfactual commitments.

Salvaging the Distinction

OK, that’s my main point. Doing history inevitably involves taking on commitments about counterfactuals, because doing history involves making causal claims, and once you’re making causal claims, you’re taking on commitments about counterfactuals. Still, isn’t there something right about the idea that some kind of particular/general distinction tracks the distinction between more humanistic and more social scientific approaches to history? I think there is. But to see it, we first need to ditch the idea that the humanistic historian is distinctive in eschewing laws.

As it turns out, the idea that science involves looking for general laws is a bit out of fashion, and I think for good reason. A standard potted history of the philosophy of science says that in the mid 20th century, philosophers treated fundamental physics as a kind of model for all of science. Fundamental physics looks for general laws—of gravity, electromagnetism, and the like—so we should understand chemistry, biology, and the rest of the sciences as doing something similar. Social scientists look for economic laws, or historical laws. But over time—starting in the 70s and 80s, and still holding today—a rough consensus emerged that lots of science isn’t fruitfully understood as searching for or discovering laws. Evolutionary biology has made great progress, but it’s hard to identify exceptionless, universally applicable laws of evolution. For almost any such putative law, you’ll be able to find counterexamples. Rather, progress in evolutionary biology has consisted in developing more and better models—none of which are universally valid—and in getting better at fitting these models to real-world biological phenomena.

Once we stop thinking of science as the search for universal laws and instead think of it as the construction of models with limited domains of applicability, the contrast with history looks different. Cliometric historians specify models explicitly and generate counterfactuals by varying parameters within those models—stuff like rainfall, or prices, or population. But we should also understand humanistic historians as constructing models, albeit qualitative, implicit ones, which are usually relatively limited in their scope. Our humanistic World War Two historian can be understood as constructing a kind of model of Truman’s psychology, which will carry implications concerning how he might have decided differently had the factors he was concerned with been different. The difference is not that one traffics in counterfactuals and the other does not. It is that one works with formal, parameterized models, while the other works with informal models whose counterfactual implications are less explicitly articulated.

None of this is meant as a criticism of the humanistic approach. The question of what motivated Truman is a perfectly good question, and it is pretty clearly not the sort of question that lends itself to quantification. We can’t capture the space of possibilities of interest—concerning the ways in which the factors affecting Truman’s decision could have varied—in terms of the values of some set of mathematical parameters. My point is not that such work should be replaced by regression analysis, but that even here the work of distinguishing between more and less plausible historical narratives inevitably involves judgments about what would have made a difference.

I think this way of understanding the contrast between humanistic and social scientific approaches to history can also help us make sense of the typical skeptical take of professional historians towards what I’ll call “full-blown” counterfactual history, without foreswearing counterfactuals entirely. A classic of the genre is the collection of essays, “If It Had Happened Otherwise.” It contains a piece by Winston Churchill himself, speculating about how World War One might have played out differently—or as it turns out, not at all—if Robert E. Lee had won the battle of Gettysburg. While I enjoy this sort of writing as fiction, I sympathize with historians who think it’s very different from what they aim to do. But the problem with such exercises is not that they involve counterfactuals. Rather, it’s that the counterfactuals at issue involve such distant possibilities, in which so many background conditions are simultaneously altered, that the writer of full-blown counterfactual history lacks the constraints that keep traditional historians grounded. Instead of asking how a particular decision might have changed had its immediate circumstances been different, the full blown counterfactual historian needs to imagine a major change and then project all of world events forward fifty years in which they take a very different trajectory—who can guess where things would have ended up?—to ask how World War One might have been different had the Confederacy won the American civil war. Carr was right that some “might-have-beens” are parlour games. He was wrong to think that counterfactual reasoning as such is alien to history.

Ultimately, I want to paint what I expect will be a familiar picture of a certain kind of identity or self-conception—that of a humanist—and to explain why it’s not mine, and why I think it can be a dangerous one to adopt. While there’s nothing per se wrong with being primarily interested in the humanities, it’s a near occasion of (epistemic) sin.

Some people are more excited about literature, foreign languages, and the arts, while others are more excited about math and science.2 So far, no problem. But to a man with a hammer, everything looks like a nail. If your interests and skills are primarily humanistic, it can be tempting to see more questions as tractable via humanistic methods than really are. The characteristic temptation of the humanist is bringing exclusively humanistic tools to bear on questions that demand empirical evidence.

Many of the questions treated in great books fall within the domain of disciplines that, in the contemporary university, are not grouped with the humanities, and properly so. When Socrates and Plato ask whether and how it’s possible to knowingly do wrong—to choose A, even when you know you ought to choose B instead—they are asking questions we’d now classify as part of (moral) psychology. Likewise when Seneca writes about the nature of emotions like anger, and the extent to which they are under voluntary control. When Aristotle or Aquinas write about the function of money, and institutions like credit and interest, they’re doing what we’d now classify as economics.

I’ll give some examples later on where I think the humanistically inclined put too much weight on the authority of great books—rather than on evidence from contemporary empirical social science—in thinking about these sorts of questions. But first, I want to emphasize that worries in the ballpark of the ones I’m discussing are not new.

Subscribe now

Two Cultures Redux

One of my favorite readings from college was CP Snow’s “The Two Cultures,” a lecture he delivered in 1959. He described and lamented a divide between a literary/humanistic culture, and a scientific culture. As both a physics PhD and a novelist,3 he had feet in both cultures, and saw both scientists ignorant of literature as well as humanists ignorant of science as missing out on some of the best that humanity has produced. After the more familiar criticism of philistine scientists, he turns to scolding the humanists:

But what about the other side? They are impoverished too—perhaps more seriously, because they are vainer about it. They still like to pretend that the traditional culture is the whole of 'culture', as though the natural order didn't exist. As though the exploration of the natural order was of no interest either in its own value or its consequences. As though the scientific edifice of the physical world was not, in its intellectual depth, complexity and articulation, the most beautiful and wonderful collective work of the mind of man. Yet most non-scientists have no conception of that edifice at all. Even if they want to have it, they can't. It is rather as though, over an immense range of intellectual experience, a whole group was tone-deaf. Except that this tone-deafness doesn't come by nature, but by training, or rather the absence of training

As with the tone-deaf, they don't know what they miss. They give a pitying chuckle at the news of scientists who have never read a major work of English literature. They dismiss them as ignorant specialists. Yet their own ignorance and their own specialisation is just as startling. A good many times I have been present at gatherings of people who, by the standards of the traditional culture, are thought highly educated and who have with considerable gusto been expressing their incredulity at the illiteracy of scientists. Once or twice I have been provoked and have asked the company how many of them could describe the Second Law of Thermodynamics. The response was cold: it was also negative. Yet I was asking something which is about the scientific equivalent of: Have you read a work of Shakespeare's?

While Snow’s concerns were a bit different from mine—in particular, social science isn’t really on his radar—I do want to take from him the idea that there are pitfalls to adopting a self-conception that functions both as an excuse for ignorance and for devaluing that which one is ignorant of.

Ultimately, where I want to end up is with the idea that the boundaries between humanistic and social scientific treatments of big questions about humanity are porous, and that neither can legitimately ignore the other. So unlike Snow, I’m not trying to point to a sharp divide where people on one side go wrong by ignoring the valuable work on the other, but rather to a mushy divide that we make a mistake by treating as sharp. So it will help to have a different failure mode to use as an analogy—a case of people wrongly treating a disciplinary distinction as much sharper than it is—rather than the kinds of mistakes Snow skewers.

Noma? No Thanks

Stephen Jay Gould’s idea of science and religion as “non-overlapping magisteria” (NOMA) is a particularly clear example of the kind of mistake I want to discuss. Stephen Jay Gould’s main intellectual contribution in his home field of evolutionary biology was the theory of punctuated equilibrium—an attack on the Darwinian idea that evolutionary change is constant and gradual. Philosophers are perhaps most likely to have encountered his work with Richard Lewontin, critiquing adaptationism in evolutionary biology. But by the 1990s, much of his work was more public facing, and focused on cultural and political questions, and his idea of NOMA should be understood in that context.

Back then, there were various political battles in the United States over whether and how evolution should be taught in public schools, and how any potential conflicts with religion should be addressed. There were (and are) plenty of people in the United States who professed to believe that humans were created by God in their present form within the last 10,000 years, and who didn’t like the idea of high school biology teaching otherwise. NOMA was a kind of peace offering:

The text of Humani Generis [a papal encyclical] focuses on the magisterium (or teaching authority) of the Church—a word derived not from any concept of majesty or awe but from the different notion of teaching, for magister is Latin for "teacher." We may, I think, adopt this word and concept to express the central point of this essay and the principled resolution of supposed "conflict" or "warfare" between science and religion. No such conflict should exist because each subject has a legitimate magisterium, or domain of teaching authority—and these magisteria do not overlap (the principle that I would like to designate as NOMA, or "nonoverlapping magisteria").

The net of science covers the empirical universe: what is it made of (fact) and why does it work this way (theory). The net of religion extends over questions of moral meaning and value. These two magisteria do not overlap, nor do they encompass all inquiry (consider, for starters, the magisterium of art and the meaning of beauty). To cite the arch cliches, we get the age of rocks, and religion retains the rock of ages; we study how the heavens go, and they determine how to go to heaven.

To my ears, it’s hard not to hear Gould’s principle as an expression of wishful thinking. If only religion would stay in its lane! In fact, almost all world religions offer not only moral teachings, but also distinctive factual claims. Christians don’t just think that Jesus preached some nice sermons, but also that he was bodily resurrected. Moreover, it’s because they think he was the son of God that they take his moral teachings so seriously. In general, the moral teachings of religions are often closely bound up with their factual claims—about the soul, the afterlife, or the divine provenance of this or that text. The idea that religion gets exclusive rights to talk about value and science gets exclusive rights to talk about fact is better understood as an ambitious—presumptuous!—proposal about how to divvy up territory, with religion ceding a lot of ground, rather than a description of some already accepted norm.

By the way, it’s not just religion that would be getting a raw deal if Gould’s terms were accepted. Consider Derek Parfit’s remarks at the end of Reasons and Persons:

Belief in God, or in many gods, prevented the free development of moral reasoning. Disbelief in God, openly admitted by a majority, is a recent event, not yet completed. Because this event is so recent, Non-Religious Ethics is at a very early stage. We cannot yet predict whether, as in Mathematics, we will all reach agreement. Since we cannot know how Ethics will develop, it is not irrational to have high hopes.

“Not so fast Derek!” says Gould. “Ethics is within the magisterium of religion; ‘non-religious ethics’ is an oxymoron!” Gould’s treaty is not a happy one for secular ethicists. Oh well, perhaps they’ll agree it’s a price worth paying to save high school science from the Southern Baptists.

Of course I’m being a bit tongue in cheek. My point is that Gould’s attempt to carve up intellectual life into non-overlapping magisteria was a pretty dismal failure, however noble its motivations may have been. To be clear, none of this is to say that the “conflict” or “warfare” between science and religion which Gould lamented is inevitable. Rather, it’s just to say that any successful reconciliations won’t be along the lines of NOMA. Where scientific and religious claims apparently conflict, and how those conflicts can or cannot be resolved, are big questions that I don’t hope to even touch on in this post.4

Great Books, Lousy Evidence

I don’t know of anybody who explicitly defends the idea that the humanities and the (social) sciences represent non-overlapping magisteria a la Gould; it’s a hard idea to defend! Still, I often see humanists arguing for claims that—when stated baldly and clearly—are manifestly empirical, causal claims, but supporting them by appeal to the authority of the mighty dead. While I never see principled arguments to the effect that empirical evidence would be irrelevant to such claims, I do see puzzling absences—arguments stated with less humility than would be called for if their empirical limitations were frankly acknowledged. As I think this is a tendency that pops up all over the political spectrum, I’ll give examples from both the right and the left.

One of the best known right-wing thinkers who gets on my nerves—epistemically speaking—is Jordan Peterson. Maybe a weird example for me to pick for this post, as he’s not technically a humanist; his PhD is in psychology. But while he’s done plenty of traditional empirical work, I get the sense that, at least in the last couple of decades, that’s not where his heart is; his intellectual style is much more literary/humanistic than (social) scientific. He’s apt to make big, sweeping claims about human nature that are supported primarily by situating them in the interpretive framework of Jungian archetypes, rather than by appeal to research in which those claims are operationalized and tested.

One of the central explanatory claims Peterson makes using this Jungian framework concerns the psychological effects of uncertainty, and the role that hierarchy and tradition play in managing it. The thought is that exposure to uncertainty reliably produces fear, and that this is why human beings seek refuge in tradition, hierarchy, and stable social roles.5 This is presented as a claim about human psychology with evolutionary roots: the unknown is dangerous; we have both biologically and culturally adapted to treat it as such; myths and rituals function to domesticate it.

It’s a cool idea! And not an implausible one. There’s nothing wrong with treating writers like Jung as a rich source of hypotheses. But hypotheses are supposed to be probed with evidence. And maybe I’ve missed it—Peterson has written and said quite a lot—but I’ve never heard him tell that part of the story. What would it take to operationalize a claim like this? Under what conditions does uncertainty produce fear rather than curiosity or excitement? Doesn’t that sometimes happen too? To the extent that you do see people embracing tradition or hierarchy, how could you tell if it was best explained as an attempt to mitigate psychologically painful uncertainty?

Granted, these are hard questions. I don’t know how I’d go about testing the Jungian picture. But that’s part of the value of the (social) scientific method. When you force yourself to specify how a claim could be tested, even in the happy case where it’s true, you come to better understand its scope and its limits. After all, there are many ways in principle to reduce uncertainty—the “dark room problem” is just one illustration—so why, if at all, should we expect the particular, socially conservative responses that Peterson emphasizes, rather than others, to be the dominant or characteristic way people handle uncertainty, even granting the hypothesis that uncertainty-reduction is as fundamental a drive as he supposes?6

Along the same lines, I enthusiastically recommend a recent essay by that raises a similar criticism against a different target. He’s discussing a politically conservative strand of Straussianism, an approach to interpretation named for the 20th century German-American philosopher Leo Strauss. The basic idea is that many great thinkers, writing under conditions of persecution or censorship, concealed their true views beneath the surface of their texts. Careful readers are therefore encouraged to attend to puzzles, contradictions, and apparent missteps as clues to hidden, “esoteric” teachings. Economist, blogger, and polymath extraordinaire Tyler Cowen frequently offers Straussian interpretations of various cultural products: e.g., the hidden message of Star Wars is that the Jedi are not to be trusted.

To put my cards on the table, I have nothing against the idea that a Straussian approach is sometimes the best way to understand an author’s message. And even when it’s not—come on, George Lucas is not that subtle—it can still be fun.

So what’s the problem? Traldi argues that Straussians can become so focused on questions of interpretation—on their methods for reading between the lines to identify a great author’s secret teachings—that they fail to notice the substantial gap between establishing that someone held some view, and establishing that the view is true. Here’s how he describes the Straussian approach to reading Great texts:

under the assumption that the writer is a capital-g Great, capital-g Genius, the philosophy student is to apply this method to elicit that writer’s true beliefs, then adopt them.

That is, all of the Straussian’s argumentative energy is expended in showing that this or that Genius believed something, and little if any work is done showing that the Genius was right to believe what they did (which would generally require very different, usually less humanistic methods). I’m not familiar with enough Straussians to know how representative this picture is, but my hunch is that Traldi has identified an important failure mode.

On the left, I see analogous slippage in many critiques of capitalism. Elizabeth Bruenig’s case for socialism is a paradigm example. She draws on Great books—Plato, Aristotle, St. John Chrysostom—in characterizing a healthy, well-ordered soul, in which reason is properly in control of the passions. You might think I’d already object here, but I’m happy to follow her this far. While contemporary psychologists tend to use different vocabulary, I think there’s plenty of continuity between ancient and contemporary discussions of the soul/mind that can support the ideals Bruenig is identifying.7 Where my hackles are raised is when she moves from there to discussion of the deleterious psychological effects of living under capitalism. She makes explicit causal claims, but supports them by quotation rather than reference to empirical evidence:

Wherever the bourgeoisie has got the upperhand, Marx and Engels observed, “it has drowned the most heavenly ecstasies of religious fervor, of chivalrous enthusiasm, of philistine sentimentalism, in the icy water of egotistical calculation.” Capitalism demanded rationalization: the weighing and measuring and ordering and specifying of everything, not least because capitalism simply isn’t possible without rigorous and extensive bookkeeping.8 The objective of capitalism is the generation of profit, and to maximize that end it helps to be thoroughly, coldly calculating about things. It wasn’t as though no one in the Western world had ever thought or behaved this way before: “Rather,” the late Christian socialist scholar John Hughes wrote, “it is that this practice came to be differently evaluated, and this novel interpretation of capitalism as moral came to a new dominance.”

Any time you want to make claims about the causal effects of living under capitalism, you face some serious challenges; good evidence for causal claims requires controls, and it’s hard to find collections of societies that are otherwise similar but vary in how capitalistic they are. At the very least, absent this sort of evidence, one might hope for some humility about the strength of the causal claims you’d be in a position to make.

But as it turns out, there is an empirical literature in the ballpark, and it’s not all that favorable to the picture Bruenig paints. The quote below is from the conclusion of a survey article on the relationship between markets and morals:

Many people – both scholars and non-scholars alike – think markets transform us into the worst versions of our moral selves. While the evidence does not uniformly nor definitively tip the scale to a side of the debate, as our survey of the literature in Section 4 hinted, the bulk of the evidence suggests that they may not be correct. Most regression-based studies and experimental studies on prosocial behavior in economics suggests that markets encourage and promote prosocial behavior, attitudes and orientations. There was, indeed, some empirical support for markets being associated with socially harmful behaviors, attitudes and orientations. However, the weight of the quantitative studies seem to support the position that the market generally encourages and promotes prosocial behavior, attitudes and orientations.9

While the piece itself is from after Bruenig’s—and so she can’t be blamed for failure to find it—it surveys a large literature, which includes work from long before she wrote. I don’t bring it up to suggest the case is closed in favor of the salutary psychological effects of markets and/or capitalism. Rather, my point is just that this is the sort of work that one ought to be inclined to look for, if one is going to make the sorts of claims she does. While I can’t be sure that it’s an excessively humanistic orientation that explains Bruenig’s failure to do so—that’s a causal claim, and I don’t want to be hoist by my own petard—that is my suspicion. And suspicion is enough, I think, to be on guard against that possibility in myself. I don’t want to identify as a humanist, because I don’t want to be the kind of person for whom it feels natural to let quotation substitute for evidence when empirical questions are at stake.

In a future post, I plan to discuss how concerns like the ones I’ve raised here bear on questions of disciplinary gatekeeping—especially in light of recent criticisms of philosophers for being too quick to ask questions like “how is this paper philosophy?” As a preview: while I admit the question can be used badly, I think it can also reflect a healthy caution about the dangers of what others have called “epistemic trespassing.” Stay tuned!

This post is about none of those questions. My topic is humbler, but still (personally) urgent. How should university faculty teach—and especially assess—students now that they have access to large language models?

In fact, it’s even narrower than that. Because I’m sure the answer varies from discipline to discipline depending on what assessments were already being used, and I’m sure the answer may change quickly, as the technology is changing quickly. So really, my topic is: how should philosophy professors structure their assignments in 2025-26? A small question, perhaps, but one I’m de facto required to grapple with, since I’m teaching philosophy classes this year.

Two Poles

I find it fruitful to distinguish two polar opposite approaches, as I’ve seen people pursuing both of them, and my preferred approach falls somewhere in the middle. At one end of the spectrum, we can “keep on keeping on.” Basically, change nothing, and ask students not to use AI. At the other end of the spectrum, we can completely AI-proof our assignments. Perhaps grades should be entirely based on some combination of in-class tests, oral exams, and class participation. I’m less interested in arguing against alternatives than I am in describing what I’m doing and how it’s going, but I do want to say a bit about why I’m not a huge fan of either of those approaches.1

First, why not “keep on keeping on”? Well, partly it’s that I don’t like having rules that I know I can’t enforce. I used to have a practice of asking for brief weekly reading responses, that would serve both as a nudge to get students to do the reading, as well as a way for me to think about what topics I might want to cover in lecture. Now, I think it would be very easy to upload pdfs of the readings to an LLM, ask for responses, and get very good results. I typically didn’t ask for more than a page, and I just asked for questions—not for sustained argumentative engagement with the texts. I’m sure LLMs could ace this assignment, and as much as I think my students are great, I wouldn’t trust them to resist the temptation to cheat. Moreover, I wouldn’t trust myself to spot LLM-generated responses. I’d have suspicions, sure, but I wouldn’t be confident enough to penalize a student for using AI, and I don’t trust supposed detection tools enough to rely on them for enforcement.

Why not completely AI-proof the class? While I have moved partially in that direction—more below—I’m not willing to completely give up on the take-home essay. It’s a cliche, but it’s true: learning how to write is learning how to think. Writing a paper isn’t just the process of translating thoughts that already exist in your head onto the page; rather, it’s only when you have to commit your thoughts to writing that you are forced to clarify them to yourself. And one of the the skills we want to teach people in philosophy classes is to think thoughts that require a lot of time and pages to develop. State a thesis, clearly explain it to the reader, raise the best objections you can think of, evaluate those objections, and perhaps iterate the process a few times; maybe you’ll decide your thesis needs to be revised in light of the objections, and the process begins anew. If the only essays we have people write are for in-class exams, then we’re substantially lowering the bar for how much of that sort of thinking we’re asking students to do. So if it’s at all possible to hang on to the take-home essay, I want to. As you’ll see, I think it is, at least for now.

What I’m Doing

No more reading responses. Instead, I’m assigning in-class reading quizzes. They’re short, one-question quizzes that are designed to be easy if you did the reading, and hard otherwise. An in-class, proctored final exam. The idea is to shift the weight of the grade towards AI-proof assignments. But I am still assigning two take-home essays. I’m telling the students they’re welcome to use AI in writing the essays, but I’m trying to make them hard to do a good job on even with the assistance of AI. How is that possible? I’m making them a bit longer than I’ve made them before, and I’m not assigning essay prompts. No prompts? That’s weird! But my fear is that prompts would make the task too easy for LLMs. This is a class on contemporary epistemology, so most of the papers we’re reading are relatively recent, and the internet is not awash with discussions of them. What I’m telling students is, roughly, “find something interesting to say about the readings thus far”, and pointing them to general guidelines on writing philosophy papers, along with a few model papers. How easy a task is this for contemporary LLMs?

As far as I can tell, it’s still not easy. In my judgment, as of late 2025, LLM-generated undergraduate philosophy essays still have “tells” that not only make it possible to identify them as such, but which make them bad, qua undergraduate philosophy essays. So I can mark them down not because I strongly suspect but can’t prove that they’re LLM-generated, but for the more traditional reason that they’re just not good work. I’ll explain below why I think this is the case, but first, I’ll propose a test. After I wrote this post, I asked ChatGPT5.1Pro (gotta spend that research $ on something) to stick a paragraph in, somewhere in the essay. So most of the post is me-generated, but there’s a stray paragraph that’s not. Guess in the comments which one it is. I hope you’ll be largely correct, and that the fact that you’re correct will be some evidence for my claim here.2

In a class of about 30 students, I got two essays that I suspected were almost entirely AI-generated. One student of mine—a very good student—in addition to giving me his actual paper, generated a paper using AI as an experiment; he uploaded all the readings for the class, all the handouts from my lectures, and all the instructions I’d given. The flaws of this paper were very similar to the flaws of the papers I received that I suspected (but didn’t know for sure) were AI-generated. At least for now, asking contemporary LLMs to write a philosophy paper, at least without a lot of further prompting, does not generate the kind of sustained defense of a thesis that we ask students to produce. Rather, it produces a set of loosely connected meditations on a theme. The prose flows, but the ideas are superficial and the writing is glib.

At the same time, I’ve noticed that AI alters not just how students write, but how they approach the very idea of an assignment. When a model can produce something that looks like a paper in a few minutes, the temptation is to treat a prompt as a request for a product rather than an invitation to think. That shift subtly changes the background norms: instead of seeing assignments as occasions for intellectual exploration, some students begin to treat them as exercises in efficient summarization. The result is a style of work that is polished on the surface but oddly non-committal, as though the prose wants to gesture at an argument without fully taking ownership of it.

What’s the paradigmatic “tell” of AI-generated prose? “It’s not X, it’s Y.” On its face, you might think a tic of constantly drawing distinctions would be great for writing philosophy papers. Philosophers love distinctions! William James once referred to the great scholastic adage, “whenever you meet a contradiction, you must make a distinction.” But drawing a distinction is like writing a check—you incur a debt, and the debt must eventually be paid. That is, you have to explain the distinction, and then go on to use it in the rest of the essay in a way that’s consistent with the explanation you’ve given. And that’s where contemporary LLMs absolutely fail.

One of the most unobvious and helpful pieces of writing instruction that philosophy teachers standardly give in introductory classes is the following: don’t vary your vocabulary just for the sake of variety. In other contexts you might be told not to use the same word—especially if it’s an unusual one—two sentences in a row. In philosophy, even if it sounds clunky, it’s much worse to avoid that clunkiness by reaching for the thesaurus. Don’t write about “justice” in one line, “morality”, the next, and “virtue” in the third, if you’re just using them all to mean the same thing. Rather, if you use three different but related words, I’m going to assume you mean three subtly different things. And if you use three different but related words without explaining how you’re using them differently, I’m going to pull my hair out.

Here’s an example of how LLM-philosophy makes me pull my hair out. The experimental LLM-generated paper my student gave me included the following sentence: “the resulting picture is layered rather than hierarchical.” That sounds interesting, in isolation, but it’s far from obvious what distinction is being drawn; in many contexts, “layered” and “hierarchical” mean roughly the same thing. The obvious example to reach for—given the topic of the present post—is hierarchical neural networks, which are so called because they are composed of artificial neurons arranged into layers. Does this mean that “layered” and “hierarchical” can’t be fruitfully distinguished? No, but it does mean that if you’re distinguishing them, you have some explaining to do. But in that paper, the words “layered” and “hierarchical” each appeared just the once, in that sentence. The distinction was drawn, and quickly discarded. The sentences that followed the one I quoted continued to draw other distinctions, but they didn’t at all illuminate why the picture being described was layered but not hierarchical.

More generally, LLM-generated philosophy is full of turns of phrase that combine rhetorical crispness with semantic murkiness. Drawing distinctions without differences is just one particularly obvious way of generating the appearance of subtle insight without the substance.

Why are LLMs Still Bad at Philosophy?

Supposing I’m right that it’s still pretty hard to get a good undergraduate philosophy paper out of an LLM—at least in one shot, without a lot of back and forth—why is this? There’s a lazy explanation that I absolutely reject: LLMs are mere “stochastic parrots” that can’t actually engage in the sort of complex thought necessary for producing good philosophy. Whatever you say about whether LLMs think, this sort of explanation fails because it doesn’t explain the pattern of LLM failures and successes. In some domains, LLMs are knocking it out of the park on tasks that—for humans—require thought no less complex than that involved in writing undergraduate philosophy papers. From what I’m hear, contemporary LLMs have already rendered take home problem sets in economics and computer science largely obsolete. And other closely related AI tools—AlphaEvolve—are helping push the frontiers of research in mathematics. Even if you think LLMs are stochastic parrots, you haven’t thereby explained why stochastic parrots don’t produce highly competent undergraduate philosophy, given everything else they clearly can do.

So why do I think they still fall down flat when it comes to philosophy? My speculative answer is that it’s because of the training data. When it comes to tasks that LLMs do well on, the internet is full of samples of excellent performances of those tasks. I’m sure the existence of sites like Stack Overflow—a longstanding resource for programmers to ask and answer questions—played a big role in supplying the training data that explains why current LLMs are so good at coding. Any subject with lots of questions and solutions online—this probably also goes for economics—will be such that LLMs have trained on exemplary answers to the sorts of questions professors want to pose to their students.

But I don’t know of any large—or even medium sized—corpus of excellent undergraduate philosophy papers that LLMs might train on. And if you’re just scraping the internet as a whole, there’s plenty of dreck. R/philosophy is pretty big, but I wouldn’t particularly expect an LLM trained on it to do a good job of writing philosophy papers. While I’m sure there’s good philosophy out there, it’s mixed in well enough with the bad that I think an LLM would have a hard time identifying the patterns we want students to emulate, without also picking on the ones we don’t. Moreover, my perhaps vain hope is that things may stay this way for awhile. While there is obviously lots of money to be had in automating coding, there’s as yet no profit to be had in churning out competent undergraduate philosophy papers at scale. So even if I can imagine strategies for generating the kind of training data you’d need for LLMs to get good at writing philosophy,3 I suspect the economic incentive to pursue those strategies doesn’t exist.

Of course, maybe in very little time this post will seem quaint. The last few years have seen LLMs blowing through one benchmark after another, and I would be disappointed but not shocked if, this time next year, I find myself deciding that full AI-proofing is the only way to go. What do you all think? If you’re an undergraduate, how are you using AI tools? If you’re a professor—not just in philosophy—how, if at all, have you changed your assignments? Which paragraph above was AI-generated? Let me know in the comments.

Update: I’ve revealed which is the AI-generated paragraph in a comment below, so guess before reading the comments.

Some idealists set out to build a new community from scratch. They saw themselves as unusually clear-headed and logical—people determined to build a society based on reason rather than on the accidents of tradition. If there was a better way to do something, they wanted to find it.

At first, the experiment went smoothly. They shared work, rotated responsibilities, and debated policy late into the night. But before long, they started to complain about English. It’s irrational, they said. Silent letters everywhere, no phonetic consistency, spelling rules that dissolve the moment you learn them. Why do “though,” “through,” and “tough” all look the same but sound nothing alike? And the idioms! Why should “kick the bucket” mean “die”? Surely a truly rational society can do better, they thought.

So when the first children in the commune were born, factions emerged around which language the next generation should speak.

• One faction rallied around constructed languages—Esperanto, or perhaps Lojban. Why cling to a broken tool when we could use one optimized for ease of learning and efficiency of communication?

• Another faction pushed for Mandarin. It was already the most widely spoken language on Earth. If children should be prepared for the future, shouldn’t they learn the tongue of global commerce and power?

• A third championed Latin. They saw it as venerable, orderly, and the foundation of much of Europe’s culture. They argued that it would provide an excellent foundation for learning other languages, should children choose to. To them, it seemed both rational and dignified to return to such a source.

Each group had understandable, albeit very different reasons for their respective positions. But precisely because nobody could win the argument, no consensus was reached. Ultimately, each family in the society made their own decisions about linguistic education, and the children grew up exposed to various different languages at home, but none of them was universal in the commune. What did they fall back on? English—the one language they all still sort of shared.

But it wasn’t the English their parents spoke. Having received little to no English education, they spoke a stripped-down, halting version. They used fewer words, the grammar was rough, their range of expression narrower. The common language survived, but thinner and weaker than before.

The commune set out to improve on English and ended up with something worse: a diminished, half-forgotten version of the very thing they were trying to replace.

Great Books

I haven’t posted in awhile, and that’s because the semester is back in swing. My parental leave is over, and I have a full complement of teaching, advising, and administrative work. Part of what makes that so time consuming is that one of the courses I’m teaching is the philosophy strand of our great books sequence (in addition to philosophy, the students taking the sequence study literature, and history & political thought). It’s a wonderful class to teach, in part because it selects for particularly enthusiastic/masochistic students. To take it is to opt in to huge amounts of reading and writing. They also have to apply for the program, though my sneaking suspicion is that we keep the application process in place not so much as a filter, but because without it, there would be less interest in the program. Yale students are a bit like Groucho Marx. They don’t want to belong to any clubs that would have them as members without some sort of selection process.

But since I have to do the readings too, my thoughts turn to the old question of Canon. Why do we still read Plato, Aristotle, and Kant? In this post I want to address two very different sorts of skeptical characters. The first is the technical, scientific-minded skeptic. Maybe a bit like the characters in our parable. Here’s his speech:

If philosophy is really about truth-seeking, as you often say, why is the pedagogical approach so different from the approach in those disciplines that are best at generating reliable knowledge, namely, the natural sciences? Of course Isaac Newton was a genius, but budding physicists don’t read Newton; they read contemporary textbooks that present his ideas—not all of them, we leave out the alchemy—in a maximally accessible form that provides a sound foundation for further learning.

Even if Plato has some good ideas—which I’m not sure of but I’ll grant for the sake of argument—there’s no chance we couldn’t express them more compactly and clearly than he did. The fact that you still read Plato’s own works, rather than textbooks that distill whatever intellectual value his works had, tells me what you do is more like cultish hero-worship than genuine intellectual inquiry.

The second critic is happy with reading historical works, but objects that our focus is too narrow:

If you’re really interested in reading the best philosophy, why do you read so many European men? There were extremely rich philosophical traditions in the non-Western world that your great books classes completely ignore. And even within the Western world, there are lots of neglected female authors who don’t show up on your syllabi. The chance that the best philosophical works in history would be so culturally and demographically skewed is basically nil—more likely, you’re just reading books that have made their way into the Canon through accidents of history, helped along by racism, imperialism, and sexism, rather than because they’re uniquely deserving of their exalted status.

Some people would want to give full-throated replies to these objections. While I’m not Catholic, I have friends who are, and I can imagine a speech like the following:

The reason we should read roughly the books we already do in great books courses is that Aquinas basically got everything right. But to understand Aquinas, you really need to read Aristotle (who was half right, which is better than almost anyone else except Aquinas). But Aristotle won’t make much sense to you without seeing how he’s responding to Plato…

And of course it’s not just Catholics who can give speeches like this. I have a close colleague who I imagine would say something pretty similar about Spinoza. I know others who think the history of philosophy basically culminated with Kant. Plenty of contemporary philosophers will be able to give unapologetic, full-throated defenses of the Canon if they think some figures in the Canon are largely correct, and that the rest is essential for understanding them. This is a natural pairing with declinist narratives—it’s not just that some Canonical figure got things right, but that we’ve collectively intellectually regressed since then, so later statements of similar views tend to be worse than the original. Alasdair MacIntyre tells a story along these lines.

I want to acknowledge this sort of view, but it’s not mine. I feel the pull of the objections, especially the former. My intellectual instincts are what some would pejoratively call “scientistic”. I think the ancients and early moderns were missing too much scientific knowledge—and good philosophy is too closely integrated with science—to think any of them were basically right in most of their big picture views. So I want to give a kind of concessive reply—one that grants the main points the objectors are making, and still resists replacing Canonical philosophical texts with contemporary textbooks, or radically expanding the Canon.1 And as is a recurring theme on this Substack, I want to do so by drawing on some concepts from economics.

Coordination Games and Schelling Points

Schelling points are features of (some) coordination games, so to understand what Schelling points are, we first need to understand coordination games. And while they have a technical definition, I tend to think the concept is best understood by starting with examples. Consider the problem of which side of the road to drive on. In some places, cars drive on the left. In others, they drive on the right. Either convention works fine. It’s far more important that drivers coordinate—that everybody drive on the same side of the road—than how they coordinate. Language is the ultimate coordination game. A rose by any other name would smell just as sweet. That is to say, it doesn’t matter whether we refer to roses with the word “rose” or something else. What matters is that we coordinate on some shared word, so that I can use that word to get you to think about roses. With driving, I only know of two stable, workable patterns of coordination. With language, there are at least as many solutions to the coordination game of communication as there are spoken languages, and clearly there are far more; there are lots of merely hypothetical languages that no actual community speaks, but which could in principle work just fine.

Coordination isn’t always easy. Thomas Schelling—writing long before cell phones and text messages—imagined an example where traveling companions in an unfamiliar city get separated. Let’s say it’s Paris. They face a coordination game in that they want to end up in the same place to continue their trip together, and they care far more about meeting each other than they do about where they meet. How might they find each other? He thought they would likely pick some unusually salient location—the Eiffel tower, perhaps?2 The general idea is that when it’s more important that we coordinate our choices than it is what we choose, we’ll often do well to pick options that we know the people we’re trying to coordinate with are likely to be thinking of.3 Such choices—solutions to coordination games that we’ll naturally converge on when we can’t explicitly talk things out—are called “Schelling points”.

One crucial feature of Schelling points, for my purposes at least, is that they don’t need to be optimal. To stick with the example of Paris, if the place we get separated is not all that close to the Eiffel tower, an optimal solution to our coordination problem would be somewhere else; maybe halfway between us, or the nearest bistro that we can both comfortably walk to. Still, because we can’t explicitly talk things out, searching for the optimal solution to our coordination problem is unwise; we’re unlikely to both find it, and we might disagree on what it is (maybe you prioritize a short walking distance, while I’d really like a bite to eat). We’re better off sticking with the Schelling point.

Great Books

I hope the point of my title is pretty clear now. There are at least some similarities between the choice of what to read, and coordination games. I admit the analogy isn’t perfect. If I want to curl up with a fun mystery novel on vacation, I might not care if anybody else has read what I’m reading; I’m just aiming for private enjoyment. That’s not a coordination game. But for other books—especially works of philosophy—part of why we read is to share a kind of common language for discussing ideas. And to the extent that this is so, it’s more important that we read the same books—or at least, that there be substantial overlap in what we read—than it is that we read the best books.

Look around and you’ll find lots of examples illustrating the point. This post itself is one example. The story of the Tower of Babel is at least as old as Genesis. But because I can assume that you, reader, have heard the story, I can quickly and efficiently convey various associations by alluding to it. If I were designing a society from the ground up, including its mythology, would I include the Tower of Babel as one of the shared myths? I have no idea. But the question is beside the point. I’m not designing a society from the ground up, but instead I exist within a society that already has a stock of tropes, and the best I can do to communicate my thoughts is to draw on the images and touchstones I know we’re likely to share.

In an earlier post on free speech, I framed my discussion as a response to an argument from John Stuart Mill. Why? Because contemporary debates about free speech are heavily indebted to Mill, and I judged that the best way to convey my own perspective was to emphasize points of contrast between it and a broadly familiar one. Even in decidedly middlebrow publications like Slate, a few years ago you could find people drawing on a concept from Plato—the Noble Lie—to criticize Covid policy. One last example. The familiar econ 101 distinction between positive and normative analysis has its roots in Hume’s famous discussion of the is/ought gap, and it’s easy to find invocations of that distinction all over the place in contemporary discussions aimed at a broad audience. For example, a few years ago Tom Rachman wrote in the pages of the Atlantic about the difficulties facing contemporary journalists under the heading “America’s Is/Ought Problem.” If you try to explain to a middle schooler the—putative?—difference between news reporting and opinion reporting, you might have an easier time of it if you remember your Hume. Allusions and references to the Canon are all around us, and reading the Canon puts us in a better position to interpret and communicate with other people who have also done so.

Let’s return to the objections. Why not teach via textbooks? In the more technical areas of philosophy—logic, especially—there’s enough agreement about what are the core ideas students should know that instruction does largely proceed via textbook. While there are many logic textbooks—just like there are many introductory chemistry textbooks—because there’s enough agreement about what needs to go into them, they’re close enough to interchangeable. In my role as director of undergraduate studies for the philosophy major, I’m happy to accept transfer credits in logic from other universities, even if their class uses a different text.

But when it comes to the greats of the history of philosophy, my strong guess is that if we tried to distill their ideas into textbooks and teach exclusively from those, there would be enough disagreement over just what the textbooks should look like—different textbooks would place substantially different emphasis on different ideas—that we’d end up essentially having failed to coordinate. I don’t put this forward as an ironclad rule. I very much like the idea behind utilitarianism.net, and I have assigned sections from it in less historically oriented courses. But I’m also happy that our great books sequence has students reading John Stuart Mill’s Utilitarianism. I think it’s a strictly worse presentation of the utilitarian worldview than utilitarianism.net. But I’m confident that students who’ve read Mill will have a common language for discussing (aspects of) ethics with the countless other people who read Mill at some point or other, whereas I worry that if I completely switched to utilitarianism.net, the world wouldn’t switch with me, and I’d effectively be abandoning a valuable Schelling point.

What about the multicultural critique? I fully concede that there’s lots of historical contingency in which books made it into the Canon. In an alternative universe where India was the conqueror and Britain the colonized, I can easy imagine the Buddhist philosopher Nagarjuna occupying a central role in the Canon, and tweedy Occidentalist scholars making the case that the obscure Scottish philosopher David Hume actually had some really interesting ideas that are a bit like Nagarjuna’s if you squint, and there’s a strong case for reading him alongside the Madhyamakas. I’m not at all sure that, intellectually speaking, we’d be any worse off in that world. We’d have different Schelling points, but they might not be inferior. Or let’s grant for the sake of argument that women like Margaret Cavendish, Ann Conway, and Mary Astel did excellent philosophy, and the fact that their works haven’t occupied places in the Canon comparable to Hobbes, Leibniz, and Locke is largely a function of historical and ongoing sexism.

That’s only a reason to significantly expand the Canon if the aim is to read the best books. If we think of syllabus construction as a coordination game, then this is a mistake. Of the “non-Canonical” philosophers named in the previous paragraph, I bet almost none of you had heard of any of them. If you’d heard of all of them, you’re almost certainly an academic philosopher involved in or at least deeply familiar with the New Narratives project in history of philosophy, which is not many people. That’s just to say we’re very far from a world where all of these figures have entered the Canon and can serve as a source of shared reference points for public discussion. The fact that there are so many ways we could take the multicultural critique on board means that trying to do so in a robust way would in practice lead to fragmentation. Instead of a larger Canon, we’d just have less of one—fewer overlap between different great books classes, because each instructor has tried to expand the Canon in different directions. We’ll be a bit like the second generation in the rationalist commune.

Does this mean the Canon must be etched in stone, never to change for all history? Not exactly. But I do think it means the Canon should change very slowly. The great books sequence at Yale was founded in 1947. My strong guess is that back then, they read Plato, Aristotle, Aquinas, Descartes, Hume, and Kant, and I can attest that we still read them today. But I also imagine that there have been some changes on the margins. From year to year, we’ll add or subtract a text—we won’t subtract one of those texts, but in a year where the students howl too much about Hegel, we might try out a different German Idealist—Fichte?—the next year. If enough people across the country or world decide to add the same text to similar courses year after year, I see no reason why such a text couldn’t enter the Canon as a new junior member. But I wouldn’t expect it to happen often.

When coordination games are played with a large number of players who differ in their views about where the optimal coordination points are, and who are too diffuse to engage in effective collective deliberation, we abandon Schelling points at our peril. We’re more likely to fail to achieve the value of coordination than we are to move to a better equilibrium.

One of the fundamental concepts in economics is that of an “externality”. An externality is a cost or benefit associated with a transaction that affects people who aren’t party to the transaction, and which is not reflected in prices. If my railroad throws sparks onto nearby fields that sometimes cause fires that burn crops, that’s an externality. Usually, discussion of externalities as a practical problem concerns how to get parties to “internalize” the costs of their activities. Perhaps the railroad should be subjected to a Pigouvian tax. Or maybe if the railroad owner can bargain with the owner of the nearby fields, they’ll reach an efficient solution.2

Spending resources and effort to move up a ranking is an unusually pure example of an externality. We might call it an “intrinsic” externality. It’s not intrinsic to operating a railroad that it must cause costly fires. We can imagine all sorts of ways the value associated with operating a railroad could be achieved without the cost of starting fires. Maybe some technological innovation could prevent the railroad from throwing off sparks. Or maybe it could be routed in such a way that it doesn’t pass near any tinderboxes. Typical examples of externalities are “extrinsic” externalities. But there’s no way someone can become #1 in a ranking without knocking someone else out of the top spot. It’s intrinsic to Mast getting what he wants—having the bigger yacht—that Crewman fails to get what he wants.

Subscribe now

If the only things we cared about were our positions on rankings like these—in the economists’ lingo, if all goods were positional goods—then we’d be left with an extremely bleak picture of the human condition. It would be impossible for us to collectively improve our lot.3 Luckily that’s not the case. While we do care about our positions in rankings, that’s not all we care about. In the example as I imagined it, up until 300 feet, there’s room for both Mast and Crewman to get more of what they want, yacht-wise. While some economic activity looks like trying to build the bigger yacht, plenty does not. I have no idea which of my acquaintances, if any, suffer from hemorrhoids; people tend to be private about that sort of thing. If I do have any such acquaintances, I won’t be worse off in any way if they pay the makers of Preparation H for rectal relief.

This suggests a kind of public policy objective that I don’t think I’ve seen much discussed. We should be on the lookout for ways to shift people’s economic activity away from pursuing zero-sum status competition, and towards everything else; fewer yachts, more hemorrhoid cream. What might that look like in practice? I’m not really sure. In our example, through the power of stipulation we could determine that a ban or prohibitive tax on >300 foot yachts would be efficient. In the real world, it’s much harder to say, as it’s much harder to identify economic activity whose sole effect is to improve the relative position of the actor in some ranking. Moreover, I’d be nervous about empowering anyone—even democratic majorities—to decide which apparently self-regarding activities really amount to negative sum status competition. Public choice and all that. So I don’t think there are obvious public policy implications of the above, though I do think it’s a rich vein to mine.4

Perhaps the best example I can think of would be certain kinds of cosmetic surgery. Consider height. Within the normal range, being taller has only relative benefits. Famously, taller people earn more, and do better on the dating market. When someone who is short has costly limb-lengthening surgery,5 they use medical materials and the time and expertise of medical professionals for the purpose of moving up in a ranking. They’re only better off to the extent that everybody else they’re now taller than is now (a tiny bit) worse off. I don’t think it’s absurd to think that it would be good public policy to discourage purely cosmetic limb lengthening surgery, perhaps through taxation.

The Hedonic Treadmill

But I don’t really want to talk about public policy in this post. Rather, I wanted to use the somewhat analytically cleaner example of zero-sum status competition as a kind of jumping off point to talk about the related, but distinct, set of issues known as the “hedonic treadmill”, and how it can be addressed through private policy. The metaphor of the hedonic treadmill involves expending effort to get happier, while in fact staying right where you are. Just like Mast and Crewman, who collectively spend 1100-yacht-feet-worth of resources just to end up where they started, if you’re on the hedonic treadmill, your efforts to make yourself happier are ultimately fruitless.

The hedonic treadmill might involve status competition. I’m an academic philosopher, and we have lots of rankings. We rank journals, we rank departments, and like any other academics of course we can count citations. Here’s a fictionalized, but I think pretty realistic, picture of how an academic philosopher—call her Sophie—might find herself on a status-inflected version of the hedonic treadmill. In college, she decides to apply to graduate school. She reads and posts on message boards like Gradcafe, which is full of people sharing stories of getting rejected from lots of the programs they apply to. Relative to this population, people who get into top graduate programs seem extremely fortunate. They must be so happy! Lo and behold, Sophie is lucky enough to be accepted into her top choice graduate program. But now that she’s surrounded by other people who were also accepted to that program, it no longer feels unusually fortunate to have done so. Moreover, her peers don’t seem particularly content; they’re all stressing out about the academic job market. Plenty of people, even at top programs, don’t land a faculty position. What happens to such people? Does anybody know? There are rumors that some of them find work outside of academia(!), and even enjoy it(?!?!?), but that’s hard to imagine. No, the ones who get faculty jobs are the lucky ones, and Sophie is sure that if she manages to enter their ranks, all will be well.

You see where this is heading. Sophie gets an academic job, but her standards keep shifting. You know you can look up salaries at public universities? Do you see what the philosophers are paid at Rutgers? (It’s a lot.) Even the very few, extremely influential, can unfavorably compare themselves to the mighty dead, or can worry that they’ll be a failure if their work won’t still be read in 200 years.

Seems like a bad state to be in! And sadly, it’s not just limited to academic philosophers. Of course, this is a very old observation. Entire philosophical traditions have been built around the idea that the satisfaction that comes from getting what you want is fleeting. Buddhism takes it as a central insight; Stoics counseled detachment from the things that slip away; I’m given to understand it’s a major theme in Schopenhauer’s work too.

From an evolutionary point of view, our tendency to keep shifting the goalposts so that they’re always out of reach may be a feature rather than a bug. Being content is maladaptive. If things are going well—if you have food, shelter, resources—that’s not a reason to relax; it’s a reason to push harder, to have more children, or to better position the ones you already have to reproduce in turn. A creature wired to always want a bit more, never quite satisfied, is a creature more likely to flood the next generation with copies of its genes.

But I’m an optimist by disposition; I don’t think our lot is quite as hopeless as the above might suggest. As I see it, there are two broad strategies—distinct, but compatible—for mitigating and/or avoiding the hedonic treadmill.

First, we can lean in to the human tendency to judge how well things our lives are going comparatively, but manipulate our bases for comparison so that we’re doing pretty well. (This strategy will further subdivide into two variants.) Second, we can try to find satisfaction in activities and projects whose value is non-comparative.

Cheating the Thief of Joy

How can we manipulate our bases for comparison? One strategy is nicely illustrated by a scene from the 2020 Netflix adaptation of Dracula. It uses the old Rip van Winkle trope of letting a character who’s been asleep for a long time see the present through the eyes of the past. Here’s how Dracula interacts with a contemporary, lower-middle-class English woman, Kathleen:

Dracula: Kathleen, isn’t it? What’s wrong with your servants, Kathleen? Is it their day off? I’m assuming you have staff. You’re clearly very wealthy.

Kathleen: …wealthy?

Dracula: Well look at all this. [He points at the telly] That thing? These things? [He’s walking round the kitchen area, yanking open doors] All this food?? Food in boxes with pictures of food on the front - there’s nothing like labouring the point, is there? That machine outside—Bob calls it a car—is it yours?

Kathleen: [nods]

Dracula: This treasure trove is your house?

Kathleen: It’s a dump.

Dracula: But look at it all, it’s amazing. It’s so warm. There are only two draughts. The furniture is new. That’s the most extraordinary clock I’ve ever seen. Kathleen, I’ve been a nobleman for four hundred years. I have lived in castles and palaces, among the richest people of any age. I have never stood anywhere, in greater luxury, than surrounds me now. This is a chamber of marvels. There isn’t a King or Queen or Emperor I have ever known - or eaten - who would step into this room and agree to ever leave it again. I knew the future would bring wonders. But I didn’t know it would make them ordinary.6

One more example from fiction. One of my favorite novels is The Good Earth, which tells the story of an early 20th century Chinese peasant, Wang Lung. At the beginning of the book, he’s desperately poor. He and his family are constantly on the brink of starvation, one child grows up as a “poor fool” due to malnutrition-induced mental disability, and his wife kills another at birth when she knows there will not be enough food to keep the infant alive. By the end of the book, through a combination of hard work, luck, and theft, he’s rich. You’d think he’d be grateful! But now he has a new set of problems. For instance, his wife is old and ugly, with broad, coarse features; he wants a young concubine like his new friends have. And he is willing to cause some considerable tension in his household to get one. While the portrayal of Wang Lung is generally sympathetic, by the end of the book you sort of want to scream at him to be grateful for how incredibly lucky he is. And yet, almost anybody reading the book today—certainly myself included—has a life that’s more comfortable than Wung Lung’s even at his richest.

As much as possible, I want to view my life through Dracula’s perspective, and to avoid Wang Lung’s mistakes. I think just having these archetypes in mind and frequently reminding myself of them helps. As I was writing this I went to the refrigerator to eat some strawberries. Most emperors in most of history couldn’t have cold strawberries on a whim. I was grateful.

So my #1 tactic for hacking our tendency to judge our lives comparatively is to try to constantly remind ourselves of how big the world is, both spatially and temporally, so that we get into the habit of comparing our lives to those of the vast numbers of people far less fortunate than us—if you’re reading a post on Substack, you are included in the “us”—rather than our similarly fortunate neighbors with bigger houses, or newer cars, or more citations.

But suppose you just can’t shake the habit of judging how well your life is going by comparing yourself to members of your (comparably fortunate) social circle. All is not lost! While status competition may seem to be inherently zero sum, it doesn’t have to be. I was recently reminded, reading a post by Brad Skow, that Robert Nozick pointed out that if there are multiple dimensions of status that people care about, and if people vary in which dimensions of status they care more about, then status competition can be positive sum. People can compete to be higher on the rankings they care more about—or can end up caring more about those rankings where they’re already towards the top—and can psychically devalue the rankings where they’re towards the bottom. Suppose there’s wealth and artistic talent, and the artists tend to be starving, while the wealthy tend to be philistines. Then everyone can be happy; the wealthy can congratulate themselves on their material success, and the artists can congratulate themselves on their creativity.

The best is when people specialize in multiple, uncorrelated—or even better, negatively correlated—dimensions of status, and shift which ones they emphasize based on who the salient comparison group is. Here’s how I do it. I’m a professor at a fancy university. I’ve progressed a ways along the treadmill Sophie was on. But if I put all my self-worth eggs in that basket, there’s no shortage of people I can unfavorably compare myself to. If I go to a philosophy conference, or even just a faculty meeting, I’ll have no trouble finding people who have published more, made more of an impact on their respective fields, etc. “But”, I can tell myself, “I could beat them up.”

Don’t laugh! I’ve been training in Brazillian Jiu Jitsu for quite awhile now, and regularly participate in small, Connecticut tournaments at the black belt level, where I win some and I lose some. The level of competition is not particularly high. When I occasionally find myself matched against elite grapplers slumming it in the local competition circuit, the vast gulf in skill is palpable. But fortunately, none of those people are professors!7 If there were an academic philosophy/Brazillian jiu jitsu analog of chessboxing—if you’ve never heard of it, you should follow the link right now—I might well be the best in the world.

You may think it’s childish for me to take any pride in the knowledge that I could beat somebody up. It probably doesn’t bother you at all, reader, that were we to fight, I’d win. And that’s sort of my point. While I rank above almost(?) all of my academic peers when it comes to who would win in unarmed combat—and I get a kick out of knowing that—it doesn’t bother them that they rank lowly on that dimension. So my advice to you—only somewhat tongue in cheek—is to cultivate some idiosyncratic specialties. If your main project in life is A, try to have some side project B of the sort that people who focus on things like A would be very unlikely to be good at stuff like B.

The ultimate idiosyncratic project, however—and this is where we transition to the final portion of the essay, in which we turn our attention to non-comparative sources of value—is parenting. There are plenty of other people who are better than me at philosophy, or (not and!) jiu jitsu. But nobody could be a better father to my children than me.8 By and large, the ups and downs of parenting have nothing to do with anybody else’s parenting. When I take my baby and toddler to the playground, and the toddler demands the baby be put in the swing next to her and be given a big push just like she gets, and then they both laugh their heads off, the joy I get is not in the least diminished if another couple of kids are having a very similar experience with their parent a few swings down. When I rewatch the adventure movies I loved as a child—Indiana Jones, The Princess Bride, Willow—with my two older children, and then they spend the next week reciting their favorite parts, I’m happy if I hear that my buddy’s kids also loved them.

I doubt parenting is unique in this respect. I suspect many of the activities calls “atelic”, as opposed to “telic”, have this character. As he puts it:

Atelic activities are different. Listening to music, spending time with your new friend, working thoughtfully and well: these activities aren’t defined by endpoints that exhaust them, but are forms of process. You can stop doing these things, and you eventually will, but you can’t complete them, leaving no more left to do.

Setiya sees midlife crises as stemming from being overly focused on the opposite, telic projects:

The problem with projects is that they place fulfillment always in the future—the goal has yet to be achieved—only to consign it instantly to the past. No wonder the present feels empty.

My minor addition to Setiya’s rich discussion is that telic projects lend themselves to comparison, and thus, the hedonic treadmill, much more than do atelic activities. If you aim to hike the Appalachian trail, then it’s natural to start wondering what other trails you might hike—for instance, there’s also the Pacific Crest Trail, and the Continental Divide Trail. Over 700 people have hiked all three…Better, perhaps, if you can just enjoy hiking.

Parenting certainly can be approached as a telic project. For instance, you can set yourself the aim of raising chess prodigies, as did László Polgár. He wanted to prove that geniuses are made, not born, so he trained all three of his daughters to play chess from a young age. By any measure, he wildly succeeded: his daughters grew up to be among the best chess players in the world. But having succeeded, just as Setiya might predict, he needed a new project:

In 1992, Polgár said that he now wanted "to break the racial barriers in the virtually all-white chess world" by adopting "a black infant from the Third World" whom he would train to become a chess prodigy. Susan [his daughter] recalled in 2005 that, about 15 years earlier, "a very nice Dutch billionaire named Joop van Oosterom" had offered to help Polgár "adopt three boys from a developing country and raise them exactly as they raised us." Polgár, according to Susan, "really wanted to do it, but my mother talked him out of it. She understood that life is not only about chess and that all the rest would fall on her lap."

I think—hope?—most parents will recoil from the idea of treating children as projects. Psychologist Allison Gopnik offers a wonderful contrast between the parent as gardener and the parent as carpenter. The carpenter parent is trying to shape their children into a particular form, like a carpenter building furniture to an exact plan. The gardener parent is merely providing conditions for their children to grow, while allowing that lots is out of their hands; gardeners have much less control over the dimensions of their crops than carpenters do over their handiwork. As may already be clear, Gopnik favors the gardener metaphor.

Parenting is just one example—an especially vivid one for me—of an activity whose value isn’t comparative. My point isn’t that everyone has to be a parent to get off the treadmill. The point is that some goods are best enjoyed for what they are, not because they rank higher than someone else’s. To stick with Setiya’s examples, listening to music and spending time with friends are not diminished by how anyone else is doing.

Summing up, I think there are three main ways to resist the treadmill. First, perspective: like Dracula, we can try to see just how extraordinary our ordinary lives are. Second, idiosyncrasy: if we must rank ourselves against our contemporaries, we can do so in specific ways where we come out on top. And third, non-comparative goods: the kinds of activities whose worth doesn’t depend on ranking at all. None of these are foolproof, and none of them let us escape the hedonic treadmill entirely. But they’re the best I’ve got to offer.

I’ll close by noting that these strategies have a cost. They breed complacency. If you get good at finding favorable ways to compare yourself to others, or satisfaction in things that don’t involve winning, you also get worse at ambition, at striving, and at being outraged when you get a raw deal.9 And I don’t think you can have the benefits without some of that cost. I think I’m both happier and less driven than I was earlier in life. It’s a genuine tradeoff! Personally, I think it’s worth it, but I don’t want to pretend the tradeoff isn’t there.

We’ll ultimately get to the idea that there is a single hair—#32,207, perhaps?—that marks the boundary between being bald and not being bald. But getting there will require going via some general questions about the relationship between science and philosophy, and what the latter has to learn from the former.

A recurring idea in much recent philosophy is that there is no clean break between philosophy and other sorts of inquiry. Maybe Quine was the first person to explicitly defend the idea of a continuity between philosophy and science. He did so on the basis of controversial arguments, most centrally the rejection of the distinction between the a priori and the a posteriori. But lots of other philosophers who don’t accept Quine’s arguments still agree that coming up with a demarcation criterion cleanly separating science from philosophy is hopeless.

Go back a few hundred years, and while you’ll have a hard time finding anybody explicitly defending Quine’s position, that’s only because they didn’t conceive of “science” as its own endeavor distinct from philosophy. Rather, they talked about “natural philosophy”, which looks in retrospect like what we’d call science today, but with plenty of what we’d call philosophy mixed in. Today we think of Galileo and Newton as scientists, and Descartes and Leibniz as philosophers. But in their own times no such distinction was recognized; all of them would have been understood as doing natural philosophy.

Subscribe now

While this position can be defended by abstract, principled arguments, I think the most powerful case is made by example. A few years ago James Madole and Katherine Paige Harden published a paper called “Building causal knowledge in behavior genetics”, which addressed the question of how we can identify genetic causes of human behavior. A major aspect of the discussion, however, turns on questions about causation in general, and they engage deeply with the recent philosophical literature on the topic (e.g., the work of David Lewis.) If you’re interested in whether and in what sense genes cause behavior, it seems to me you can neither ignore what look like metaphysical questions about the nature of causation, nor can you stay in your armchair and avoid engaging with data. Or to take another example, lots of salient questions about contemporary artificial intelligence—does it genuinely understand? Does it reason? If not yet, what would it take? What evidence that we currently lack ought to convince us?—can only be addressed at the intersection of computer science, cognitive science, and philosophy. Here’s a paper from a computer science conference citing and discussing, among other philosophical works, Michael Dummet’s book on Gottlob Frege’s philosophy of language.2

The idea that the boundaries between philosophy and science can be blurry should be relatively uncontroversial. But it can be a jumping off point for wildly different arguments. Here’s one direction it can be taken. Philosophers who think it’s regrettable that contemporary philosophy is largely an armchair endeavor can appeal to the blurriness of the distinction between philosophy and science, and the history of philosophers asking what we now think of as scientific questions, to argue that there’s no principled reason to reject the idea that philosophy ought to be much more interdisciplinary and empirical than it currently is; in particular, the idea that if philosophers get their hands dirty engaging too much with a posteriori questions they’re no longer doing philosophy is one that’s hard to square with much of what we tend to regard as great historical work.3

That is very much not the direction Timothy Williamson has taken the idea. Rather, he’s leveraged the difficulty of drawing crisp distinctions between philosophical and scientific methods as a way of defending armchair, non-empirical philosophy against the charge that it commits some methodological mortal sin. This was one of the main themes of his first book on philosophical method, The Philosophy of Philosophy, where he argued against the idea that there is some distinctively disreputable method philosophers use called “appeal to intuition”, which isn’t used in the sciences, and which we can and should do without. Instead, he contends that while it’s true that philosophers must make use of their judgment to identify suitable premises for their arguments, the use of judgment is an ineliminable part of inquiry everywhere, including in science. Arguments that attempt to cast doubt on a supposedly peculiar use of “philosophical intuition” end up collapsing into general skeptical arguments that would undermine all knowledge—and which should therefore be rejected—rather than targeted attacks on philosophical methods.

The more recent book I’m reviewing makes use of the blurry distinction between philosophy and science in a different way. Here, Williamson’s targets are not bomb-throwing critics of philosophy as a whole, but rather a motley assortment of philosophical views he thinks are the product of the methodological sin of overfitting.

Overfitting

Imagine a biologist who wants to predict a plant’s growth rate from sunlight, water, and soil nutrients. She records 100 data points and feeds them into a computer. The software can generate anything from a neat linear model with just a few parameters to a 100‑parameter monstrosity that perfectly fits every noisy measurement. The latter looks spectacular on the training data—but give it a fresh plant and it does a poor job making predictions. It has learned the random wiggles, not the underlying patterns.

Modern statistics and machine learning have a whole toolbox for avoiding that trap: train/test splits, cross‑validation, complexity penalties, Akaike and Bayesian Information Criteria—all formal ways of rewarding models that explain a lot with a little, and punishing those that “explain” data only by memorizing it.4 Williamson’s thought is that even in less formal philosophical contexts, where the “data” to be explained are judgments about things like knowledge, meaning, morality, or freedom, we don’t really explain them if we try to capture them with theories that are sufficiently flexible that they’d be able to accommodate any judgments. Just like scientists, we should prefer theories that are rigid enough to be potentially inconsistent with the evidence they’re meant to explain.5

Here’s a case study at the intersection of philosophy and economics. In those disciplines, a standard way of modeling an individual agent’s beliefs is with a set Ω of live possibilities. An agent believes some proposition A when A is true at every possibility in Ω, the agent disbelieves A when A is false at every possibility in Ω, and otherwise—when A is true at some possibilities, and false at others—the agent is uncertain. We represent finer-grained distinctions in her opinions—facts about which propositions the agent is more or less confident in, among the ones she’s uncertain of—with a probability distribution over the subsets of Ω.6 Two surprising features fall right out of this framework:

1. Closure under logical consequence. If the agent believes A, and all the A-possibilities in Ω are B-possibilities, she automatically believes B.

2. Indistinguishability of equivalent sentences. If two sentences are true in the same possibilities in Ω, the framework cannot represent the agent as believing one while failing to believe the other.

These features are sometimes called “logical omniscience”, and the fact that standard models of belief represent agents as logically omniscient is often thought to be a problem; isn’t it possible to believe P, where P entails Q, but to fail to believe Q? While that’s certainly a mistake, we should have a way of representing that sort of mistake, shouldn’t we?

It’s not so obvious. Let’s work through a simple example. Suppose a fair six‑sided die is rolled behind a screen. Initially our agent’s set of live possibilities is:

One possibility for each face the die might have landed on, all at probability 1/6. Then, she’s told that the die landed on a prime-numbered face. So her new set of live possibilities is:

Assuming she updates her probabilistic opinions in the standard way, her new probability for these possibilities is 1/3 for each of them. A nice thing about this framework is that it meshes neatly with expected utility theory; we can make predictions and/or recommendations about which bets she’ll accept before and after learning that the coin landed on a prime-numbered face. Moreover, these further claims about which bets maximize expected utility are not ad hoc additions—rather, they’re what inevitably follow given our reasonably natural starting assumptions.

But what if we want to represent a logically ignorant agent who isn’t sure which numbers between 1 and 6 are prime? Can we represent her as learning with certainty that the die landed on a prime-numbered face, but not learning with certainty that it didn’t land on 4? The short answer is that the most straightforward way of doing that is by using a framework for representing belief that is so flexible that everything—in particular, everything about how our agent’s beliefs will change in response to learning new information—has to be put in by hand. E.g., We can use a bigger set that doesn’t just include 6 possibilities, but instead includes 12—for each of the original possibilities, we can include a possibility where that number is prime, and another one where it isn’t.7 By brute force we’ve added in a possibility where 4 is prime, and allowed the agent to leave open that possibility when she learns that the die landed on a prime-numbered face. But we’ve done so at the cost of robbing our toy model of any predictive or explanatory power when it comes to the agent’s actions, or belief change over time; learning that the die landed on a prime number no longer has any automatic implications concerning what number—1, 2, 3, 4, 5, or 6?—it landed on.

How will the agent’s willingness to accept a bet to the effect that the die landed on 4 change in response to learning that the die landed on a prime number? In the standard model, with logical omniscience built in, we can get some non-trivial, surprising answers to questions like this. Granted, our example was simple enough that the answers aren’t that surprising. But in more complex cases, the framework can generate counterintuitive, but potentially illuminating results.

However, when we allow agents to be logically ignorant, we effectively rob ourselves of the ability to plug in some assumptions about an agent’s initial beliefs and values, and then see what they entail; once agents can be logically ignorant, any initial assumptions you start with will be compatible with any conclusions you like about how they’ll update their beliefs in response to new information, or what bets they’ll be willing to take.

The moral is that a noble aspiration—to accommodate the data that humans sometimes make logical mistakes—leads philosophers to a non-explanatory, “overfitted” theory of belief.8 Better to work with a theory that doesn’t accommodate all the data, if that’s the cost of explaining some of them.

Bullet Biters and Particularists

There’s a recurring divide in philosophy—sometimes explicit, sometimes just under the surface—between people who put more weight on abstract, general principles, and people who put more weight on judgments about specific cases. In a gem of a paper, Roderick Chisholm called the latter group, in which he placed himself, “particularists”, and the former he called “methodists”, largely as a joke. I’ll call them “bullet biters”. The bullet biter wants to find some simple, general theory—something clean and principled—and then follow where it leads, even if that means going to some strange places.9 The etymology, as far as I understand it, comes from the practice of soldiers being given a bullet to bite during amputations, when anesthesia was unavailable. Biting a bullet is a way of bracing for pain. The particularist, by contrast, starts with their judgments about individual cases, and is wary of any theory that strays too far from them.10 Particularists, in short, avoid (philosophical) pain.

The contrast is very clear in ethics. A utilitarian, for example, starts from a simple principle—maximize happiness—and lets that principle decide every case.11 If that means lying, breaking promises, or pushing one person in front of a trolley to save five, so be it. These are the bullets that utilitarians famously bite.12 Principled Kantians are bullet biters too. I’ve heard it claimed that the most influential contemporary Kantian, Christine Korsgaard, thinks its immoral to throw people surprise parties, because of the deception involved. At the other extreme, we have the moral particularist, who holds that “there are no defensible moral principles, that moral thought does not consist in the application of moral principles to cases, and that the morally perfect person should not be conceived as the person of principle.” The moral particularist doesn’t need to bite any bullets—if she judges that it’s wrong to push one person in front of a trolley to save five, her theory lets her do so—but she also can’t explain or justify her judgments about particular cases by appeal to general principles, because she doesn’t endorse any; according to the particularist, ethics is a matter of good judgment, and good judgment can only be imperfectly captured by general principles.

But it’s not just in ethics that we see this sort of divide. Some metaphysicians want a simple ontology that obeys a few tidy laws, even if it leads to surprising or revisionary consequences (e.g. nothing exists other than elementary particles). Others are more pluralistic, building complicated taxonomies to better accommodate our pre-theoretical beliefs about what sorts of things exist, and what sorts of things do not. It’s natural to see physicalists about consciousness—people who think that facts about conscious experience are ultimately nothing over and above facts about brain function, even if it’s admittedly hard to see how to explain consciousness in physical terms—as bullet biters. By contrast, people who insist that consciousness is too weird and distinctive to be explained in reductive physical terms are, on that debate at least, closer to the particularist end of the spectrum.

Seen through this lens, Williamson is a paradigmatic bullet biter. The book that made his name in philosophy was a defense of the idea that vague terms like “bald” or “tall” have precise boundaries, though we can’t know exactly where those boundaries are. This position—called epistemicism—entails that there’s a specific number of hairs that marks the dividing line between being bald and not. Most people find that hard to believe, to put it mildly. But he does a much better job than I at least would have thought possible of arguing that all the alternatives on offer are just as incredible. Very roughly, he argues that his view allows him to preserve the power and simplicity of classical logic, where every statement—including statements to the effect that someone with 32,207 hairs is bald—is either true or false. It’s very hard to avoid the consequence that there is some smallest number of hairs that suffices for not being bald without revising classical logic, or holding that logic is inapplicable to natural language. Both options involve serious costs.

In the recent book, various positions Williamson has argued for elsewhere, and which I won’t explain here—not just epistemicism about vagueness, but also the view that the natural language indicative conditional is the material conditional, as well as the view that formal semantics should be done in an intentionalist, rather than hyperintentionalist framework—are all defended by arguing that accommodating the data that counts against them would amount to overfitting. Better, he says—though not in this language—to bite the various bullets that are involved in holding the views just mentioned.

The military metaphor of bullet-biting can make it seem like it’s motivated by a kind of macho bravado: do you want to be a weak, wilting particularist? Are you afraid of a little (philosophical) pain? As I read Overfitting and Heuristics in Philosophy, it’s a defense of bullet biting that aims to put the practice on sounder footing than the previous would suggest. The reason to bite bullets isn’t that it’s fearless or badass. Rather, a willingness to bite bullets is the price of admission to a certain kind of philosophical-cum-scientific theorizing. If we want theories that do more than trace the shape of our pre-theoretical judgments—if we want theories that explain—then we need accept the kinds of constraints that sometimes force us to bite bullets.

In the late Twentieth Century, competition in the tech industry looked a lot like Schumpeter’s vision. In the 1970s and early 1980s, the dominant tech company was IBM. But it was Microsoft, not IBM, that developed the leading operating system. In 1995, Bill Gates wrote a memo warning that the internet was a “tidal wave” that would reshape the tech industry.1 But it was Google, not Microsoft, that developed the leading search engine. Google tried to compete in social media—remember Google+? But it was Facebook, not Google, that developed the leading social network. In each case, a startup figured out how to commercialize a new technology more effectively than an incumbent. And in each case, the startup’s success created a new large tech company.

In the last two decades, though, this pattern of creative destruction has broken down. As I write, the top six corporations by market cap are NVIDIA, Microsoft, Apple, Amazon, Alphabet (Google), and Meta (Facebook). All of these companies are old enough to drink. Microsoft and Apple were founded in the 1970s. NVIDIA, Amazon, and Google were founded in the 1990s. Facebook was founded in 2004. If you exclude spinoffs and conglomerates, the most valuable public company founded in the last two decades is 82nd-place Uber, which is valued around $190 billion. By comparison, the smallest tech giant, Meta, is valued around $1.8 trillion. Why have none of the startups founded in the last two decades brought competition to the aging tech giants?

In this post, I’ll argue that we should be concerned about the state of competition in the tech industry. For now, I’m going to put aside the difficult questions about what—if anything—antitrust enforcers can do about it. I’ll come back to those questions in my next post.

In this post, I’m going to draw on arguments from Coopting Disruption, a paper with Mark Lemley published in the BU Law Review, and No Exit, a paper with Brian Broughman and Sam Weinstein forthcoming in the NYU Law Review.

What Are VCs Good For?

If you want to predict our economic future, you should look to the venture capital market. Less than half of one percent of all new businesses raise venture finance. But that half of one percent has an outsize influence on the economy generally and innovation specifically. Venture-backed startups represented 47% of all IPOs between 1995 and 2019.2 And among the companies that went public during those years and were still public by the end of 2019, former venture-backed startups accounted for 76% of the total market cap and 88% of the total R&D spending. Each of the current tech giants—NVIDIA, Microsoft, Apple, Amazon, Alphabet, and Meta—was once a venture-backed startup.

The reason why venture finance is so important is that it funds risky business ideas that have the potential for exponential growth. Here’s how we put it in No Exit:

Imagine that it’s the 1990s, and you want to start a business. If you wanted to open a pizzeria, it wouldn’t be hard to find capital. You could use your own savings, borrow from friends and family, or get a loan from a bank. But suppose you want to build a software company. You’ll need a lot of capital to buy the servers and pay engineers. Unless you’re a billionaire or hang out with billionaires, you won’t be able to bootstrap your company. And you’re also not likely to get help from a bank. A loan officer doesn’t care if your business has a small chance of being worth billions one day. They just care whether you’re likely to make the interest payments on the loan and pay back the principal. If your business plan won’t generate revenue for many years, you’re not worth the risk.

The answer to your predicament is venture capital. VCs specialize in betting on high-risk, high-reward business ideas. They invest with equity rather than debt, so they care more about upside potential than downside risk. If your software company grows to be worth billions, their shares in your company will be worth tens or hundreds of millions. VCs are willing to take this kind of risk because they are diversified. They invest in a portfolio of startups knowing that many will fail but hoping that one of them will generate exponential returns that will make the portfolio a success.

Now of course, there are dangers to investing in high-risk businesses. A full accounting of the social impact of venture capital would need to include not just Google and Facebook but also Theranos and FTX. But for now I want to focus on something VCs used to be inarguably good at—turning risky startups into successful tech companies that challenged incumbents.

The Rise of Selling Out

In the 1990s, venture-backed startups had a predictable lifecycle. If a startup kept growing, it would raise another round of funding every 12-24 months. After a few rounds of funding, it would look for an “exit.” The word “exit” has two subtly different but overlapping meanings. An investor “exits” an equity investment in a company when they sell their shares for cash. A startup “exits” the private capital market when it goes public or is sold to another company.

The reason why a startup and its investors need an exit is that startups are—or used to be, more on this in the next post—illiquid. Startup founders and VCs can be worth a lot on paper yet be unable to convert that paper wealth into cash. An exit lets investors cash out. And in the case of VCs, that cashout is critical because they are investing other people’s money. VCs generally have ten years to deliver their returns to their own investors.3

Until recently, the path to exit was clear. The most successful startups would have an IPO. Moderately successful startups would be acquired. And the rest would simply go out of business. The VCs made their returns on the IPOs. Acquisitions were a consolation prize.

But after the dot-com bust, that path to exit changed. More and more startups started to exit by acquisition than IPO. Here’s a chart from No Exit that illustrates the trend:

In the spirit of Grecovian intellectual honesty, I should say that a dollar-weighted chart would look a lot messier. We don’t know exactly what the relative dollar value of IPOs and acquisitions is each year because for many acquisitions the price is undisclosed. But it’s fair to say that while a small number of IPOs are still generating a lot of value, the overall trend is more startups are choosing to exit by acquisition. Why?

One theory is that increasing regulation of public companies—particularly the Sarbanes-Oxley Act of 2002—made IPOs less attractive. Another theory is that technological changes increased economies of scale or scope, which made acquisitions more attractive. Incumbent tech companies built around large platforms, this theory says, are willing to pay more for startups with technologies that complement their platforms. But there is evidence that at least some of these acquisitions were anticompetitive.

Fun Emails by Mark

Some of that evidence comes from Mark Zuckerberg.4 In 2012, Facebook acquired Instagram for about $1 billion. Earlier that year, Zuckerberg wrote, in an email to Facebook’s CFO: “One business question[] I’ve been thinking about recently is how much we should be willing to pay to acquire mobile app companies like Instagram . . . that are building networks that are competitive with our own.” Zuckerberg explained that, though startups like Instagram are small, they are growing quickly, “the networks are established, the brands are already meaningful and if they grow to a large scale they could be very disruptive to us.”

After Zuckerberg sent that email, he likely spoke with a lawyer. Because he suddenly discovered some reasons to buy Instagram that weren’t anticompetitive. He wrote an email the next morning to clarify: “I didn’t mean to imply that we’d be buying them to prevent them from competing with us in any way. Buying them would give us the people and time to incorporate their innovations into our core products.”

Unfortunately for Facebook’s lawyers, Zuckerberg’s enthusiasm for merger synergies didn’t last long. In a private message on the day that Facebook bought Instagram, he wrote to a different colleague: “I remember your internal post about how Instagram was our threat and not Google+. You were basically right. One thing about startups though is you can often acquire them.”

The Coopting Disruption Hypothesis

Let’s put this together. We know that the incumbent tech giants haven’t faced a successful challenge since the rise of Facebook. We know that their strongest potential challengers—venture-backed startups—have been increasingly opting to sell rather than to go public. And we know that in at least some cases, the incumbents have intentionally acquired startups to choke off competitive threats.

But why only recently? Why didn’t Microsoft acquire Google when it had the chance?

Mark Lemley and I think the answer is that tech companies have learned how to coopt disruption. In 1997, the business scholar Clay Christensen published The Innovator’s Dilemma. In the book, he argued that incumbent tech companies face internal organization problems that cause them to miss out on innovation. The incumbents’ engineers, Christensen said, focus on delivering sustaining innovations—incremental improvements to their existing products that better serve their existing customers. But startups are more willing to pursue disruptive innovations—fundamentally new technologies that create new markets. Disruptive innovations can be inferior to existing products in some ways, but they can grow and improve and ultimately cannibalize existing markets.

To make this idea concrete, consider the technology I study for a living—automated driving. In a few warm-weather US cities, you can ride a robotaxi across town. Today’s robotaxis are inferior to conventional vehicles in some ways. They can only drive in small, geofenced areas. They won’t drive on highways. They sometimes get stuck and need remote assistance. But market prices already suggest that automated driving could upend the industry.

Automated driving was developed by the auto industry. The leading automated driving company is Waymo, an affiliate of Google. And its main domestic competitors are Zoox, an affiliate of Amazon, and Tesla—not the incumbent automakers. Some of the automakers—including Ford and GM—tried to develop automated driving but ultimately gave up. Instead, they have focused on developing driver assistance technology, a sustaining innovation that they can add as a feature to their existing vehicles. As Christensen observed, companies don’t like to develop technologies that will undercut their own revenue streams.

That same principle applies to tech. In automated driving, Google was the disruptor. But in search engines, Google is the incumbent. And AI represents a potentially disruptive threat to search engines. We doubt Google’s corporate development managers are naïve. In fact, we bet they’ve read Christensen or at least internalized its lessons. They know incumbents aren’t good at disrupting themselves. So how did Google respond to the disruptive threat of artificial intelligence? In 2014, it acquired DeepMind, then one of the most prominent AI startups.5

Other tech companies would have liked to follow Google’s lead in acquiring AI startups. But starting around 2020, antitrust enforcers started to scrutinize startup acquisitions more closely. So big tech has come up with more subtle means of influence. Microsoft poured $13 billion6 into OpenAI. Amazon poured $8 billion7 and Google $3 billion8 into Anthropic.

Why Worry?

I could imagine some readers thinking, sure, large tech companies may be acquiring nascent competitors—but what’s the harm? Who cares if AI is developed by big tech firms instead of startups? It’s hard to develop AI without a lot of cloud computing resources anyway, which makes partnering with big tech attractive.

But consider this. One of the DeepMind founders was a British engineer named Mustafa Suleyman. He claims that after Google acquired DeepMind, he pitched Google managers on creating a personal AI assistant.9 In his vision, the user would ask the assistant a question in natural language, and the assistant would provide its best answer. Suleyman said Google’s managers were unenthusiastic. Giving the user one best answer at the top would make it harder to sell ads. And Google’s business was selling ads in search results.

The world we may be heading towards is a world in which a few large tech companies set the innovation agenda. Entrepreneurs still found startups, and VCs still fund them. But the end game isn’t competing with the incumbents. It’s selling out to incumbents or partnering with them. And when incumbents acquire disruptive startups, will they give them the freedom to keep pursuing disruption? Or will they divert the efforts of the former startup’s engineers to the incumbent’s sustaining innovations?

Yes, there is fierce competition among the tech giants. And yes, that competition generates some innovation. But is it creative destruction?

Subscribe now

I’ve been interested in this topic for a long time. As a sophomore in college I took a class where we read Peter van Inwagen’s “It Is Wrong, Always, Everywhere, And For Anyone, To Believe Anything Upon Insufficient Evidence.” In it, van Inwagen responds to a familiar challenge to religious belief: that it’s irrational to accept the tenets of any one religion, given that there’s lots of religious disagreement, and no religion has anything close to majority support among people familiar with the historical evidence and philosophical arguments relevant to the distinctive claims it makes.

Van Inwagen’s response is what philosophers call a “companions in guilt” argument. If disagreement poses a challenge to the rationality of religious belief, he argues, then it’s also a problem for believing just about anything at all in philosophy, politics, or economics—areas where disagreement is rampant, and expert consensus elusive. But we don’t take disagreement as a reason to suspend all our controversial beliefs in those areas. So why should religion be held to a stricter standard?

I found van Inwagen’s claim that disagreement doesn’t pose a distinctive challenge to religious belief compelling. But where van Inwagen saw a reductio ad absurdum, I saw a powerful, general skeptical argument: a challenge not just to the rationality of religious beliefs, but to the rationality of confidently held beliefs on lots of disputed questions.

Here’s the rough shape of that challenge: On lots of hard questions, there’s no special reason to think your own judgment is more reliable than the judgments of others—especially people who’ve thought more about the question, who know more of the relevant evidence, and who have better natural or acquired abilities for evaluating that evidence than you do. If there is a consensus among such people that differs from your own opinion, you should defer to that consensus. If there is no consensus among such people—if they disagree—then you should be agnostic. Either way, you shouldn’t think for yourself. Or rather, think for yourself all you like, but when it comes to what you ultimately believe, go with the distribution of expert opinion.

There's been something of a cottage industry in philosophy on the epistemology of disagreement in the last couple decades. While I think lots of great stuff has been written, unsurprisingly, nothing like a clear consensus has emerged.1 This lack of a consensus is itself a topic within the field; how should views about the epistemology of disagreement handle the fact that people disagree about those very views? But my aim in this post will be to avoid taking a strong stand in that debate, while still drawing out some surprising consequences of what I take to be some broadly shared common ground.

Everybody will agree you shouldn’t think for yourself about some topics. Take string theory. If you want to know whether it’s a promising candidate for a unified physical theory, you shouldn’t try to evaluate the arguments yourself, unless perhaps you’re a theoretical physicist. The evidence is highly technical. The debates are complicated. If you're not in the field or at least pretty close,2 you’re probably not in a position to weigh the competing considerations. You’re better off deferring to the physicists. Not trusting any specific one of them—I’m sure you can find overconfident egomaniacs on both sides—but trusting the distribution of their opinions. If you can find a poll, or some other measure of what experts in the field believe, that’s your best bet.

Unsurprisingly, the case of religion is much more disputed. Both believers in the claims of particular religions as well as atheists will tend to think they can reasonably maintain their strong opinions, despite the existence of smart, well-informed people who disagree with them in good faith. What's the difference? Here's my take. In the hard cases, where arguments from disagreement seem to get less traction, there isn't a way of identifying the relevant experts—the people who it makes sense to trust—that's any less controversial than the disputed question itself. The question of who's competent to judge whether God exists is pretty much inextricable from the question of whether God exists. If you believe in God, you probably think that theologians are among the relevant experts whose beliefs deserve some kind of deference on the question. If you start out as an atheist, you’ll see the beliefs of theologians as much less reliable. Your view about who counts as an expert depends on your prior views about the subject matter. And if you can’t identify the relevant experts without first deciding the question under dispute—or at least engaging in a great deal of independent reflection about it—then you’re stuck thinking for yourself.

Now you might think that will apply pretty broadly. Arguments to the effect that you shouldn't think for yourself, but should instead defer to some kind of average of expert opinion won't get traction on most controversial questions—the sorts of things people argue about on Substack—because such cases will look like the case of religion; people's views about who's reliable and trustworthy will almost perfectly track their prior views on the substantive question under dispute. It’s easy and painless to defer to the “experts” if the way I identify the experts is just by looking for the people who agree with what I already think.

Index Fund Epistemology

This might sound like a general license to trust your own judgment on controversial topics. I think that’s too quick. Let me shift gears and introduce what might seem like a totally unrelated topic. It’s not.

Here’s some boring, normie, but very good investment advice: unless you work at a hedge fund or have deep financial expertise, you should invest in index funds. Buy them, hold them, and don’t try to pick winners. Don’t try to time the market. Don’t try to outguess the professionals. There is room for individual variation based on your risk preferences and time horizons—as you get closer to retirement, you’ll probably shift towards bonds and away from stocks—but it shouldn’t be your idiosyncratic view of where the economy is heading that drives your idiosyncratic investment decisions. This should sound familiar and obvious—I guess if you disagree, I’m happy to argue in the comments, but mostly I want to show you that this bland practical advice has some pretty far-reaching implications for what you should believe about lots of stuff.

Because it isn’t just practical advice. Rather, it’s epistemological advice with practical implications. The reason you should follow this investment strategy is because—at least if you’re not a professional—you probably shouldn’t think for yourself about asset prices. Rather than trusting your own judgment, you should trust the aggregated judgments of people with more time, more expertise, and more money on the line than you. Market prices already reflect their beliefs. If you think you know better, you’re probably wrong.

There’s a classic joke about this. Two economists are walking down the street. One sees a $20 bill on the sidewalk and bends down to reach for it. The other says, “Don’t bother. If it were real, someone would’ve already picked it up.”

The joke is absurd because people sometimes do drop $20 bills, and when it happens there’s no reason to think you shouldn’t be the lucky one to find it first. But opportunities for profit on the stock market aren’t like that. It’s a bit like if I were extremely shortsighted, walking very slowly down a street filled by speedy, eagle-eyed people looking for fallen money. In that case, I really should doubt that I’ll ever be the first to spot the cash.

I don’t need to assume anything like a strong or even semi-strong form of the efficient market hypothesis here. For all I’m saying, there may very well be bubbles, and prices may sometimes fail to reflect fundamentals. All I need is the weak, plausible claim that most of us should never take ourselves to be in a position to spot such cases.

Here’s the punchline. If you accept the logic behind index fund investing, you’re already accepting a big constraint on how much you should trust your own judgment. Because lots of your political, economic, and technological opinions likely have implications for asset prices. If you think AI is massively overhyped, or if you’re convinced that it will wipe out most white collar jobs in the next ten years, or that Trump's tariffs will send inflation soaring, or that they’ll revitalize America’s industrial base—all views you can find defended on this platform—those stances should, in principle, inform how you invest.

If you're confident that AI is massively overhyped, you should probably short NVIDIA.3 Alternatively, if you think AI will render huge swaths of jobs obsolete and trigger major productivity gains (and profits for those who control the models), then you might want to double down on tech giants, chipmakers, cloud infrastructure providers, and maybe even labor-replacement adjacent firms—think robotics.

If you're confident Trump will cause serious inflation, then buy TIPS (Treasury inflation-protected securities) and short T-notes. You might also look to hedge with commodities, gold, or energy stocks, which tend to do well in inflationary environments.

Alternatively, if you think Trump will revitalize American manufacturing, then you might want to buy GM and Ford, sure—but also construction materials, steel producers, and defense contractors.

But if you’re not doing any of that—if you’re just holding index funds like everyone says you should—then you’re tacitly admitting that you don’t really trust your own judgment on those questions. Good for you!

Two final points to close.

First, I want to acknowledge that none of this helps with the hard cases we started with—religion, morality, deep political disagreements. There are no market prices for the existence of God or the permissibility of abortion. So nothing I’m saying here helps resolve those cases. That’s part of why I think the point I’ve been making should be broadly acceptable, even to people with very different views about the epistemology of disagreement.4 I’ve been trying to identify a special case where it will be widely accepted that most of us shouldn’t think for ourselves, and to convince you that it has broader implications than you might otherwise have thought.

Second, if you like thinking for yourself, and are threatened by the idea that you should do less of it, don’t despair! It takes a lot of thinking for yourself to take the information encoded in markets—their best guesses about things like future inflation, energy demand, or the pace of technological change—and weave it into a broader view of the world. A disposition to trust the judgments encoded in asset prices isn’t really an alternative to thinking for oneself, so much as a check on it; if you find yourself with a bunch of beliefs that imply things about asset values—and once you get in the habit of asking the question, you’ll probably realize this happens more often than you might have thought—then you should go and check to see what the prices actually are. Maybe you’ll find that your views are already reflected in them. If so, great! But often, you’ll find they’re not. When that happens, you have reason to think for yourself about where you might be going wrong.

I tell this story mostly because I like it, but also to establish my credentials as someone for whom the economist’s way of looking at the world comes pretty naturally. I’ve co-taught a class called “Rational Choice Theory and its Critics” several times now with a colleague of mine in the economics department, but both of us were more sympathetic to rational choice theory than to the critics. And my book on idealization in epistemology was probably influenced more by Dani Rodrik’s book on economic methodology than by any other single work. So I don’t come to this topic as a skeptic of economics. Rather, in the spirit of my earlier post on Mill’s Trident, I aim to raise awkward questions for a position I’m ultimately sympathetic to.

I read the IGM surveys with interest, and tend to trust the collective judgment of the economists they poll over my own on the sorts of questions they’re asked.1 But not everybody does; you won’t have the scour the internet for all that long to find commentators who are much less disposed to defer to the judgment of economists than I am. My sense is that in the past, skepticism about the reliability of economists was more likely to come from the political left. Perhaps the edifice of academic economics is best understood as part of the ideology of the ruling class, whose function is to provide rationalizations for maintaining the status quo rather than to be a genuine, knowledge-producing social science. Then economists’ opposition to policies like rent control, or laws against price-gouging, can be safely dismissed. Today, distrust of economists seems about as likely to come from the political right, who need some strategy for explaining away the fact that economists overwhelmingly favor free trade. But I aim to raise questions about what grounds outsiders have for trust in the discipline without any particular political agenda, though I admit it’s their contemporary political relevance that has me thinking about them.

What distinguishes the economist’s approach to economic questions—at least in its contemporary, academic form—is its commitment to mathematical modeling. Starting around the middle of the twentieth century, with figures like Paul Samuelson, mathematical formalism became the defining feature of serious economic work. Today, an aspiring academic economist needs to take real analysis; it’s often math majors, rather than economics majors, who are best prepared for graduate study in the field. This modeling orientation is part of what gives economists their distinctive claim to authority. It also creates barriers to entry. Many people have intuitions about markets, prices, and trade. But the economists’ claim to special insight rests in part on the idea that informal reasoning is unreliable in this domain—that it leads to fallacies or overlooks important equilibrium effects that only formal models can capture.

But why should we trust that mathematical modeling in economics is reliable? Even in the natural sciences, which provided the inspiration for the turn towards mathematical modeling in economics, the practice can seem “unreasonably effective.” There’s no clear a priori reason why the highly abstract structures studied by pure mathematicians—e.g., in group theory, or non-Euclidean geometry—should turn out to have such important applications in physics. Paul Dirac put it memorably:

“The mathematician plays a game in which he himself invents the rules while the physicist plays a game in which the rules are provided by nature, but as time goes on it becomes increasingly evident that the rules which the mathematician finds interesting are the same as those which nature has chosen”

The economic theorist is playing a game that can look a lot like the game of the pure mathematician, studying and proving things about the behavior of idealized populations of perfectly rational, utility maximizing, computationally unbounded agents. But it’s at least not obvious that economic theorists are as lucky as pure mathematicians in having an aesthetic preference for abstract structures that turn out to be useful in predicting and manipulating real-world phenomena.

In the wake of the financial crisis, Paul Krugman wrote a famously pessimistic essay, lamenting that “economists, as a group, mistook beauty, clad in impressive-looking mathematics, for truth.” Now I’m sure that even then, Krugman did not think the Samuelsonian revolution was a wrong turn. He did not recommend that economists return to the 19th century and trade in their equations for prose. But for outsiders less inclined to trust, and less invested in the toolkit that Krugman and others are so specialized in, what assurances can be offered that the methods of contemporary economics are fit for purpose?

With the physical sciences, engineering is what lets outsiders know that the models are appropriately tethered to reality. Most people don’t understand mechanics, but they can also see that people using models based on ideas from physics are able to build working airplanes. It would be miraculous for people relying on thoroughly erroneous principles to nevertheless succeed in getting roughly a million pounds of metal, luggage, and flesh from New York to LA in under six hours.2 So the best case that could be made, to a skeptical outsider, that economists’ models are similarly tethered to economic reality would be to point to economic analogues of airplanes. That is, we’d like to see economic artifacts that obviously perform a useful function, and whose design essentially relied on methods peculiar to economics.

When I’ve put this question to economists, and since then to AI, the first example that always comes up is auctions—especially spectrum auctions—usually followed by other examples of mechanism design (e.g., the medical school residency match system). Governments are often in a position where it makes sense to sell goods or rights via an auction. It makes sense for different parties to have rights to different part of the broadcast spectrum—so that signals won’t interfere with each other—so you need some way of determining who gets to broadcast on which part of the spectrum. An auction is a natural way to do this. But governments designing such auctions face lots of pitfalls. Bidders may collude to keep prices down: e.g., “if you promise not to outbid me on this lot, I’ll promise not to outbid you on that one”. And because the goods being sold are heterogenous—different parts of the broadcast spectrum are best for different purposes—it’s important for them to go to the parties that will make the best use of them. Different ways of designing auctions will be more or less likely to efficiently allocate the goods being sold.

My understanding is that spectrum auctions have gone much better, by uncontroversial metrics, when economists have played a central role in designing them.3 There were auctions before auction theory, and auctions afterwards, and the ones afterwards raised much more revenue than the ones before. Auction design is also important in the private sector; e.g., Google hires economists to design the auctions it uses to sell advertising.4 So suppose our skeptic is willing to grant this much: auction design, and perhaps mechanism design more broadly, enables economists to reliably achieve uncontroversially better results than would be possible without the use of highly abstract and sophisticated game theory.

How far does this get us towards establishing a general trust in economics, of the sort that would cut ice in discussions of rent control, or tariffs?

Not all that far, I fear. First, have you ever participated in a spectrum auction? Didn’t think so. Have you ever flown on an airplane? (or used a computer, or a gps system, or…) If mechanism design is the economic analog of airplanes…it’s a lot less important and impressive than airplanes!

While that may seem glib, the fact is that mechanism design is fundamentally simpler than most economic policy. To borrow an analogy from the physical sciences, mechanism design is like studying a closed, highly controlled system—a chemostat in a lab, where the experimenter can set pH levels, temperature, the concentration of key nutrients, and pretty much everything else relevant to bacterial growth. In such conditions, the effects of interventions are highly predictable, and precise manipulation of the system is possible. But most economic policy is more like trying to intervene in a real-world pond: an open system potentially influenced by all manner of of unobserved variables.

In an auction, the economist gets to set the rules and define the terms of engagement. It’s a system built from scratch, with limited variables and clear constraints. But broader economic policymaking rarely affords that kind of control. Policymakers step into a living system already shaped by history, politics, and culture—one they don’t fully understand and can’t easily steer. The agents involved may not behave as models predict and the effects of policy are filtered through layers of institutional and social complexity.

Once we move past the best cases of mechanism design, while it’s easy enough to find institutions whose workings depend crucially on esoteric economic modeling—central banking being an obvious example—it’s much harder to make a persuasive case that the modeling is essential to the institution’s functioning well. What would happen if central banks used simple, “set-and-forget” rules that didn’t require sophisticated modeling of the economy to apply?5 Or if central banks had never been established in the first place? Would we be worse off? These are extremely difficult, opaque counterfactuals; they’re nowhere near as obvious as questions about what would happen if amateurs tried to design rockets.

A natural objection to the pessimism I’ve been offering up so far is that for the last several decades the real action in economics hasn’t been in abstract theory but in empirical work. The “credibility revolution” has shifted prestige toward studies that isolate causal effects using natural experiments, or in the best case actual randomized controlled trials. For instance, if a policy is implemented in one state but not another, economists can compare outcomes in similar neighboring counties that are newly subject to different laws. Or if admission to some specialized education program is conditioned on achieving a certain score on a test, economists can compare students who just barely scored high enough and were admitted to the program to those who scored just below the threshold, on the theory that they’ll be otherwise quite similar, and thus that the comparison will isolate the effect of the program. Or in the best case, aid programs with limited budgets can randomize how their aid is dispersed, and aid recipients can be compared to a control group.

Maybe my lack of skepticism is showing, but I think it’s hard to read about these methods and not conclude that, when used properly, they can tell us a lot about what effects policies had in a particular time and place. But even the best economic experiments—natural or otherwise—get us a lot less than we’d like, as they typically face difficult questions of generalizability. You can be pretty sure that electrons in China behave much the same as electrons in Mexico. But the causal effect of an economic policy in one setting may not travel well to another, because it depends on cultural norms, informal institutions, or political dynamics—factors that vary from place to place in ways that economists lack good theories of. So even if I become convinced that, e.g., giving people lentils as an incentive worked to increase rates of vaccination in rural India, I might remain just as agnostic as before about what would work in Idaho. Moreover, many of the most pressing economic policy questions can’t really be studied by natural experiments, as they’re never implemented in random or quasi-random ways in otherwise similar places and times.

So far I’ve been trying to steel man the position of someone who supports rent control, or laws against price gouging, or steep tariffs, and isn’t much bothered by the fact that economists overwhelmingly doubt that such policies will produce their intended effects.6 But I do want to say a bit about why I still put such stock in the collective judgment of economists, despite basically agreeing with the criticisms above. To be clear, this is offered in the spirit of autobiography rather than argument; from here on out I’m not trying to be persuasive, but just reporting my own state of mind.

There’s an old joke about two hikers who run into a grizzly bear. One kneels down to tie his sneakers. The other says, “What are you doing? You can’t outrun a bear.” The first replies, “I don’t need to outrun the bear—I just need to outrun you.”

That’s more or less how I feel about trusting economists. I don’t have to think they’re especially good at predicting the effects of economic policies in absolute terms. I just have to think that the alternatives—relying on my own intuitions, partisan punditry, or critics armed with cherry-picked case studies—are even worse. The most serious critiques of economics rarely offer better methods. At best, they point to places where mainstream assumptions failed. That’s important. But pointing out that the quarterback threw an interception doesn’t mean you could’ve read the defense better, let alone that the team would be wise to put you in for the rest of the game.

For all its limitations, economics as a discipline at least tries to learn from its mistakes. While economics is not physics and never could be, imitating the hard sciences has some real benefits: economists care about falsification. They are expected to test their models against data and to revise or abandon them when the fit is poor. That’s an enormously valuable epistemic practice—and one that shouldn’t be taken for granted. Absent such norms, the natural human tendency—and the one I see in much criticism of economics—is to present your views in the best possible light, rather than to subject them to severe tests. And while the limitations I’ve discussed are real, I haven’t dwelled on the various counterintuitive phenomena that economics gets right; none of these successes are as impressive as airplanes, but they’re also nothing to sneeze at. So I’ll keep reading the IGM surveys.

I’ve always loved reading fiction. I remember a trip I took as a child at summer camp when the other kids were fishing off the side of the boat—that was the whole point of the excursion—and I was inside the boat reading. At least one of the counselors was concerned for my mental health, but I was perfectly happy. Now, one of the highlights of my day is reading aloud to my children; right now we’re in the middle of the Lord of the Rings, and it’s loads of fun.

But many people think reading fiction is more than just fun. Rather, reading fiction—or at least, reading great fiction—is supposed to teach us deep truths about the human condition. I want to focus on a particular claim in that ballpark, which is the idea that in reading fiction, we gain an ability to know what it’s like to live lives other than our own. Neil Gaiman puts it nicely:

Fiction gives us empathy: it puts us inside the minds of other people, gives us the gift of seeing the world through their eyes. Fiction is a lie that tells us true things, over and over.1

In a podcast conversation between Russ Roberts and Julia Galef, the topic turned to how we can rationally make transformative life decisions—whether to have children, for example. Galef said she’d ideally want robust empirical data: longitudinal surveys tracking life satisfaction across different demographics based on what sorts of decisions they made. Roberts was skeptical. He suggested that literature and poetry might offer deeper insight—that fiction is a better tool for understanding what it’s like to make such choices.

One more example. While I was thinking about these questions, I got an email asking me to share an announcement about a new course in the English department with the philosophy majors at my university. The course description included the following passage:

In this course we tour and practice some of the key capabilities literary education seeks to cultivate. These include (among many others!) the ability to imagine the minds and lives of other people.

But does reading fiction in fact give us these epistemic powers? Do regular fiction readers become better at imagining what it’s like to be other people, or knowing what it would be like to lead alternative versions of their own lives? To be clear, I’m not just asking whether fiction expands the bounds of what we can conceive; I’m interested in accuracy. When we try to picture what the world looks like through another person’s eyes, we can do a better or worse job; our imagined version of someone else’s inner life might match their actual inner life well, or poorly. Does reading fiction help us with this matching task?

I’m skeptical. I think it’s a deeply unobvious, empirical question, and I suspect that the widespread tendency to attribute various spillover benefits to reading fiction has more to do with social desirability bias—you don’t want to sound like a philistine, do you?—than with an appreciation of powerful evidence or arguments.

Isn’t this a question for social scientists?

I just said that it’s a difficult empirical question whether reading fiction has the epistemic benefits often claimed for it. So shouldn’t we just see what our best social science says? Unfortunately, it’s a very difficult question for social science. Of course, it’s easy to take surveys where you ask people how much they read, ask them a bunch of other stuff, and then see how well their reading habits predict the other stuff. That’s been done for all sorts of traits.

The big problem for this sort of approach is that habitual readers differ from habitual non-readers in lots of ways. E.g., if people who read a lot also live longer, is that because of the reading itself, or because of something else? E.g., maybe the correlation exists because people who have more free time are less stressed, which leads to longer life, and also means more time to read. If that were so, then reading would have no effect on longevity—both the reading and the longevity would have free time as a common cause. That would be fine if you could measure free time, and see if reading and longevity are still correlated even after controlling for free time. But maybe your data set doesn’t include free time. Or maybe there are lots and lots of other factors that could be playing a role; maybe it’s about wealth, or personality, or who knows what else.2

The standard response to these sorts of problems in social science is randomization. If you want to figure out whether some action affects some outcome, you can’t just observe people who do the action, and people who don’t, and compare outcomes. That approach has the problem above: you might actually be measuring the effect of something else that both causes them to do the action, and affects the outcome. Instead, you’ve got to take a bunch of people, randomly force some of them to do the action, and some of them not to, and then measure outcomes. That way, you know it’s the action that’s making the difference to the outcome, rather than some underlying difference that explains why some people do the action and others don’t.

When the action is something like: “take this drug, available only in the context of the study I’m now running”, that’s reasonably easy. Everybody wants the drug, and for the ones you randomly assign to take it, they’ll happily take it when you give it to them. For the ones you randomly assign not to take the drug, you just refrain from giving it to them. This is why randomized drug trials are possible, and informative.

But what if the action whose effects we want to study is habitual reading over the course of one’s whole life? Imagine trying to take a group of kids, and randomly forcing some of them to read a lot for the rest of their life, and forcing others to read much less, and then measuring them at age 50. Cool idea! Not gonna happen.

What can be done, and has been done, is to put people in a lab and make them read for a short amount of time, and then give them various tests, like inferring people’s emotions from their facial expressions. Initially it looked like people were better at these tests after they read some literary fiction. Later studies failed to replicate the result. But even if the result had held up, I don’t think it would have been very informative. A constant and frustrating feature of psychology is that there’s often a vast gulf between what we’re really interested in, and what we can measure. What’s the relationship between the short term effect of reading a story in the lab, and the long term effect of a lifetime of habitual reading? What’s the relationship between scores on a lab test of inferring emotions from facial expressions, and the real-life task of empathizing with other people? The research can’t really help here.

Why So Skeptical?

That’s why I’m pessimistic about social science telling us all that much about our question. But we can still speculate! From the armchair, should we expect that a lifetime of reading gives people the power of cognitive empathy—the power to know what it’s like to be other people, and what it would be like to live hypothetical alternate versions of their own life?

How would that work? The natural, almost inescapable thought is that it would involve a kind of pattern recognition and extrapolation. If you’ve read a bunch of fiction—or at least, fiction of the right sort—you’ll have encountered a wide variety of characters with complex inner lives, and you’ll have seen how those inner lives were affected by external events, as well as how they manifested in behavior. For instance, even if you’ve never experienced the loss of a sibling—maybe you’re an only child, so it’s doubly hard to imagine—if you’ve read fiction in which characters have, then you have available a kind of lens through which to interpret people you meet in real life who have suffered that misfortune. Read enough fiction, and you’ll have a large enough body of “data”, and patterns you’ve learned from that data, that when it comes to interpreting real people and imagining what their inner mental lives might be like, you’ll be much better at extrapolating from the public, observable facts you know about them to the unobservable facts about their private experiences.

The implicit model I’m offering here is one where the task confronting a would-be empathizer—someone who wants to know what it’s like to walk in someone else’s shoes—is a kind of learning problem. Given limited data (what you can observe about someone), draw inferences about unobservable hypotheses (what their mental life is like). Especially today, we’re familiar with the idea of general purpose learning algorithms—like large language models—doing an exceptionally good job of drawing reasonably accurate inferences, at least when trained on a large and representative enough set of data. Effectively, I’m suggesting that you can think of the way fiction could work to make you better at imagining what it’s like to be other people as operating on a similar principle. Fiction could act as extra training data for the inference engine in your head.

One thing I like about this picture is that, at least in principle, we could test it. Imagine loading up a version of ChatGPT that’s been trained on all the great literature that’s ever been written (you can decide how to curate the list of the greats). Now, start telling it things about your own life that a casual acquaintance might know, and ask it to guess at facts about your inner mental life—your hopes, fears, anxieties, and so on—that are known only to you. How good a job do you think it will do? I haven’t done the experiment, but here’s my guess at how it would turn out.

My guess is that it would predict that your inner mental life is more interesting, more unusual, more compelling to read about, in a word, more novelistic, than it really is. There are systematic differences between what tends to happen in fiction and what tends to happen in the real world, and our hypothetical instance of ChatGPT will be biased towards interpreting your life as more like the former than the latter. This is easiest to see when we think not about characters’ inner mental lives, but about plot and narrative structure. Consider the principle of “Chekhov’s Gun”: “If you say in the first act that there is a rifle on the wall, in the second or third act it must absolutely go off. If it’s not going to be fired, it shouldn’t be hanging there.” While this rule is adhered to in much fiction, the real world isn’t constrained by any such canons of narrative construction. Actual human lives are like bloated manuscripts, full of loose ends, unresolved plots, and extraneous detail that a good editor would delete. Indeed, you could do worse than summarizing the paranoid, conspiracist style of thinking as one that involves interpreting real-world events as if they had the coherence and tidiness typical of fictional narratives. It’s probably hypocritical to cite fiction in making my point, but the pitfalls of interpreting real life through the lens of fiction is a theme in Jane Austen’s Northanger Abbey. In that book—mild spoilers ahead—the character Catherine Morland has subsisted on a diet of Gothic novels, and consistently goes wrong in interpreting the people in her own life through that lens; when she gets to Northanger Abbey, she views the family patriarch as a Gothic villain, and looks for signs that he murdered, or is secretly imprisoning, his late wife. In fact, the household is much more boring and unremarkable than she imagines.

The point I’m making, while easiest to accept when we’re thinking about extrapolating from fictional plots to real-world sequences of events, doesn’t really lose much force when we move to the similar task of extrapolating from fictional characters’ inner mental lives to those of real people. Just as fictional plots tend to conform to canons of narrative construction that lead them to systematically differ from reality, the same is true of the relationship between fictional characters’ mental lives, and actual people’s mental lives.

In fiction, characters often exhibit a sort of psychological compression: they’re built around a small set of consistent traits or conflicts that readers can easily recognize and track. Fictional characters tend to have a logic to them that real people often don't. Think about Jay Gatsby—spoilers again. He’s one of the most famous and memorable characters in American fiction: romantic, mysterious, tragic. But part of what makes him compelling is that his entire inner life can be explained through a single obsession—his desire to recapture a perfect past with Daisy Buchanan. Every choice he makes, from the house he buys to the parties he throws, serves that purpose. That’s great character construction. But it’s also deeply unrealistic. Real people aren’t usually built around one driving motive that explains everything. They’re messier. They start projects that don’t go anywhere, and then later forget that they’d been so invested in them. They act out of habit, or boredom, or vague half-formed wants that don’t tie together into a satisfying arc. Gatsby’s psyche is artfully pruned for symbolic and narrative effect. Most people’s minds are more like overgrown gardens.

Basically, I’m suggesting that we should think of the stock of inner lives that we encounter in great fiction as a source of data, yes, but as a systematically biased source of data.

None of this is to deny that fiction can be illuminating in particular cases. If you want to understand how Nora Ephron experienced her divorce, Heartburn would probably help; if you want a feel for post-Civil War life in the American Midwest, Little House on the Prairie is a natural place to start. But those are targeted uses, where the match between fiction and reality is either biographical or tightly constrained. They don’t license the broader claim that reading fiction generally improves our ability to understand people. If anything, they highlight what’s missing in the general case: some independent reason to trust that the fiction tracks the reality.

I don’t mean to defend philistinism. There are many reasons to read fiction that have little to do with its supposed cognitive benefits. We might read for aesthetic pleasure, for the beauty of language, or simply for the joy of getting lost in a story. Sometimes we read to feel things more vividly—to echo or intensify emotions we’re already carrying, much like we might listen to sad music when we’re sad. Fiction can help us name what we’ve felt but couldn’t articulate, or imagine possibilities that hadn’t occurred to us. What’s less clear is whether reading fiction makes us better at understanding other people. That idea is tempting—but perhaps we’re tempted by it for the same reason we enjoy fiction itself: because it’s a compelling story. We should enjoy fiction without needing it to justify itself with grand psychological promises. Sometimes a good story is enough.

I mostly intend to avoid writing about hot button issues in contemporary politics on this Substack. First, because lots of other people are talking about them, it’s hard to say anything distinctive. And second, talking about politics tends to bring out the worst in us, myself included. Plenty of people who are careful, subtle, and charitable when writing about Kant or music or quantum mechanics become simplistic and Manichean when the topic turns to politics.1 They rigorously scrutinize evidence that purports to undermine their convictions, while breezily endorsing sloppy arguments for conclusions they agree with. Statements from opponents are inspected for unsavory associations, while statements from allies are read as charitably as possible. These temptations exist for everyone, and the easiest way to avoid them is to stick to topics that don’t get our hackles up, where it doesn’t feel like the potential answers to the questions we pose already come associated with one or another political team.

So mostly that’s what I’ll do. But when I do write on politics, here’s how I’ll attempt to avoid these common pitfalls. I’ll try to limit myself to friendly fire; I’ll raise problems for views that I’m largely sympathetic to, rather than attacking positions I think are thoroughly misguided. I’ll focus on the planks in my eye, rather than the specks in others’. Instead of preaching to the choir, I’ll ask whether they’re singing off-key.

Today, my topic will be an argument called “Mill’s Trident”. Here’s how Greg Lukianoff, president and CEO of FIRE, summarizes it:

There are only three possibilities in any given argument:

1. You are wrong, in which case freedom of speech is essential to allow people to correct you.

2. You are partially correct, in which case you need free speech and contrary viewpoints to help you get a more precise understanding of what the truth really is.

3. You are 100% correct, in the unlikely event that you are 100% correct, you still need people to argue with you, to try to contradict you, and to try to prove you wrong. Why? Because if you never have to defend your points of view, there is a very good chance you don’t really understand them, and that you hold them the same way you would hold a prejudice or superstition. It’s only through arguing with contrary viewpoints that you come to understand why what you believe is true.

Most of the times I see this kind of reasoning deployed, I agree with the position it’s being used to defend. One of the first things I did after getting tenure was sign up as a member of Heterodox Academy. A few years letter, I was happy to add my name to a letter encouraging, among other things, “activities on campus intended to foster greater tolerance for diverse points of view.” As a materialist atheist, I co-taught a class on issues in the philosophy of religion with Ross Douthat, who thinks everyone should be religious, and in admitting students to the class we prioritized having a wide variety of religious and non-religious views represented. I’m trying to establish my bonafides as someone who is very much a supporter of dialogue across disagreement, and for whom a broadly Millian attitude towards free speech, open inquiry, and viewpoint diversity feels deeply correct.

That said, I think abstract, content-neutral arguments about these values—arguments like Mill’s trident—can’t get us all that far. In particular, in contexts where the question isn’t merely about permitting inquiry, but supporting it—with government grants, tenure track professorships, and the like—it’s impossible for all views to be equally supported, and anyone allocating limited resources needs to make substantive decisions about which projects are the most promising. In such contexts, Mill’s trident is blunt.2

That’s where I’m going. But I’ll take a detour via Karl Popper’s philosophy of science. I’ll argue that some familiar problems for Popper’s philosophy of science closely parallel problems for Mill’s trident.

Popper

Philosophers love to hate Karl Popper.3 Why? Certainly part of it is jealousy. Plenty of philosophers have science envy, and scientists love Karl Popper.4 But scientists also love Einstein, and philosophers love to love Einstein. Philosophers love to hate Popper because they think he’s undeservedly popular. Why would they think that?

Popper is mostly known among the public for his view that what’s important about science is falsifiability. A good scientist makes concrete claims that risk being proved wrong, rather than hiding in the refuge of unfalsifiable pseudoscience. In my view, this is a fundamentally sound idea, but it’s not really all that distinctive to Popper—plenty of other contemporaneous writers in the positivist tradition had similar views about the importance of making empirical claims that could be tested by observation.5 What’s truly unique to Popper among recent philosophers of science are his views about induction: the practice of extrapolating from past observations to predictions about the future.

Hume famously raised deep skeptical puzzles about induction. In a very small nutshell, he argued that it was impossible to offer a compelling, non-circular defense of the practice.6 Most people who encounter Hume’s discussion of induction see it as a paradox to be resolved. Kant famously called it a “scandal” that philosophy as yet had no satisfying response to Hume’s skeptical challenges. But not Popper. He thought the straightforward, skeptical lesson of Hume’s discussion was just correct. There really is no good answer to Hume’s problem of induction: observations of the past give us no reason to form beliefs—not even tentative, probabilistic guesses—about what the future will be like. So we should interpret scientific practice in such a way that it doesn’t involve doing anything of the sort. I know this sounds crazy. But he’s pretty explicit. Here’s a representative quote from the preface to Conjectures and Refutations:

The way in which knowledge progresses, and especially our scientific knowledge, is by unjustified (and unjustifiable) anticipations, by guesses, by tentative solutions to our problems, by conjectures. These conjectures are controlled by criticism; that is, by attempted refutations, which include severely critical tests. They may survive these tests; but they can never be positively justified: they can neither be established as certainly true nor even as 'probable'

So if scientists aren’t trying to show that their conjectures are true, or even probable, what are they trying to do? They’re not trying to verify or confirm, but only to falsify. The best case scenario is that a conjecture has survived many tests—it has withstood repeated attempts at falsification. But even then, we don’t get to believe it’s any more likely to be true.

There are lots of reasons to find this view absurd, but I’ll focus on just one. An important aspect of scientific practice is that scientists try to test theories in novel ways. They build giant, multibillion-dollar particle accelerators so they can smash stuff together harder and faster than ever before, in the hopes of getting new insights into the fundamental fabric of the cosmos. But that only makes sense against the background assumption that we need to spend all that money devising novel experiments because rerunning old, familiar experiments would be uninformative. If you want to test general relativity, why not just keep dropping a ball in your lab, measuring the fall each time? On Popper’s view, it’s hard to see why that’s not just as good a test as what we’d get from observations of previously unseen distant galaxies; both provide opportunities for falsification.

The reason we don’t keep dropping the ball is because we already know what we’ll see if we do so. By contrast, we don’t already know what we’ll see when we build the new particle accelerator or space telescope. Or, if you prefer to think about our beliefs in degrees, at least grant that we are nearly certain what will happen when we drop the ball over and over, while we are highly uncertain what we might learn from novel sources of evidence. The potential for resolving that uncertainty is why we value novel experiments, and the lack of uncertainty to be resolved is why we’re uninterested in rerunning old experiments. But on Popper’s view, we know nothing about the future—we don’t even get to assign probabilities. And so it’s hard to see what rationale there could be for prioritizing new sources of evidence over old ones. Yet clearly that’s what scientists do, and rightly so.

Mill

Popper’s error amounts to imagining scientists as more humble and open-minded than they really are or ought to be.7 Mill’s trident involves a similar mistake. Just as Popper’s view of science becomes hard to square with how scientists actually behave, so too Mill’s defense of free speech starts to look questionable once we notice that good thinkers, like good scientists, eventually shift their attention away from the already-settled. The parallel is this: Popper’s view implies we never get to be confident about what we’ll see when we rerun old experiments, so we have no reason to do new experiments. Mill’s Trident—the third prong in particular—implies we never get to rationally disregard a viewpoint, so we have no reason to stop engaging with it. In both cases, there’s no license to move on.

But of course we do move on. Scientists stop re-running the same experiment once they’re confident what the results will be, and shift their attention to more uncertain areas. Similarly, thinkers and institutions stop engaging some arguments—not because they’ve been censored, but because they’ve been settled. That’s not always a mistake. Sometimes, the Overton window narrows for bad reasons. But sometimes it narrows for good ones—for reasons that are themselves the product of open inquiry, evidence, and debate.

This argument is difficult to make for philosophers, who are temperamentally averse to treating anything as settled. Aristotle proposed the law of non-contradiction over two thousand years ago, but there are well respected philosophers today who argue that some contradictions are true. If we can’t treat basic laws of logic as settled, there’s little hope we’ll agree on much else. But not all disciplines are philosophy, and thank goodness for that. What’s the point of all this searching for truth if we never get to decide—at least provisionally—that we’ve found it?

If Mill’s trident shows anything, it is that there’s always some possible benefit from engaging an argument you disagree with, even if you’re sure it’s wrong. But that’s a very weak claim. There’s also some possible benefit from dropping a ball and measuring its rate of descent, just in case this time it falls faster than current physics says it should. But in both cases, if there’s anything else you might be doing with your time—if you have limited resources to devote to your inquiries—you might reasonably decide that slight chance of benefit is not worth it.

So What?

Arguments like the ones I’ve just offered can be misused. They’re often deployed to deflect concerns about ideological homogeneity: “Sure, we don’t have any creationists in the department of ecology and evolutionary biology, or flat earthers in the department of geology, or holocaust deniers in the history department, but that’s fine: not every argument is worth engaging.” Fair enough. We shouldn’t demand that every view be institutionally represented, regardless of its merits.

But that concession shouldn’t be the end of the discussion. It should be the beginning. If you think the narrowing of academic debate is a problem, you can’t just point to the fact that some views aren’t represented; you need to make a targeted case that we’d be better off spending more of our intellectual energy discussing the views you worry we’re missing. But the same goes for defenders of the status quo. It’s not enough to note that some views are beyond the pale and gesture vaguely at crackpots and bigots. If you think we haven’t lost much by marginalizing certain kinds of dissent, then you too need to make a substantive case: that the views in question are sufficiently implausible or intellectually barren that there’s little to be learned from engaging them. As it turns out, I think that in plenty of parts of academia there’s a powerful case to be made that we need more political diversity; we’re missing out on perspectives that are not relevantly similar to holocaust denial, and our research suffers for it.8 But the case will always turn on a series of messy, contestable claims, with no knockdown arguments or silver bullets to be had. No wonder so many people prefer to stick to the more comfortable realm of abstract principle.

Bayesianism: Beliefs should come with numbers, and the numbers should be updated with Bayes’s rule.

Fallibilism: Those numbers should never be 1 or 0 (certain truth or certain falsity, respectively).

Sean Carroll—who I’m picking on only because I have immense respect for him, and so linking to him makes clear I’m not targeting a straw man—explicitly endorses both ideas right next to each other. And he’s not alone. Rather, the ideas form an important part of the Bay Area “rationalist” worldview. However, as I’ll explain, they’re inconsistent with each other. Applying Bayes’s rule requires that you be 100% certain in some things. Moreover, the inconsistency is not easy to avoid. If you like both ideas—as I do—you probably have some serious thinking to do.

Let’s start with Bayesianism. Bayesianism’s most basic commitment is that mathematical probabilities are a good model for rational beliefs. Instead of thinking of the attitudes you can take towards a claim as yes/no/maybe—you believe it, you disbelieve it, or neither—we can think of them as coming in degrees, where the more confident you are of something, the higher the probability you assign it, with 1 as the maximum and 0 as the minimum.1 Another kind of attitude Bayesianism recognizes is a conditional degree of belief. For example, if somebody is tossing a die, my degree of belief that it will land showing 6 is 1/6: there are six possible outcomes, each of which is equally likely. But my conditional degree of belief that it will land showing 6, given that it lands showing an even number, is different. Now, there are just three even faces (2, 4, 6) and each is equally likely, so the chance it lands on six, given that it lands on an even number, is 1/3. We express these beliefs in language with words like “if”. When I say “if my toddler has a nap this afternoon, she probably won’t get a good night’s sleep” I’m expressing a high degree of belief that she won’t get a good night’s sleep, on the condition that she naps in the afternoon. None of this is uncontroversial, and you can find plenty of philosophers and psychologists who already want to get off the bus.2 But I don’t; I like Bayesianism, and will take it for granted in what follows.

The second component of Bayesianism is a rule—Bayes’s rule—for how your degrees of belief should change as you gain new evidence. If this is old hat to you, feel free to skip to “Bayes’s Rule Demands Certainty” below. But if it’s not familiar—perhaps you’ve heard of Bayes’s rule or Bayesian this or that, but have never actually engaged with the equation itself—read on. Ultimately, the explanation for why Bayes’s rule requires certainty will require getting a little bit granular, so we need to know what the rule is and why it works, before we can see why it’s incompatible with fallibilism.

Bayes’s rule says that your new probability in a hypothesis H, after you’ve gotten some new evidence E, should be determined by three things: your old probability in H, your old probability in E, and your old conditional probability in E, given H. This latter value is often called the “likelihood”. When the likelihood is high, the evidence is just what the hypothesis predicted, and that counts in favor of the hypothesis. When it’s low, the evidence is not what the hypothesis predicted, and that counts against it.

More specifically, Bayes’s rule says your new probability in H after you’ve learned E should equal your old probability in H, multiplied by the likelihood, divided by your old probability in E. That’s all displayed in the following equation. If equations scare you, be not afraid! I’m about to explain why the equation makes sense:

I think the best way to understand this equation is to think about how values on the right could increase or decrease, and to see that they have the impact they intuitively should have on the value on the left. Why does Pr(H) appear on top on the right? Because our old opinions shouldn’t be completely washed away by new evidence; the more likely a hypothesis was before the evidence comes in, the more likely it is afterwards. Why does the likelihood, Pr(E|H) appear on top on the right? Because it counts in favor of a hypothesis when it said we should expect to see some evidence—when Pr(E|H) is high—and then we do in fact get that evidence. And it should count against a hypothesis when it said we should expect not to see some evidence—when Pr(E|H) is low—but we get that evidence after all; as the likelihood increases or decreases, so does the new probability for H. Dividing by the old probability of E is a bit less intuitive, but I think the best way to see it is to think of it as reflecting the power of unexpected evidence. When your old probability for E was high—when E was the evidence you expected to see—then you’re dividing by something close to 1. (Remember that all these values are between 0 and 1.) And just as anything divided by 1 is itself, anything divided by something close to 1 is close to itself. So when Pr(E) is high, we can basically ignore it—the new probability for H will be determined mainly by the values on the top. But when the old probability for E was low, then it can make an important difference. The interesting case is when the old probability for H was low, but so was the old probability for E, while the probability for E|H was high.

Imagine the crazy guy in Times Square—the one who spends all day talking about the illuminati, lizard people, and the like—telling you that you’ll inherit a private island in the Caribbean. Initially, you think the chance that he’s a reliable source of information is very low.

And you also think it’s extremely unlikely that you’ll inherit an island in the Caribbean. But you think that the chance that you’ll inherit an island in the Caribbean, if he’s a reliable source of information, is a good deal higher. And then, lo and behold, the next day you get a letter from the executor of the will of Lord Percival Wadsworth III informing you that you are now the owner of the island of Saint Aurelia. Intuitively, you should now think this guy might in fact be a reliable source of information after all (you should also start worrying about lizard people…) Dividing by Pr(E) is what makes this possible. Let’s rewrite the equation above, subbing in “Reliable” for H, and “low” “low” and “high” for Pr(H), Pr(E), and Pr(E|H), respectively:

Now, the fact that “low” appears on the bottom on the right of the equation allows it to cancel out the “low” that appears in the numerator—which represents your initial skepticism—letting the probability that the apparently crazy guy is in fact reliable end up high.

Bayes’s Rule Demands Certainty

That was our whirlwind tour of Bayesianism. What about fallibilism? I already mentioned Carroll, who puts it as follows:

The other big idea is that your degree of belief in an idea should never go all the way to either zero or one. It’s never absolutely impossible to gather a certain bit of data, no matter what the truth is—even the most rigorous scientific experiment is prone to errors, and most of our daily data-collecting is far from rigorous. That’s why science never “proves” anything; we just increase our credences in certain ideas until they are almost (but never exactly) 100%.

It’s compelling. And not new: a standard narrative in epistemology is that Descartes tried to rebuild knowledge on a foundation of absolute certainties, but failed. Modern epistemology has learned to live with our pervasive fallibility.

So what’s the problem?

The problem is that it’s built into Bayes’s rule that the new evidence is learned with certainty. Initially I described it as a rule for how to update our beliefs when we get new evidence. But what it is to get new evidence? As it turns out, the rule requires that getting new evidence amounts to becoming newly certain in the truth of that evidence. To see what the rule says about our belief in a piece of evidence once we’ve learned it, we can just treat the evidence as if it were the hypothesis in the rule. That is, we can just substitute “E” for “H” in the equation. After all, it’s an equation that’s supposed to tell us what our new probability should be for any old hypothesis H when we learn any new evidence E. Well, what should our new probability for E be, when we learn E? Here’s what the rule looks like when we just replace the H’s with E’s:

You’ll notice that the old probability for E occurs on both the top and the bottom on the right side of the equation. They cancel out, so we’re left with:

The new probability for E is the old conditional probability of E, given E. But Pr(E|E) is 1—of course, if E is true, then E is true. This means that the new probability for E is 1. What this reveals is that built into Bayes’ rule is the presupposition that getting new evidence means becoming certain in the truth of that new evidence.

I’m being a bit cute in calling this a dirty secret. It’s not a secret. It’s been known for a long time, and philosophers who like both Bayesianism and fallibilism have worried about it. But there are no easy fixes. Richard Jeffrey famously proposed a generalization of Bayes’ rule that allows for evidence to be learned, but not with certainty. He imagined a case where you catch a glimpse of a piece of cloth in poor lighting, and can’t quite tell whether it’s blue or green. Perhaps initially you thought it was equally likely to be either, but after seeing it, you now think it’s 80/20 green versus blue. He developed a mathematical framework that would let you learn this new 80/20 probability—which would then have further ramifications for the rest of your beliefs—rather than learning any evidence proposition with certainty. Without getting into the details, I think it’s fair to say that while Jeffrey’s updating rule is of theoretical interest, it’s not really a practical, general alternative to Bayes’s rule.

Take one of the best known contemporary uses of Bayesian probability: election forecasting. People like Nate Silver build elaborate Bayesian models of elections, public opinion, and polling. His models treat polls as data—when a new pollster releases a result, his model henceforth treats it as certain that the pollster did in fact obtain that result.3 That has further ramifications for what the model predicts about public opinion, which has still further ramifications for what it will predict about elections. It’s very hard to see how such models could run using Jeffrey’s rule; instead of learning that some poll had a particular outcome, what would they get to learn? Some new set of probabilities for all the different outcomes the poll might have had instead? This would both face problems of arbitrariness—which new set of probabilities should be used? There are infinitely many options—as well as, I strongly suspect, computational intractability: if for every poll, the model had to keep track not just of the result, but of probabilities assigned to all possible results, and for all further computations to be sensitive to all these probabilities, that’s a lot more information to process. It’s no accident that basically no practicing statisticians use Jeffrey’s rule in their models.4 Moreover, I suspect the same factors that make it cumbersome for explicit, intentional use also mean that it’s not what’s going on “under the hood” in automatic, unconscious cognition.

Still, claims about what results pollsters obtained don’t seem like great candidates for infallible certainty. Data falsification, or even just typographical errors, are possible. So where does this leave us? What can be salvaged of fallibilism, if we also think that Bayes’ rule captures deep truths about how we do and should learn from evidence?

At this point I’ll be coy, and say merely that I don’t think there’s an easy answer, and that I won’t try to provide a hard answer in this post. I do have academic work—both previously published and in progress—that treats these questions at greater length, but I don’t get all that far there either. Suffice to say, I feel the pull of both Bayesianism and fallibilism, and I think it’s a worthwhile but non-trivial project to find attractive versions of them that can be jointly maintained. Are you a Bayesian fallibilist? How do you think we should reconcile the tension above? Let me know in the comments.

Ever since a famous paper by Larry Tribe, legal theorists and philosophers have worried about what to say about cases like the following:

Blue Bus: Someone is hit by a bus. No identifying information about the bus is observed—nobody sees the license plate, tire tracks, a driver, or anything like that. However, it’s known that 80% of the buses that operate on the street where the victim was hit are owned by Blue Bus Company. This is the only relevant information on which we can judge how likely it is that Blue Bus Company is responsible for the injury.

The standard of proof in civil cases in the US is “preponderance of evidence”. This is usually characterized as meaning greater than 50% probability—if someone can show that their claim is more likely to be true than false, then they’ve met the preponderance of evidence standard. And if that's what preponderance of evidence means, then the case that Blue Bus Company should be held liable for the victim’s injury looks like a slam dunk; given the paucity of evidence, it’s 80% likely that they’re responsible, and 80 is greater than 50. But most people who hear the case think there’s something wrong—unjust, unfair—with finding liability on the basis of this sort of “bare statistical evidence”. So there has arisen a kind of cottage industry of trying to explain how we should understand the civil burden of proof, so as to be able to say that it’s not met in a case like Blue Bus.

It's not just civil cases that present this sort of challenge. In the US, the standard of proof in criminal cases is “beyond a reasonable doubt.” Suppose that can be understood probabilistically—“beyond a reasonable doubt” means >n% likely, for some n close to 100. (95? 99?) Here’s an abridged version of an example presented by Charles Nesson that raises the same basic issues as Blue Bus:

Prison Yard: 100 prisoners are exercising in the yard. 99 of them get together to attack the guard. One of them stays in the corner and doesn’t participate. This is all observed by another guard on a faraway tower, who’s unable to specify anything about the one prisoner who didn’t participate in the attack. No other evidence exists to help us distinguish the 1 prisoner who didn’t participate in the attack from the 99 who did.

Here, it’s plausible that for each prisoner, given our evidence, there’s a 99% chance they participated in the attack. So can we find all 100 of them guilty beyond a reasonable doubt? If not, would things change if we upped the number to 1000 prisoners, of whom only one didn’t participate? (Now, for each of them, the chance they’re responsible is 99.9%) 10,000? If you want to keep answering “no”, no matter how big the numbers get, then you’re probably committed to the idea that we need some way of understanding “beyond a reasonable doubt” that’s more fancy than the simple picture where it just means "above some probabilistic threshold."

It might seem a bit odd to worry about cases like the ones above. They're can seem artificial and stylized; creatures of philosophers’ thought experiments rather than pressing real-life questions. As it turns out, fact patterns that look a lot like Blue Bus aren’t all that unrealistic. Here’s a typical case. Somebody could prove they got mesiothelioma—which is typically caused by asbestos exposure—and that as part of his job for many years he used an arc grinder on brake drums, which often contain asbestos. He could prove the brake drums were manufactured by a specific company. While he couldn't prove that the specific brake drums he worked with contained asbestos—these cases are always brought many years after the fact, as mesothelioma takes a long time to develop—he could prove that the overwhelming majority of the brake drums manufactured by that company at the time did contain asbestos. Should he have been able to recover damages? (In fact, the he was able to recover, as were many others in similar situations.)1

Moreover, these cases can be seen as a special case of the more general question of whether statistical evidence ought to be treated with suspicion, as somehow different from other sorts of evidence. While cases where the main evidence at issue is statistical present a particularly pure version of this challenge, it comes up in other ones as well. In one case, a pair of brothers were charged with smuggling opium in the Minneapolis area. Should the prosecution have been able to mention—not as their only evidence, but as one argument among many—that the brothers were of Hmong descent, and that 95% of the opium smuggling cases in the area involved people of Hmong decent? (They weren't allowed to mention it.)

If evidence for a claim is just any collection of facts that raises the probability of the claim, then it's hard to deny that these facts are evidence—far from dispositive, but still relevant—in favor of their guilt. This is easiest to see if you imagine the impact of learning that they're not of Hmong descent. Even if they were at the scene of the crime, and there's some physical evidence linking them to it, if 95% of the opium smuggling cases involve Hmong people and these are a couple of Kenyans, that's some reason to think that they were just in the wrong place at the wrong time. But it's a general principle of probability that if learning E (they're not Hmong) would raise the probability of H (they're innocent), then learning not-E (they are Hmong) must lower the probability of H (ie, must make it more likely that they're guilty).

As it turns out, I like the simple picture that evidence in the law should be understood in terms of probabilistic relevance, and burdens of proof should be understood in terms of probabilistic thresholds.2 So I think we should get out of the business of trying to concoct explanations for what’s wrong, unjust, or unfair with findings of liability—civil or criminal—based in whole or in part on statistical evidence. I recognize that may sound like a big lift. For now, I want to discuss a more abstract question of methodology. The reason is that I think the surprising claims I just made can be supported by independently attractive approaches to thinking about what the law should be.

Casuistry vs. Policy Engineering

Here's one common way of thinking about questions like the ones we've discussed so far. We could neutrally call it the method of cases, or slightly more pejoratively (but still accurately!) the method of casuistry. A casuist considers starts with intuitive judgments about how various cases—hypothetical or actual—ought to be resolved, and treats these as data. They then try to construct abstract legal principles that best explain these data. They may be willing to reject data—to revise their initial judgment about how some case ought to be resolved, if that's the only way to hold on to an otherwise attractive general legal principle—but there's a strong presumption against doing so. This is the method most writers in the literature on statistical evidence implicitly adopt.

Contrast this with a different method, whose motivating analogy I heard from my co-author on this substack, Matt Wansley. It goes as follows:

Engineering : Natural Science :: Law : Social Science

Engineers take general principles and theories from natural science, and then apply them to figure out how best to fulfill various goals, e.g., building sturdy bridges. Similarly, on this picture, lawmakers should take insights from social science and apply them in the service of fulfilling various goals. What goals? Lots of them—good laws provide people with physical security (e.g., by deterring and incapacitating those would harm them), make possible various forms of cooperation and coordination (e.g., through contracts), and much more. Let’s call this approach policy engineering.

Before getting into how they help us think about statistical evidence, I want to say a bit about what each method has going for it.

In defense of policy engineering, it’s a way of capturing the extremely natural idea that institutions like laws are merely instrumentally valuable. (This was a theme in Matt's post on the capitalist alignment problem.) Why is it good to live under the rule of law? Not because being subject to laws is intrinsically good. Rather, because there are various other goods—things like life, liberty, and the pursuit of happiness—that we need the rule of law to secure. As David Hume memorably argued, the point of institutions like property rights is that, absent them, nobody would be willing to engage in long term projects like agriculture; if the reward for tilling, planting, and harvesting is that bandits steal your crops, why do all that work in the first place? But of course, a society where nobody is willing to invest in the future would be immeasurably poorer in all sorts of respects. So we shouldn’t think of property rights as good in themselves; rather, we should think of them as a necessary piece of social technology for enabling people to engage in valuable long-term projects.

What does this have to do with questions about standards of proof and statistical evidence? It suggests that the right way to approach them is to ask what values we want the institutions of criminal and civil trials to achieve and to then use our best social science to figure out which ways of structuring those institutions would achieve the best tradeoffs of the relevant values. It’s not obvious why that process needs to involve anything like the casuistic method of consulting our intuitions about what it would take to do justice in individual cases (hypothetical or actual). To return to the analogy to engineering, abstract intuitions about what sorts of bridge designs would work shouldn't be given much weight in civil engineering--better to reason scientifically from what we know about the properties of various materials to conclusions about what sorts of bridges should be built. If we come up with building designs that don't intuitively seem like they should work, so what? Scientific physics trumps folk physics.

But taking this analogy and running with it is a recipe for unconstrained hubris. If you’ve read your Scott, you’ll probably suspect that a policy engineer trying to reason from first principles about the best way to structure this or that institution—some version of which already exists, and has evolved over time in response to real-world pressures—and given free rein to remake the institution according to his ideal, will tend to make a hash of things. Preventing overly ambitious policy engineering is, I suggest, the proper function of casuistic methods in the law.

Here's a picture of how judges in the common law tradition (ideally) behave. They are trying to do policy engineering, but with a large dose of epistemic humility. As policy engineers, they try to construe the law in such a way that it has good results; so that the law enables cooperation, prevents harm, and the like. But because they're epistemically humble, they give a lot of weight to the fact that similarly situated judges over many generations have found some rule to be workable. If there's a longstanding precedent a judge has the option to overturn, he'll think hard before doing so. (Recall Chesterton's fence.) So rather than treating each case as an occasion to do policy engineering from scratch, they instead assume that the body of case law they encounter—in both its general principles, and in how those principles have been applied to particular cases, is reasonable by default. As a result, they prefer to limit their attempts at policy engineering to cases where precedent doesn't speak, or where there are conflicting precedents, and so overturning some precedent is inevitable. This respect for precedent ends up looking a lot like casuistry, in that it treats lawmaking as largely an exercise in extending the law in a way that treats prior decisions as (relatively) fixed points, rather than unconstrained armchair reasoning about how to achieve socially desirable results.

This picture about the function and value of casuistic methods does suggest some differences from how casuistic reasoning about the law is applied in much academic literature, however. Specifically, it justifies caring a lot more about how laws fit with actual precedents than about how they fit with intuitive judgments about hypothetical cases. Why? Because actual precedents have had the opportunity to be tested. If the holding in an actual case leads to absurd results, sooner or later those results will be brought to light in some later case, and the precedent will likely be overturned. If a precedent is longstanding, that suggests it hasn't led to such absurd results. By contrast, judgments about hypothetical cases carry no such guarantee. Some of the principles floated in the literature on statistical evidence, if adopted by judges and made into law, would likely have radically revisionary consequences for the use of DNA evidence, effectively barring it from playing a significant role in criminal prosecutions. Here, while our abstract intuitions about justice may push us in a certain direction, if those intuitions haven't yet been put into law and tested, we have no guarantee that they can integrate with a well-functioning legal system that achieves the sorts of aims policy engineers should want. Returning to the analogy to engineering, we already saw that abstract intuitions about sound design should get little weight. But if some design has actually been used to build bridges, and those bridges have stood for a long time without falling down, then we have much more reason to trust the design. In fact, engineering often precedes science; the Wright brothers achieved powered flight well before anyone had the necessary theoretical understanding of aerodynamics to comprehend how they managed it.

Where does this leave us? Both the policy engineering and casuistic perspectives on lawmaking get something right. The policy engineer is right that institutions like trials and standards of proof are best understood as social technologies whose function is to enable us to enjoy various goods. And the casuist is right that judges should generally not attempt to reason from first principles about which designs for such institutions would best secure those goods, but should instead have a healthy respect for precedent.

Back to Statistical Evidence

What does any of this have to do with statistical evidence? In a nutshell, I'll argue that a naive policy engineering approach provides no reason to see statistical evidence as in any way suspect. And this verdict doesn't change when we move from naive to sophisticated policy engineering by adding in respect for precedent, because precedents concerning statistical evidence don't speak with anything like a consistent voice.

Why does a naive policy engineering approach to thinking about standards of proof and admissible evidence favor welcoming statistical evidence? Because just about any aims you'd want the legal system to achieve are furthered by making it more accurate—more likely to find people guilty/liable when they really did the crime/tort, and less likely when they didn't—and considering statistical evidence makes the legal system more accurate. It's standard to distinguish four purposes of criminal punishment: rehabilitation, retribution, incapacitation, and deterrence. Set aside deterrence for now—we’ll come back to it shortly. Each of those other aims is clearly better served by a more accurate legal system. Innocent people don't need rehabilitation, retribution, or incapacitation, and guilty people do. Civil law, in addition to aiming at deterrence, aims at restitution. And people who don’t cause damages shouldn’t be paying restitution, while people who do, should—accuracy is clearly extremely important.

Deterrence is a bit more complicated. Some writers have argued that considering statistical evidence does not serve the aim of deterrence, in either the criminal or the civil context. The basic worry is that potential wrongdoers generally don't have any control over what statistical evidence suggests about their likelihood of having committed a crime. Let's go back to the blue bus case. Suppose Blue bus company operates 80% of the buses in town, and Grey bus company operates 20% of the buses. If Grey bus company is considering whether to take some action that makes accidents more likely, they'll know that whatever they do, the statistical evidence will still tell against their being responsible for accidents. So a legal system that considers statistical evidence might seem to frustrate the aim of deterring Grey bus company from causing accidents. After all, if they cause an accident—and if there's only statistical evidence to go on—it will be their competitor who ends up paying the price. One can imagine situations where knowledge that statistical evidence will be used creates perverse incentives.

While there's much to say here, I'll boldly claim that these problems largely disappear if we make the reasonable assumption that potential wrongdoers are almost never highly confident, ex ante, that only statistical evidence will be available if they commit a crime. The way to see this is to realize that the same objections we can raise to considering statistical evidence can also be raised about other forms of evidence that we’d never think of excluding from trials, such as evidence about motive or opportunity. If I have a rich uncle who's made me his sole heir, I have a motive to kill him. Even if someone else kills him, I'll still be a potential suspect, due to my motive; in that sense, I can't control what the motive evidence says about me, so we might worry that considering motives doesn’t help with deterrence. If the motive evidence will tell against me whether I kill him or not, I may as well go ahead and kill him, right? This argument is clearly unpersuasive, but it’s really not any better than the argument against considering statistical evidence. Allowing the legal system to consider evidence about motives is clearly a good idea, and doesn’t undermine the aim of deterrence. People with strong motives to kill are exactly the sorts of people we should be extra concerned with deterring—they're the ones more likely to kill—so knowing that their motives will mean they'll face extra scrutiny creates stronger reasons for them to avoid creating any further incriminating evidence against themselves, of the sort that they'd create by actually murdering their potential benefactors. To the extent that this means people without motives are deterred less, it's a reasonable tradeoff; it's more important to deter the people who are more likely, ex ante, to commit the crime in the first place. The same reasoning applies when we're considering people who are more likely to commit crimes because of statistical evidence against them, rather than motive evidence.3

More generally, from the policy engineering perspective, it’s hard to see how statistical evidence is importantly different from other kinds of evidence. Yes, if we’re willing to hold people liable—whether civilly or criminally—on the basis of statistical evidence, we’ll occasionally make mistakes. In Prison Yard, the mistake is very salient—one of those 100 people would be wrongly convicted if we decided that 99% probability was enough to meet the threshold for conviction. But the same basic problem arises for eyewitness evidence, motive and opportunity evidence, and any other fallible means of determining whether somebody really did the crimes or torts they’re accused of. Considerations about the badness of wrongful convictions support setting the threshold for beyond reasonable doubt high; it’s much harder to see how they could support restricting what kinds of evidence can be brought to bear in meeting that threshold.

That's my case that if you take the flatfooted policy engineering approach to law, you'll be happy to treat statistical evidence like any other kind of evidence. How do things change if we adopt the method of casuistry? I admit that the cases I started with—Blue Bus, and Prison Yard—seem like cases where finding people liable on the basis of statistical evidence is unfair. Likewise, it seems unfair that just because other Hmong people are disproportionately involved in opium smuggling in Minneapolis, an individual Hmong person in Minneapolis should come under extra scrutiny.

If there were a long and rich tradition of treating statistical evidence with suspicion in the law, then accepting the arguments I've been offering would amount to a radical revision, and a cautious casuist would be loathe to do so. But there is, in fact, no such clear tradition; as the seemingly contradictory cases I’ve already mentioned in this post suggest, it’s very hard to extract general principles from precedent here.

Here’s a perhaps cynical hypothesis about what’s going on. In cases where limiting the use of statistical evidence would seriously frustrate the aims of the law—e.g., where it would make it impossible for lots of people injured by asbestos to recover damages—courts have tended to allow the use of statistical evidence, as good policy engineers would. But in cases where accepting limits on statistical evidence seems much less likely to have far-reaching ramifications, courts have been happy to indulge the intuition that doing so is unfair. E.g., in the case of the alleged Minneapolis Hmong opium smugglers, it’s highly unlikely that prosecutors would have been able to clear the reasonable doubt threshold if, but only if, they were allowed to offer statistical evidence about the involvement of Hmong people in the opium trade. Even while the majority of opium smugglers were of Hmong descent, the vast majority of people of Hmong descent were not opium smugglers, so in any individual case, lots of non-statistical evidence would have been required for conviction, and it’s unlikely that the statistical evidence would have often been the difference-maker. But I suspect that if cases like Prison Yard somehow became extremely common—if there was a large group of guilty people that could plausibly only be convicted by taking statistical evidence seriously, with error rates comparable to what we accept with other sorts of evidence (eyewitnesses are notoriously fallible!)—then we’d eventually reconcile ourselves to letting statistical evidence suffice for finding criminal liability.

Perhaps I’m wrong about this. But even if so, I hope the perspective urged in this post may change some minds about what kinds of arguments would have to be made to justify treating statistical evidence as distinctively deficient. It’s not enough to appeal to the intuition that the use of statistical evidence (whether on its own, or as part of a cumulative case) is unfair. Rather, a statistical evidence skeptic should identify some goals we hope institutions like trials will achieve, and show how they’re frustrated by treating statistical evidence just like other kinds of evidence. I haven’t yet seen any persuasive arguments to that effect, and my suspicion is that there are none to be found.

Paperclips

In 2003, the philosopher Nick Bostrom argued that artificial intelligence might become dangerous even if it wasn’t designed with malicious intent.[1] He offered a thought experiment to illustrate the point. Imagine, he said, that we design an AI to manufacture paperclips. If that AI becomes powerful enough, it could start “transforming first all of earth and then increasing portions of space into paperclip manufacturing facilities.”

In today’s debates about AI policy, the worry that Bostrom identified is called the AI alignment problem. We might design an AI with a goal that isn’t fully aligned with our values, and that misalignment might cause harm. It’s useful to build paperclips. But there are many other things that we wouldn’t want to sacrifice to build more paperclips.

The AI alignment problem is hard to solve in part because it’s hard to specify all of our values. None of the leading ethical theories—consequentialism, deontology, contractualism—seem to capture all of our moral intuitions. And even if we were confident that one of these theories was right, it might not generate prescriptions that are clear and exhaustive enough for an AI to follow. So if we develop a powerful AI, we may not be able to fully specify its goals or the constraints under which it should operate. We may need to be vigilant to prevent subtly misaligned AIs from causing harm.

I think the AI alignment problem is important. But in this post, I won’t try to grapple with it. Instead, I want to argue that the AI alignment problem is just a special case of the larger alignment problem built into the structure of capitalism.

Windshield Wipers

In 1989, a professor from the United States took a trip to the Soviet Union and wrote a report for The New York Times. He started with this observation:

“Travelers driving through the Soviet Union can easily distinguish foreign tourists' cars from those of Soviet citizens. The latter don't have windshield wipers. The former do—but not for long.”[2]

Wiper blades, he explained, were often stolen. And once stolen, they were hard to replace. Why?

In the Soviet economy, production was organized by politics. Central planners had to decide which goods would be produced, how many would be produced, and what resources would be allocated to producing them.

In theory, one advantage of organizing production this way is value alignment. The planners could prioritize the production of the material goods that they believed were most important for human wellbeing. This is roughly the idea captured in the second half of Marx’s slogan: “to each according to their needs.”

But suppose that you were a government planner. How would you figure out what consumers needed? You might be able to estimate demand for big-ticket items from population statistics. You might decide, for example, that the average household needs two beds, one refrigerator, and one car. But how many of those cars’ windshield wipers would need to be replaced each year? What about brakes, headlights, and timing belts? And how would you calculate the right amount of resources to devote to producing these replacement parts at the expense of producing more bread or more jeans?

This is the socialist calculation problem. Even Trotsky understood it would be maddeningly hard to solve. He wrote: “The innumerable living participants in the economy, state and private, collective and individual, must serve notice of their needs and of their relative strength not only through the statistical determinations of plan commissions but by the direct pressure of supply and demand.”[3]

Unfortunately, Trotsky didn’t get his way. For decades, the Soviet Union tried to organize production without the direct pressure of supply and demand. And it made life miserable. The planners proved unable to reliably predict what people needed. Consumers routinely faced shortages for basic goods and had to wait in lines. The planners also struggled to police the quality of production. Consumer goods manufactured in the Soviet Union were generally inferior to goods manufactured abroad. And a massive black market emerged to satisfy demands that the planned economy wasn’t satisfying.

Centralized planning can’t be blamed for all the problems of the Soviet Union. A planned economy doesn’t require secret police and gulags. But it can be blamed for the lack of windshield wipers. It’s not that Soviet auto factories couldn’t manufacture wiper blades. The planners just didn’t have the information or incentives to provide enough blades to replace ones that broke or got stolen.

Today we don’t worry much that our windshield wipers will be stolen. There are auto parts stores that are eager to sell us replacements. The managers who run these auto parts stores aren’t more benevolent or more clairvoyant than Soviet planners. But they don’t have to be.

In a market economy, production is organized by profit. We don’t task any person or institution with deciding what consumers need. Instead, we have agreed on a set of rules that facilitate decentralized production. We protect private property, so people can use the resources they own as they see fit. We enforce contracts, so we can specialize and realize gains from trade. We allow general incorporation, so anyone can start a business. We permit open entry, so businesses can move into new markets and introduce new products. And we make it easy to raise capital, so companies can undertake projects that won’t pay off for years.

There are, of course, exceptions to each of these rules. Property can be taken by eminent domain. A contract to sell narcotics won’t be enforced. Entrepreneurs who want to incorporate a bank need a government charter. Businesses that want to start selling electricity or airplanes need regulatory approval. And companies that want to raise capital from retail investors need to disclose certain information to the public. But for the most part, if you identify an opportunity to profit, you can take it.

Decentralized production works because prices solve the calculation problem. As Hayek explained, the information that society needs to organize production is dispersed across the population.[4] Individual consumers know best what they need and want at any given moment. In a market economy, consumers reveal what they want by the prices they are willing to pay when they make purchases. Businesses use the information about aggregate consumer preferences that prices convey to guide their production decisions.

This way of organizing production has important advantages. A market economy can adapt to change quickly. Businesses aren’t beholden to a Five-Year Plan. If a streak of unexpectedly bad weather causes more wiper blades to break, consumers will buy more replacement blades, and prices will rise. Auto parts companies will respond by manufacturing more blades. A market economy also isn’t dependent on a small number of fallible decisionmakers. If an incumbent business—or even a whole industry—isn’t satisfying consumers, an entrepreneur can start a new company and compete with the incumbents.[5]

In the long run, the most important advantage of a market economy is its effect on innovation. Governments play a critical role in innovation by funding basic research. Indeed, the Soviets excelled at math and science—they made important discoveries we are still building on today.[6] But science itself doesn’t raise living standards. New technologies must be turned into products. And that process is too risky for many government officials. Ask yourself whether you heard more about the Obama administration’s failed investment in Solyndra or its successful investment in Tesla.

In a market economy, venture capitalists will fund the commercialization of new technologies. VCs are willing to make high-risk bets on startups because when they pay off, they can generate exponential returns. And some of those bets deliver socially valuable technologies. That’s how we got search engines, mRNA vaccines, and AI.

So there is a lot to be said for an economy that lets consumers’ willingness to pay—communicated through prices—guide production. Part of the appeal of capitalism is that its default rules aren’t paternalistic. Consumers decide what they want themselves. But we should never forget that consumer willingness to pay is not quite the same thing as human flourishing. One of the problems of capitalism is that they are subtly misaligned.

OxyContin

OxyContin is the brand name for oxycodone, a powerful opioid. In the 1990s, Purdue Pharma convinced the FDA to approve OxyContin for use as a pain medication. Purdue aggressively marketed the drug by downplaying its potential for addiction. And the company kept promoting the drug despite increasing evidence that it was addictive. Over the next two decades, hundreds of thousands of people in the United States died from opioid overdose.

Purdue and some of its executives plead guilty to crimes. The company is now bankrupt. In March, Purdue and its wealthy owners—the Sackler family—agreed to a new settlement in which they will pay around $7 billion. It’s easy to see this story as a group of rich villains committing crimes in an otherwise healthy society.

But it wouldn’t have been possible for Purdue to cause this much harm alone. They needed doctors who were willing to prescribe the drug despite the mounting evidence of its risks. They needed pharmacies that were willing to fill those prescriptions even when the same patients came back for years. And they even needed McKinsey, arguably the country’s most prestigious management consulting firm, to develop the marketing campaign. It was McKinsey consultants who proposed that Purdue could improve sales of OxyContin by paying distributors a rebate every time a patient overdosed on a drug they sold.[7]

Many actors worked together to make the opioid crisis as deadly as it was. But they didn’t need to join a grand conspiracy. They just all had parallel incentives to sell a product that consumers were willing to pay for. The same market mechanisms that help organize the production of windshield wipers can organize the production of opioid addiction. Consumers’ willingness to pay, in this case, wasn’t aligned with broadly accepted social values.

It's not that we don’t realize the risks of misalignment. In fact, we are so wary of the risks of pharmaceuticals that businesses can only bring drugs to market after they persuade regulators—based on clinical data—that the drugs are safe and effective. But even well-crafted regulation can’t always overcome the power of market forces. When the primary FDA official responsible for the approval of OxyContin left the agency, he started working for Purdue.[8]

The rise of AI may make the misalignment between corporate goals and social values more consequential. Until recently, businesses have had to rely on warm-blooded human beings to set policies and implement them. In some cases, their consciences have served as a check on misalignment. But what happens when an AI replaces some of those people?

I’d rather live in a society with an alignment problem than a calculation problem. Market economies have a better track record of protecting political freedom and improving material conditions than planned economies. But when we make public policy, we should remember that we are operating within a fundamentally misaligned system. Capitalism is like an AI—useful but hard to control and dangerous.

[1] https://nickbostrom.com/ethics/ai.

[2] https://www.nytimes.com/1989/08/27/travel/behind-the-wheel-in-the-soviet-union.html.

[3] https://www.marxists.org/archive/trotsky/1932/10/sovecon.htm

[4] https://statisticaleconomics.org/wp-content/uploads/2013/03/the_use_of_knowledge_in_society_-_hayek.pdf.

[5] As Coase pointed out, much economic activity is organized by firms rather than by markets in part because there are benefits to central planning. So, put differently, the advantage of a market economy is that it allows competition among multiple central planners, whose decisions are guided by prices. https://onlinelibrary.wiley.com/doi/10.1111/j.1468-0335.1937.tb00002.x.

[6] https://en.wikipedia.org/wiki/Science_and_technology_in_the_Soviet_Union.

[7] https://www.nytimes.com/2020/11/27/business/mckinsey-purdue-oxycontin-opioids.html.

[8] https://www.newyorker.com/magazine/2017/10/30/the-family-that-built-an-empire-of-pain.

Subscribe now