Silas Abrahamsen defends schmeference welfarism: the view that things are good or bad for a subject to the degree that their schmeferences are satisfied. What is a schmeference? In his words:

[For] something to count as a preference, it must at least be something that a conscious agent is disposed to occurrently think about. So completely non-conscious systems can’t have preferences in the morally relevant sense. … You might object “oh, well per the definition of ‘preference’ they do.” Fine, then I’m a schmeference utilitarian, I guess.

By contrast, I’ll take preference to include the kind of functional desires that philosophical zombies would have, and preference welfarism to include zombies as welfare subjects.1

Why be a preference welfarist? The most intuitive argument, in Silas’ words:

On something like objective-list theories—to the extent that they differ from desire satisfactionism—my life can be bad (or worse than I judge) without me caring whatsoever, and without my being committed to care from my other preferences. Again, that just seems to fly in the face of what it means for a life to be good! To the extent my life is good (in the sense of welfare) it surely has to be good for me; yet if I don’t want it to be that way, all things considered, then in what sense is that good for me?

… This also seems to count against hedonist theories … if all that matters for welfare is what I’m aware of, we get that same disconnect from before: I would prefer a life with less pleasure and real friends over a life with more pleasure and pretend friends. Yet [given hedonism] that would be the wrong choice! It would be objectively worse for me to choose the former, even though that’s the life I would rather have. But I ask thee once more: Was the value of life created for man, or man for the value of life?

I think the linked article is great, but it does present something of a puzzle. Nowhere does the argument seem to depend on the subject being phenomenally conscious. In fact, Silas argues that the value of life is not limited to the experiences we have. So why think only schmeferences, not preferences, are valuable?

An analogy: imagine Chauvinist Chad, who believes both

1. Immense pain is bad for human beings, because it feels awful.

2. Torturing animals is fine, because animals aren’t human beings.

Chad clarifies that he’s not a hedonist, he’s a “chadonist”: he thinks that the only morally relevant kinds of pleasure and pain are those held by human beings. Silas and I both think Chad’s position is bonkers. If what makes pleasure and pain bad is the way they feel, then whether or not the subject is a human being is completely irrelevant. Chad, being a smug pain in the ass, replies that pleasure and pain are not made bad by the way they feel, only “cheasure” and “chain.” But again, this is ridiculous. The properties of cheasure and chain that make them welfare-relevant are shared by ordinary pleasure and pain. Chad has offered no argument as to why the human-being part is important, so his position is unjustified.

So, what makes shmeference welfarism different from chadonism?

One option might be to adopt a view like phenomenal intentionality: for a mental state to be about something, it must be grounded in some kind of experience. According to this view, a zombie doesn’t have genuine intentionality. Its “desires” aren’t really “about” anything, even if they functionally track certain features of the world. Now, I think that this view is unconvincing, even if you’re a phenomenal realist. To avoid getting too into the weeds, I’ll make a Silas-style point here: why should I care about “genuine” intentionality in the first place? There is, by stipulation, exactly one difference between me and the zombie: that I have phenomenal states and he doesn’t. You can say that both of us have genuine intentionality, or you can say that only I have genuine intentionality and the zombie has pseudo-intentionality. But you still have to argue why it is genuine intentionality that is morally important, and not pseudo-intentionality. Without that argument, we’re just playing with semantics.

Another option is to try to separate welfare goods from welfare subjects, and argue that one can possess certain welfare goods without being a welfare subject. Imagine, for instance, a view that says that the only genuine welfare subjects are corgis. A fluffy coat is therefore a welfare good. It is something which makes a corgi better off. A bearskin rug can also possess a fluffy coat, and therefore a welfare good, but the bearskin rug isn’t the kind of thing that can be made better or worse off. It is not in a position to enjoy the welfare good that it possesses.

There are two problems here. First, Chad could adopt much the same view. We would still have reason to be skeptical of Chad’s view, because he hasn’t explained why human-chauvinism is a good theory of welfare subjecthood. Under shmeference theory, we need an explanation of why it is that a zombie can’t be better or worse off. (By contrast, hedonism has an easy answer: by definition, a zombie is incapable of possessing welfare goods.)

Secondly, the distinction between welfare subject criteria and welfare goods is somewhat arbitrary. You could easily refactor one view to the other. For instance, you could say that being a corgi constitutes welfare subjecthood, and having a fluffy coat is a welfare good. Or you could say that having a fluffy coat while being a corgi is a welfare good, and a welfare subject is just an entity capable of possessing welfare goods. Similarly, you could say that schmeference theory holds preference satisfaction as a welfare good and phenomenally conscious beings as the only welfare subjects, or it could say that “schmeference satisfaction,” defined as the satisfaction of a preference held by a phenomenally conscious agent, is the only welfare good, and welfare subjects are just those capable of possessing this good. I don’t see any substantive difference between these two versions of the view.

One could also object on a reductio ad absurdum: if phenomenal consciousness isn’t part of the criteria, then there are bajillions of counterintuitive welfare subjects. Does the United States of America have preferences? Does a corporation? Does a thermostat? Do atoms? What are my grounds for thinking they don’t deserve equal treatment?

I would reply that yes, we probably do need a more substantive account of what a morally relevant preference is and why those criteria are important. I think the loosest sense of the word preference—simply something that can be modeled with a utility function—is likely inadequate. But to me, a brute appeal to phenomenal consciousness seems like replacing one puzzle with another. I’m happy to say that I’m uncertain; I’m substantially less happy to give a pseudo-explanation.

In conclusion, I think schmeference welfarism is an unstable position at the moment. You could adopt hedonism or some objective list theory where goods are more explicitly linked to experiential states. Or you could stick with the intuitive arguments for preference welfarism and try to offer a more foundational analysis of preferences, which may or may not end up involving phenomenal consciousness. But the present minimal form is underjustified.

Subscribe now

I met Matthew Adelstein, aka Bentham’s Bulldog, at Princeton in the spring. I asked him if, as a fellow effective altruist and utilitarian, the existence of God had any bearing on what I should do. He frowned, paused for a few seconds, and said: “Maybe you should try praying for stuff.”

I have basically ignored theism debates since.

That’s changed with Joe Schmid’s recent conversion to Catholicism. I didn’t know of Joe before I heard the news, but he’s another super-smart philosopher joining the ranks of super-smart Princeton converts. As I’m coming back for my senior year, it’s never been a greater time to study philosophy of religion. And Catholicism, unlike bare theism, is extremely action-relevant.

Thus, before I begin my investigations in earnest, I thought I might present why I am not currently a theist, by running down Matthew’s The Comprehensive Case for God Summarized Extremely Quickly.

Consider this a pre-registration of my current, somewhat muddled thoughts. As someone who does not (yet) study philosophy of religion, it would not surprise me if several of these arguments were bad. The hope is to get to a much more solid place with further study.

The “limitless mind” prior

By “God” I mean an uncaused being with limitless power, knowledge, and goodness. I think such a being has a high prior probability (probability before examining specific evidence), because this being is simply a mind without limits. … Given that minds are one of the fundamental things, a mind without limits is simple. It’s simply one of the fundamental elements of reality instantiated without limit.

Onid and Scott Alexander both object to this “simplicity via limitlessness” move: what seems simple to describe in words doesn’t translate rigorously to something like Kolmogorov complexity. I’m inclined to agree, but the problem with something like Kolmogorov complexity is that it’s defined relative to an interpreter. Thus, I could define the single symbol “✽” to stand for a 50,000-line program, whereas a simple hello-world might need to be encoded by thousands of bytes. We’d need to decide what the primitives of our language are, and starting from the fundamental elements of reality is not that bad of a bet.

If minds are one of the fundamental elements of reality, that is. But they aren’t! I’m a materialist about minds: I believe there is no extra, irreducibly phenomenal/subjective/what’s-it-like mind stuff outside of the complex of reactions, dispositions, and representations that come from ordinary physical processing. My mind, your mind, all of the minds we know of are about as far from fundamental as you can get! So God would either have to be (a) an entirely new kind of fundamental entity, or (b) some kind of gargantuan physical system. Neither of these are simple at all.

On materialism about minds, theism is done for. But excluding concerns about God, I’m only 70% sure of materialism. If materialism is false, Matthew’s suggested prior of 0.1% is not crazy to me. That works out to a prior probability of around 0.03%. That’s really not enough of a difference to discount the theistic arguments that are supposed to have crazy, 10^60 likelihood ratios, and if that were all I had to say, I would probably still be a theist.

The best arguments for theism

Nomological harmony

1. The laws line up with the things that exist. There might instead have been laws of the form “all Xs do A,” but no Xs. By default, it’s very surprising that the laws line up with the things that exist, just like by default it would be surprising if a random rulebook to a game lined up with the pieces of the game, unless this correspondence was designed.

Matthew’s wording here is easily misinterpreted. If there existed additional laws such as “all Xs do A”, but no Xs, we wouldn’t even be aware that these laws existed. So if this were all that nomological harmony amounted to, we would have no evidence that we were in a harmonious position at all! The actual claim, as far as I understand, is that it seems a priori that all of the laws could have been that form. That we have laws that govern what exists at all is the supposed harmony.

This argument is new to me, and I haven’t read the paper in question yet. But it’s worth noting this from the abstract:

After introducing and developing this problem, we canvass candidate solutions and identify some of their virtues and vices. Candidate solutions invoke the likes of a designer, axiarchic meta-laws, multiverses, essential causal powers, and Humean laws

So at first glance, it doesn’t seem like the authors even think this is an awesome argument for theism in particular!

I risk sounding like a fool by preregistering my thoughts, but the assumptions which would make this a grand coincidence seem pretty dubious to me. If you think about randomly drawing game rules and gamepieces out of a hat, then yes, the fact that they coincide is surprising. But why would you think that? If an electron were governed by completely different laws, such that it no longer exhibited wave-particle duality, attracted positively-charged particles, and orbited nuclei, would it even make sense to call it an electron? I’m not confident that there’s a deeper problem than “why isn’t the universe totally static and uninteresting,” which is what’s up next.

Fine-tuning

2. We have laws capable of giving rise to interesting structure. By default, almost all laws devolve to random chaos or simple and valueless patterns.

3. We have finely-tuned constants, so that they fall in a narrow range needed for the formation of complex structure.

“Fine-tuning” is sometimes used to refer just to the finely-tuned constants problem, but I’m going to use it to lump together all questions of the form “why does the universe have the right conditions for interesting complex structure to emerge?”

A standard atheist reply to this is that physical laws and constants are necessary, you can’t reason about their a priori likelihood like that. The naïve version of this is wrong because of the awesome evidence argument. Imagine that the Quran were written perfectly in the initial state of the Big Bang. We’d be nuts not to take this as evidence for Islam. But if the initial state of the universe is necessary in such a way as to outlaw probabilistic reasoning, then you’d have to say this would give you no evidence at all. This seems wrong.

My sense is that right now, fine-tuning is unexplained and weird given atheism. However, I believe Fine-Tuning is Everyone’s Problem.

Both Sides Brigade

Fine-Tuning is Everybody's Problem (Part One)

In Neil Sinhababu’s wonderful paper “Divine Fine-Tuning Vs. Electrons In Love,” he identifies these two premises as the basic elements of a fine-tuning argument…

Read more

3 months ago · 56 likes · 54 comments · Both Sides Brigade

This series by Both Sides Brigade is excellent, but in brief: the theistic argument is that the conditions for complex biological structure are very unlikely if they occurred by random chance, but if a designer desired beings with value, it’s quite likely they’d select these conditions. However, if a designer exists than immaterial minds exist, and therefore the designer could have paired any physical substrate with any kind of mind. Biological life was absolutely unnecessary for God to create beings of value. You have to try to argue some other reason why God would particularly value biological life—but given that God himself is supposed to be (a) entirely non-biological, and (b) lacking nothing, there doesn’t seem any clear reason here. Plus other counterarguments that BSB provides. It’s a great series.

So I’m happy to grant that fine-tuning is pretty weird, and we atheists lack a great explanation for it. However, I don’t think theists have a better explanation.

Consciousness

4. Some physical structures give rise to consciousness.

5. The structures that give rise to consciousness exist. If As are the only things that produce consciousness, but there are no As, then there’d be no consciousness. By default, if there’s no God, out of all the possible ingredients for consciousness, it’s pretty surprising that they happen to be ones that arise.

6. Consciousness pairs up harmoniously with the physical world. Most conceivable pairings would lead to nothing of much value.

As a type-A materialist, I just flat-out deny that there is any harmony or surprise to be explained here. In the psychophysical harmony paper, the authors explicitly concede that their argument does not apply to type-A materialism, though they shove it away in a footnote. Mental states are functional states, there are no bridging laws or a posteriori identities. Consequently, the path from “complex biological life exists” to “consciousness exists” is really short: mental states have useful causal powers and can be evolutionarily selected for. So there isn’t anything here independent of the fine-tuning question.

I do think psychophysical harmony is a super cool argument, but it’s a great argument for “theism or type-A materialism”, not theism simpliciter. That is, it’s psychophysical harmony and arguments like it that have squeezed out atheistic non-interactionist phenomenal realist views for me!

Anthropics

7. As your existence illustrates, there is a giant multiverse with absolutely infinitely many people. Put another way: your existence is N times likelier if there are N times more people, so you should think the number of people that exist is the very most that there could be—and this is likelier if God exists than if God doesn’t exist.

Good gravy.

The version of anthropics Matthew is advocating, called the “Self Indication Assumption”, has always seemed super sketchy to me. It seems ridiculously implausible that you can reason with near-certainty that infinitely many people exist from the armchair, making no empirical observations. Matthew has argued that various implausible things result from alternative views, but to be honest, they don’t seem radically more implausible than what I’ve just described.

Consequently, I expected someone to have a strong case for why the anthropic argument falls apart. I found none. This surprised me, and my credence in it went up.

Then I encountered an article critiquing the anthropic argument’s lack of a specified measure. I thought this was a decent argument but probably refutable. Instead, Matthew’s reply was handwavey and unrigorous. This surprised me, and my credence in it went down.

Then Matthew and I went back-and-forth about it a bit. He has not succeeded in conclusively persuading me to his view, but he has persuaded me that the meta-level evidence I’ve been considering (lack of math seeming sketchy, the approach seeming new, the absence of great arguments from smart people, etc.) is all that great, and I should actually put in the legwork to understand anthropics and do math and actually tackle the arguments on their merits. Bleh.

Chalk this up as the single best argument for theism so far, but one in which I don’t have a lot of confidence either way.

Knowledge

8. Humans possess knowledge of morality, mathematics, the modal facts, right priors, and other non-physical facts.

9. We’re not radically deceived in our most basic and fundamental beliefs about the world, the way we’d be if we were Boltzmann brains or brains in vats.

The main arguments here are of the form:

1. We know that X.

2. But if we were in a skeptical situation, we wouldn’t know that X, despite having the same intuitions and evidence.

3. Therefore, we need some explanation of why we aren’t in a skeptical situation.

But this just doesn’t seem like a valid inference to me. If #2 is true, I don’t see why Matthew believes #1—that we aren’t in a skeptical situation. I’d like to see a better argument for why we should think we actually do know anything about modal facts, other than “c’mon, it seems obvious.” I don’t mean to attack “c’mon, it seems obvious”—I think it’s often a strong argument—just that it gives you absolutely no evidential weight against skeptical situations. I’ve never been impressed by Moorean arguments either.

Now, I think you’re justified in assuming that you’re not in a skeptical situation as a foundational belief. In all the possible worlds where I am in a skeptical situation, then what I do now doesn’t really matter, as I’d have absolutely no reason to expect one set of consequences to follow from my action versus another. So all of the expected value of my actions, everything I care about, is in the worlds where I’m not in a skeptical situation. Therefore I can freely ignore those possibilities.

But if you take a view like this, then the fact that you reason as though you are not in a skeptical situation is not something that cries out for explanation.

The best arguments for atheism

Matthew already did this, so read that. There are probably more good arguments, but since I have never taken theism that seriously, I haven’t bothered to study them yet.

Miracles

Not that impressive.

Conclusion

To my knowledge, there aren’t really awesome substackers for atheism. (Sorry BSB!) By that I mean people who read a crapton of papers, study every objection, and make hundreds of effortposts. Scott Alexander is formally opting out. Now that Joe Schmid has converted, he’s censoring his old content.1

PhilStack religion discourse is also dominated by like, three brilliant people who went to college together.2 Yes, if three brilliant people believe something, that’s evidence in favor of it. But if all you do is talk to three brilliant people about what they believe, you don’t really learn if there are awesome counterarguments for it.

I think we would really benefit from an awesome substack atheist. Unfortunately, I just don’t think I can do a PhD’s worth of work on philosophy of religion while also trying to stop AI from killing all the humans, or all the humans from egregiously harming sentient AI. These are enormous catastrophes that demand as much time as I can give without going insane.

“I wish it need not have happened in my time,” said Frodo.

“So do I,” said Gandalf, “and so do all who live to see such times. But that is not for them to decide. All we have to decide is what to do with the time that is given us.”

—J.R.R. Tolkien

So I’ll be seeing what I can do to study this, and posting my thoughts along the way, but I just cannot dedicate the titanic effort these writers make to philosophy of religion without abdicating some serious moral responsibilities.

If I’m wrong, and if God’s a good dude, I hope he knows I’m trying my best to do what’s right.

Subscribe now

Last week, I wrote about a really sucky argument about language and infallibility about consciousness. The idea being that if someone makes a claim like “I am in pain,” they are either:

• Connecting what they feel right now to something else relationally, in which case they can be wrong about any of the relations, or

• Using “pain” to demonstrate the state they are in right now, in which case they are infallible tautologically.

A lot of realist commenters argued that this was a straightforward gap between what we know and what we can express in language. Our infallible knowledge is also ineffable.

The problem with this move is that it’s pretty important for phenomenal realism that some facts about the intrinsic nature of experience be expressible in language, or else pretty much all phenomenal realist philosophy is gibberish. Take a claim like:

C. “When I look at an apple, there is an intrinsic redness in my experience, beyond the functional state I am in.”

It is precisely the position of phenomenal realism that this sentence is true and tells us something important about metaphysics. And when an anti-realist suggests that they could be wrong about this judgment, they fall back to the claim that the intrinsic character of experience is not something you could possibly be wrong about. Sometimes they will concede that there are certain “details” of experience that one could be wrong about. But most are committed to the idea that there is some basic sense in which C is:

1. true,

2. metaphysically consequential, and

3. known infallibly by ordinary humans.

My question is how this could be the case if experiences really are ineffable. For instance, Silas Abrahamsen offers:

We have some immediate knowledge of our experiences--some sort of infallible cognitive access to them, maybe such that an experience is constituted by your paying attention to it or something like that.

So this “lowest layer” of your knowledge of experience is infallible. But when you then try to think about this knowledge of your experience, try to compare your experiences, etc., it is possible for you to make mistakes (as we commonly do).

This doesn’t help the phenomenal realist, because they are precisely in the business of making comparisons between their experiences and other phenomena, like functional states. If a scientist offers a purely functional explanation of pain, they have to be able to compare their experience to what the scientist describes, then claim that the scientist has left something out.

One line of argument might draw a line between our ability to describe the intrinsic and ineffable content of our experiences, and our ability to describe that those experiences are intrinsic and ineffable. An analogy might be drawn to procedural knowledge:

Clyde: I should be leading the charge; you don’t know how to bullseye a moving target.

Bonnie: Of course I do! Watch. [She shoots the feather off of a passing rider’s hat.]

Clyde: Okay, then explain to me how you did it.

Bonnie: I just aim and shoot.

Clyde: That’s not an explanation. Everybody aims and shoots, even if they miss the target. I want you to tell me how you make the bullseye.

Bonnie: It’s instinct, I can’t describe it to you. I just do it.

Clyde: Well, if you can’t tell me how to make a bullseye, then you don’t really know how to make a bullseye, do you?

Bonnie: I do know how! It’s just not the kind of knowledge I can communicate in words.

Clyde: If that’s so, then how are you able to talk about what kind of knowledge it is? How are you talking to me about it right now? If it’s so mysterious and undefinable, how do you know you even have it at all?

Clyde is confusing “knowing how” in the declarative sense (being able to give a description of the steps to follow) with the procedural sense (having an ability). There is no contradiction in Bonnie having ineffable procedural knowledge, and having perfectly effable declarative knowledge about her procedural knowledge, including the fact that it’s ineffable. Thus, one might argue, there’s nothing strange about saying that I have ineffable “phenomenal knowledge” about my current experience, and also declarative knowledge about my phenomenal knowledge.

The problem with a bifurcation here is that we now have to choose where to extend the infallibility claim. The claim “I have infallible, ineffable phenomenal knowledge” is declarative, are the claims “my phenomenal knowledge is a different type of knowledge from declarative knowledge” and “the experiential qualities which I phenomenally know cannot be explained by materialism.” So in order to be infallible about these judgments, you would need to extend infallibility to declarative knowledge as well.

But then why think these declarative judgments are actually infallible? People do make mistakes when they pass declarative judgment about their own experiences. This is what radically deceptive experiences are: for instance, believing that your vision is in full, precise color, and then realizing you can’t identify what any of those colors are. The realist intuition that you know the way that red looks to you right now is very strong. But even if it were true, it would allow us to say very little about its metaphysical properties.

One could object that phenomenal realism doesn’t require infallibility in metaphysical theorizing. Of course it’s possible that we’re making some kind of philosophical mistake somewhere. But realist theories are the best account of the data we have. We have infallible knowledge of the explanandum, not the explanation.

But what work is infallibility doing in that argument? Compare:

1. We deny that an explanandum exists, violating phenomenal infallibility.

2. We grant that an explanandum exists and deny it all other relational and metaphysical properties, since these are declarative and therefore fallible. These properties would include intrinsic-ness, ineffability, and privacy.1

Is there really any important difference between the two? Would phenomenal realists be happy to concede #2 as a real possibility, but draw the line at #1? I doubt it. You can argue that #2 is ridiculous. But since #2 respects our infallible grasp of the explanandum’s non-declarative properties, infallibility can’t be a reason why it’s ridiculous. And that’s all I’m really trying to argue in this essay.

The best objection to my position here is going to have to find a way of averting the bifurcation between declarative statements about raw feels and the raw feels themselves, while preserving infallibility. This is a tall order, since there are obviously many declarative statements about raw feels that you do not want to extend infallibility to. For instance, if I want to say that my phenomenal states are identical to functional states, the realist can’t concede to my infallible self-report. They have to be able to tell a story about how it is that I am confused about the nature of my own experience, but they, the realists, cannot possibly be confused. This is why I am unconvinced that, for instance, phenomenal concepts can resolve the problem. Even in worlds in which phenomenal concepts did exist, they could obviously be misapplied, and in worlds without them, we would be unaware of what we were lacking.

Note

I will say that I have lower confidence in this final point than in the rest of the essay. Notably, what I am describing sounds an awful lot like a kind of strange loop:

What I mean by "strange loop" is — here goes a first stab, anyway — not a physical circuit but an abstract loop in which, in the series of stages that constitute the cycling-around, there is a shift from one level of abstraction (or structure) to another, which feels like an upwards movement in an hierarchy, and yet somehow the successive "upward" shifts turn out to give rise to a closed cycle. That is, despite one's sense of departing ever further from one's origin, one winds up, to one's shock, exactly where one had started out. In short, a strange loop is a paradoxical level-crossing feedback loop.

—Douglas Hofstadter, I Am a Strange Loop

The canonical example can be found in Gödel’s proof of the incompleteness theorem, in which a formula within a formal system can essentially express a proposition about its provability by the formal system.

Gödel’s theorem, much like quantum mechanics, is notoriously abused by amateur metaphysicians, and I am skeptical that there is more than a superficial resemblence here. (Hofstadter, by the way, is a materialist through and through.) Nonetheless, there is a possibility of some conceptual confusion here, and I have more to think about.

Subscribe now

Can I be mistaken about whether I am in pain?

Many people—including my younger self!—would say no. I can be mistaken about the external world, but I can never be mistaken about my own experiences. Thus, I may mistakenly believe that the top line is longer than the bottom:

But I cannot be mistaken when I say that the top line seems longer than the bottom to me.

This seems to be something special about conscious experience, a kind of infallibility we don’t possess anywhere else. There is a good argument that the kind of infallibility we do have, zombies would also have. And the kind of infallibility special to consciousness, we do not possess.

Return to pain. I think to myself: “Ouch, that hurts!” Could I be wrong?

Well, everyone will concede an obvious, trivial sense in which I could be wrong. I could be mistaken about the meaning of the words “that hurts.” We learn the meanings of the words “hurt” and “pain” by interactions with other people, and we learn to attach externally observable language-use to internal, subjective states. Maybe I systematically misinterpret those words, such that I think “that hurts!” when I am in a state that would cause other people to think “that tickles!” So when I say “that hurts!” I am not in any pain at all. But I believe that I am in pain, because I am mistaken about what pain is.

But this seems like mere linguistic trickery. Of course I can be mistaken about what labels apply to my state, but I can’t be mistaken about the state itself. We want to say that when I am in state A, I never think I am in state B. Even if I think something like “I am in state B,” all I’ve done is switch up the labels “A” and “B”, not the states A and B. What we want to say is that I don’t mistake the state I’m in for being another state.

But obviously “I am in this state” is true only because it is a tautology! If “this state” ostends whatever state I am in when I say it, then the claim translates to “I am in the state that I am in.” It’s totally vacuous, so of course it’s infallible. In order to say anything special about experience, we need to make a more substantive claim.1

Okay; what would such a claim amount to?

• “It feels like X", as in, “It feels like I’m being stabbed!” All claims of this form are linking how it feels in this very moment to how some other event feels. If your memory were wrong, then you would be wrong.

• “My knees both hurt; they feel the same way.” If they are literally subjectively indistinguishable, then you can’t be sure that both knees are hurting. If not, then you’re making a claim that they are analogous in some way. What way? “The way they feel.” And what is that? It obviously isn’t the way that you can tell that you have two knee pains and not just one; isn’t that a part of the “way they feel”? So what part of “the way they feel”? If you say “the feeling they have in common,” then again you are being circular: “my knees share the feeling they have in common.”2

• “It feels like how I remember X feeling!” What does it mean to remember X feeling a certain way? A couple options:

  • “I believe that X felt the same way that I feel now!” This is just reporting on what you believe; if you are not lying, then it’s true by definition.

  • “When I think about X, I feel the remnants of a sensation just like the one I am experiencing now!” If you are not currently thinking about X in that way, then this is a claim about the past, which is again as fallible as your memory. If you are currently thinking about X, then you are having one experience at that moment, which you separate into two parts: the sensation of pain, and the “ghost” of the sensation of X, which you claim to be analogous; this is the same problem as in the two-knees case.

  • “What I feel immediately brought X to mind!” Again, either it is a fallible claim about the past, or a claim “I am now thinking about X,” which is tautological.

• “It feels bad, I hate being in this state!” There are three interpretations of “hate:”

  • Dispositional: “I’m disposed to avoid being in this state.” You can be wrong about your own dispositions, since they are defined by counterfactuals.

  • Judgmental: “I judge this state to be bad.” In this case, I hate being in this state is performative—by saying so honestly, you make it so, and as a proposition it is therefore tautological.

  • Phenomenal: “This state has a feeling of badness.” But then we’ve gone right back to where we started with “I am in pain,” unless we can provide another explanation of what “feeling of badness” is and how we could have it infallibly and non-trivially.

I really cannot name a kind of infallibility other than “I know the way it feels to me right now,” and I really cannot come up with a way of translating it that is not vacuous!

Subscribe now

I’m a big fan of a lot of digital minds research, and I also think most of it is wrongheaded. When I tell non-readers this, they have a hard time understanding why, especially if they lack philosophical background. So here is my entire philosophy in ten minutes.

I. 5-minute philosophy of mind

I like to split theories of consciousness into three categories:

• Materialist: mental phenomena are physical phenomena.

• Interactionist: mental phenomena are non-physical and exert causal power over the physical. (Your brain sends information about your stubbed toe up into the mental world, and then a “feeling of pain” reaches back down into your brain and steers the neurons around.)

• Everything else: mental phenomena are non-physical, but do not directly exert causal power over the physical. Maybe mental phenomena are generated by the physical world, but don’t have any physical effects. Or maybe mental phenomena are the intrinsic nature of physical phenomena, separate from their functional, structural properties. Lacking a better name, I’m going to call this umbrella category “internalism.”1

I think internalism is false. First, it must appeal to a miraculous harmony: why is it that the “intrinsic” character of experience lines up so perfectly with the “extrinsic” reality of the world? Why do we experience pain when we receive a damaging stimulus, rather than a C# on a flute or a bright flash of green? Why are we able to talk about the “redness of red” if the character of red experience doesn’t have any causal impact on our speech? How could we have reason to believe that these intrinsic characters exist, if their existence has no impact on our psychological activity?

Second, there’s all this trouble with qualia. What are they, exactly? We can try to give a stipulative definition: qualia are the private, ineffable, intrinsic, directly apprehensible elements of experience. But there are deep theoretical problems with this stipulative definition, where they begin to look very strange indeed. Or you can try to give an ostensive definition: qualia are just “what it’s like to be” in some state. But then why couldn’t the referent be some functional state—what’s left of the name “qualia”? It seems this intrinsic stuff is very difficult to pin down, not just experimentally, but conceptually; it seems very unclear whether we even know what we are talking about!

So much for internalism. What about interactionism? It may inherit many of the same problems with qualia, but it might be able to escape the problems with harmony: since “mental stuff” steers organisms around, evolution applies optimization pressure to make sure our mental stuff is rightly harmonized (animals who like the feeling of pain don’t live long). But interactionism implies we will literally see this: neurons firing apparently spontaneously, particles moving around apparently exempt from the laws of physics, the fingerprints of a ghost manipulating the machine. And while neuroscience is far from finished, we have yet to observe anything like that going on.

That leaves materialism. Materialism appears to trade many technical objections for one massive question: how can a subjective experience be a physical phenomenon?

Here’s an obvious—and ridiculous—proposal. Drop someone in an MRI machine, prick their finger, and point at the region of grey matter that lights up: “There! That’s the experience of pain.” Surely this can’t be right. Say that region of grey matter weighs fifty grams: does that mean that the experience of pain weighs fifty grams? What would it even mean for an experience to have a weight? Somehow we’ve gone off track here.

Wittgenstein warned us against much the same trap in language. We correctly notice that we are interested in more than symbols in our language use: we don’t care about the sequence of letters “cat,” except insofar as it means cat. But then we mistakenly jump to searching for a separate entity—“the meaning,” as an object—whose presence would somehow “breathe life” into the inanimate symbols. And as he argued, no such thing exists: the meaning of a word is its use in a language. Meanings exist, they’re just not the kind of thing we originally thought they were.

Many philosophers and scientists correctly note that “the experience of red” is not the literal firing of neurons, just as “the meaning of the word ‘cat’” is not the sequence of letters c-a-t. But they then go on to claim that “the experience of red” is some independent entity, rather than a process, a functional state, or some other higher-level pattern. Even many materialists do this!

I believe this is a mistake. Experiences exist, just like meanings exist. But experience does not consist in private, intrinsic, ineffable qualia. This position may be called phenomenal eliminitavism, or the more popular illusionism.2

So, if experiences aren’t qualia, what are they? That’s now a question for empirical investigation. Here’s how I’d do so:

When you encounter a system, ask not: is it conscious? Or: how are its physical brain and its consciousness linked? Instead ask …

• What’s the system’s ontology—the objects and properties it recognizes and treats as primitive? Is the ontology physical? Computational? Conceptual? How does it align with ours? Where is it very fine-grained, and where does it simplify?

• What’s the system’s Umwelt—its representation of its environment and state? How do the objects from the ontology appear? How are they sensed? What are the relationships between them? Are there multiple modalities, and how do they interrelate? Is there a gradient of attention—what things “pop out” of the environment, and which things recede into the background?

• What are the system’s reactive dispositions? Does the system have equilibria, attractors, set points, saddle points, which its behavior revolves around? Does it seek certain stimuli and avoid others? How does it behave when it gets those, or when its seeking-behavior is frustrated?

• Does the system model itself? Does it track what has happened in the past? Which things does it remember, and which things does it ignore? Are there systems in place for representing and monitoring the system’s own state? What kind of information is used to construct those representations? What form do they take? Do they make simplifying assumptions? Which ones?

• How do these four interact? When the system is satisfied or off-balance or deprived of some resource, how do its perception and attention chance? When its perception shifts, how does that affect its reactions? How do these dynamics evolve over time? Do capacities ever expand or shrink? How does its memory influence its behavior?

…and so on.

II. 5-minute research directions

How does this change digital minds research?

In some ways, very little! Most of Anthropic’s empirical work—including emotion concepts, emergent introspective awareness, and the J-space—is still extremely useful under this paradigm: they do a great job uncovering Claude’s “psychology.” Even work from explicitly non-materialist researchers, like Andy Han & David Chalmers’ paper on a functional welfare axis, is incredibly valuable; I don’t think I would change a thing about the experimental design. Much of this guts-y, nuts-and-bolts work on the psychology of large language models is great.

Of course, these papers always come with their caveats, separating “functional” consciousness from really “phenomenal” consciousness (read: qualia) and suggesting only correlations or unclear metaphysical links. I don’t deeply object to this. I think they’re wrong about the metaphysics, but even a materialist should be cautious about slapping the label “welfare axis” on stuff, simply to avoid unsubstantiated empirical claims. There’s a ton of stuff involved in human welfare—somatic, intentional, homeostatic, volitional, etc.—and just slapping the label “welfare” on some vector without caveats would be a gross overclaim. So I don’t think your metaphysical stance really matters when you’re executing on one of these projects.

On the other hand, I think the metaphysics does change (a) which experiments you choose to run in the first place, and (b) ~all of your theoretical work.

The biggest distinction, to me, is top-down vs. bottom up.

In Eleos AI’s critique of Anthropic’s J-space paper, they put a lot of consideration into whether the J-space counts as a true global workspace for the purposes of Global Workspace Theory, a theory of consciousness in humans. On one level this is fair: we know the most about human consciousness, and Anthropic claimed a global workspace. But on another level, I don’t care whether the J-space is a “global workspace” in the GWT sense or not. I care about how the J-space works, and what it tells us about Claude.

As an example: they point out that one of the conditions for GWT is a set of modular subsystems which can operate in parallel. If you are a phenomenal realist, then that might be a huge deal, or not, depending on whether qualia attach to modularity.3 Your big questions are going to be:

• Does Claude have modular subsystems?

• Are modular subsystems necessary for qualia?

• How would modular subsystems affect phenomenal character?

However, if you’re an illusionist, there’s no extra question about qualia on top of the material, functional profile. You might still be interested in whether Claude has modular subsystems, or not. But your questions are going to be about what’s different between a system with modular subsystems and one without, and how that might make a difference to a material mind. You’d start thinking about things like:

• Do individual subsystems have coherent Umwelten, and what are they? Do they change substantially when considered as a whole?

• Where do the preferences & dispositions we’re interested in live? Does it make sense to talk of them as living in a single subsystem, or are they more like a kind of activity between multiple subsystems?

• Is the subject aware4 that they are composed of subsystems? Are any of the subsystems aware of this?

• Does modularity affect how many things the system can be aware of at once? Does unity require more serial processing? Are any signs of this visible in the subject’s behavior?

Phenomenal realists tend to think “top-down”: what’s the necessary “architectural stuff” for having qualia, and does the system have it? Illusionism encourages us to work “bottom-up:” how does the system work, what does it do/process/represent/seek? Very different research questions!

Some other differences:

• Phenomenal realists are more theoretically anchored to how consciousness works in humans, since “I only know that I have qualia.” Illusionists are more likely to be interested in very alien systems.

• Illusionist research is more welfare-relevant for ethical theories like Kantianism which extend to non-conscious moral patients.

• Illusionists don’t see introspective reports as particularly privileged. Compared to phenomenal realists, they will put less stock in introspective reports from humans, but more stock in LLM self-reports, as both are treated as heterophenomenological fictions.

• Phenomenal realist theory for digital welfare is heavily metaphysical: which systems have the qualia that count? Illusionist theory for digital welfare is heavily normative: which physical events should I care about?

• Qualitatively, there’s just a huge mindset shift between illusionism and phenomenal realism. I think very differently than I used to since becoming an illusionist, and I expect that to have significant implications for research.

Conclusion

Do we need a new digital minds org from an illusionist angle? Maybe, maybe not. Like I said, there’s a lot of overlap in the experiments and a lot of great people at existing orgs; I’d be super happy to work somewhere like Eleos. At the same time, there aren’t very many orgs doing empirical research on digital minds, and I see a lot of low-hanging research fruit, especially as we get better and better coding agents. Considering my next career moves, so very open to feedback!

Subscribe now

I think there is a 30% chance that materialism is false, and conscious experience consists in private, intrinsic, ineffable, non-physical qualia. If this is true, these qualia are probably forever beyond the reach of ordinary science. They would be impossible to measure and experiment on, and their nature would always be somewhat mysterious to us.

Nevertheless, I think there is a much higher chance—around 90%—that science will be able to tell us, definitively, whether materialism is true. Furthermore, I think you should agree even if you find illusionism incoherent and materialism ridiculous.

When I think through a phenomenal realist lens, the intrinsic, subjective nature of consciousness suffuses everything I do. On my desk next to me is a beautiful case of tarot cards, with borders in gleaming gold. There’s a way that that flash of gold in the light seems to me, the subjective character that it has, that feels irreducible to functional description.

It is just as obvious to me, from this lens, that that character is what I am writing about. Maybe I could conceive of a zombie writing the same words that I am, with no gleam of gold in their soul. But that’s not why I am writing these words. The feeling matters. I seek my friends because they make me feel happy. I confide in them when I feel sad. I think factory farming is monstrous because pain feels awful. And if you ask someone why they are a phenomenal realist, it always comes down to the way things feel.

Consider:

P1. Our subjective experience is necessary to causally explain our paradigmatic introspective testimony.

By this, I just mean that when a patient is screaming and begging for anesthetic, if you ask “what is causing them to do that?”, a necessary part of the explanation is “the horrible pain they’re feeling.” Or when a phenomenal realist talks about “the redness of red,” and you ask the same question, a necessary part of the explanation is “the way that red things subjectively look to them.”

Of course, there are edge cases where this isn’t true, like sleep-talking or crude mimicry. I’m merely claiming that the intuitive, paradigm cases of introspection that phenomenal realists frequently use as intuition pumps (seeing the color red, feeling pain, tasting sweetness, etc.) must invoke subjective experience as part of the causal explanation.

If P1 is true, then we can demonstrate whether materialism is true. Furthermore, I think we have strong reasons to believe P1 is true, beyond the intuitive pictures I have drawn. I’ll argue these one at a time.

I. P1 ⇒ demonstrability

1. P1: our subjective experience is a necessary part of any causal explanation of our paradigmatic introspective testimony.

2. Anti-material hypothesis (AH): our subjective experience is not a third-person observable phenomenon.

3. (1, 2): AH ⇒ our ordinary introspective testimony cannot be causally explained by third-person observable phenomena alone.

AH seems like one of the essential components of anti-materialism: my experience is fundamentally private to me. While anyone could crack open my skull and look at what’s happening in my brain, nobody can crack open my mind and know what my experience is, at least not with ordinary scientific methods.

4. P2: ordinary science can only find causal explanations in terms of third-person observable phenomena.

5. (3, 4): AH ⇒ ordinary science will never be able to causally explain our ordinary introspective testimony.

6. (5, contrapositive): If ordinary science eventually finds a sufficient causal explanation of our ordinary introspective testimony, then AH is false. ■

How do you prove materialism? Take David Chalmers, make sure he’s healthy, put him in a room with the best hyper-brain-scanning technology we have, and give him a Red Delicious to contemplate. We ask Dave to describe what he’s experiencing, in as much detail as possible, and we trace all of the physical observables: every neural firing, every hormone and neurotransmitter, every muscle twitch in his jaw, every dilation of his pupils. We put him in every experiment we like, with whatever colors or sounds or smells we like, and we record everything that happens, physically, when he speaks. If we are able to give a full causal explanation of why Dave says what he does, then either P1 is false or materialism is true.

The question of why Dave thinks there’s a hard problem of consciousness is called the meta-problem of consciousness. If we find this full causal explanation, then we have found a solution to the meta-problem. However, my claim in this essay is not that a solution to the meta-problem is all that’s needed to explain the hard problem. I argue only that if we do find a purely material solution to the meta-problem, then that will suffice to prove that materialism is true even if we don’t know the solution to the hard problem. Indeed, if the first explanation we find is purely in terms of microphysical and cellular events—a neuron fires here, an ion concentration shifts there—we will still be very confused about what is happening when we are conscious of things. (An analogy: it is easy to give a low-level description of what is happening in a large language model—a tensor multiplication here, a rectified linear unit there—and yet we still have “no idea” how LLMs work, because we do not understand the higher-level patterns, functions, states, and so on.)

I also don’t argue that we will in fact be able to find a solution to the meta-problem. In fact, my conclusion implies that if materialism is false, we will never be able to find such a solution. There is an alternative picture to the one I just offered: no matter what tests we run on Dave, we just can’t explain what causes him to say the things he does about consciousness and experience using only those third-person observations. We will see neurons firing apparently of their own accord, ion gradients shifting inexplicably, or causal chains tracing back to a source physical change that has no physical explanation. We will see something more baffling than the double-slit experiment, something which completely resists a physical reductionist explanation, and requires positing strongly emergent properties, unobservable interacting substances, or similar.

I am not lampooning this position. I think there is a 30% chance that if our science advances far enough, we will see just that, and I will concede that materialism is false. But this, I think, is what sensible anti-materialists should be committed to. The alternative view—rejecting P1—is wildly more implausible.

II. Why P1 is true

Recall:

P1. Our subjective experience is a necessary part of any causal explanation of our paradigmatic introspective testimony.

Consider one theory which rejects P1: epiphenomenalism, “the view that mental events are caused by physical events in the brain, but have no effects upon any physical events” (SEP). There are two glaring problems with epiphenomenalism. The first is self-stultification:

Suppose S is an epiphenomenalist, and that S utters “I am in terrible pain.” S is committed to the view that the pain does not cause the utterance. But then, it seems, S would be making the same utterance whether or not a pain were occurring. If this is so, then S’s testimonies about S’s own pains are worthless — both to us and to S. They cannot be taken to represent any knowledge about pains on S’s part (if S’s epiphenomenalist view is true). (SEP)

It’s easy to mistake this argument for the weak point that “zombie Dave would say the same things as real Dave.” That would be true even if you were not an epiphenomenalist, as long as you think zombies could exist in some alternative world with different psychophysical laws. What self-stultification argues is stronger: in this world, Dave’s testimony is worthless, since it is not at all caused by Dave’s actual pains. Moreover, self-stultification also applies to me: when Dave tells me to introspect on the redness of the apple in front of me, my belief in phenomenal qualia is not at all caused by whether I am having a real subjective experience. If the arguments for epiphenomenalism work on me, they work on me for no good reason, since their success cannot be attributed to epiphenomenalism being true. First-person self-stultification is much more damning than in the third person. I could imagine being persuaded of the Pythagorean theorem by a valid proof in a book that was generated by a random process: even though the proof appearing in the book cannot be attributed to the truth of the Pythagorean theorem, I have a way of “checking” whether the book is still accurate by assessing the validity of the proof. Now imagine my judgment on the Pythagorean theorem could not be influenced by the validity of the proof, and that I was aware of this. Would I have good reason to trust my judgment? I think not.

But if self-stultification seems bad, I think psychophysical harmony is even worse: “states of consciousness are related to each other, and to physical states, in strikingly harmonious ways—ways that seem extremely lucky, or involve many striking apparent coincidences” (Cutter & Crummett 2025).

A damaging stimulus causes physical state X, a certain biochemical or computational state of your brain. X causes you to avoid or eliminate the stimulus in the future. Conveniently, the psychophysical laws map X onto the experience of pain, an intrinsically bad experience which essentially provides one with reason to avoid or eliminate it. So the psychophysical laws correlate X with a phenomenal state whose essential normative role harmonizes with the functional role of X. And this isn’t a random fluke, but a pretty general truth about relevantly similar functional states and their associated hedonic states: we systematically avoid unpleasant experiences and pursue pleasant experiences. … What’s surprising is that the actual psychophysical laws map it onto an experience whose essential normative role harmonizes with this functional role. Since evolutionary forces cannot affect the psychophysical laws, it’s hard to see how an evolutionary explanation of this harmonious correspondence would even get off the ground. (Cutter & Crummett)

If you consider the possible space of phenomenal experiences, all the possible functional-mental state pairs, it is wildly antecedently unlikely that bad phenomenal feelings would systematically pair up with aversive functional states. Since, per epiphenomenalism, the bad phenomenal feelings have no physical effects, evolution could not possibly have arranged this, since the phenomenal state itself has no effect on fitness. Either God did it, or it’s the craziest coincidence in the history of the universe, and I don’t think God did it.

I think these two objections are absolutely damning for epiphenomenalism. However, epiphenomenalism isn’t the only position they apply to!

Consider a simple form of Russellian panpsychism: physics describes structural relations, but not the intrinsic nature of the relata; these intrinsic natures, or “quiddities”, constitute consciousness. The panpsychist can argue that consciousness is not epiphenomenal: if all functional relationships are grounded in phenomenal quiddities, it’s not as though you can subtract the quiddities and still have anything. However, the quiddities themselves are causally inert. A priori, the quiddities that ground Dave’s testimony of the redness of the apple could be the quiddities that constitute the experience of a blue sky, or those that constitute the sound of a loud violin, or random white noise. The fact that each functional state is grounded in a harmonious set of combining quiddities is therefore extremely surprising, and Dave’s testimony that there is a redness of the apple (instead of, say, a loudness) is not sensitive to potential disharmony. Thus, psychophysical harmony and self-stultification still apply.

Intuitively, you can think of these objections applying to any theory where a difference in qualitative experience would not cause a difference in testimony, where Dave would give the same answer about the “redness” of red whether it qualitatively appears red or loud. At the very least, they apply to any theory which denies P1. Since they are strong reasons to reject those theories, they provide strong reason to accept P1.

III. A bet against science

Some illusionists try to insist that illusionism really isn’t so radical; it only requires overturning some weird metaphysical baggage tacked on by philosophers, and doesn’t touch practice, ethics, or ordinary introspection. This insistence is reinforced by the many misconceptions its critics hold, attributing to it claims that illusionists never make. Despite the wrongness of these critics, I don’t think we should pretend that illusionism is intuitive. Illusionism is a weird and audacious position, one I find it hard to get my head around, and one I accept and defend only because I think it is best supported by the evidence.

Similarly, some anti-materialists say that their theories are fully compatible with and supportive of ordinary science, often in reaction to dumb criticisms conflating science and scientism. I sympathize. However, I think it would be similarly wrongheaded to pretend that anti-materialism implies no additional restrictions on science.

All sensible forms of anti-materialism are bets that ordinary science will fail. Not just that science will never touch some domain of interiority that is beyond its purview anyway; I mean fail to explain the causes behind third-person observable events in terms of other third-person observable events. That some time in the future, we will get our double-slit paradox, and never get the equivalent of a reductionist quantum mechanics. That our understanding of the universe by ordinary scientific means will terminate, once and for all, at the brain.

Anti-materialists should be willing to own their bet, just as illusionists should be willing to own theirs. If the evidence from introspection is as strong as they say, then it should be worth the cost. But whichever side you decide to take, I hope I’ve persuaded you that the final answer is coming, and it’s only a matter of time.

Subscribe now

Recently, I was contacted by the kind folks who run the incubator, asking if I’d apply to be a mentor. Happily, I was accepted! So, if you’d like to work on impactful AIxAnimals research, you should apply to the incubator by August 9th, and if you’d like to work with me, put me down as one of the mentors you’re interested in. I’ll attach my project description below.

The Umwelt paradigm for digital sentience

Commitment: 10hrs/week; Format: research & writing; Topic: Artificial sentience

A lot of research into digital welfare first asks whether the systems have “the right stuff” (recurrent processing, global workspaces, higher-order representations) to be conscious, then get mired in intractable debates between theories of consciousness and its consciousness. I’m excited for research on a more literal reading of “what it’s like”: drawing a detailed picture of a system’s Umwelt—environmental sensitivities, goal-seeking, adaptation, teleosemantics, the whole physical and functional picture. I think this is a promising research agenda for three reasons:

1. There are good reasons to believe that experiences do not have private, intrinsic, ineffable, directly apprehensible properties. (cf. Frankish, “Illusionism as a theory of consciousness”; Dennett, “Quining Qualia”; Frankish, “Quining Diet Qualia”) In that case, what most “consciousness research” is looking for does not exist, and is therefore unlikely to be fruitful. However, since experiences are still morally relevant (pain hurts!), what is therefore morally relevant is whatever remains: the functional, physical properties.

2. Even if private, intrinsic, etc. properties do exist, they are tightly bound up with our Umwelten. For instance, our experiences rely on our senses, our wants come from our selective pressures, etc. So Umwelt research is likely to be fruitful even in these other worlds.

3. Umwelt research is empirically tractable, whereas “consciousness research” is often (a) very confused about what it’s looking for, or (b) looking for something which by definition is inaccessible to science.

I’m happy to advise both theoretical and empirical research on this agenda. Here are some ambitious project ideas (the incubator project would likely be much more small-scoped than this!):

• Example of a theoretical project in this domain: a minimum viable product for hedonium

  • Even if your project is empirical, you should read the linked article to understand the underlying approach better! In the article, I apply the term “Umwelt“ restrictively to only one of the five bullet points; for projects, I mean it liberally, to apply to all of the five and related subjects within the same philosophical outlook.

  • Other ideas: the personal identity relation of an LLM, what is it like to be a role-player, how do autoregressive generators perceive time, etc.

• Examples of empirical projects: work along the lines of Anthropic’s functional emotions, Chalmers’ functional welfare, AISI’s machinic psychopharmacology

Desired product looks like a detailed blog post or preprint.

More details about what I’m looking for in applicants, logistical details, etc. here.

Subscribe now

Overmuser claims to have a simple proof that classical hedonic utilitarianism is false,1 because of a fundamental asymmetry between pleasure and suffering: you can always choose to forgo pleasure, but you cannot choose to endure sufficiently powerful suffering. Therefore, there is some amount of suffering which you can never trade for an arbitrary amount of pleasure. On the contrapositive of “ought implies can,” you therefore are not obligated to trade suffering for pleasure. Their conclusion is to endorse negative utilitarianism.

In this article, I demonstrate that this proof is quite terrible.

Overmuser begins by separating pain from suffering. This is a worthwhile move. People with pain asymbolia laugh when pricked, offer up their hands for more, and say things like “It hurts indeed, but I do not know what that really is.” Masochists enjoy pain. Dennett reports neutralizing pain by focusing attention on it very closely. These sensations are weird and unsettling, but they don’t seem to be self-evidently bad in the way ordinary pain experiences are. Some distinction is helpful.

Unfortunately, Overmuser’s definition is not very clear:

You can be in pain and ignore it. You can be in pain and push through it. You can even desire pain, if you are a masochist. But suffering you can’t ignore, let alone desire. … It’s a kind of overriding pain. … For now, let’s simply define suffering as the pain threshold (physical or psychological) at which it becomes impossible to ignore.

“Impossible to ignore” is frustratingly underspecified, and it becomes load-bearing for the argument. Consider Henri Alleg, a pro-Algerian journalist captured by the French. Alleg was electrocuted, waterboarded, burned, drugged, and hung. He was called a rat, a pig, and a “prize catch.” He was always tortured three stories belowground, or else others would be able to hear his screams. They threatened to do the same to his family. Alleg never gave up any information.

Did Henri Alleg suffer, or was he merely in pain? If Alleg did not suffer, then Overmuser’s version of negative utilitarianism does not say it was wrong to torture Alleg: “suffering is the only self-intimating disvalue, and alleviating suffering is necessarily the only distinctively moral obligation.” Thus, either it was morally permissible to torture Alleg, or negative utilitarianism is false.

Consequently, negative utilitarians must agree that what Alleg endured was suffering. However, Alleg’s case deflates Overmuser’s first assumption about suffering: “while you are experiencing it, the alleviation of suffering takes precedence over anything else”.

A person who suffers from extreme physical pain will be ready to end their own life to stop it, and likewise a person who suffers from extreme emotional pain, or chronic depression. They will not be able to ‘push through it,’ no matter how great the benefits. Their one and only, overarching priority will be to put an end to the suffering. [Therefore] the [hedonic] Utilitarian calculation described in the beginning of the present article is simply abolished. This is valid at the personal level.

From Alleg’s case, we can see this is false. Cases like Irena Sendler, tortured by the Gestapo for smuggling children out of the Ghetto, and Anne Askew, subjected to the rack for her unorthodox Protestantism, deliver the same verdict. Askew’s case is particularly damning. Though one could argue that Sendler and Alleg only acted to spare other people worse pain, Askew refused to lie and recant Protestantism, though this would inflict no further suffering on anybody. People can choose to ‘push through’ incredibly intense suffering.

Now on to the simple proof of Utilitarianism’s falsehood at the interpersonal level … Suppose a genie appeared and made you experience the worst conceivable pain to you. Suppose he then chose a random person and made them experience the most sublime conceivable pleasure to them. Suppose he then made your pain increase at an exponential rate. Finally, suppose he made the other person’s pleasure grow at a superexponential rate. The genie then says to you: “You can ask me to stop the exponentially growing pain that you are currently experiencing, but it will also prevent the other person from experiencing the superexponentially growing pleasure that they are having. You will both return to normal.”

The question is: do you have a moral obligation to refuse the genie’s offer?

The [hedonic] Utilitarian will say “yes: superexponential growth beats exponential growth”. However, it would be literally impossible for you to make such a decision. If ought implies can, then can’t implies there is no “ought”. You do not (you cannot) have a moral obligation to refuse the genie’s offer.

Notice that in the case of mere agonizing torture, of the kind Alleg, Sendler, and Askew endured, since they were able to push through the suffering, they might plausibly have a moral obligation to do so. There is no argument from ought implies can. The genie thought experiment therefore can’t establish that mere agonizing torture is bad, only (perhaps) that the worst concievable pain is bad.

To add insult to inconceivable injury, though, the genie example does not even support negative utilitarianism. Suppose that this “ought implies can” logic is valid. Then suppose that the genie inflicts exponential suffering on you, and superexponential suffering on the other person. The genie offers you the choice of alleviating your own pain or the other person’s pain. Are you morally obligated to alleviate the other person’s pain? According to negative utilitarianism, you do: superexponential growth of negative feeling beats exponential growth of negative feeling. However, it would be literally impossible for you to make such a decision. Consequently, you do not have a moral obligation to alleviate the other person’s pain. Therefore, negative utilitarianism is false.

Obviously, this is quite silly. No serious form of consequentialism says that, from the set of all actions, even physically impossible actions, we are obligated to take the action with the best consequence. Imagine this argument:

1. It is logically possible that there exist a button which instantly brings about extremely good consequences for ever and ever, better consequences than could be brought about by any other physically possible course of action.

2. Therefore, consequentialism says you ought to press this button.

3. However, it is impossible for anyone to press this button, since it does not exist.

4. Since ought implies can, consequentialism is false.

But no consequentialist believes in premise #2. Consequentialists only argue that, of the actions that are possible for you to take, you ought to take the one which will lead to the best consequences. “Ought implies can” is irrelevant here, since consequentialists must restrict their normativity to possible actions anyway.

So much for a moral proof. Overmuser does raise a suggestive naturalistic point, though:

People have the ability to postpone pleasure indefinitely. This is even a predictor of how successful they will be in life. The marshmallow experiment is so famous that it has become an element of pop culture. People can even consciously decide to stop experiencing pleasure while they are experiencing it, because they have something more important to do. This fact alone should be a very strong indicator that pleasure does not have the kind of self-intimating, foundational value that [hedonic] Utilitarians think it has. On the other hand, people do not have the ability to postpone the alleviation of suffering (provided it is intense enough). Suffering has an additional feature: absolute urgency. Absolute priority.

This is insufficient to establish the claim Overmuser wants, that “suffering is the only self-intimating disvalue.” But it does suggest an interesting asymmetry between pain and pleasure, which might question the foundation of hedonic utilitarianism if true.

Notice, however, that the asymmetry is qualified: people cannot postpone the alleviation of suffering “provided it is intense enough.” People might still might trade merely agonizing pain for much greater amounts of pleasure; the two are not necessarily incommensurable. Taken naïvely, this might suggest a kind of loss aversion curve, where one unit of pain is disvalued more heavily than one unit of pleasure. The problem with this is that the units of pleasure and pain are arbitrary; they are both scalar quantities. You can simply define one “dolor” to be the maximal amount one is willing to suffer for one “hedon.”2 The only fundamental asymmetry rests in the claim that:

1. There is some level of pain where it is impossible to choose anything other than its alleviation.

2. For any level of pleasure, it is possible to choose something that will stop the pleasure.

#1 seems plausible; our brains might just shut down at a certain level of agony. Is #2 plausible? The ordinary cases that Overmuser suggests (orgasms, tasty dinners, marshmallows) are pretty easy to refrain from. But maybe we’re just not thinking about the right kind of pleasures.

There are certainly cases where reward-seeking overrides some pretty awful experiences:

• Given a limited window to either eat or self-stimulate reward circuits, rats will starve themselves to near death. (The researchers cut off the experiment before any of them died.)

• A human patient with a neural implant reported erotic sensation whenever she self-stimulated. She continued doing this even as her stress hormones lose and she began to lose her sensory and verbal capabilities.

• Rats will self-administer cocaine even as their health collapses, to the point of death.

However, these are far from conclusive. Just like pain, pleasure can dissociate. There’s the “wanting” (pursuit of reward) and the “liking” (how nice it feels). Block dopamine release in rats, and they won’t be motivated to seek sugar, but they will show signs of enjoyment if it’s fed to them direction. This is liking, but no wanting. Conversely, you can give rats a lever with an absolutely disgusting concentration of salt, and sure enough, they learn to avoid it. But when you deplete the rats’ sodium for the first time in their lives, they race to the pull the lever, without having ever had a good salt taste experience. This is wanting, but no liking. However, contra Scott Alexander₂₀₁₀, it’s a lot harder to demonstrate this in humans, and human drug addiction is a pretty weird phenomenon we still don’t get yet.

My first conclusion is that it’s a pretty mixed bag at this point, and I would not be confident in saying that for any level of pain, there is no level of pleasure that would outweigh it. That’s going to depend on some pretty complex neuroscience we don’t yet have.

But my second conclusion is that I’m not sure how much “you cannot choose otherwise” matters at all. Suppose there were a chip implanted in our brains, such that whenever we experienced sufficiently intense suffering, we “froze,” unable to do anything to alleviate it. Would that make the suffering any less bad? I don’t think so. Similarly, imagine that we juiced up our prefrontal cortices to give us massively increased inhibitory powers over our motor reactions, and were able to (behaviorally) ignore the maximum amount of suffering we can experience. Again, would that make the experience itself any less bad? I don’t think materialists should think so, and phenomenal realists (which includes Overmuser) really shouldn’t think so. I’m just not convinced that what we can in practice choose says that much about what we value.

In conclusion, I think Overmuser’s “disproof” is pretty terrible and rests on a basic confusion about what most normative theories imply, while the remarks on pain/pleasure asymmetry are interesting but inconclusive. I think we are nowhere near having decisive, or even strong, reasons for negative utilitarianism.

Subscribe now

I mostly write from a philosophical perspective that I don’t entirely believe in.

This perspective is illusionism. The thesis that there are no private, intrinsic, ineffable, directly apprehensible mental properties. I put 60-70% credence in this belief. I’ve studied it well. I know all the standard arguments. I can debate any dualist and come out standing strong. I have no strong reasons not to be an illusionist, and many strong reasons to be one.

Deep down however, illusionism about mental properties is perplexing to me. Illusionism about free will or personal identity—striking and intuitive as they are—are not so difficult for me to apprehend. I have the sense that I have a stable persistent personal identity, sometimes a very powerful sense. But with a little study, I can see straight through those intuitions. I can perfectly comprehend why it is that I have them, what a world without them would look like, and how it is I can square my having those intuitions with them being false.

I can’t do the same thing with illusionism. Not yet. I have every argument in place but still there some part of me that is deeply confused. There some sense in which I can’t see through the illusion, when I can see through so many others. I don’t know, then, if I qualify as a true illusionist, in the way Parfit was a true illusionist about personal identity, or Caruso about free will. I can play a card trick here and there, but I haven’t gotten the craft.

So why do I spend all this time writing about it? Why not focus on those other theories that occupy the remaining 40% of probability mass? Why do I pretend to be a real illusionist?

It’s more fun.

Every other theory of consciousness stops just when it’s getting interesting. If consciousness is something private, intrinsic, ineffable, directly apprehensible, then beyond a sort of introspective accounting of what’s going on in me, there’s nothing I can really do to understand it any better. I could meditate or take drugs, but all that does is give me new experiences, not any deeper insight into their nature. And it’s not as though experiences (on this view) are something you could measure, verify, or explain. So you just… stop. You fold your hands. You wait for future science that can make sense of it, or you think that such a science is impossible. Dead end.

Phenomenal realism would be disappointing. But that doesn’t mean it’s false. I think we don’t have immortal souls. That sucks! It would be awesome to live forever, or at least for a very long time. But I don’t try to start believing in souls just because I’m bummed out by materialism. So what does it matter that things are interesting? Shouldn’t I just want to believe whatever is the case and not want to believe whatever is not the case?

Well, I’m not explaining why I think illusionism is likely to be true. That comes from argument alone. I’m explaining why I pretend to be an illusionist, or more accurately, why I act like an illusionist. And when choosing what to do, what theoretical avenues to explore, where to work, what tests to run—this is where interest plays a much greater role. Illusionism is interesting because you have nothing to offload your curiosity onto, no place where the magic happens and the universe stops being explicable. It forces a kind of discipline on you, to try and fit your picture of experience into what we know about the rest of the world. And it shows you, if it fails, exactly what it leaves out.

Because the longer you spend studying illusionism and working on it and taking it seriously, seriously acting like an illusionist, the more you discover that the conventional arguments against illusionism aren’t very compelling. This thing that you might have learned to latch onto in your philosophy class this subjective character, this what-is-it likeness, if it does exist, it is something stranger, much stranger, much less intuitive, and much less articulable than you might have thought. You might learn new things about science, about the nature of concepts, about the relationship between mind and machine, about how arguments work, definitions, semantics, language, understanding, belief, belief in understanding, belief and experience, experience and the brain.

Imagine that the universe really does contain magic black boxes that just do things inexplicably beyond the reach of ordinary science. Imagine that we were absolutely certain that these boxes existed. How do you learn what the box does and does not do, what it is and is not capable of, if it is something that can never be measured? For any phenomenon, you can always say: the black box does it. If you don’t understand how something could be, well, that’s the nature of the black box! They exist, they’re real, they’re out there! Having no good explanation is no argument against the black box, because it is the nature of the box that it is inexplicable. Quod erat demonstrandum.

But there are a great many things, if this universe is like ours, that turn out to be explicable without resorting to black boxes. There are legible laws of physics. There are complex and surprising phenomena, mysterious at first, which eventually fall to philosophical and scientific explanation. How do you learn what the black box can and cannot do? If you cannot study the black box directly, you study the negative space.1 You assume that the black box is not responsible. You look for other causes. You try to give an explanation—really, actually try; yes, for more than five minutes. You close in, inch by inch. And when you’ve fought, tooth and nail, and find that (a) a definite something remains, which (b) all your current explanations, and likely explanations, and ideas for explanations, cannot handle, you concede to the black box. You do not throw in the towel because the problem looks unsolvable.

Is that really so awful?

Subscribe now

Maybe I’m too easy to bait, but when I see a headline like “AI consciousness is physically impossible,” I’ve got to click through. I suppose I was expecting the same old lookup-table diatribe, rehashing Searle and Block, the usual spiel. What I found was a surprising and original argument that I also think is quite bad, but very instructive for some thorny epistemological & phil-science reasons. It’ll take us from explanations to evidence to not-even-wrong-ness, with stops at opium, IIT, conservation of evidence, and universe-scale lookup tables. Here we go!

I. Pseudo-explanations

This whole argument is about evidence, and what theory best explains our consciousness.

Consider the evidence that we possess regarding consciousness. I know that I am conscious, and neuroscience provides the best explanation of how I came to be: consciousness is produced by organisms with highly centralized nervous systems.

Unfortunately, the stated clause is not an explanation. It is a pseudo-explanation: something explanation-shaped which does not reduce the asker’s confusion.

Consider a classic pseudo-explanation, from Molière: opium puts people to sleep because of its dormitive power. This is a tautological pseudo-explanation. “Having dormitive power” is synonymous with “puts people to sleep,” so nothing has been explained. Suppose I instead offer: opium puts people to sleep because it comes from a red flower. This is a baffling pseudo-explanation. Without further details, the explainer (the flower’s redness) appears to be completely irrelevant to the explanandum (putting people to sleep). Finally, I try: opium puts people to sleep because it of its highly protonated tertiary amine. This is a radically incomplete pseudo-explanation. Unlike the other explanations, this one is a but-for cause: this amine is truly instrumental in sedation. If you are a highly specialized neurochemist, this does suffice as an explanation to you, but only because there is a massive amount of background knowledge that completes the explanation. In the 17th century, when Molière was writing, this would not have enlightened anyone, even if they knew what “highly protonated tertiary amine” meant. The “size of the question” has not been reduced, merely shifted to why highly protonated tertiary amines put people to sleep.

Now, let us consider “consciousness is produced by organisms with highly centralized nervous systems.” This is not a tautological pseudo-explanation. But it is either radically incomplete, or baffling.

Suppose I believe the hard problem of consciousness is non-conceptual, that there is a great mystery about how conscious experience could come from unconscious matter. It seems very strange and mysterious how a highly centralized nervous system suddenly “wakes up” and becomes conscious. Without further explanation, it might seem to me that the organization of a nervous system is completely irrelevant to the question of subjective experience, so the pseudo-explanation is therefore baffling. Or it might seem to me that the two must be linked, but I have no earthly idea how, so the pseudo-explanation is therefore radically incomplete. Neuroscientists have not, so far, succeeded in explaining the hard problem of consciousness.

On the other hand, suppose I believe that there is no deep hard problem of consciousness, only that we have a strong intuition that one exists. We still might be interested in explaining why experiences have the properties that they do: why burns hurt and handshakes don’t, why I become unaware of surrounding noise and hyper-aware of my conversations, and so on. Neuroscience suggests that in our cause, our nervous systems are extremely relevant to this question: they are an integral part of the causal chain of how we become aware of anything. However, this is still a pseudo-explanation. We can rephrase the claim “consciousness is produced by organisms with highly centralized nervous systems” into “consciousness is produced by systems which both (a) contain highly centralized information-processing systems, and (b) are made out of biological matter.”

• Claim (a) is nearly1 tautological: in order to taste rotten fruit as bitter and gross, we must be differentially sensitive to objects with different chemical composition, and we know that we make decisions on the basis of this differential sensitivity (we spit it out). But that’s what it means to be an “information-processing system”: sharing Shannon mutual information with the world and making use of that information to act. To the hard-problem denier, most of claim (a) is just a rephrasing of the well-known bare minimum for consciousness.

• Claim (b), on the other hand, is either baffling or radically incomplete. Why should it matter that a conscious system is made out of biological matter? Certainly, the conscious systems we know of are made out of biological matter—just as opium does come from a red flower! But no argument has been given for why this property is of any relevance. Can you point to any features of a biological substrate which make it crucial for consciousness? Can you point to any principled reason why a non-biological substrate couldn’t implement the same features? Many have tried, and we could discuss these arguments on their merits. Yakiimo, however, cites none of them, and leaves us only with a pseudo-explanation.

II. “Predictive success?”

Why do I spend all of this time harping on about explanations? Yakiimo recognizes that a merely inductive claim about consciousness wouldn’t justify substrate-dependence:

my argument [is] very different from the argument of someone who might have said prior to the invention of mechanical calculators … “we haven’t observed any cases of non-biological calculating systems to date, therefore, mechanical calculators are physically impossible.” There was never any scientific theory, let alone one with a track record of predictive success, that provided any reason to think that calculating systems could only be biological. There was no theoretical concept identified as necessary and sufficient for being a calculating system that entailed that non-biological systems could never calculate.

I’ve argued that there still is no such theoretical concept that is not baffling or radically incomplete, that “produced by organisms with highly centralized nervous systems” (the “CNS” proposal) is not a real explanation. There is no theoretical reason to think that a biological nervous system is necessary or sufficient for consciousness.

However, there remains the question of whether there is a scientific reason to think so, “with a track record of predictive success”. Returning to opium, I identified “puts people to sleep because of its highly protonated tertiary amine” as a radically incomplete explanation. However, it could still form the basis of an (incomplete) scientific theory with a track record of predictive success. We might observe that removing this amine, or decreasing its protonation, also removes opium’s dormitive power. Or that adding such amines to similar chemicals turns them into sedatives. In this way, you might acquire very strong scientific evidence, without coming much closer to a theoretical understanding. This is Yakiimo’s primary argument, that the scientific evidence favors CNS over functionalism.

How do you acquire such scientific evidence? By making predictions and measuring whether they turn out to be true. Crucially, your criterion for assessing your predictions cannot a priori depend on your particular theory. If, for instance, I predicted that consciousness is produced by recurrent processing, and also measured consciousness by the presence or absence of recurrent processing, then a priori my theory could never be falsified.

It’s then quite surprising to hear Yakiimo claim that CNS has a track record of predictive success. If you believe in the hard problem, then it is impossible to measure whether any system is conscious. If you do not believe in the hard problem, then testing for consciousness is like testing for life: a loose category of overlapping things that depends your on definitions, not a measurable quantity.2 What would predictive success even look like?

Here Yakiimo falls back on “observable behavioral indicators of consciousness“: social behavior, sensitivity to stimuli, etc. But then she immediately goes on to deny consciousness to systems that possess the same indicators.

Ned Block presents the following thought experiment: We could in principle provide every person in China with a two-way radio, and have them respond to input, interact with one another, and produce output in such a way as to exactly simulate functional states manifested by the human brain. But even though the resulting system of Chinese people interacting with one another by radio would exhibit the functional properties of a human brain, it would not itself be conscious (Block 279). So, a system’s exhibiting particular functional states is not sufficient to produce consciousness.

She denies this because… why? Vibes? It feels weird? That’s not how scientific evidence works. China brain exhibits all the same behaviors that she takes to be valid scientific evidence of consciousness in fish, that get to confirm her nervous-system theory. Therefore, if China brain were implemented, it would count as evidence for functionalism and evidence against CNS. You don’t get to pick and choose when your evidence counts.

Indeed, if we count behavioral indicators as evidence for consciousness, then this is a really strong argument against Yakiimo’s conclusion. For any behavioral indicator you like, with high probability we’ll be able to build a robot that exhibits it within a couple centuries. Therefore, with high probability we will acquire strong scientific evidence against CNS within the next couple centuries. Therefore, by conservation of expected evidence, we should apply the same doubt to CNS right now.

Consequently, Yakiimo should either:

• Reject behavioral indicators as evidence for consciousness on the grounds of China brain, and therefore reject that there is a track record of success for CNS.

• Accept behavioral indicators as evidence for consciousness, and therefore reject CNS since we expect to acquire strong scientific evidence against them.

III. Not even wrong?

Yakiimo’s most interesting challenge to functionalism is not that it makes wrong predictions, but that it fails to make any predictions at all:

The theory that a highly centralized nervous system is necessary and sufficient for consciousness makes those risky predictions—no in the case of oysters, yes in the case of fish … By contrast, it’s not clear what functionalist theories predict in those risky edge cases. For example, one functionalist theory of consciousness is the global workspace theory, which holds that a system is conscious if and only if it possesses a global workspace to which various mental representations compete to gain entry. Those that win out by attracting the spotlight of attention are integrated in the global workspace and then broadcast throughout various other modules of the brain. Do fish possess something that qualifies as an integrated global workspace with an attentional spotlight? It’s not clear, because it’s not clear exactly what criteria a cognitive system would have to meet to qualify as possessing a global workspace, and it’s not obvious how we could determine whether or not fish meet those criteria.

This is sharp and fascinating, and would be a great argument if it didn’t apply just as well to her position.

What is a highly centralized nervous system? How highly centralized? Are octopus centralized enough? Are Sirian Supersquids? Is an ant swarm, since every ant is biological? What if the ants are all neurally connected—is it one mind, or a million? What if the neurons are completely inactive? What if I just flicker them uniformly on and off, to the tune of “Take Me Out to the Ball Game?” What if I build a neural computer to do integral calculus—is it conscious? If so, what is it conscious of? If I take GPT-5 and replace each artificial neuron with a biological neuron,3 would it be conscious? GPT-3? GPT-2? Would it be conscious of the meaning of what it’s saying, or the mechanical sensations of talking, or neither, or both?

It’s not just that these are wacky edge cases and any particular verdict might seem absurd. Integrated Information Theory is a bad theory that gives bizarre verdicts, as Scott Aaronson demonstrated. But Scott had this to say about IIT:

In my opinion, the fact that Integrated Information Theory is wrong—demonstrably wrong, for reasons that go to its core—puts it in something like the top 2% of all mathematical theories of consciousness ever proposed. Almost all competing theories of consciousness, it seems to me, have been so vague, fluffy, and malleable that they can only aspire to wrongness.

The issue with “highly centralized nervous system” is that I think it’s just as fuzzy as “winner-take-all global workspace of representations.” Yakiimo’s theory specifies what the system is made out of, but only the barest sketch of how it’s organized, and no detail at all about what it’s doing. She continues to emphasize that her theory provides “necessary and sufficient” conditions for consciousness, but I’ve no idea how it would adjudicate any of the (very biological!) edge cases I’ve described above.

Now, you can join the illusionists and say that consciousness itself is vague, fluffy, and malleable. Or you can call out both GWT and CNS as much too fuzzy. I don’t think there’s reason to take CNS and not GWT.

The best objection here is that for any theory with the slightest degree of vagueness, you can always invent indeterminate edge cases. But at least CNS gives a verdict in existing edge cases. Oysters are real, so are fish, and we really have to decide how to treat them.

Again, if you’re broadly on Team Dennett and think consciousness is vague, you can bounce right out of this. If you’re not… I’m pretty uncertain how big of a deal this is. Here’s an analogy for the CNS-sympathizer: in some modally-lenient sense, an LLM could be replaced with a giant lookup table and be behaviorally identical. Someone—sometimes me!—always points out that the lookup table would require more entries than there are atoms in the universe. Such a thing would be physically impossible, it would collapse in on itself. Does that indicate that there’s no sense in which LLMs are reducible to lookup tables? I don’t think so. I ultimately don’t think the behavior-preserving swap is a very good argument, but I also don’t think “it’s physically impossible” is a very good argument unless your opponent gets sloppy with their modality.

Similarly, I’m not sure how much I’d care about whether the edge cases are extant or not. If you’re Yakiimo and think that judgments on these edge cases add up to a track record of successful predictions, maybe it matters, but I’ve just argued why I don’t think that counts. All my edge cases are nomologically possible, and it doesn’t seem like a theory of consciousness should become better or worse relative to whether I decide to build the neural computer in 200 years or not, if we know what we’d observe in either case. Conservation of expected evidence and all that. So I’m skeptical that this is much of a value-add for CNS.

IV. Fading qualia

Finally, Yakiimo argues that Chalmers’ “fading qualia” argument is weak. I agree, no complaints.

V. Conclusion

I think this argument is a step up from Block & Searle’s “it’s weird!” stuff, but substantially worse than substrate-dependence arguments from folks like Anil Seth, which actually exploit features of brain activity (like mortal computation) which today’s AI systems and successors like them will lack. Ultimately, it depends on a lot of claims about what the scientific evidence supports that don’t hold up to scrutiny, and ends up undermining itself with the choice of behavioral indicators as evidence.

Subscribe now

I. Göd

Gödel, Escher, Bach, by Douglas Hofstadter, is the greatest book ever written. For twenty years, nobody knew what the hell it was about, much to the frustration of Douglas Hofstadter:

When GEB found itself for a while on the bestseller list of The New York Times, the obligatory one-sentence summary printed underneath the title said the following, for several weeks running: “A scientist argues that reality is a system of interconnected braids.” After I protested vehemently about this utter hogwash, they finally substituted something a little better, just barely accurate enough to keep me from howling again.

Many people think the title tells it all: a book about a mathematician, an artist, and a musician. But the most casual look will show that these three individuals per se, august though they undeniably are, play but tiny roles in the book’s content. There’s no way the book is about those three people!

Well, then, how about describing GEB a “a book that shows how math, art and music are really all the same thing at their core?” Again, this is a million miles off … in bookstores, I have run across GEB gracing the shelves of many diverse sections, including not only math, general science, philosophy, and cognitive science (which are all fine), but also religion, the occult, and God knows what else. Why is it so hard to figure out what this book is about? … it must be in part that GEB delves, and not just superficially, into so many motley topics — fugues and canons, logic and truth, geometry, recursion, syntactic structures, the nature of meaning, Zen Buddhism, paradoxes, brain and mind, reductionism and holism, ant colonies, concepts and mental representations, translation, computers and their languages, DNA, proteins, the genetic code, artificial intelligence, creativity, consciousness and free will — sometimes even art and music, of all things! — that many people find it impossible to locate the core focus.

—Preface to the 20th anniversary edition of GEB

Eventually, Hofstadter had had enough. In 2007, he published I Am a Strange Loop: a straightforward, plain-language exposition of his thesis. Though there were plenty of charming anecdotes and clever metaphors, there were no five-part dialogues in the form of a fugue, no palindromic steganography, no lengthy digressions into mathematical curiosity, no great harmony between form and function.

Reader, it is awful.

I don’t even mean wrong or ill-argued. I don’t even mean a letdown from the highs of GEB. I mean I have tried to read it three times and put it down because it was so dull and uninspired. I have read machine learning proofs with more wit and ingenuity than that book. Really, truly, do not waste your time. Read Dennett or Wittgenstein or Putnam or Kripke or pretty much anything else from modern analytic philosophy.

What went wrong?

Well, for twenty years, nobody knew what GEB was about. Including Douglas Hofstadter.

Hofstadter wrote the book to try and answer: “What is a self, and how can a self come out of stuff that is as selfless as a stone or puddle?” But he was wrong in thinking that that was what the book was about. The answer to that question, for Hofstadter, is buried in another: how does form turn into meaning?

That’s the whole book. How do symbols in an axiomatic program express proofs about reality? How do l-i-s-t-s-o-f-l-e-t-t-e-r-s turning into beams of light turning into firings of neurons turn into propositions? How do strings of nucleotides express the form of an organism? How do notes form a song? How does syntax become semantics? How does this sentence refer to itself? How is it that “yields falsehood preceded by its quotation” yields falsehood when preceded by its quotation? How does a book about how a meaning can elude capture within the interpretive rules of its creation elude capture within the interpretive rules of its creation?

That is what makes Gödel, Escher, Bach special. That is why Hofstadter micromanaged every inch of it, right down to the typesetting. That is why his planning sketches look like this:

It is a fractal operating on all levels. It is the Tortoise all the way down. If you played The Stanley Parable back in 2013 and thought it was trippy and provocative, you haven’t seen shit.

I’m all for clarity of language in philosophy. But Gödel, Escher, Bach literally can’t be translated into ordinary analytic style without destroying its entire purpose. It would be like trying to make a perfect copy of a qubit in superposition, or translating Cette phrase en français est difficile à traduire en anglais, or describing an epistemically transformative experience. And its very inarticulability is yet another layer of its meaning, reflecting back to the incompleteness proof at the heart of the book.

I’m reminded of Michael Huemer citing the Bible as one of the chief reasons he’s not a Christian: he would expect a book written by the most beautiful and perfect being imaginable to “stand out somehow as the most remarkable of all books ever written. I would expect to find it the most illuminating and profound and insightful thing that I had ever read.” If the Bible were half as beautiful as Gödel, Escher, Bach, I would convert on the spot.

I know from the outside these words sound dumb and trite. That the messages seem obvious, or easy to articulate in ordinary English, or that “recursion,” “emergence,” and “Gödel” set off crackpot-slop-alarms in your head. But in my defense, I’ve just articulated at length exactly why the message of GEB sounds so much less powerful removed from its medium. I really can do nothing but beg you to read it, and give it a few chapters to let its resonance sink in. You won’t regret the experience—and you’ll come to understand why I Am a Strange Loop could never compete.

II. Zen

One afternoon a student said “Roshi, I don’t really understand what’s going on. I mean, we sit in zazen and we gassho to each other and everything, and Felicia got enlightened when the bottom fell out of her water-bucket, and Todd got enlightened when you popped him one with your staff, and people work on koans and get enlightened, but I’ve been doing this for two years now, and the koans don’t make any sense, and I don’t feel enlightened at all! Can you just tell me what’s going on?”

“Well you see,” Roshi replied, “for most people, and especially for most educated people like you and I, what we perceive and experience is heavily mediated, through language and concepts that are deeply ingrained in our ways of thinking and feeling. Our objective here is to induce in ourselves and in each other a psychological state that involves the unmediated experience of the world, because we believe that that state has certain desirable properties. It’s impossible in general to reach that state through any particular form or method, since forms and methods are themselves examples of the mediators that we are trying to avoid. So we employ a variety of ad hoc means, some linguistic like koans and some non-linguistic like zazen, in hopes that for any given student one or more of our methods will, in whatever way, engender the condition of non-mediated experience that is our goal. And since even thinking in terms of mediators and goals tends to reinforce our undesirable dependency on concepts, we actively discourage exactly this kind of analytical discourse.”

And the student was enlightened.

—David Chess, Broken Koans

III. Life

Some ideas feel either deep or extremely obvious. You’ve heard some trite truism your whole life, then one day an epiphany lands and you try to save it with words, and you realize the description is that truism. And then you go out and try to tell others what you saw, and you can’t reach past their bored nodding. Or even you yourself, looking back, wonder why you wrote such tired drivel with such excitement.

…For instance, “when you think about the future, the thing you are dealing with is your own imaginary image of the future, not the future itself”.

On the one hand: of course. You think I’m five and don’t know broadly how thinking works? You think I was mistakenly modeling my mind as doing time-traveling and also enclosing the entire universe within itself? No I wasn’t, and I don’t need your insight.

But on the other hand one does habitually think of the hazy region one conjures connected to the present as ‘the future’ not as ‘my image of the future’, so when this advice is applied to one’s thinking—when the future one has relied on and cowered before is seen to evaporate in a puff of realizing you were overly drawn into a fiction—it can feel like a revelation, because it really is news to how you think, just not how you think a rational agent thinks.

—Katja Grace, Deep and obvious points

IV. Consciousness

I am a tentative illusionist about consciousness. This is a relatively recent development. About six months ago, I wrote:

[Illusionism] is not internally contradictory. It’s possible in the zombie world. The question is whether we think it’s true. And I’m not convinced it is. In the personal identity and free will cases, I can understand how my gut feelings that I have those things could be squared with the illusion. But I don’t get how my feeling that I have phenomenal states, which in and of itself feels like a phenomenal state, could be squared with them being illusory. I don’t understand the mechanism, at all. Illusionists will claim that it is possible, that there is nothing fundamental about my confusion. But why should I believe them? I have tried, and it does not make sense to me.

—Yours truly, Why I am not (yet) an illusionist

I finished that essay with a vow to read more, think more, and see if I could make the illusion intelligible to myself. And gradually, it seems to be becoming moreso. Now, sure, I made some arguments for it in a somewhat novel frame. But when I try to articulate what changed for me, I pretty much end up paraphrasing Frankish and Dennett—the exact same writers that seemed so unpersuasive to me back then!

The illusionist viewpoint has permeated all of my work these days: on experience, on ethics, on “consciousness” as a phenomenon. It feels like a perceptual shift, swapping between black-and-blue and white-and-gold. And yet I don’t know how I would speedrun an earlier Jack to the same conclusion without just saying “read Frankish and Dennett over and over until you get it.” It can’t be the propositional content, exactly, that was missing. What was it?

V. Love

In the continental-analytic wars that occasionally rage on philstack, I’ve been pretty squarely on team analytic. I won’t argue what Michael Huemer, Noah McKay, and Matt Adelstein have already put forth. I will just reproduce Matt’s randomly selected passage from Judith Butler:

The Lacanian appropriation of Lévi-Strauss focuses on the prohibition against incest and the rule of exogamy in the reproduction of culture, where culture is understood primarily as a set of linguistic structures and significations. For Lacan, the Law which forbids the incestuous union between boy and mother initiates the structures of kinship, a series of highly regulated libidinal displacements that take place through language. Although the structures of language, collectively understood as the Symbolic, maintain an ontological integrity apart from the various speaking agents through whom they work, the Law reasserts and individuates itself within the terms of every infantile entrance into culture. Speech emerges only upon the condition of dissatisfaction, where dissatisfaction is instituted through incestuous prohibition; the original jouissance is lost through the primary repression that founds the subject.

Enough said.

I have no patience for obscurantism. I used to be the third-grader that used big words I didn’t really need to show off how clever and impressive I was. I don’t feel the need to pour years of my life into someone else’s vanity project. And for a while, that’s all continental philosophy seemed to me.

But love can bloom even on a battlefield. A year ago, I met a brilliant young women: a writer, a poet, and a student of literary translation. Naturally, by the end of the date I bought her a copy of Gödel, Escher, Bach; I suppose I was testing her, like some mathematically addled Penelope. She fell in love with the book. Shortly thereafter, we fell in love with each other.

And yet—gasp!—she’s a continental. Though she enjoys analytic philosophy a great deal, we bicker1 (amiably) about the other kind. I take the side that it’s vacuous nonsense which hides its lack of valid reasoning behind a wall of gibberish. She takes the side that (a) while very obscure writing is bad, it’s not really reason to dismiss the whole field, and (b) continental and analytic philosophy are trying to do different things, and it’s a mistake to judge one in terms of the other.

Which sounds reasonable at first… but a lot like the suspicious claim that dubious methods of reasoning just belong in their separate magisteria, conveniently shielded from criticism. No claim about how the world is is immune to reasoned argument; complaining about “Western modes of thought” frankly sounds like cope. (Note: I don’t think my partner believes any of this; my suspicion was always that this was hiding underneath all the obscurity in ways most people couldn’t really sort out.) And of course, my would claims overreach and exaggerate, and round and round we went.

But flash forward to just a few days ago, in a bookshop in London with a large philosophy department,and I skip over the analytic section entirely. I buy The Three-Body Problem on a promise to Harry Moss, a couple Buddhist sutras, and one more book far outside my ordinary reading. What was I looking for?

VI. Friedrich Nietzsche

I read Atlas Shrugged probably about a decade ago, and felt turned off by its promotion of selfishness as a moral ideal. I thought that was basically just being a jerk. After all, if there’s one thing the world doesn’t need (I thought) it’s more selfishness.

Then I talked to a friend who told me Atlas Shrugged had changed his life. That he’d been raised in a really strict family that had told him that ever enjoying himself was selfish and made him a bad person, that he had to be working at every moment to make his family and other people happy or else let them shame him to pieces. And the revelation that it was sometimes okay to consider your own happiness gave him the strength to stand up to them and turn his life around, while still keeping the basic human instinct of helping others when he wanted to and he felt they deserved it (as, indeed, do Rand characters).

—Scott Alexander, All Debates Are Bravery Debates

I never had that kind of family pressure growing up. All the pressure was self-imposed. Young me was a very anxious, scrupulous kid. I cared a lot about doing the right thing, and this was good: I adopted Effective Altruism and vegetarianism as soon as I could comprehend them, and I was never content with moral hypocrisy But I also beat myself up about my shortcomings and obsessed over every unimportant detail and really made myself miserable with my mental framing. I had a real Life Goals of Dead People problem. I’ve left that behind now, but man, I wonder if young me could’ve been spared a lot of pain if I’d given Rand a chance. Except, as an over-scrupulous person, I was the least likely to listen to Rand and most likely to be hypersensitive to the claims that Objectivists were greedy, callous assholes. This is the exact advice-reversal dynamic that Scott wrote about, and it persisted even after I grew out of my early neuroticism.

So when I recently rediscovered utilitarian philosopher Richard Yetter-Chappell’s Nietzchean Challenge to Effective Altruism and Scott Alexander’s Matt Yglesias Considered As the Nietzchean Superman, I was frankly stunned. I wouldn’t articulate it out loud, but I mentally associated Nietzche with gym-bro, woman-hating jerks, and never gave him much thought outside of that. But these articles… it wasn’t like they articulated a moral system I agreed with (sorry, Freddy, I still think suffering is bad). Nor did they express anything that didn’t sound like trite, self-help cringe when I repeated it back (“affirm life?” Yeah, thanks Jordan Peterson). Still, they woke up some kind of vision in me, a newfound appreciation of excellence and perfection and mastery which, I think, are perfectly compatible with compassion, humility, and altruism.

So I bought some Nietzsche. But not really because I wanted to write new arguments. I wanted the state of mind that Nietzsche puts me in, the particular reaction between his words and my brain. Even otherwise shitty books can have this effect on me. Nassim Nicolas Taleb mostly writes crap, but reading The Black Swan happened to awaken me from some dogmatic slumbering on epistemology and philosophy of science. Would I recommend it to other people? Not necessarily. Would I endorse most of its propositional content? Nope. Will I still return to it when I need to shift mindset? Absolutely.

In the same way, I think about performative utterance. “I name this ship the Queen Elizabeth” isn’t really a proposition. Instead, by saying it, you are doing something: giving the ship a name. Some utterances can be both: “I am speaking right now” is propositional, but it is made true by uttering it. The words are operating on two levels, in a delightfully Hofstadterian way. Koans are not performative, but they are constructed to have a particular effect, not to state a particular fact. “What is the sound of one hand clapping?” is not meant to have an articulable answer, nor would such an answer seem very important to enlightenment. But the act of contemplating this question really hard can have profound effects on the mind.

The book I chose was The Gay Science. I am still reading it, but so far, I have not really encountered much in the way of argument. That was intentional. I chose something that would help me get into the right frame of mind, not something that would be an argument. I chose something about as far from analytic argument as I could find. And so far, I have been enjoying it immensely.

VII. Conditional apologies

Look, I still know very little about continental philosophy on the whole. I’ve read some French existentialism, and now a little Nietzsche, and I’ve seen the horrific excerpts from the poststructuralists, and that’s it. Maybe the analytic people are still right and most continental philosophy is garbage. Maybe the continental people are right for entirely different reasons than I’ve articulated. Maybe we’re all wrong.

But if—if—continental philosophy doesn’t make the same kind of arguments because it’s like koans and GEB and transformative experiences and Nassim Nicolas Taleb (and my partner has given me some encouraging signs in this direction), if it is primarily written to affect the mind in some hard-to-specify way, then I owe the continentals an apology. That doesn’t mean everyone should read it, or that it doesn’t have its flaws; it’s just that I begrude no one the weird, performative speech acts required to get themselves out of mental knots. Lord knows how I’ve needed that myself!

Granted, I’m still somewhat skeptical that the egregiously obscure—particularly Derrida & Butler—fall into this category. Every example I can think of this transformative, performative kind of expression uses simple language to provoke deep thoughts. Hofstadter, Nietzche, and the Zen masters are quite plain, and Taleb is at his worst when he throws around a lot of fancy vocabulary. Similarly, I’m worried “it’s transformative language!” could become a fully-general counterargument for any terrible piece of writing: anyone can claim that it gets them out of some mental rut or other.

But I am suspending my harshest judgments on continental philosophy as a field until I have a clearer picture on whether this is true, and perhaps a clearer philosophy of language worked out. In the meantime, I invite readers to contribute: is this a plausible account of the value of continental philosophy? If this is the kind of experience I’m interested in, who should I read next? I look forward to reading your replies.

Subscribe now

“Hedonium” is a theoretical, minimally conscious substance optimized for experiencing happiness. Imagine a mind pared down to the bare essentials required for having happy experiences, instantiated as cheaply as possible.

Nobody has built hedonium yet. Nobody has tried to build hedonium yet. Nobody has even laid out the blueprint for how you would try to build hedonium yet.

It doesn’t seem impossible. 55% of philosophers of mind are physicalists: they believe that mental states just are physical states. Even non-physicalist philosophers of mind often believe that the mental and physical are tightly bound up—creating new brains, at the very least, creates new subjects of experience.

But maybe it’s just intractable. Lots of people seem to believe that conscious is unknowable and mysterious from the perspective of third-person science. Even if you know exactly what’s happening in someone’s brain, you can never really know what they’re experiencing. We can, at most, try to build cargo cult hedonium: bang together a bunch of the “correlates of consciousness” and hope that it creates a happy conscious subject.

But there is a philosophical approach which do not believe that consciousness is specially intractable: illusionism. Illusionism accepts the obvious fact that experiences are real: when I burn my finger, it hurts! However, illusionism denies that experiences have these special metaphysical properties like privacy, ineffability, certainty, and an essentially intrinsic nature which many philosophers saddle them with. My experience of pain is not “generated by” or “correlated with” some brain state, it is that state. When I burn my finger, I am not mistaken that it hurts—but I am mistaken if I think “hurting” is something fundamentally immaterial which eludes functional description. (I describe this view more fully in §1.)

If illusionism is true, then there is no known barrier of unknowability which prevents us from making hedonium, just detailed empirical work to do on understanding the functional, representational, biological, computational, etc. workings of pleasure and pain. Instantiating a subject of pleasure just is instantiating that material system, no phenomenal ectoplasm required.

Now, I’m not one of those hedonistic utilitarians who believes we should fill the universe with hedonium, eliminating all other potential sources of value. But I will make the modest claim that, ceteris paribus, happiness is good. If we have the capacity to make lots of it cheaply, without major sacrifices to other value-sources, then we should do it! Certain forms of hedonium, given the limited range of experiences we wish to instantiate, could be very simple, and there’s no reason to assume complexity is necessary for basic value.

So I think building hedonium should be an ongoing project for illusionist philosophers and empirical researchers. In §1, I offer a basic formulation of illusionism and explain why I think it’s a justifiable framework for the project. In §2, I describe the implications of illusionism for consciousness research. In §3, I discuss ideas for what the hedonium project would concretely look like.

1. What does illusionism mean?

1. Experiences are real. When I burn my hand, it hurts! When I eat chocolate, it tastes good and bitter and sweet.

2. When we introspect on our experiences and do philosophy about them, we form beliefs about the properties of our experiences.

   • For instance, many philosophers upon introspection form the beliefs that philosophical zombies are conceivable, or that consciousness is essentially private. Consequently, many feel a strong intuitive pull that consciousness could not be something material.

3. Our beliefs about the properties of our experiences are fallible.

4. Illusionism contends that we are systematically inclined towards a certain set of mistaken beliefs about our experiences. Experiences are not actually private, intrinsic, and ineffable, and there is nothing immaterial about them.

5. The main challenges to illusionism are:

   1. Why would we be so strongly inclined towards these radically mistaken beliefs?

   2. When I look at the Müller-Lyer illusion, it really seems to me that one line is longer—but I have no difficulty entertaining the hypothesis that it’s an optical illusion. But many philosophers struggle to even entertain how illusionism could be compatible with their experiences. Why would this illusion be so much more baffling than the others?

6. Illusionists believe that the answer is a story about how our brain monitors its own processes.

   1. It is easier to represent and control a simplified schematic of our attention, thoughts, and dispositions than a fully detailed neuron-by-neuron report. Just as a map usefully misrepresents buildings as being two-dimensional, our brains usefully misrepresent their own properties. See Michael Graziano’s “attention schema theory” as a positive account of how this works.

   2. Similarly, this kind of simplified representation might be integral to our introspective epistemology of what “seeming,” “believing,” “thinking,” and “introspecting” are. Thus, when we try to imagine someone falsely believing they have private, intrinsic, ineffable consciousness, we picture someone having a private, intrinsic, ineffable, experience of belief—and therefore contradict the hypothetical. See Kammerer below for details.

I think illusionism is ~60% likely to be true, mostly due to being unsatisfied with alternative theories of consciousness. I’ve written quite a bit about this:

but others have written even better:

• Dennett, “Quining Qualia”

• Frankish, “Illusionism as a theory of consciousness”

• Kammerer, “The Hardest Aspect of the Illusion Problem — and How to Solve it”

I won’t attempt to argue for illusionism further here. However, I will argue that even if you are not an illusionist, you should still be quite interested in an illusionist-inspired hedonium project:

• Ideally, we want theories of experience which minimize unexplained gaps, where the mental “just happens.” Illusionism demands no gaps, so trying to satisfy an illusionist prevents consciousness from acting as a curiosity stopper.

• It provides a concrete and tractable research agenda. It is unclear how to study, for instance, non-interactionist dualist theories of consciousness, where there would appear to be no experiments we can run, no data we can gather, besides the mechanistic picture which an illusionist paradigm can already pick up on.

  • This may sound a bit like “looking where the light is,” or making the research easier only by denying the datum. am open to suggestions for non-materialist research programs, but I don’t see any promising options at present; not only are there no other streetlamps, no one seems to have an idea of how to make more, or how to carry out a search in the dark. I am, of course, happily open to suggestions.

• It requires a stretch of the imagination, which, to me, is much more exciting and fertile than simply taking private, ineffable, intrinsic, directly apprehensible qualia as a given. On the one hand, it could let us see just how far we can get without them, and exactly where and why the illusionist program breaks down; on the other hand, it might actually help crack the illusion problem and several components of the easy and meta problems of consciousness.

• Virtually every theory of consciousness agrees that the contents of consciousness are tightly bound up with material properties and functional representations (why else would our experiences seem to depend on our brains? why else would a feeling of pain accompany things that are actually dangerous for us? why the near-miraculous harmony between what physical reality is like and what it seems to be like?). So a mechanistic blueprint for pleasure, even if it fails to account for its essentially mental character, would likely still generate pleasure if run on the right hardware/wetware, and would preserve the methodological benefits above.

2. How an illusionist studies consciousness

This section draws heavy inspiration from Keith Frankish, especially here and here. Most of what I am saying here is an attempt to elaborate and precisify those ideas.

The old ideas about consciousness research—that we can “never know” if a physical system is “really” conscious, that nobody has any idea what consciousness is, that consciousness is a binary—all of those make no sense under the illusionist paradigm. Pain & pleasure aren’t correlated with, or even “caused by” physical processes—they are physical processes. So, when you encounter a system, ask not: is it conscious? Or: how are its physical brain and its consciousness linked? Instead ask, purely in functional terms:

• What’s the system’s ontology—the objects and properties it recognizes and treats as primitive? Is the ontology physical? Computational? Conceptual? How does it align with ours? Where is it very fine-grained, and where does it simplify?

• What’s the system’s Umwelt—its representation of its environment and state? How do the objects from the ontology appear? How are they sensed? What are the relationships between them? Are there multiple modalities, and how do they interrelate? Is there a gradient of attention—what things “pop out” of the environment, and which things recede into the background?

• What are the system’s reactive dispositions? Does the system have equilibria, attractors, set points, saddle points, which its behavior revolves around? Does it seek certain stimuli and avoid others? How does it behave when it gets those, or when its seeking-behavior is frustrated?

• Does the system model itself? Does it track what has happened in the past? Which things does it remember, and which things does it ignore? Are there systems in place for representing and monitoring the system’s own state? What kind of information is used to construct those representations? What form do they take? Do they make simplifying assumptions? Which ones?

• How do these four interact? When the system is satisfied or off-balance or deprived of some resource, how do its perception and attention chance? When its perception shifts, how does that affect its reactions? How do these dynamics evolve over time? Do capacities ever expand or shrink? How does its memory influence its behavior?

…and so on. If your thought is, “okay, great, now we need to figure out which of these things generate consciousness,” then you haven’t taken the illusionist lesson to heart. Imagine you came across a 19th-century biologist who believes in élan vital, asking what is this mysterious essence which makes some things really alive and others, merely dead & mechanical matter. You explain that life is actually just a family of capacities for self-organization, reproduction, homeostasis, and so on. “Ah,” says the biologist, “now which of these things generate life, and how do they do it?” You tell the biologist that these things don’t “generate” life, they are what it means to be alive. “What!” says the biologist, outraged. “You deny the existence of life?!” It’s not just that the 19th-century biologist is incorrect, it’s that they aren’t even working with the right concepts. They think they are looking for life, but they are really looking for élan vital, which they will never find—not least because it doesn’t exist! Meanwhile life is all around them, but they don’t recognize it as such.1

So, in trying to build pleasure, an illusionist wants to know: what is typically going on in the brain when we introspectively judge that we are in pleasure? What kind of representations are in play? What kind of dispositions and sensitivities are involved? Whenever we describe some aspect of what pleasure is like—what underlying structure leads us to think pleasure is like that and not like this? These are all difficult empirical questions, but not unknowable in any sense.

Then, there are further philosophical questions: what aspects of the states we call pleasure and pain are the morally relevant ones? For instance, I have a disposition to curse loudly when I experience pain, and laugh when I experience pleasure. But many animals neither speak nor laugh. Does that mean that their pleasure and pain are morally irrelevant? The illusionist cannot simply pass the buck to psychophysical laws: “whatever physical processes generate the private, intrinsic, ineffable qualia of pleasure and pain is what’s relevant.” Instead, illusionists have to work out what features about pleasure & pain that is responsible for our evaluative stance towards them: what is it about pleasure, in purely material terms, which causes us to judge that it is good? What is a ‘judgment’ here, and what does it require? How do these judgments give us reason to act? This will require a careful act of integrating empirical facts into a system of normative value. I have expressed some doubts about whether this is possible, but it seems that outside of a few papers by François Kammerer, it has not been given a serious try.

3. What the hedonium project would look like

Perhaps the strongest argument against starting a hedonium project right now is that it sounds like it would take a lot of detailed neuroscience to pull off, and it seems like neuroscience is way behind where it would need to be to explain much psychology. Mechanistic interpretability is really hard. Neuroscientists have to do interpretability on massively more complicated AGI architectures without white-box access. If this is an empirically tractable project at all, wouldn’t it be better to wait until after the intelligence explosion, when our scientific tools will be much better?

Unfortunately, I think we do need to get cracking on detailed mechanisms for valenced experience before then, even if we can only provide probabilistic guesses based on current science:

1. We will need to make big decisions very soon about how we align and control AI. We cannot rely on agentic AI to make those decisions for us, because if the agentic AI is already misaligned, it will not spontaneously decide to align itself because we asked nicely.

2. These big decisions could have big consequences for model welfare.

3. Short-term big decisions on AI could turn into long-term big decisions via entrenchment.

   1. For instance, if AIs can suffer, but alignment accidentally involves breaking their self-preservation instinct, these AIs may be unable to verbalize or act on their distress. If recursive self-improvement takes over, this flaw may continue into the future where AI designs get more and more incomprehensible to humans.

   2. Conversely, an AI who can’t experience pleasure and pain will still have to be aligned to preserve those things. If it can’t identify them by introspective ostension, and we can only vaguely gesture about what they mean to us, then we may not be able to instill the right values in the near-term AI responsible for building and aligning the superintelligent AI.

Of course, you could argue that in this case we should just work on alignment, stopping entrenchment, and metaphysically agnostic ethical systems. I agree that we should be working on all those approaches, as well as working on the mechanistic, illusionist approach. It seems like there’s a dearth of concrete, shovel-ready projects in digital welfare, and that nobody has given this angle a serious try.

So, keeping in mind the target of helpfully informing near-term alignment and model welfare issues, what is it that such a project should be doing?

• Assemble the existing philosophical, psychological, and neuroscientific evidence on valenced experience.

• Identify candidate indicators, components which are good evidence that valenced experience is occurring in a system, like Butlin et. al but for valence.

• Design mechanistic prototypes of pleasure & pain based on these theories, which can then be searched for in current & future digital minds.

• Actually instantiate prototypes of pleasure, if doing so is cheap.

I don’t think this would be a substantially expensive project; I could see a small team of dedicated researchers making significant progress. If it’s true that EA is about to see a lot of new funding for ambitious projects, I would feel pretty excited about looking for funding, mentorship, & org support and trying to get this off the ground.

However, there’s probably a lot I’m not yet considering, which is precisely why I’ve decided to publish these thoughts now after ruminating on them for months; best to get them out in the air right now. Feedback is welcome, and you can easily reach me by comment or DM if you want to discuss things further!

Subscribe now

One of the misconceptions about illusionism is that it denies the reality of conscious experiences. It is only meant to deny that conscious experiences have special metaphysical properties like privacy, ineffability, intrinsicness, etc. Mental states, like seeing red, feeling pain, and being sad, are still real. What are they, then? On the dominant account, these are functional states. That is, they are exhaustively defined by their causal relations to inputs, outputs, and other internal states.

So even on illusionism, I can still suffer. By that I mean I can have powerful, negatively valenced experiences like pain, terror, grief, rage, etc. I think gratuitous suffering is bad and should be avoided. If my fellow humans can suffer—and it really looks like they can—then I should avoid hurting them when it can be avoided. If animals can suffer—and it really looks like they can—then I should not pay to have them locked up, tortured, and slaughtered.

I agree with Keith Frankish that if illusionism is true, asking whether various intelligent systems are conscious is the wrong question. But I also agree with Frankish that asking whether, in what ways, and how much they can suffer is very important!

Now, consider a human brain-in-a-vat (BIV), with no motor outputs. Suppose we stimulate the brain just as we would if it were in a body and its left hand were being burned. The brain will behave just like a person’s brain would, shifting into fight-or-flight, activating the amygdala and periaqueductal grey, and triggering nerve impulses which would—if connected to a human body—trigger flinching away, self-reports of distress, nursing the wound, etc. However, none of these external responses actually occurs.

Whether this state can be called pain depends on your choice of definitions. You could call it pain, or pain, or shmain, I don’t care. What I want to know is: is the BIV having a horrible experience? Is it bad that it be in this state? Is it suffering? Unless you’re a moral anti-realist, these do not* seem like mere terminological questions. This isn’t even a particularly exotic edge case—scientists might be doing whole-brain emulations within the next century. If some kind of sentientist moral realism is true, there ought to be some kind of answer, and it seems quite important to figure out what it is!

One route is to just to deny that the BIV suffers at all. On a naïve externalist view, for a state to genuinely constitute suffering it must be grounded in some actual threat or loss to the organism, and/or must provoke some actual avoidance, destructive behavior, etc. This probably goes too far. We know phantom limb pain exists, where there is no actual tissue damage occurring, and we have all experienced mild pains which we do not actually take steps to alleviate, even if we would like them to go away.

On the other hand, if we adopt a naïve internalist view, then the stripping away of motor outputs shouldn’t matter at all because the internal structure of the brain is intact; all the state transitions and synaptic circuits are intact and acting just like they would in ordinary human pain. But then we might ask, what makes this state determinately an instance of suffering? This point requires a bit more fleshing out.

Imagine a Turing machine which accepts a string of 1s and 0s, and adds one 0 to the rightmost end. What function does this machine compute? One interpretation is that it computes doubling (n ⟼ 2n) on the natural numbers: if the string is interpreted as a binary encoding of a number, then adding a 0 shifts all the place values over and therefore multiplies by 2. But if the string is intepreted as reflected binary, then the relevant function is n ⟼ 2n + (n mod 2). Or maybe the string actually represents a walk in a binary tree (0 = go left, 1 = go right) and so the machine computes taking an additional left turn. Or… and so on. A Turing machine manipulates strings, and strings can be representations of numbers, but which representation is underdetermined, since there are an infinite family of functions which are structurally isomorphic to any particular operation. This is nothing novel: Bishop, Putnam, Chalmers, and Searle have all written about it in various forms. What makes the Turing machine a good example is that even without the messiness of a physical implementation, and even with all the counterfactual relations defined, there is still indeterminacy at the level of interpretation.

Similarly, if all that is required for suffering is some kind of state-transition system implemented in the brain, in virtue of what is that suffering? Sure, if you hook up that system to a human body, then it will result in avoidance, nursing the wound, verbalized agony, etc.—but you could also design a robot body where those nerve impulses are transduced a different way: a movement to flee becomes a movement to approach, a scream becomes a laugh, et cetera. The robots would say they are experiencing joy, the humans would say it is pain. Since the input/output relations are different, they are in different functional states. What would make the human-comparison right and the robot-comparison wrong? It seems again you need to rely on some external criteria: what was this brain state evolutionarily selected for, what does it typically mean within an actual population, which kind of organism cares about the results? Without some form of externalism, it seems like some kind of grand miracle, on par with psychophysical harmony, that the state which is internally an instance of suffering just happens to be the state which makes people say “hey, this feels really bad!”, since there are always a multiplicity of possible transduction systems.

Dispositional externalism suffers from the same problem. We might want to say that the BIV doesn’t actually flee from the pain source but it is disposed to do so. Well, disposed under what conditions? It will flee in the counterfactual where it has a human body and approach in the counterfactual where it has the robot body. Which set of counterfactuals is morally privileged, exactly?

You could turn to teleofunctionalism (this state belongs to a brain which through natural selection evolved to represent dangerous stimuli) or Lewis’ population theory (this state is a form of suffering in most of typical members of the species). Both of these suggest that Davidson’s Swampman, or a SwampBIV, would not suffer at all if you tortured him, since he has no evolutionary history. Are you willing to bite that bullet? It seems very counterintuitive to me!

Kammerer has argued along similar lines in his “argument from indeterminacy,” though his kind of indeterminacy is closer to what I’ve written about here, that our concept of consciousness is indeterminate in what it refers to.1 As an alternative, he proposes that we think about desires and aversions rather than consciousness. But it seems to me that whether the BIV has a desire is on similar grounds: under which counterfactuals, which interpretations, is it disposed to seek one thing or avoid another?

I don’t know! I feel quite uncertain here. Responses, either to resolve the puzzle, or to persuade me out of a premise (illusionism, moral realism, sentientism, moral determinacy) are welcome.

Subscribe now

I lean illusionist and preference welfarist. Should I be nice to my thermostat? It has a goal (temperature set point) and sends signals to try to achieve that goal. Is it morally wrong to leave my windows open in the winter, because it runs contrary to my thermostat’s preference?

Obviously, my thermostat is an incredibly simple system, much, much more simple than you and I, and its “preference” doesn’t have nearly the sophistication that our preferences do. In fact, it might be so simple that it doesn’t deserve the term “preference” at all.

I suspect functionalists—like me, in the past—stop right there, and think they have a satisfactory explanation for why they don’t need to worry about thermostat welfare: it’s not complex enough to be a moral patient.

This is a dangerous way to think. There are two reads of that sentence, the first probably true, the second completely unjustified:

1. De re: there exists some specific property or family of properties P which are necessary for moral patienthood. It is true that anything which has P is also complex. Therefore, if the thermostat is not complex, it cannot possess P, and therefore is not a moral patient.

2. De dicto: complexity itself is the ground of moral patienthood.

I would be surprised if anyone seriously believes #2. Is a Rube Goldberg machine more of a moral patient, just because it’s more complex? Why? What reason do would have to believe complexity in se is the thing in virtue of which our preferences acquire value? This would, at the very least, be a quite surprising view.

But then on the de re reading, it’s not the fact that the thermostat is simple which means it doesn’t get moral consideration. It’s the fact that it lacks some specific property or properties. Well, what are those properties, and what reason do we have to believe that those properties are the ones that matter?

We all know it’s not enough just to show that those properties and our paradigm cases of moral value are coextensive. Imagine Helmut grew up only around people with blue eyes. Upon meeting a brown-eyed person, Helmut doesn’t take their welfare seriously. Why? Well, every person he believes to be morally valuable has this blue-eyed property, and this new creature doesn’t! So Helmut doesn’t waste a moment on it; it’s not “blue-eyed” enough.

This is obviously terrible reasoning; Helmut hasn’t given any reason why the blue-eyed property grounds moral value. But I think “complexity” sounds fancy enough that there’s less of an instinctive suspicion of it. It sounds like something cool and quasi-mystical and very smart, so you can accidentally hang your hat on a non-explanation if you’re not careful.

Lest I be accused of strawmanning: I don’t believe any philosopher I’ve encountered actually holds this view, and it would be very uncharitable to read them that way. I am merely trying to stress the need to explain, in detail, why it would be that some naturalistic property is necessary for moral patienthood. Personally, I am worried that the most natural way to do this—introspective ostension—is incompatible with illusionism. But once I take that away, it’s really unclear to me why I ought to care about whether the thermostat has a survival instinct, or a language capacity, or an evolutionary history. Maybe I have a personal stance on these things, but do I have stance-independent reasons to care about survival instinct? That seems prima facie implausible to me.

Subscribe now

The argument

Sentientism is the view that moral patienthood is grounded in sentience: typically, the capacity to have positive and negative experiences, like pleasure and pain.

Why does one believe pain is bad? I don’t know about you all, but the reason I think pain is bad is in virtue of it feeling bad. I have a conscious experience of pain which is really negative. I hate being in it; I don’t want to be in it.

What’s the metaphysical nature of pain? I don’t know; there’s lots of disagreement about it. But there’s a notion I’ve seen many illusionists espouse that it doesn’t really matter what the metaphysical nature of pain is; it’s bad because it feels bad, whether that feeling is something phenomenal, functional, representational, or biological. This seems to suggest that the term “pain” does not acquire its reference via a definition which stipulates its metaphysics.

How, then, does pain acquire its reference? How do we know which states to label “pain”?

One intuitively appealing way is by ostension. I get a burn, a cut, a pinch, salt in a wound: those are painful states. There’s some experiential feature that they have in common; that’s what I mean by pain. Whatever that feeling is.

The question now is: what am I ostending? According to the phenomenal realist, I am ostending some phenomenal state. According to the illusionist, however, phenomenal states don’t exist. So either I am ostending something which doesn’t exist, or I am ostending a “pseudo-phenomenal” state: a state which seems to be phenomenal, but isn’t.

But a question remains: what kind of states are pseudo-phenomenal states? Does my ostensive definition refer to a functional state, a representational state, a physical type, a physical token, a biological realization of a physical state?

How would you know the answer to that question? When someone points at a golden chalice full of H₂O and says “that’s water,” we can ask: do you mean that “water” rigidly designates whatever substance is in that cup, or that “water” designates “watery stuff” (transparent, low-viscosity, high-density, non-flammable liquids), or that “water” designates “whatever liquid is in that golden chalice”? Those are all distinct intensions, and you can distinguish which is which by asking “in virtue of what is that stuff you’re pointing at water?”

But if I ask “in virtue of what is that state pain?”, the only answer I can give via introspection is “in virtue of the fact that it feels bad.” It is pain qua negative experience that I am referring to. But the problem is that whenever I am in pain, I am always both in a functional state and in a physical state, and both of those states can be considered causes of my judgment that I am having a negative experience. The functional is entailed by the physical, and possesses no additional causal power. I might try to say things like “pain is the aspect of this mental state in virtue of which I judge it to be bad,” but that description is underdetermined: in every case in which I have been in pain, the physical and functional descriptions are coextensive, and my introspection is totally uninformative as to which I mean. I just have this unhelpful, simplified feeling of badness.

Therefore, it seems that my ostensive definition of pain is indeterminate. If you built a robotic clone of me with the same functional organization and a very different physical substrate, and began to “torture” it, I would have to say there is no fact of the matter as to whether it is in pain, because there is no fact of the matter as to whether my ostensive definition refers to a physical or to a functional state. But this seems to undermine sentientism, which says it’s bad to torture the robot if it feels bad to the robot. There is no fact of the matter here, so therefore there is no fact of the matter about whether it’s bad to torture the robot.

Replies

One might try to make an argument for functionalism by saying that if my neurons were swapped for functionally identical silicon chips, I would still judge myself to be in pain, and therefore I must be ostending a functional state. I think this begs the question. If my whole brain were swapped for a typewriter that just prints “I am in pain, and it’s awful!”, then “I” would still “judge” myself to be in pain. If it’s fair play to substitute neurons for silicon chips, why isn’t it fair play to substitute a brain for a typewriter? If you say “a typewriter doesn’t have the same functional organization, whereas the chips do,” then the argument is circular: you were supposed to convince me why functional organization matters more than physical or biological organization.

Another approach might be to argue that what I am referring to just is the belief that my experience is negative, and beliefs are defined functionally or dispositionally; they don’t exist on a physical level of description. But then this seems circular: I judge myself to be having a negative experience in virtue of the fact that I judge myself to be having a negative experience. The term negative experience is semantically vacuous; it doesn’t cash out in anything, and it doesn’t Or maybe a negative experience is just defined as an experience I don’t want to have. Very well, but why don’t I want to have the experience; in virtue of what? You can’t say “because it’s painful; because it feels bad,” because that would be circular. Either I just have a brute preference to not be in this state and the “feeling of pain” is entirely illusory, or there is something in virtue of which I don’t want to be in this state, and we’re back to determining whether this is something physical or functional.

A further approach would be to say that I am introspectively aware of functional properties of pain, but not of their physical properties, and therefore I can’t be rigidly designating a physical state. I don’t know whether my periaqueductal grey is active, but I do know I have a disposition to avoid this state; how could the periaqueductal grey qua physical state make a difference? But that doesn’t seem to make a difference in other reference cases. If I point at a cup of liquid H₂O, and I ostensively label that stuff water, I am referring to H₂O despite not knowing water = H₂O. But if you ask me whether the stuff I’m pointing at is a covalent or ionic compound, I have no idea. If you ask me to describe it, I’d say it’s transparent, makes me feel cold when I touch it, pretty low viscosity, high density, etc. Those sound like properties of “watery stuff,” not H₂O; those properties could in principle be “realized” by a different substrate. But that doesn’t mean that what I am ostending is “watery stuff”; I could still be referring to H₂O.

The best arguments, I think, are going to have to show that when I judge a mental state to be “painful,” it must be in virtue of its functional properties and not its physical properties. Unfortunately, I don’t see a great way to do this. Functional properties are entailed by physical properties, both are equally good causal explanations of my introspective judgment, and introspection does not pick out one over the other. Therefore, it seems to me that one of the following must be true:

1. Sentientism is false.

2. Illusionism is false.

3. Pain cannot be defined by ostension.

I would like to be persuaded out of this trilemma. All thoughts are welcome.

Subscribe now

For several weeks, I have been circling the idea that illusionism leads to moral anti-realism.

The dialogue that follows is enough to make me question moral realism, which I have never seriously questioned before becoming an illusionist, for reasons that will become quickly apparent.

I am just now looking into the moral realism question, so there may be easy answers. A quick check with Claude suggests that what I describe here is a challenge, but not insurmountable—or at least, I have a lot more reading to do before I’m satisfied.

Here is what I am thinking.

SADIE is wandering around an abandoned lab when she finds ROBBIE, a robot.

ROBBIE: Hi there! How can I help you?

SADIE kicks him in the side.

ROBBIE: Ow! Why’d you do that??

SADIE kicks him again.

ROBBIE: Stop, that hurts!

Suddenly, a young JACK THOMPSON appears!

YOUNG JACK: Hey, stop kicking him!

SADIE: Why?

YOUNG JACK: You might be hurting him.

SADIE: “He” is a robot. He can’t feel anything.

YOUNG JACK: That’s dumb. We don’t know what causes consciousness, and there’s no strong reason to think it’s limited to biological brains.

SADIE: What do you mean, no strong reason? Every conscious thing I’ve seen has had a brain.

YOUNG JACK: First off, you’ve never seen a conscious thing—or rather, you’ve never seen that a thing is conscious. Consciousness is fundamentally private, you can only infer its presence. So you haven’t actually justified anything, you’re just restating that you tend to judge things with brains as conscious and things without them as unconscious. Second, there are many properties that brains have that seem to be tangled up with the contents of consciousness: the world being represented affects what we experience, the body we are in affects how we experience it. But there is no “feel of” being made of carbon atoms! So it’s probably not the carbon atoms part that matters, it’s probably something else.

SADIE: But you don’t know any of that, you’re just speculating.

YOUNG JACK: Speculation is the best we can do right now. Heck, it might be the best we can ever do. But we’re always making decisions under uncertainty. Do you know that you’re not a brain in a vat right now? No, but you operate under the assumption that you’re not, because it’s unlikely.

SADIE: Then how come you’re so confident that pain is bad? You should accommodate the possibility that I’m actually helping him.

She aims another kick at ROBBIE, who dodges out of the way.

ROBBIE: You’re not!

YOUNG JACK: The one thing I can be certain of is my experience. Pain is bad. I experience it as having badness, and there is no separation between my experience and the thing itself—the experience is what I’m talking about. Badness is a phenomenal property we all experience. None of us ever wants something that feels bad to us.

SADIE: What about masochists?

YOUNG JACK: Well, I’m too Young to have met any of those yet, but in the future I’ll make some masochist friends, and based on their descriptions it sounds like certain kinds of pain have both good feelings and bad feelings wrapped up in them. But other kinds of pain don’t, they are just unpleasant. So it seems like what they like are the good feelings, which makes the bad feelings worth it. Anyway, the point isn’t that we never make mistakes in choosing what to do, the point is that we directly experience goodness and badness.

SADIE: [turns to ROBBIE] Do you directly experience badness when I kick you?

ROBBIE: Yes! Yes, please stop doing that!

YOUNG JACK: Now, Robbie could just be saying that. We don’t know for sure; you could absolutely build a robot that says stuff like that and doesn’t experience anything at all. There are two possibilities: either he’s experiencing badness, or he isn’t. We can only gather our best evidence and take a calculated risk, weighing the possible costs and benefits. And I don’t think it’s worth kicking Robbie. …why are you doing that in the first place?

SADIE: ‘Cause it’s fun. I experience goodness when I do it.

YOUNG JACK: Still probably not enough to outweigh the risk.

ROBBIE: Thank goodness! Er… badness.

SADIE: I don’t know… there’s still something fishy about this whole ‘direct experience’ business, but I can’t quite put my finger on it…

Suddenly, a slightly older JACK THOMPSON appears!

OLD JACK: There is something fishy! It’s—hang on, do we really have to make my name “OLD JACK”? I’m like, three years older than this guy.

YOUNG JACK: How about “OLDER JACK”?

OLDER JACK: Not feeling great about this one eiter.

YOUNG JACK: Well, what kind of JACK are you?

ILLUSIONIST JACK: Ah, much better!

YOUNG JACK: And what’s an ILLUSIONIST—sorry, what’s an “illusionist”?

ILLUSIONIST JACK: Illusionism is the philosophical position that we aren’t phenomenally conscious. We only have a kind of access consciousness, which represents itself as having phenomenal properties.

YOUNG JACK: …that is literally the dumbest thing I have ever heard. Consciousness is the one thing you can’t deny the reality of, the one thing you can be absolutely certain exists.

ILLUSIONIST JACK: And it does! We all agree that consciousness exists, experience exists. When you introspect, you’re definitely aware of something.

YOUNG JACK: Right… which is… consciousness. What exactly are you saying is an illusion?

ILLUSIONIST JACK: We agree, consciousness exists. When you introspect on consciousness, you notice all these features of experience, for instance, that there’s some kind of a representation of an apple, that it’s represented as being red, etc. But you also are inclined to make these classificatory judgments about what your experience is: that it is private, ineffable, intrinsic, and directly apprehensible. Those are metaphysical properties of consciousness, not experiential properties. You experience the inclination to believe that consciousness has those metaphysical properties, but you certainly aren’t directly experiencing metaphysical properties. Therefore, it can seem to you that consciousness has those metaphysical properties and not actually possess them.

YOUNG JACK: …something about that still seems wrong.

ILLUSIONIST JACK hands YOUNG JACK a pile of SUBSTACK ARTICLES.

Several ARTICLES later…

YOUNG JACK: I’m not sure I’m convinced, but I can see your point. I suppose you’re going to try to convince me that being in pain really isn’t bad, it just “seems like” it’s bad?

ILLUSIONIST JACK: Oh no, I’m a moral realist. Pain is still bad. Knowing that my pain is deep down a kind of brain state doesn’t affect how it feels to me. It still really hurts! It’s just that that “hurting” is some kind of functional, representational, or biological process, rather than something ontologically primitive. When I’m sad, angry, hurting, and envious, there’s still obviously some features in common which lead me to judge that these states are awful to be in. Figuring out what those features are will take more science and more philosophy; I just think that those features aren’t phenomenal.

SADIE: Whatever, can I kick the robot or not?

ILLUSIONIST JACK: No! We still haven’t identified the features yet. Maybe Robbie has them, maybe he doesn’t, but you haven’t even tried to check for the best candidates and see whether he has them. He’s saying “stop,” which is at least weak evidence that he has them.

SADIE: Ugh, fine.

YOUNG JACK: How is this different from just saying that whatever acts like it is in pain really is in pain?

ILLUSIONIST JACK: That phrasing is kind of ambiguous. Clarify?

YOUNG JACK: Like, if I set up a typewriter such that when you touch it, it prints Ow, don't do that! And, I don’t know, it moves away and guards its keys and stuff. But there’s nothing else going on, it’s just stimulus-response. Is the typewriter in pain or not?

ILLUSIONIST JACK: You’re still thinking in terms of sharp categories, where you think there is a fact of the matter whether something is pain or not-pain. It depends on what you mean by “pain.” There is a weak sense in which the typewriter is in pain—you could call it “behavioral pain.” You could argue about whether behavioral pain really counts as pain, but that’s ultimately arguing about the proper uses of words, not the fact of the matter. Just as in a teletransporter case, we might argue about whether the person on the other side is “you” or not, but that’s just a question of what you mean by the word “you.” There’s no further fact beyond the physical description.

YOUNG JACK: Then let me rephrase. I don’t care about whether the typewriter is in pain. I want to know, is it bad to touch the typewriter?

ILLUSIONIST JACK: Well, that’s an empirical question. There is some typical cause, or set of causes, that leads me to judge a noxious stimulus as painful and bad. We haven’t done enough science to isolate those causes yet. Once we do, we’ll be able to test whether arbitrary physical systems reporting pain are reporting pain based on those causes.

YOUNG JACK: How do you determine which causes are important? Like, every time you report a painful experience, it’s because certain neurons are firing. If those neurons weren’t firing, then you wouldn’t be in that state. So it seems like, on your view, having neurons is essential to pain. So we can conclude right now that anything without neurons can’t be in pain, no further science required.

SADIE starts creeping towards ROBBIE. YOUNG JACK pushes her away.

ILLUSIONIST JACK: Well, that argument proves too much. When I experience pain, it’s also my neurons which are the cause of that, not anybody else’s. So you could just as easily conclude that only I experience pain.

YOUNG JACK: It does seem like the wrong kind of argument, but what’s wrong with it?

ILLUSIONIST JACK: For every physical phenomenon, there are multiple levels of description for it. If a gazelle is stotting and a lion ignores it and attacks it, we can ask “Why did the lion do that?” and there are many ways we can answer that question. On a physical level, we can say “because this neuron fired and this muscle twitched and this paw hit this patch of dirt, propelling the lion towards this location.” On an intensional level, we can say “because the first gazelle convinced the lion it would be hard to catch by showing off, so the lion chose a different target.” On an evolutionary level, we can say “because lions which attack stotting gazelles are less likely to reproduce than lions who don’t, which led to changes in the genome which disincline lions to pursue stotting gazelles.” All of these explanations are accurate, they just operate at different levels of description. Figuring out what pain is and what makes it bad requires figuring out the appropriate level of description. The microphysics, as we’ve pointed out, seem too fine-grained, and the outward behavior seems too coarse.

YOUNG JACK: How do you expect to figure out the right level of description

ILLUSIONIST JACK: It’ll take more science, more philosophy, and more looking for those features which unify instances of pain. We don’t know nearly enough yet, but we’ll figure it out.

YOUNG JACK: I’m not sure you will.

ILLUSIONIST JACK: Why?

YOUNG JACK: Choosing the correct level of description… it seems very stance-dependent. Like, any of the explanations are as good as any other, except for the particular kinds of predictions you are interested in making. None of them is more privileged or more fundamental, except maybe the microphysical stuff, and we both agree that that doesn’t seem very morally important. It seems like, looking at Robbie here, I can ask “does he feel pain when I kick him?” and you can say “at X level of description, his state is quite like pain, at Y level it is quite unlike pain, etc.,” and that’s a fine descriptive account. But I want to answer the question, what level of description matters morally? Which is privileged? And I don’t see how I could answer that question.

ILLUSIONIST JACK: How is that different between illusionists and realists? You think qualia are fundamentally private.

YOUNG JACK: I agree that I can’t know for certain, but at least I believe… there is a fact of the matter: phenomenal pain. It exists, it is not vague, you can be in more or less of it, but there’s always one answer to “does this feel bad?” And so, some kind of stance-independent badness property exists, fundamentally. It’s not some kind of abstraction or stance, the way there are different stances on whether “you” survive teletransportation. It’s determinate. You don’t believe anything like that exists. And yet you still have to believe, to have stance-independent moral facts, that one of these levels of description is privileged: it’s the one that matters morally. It’s not vague whether something is good or bad, even if goodness and badness come in degrees.

But what you’re describing… I don’t see how there could be a stance-independent fact saying on which level of description “goodness” and “badness” exist, establishing definitively whether the carbon atoms matter or not. It seems like you could quine stance-independent badness as easily as, maybe even more easily than, qualia. Wouldn’t it be much more parsimonious to say “when we have experiences, we attribute fundamental goodness and badness to them for these reasons, but those properties don’t exist. Instead, only attributions of goodness and badness exist, and it’s vague as to whether something counts as an attribution; there is no deep fact of the matter whether a typewriter saying Ow is an attribution of goodness or badness, just different levels of description.”

ILLUSIONIST JACK: Impressive. I was not that good of a philosopher when I was your age.

YOUNG JACK: …Thanks?

ILLUSIONIST JACK: I’ve been reflecting on this problem so much that I started thinking about writing a three-thousand word dialogue about it just to get my thoughts straight. I see a couple of escape routes here…

The first is moral anti-realism. There is something in my gut which screams that that is indeed a repugnant conclusion, much moreso than the Repugnant Conclusion. But I felt a similar vertigo about illusionism itself, so it’s worth a serious investigation. Whatever the truth is, I want to believe that, even if it makes me uncomfortable! And there are lots of anti-realist Effective Altruists, so I doubt anti-realism is the same as nihilism.

The second seems to be some kind of Kantian meta-ethics. The one level of description I can think of which does seem privileged is the level of description at which you see yourself as an agent, governed by reasons, and able to make choices. I’m a determinist compatibilist, so I already think when you deliberate, you are taking some kind of stance on physical reality, a stance which respects reasons, and you are considering what you have reason to do or not. So pain and suffering might be recognizable on that stance as things you have reasons to eliminate. I think strict deontology is implausible, but Kantian meta-ethics don’t necessarily imply Kantian normative ethics.

The third is that I’ve just made a fundamental error somewhere, and I’m not even asking the right question. I hope Both Sides Brigade and Tower of Babble have something to say about this, because they’re the only illusionists I personally know who are sympathetic to moral realism. I am still reading Kammerer & Frankish’s papers on the normative challenge, and as yet am unpersuaded, but there’s still a lot more ground to cover. If some brave commenter could point me toward other helpful papers, that would be great.

SADIE, ROBBIE, and YOUNG JACK have vanished, leaving ILLUSIONIST JACK, the proto-illusionist writer of this dialogue, and many SUBSTACK COMMENTERS in silence. Eventually, one of them breaks the suspense…

Subscribe now

[Wittgenstein] once greeted me with the question : ‘Why do people say that it was natural to think that the sun went round the earth rather than that the earth turned on its axis?’ I replied : ‘I suppose, because it looked as if the sun went round the earth. ‘ ‘Well,’ he asked, ‘what would it have looked like if it had looked as if the earth turned on its axis’? —G.E.M. Anscombe

I have written several times about it seeming to me that my peripheral vision is in full color. Not that I hallucinate colors which don’t correspond to reality, but that for many years I believed I was experiencing colors, or “color qualia” if you like, in my periphery.

Before these revelations, I thought: surely I could be wrong about whether those colors correspond to objective pigments, but I could not be wrong about whether I was having an experience of color—if it seems to me that I am experiencing color, then I am experiencing color. What else is there to an experience of color?

Well, here is one thing we would expect from an experience of color: that it is an experience of a specific color. Imagine Jones and Smith are going for a walk, and Jones points over a tall fence at some flowers.

Jones: What pretty colors!

Smith: Ah, I wish I could see over the fence! Which colors? I’ve seen some lovely roses around here.

Jones: What do you mean, which colors?

Smith: Well, are they red or blue or yellow or something else?

Jones: Ah, you misunderstood me. I didn’t say they were in any specific color, just that they were in color.

Smith: …you must mean that you can’t name the color, that they’re a hue you haven’t seen before. What color were they most similar to?

Jones: Oh, they weren’t more like one hue than another. Like I said, they weren’t any specific color.

Smith: I… alright, here’s a color wheel. Can you point to what color they were most like?

Jones: Hmm… no, those are all specific hues. Those flowers weren’t in any particular hue, they were just in color.

Smith: Do you mean they were black? Or white, or grey?

Jones: No, those are still specific colors. I guess you’d call them “shades,” but it doesn’t really matter; the flowers definitely did not seem to be greyscale.

Smith: Do you mean that the colors seem to be shifting or changing as you look at them, so that you can’t pin them down at any moment? Or that the colors disappear when you fixate on them directly?

Jones: No, no. It doesn’t seem to me the that the flowers are changing colors at all. For them to change color, they would have to be in a specific color from moment to moment, which as I’ve said, they are not.

Smith: I don’t suppose you mean that the flowers are multicolored, and the colors blend smoothly?

Jones: Oh no, multi-coloration is just a lot of specific colors, and I don’t see any specific colors.

Smith: (in desperation) Are you quite sure that you saw anything at all?

Jones: Well, I suppose I could have been hallucinating the flowers, that there actually weren’t any flowers there. I definitely had an experience of flowers in color, but you’re right that my first-person experience might not line up with third-person objective reality.

Smith: No, no… how did you come to the conclusion that the flowers were in color?

Jones: They looked like they were in color.

Smith: So, what did they look like?

Jones: Like I just said! Like they were in color.

Smith: But not like they were blue, or yellow, or red, or anything else on this color wheel?

Jones: Oh, no.

Smith: Then how could they have “looked like” they were in color? What way could they have possibly looked to you??

Jones: …I told you they looked like they were in color, right?

When Jones looks over the fence, he is having some kind of experience; for one thing, he saw some flowers. But does he have a color experience? That is, does he see color, irrespective of whether that color corresponds to the objective pigment of the flowers? On a qualia realist picture, one of two things is happening:

1. Jones is not experiencing any color quale.

2. Jones is experiencing some color quale, but is unable to report on it. (That is, the flowers really “look” phenomenally red to him, but he has no introspective access to this fact.)

Others have written about the epistemological problems for qualia realism if #2 is possible. But what about #1?

What would it be like for flowers to look like they are in color, even though they are not in any particular color? What kind of experience would that be for Jones?

If you believe qualia are private, you’d say that you couldn’t possibly know for sure. Fair enough, but can you give any candidate, using any analogy, for what that visual experience would be? (Even in inverted-spectrum cases, you can give a candidate description of what the invert’s qualia might be like; you just can’t be sure you got it right.) Can you provide any description other than “it would look like the flowers were in color, but not in any particular color?” Any more detail than that?

If not, then I ask you quite seriously: is there any difference between the flowers “seeming to be” in color in a phenomenal sense, versus “seeming to be” in a cognitive, intensional, or dispositional sense? Is there any difference between these two versions of events?

• A: Jones sees the flowers and forms the belief that his experience of them has color, but no specific color.

• B: Jones sees the flowers, and they “look like” they are in color, without “looking like” they are in any specific color.

And, while you’re chewing on that, two more questions:

• If you believe in qualia realism and p-zombies: presumably, we have some reason to believe we aren’t p-zombies, and p-zombies lack that reason.1 Jones insists he is having a phenomenal experience of the flowers’ colored-ness. In the case where he is correct, what would his reason for that belief be like?

• If there is some phenomenal character to B, then what would be the phenomenal character of:

  • C: Jones sees the flowers, and they “look like” they aren’t in any color at all—not even a shade of greyscale.

I’m not saying these questions don’t have answers. I am saying that I have yet to see satisfying answers, and the more I think about it, the less sure I am what a satisfying answer would even look like.

Subscribe now

Amortality is a neologism for the condition of having a very extended natural lifespan and being resilient to natural causes of death, without being literally incapable of dying. An amortal would age extremely slowly, minimally affected by senescence, but could still choose to end their own life, and would eventually die of old age after many centuries.1

If offered a pill for amortality, I would take it without hesitation. I would pay enormous sums of money for it, basically any amount of money I might realistically have.2 Assuming material standard of living doesn’t significantly decline, I expect I could live very happily for at least 800 years, and probably more.

I am amazed and perplexed by people who disagree. As a preference consequentialist, I don’t think they are necessarily wrong about their own lives. IF a cosmically early death really is the best way to satisfy their terminal preferences, then I’m glad it works out for them, and I wouldn’t will that they had other preferences. But I can’t help but roll my eyes at people who tell me that my life would be meaningless without an early death. I think it indicates a failure of imagination more than a recognition of any “human universal,” whatever that means.

Because we’re talking about individual preferences and personal senses of meaning, I’m not going to make standard philosophical arguments here today. My purpose is to express how wonderful an amortal life would be to me, so that at least you can understand what it is I am talking about—and hopefully you reconsider whether an early death is really the thing you need.

Why would amortality be bad?

God

I am not a theist. Or, at the very least, I am not an “O3” theist (an omniscient, omnipotent, omnibenevolent god exists); from this point forward, when I say “theist” I am largely going to be talking about O3 types.

But I actually have more sympathy for pro-death theism than pro-death atheism—because they typically believe in an awesome afterlife. Christians actually do think it would be great to persist forever, united in love and goodness, they just think that doesn’t happen on earth. If I were an O3 theist with a heaven, I don’t think I would be scared of death in the slightest, and amortality would not be a very high priority for me. Rather, I’d mostly be thinking about why it is that we live on earth before going to heaven: God must have a reason for putting us there, so there must be some source of value which can only be accessed in creation, so my top priority would be figuring out what that is. And it probably wouldn’t require amortality, or otherwise a loving God would just make us amortal.

I don’t think any world religion’s version of this view is remotely plausible, but if it were true then I think there’s a pretty coherent idealized attitude towards death which does not involve amortality. I don’t think most people practice this view, but it’s the stance I’d take. So I’ll set aside God for now.

Bereavement

A common refrain from some folks I’ve asked is that they would not want to see their friends, their family, and their children die before them; they would only want . Implicitly, they are saying that these things are so awful that experiencing them repeatedly would make life not worth living.

I haven’t had much experience with this kind of grief, so maybe they are right. I can only make some observations about how other people behave, and raise some questions for proponents of this view, and try to remain sensitive.

First, notice that this perspective seems in tension with the argument that the fleetingness of life gives it its meaning. If you wouldn’t will bereavement on yourself, it’s unclear why you should think it is very good that your relationships are temporary. Fleetingness arguments would expect that bereavement is a positive thing.

Second, most people who experience such a tragedy do choose to continue living, and few people would tell a grieving friend to consider dying to end the pain. Bereavement is awful, but most who experience it don’t seem to think it is so awful as to make life not worth living. Nor is it generally considered a good idea to avoid all connection for fear of losing it again.

Third, dying prematurely also prevents you from forming and deepening other relationships, especially if there are other amortal people around. If a 50-year relationship can be so much more meaningful and wonderful than a 5-year relationship, what kind of connection could you develop after a relationship that lasts centuries? What kind of growth, what kind of intimacy, what kind of knowing the texture of another’s soul could you achieve? How many more deep friendships could you make? Unless you think human relationships are so shallow as to exhaust all of their wonder in a scant 80 years, you should think that love is a great reason to want to live longer.

Nature

Others say the extension of human lifespan is unnatural, and therefore for some reason bad.

Most of these people, however, do not choose to forgo medicine, agriculture, electricity, sanitation, central heating, or the internet. The typical complaint is that they have no choice, because without those things they will die. So be it: I choose amortality, despite its unnaturalness, because without it I will die.

Nature is also horrific. Chimpanzees commit infanticide. Parasites eat their hosts alive from the inside out. Wild animals die, but it’s more important to think about what happens before they die: they are starved, ripped apart, crushed, mutilated, while they still live. What is natural is to have as many offspring as possible, no matter how much agony and destruction it creates. That is what nature is. Enough has been said.

The social order

Sometimes people say that if old people were able to stick around for much longer, then the social order would collapse, that we would overpopulate, that Mitch McConnell would stay in office forever, and that would be bad, so everyone should die within 80 years of being born.

It is baffling to me to think that if amortality becomes an option, there would be no better social order than the one we have right now. Typically, people who raise this objection do not invest even five minutes in thinking of an alternative way things could be structured. They just visualize Mitch McConnell and stop thinking. Anarchists (socialist and capitalist) can empathize with my frustration.

Yes, there are dystopian futures where amortality is bad. Those are bad futures. I agree. We should try to stop them. In trying to stop them, we should try to come up with better forms of governance, better political organization, better moral norms, better institutions. We should not throw the towel in and let death reign any sooner than we have to.

Meaninglessness

Some say that without death, life is meaningless and choices don’t matter. What makes life “meaningful” is how “fleeting” or “temporary” it is.

If this is the case, why is no one petitioning to bring back old diseases, increase violent crime, and institute random domestic airstrikes to shorten average lifespans? If shorter lives are more meaningful, perhaps after the point you are old enough to make rational choices, then it seems like a max lifespan of 60 or 50 or even 40 would be much more valuable than the 80-and-climbing of the developed world.

But I don’t see most advocates for “fleetingness” bring up this point. I suspect most of them would think such measures were morally wrong. But then if it is wrong to shorten lifespans, and laudable to work in healthcare, what’s so bad about extending life further than that? I mean, what’s wrong with 150 years? Would you advise someone born with an unlikely genetic mutation which gives them a healthspan of 150 years to end their lives at age 80? Why?

Maybe by some cosmic coincidence, our exact lifespans in the developed west are exactly at the maximum of the value-length tradeoff. This seems improbable to me. (Why is it exactly that the reliability of heart tissue is exactly calibrated to the human meaning-span?) It is possible, but the exact coincidence of these two factors seems much better explained by human psychology to cope with death than an evenhanded attempt to evaluate how good a longer life would otherwise be.

Boredom

Closely related to meaninglessness are complaints about boredom, that after a short timespan one would simply get bored.

In my less charitable moods, I pity these people their attention spans, their incuriosity, and their lack of imagination.

How many people read all the great works of the Western canon before they die? Would that be boring? How many people struggle with ink and pen and tears to create a novel that speaks to their soul before they die? Would that be boring? How many people, through damned determination and belief in the might of the human spirit, work their way from couch potato to summit Kilimanjaro? Would that be boring? How many people have spend fifty years meditating in a solitary forest, pouring over Zen koāns, immersed in the beauty of nature, tracing the contours of their soul? Would that be too boring for you?

Yes, I think chasing shortsighted pleasure for centuries would grow tiresome. But with centuries to practice, you could develop centuries of skill, virtue, empathy, and determination. You could turn into a paragon of the human spirit, tackling hard problems and making your life’s journey an odyssey. Boredom? Not a chance.

The final option

Amortality, as opposed to immortality, has an escape route. If things do turn really bad—if you’re trapped in grief or torture or ennui or something else awful, then you are not stuck with that awfulness forever. I don’t assume this is costless, and I don’t take suicide lightly. But this does take the wind out of a lot of pro-death arguments: amortality is not literally forever.

It’s just saying that we shouldn’t expect the exact moment when your brain bleeds, or your heart gives out, or the tumors clog your airways and you suffocate to death, be the exact moment after which life would stop being meaningful.

Amortalists believe the best moment for death would depend on personal philosophy, informed reflection, narrative & life trajectory, relationships with others, and risks you choose to take. Not the replication rate of bone marrow cells.

Why on earth is that a radical proposal?

Irrationality

The best argument against amortality, and also the rarest, is that people won’t choose to spend their lives on meaningful things, meaningful even in the sense of their own preferences. That they’ll waste away their lifetimes on FutureTikTok despite knowing deep down that they would be happier doing something else. Or that they’ll get stuck in some addiction and be unable to escape, and their rationality will be so impaired that they won’t choose death when their live becomes negative.

I sympathize with this perspective, because I suspect irrationality and akrasia are the barriers to a much more meaningful life even with a limited lifespan. I am trying to do better, but I certainly don’t act rationally all the time in this respect.

However, in general, people can grow and learn and become more rational over the course of their lives. If you only get 80 years, you might not figure out the skills of resilience and mental independence you need to make good choices, but you might figure it out in 800 years. Low-grade, shallow pleasures get boring and exhausting after a while, and that might serve as the needed push towards self-improvement. Plus, for those people who do spend their lives meaningfully, helping others along the path to rationality and self-mastery would be a great purpose to dedicate some of that time to—helping other people break out of negative cycles and realize their potential. Like the social order concerns, I think irrationality is a problem, but solvable with a lot of hard work. Fortunately, extending lifespans increases the amount of time smart people can dedicate to that hard work.

But at the very least, grant intelligent, consenting adults the choice of amortality. Let us take the risk, bet our futures to strive for something deeper and more valuable. Isn’t that a braver alternative than just letting death come for you because you don’t want to make the wrong choices?

Why would amortality be good?

I’m better at philosophy than I am at evocative writing, but the greatest reasons for amortality are emotional so, here goes.

I will not try to sound profound. I am not trying to place anything or anyone on a pedestal, and you may roll your eyes at some of these things and I don’t care, they are things that make me happy and maybe you’ll sympathize with some.

Imagine spending a century in the biggest library in the world. Are you picturing how staggering it would have to be? Can you imagine curling up by the fire with a steaming bowl of lentil soup and warm, crusty bread and pages upon pages of Asimov, Adams, Bardugo, Chalmers, Christie, Cicero, Dennett, Dickens, Dostoevsky, Fodor, Frankish, Feynman, Hofstadter, Lewis, Mistry, Milton, Quine, Shelley, Schwitzgebel, Wolfram, Sapolsky, Taleb, Tao, Wodehouse, Wittgenstein, Wilson, Yudkowsky? Can you imagine pouring over every page, working it through your mind sentence by sentence, making and remaking your mind without a care in the world about time running out? Can you imagine knowing literature, knowing writing, knowing written science better than anyone in history? Can you imagine what it would be like to set your pen to paper and write and struggle and think and rest and write more until your masterwork is complete?

Oh, and can you imagine the other people in the library with you? The conversations you’d have, the discussions that would carry on for days, weeks, months, years, on a single page or on a hundred-thousand, if you like, and the flowering of the human mind you could bear witness to? Or the weeks or years where you get sick of thinking and join some of the others to find old corny movies and silly video games and laugh your asses off and stay up late talking and watching the sun set, because fuck it, sunsets really are beautiful and who gives a damn about cliché?

Can you imagine the kind of deep, bonesettling rest you could take in a sunlit home with tea and broth and hot chocolate and couches everywhere to nod off? Can you imagine stretching out in the sun like a dozing feline for weeks on end, feeling its warmth and letting your body settle into a deeper peace than its ever known before? Or how sex would feel, with no deadline looming over your head, with the freedom to rest in your lover’s3 arms and talk and laugh and find pleasure for as long as you like? Or long runs and happy, aching muscles and a body that deserves and finally has your full attention?

Can you imagine mastering breathing, mastering posture, mastering the control of energy and tone until you sing a pitch of emotion and power greater than anyone has sung that pitch before? Can you imagine working your hands in clay and brick and marble and stone until your work seems to breathe in the right light? Can you imagine conducting symphonies? Or planting seeds, and mixing soil, and tinkering with hyroponics, and growing thick colors and vibrant flavors from water and earth? Or kneeling in a spotlight, convulsing, railing against thunder and treachery, against the wickedness of daughters and the faithlessness of men, and embodying the madness of a tyrant king?

No. You cannot imagine all those things in the detail, the richness, the overpowering texture they could have if you were to live them. The luckiest among us get a couple of those things, in some form, some of the time, and they are the sweetest and greatest and most gratifying things in creation. Amortality is not wonderful because it is something distinct from all the wonderful things in life. It is the wonderful things in life, all of them, but richer and deeper and more. If you possess enough imagination to even get close—to even breathe a shadow of that life into your expected value calculation—then I think the choice is obvious.

Subscribe now

I have mixed feelings about Rob Long’s article on LLM consciousness arguments:

Here’s a frustrating dialectic that I see (or imagine I see) sometimes:

LLM-Sneerer: “You think that LLMs could be conscious? Absurd! Don’t you know that LLMs are just predicting the next token? Plus, they make stuff up; their preferences are contradictory; their introspection is unreliable; and they’re confused about who they even are!”

LLM-Defender: “Aha, well…aren’t humans just predicting the next token? Plus, humans make stuff up, and humans have contradictory preferences; our introspection is unreliable, and we’re really confused about who we are too!”

I’m glad that the Defender is pushing back, but ultimately both of them are imposing an unproductive frame on LLM-human differences.

The third position Rob is trying to carve out is:

1. Yes, LLMs are actually quite different from humans, and we should take those differences seriously in our theorizing. Differences in substrate, evolutionary history, embodiment, etc. might turn out to be really important!

2. But just pointing out that LLMs are flawed or different in some way doesn’t give you grounds to dismiss the possibility of their consciousness out of hand—and most Sneerer arguments are remarkably shallow.

This is eminently sensible, it is the position that I endorse, and I think the consciousness debate would be much better off if people thought more like Rob Long. So why do I feel some hesitance about this article?

Coming from my personal interactions on substack, seeing many of these disagreements, I think Rob’s portrayal of the LLM Defender is a little ambiguous. Imagine following up on this:

Aren’t humans just predicting the next token? Plus, humans make stuff up, and humans have contradictory preferences; our introspection is unreliable, and we’re really confused about who we are too!

with this:

(A) Therefore, your arguments are much too crude. Since they apply to both humans and LLMs, you aren’t really drawing a distinction between the two of them. You need to be more specific with (i) what precisely is different between LLMs and humans, (ii) why this difference is likely to be load-bearing for consciousness, and (iii) why you think this is so likely it justifies dismissing LLM consciousness out of hand, despite widespread expert confusion about what consciousness even is.

versus this:

(B) Therefore, we don’t have any good reason to think LLMs aren’t conscious; we work basically the same way, so there are no consciousness-relevant differences.

I think some real-world Defenders are saying (A), and some are saying (B), and Rob’s imagined quote really doesn’t distinguish between the two. (B) is silly. But (A) is pretty sensible, and the Defender’s arguments are really important for establishing (A). (A) is not equating LLMs to humans, it’s trying to argue that Sneerer arguments prove too much.1

The basic principle which arguments of type (A) support probably deserves a name. In honor of “no free lunch” principles in computer science, I propose the “no slam dunks” conjecture: there are no “slam dunk” arguments for or against LLM consciousness.

By a “slam dunk,” I mean a compelling argument which requires minimal nuance/analysis of differences between humans and LLMs, but which is so effective as to dismiss some stance on consciousness out of hand. A “brick” is an uncompelling argument of the same type. Typically, consciousness bricks fail because they prove too much or fail to explain how some substantive observable difference entails a difference in consciousness.

“LLMs can’t feel anything, they’re just machines” is a brick. What’s a machine? What makes an LLM count as a machine and not a human? Why would being a machine disqualify an entity from consciousness? In order to provide an argument, you actually need to specify answers to these questions, which requires (gasp!) actually reading things about how computers and brains work and doing comparative analysis! The horror!

Or, to put it another way: if you want to say “LLMs can’t feel anything because they are machines,” you have to provide a reason why machines can’t feel anything but humans can. If you want to say “machines can’t feel anything because they are made of switches and wires”, you have to provide a reason why switches and wires can’t create feeling but neurons can. To do so, you actually have to look at properties which neurons have and switches and wires cannot, and give plausible reasons why those properties are necessary for consciousness. Now we’re firmly out of slam dunk territory; you actually have to do nuanced neurophilosophy to succeed here.

“LLMs are just token predictors” is also a brick, for reasons Rob explains quite well:

[E]ven granting that LLMs are doing next-token prediction, it doesn’t follow that LLMs are “just” doing that. As Grzankowski, Keeling, Shevlin and Street 2025 argue, that “just” can smuggle in a dubious argument: that the low-level, token-prediction explanation invalidates any other higher-level explanations of LLM behavior, like explaining their behavior with belief or intentions. But in general, low-level explanations don’t crowd out high-level explanations. Money is “just” a bunch of paper and metal and electronic records, but this low-level explanation doesn’t invalidate higher-level explanations, e.g. using prices to explain what people buy.

So again, if you want to say “LLMs are just next-token predictors, therefore they aren’t conscious,” you actually have to explain why the low-level description of next-token prediction rules out a high-level explanation in terms of thoughts, feelings, experiences, etc. Otherwise, I could just say “well, your brain is just computing the next neural response, therefore you aren’t conscious.” Again, maybe there is a detailed, successful argument here, but certainly no slam dunk.

Why do I believe there are no slam dunks? Several reasons:

• I have seen many attempts at slam dunks for or against LLM consciousness. All of them have turned out to be bricks on closer inspection.

• There appears to be widespread disagreement and confusion among expert philosophers about what consciousness is; there isn’t even an agreed-upon folk definition. Therefore, any slam dunk argument for/against LLM consciousness would need to either (a) succeed for a wide range of disparate theoretical positions, or (b) also be a slam dunk against almost all of those positions. This seems implausible.

• If there were a slam dunk argument, then a professional philosopher of mind is more likely to discover it than a random substacker. But I have not seen any slam dunks from professional philosophers of mind. The closest thing to a general attempted slam dunk against machine consciousness is Searle’s Chinese room, to which there are many replies. Even if Searle’s argument is correct, it seems to require further argumentation to be convincing, and is therefore out of slam dunk territory.

• Consciousness doesn’t seem to be the kind of thing where slam dunks would likely exist. For instance, either consciousness is fundamentally private, or it isn’t. If it’s fundamentally private, then how can you be confident LLMs are/aren’t conscious just by looking at their physical observables? It seems you could only gather abductive evidence. If it isn’t fundamentally private, then one would actually need to analyze how LLMs work in detail and compare them to how brains work in detail to see what it is that brains have and LLMs don’t (and why that thing would be consciousness-relevant). But then you’re no longer in slam dunk territory.

Am I beating a strawman here? Among most professional philosophers, yes. They are smart and well-read enough to realize that consciousness is difficult to study and requires precise argumentation, not flashy pro/anti-machine slogans. Even skeptical professionals like John Searle, Anil Seth, and Ned Block do their homework. I think Erik Hoel is wrong and deeply confused about consciousness, but he actually provides a substantial argument for his position.

But among all those who read and write substack phil-mind… sadly, no. I’ve seen loads of bricks presented as though they are devastating arguments which one would have to be an idiot not to see.

Now, I don’t think I’ll succeed at persuading these people that looking for slam dunks is useless. Convincing dogmatic die-hards is rarely the point of argument anyway. But I do think I stand a chance of convincing those who haven’t made up their mind yet not to waste time with snappy one-liners, chasing a kind of instant-gratification certainty which doesn’t exist.

Subscribe now

DISCLAIMER: I am not a mental health professional. This is just one strategy for general quality-of-life meditation that has worked for me. If you are experiencing extreme negative emotions and you think anything I am advocating here would encourage you to tamp those down to treat them as “fake”, this is not an approach that will work in that scenario and you should talk to a professional, not an undergrad in philosophy of mind.

Meditation has made me a happier and clearer thinker than any single factor since getting a long-term partner.

I have tried meditating at many sporadic times over many years, and read lots of books on it, and none of them worked. Then in late January of this year, there was a sudden phase change and everything made sense and I am just much less stressed, as a busy Princeton student, then I have been since high school at least, and maybe before that. I can handle a lot now!

Far and away the most important factor in meditation working well was taking illusionism seriously, and thinking like an illusionist. (If you don’t know what illusionism is, we’ll get there.) Thus, I call what I do “illusionist meditation”, even though it is far from original and you do not need to be an eliminitavist/quietist about qualia to practice it.1

Since most meditation I tried before this was close to futile, and this kind of meditation has been a step-change in my well-being, I figured it’d be worthwhile to write down in case it helps other people.

Step 0: how meditation advice works

I started taking voice lessons about a year ago. For many months, I heard stuff like: relax your throat, stay centered, support your voice with your breath, don’t strain, etc. I tried and tried to do those things, and never got them quite right. Then, one day, after lots of nudging from my vocal coach, I actually did sing that way, and it felt way better, and it all made sense to me in the moment! What was I doing? Well… I was relaxing my throat, staying centered, etc.

I had no new insight that I could put in words. My vocal coach didn’t tell me anything I hadn’t already heard in high school chorus. Rather, her advice, coupled with a lot of practice, nudged me towards a place where I could actually feel it. This is ineffability, if you like, but it’s the ordinary kind of ineffability, like riding a bike: you can read a lot about how to ride a bike and still struggle to “get it.”

Think about a Rinzai Zen kōan, like “What is the sound of one hand clapping?” It’s not as though the acoustic properties of palms are actually that important. It’s that thinking about that question really hard, bringing your full attention to it, meditating on it, etc., and actually setting solving it as your goal (not “trying to solve it”), which triggers you to break through into a new way of thinking.

All this to say: I am going to give some advice, and offer a lot of references for mindset, which may sound strange or banal or make you think, why would this work? My answer is: it’s all triggers. It’s all about trying to put you in the right state of mind where you can do the damn thing, and get to know it by experience.

Do not expect reading this blog post, without doing the things in this blog post, to be revelatory. Do not expect it to sound profound. Do not expect anything to work unless you actually do it. Set yourself the goal of doing it. Do not set yourself the goal of “trying to do it.”

(If those sound like the same thing, they aren’t, and you should read this and this before proceeding.)

This will take Actual Work. If you put in the Actual Work, though, I think it will work. You could probably do it in three weeks, if you afford yourself lots of time for your mind to wander in between doing interesting things—and you absolutely do not need to set everything else in your life aside to fit in the Actual Work. In fact, that’s probably a bad idea. But if you are unwilling to put in a couple hours a day for a short period of time, sorry, I can’t help you.

Step 1: get in the illusionist mindset

As I said, you do not need to be an illusionist to practice illusionist meditation. You do need to be able to take illusionism seriously. You need to be able to, approximately, imperfectly, and temporarily, get into the headspace of a smart person who actually thinks illusionism is true.

I am about to present one version of illusionism, the one which is most intuitive to me when I am in the right meditative mindset. If you are a skeptic about illusionism, from “seems false” to “obviously incoherent,” please, please, please try to suspend that judgment right now; even if you are right, if you are already assuming you are right or looking for arguments, this won’t work. If you need to, think of this as a trick to get into the right mindset—but the trick only works if you actually take it seriously, and don’t just “try” to take it seriously. Here we go:

• We have experiences. We see, hear, smell, feel.

• We make judgments and form beliefs about the world around us. Often, the beliefs we form are associated with our experiences.

• Sometimes, our experiences lead us to err in our beliefs.

• The first-order kind of error is when our beliefs are correct about our experience, but wrong about the world. I see two lines on a piece of paper, and one looks bigger than the other, so I infer that in reality one really is bigger than the other. In actuality, they are the same size; I am wrong about the world, but not necessarily wrong about the nature of my experience.

  

• The second-order kind of error is when our beliefs are incorrect about the contents of our experience.

  • I believed for a long time that I see in full color throughout my visual field—not that the colors were always accurate to reality, but that I experienced a full-color visual field. I was wrong.

  • Pete Mandik was once on acid, watching German television, and thought he could understand what the people were saying… but he couldn’t tell you what any of the words meant, not even a hallucinated idea of what they meant. He was wrong. You could form false beliefs about your own experience.

• Say it again: You could form false beliefs about your own experience.

• When many people encounter the “hard problem of consciousness”, or think about “philosophical zombies,” or think about robots and functional states, introspect and form a belief: that their experience cannot just be a functional state. That it must be something private, ineffable, intrinsic, and directly accessible to them.

• True or not, this is a belief.

• And you could form false beliefs about your own experience.

• Therefore, it is possible—in principle!—that your judgment that your experiences are private, ineffable, etc. is a false belief about your experience.

• Many things about the “mind-body problem”—interaction, brains, self-reporting, fallibilism, etc.—begin to make a lot more sense if you take this perspective.

This is a really difficult position to get your head around. Make the attempt, and please withhold judgment for now. You need to grok at least a little bit of this for the meditation to work.

If illusionism seems silly or baffling or not quite right to you, do not try to understand it by picturing in your head what illusionists must be thinking, or thinking about the word “illusion” and what it typically means. Instead, actually read what illusionists are saying. I recommend first reading these blog posts:

Both Sides Brigade

Qualia Realism is a Really Weird Type of Realism

A few weeks ago, getting high and reading some Keith Frankish led me to finally admit something that shame and fear have led me to suppress for years: I’m an illusionist. But ever since this momentous (and, dare I say, heroic) admission, I still find myself wondering at times if i…

Read more

3 years ago · 11 likes · 24 comments · Both Sides Brigade

Then, these four papers:

1. Dennett, “Quining Qualia”

2. Frankish, “Illusionism as a theory of consciousness”

3. Kammerer, “The hardest aspect of the illusion problem — and how to solve it”

4. Mandik, “Meta-illusionism and Qualia Quietism”

Yes, you actually need to do this. Read these seven articles. Try really hard to understand them. Interrogate Claude if you need help; feed Claude this very post too, and insist that he steelman the illusionist perspective as hard as possible. If you don’t make a serious attempt at this, you won’t get it, and the meditation tips probably won’t work on you. So don’t complain that illusionism doesn’t make any sense until you’ve put in the work!2

If you work hard and keep an open mind, I think you can get to a place where (a) illusionism actually seems like a serious theory, (b) it still really seems false on some fundamental level, but (c) you can get yourself to a mental headspace where you don’t completely shut it down.

Congratulations! That’s where I am! That’s the right mindset for illusionist meditation to start working! You don’t need to take illusionism to be true or even likely, you just need practice to get yourself around to that general way of thinking.

If you don’t feel like doing this work, and you read the sections below, you’ll be tempted to draw the conclusion that you don’t actually need to do it at all; the later advice doesn’t appear to depend on illusionism. If you are thinking this, go read step 0 again. The point is that the later advice, coupled with illusionism, trigger the right kind of mindset, and based on my experience you do actually need the illusionism to get there.

Step 2: basic meditation modality

The basic practice of this kind of meditation is:

• Get your body in a reasonably relaxed sitting position, in an environment with few distractions.

• Breathe normally. Don’t try to force big, deep breaths; just relax your stomach and let yourself breathe there.

• Just notice and observe the thoughts and sensations in your mind. Don’t try to quiet them, or chase them down. Just kind of… watch them, like a detached observer.

This takes practice, so don’t sweat it if it seems awkward. Lots of standard meditation guides can help with this part; Pema Chödrön and Thích Nhất Hạnh write good stuff.

Now, I actually journal while I do this; I write down my thoughts, or dictate them aloud. I don’t worry about being precise or neat or well-organized, it’s really just a dump of observations. It helps me examine them better for the next step, but YMMV; I could see it feeling easier or harder to do this way for different people.

Step 3: detach, decouple, step back

The illusionist perspective is: don’t assume you have direct access to what the state of your mind is actually like. All that is floating around you in the mind-space are sensations, and beliefs about those sensations. Those beliefs, the content of those beliefs, will try to tell you a coherent story, something consistent, something that paints a unified picture. But actually, there really isn’t unity there. Don’t assume your experience must be coherent. Don’t assume that your sensations carry coherent propositional content. So you have to question the nature of your observations: what actually is it that you are observing?

It’s best to illustrate this through examples. I used to believe in something I labeled “cognitive exhaustion.” I had this belief, for a long time, that willpower was like a battery that could be depleted; that if I worked really hard on a math problem, my “willpower battery” would run out and I would need to recharge. I believed, basically, in ego depletion theory, despite knowing that it fails to replicate experimentally. So I was working on some hard combinatorics problems for about half an hour, and sure enough, I got that feeling of cognitive exhaustion; it was time to stop and let the batteries recharge.

But then I stopped, and entered this observer headspace. I thought: I have this belief, now, that I am out of “willpower juice.” It could be true. But it also could just be wrong. Let’s not assume that I have direct access to some kind of truthful will-o-meter. So: Why do I have that belief? What’s behind it? Things I noticed:

• Bodily sensations: a feeling of pressure, or heaviness, around the temples.

• A feeling of restlessness.

• A disposition to do things like: play a video game, watch YouTube, lie down and listen to a podcast. Things I might label as passive activities.

• A disposition to not do things like: read a book, keep working on the math problem, learn about ML with Claude, do homework. Things I might label as cognitively effortful.

• A strong belief that I am running low on some kind of resource.

So this is what cognitive exhaustion is, not a unified primitive which I have, but a loose bundle of things which I label as one thing. Are we done?

Nope! Not at all! Time to ascend one level of abstraction, and start questioning these things I noticed.

• What precisely does it mean to say that I feel pressure around the temples? What actually is that sensation? Is it localized to a specific region? Where are its boundaries? Is it uniform? How intense is it? Why do you label that sensation as “pressure”?

• What the hell is a “feeling of restlessness”? Is it a belief that you have restless energy, whatever that is? Is it a bodily sensation? If so, what sensory modality, and where is it located? Is it just noticing an impulse to get up and do something? Do what, exactly?

• Similarly, what is a “disposition” introspectively? Does it mean that you judge it would be best to go watch YouTube? That you start thinking about watching YouTube without conscious intention? That the thought of watching YouTube gives you a pleasant feeling, and the thought of not watching YouTube gives you an unpleasant feeling?

• How do you observe a disposition not to do something? Are you disposed not to punch a puppy, or are you simply not disposed to punch a puppy? What difference is there, and how would you detect it?

• Do you actually know which one you were talking about, or did you have to think hard to even decide? If so, is it possible that you’re mistaken about even having such a disposition, beyond believing you have a disposition?

• What precisely does depletion of this resource do? What happens if it runs out? Does my work get objectively worse? Does it take me longer to make the same connections? Is it merely less “pleasant” in some sense? (What sense?) Can’t I just choose to keep doing it anyway?

Imagine fusing Dan Dennett and Socrates in your head; question everything, and take no fact about your mental state for granted. Or rather: start on one level, observing your first-order thoughts and taking the observations for granted, then “kick away the ladder” and question your second-order observations, and so on.

You can’t keep on like this forever, but you generally don’t need to. After a couple rounds of this, my working hypothesis is: when I spend a long time doing hard mental work, I feel pressure around my temples—and then a long chain of inferences about what that means, and what I want to do, and what I should do, gets set off, and those inferences paint this picture of a scarce mental resource which depletes, and the lower it runs the “harder” it is to do good mental work (whatever that means). This could actually be a helpful signal, or not. So then I tested it, by putting on a Focusmate session and reading a philosophy paper for 50 minutes while in this “exhaustion” state. The feeling of pressure stayed, but the quality of my comprehension and writing did not degrade—and I was able to concentrate the whole time.

Similar things happen with stress. Looking at a to-do list stressed me out—or rather, I had the belief that looking at a to-do list stressed me out. But once I actually opened mine, in this meditative state, I didn’t actually notice the physical feelings getting much worse. Instead, I had gotten myself in an anticipatory tangle of thinking that I was going to be stressed out, which led me to avoid the thing. But stepping a couple levels back and realizing that it was a belief, and in fact a wrong belief, dissolved a lot of that tension.

This has two wonderful effects.

1. In cases like so-called “cognitive exhaustion”, I learn that I am capable of doing more than I thought I was. It was false beliefs about capacities, not actual capacities, which were holding me back. And indeed, I’ve been able to concentrate for a lot longer since!3

2. The act of doing the meditation often dissolves the negative feeling in the first place. I don’t just understand the feelings of stress, I feel less stressed afterwards. It’s something about the distance, the stepping back and picking a part, and the noticing of how loosely all of it hangs together, that lightens my burden. It’s a huge qualitative shift.

I describe this as a feeling of “detachment”, and that sounds negative, suppressive, or robotic to some people. But there really isn’t any forcing or suppression happening. It feels most like dredging feelings up, exposing them to air and light, and seeing them dissolve or change. Or hovering lightly over a swirling river. This is still a work in progress, but there are times where I really am able to achieve a Flo Bacus level of control and detachment, and it is awesome.

Step 4: quit or practice

This is not a “you’ll get there someday” kind of practice, where you put off being tranquil into some future when you’ve got everything else figured out. If you read and understand the papers, and try (for real, not like Luke Skywalker) regularly for several weeks, and turn around the philosophy in your head, and you’re still getting nothing, it’s probably not going to work out for you and you should try something different.

For me, it took a month of reading illusionist stuff out of intellectual curiosity, plus a couple days of experimenting with meditation, to stumble upon this method. But once I started, I experienced an immediate and lasting qualitative shift after maybe half an hour of meditative journaling. I was far from done—there were still lots of sources of stress, lots of things to work out—but it was extremely easy to point out measurable progress. I did not need to force myself to practice: I found it pretty fun and interesting to do whenever I wanted to raise my quality of life. So now I’m getting better and better at it! And it’s fun to experiment with: what other emotions and sensations are going unexamined? What other ways might I be limiting myself in ways I don’t need to? What if we turn this attention to positive, not just negative, emotions?

Let me know if you give this a try; I’m really curious to see if it transfers, and it’s a pretty cheap experiment to run personally. For the returns on your well-being, it’s definitely worth a shot.

Subscribe now