For one thing, many dramatic improvements require zero effort from those who will benefit (like with faster communication, shipping, computers, etc.) This is no accident, of course. When a company creates a new product, they must make it easy to use, and take advantage of customers’ existing behaviors and expectations.

Also, human minds can perform many creative feats without any conscious effort. We can walk up stairs without having to think about it, even if they’re unlike any set of stairs we’ve seen before (e.g. a different color, height, surface texture, etc.)

So, while many improvements require no direct behavior changes at all, many more require only imperceptible ones, like adjusting to a slight change in stairwell design. It turns out that billions of people can adapt to thousands of changes over the course of a year, and never be overwhelmed by any of them. A single trip to renew your driver's license can cause more angst than riding this exponential wave.

Having said that, not all changes are imperceptible or effortless. Some are difficult and frustrating, but I think these are the exception.

And one has to distinguish between those who *make* the improvement and those who *use* it. Using an improvement requires many orders of magnitude less effort than making it. So, while one person spends years creating a new technology, millions can take advantage of it within moments, potentially.

Ditto for prime numbers. There are infinitely many of those too, and infinitely many integers. So, there are as many primes as there are integers. Prime numbers aren't rare in that sense either.

However, that's only true if you talk about primes and integers in the abstract.

If you look at the number line, primes do get rarer. And, if you alter primes using familiar operations (adding, subtracting, multiplying, dividing), you almost always break the prime, turning it into a non-prime (aka "composite") number.

There's no number line for ideas, but you can talk about altering good ideas in familiar ways (adding, subtracting, switching parts of it), and this usually breaks the idea, turning it into a non-good (aka "bad") idea.

So, in this sense - perturbation by familiar operations - primes and good ideas are rare. Or, not "rare", but easy to break. As David Deutsch would say, they're hard to vary.

Actually, good ideas are rare. There may be infinitely many of them in the set of all ideas, but consider a different set: the set of variants. In that set, good ideas are indeed rare.

There may be infinitely many ways to design an excellent computer, but most ways of changing a good design will just break it.

Maybe yet another way to put this is: good ideas aren't rare, but good variations or alterations are rare.

When thinking about the mind, it's helpful to consider five nested sets, where each one is a subset of the next:

Active ⊆ Existing ⊆ N-Reachable ⊆ ω-Reachable ⊆ Possible

1. Active. The set of things currently active in the mind. This is usually a tiny set, focused only on what's relevant at any given moment. The fact that we can focus like this helps us solve problems more efficiently than just trying things at random.

2. Existing. All the knowledge in the mind. Many of these things haven't been used in years, but they're in there, and might be used if they suddenly become relevant. Ideally, you'd have vast knowledge, and know exactly when to use each thing - when it should become active.

3. N-Reachable. All the knowledge you don't yet have, but might reach in N steps. e.g. All the things that are one thought away, say. The point here is that many things are just a few thoughts away. Also, if your mind contains good ideas, then many improvements might be just a few steps away.

4. ω-Reachable. All the things you can ever reach, given unlimited time and resources. The point here is that the human mind might be able to create any thought, while AIs or raccoons might never reach those things, even with infinite time. (The symbol ω is "omega" and refers to infinity.)

5. Possible. All the things that are possible in reality. Some systems can do all these things - perhaps we humans can. If the set is finite, then there is a best possible state, and things can never improve after that. If the set is infinite, then perhaps things can get better forever.

It’s creation and discovery. You construct an alternative way to represent or approximate a thing, often using new building blocks that aren’t even in it.

e.g. 10101010101010101010 can be compressed as: “10” x 10.

Note that the original doesn’t contain quotes “” or mathematical symbols like x.

In other words, you didn’t merely throw out some of the original, but added new things it never contained, like a notion of multiplication or repetition.

Likewise, when we “compress” the complexities of the world, we aren’t just throwing out large parts of it because it’s too much to handle. We are introducing new, powerful ideas it never contained.

Computers are unlike every other machine. We routinely use or create new software. That is, we get a machine which we bought years ago to do something new - only thought of yesterday. That's like buying a car, and then years later, easily getting it to bake a cake, because it turns out that it can do that too, oddly enough.

And people change. Radically. Like a computer, it turns out they can speak one language, and later learn another. While "personality" may seem consistent over time, you have almost zero ability to predict the details of what a person will be thinking and doing in a year, or in a day. That's unlike every other organism. A squirrel doesn't eat acorns today, then open YouTube to learn a recipe, and cook acorn pie tomorrow. It's acorns all day, every day.

Computers and people are weird, weird, weird, and for much the same reason. They have a hidden hyperflexibility.

This is true for any system.

• a giraffe, evolving to be taller or stronger

• an AI, training to be better at math problems

• an animal, playing and learning to fight

• a human, pondering new recipes or new physics

Novelty

If you try new things, there's no guarantee they'll be better than what you started with. But, if you never try anything new, you ARE guaranteed to never improve yourself. (Though things outside your control may make your situation better or worse, like a war or random cash giveaway.)

Novelty just means doing something new. That said, there are different measures of novelty. A box of bouncing molecules is always doing something new - you'll never see the molecules in the same configuration twice. But, if you ask what it looks like, it always looks random. In that sense, it never does anything new. There are infinitely many ways to look at the box and what it's doing. Infinitely many measures of novelty.

What is a measure of novelty? It takes a thing - a box of molecules, say - and asks a question about it which has some number of different answers. Here's one question: Does the box contain life? That has two answers, yes or no. If the box doesn't contain life, and then it does, then it the life question gets a new answer. By that measure, the box has done something novel.

A different question might be: what is the temperature of the box? To achieve novelty in this sense requires the box to reach a new temperature - something higher or lower than it has before.

A musician never really creates new notes, but they can make new melodies, or inspire new feelings.

Taste

Like measures of novelty, a system needs measures of quality. Which animal is fastest or strongest. Which machine learning model is best at face recognition. Which recipe tastes best. Which scientific theory make the most sense. Without such measures, a system is exploring the possibilities at random, and unlikely to find good stuff, and especially unlikely to recognize good stuff and focus on it.

Control

It's one thing to have a sense of quality - what's good and what's bad, and in what way. But, you have to be able to *do something* with that information. If you couldn't, then you'd be like a consultant at a company that nobody listened to. You might know what to do, but that will have zero impact on what happens. If the company is making a mistake, then they'll go on making it.

When a lion catches a gazelle, two things happen. First, it has established the gazelle is slow (either at running, or at recognizing danger), and then... it eats the gazelle. That removes its genes from the gene pool. It is a biological consultant that is being listened to. The alternative would be that the lion catches the gazelle, then lets it go. That gazelle has been found to be slow, but that information gets ignored by evolution, and the fast gazelles and slow ones reproduce equally, and the species never evolves to become faster.

Memory

To make continued progress, you need not only to *make* improvements, but *keep* them, so you can build on them. If you have amnesia and your library burns down every day, then you have to start from scratch, and can never discover or create anything that requires more than a day of work. If you want to invest centuries into a cathedral, or a billion years into ever-more-complex living organisms, you'll need to save your work along the way. The bricks of the cathedral need to stay where you put them. The genes that specify an organism need to persist - either in one instance, or in copies. Otherwise, evolution would have to start from scratch - and giraffes can't be built in a day.

Without novelty, things remain the same. Ask: what are the sources of novelty in the system? What kinds of novelty?

Without taste, the good and the bad are indistinguishable. Ask: how does the system distinguish the good from the bad? What measures does it use? Can these change over time?

Without control, taste has no impact. Ask: if a system has taste, how does that impact what happens? Once something is determined to be good or bad, what is done with that information?

Without memory, all prior work is lost. Ask: when something happens in the system, how does affect the distant future of the system? How are the results saved? Where is the information stored?

Lose any of these four things, and sustained progress is impossible. You need all four. That said, they can each be achieved in innumerable ways. Evolution did them one way. Animals like giraffes and chimps do them in other ways. LLMs yet another. And humans achieve them in a uniquely powerful way - and that's why a human is writing this, a human is reading this, and humans invented the hardware and culture involved in making all that happen.

Some news: First, I made myself a lovely lightboard, as you can see below. Secondly, I used it to make an hour-long introduction to the ideas of David Deutsch’s The Beginning of Infinity.

Read more

Some news: I’m hosting a 7-day Popperian workshop next week - starting on Monday - and you’re invited! Just click the link at the bottom of the post.

Read more

After all, life on our planet’s had 3,700,000,000 years to get to the moon. Yet it only did so 54 years ago, when we reached it. Why? Two reasons.

Universality

Unlike every other animal, we can extend our abilities without limit. We can think and do anything that is physically possible, once we figure out how. That’s why it was possible in principle for us to reach the moon.

Efficiency

We can efficiently find good ideas and actions, despite the fact they are stupendously outnumbered by bad ideas and actions. That’s why it was possible in practice for us to reach the moon.

Why do human minds possess this unique universality and efficiency? The full answer is still a mystery. Nevertheless, we can break these mammoth mysteries down into more manageable pieces, and that’s the aim of this book.

Are humans really special? Why care?

They are, and I hope to show precisely why. I think our minds are superior to any other animal or algorithm yet discovered. Each individual among us has literally unlimited potential for thought and action. This also makes human civilization uniquely powerful. I think these facts ought to make us optimistic, both as individuals and as a society.

I’m optimistic that, by understanding our minds, we can upgrade them. After all, despite our success in creating tools like paper and computers, we have not upgraded our mental hardware - our brains - in the entire history of our species. While Apple releases a new iPhone every year, we’ve been stuck with the same brain for 300,000 years. It’s embarrassing. And it matters, because upgrades could:

• Make death optional - not mandatory

• Allow us to create, explore, and inhabit wild new virtual realities

• Speed up our thinking, giving us more time to address the dangers of the cosmos

Lastly, you should care about human exceptionalism because it’s so damn interesting. In the attempt to explain it, you’re driven to consider so many different fields and areas of human concern: epistemology, evolution, progress, computation, language, logic, neuroscience, machine learning, and more. After 5+ years of research, I still feel like I’m only just beginning!

What’s next: reading, writing, and remunerating

Reading. I hope to share a new section of the book each week. If you want these delivered to your email, subscribe below:

Subscribe now

Writing. I’m always on Twitter, so tweet at me with your encouragement, feedback, questions, and ideas.

Remunerating. My personal savings were enough to support 5 years of full-time research, but now I need your help to continue. To cover my living expenses, I need 200 supporters, but I currently only have 10.

To become the 11th, just click the button above and pick a plan. (The annual plan is best for me, since it provides me with $120 immediately, and my savings are quite low at the moment.)

By supporting me financially, you’ll:

• Accelerate the writing of this book, by letting me continue working on it full-time

• Fund fundamental, curiosity-driven science

• Allow me to share this book freely for anyone to read

• Have a well-earned place in the book’s acknowledgements :)

Speaking of acknowledgements, I’d like to thank my supporters!

1. David Kedmey

2. Neil Hudson

3. Daniel Gray

4. Brad Ingarfield

5. Gleb Posobin

6. Brian Joseff

7. Tom Hyde

8. Adrien Chauvet

9. Velimir Gasparovic

10. James Titchener

11. You?

I used to have trouble finishing songs. Musical variation helped. But, out of fear of being boring, I avoided repetition. The results were often chaotic. So, to create more coherent music, I've thought about how and why to use more repetition in my songs. (I’ve also looked at some Rush songs to see their approach.)

Repetition is boring. Or is it?

Repetition is boring.
Repetition is boring.
Repetition is boring.
Repetition is boring.

That sort of exact, continuous, predictable repetition is boring. But, repetition can also be inexact, discontinuous, unpredictable, and not boring:

Repetition is boring.
Repetition is boring.
Not always!
Not today!

Repetition isn't boring?
Repetition isn't boring?
Not always!
Not today!

Repetition is boring!
Repetition is boring!
Not always!
NOT TODAY!

Better! Let’s look at creative ways to repeat an idea, some of which I've used above.

Approaches to repetition

AA' - Change the idea slightly.

ABA - Return to the idea after exploring others. This can provide a strong sense of coherence, closure, and homecoming.

ABACA - Return to the idea again and again as you explore. This ties a song together, giving it coherence.

ABABA - Alternate between two ideas. Incidentally, this creates strong expectations you can then break for dramatic effect. Rush's La Villa Strangiato alternates awhile, but then surprises you when the now-familiar transition leads someplace new.

AB AB...AC AC - Given an idea with two parts A and B, reuse only one part later.

AAAA...AAAA
BBBB.…CCCC - Bring an idea back in a new context (e.g. a new beat).

ABAB (ABAB)’ ABAB (ABAB)’ - Group an idea with others, then repeat the group. You can do this several times, ending up with a group of groups of groups, etc. In Rush’s YYZ, one section is built by repeatedly doubling just two short ideas - from 2 to 4 to 8 to 16 bars:

1. AB - start with two complementary ideas

2. ABAB - group is repeated

3. ABAB (ABAB)’ - new group is repeated, but transposed up

4. ABAB (ABAB)’ ABAB (ABAB)’ - new group is repeated

C………
B…B…
AAAA - Loop layers at different rates (e.g. every 1, 2, 4, or 8 bars). The 16-bar intro to Rush’s Bravado consists of three overlapping loops:

• 8-bar bass (repeats 2x)

• 4-bar drum (repeats 4x)

• 2-bar guitar (repeats 8x)

You can mix and match approaches:

ABCD EFG ABCD - In Rush's Tom Sawyer, the first third of the song has many parts, and the whole sequence is repeated again at the end. This is a sort of ABA pattern where A and B aren't individual ideas, but groups of ideas.

ABAB CDEFG (ABAB)’ - Group-and-repeat A and B, then return to the group after exploring other ideas. Instead of returning to the exact same idea, return to a slightly-changed version. This combines three things: group-and-repeat, leave-and-return (ABA), and change-it-slightly (AA').

Purposes of repetition

Appreciation. A good idea is worth hearing again - maybe many times. If it flashes by too quickly, the listener may not have time to understand it, much less appreciate and enjoy it. Ask yourself: Have I given the listener enough time to appreciate this idea?

Building. Once the listener is comfortable with an idea, you can build on it without creating confusion. This ultimately allows you to work your way up to more complex, multi-layered textures. Have I given the listener enough time to understand this idea?

Effort. By repeating ideas, you don't have to come up with so many of them, and the listener doesn't have to work so hard to understand them all. You can both relax. Of course, you want to challenge yourself and the listener at certain moments, but not every moment. Have I given the listener enough time to relax?

Surprise. Changes are more striking when the music has become somewhat predictable. Have I given the listener enough time and clarity to form expectations, so that the next change will be a special surprise?

Returning. If you get the listener familiar with an idea, it will be recognizable later, and can provide a strong sense of coming home, especially at the end of a song. (Meanwhile, returning to an unfamiliar idea just sounds like something new.) Has the idea had time to stick in the listener's memory?

Legitimacy. Once is a mistake. Twice is jazz. A weird, new idea can sound arbitrary. Returning to it can make it sound more like it belongs. (Rush’s Spirit of Radio does this with an unexpected reggae section at the end.) More generally, you can repeat it in different forms, perhaps hinting at it in the background or otherwise weaving it into the fabric of the song. Have I made this idea sound like it belongs in the song?

Coherence. When an idea is used throughout a song, it ties everything together. By mixing, alternating, or surrounding new material with familiar material, you can avoid things sounding arbitrary. Often, ideas from the beginning show up at the end. Have I tied the song's parts together? What material would be best for that? Where should it be placed?

Distinctiveness. You can fill a song with a hundred ideas that occur only once, but then it has no signature ideas. Ideas that stand out. Ideas you've become familiar with. Ideas that you can recall and recognize. Focusing on a few heavily-repeated ideas gives the song its signature. Does the song have any signature ideas? Any memorable hooks? If not, which ideas could play that role? How can I focus more attention on those?

Conclusion

With a clearer picture of the possibilities and purposes of repetition, I think I can look forward to trading musical chaos for coherence!

Repetition is boring.
Repetition is boring.
Not if it’s clever!
Not if it’s creative!

Never repeat!
Never repeat!
But it’s chaotic…
But it’s confusing…

Repetition is boring!
Repetition is boring!
Not if it’s clever!
NOT IF IT’S CREATIVE!

A melody isn’t just a melody. A riff isn’t just a riff. A bassline isn’t just a bassline. With the right knowledge, each one is a gateway.

A gateway to more ideas, so you can finish songs instead of getting stuck.

A gateway to better ideas, so you can make more interesting songs. More emotionally powerful songs. Songs that, at every point, are tuned to provide just the right levels of novelty and coherence. Never boring, never chaotic.

Variables

If an idea is a gateway, how do you open it and walk through? First, stop seeing an idea as a single, complete thing. Like a machine with many parts, every musical idea consists of many choices - each of which can be changed.

Here are ten such choices: pitch, rhythm, register, articulation, harmony, dynamics, timbre, key, tempo, and meter.

That makes ten musical directions you can travel in. Ten new ideas, just a step away. Pick one and you'll be standing on a new idea, with ten different ideas surrounding you. Just two steps away, hundreds of ideas are in reach. Three steps, thousands.

(In three steps, for instance, you could turn a slow, high-pitched piano melody into a fast, low-pitched guitar riff.)

Working with notes

Above all, you'll work with notes. You'll add and remove them. Make them longer and shorter. Move them up, down, left, and right. By different amounts. Individually and in groups.

These operations are difficult to do in your head. They’re easier with a music notation program like Dorico, where you can see every note and you’ve got total control over each one.

Indeed, it can be a revelation to work visually. To think in terms of notes and contours, not fingers, fretboards, or gut feel. You can see the shape of a melody - the peaks and valleys, the steepness of its rises and falls. You can see the busy spots and the open spaces. You can see how to split an idea and rearrange its parts. You can end up creating things that would never occur to you otherwise.

By the way, if you can use a program like Ableton, you can use a music notation program. You don't need to sightread music. (I can't.) Instead, Dorico plays the music for you while you compose, and, when you want to record, it automatically generates easy-to-customize tabs for what you've written. (I use the free version.)

High-level variables

When exploring variations of an idea, you can do more than change low-level variables like pitch. You can also change things like intensity, emotion, and genre. This usually involves changing several low-level variables at once.

For instance, to turn a classical symphony into a rock song, you'd need to change the instruments, add a drum part, and maybe transpose all the notes to make them playable on guitar and bass.

Check out David Bruce's video on variation, and Nahre Sol's series on sounding like different composers.

Starting small

Two promising bars. A motive. That's all you need to start. Empowered by the tools of variation, you can extend and improve a tiny idea until it's a complete and compelling melody, or even an entire song.

My starting ideas usually come from messing around on piano, guitar, or bass. If I find something I like, I record it as a voice memo on my phone. If I still like it later, I transcribe it into Dorico and start developing it.

For more on motives, check out the first chapter of Alan Belkin's book, Musical Composition: Craft and Art.

Building melodies, riffs, and other parts

Usually, a small idea is improved and extended to form part of a song, not the entire thing. Different ideas are used for each melody, riff, bassline, beat, etc.

Then again, the same idea can be played by each instrument - all at once, or at different times. Or it can be reformulated - as a flowing melody, a badass guitar riff, or a funky bassline.

Once you have melodies, riffs, and other parts, you can combine them in different ways. A vocal melody can appear alone or with the entire band. In some genres, actually, the individual melodies, riffs, and other parts stay the same throughout the entire song. Things only change when various parts are turned on or off.

Small ideas become parts. Parts become songs.

Conclusion

Mess with a guitar. Find a couple good bars. Transcribe them into Dorico. Develop an 8-bar melody. Then a riff and a bassline. Play with all the variables.

Like food, a musical idea can be reinvigorated with a dash of spice.

Like a Lego creation, it can be broken down into tiny pieces and endlessly recombined.

Like a spoken word, it can be voiced in any of a thousand emotions, from mild curiosity to titanic anger.

“I’m trying to analyze a big, messy dataset for the first time, and it’s going okay. But, I’m a web developer - not a data scientist. I’m guessing there’s a better way to go about it… any tips?”

Sure! While you might get through the task with the tools you already know, the wide world of Python data science is here to help you. But, before you dive into specific tutorials, books, and courses, let me sketch the big picture by explaining the three tools you’ll use in every data analysis.

Plots

Exploration, debugging, and checking correctness all involve constantly printing and plotting things. The more this becomes second nature to you, the better you’ll be at asking and answering questions of your data (and catching mistakes).

A great library for plotting in Python: Matplotlib.

Notebooks

You'll often want to refer to prior results - things you've plotted and printed. Working in a terminal, text editor, or traditional IDE makes this difficult. Plots either disappear or stack up in a mess, and are always detached from the code that created them. Notebooks solve these problems, making it easy to explore without fear of losing prior results.

The first and most popular notebook for Python: Jupyter.

Dataframes

Lists and dictionaries are slow (and conceptually awkward) when it comes to data analysis. What you really want is a spreadsheet, a table. Something with rows and columns you can refer to and manipulate easily. In the world of Python data science, this is called a “dataframe”. Your first step in almost any analysis is to get your data into a dataframe (or perhaps several), at which point you have a huge set of tools available.

The undisputed king of Python data analysis libraries: Pandas.

Additional Q&A

• How should I download these libraries? You can download these libraries individually using their instructions, or download these and many other useful libraries all at once in the Anaconda distribution.

• Do I need to connect to a database? Not that I’ve ever heard. You just download the data you need as a file, from whatever source - database or otherwise. Usually, that takes the form of a csv or something. Then you load that data into memory as a dataframe, and go from there.

• Should I save my results as I go? Usually, you do most of your work in memory, and only save new files to disk when you’ve performed a lengthy, expensive operation you really don’t want to do again. Often, an analysis will just start by downloading the raw data, doing a bunch of things to it, and saving a few things to disk at the very end.

• How do I make my code faster? Snakeviz is great for discovering bottlenecks. Also, as a general rule, it’s best to do as much as possible with Pandas itself. It’s much more efficient than Python’s lists, dictionaries, and loops. You may eventually also want to try Numba, which can compile certain Python functions to ultra-fast machine code.

• What if I have a lot of data? Depends how much. Pandas can handle much more than Excel. It’s only limited by how much data can fit in your machine’s memory. Here are some recommendations for scaling to large datasets. Also, though it’s perhaps more trouble than it’s worth, you can rent machines in the cloud for ~$10/hour that have more than a terabyte of memory (see this list).

Create a grid of zeros. Put a one in the middle. Then, replace each cell with a new value, calculated by adding up all its neighbors, then applying some oscillating function like mod or sine. Do this 50-100 times, then look upon the magic ye hath wrought.

The mod function, like a clock, lets values can grow up to a point, then makes them periodically start over. (e.g. 1, 2, 3, 1, 2, 3, 1…) Sine is another periodic function. It always outputs values between -1 and 1.

You end up with patterns like this:

Which you can post-process (e.g. with Prisma):

Or, if you throw a word on top and run the program some more, you’ll get this:

Those are all up-down and left-right symmetric, but you can start off with a grid that’s only left-right symmetric, and end up with things that look like alien faces. (See video.)

With these and other tools, the future is bright for the symmetrists among us!

For more pics and technical details, see my thread, or these two threads from my Twitter buddy Murat (they were the inspiration for all this, so thanks Murat!)

If you liked this post, you’d probably enjoy these two too:

• Xenotypography 101

• Bottom-Up Art

It’s not just a cool band name (or ship name). It happened to me once! After days of doodling, I’d filled a page in my middle school notebook with detailed abstract designs. A line here, some dots there, a nice sweeping curve, etc. Then a friend walked by and asked, “Is that a donkey?” After denying this absurdity, I held my handiwork at arms’ length and...

Not only was it the face of a donkey, it was a better donkey than I’d have drawn intentionally. It boldly leapt from the page, and all who saw it instantly accused me of drawing under the influence of LSD. To top it all off, you could flip the damn thing upside down, and it was a different damn donkey.

How on earth did I accidentally draw a reversible psychedelic donkey face?

Well, it’s actually not as unlikely as it sounds.

1. It was sure to be psychedelic-looking, because it was chocked full of weird little patterns like dots instead of photorealistic features like shaded surfaces.

2. My intuitions about good-looking lines and patterns depend partly on everyday objects, including animals and plants. It’s hardly surprising that biology-inspired lines might accidentally produce something biological.

3. I could’ve easily ended up drawing a dog, cat, or any other object. They’d all be as impressive as the donkey.

4. It’s hard to draw an abstract design that doesn’t look like anything. Stare at a cloud long enough, and it’ll look like something.

So, perhaps the sudden appearance of a donkey is not so mysterious after all.

Three ways of working

The lesson here is that you can work in several ways:

1. Start with a subject like a person or mountain. Then, choose how to represent it using colors, lines, etc. (This is called representational art.)

2. Start with a graphical idea, like a shape or symmetry. Explore it, but avoid representing any real-world objects. (This is called abstract art.)

3. Start with a graphical idea. Then, as the piece develops, see what object it might look like, and try to bring that out. Or, after the piece is complete, see what it looks like, and give it a title to shape how people see it. (Perhaps I could call this semi-abstract art, or bottom-up representational art.)

Here’s a piece I drew (and then post-processed and colorized using Prisma). Like the psychedelic donkey, it was drawn using method 3. Does it look like anything?

I gave it the title Cliffs of Dover. Does that change your perspective? Here’s one more:

I call it Circus. It’s got color and it’s got tents - such is my defense.

From love-sickness to line-thickness, every artwork depends on a wide range of high and low-level ideas. Use them how you like!

As I’m constantly writing new music, and looking for new ways of writing music, I wonder: could I write an album based on the book, with one piece per chapter?

I think so! As a test, I’ll take the first few chapters, summarize their main points, then describe a way to represent them in music.

(By the way, the Swedish pianist and composer Ernst Erlanson - a good friend of mine - also did something like this.)

Introduction

• Progress is real: Grow in speed, volume, complexity, and joyfulness throughout.

• Progress has a necessary beginning: Start with a unique event - heard only once.

• Progress need never end: On several occasions, appear to end, but resume again.

1. The Reach of Explanations

Good explanations can apply to situations far beyond where they were created. Einstein’s theory of general relativity was discovered on Earth, but applies to the entire universe.

Start with a memorable melody, then repeat it again and again while changing everything around it. The rhythm, tempo, genre, harmony, etc. Show that the melody works in many situations.

2. Closer to Reality

We observe nothing directly. In fact, we often get a better picture of reality when our observations are less direct - when things like telescopes, theories, and computer programs lie between our eyes and distant galaxies.

• Start with melody to represent a physical phenomenon.

• Then a new melody to represent observation/interpretation.

• Then an imperfect copy of first melody, to represent the errors of interpretation.

• Repeat this group of three, each time keeping the first melody the same, but lengthening the second, and making the third a more faithful copy of the first.

This represents how adding layers of interpretation can improve our understanding of what is out there. With each repetition, it’s as if we’re using larger, better telescopes and seeing the stars ever more clearly.

3. The Spark

“Like an explosive awaiting a spark, unimaginably numerous environments in the universe are waiting out there, for aeons on end, doing nothing at all… Almost any of them would, if the right knowledge ever reached it, instantly and irrevocably burst into a radically different type of physical activity: intense knowledge-creation… transforming that environment from what is typical today into what could become typical in the future. If we want to, we could be that spark.”

Divide the piece (or sections within it) in two. The first half is simple, subdued, and mostly unchanging. Maybe ethereal, like Neptune, from Holst’s The Planets. The second half starts with the sudden arrival of small-but-clear theme. Then it rapidly explodes in power and complexity.

I’ll stop there for now. The book has almost twenty chapters, and most of them could be an album of their own!

By the way, here’s a piece I wrote that was inspired by related ideas:

Seriously, though. I’ve done some work on this. First, here is a mug.

Click it to see for yourself, but that mug - and Mars - rest on the desk of Elon Musk. The red symbols are from a fictional language called Marain (not Martian) in Iain Banks’s Culture novels. I think they look cool. Elon Musk thinks they look cool. You think they look cool. We all agree. They’re cool. But I actually did something about it! Again, I’m brave.

What to do? Why not generate all possible symbols in this language? But how?

Well, the mug’s symbols all fall within a template: a plus sign and an X, all wrapped in a square (see below, left). Incidentally, The Skyward Lament, as I like to call it, can also be created from this template (see below, right).

It's amazing how many images, and how many emotions, one can conjure from this template. Take a look.

Anyway, how many possible symbols are there? Well, again, the template is a plus sign, an X, and a square. The plus sign has four line segments, the X has four, and the square has eight (two on top, two on bottom, etc.) So, that’s a total of 4 + 4 + 8 = 16 line segments. Each one can be present or absent. That’s two possibilities each. So, every character could be represented by 16 bits of information, with each bit specifying whether a particular line segment is present or absent. That leaves us with 2^16 = 65,536 possible characters. About 30% of these are either easy to confuse with each other or can't be written without lifting your pen. That leaves us with around 46,000 characters. Here are 328 of them:

They all consist of two polygons. In fact, this is an exhaustive list of all characters made up of two polygons. Here are a bunch of characters with arrows pointing to the top-right:

There are all sorts of interesting groups of characters.

But, to count as xenotypography, we’ve got to turn our attention not only to the symbols, but to the fonts! Here’s an elegant, thin-lined one:

And a bold one:

Starting only as Musk’s mug’s markings, Marain’s managed to maintain a miraculous, multi-year magnetism over my mind, and I’ve only shown you the half of it here.

The answers all flow from one idea: genes are the subject of variation and selection. Not individuals. Not groups. One of the most important consequences of this view is that genes often prosper at the expense of individuals and groups. For instance, a male might live longer if it never fought for mating opportunities. But, a gene for safe celibacy dies with its holder. Genes for dangerous promiscuity don’t - they spread. Similarly, if a gene helps one individual reproduce more than its competitors, it will spread, even if it harms the species as a whole (e.g. by reducing its population).

But what do I care? I’m interested in how the human mind works, and how to build one!

Well, though Darwin just wanted to explain the creation of complex organisms, he accidentally discovered something far more general: a theory that explains the creation of all knowledge, not just genetic knowledge (e.g. like how to build reliable anuses). This was made clear by the philosopher Karl Popper, who developed an evolutionary theory of knowledge. He argued knowledge is created by conjecture and criticism - variation and selection.

Evolution, then, isn’t really about biology. Like the mind, it’s about knowledge-creation.

This is why I’m interested in evolution in all its forms, and why I decided to read The Selfish Gene. By learning about biological evolution, I hoped to learn something about how human ideas evolve. I think I succeeded, and I learned lots more besides. In fact, here’s a Twitter thread full of interesting excerpts from the book, and thoughts I had while reading it. (Btw, make sure to read the out-of-context quotes like, “...sawing off heads is a bit of a chore.”)

1. What are the limits of biological evolution?

2. What makes explanatory knowledge special?

3. How can Popperian epistemology improve narrow AI algorithms?

4. What are the different kinds of conflicts between ideas?

5. Why is the brain a network of neurons?

6. How does the low-level behavior of neurons give rise to high-level information processing?

1. What are the limits of biological evolution?

Could evolution ever produce a wheel-and-axle? Perhaps such inventions are “all or nothing”, and are therefore impossible to create in a sequence of tiny, trivial genetic mutations. It’s a counterintuitive fact that evolution can produce stupendous complexity (if you doubt this, then try 3-D printing a hippo…), but it cannot produce even a simple thing if it is too different from what already exists. Evolution is a bit like a mountain climber that can go high as the sky, but never jump a gap. (By the way, I suspect today’s neural networks have similar limitations.)

This question about limits is interesting because there is something special about humans - something we can do that evolution can’t. But what? While we know something of the answer, our knowledge is vague. Creating artificial general intelligence may depend on a more precise understanding of the boundary between the capabilities of human minds and biological evolution.

One could call this the question of evolvability - what can be evolved and what can’t be? Incidentally, this question - about what is possible and what is impossible - has the same form as statements in constructor theory, which is about what physical transformations are possible and impossible, and why. Perhaps the limits of evolution can be expressed in constructor theory, or illuminated by ideas from the constructor theories of information, life, and thermodynamics.

2. What makes explanatory knowledge special?

In his 2012 article, Creative Blocks, David Deutsch argues that “the ability to create new explanations is the unique, morally and intellectually significant functionality of people (humans and AGIs)…” He elaborates on this in The Beginning of Infinity and his TED talk on explanation, but we have a long way to go before our understanding is sufficient to create a computer program capable of explaining anything.

In a recent post, I asked:

Explanations are about what is objectively true rather than only what is useful. What are the consequences of this? What makes this sort of knowledge more powerful, and indeed more useful, than other kinds? What is different about its structure? What mechanisms of variation and selection are required to create this sort of knowledge? What is so difficult about creating it? (After all, it’s an extremely recent innovation in the history of life on earth.)

In my last post, True vs. Useful, I tried to explore one feature of explanations that makes them especially powerful:

In the end, the search for truth entails the pursuit of logical consistency among all our ideas, and thus takes advantage of all our knowledge - not just a single, fixed idea. It subjects our ideas to a powerful form of selection - logical contradiction - not found in biological or machine learning systems. Most importantly, it provides a combinatorial explosion of opportunities for conflict - and thus for progress.

On a different note, what makes explanations so hard to create? One idea, as Deutsch argues, is that good explanations are “hard to vary,” meaning that most modifications not only make them worse, but completely non-functional - unable to explain anything at all. This makes them hard to reach in a way much like “all or nothing” ideas like the wheel-and-axle.

To visualize this property, imagine a vast cube representing the space of all ideas, where the best - the most true and useful - ideas are bright points of light while the worst ideas are invisible. In this space, good explanations are solitary, bright points. They are rare, sparse, and disconnected from other bright regions. On the other hand, useful rules of thumb exist as fuzzier clouds of points, for similar rules of thumb are about as good as one another, and so the nearest points to a given rule of thumb are about equally bright. Similarly, genetic knowledge forms a fuzzy tree without any gaps - for any two points in the tree, you can get from one to the other by following a path of bright points.

Now imagine you’re trying to navigate this space to improve your ideas, but can only see a short distance. If you find yourself in a fuzzy cloud or tree, you can look around to nearby points to see if any are better and brighter, and move to it. By contrast, finding a good explanation is far more difficult, for it is hidden in a vast space, and you might pass within a short distance of it without ever seeing it. Stumbling and looking around isn’t enough. You need something more like high-powered telescopes and ultra-accurate, long-distance teleportation in the space of ideas. How do we do that?

3. How can Popperian epistemology improve narrow AI algorithms?

In his 2012 article, Creative Blocks, David Deutsch argues Popper’s work on epistemology is key to building artificial general intelligence. I think it may also inspire unique advances in narrow artificial intelligence algorithms (which, despite their lack of generality, can still be tremendously useful). After all, Popper’s work applies to the creation of knowledge in all its forms, from biological evolution to human minds - and narrow AI.

Also, the whole point of AGI is to write a program that is a mind, so the earlier one can apply and test one’s theoretical ideas (by programming them), the better. Such tests are bound to uncover all manner of subtle theoretical issues that would otherwise go unnoticed. That’s how programming usually goes - getting one’s ideas to work in practice is harder than anticipated, and leads to a far better understanding of things.

One idea is to apply the conclusions of True vs. Useful, and focus on solving constraints rather than maximizing performance. For a visual metaphor, it’s like trying to get puzzle pieces to fit together rather than walking to the top of a hill. While this approach has a long history (e.g. logic programming in Prolog), logic-based approaches to artificial intelligence have been mostly unsuccessful. They are brittle and full of precise statements, while human knowledge is flexible and full of fuzzy statements. So, there are unsolved problems here, and perhaps deep learning and Popperian ideas can help address them.

For one thing, I think a common mistake in the history (and present) of AGI research is to take a particular cognitive tool like logic, language, or analogy and suppose it is the core of intelligence. As Popper explained, variation and selection are at the core, and other things just provide specific (and often tremendously useful) mechanisms of variation and selection. Perhaps taking this seriously will help solve the problem of how to use logic in artificial intelligence - both narrow and general.

4. What are the different kinds of conflicts between ideas?

A key part of understanding minds is understanding how ideas interact within them. How do these interactions lead to the variation and selection required for the evolution of knowledge? How are they combined and altered to form new ideas? How do they exert selection pressure on each other? What kinds of interactions spark the search for new ideas?

As I argue in True vs. Useful, logical contradiction offers one example of how ideas can interact, but a subtler example might be when you are surprised by something. In this case, it’s unlikely there’s any explicit logical contradiction at work, but there is still a conflict of ideas. If you are surprised upon entering an elevator containing three goats and a glowing block of uranium, you are experiencing a conflict between what you expected to see and what you are actually seeing.

Something similar must be happening when you find something interesting. Here, the conflict is even more subtle, though, and I don’t quite understand it. Presumably an idea appears interesting if it seems both novel and relevant to problems that one cares about. Given the vagueness of such a statement, it can no doubt be improved upon by trying to program it, as I mentioned earlier.

At any rate, the interactions between ideas are fundamentally important, conflict is one key example, and it exists in many different forms which have evolved for different purposes, like finding things dangerous, desirable, interesting, and surprising.

5. Why is the brain a network of neurons?

There are many ways to build a computer, and the only fundamental requirement is that it be Turing-complete. While brains and modern processors both satisfy this requirement, one does it with a network of neurons and the other with a von Neumann architecture.

Why the difference? Presumably because brains had to be evolvable while modern computers could be designed (see question #1 above). A network of neurons can start small and grow larger in the course of evolution and be useful at each stage. In contrast, modern computers are like the wheel-and-axle. They’re all-or-nothing. If one part of the system breaks, or has yet to be created, then it is useless.

Setting aside the question of evolvability, though, should minds be made of networks of neurons (or simulations of them)? While the way a computer is built doesn’t affect what it can do in principle, it does affect what it can do in practice. After all, the integrated circuits in modern computers are millions of times faster than their vacuum tube predecessors. Moreover, there are different algorithms for doing the same thing, and they can be wildly different in their speed and memory usage. Perhaps the network structure of the brain indicates that minds depend for their efficiency on concurrent, distributed, networked computation.

For example, consider how efficiently the brain can search its memory in the course of a conversation. I’ve sometimes wondered how it was that I recalled a perfectly-apt anecdote despite having not thought of it for years. Evidently, it was stored in such a way that, under the right circumstances, it could quickly and easily be found and shared. That is not a trivial feat, given the vast collection of memories in a mind. For instance, it would be far too slow to go through all one’s memories one by one. By the time you’d found a good story, the conversation would be over! The network structure of the brain seems to handle the problem with ease, though. The general picture (as in deep learning) is that a situation “activates” some neurons, which in turn activates others which are connected, and this cascade of activity can eventually activate a region of the brain associated with some long-dormant anecdote that’s perfectly relevant to the conversation one is having.

So, if that’s one example of the efficiency and practical value of network-based computation, what are others?

6. How does the low-level behavior of neurons give rise to high-level information processing?

Human brains must be Turing-complete (after all, they came up with the idea of Turing-completeness!) but how does one build a universal Turing machine from neurons and their connections? This is an active area of research (here’s one potential explanation).

More generally, for any given computation, how can it be expressed in terms of the behavior of neurons? The same question exists for modern computers, too, but instead of expressing things in terms of neurons, one uses the low-level instructions which a computer processor offers. Historically, engineers hand-coded these low-level instructions, then developed a slightly higher-level language to make things easier. They could write code in this language, and it would be translated, or compiled, into the relevant low-level instructions. Later, other languages were built on top of that language. This process has continued, and now modern programmers can express high-level ideas easily and then compile them into the instructions which a processor can understand and execute. Perhaps a similar process happened historically with brains, and can be used in any network-based AGI computer we wish to build.

Logical consistency is therefore a powerful constraint on our ideas. It means that any idea can in principle contradict another, sparking the search for an improvement that’s free of the contradiction. This yields a combinatorial explosion, for if any pair of, say, 10,000 ideas can conflict, that yields (roughly) 10,000 × 10,000 = 100,000,000 potential opportunities to spark progress. This also means any one idea is potentially constrained by 10,000 others. In practice, we cannot compare every pair of our ideas to check their consistency, and doing so would be a waste of time even if we could. Most pairs of ideas are irrelevant to each other. Nevertheless, each of our ideas is tremendously constrained by other, related ideas (and we can search among all our ideas to find which are relevant). To borrow the language of Darwinian evolution, the need for logical consistency subjects each of our ideas to tremendous selection pressure from other ideas.

And, knowledge-creation is all about variation and selection. In the biosphere, genes are subjected to random variation and to natural selection (e.g. predation, mating, starvation). In human minds, ideas are subjected to intentional variation and many forms of selection, usually involving some form of conflict with other ideas. Logical contradiction is just one important example. It’s these sorts of differences in the mechanisms of variation and selection which are central to explaining why human minds are vastly more capable than blind evolution at creating knowledge.

It also helps explain why human minds are qualitatively superior to today’s narrow AI algorithms, which are useless outside the narrow domain they’ve been trained for. A human idea is subject to selection by many other ideas within the mind, which are themselves subject to variation and selection (and therefore improvement). In a machine learning system, a model is subjected to only one form of selection: how well does it perform on the given task? In other words, how useful is it? First of all, this mechanism of selection is not itself open to improvement from within the system. It is subject to no variation and selection - except from without, by human minds. Secondly, even supposing different “ideas” in a model could be made to constrain one another, they are not allowed to. There is only a single, fixed form of selection: usefulness.

In a human mind, this would be like changing your ideas only in response to external reward and punishment, as David Deutsch explains in his essay, Beyond Reward and Punishment. Such an approach makes very poor use of the available knowledge in a system, quite like the way a totalitarian state suppresses all ideas but the dictator’s.

More generally, though, there is a fundamental difference between seeking ideas which are true versus seeking those which are useful. While two contradictory ideas cannot both be true, they may both be useful. This is the case for quantum theory and general relativity, both of which are famously useful and mutually incompatible. At present, many seek a unifying theory that will supersede them both. Why? Because of some practical problem? No. Their incompatibility presents a dramatic logical problem, and therefore an opportunity for improvement - one that we would be utterly blind to if we did not seek truth and thus logical consistency. Indeed, if we cared only about usefulness, the irony is that we’d be denied our most useful theories, for many were sought in response to theoretical problems rather than practical ones.

Incidentally, this is why it is a mistake to pursue only things that are known to have good consequences - altruistic or otherwise. After all, that would put an end to all research, the results of which are - by definition - unknown, and therefore the consequences of which are unknown.

In the end, the search for truth entails the pursuit of logical consistency among all our ideas, and thus takes advantage of all our knowledge - not just a single, fixed idea. It subjects our ideas to a powerful form of selection - logical contradiction - not found in biological or machine learning systems. Most importantly, it provides a combinatorial explosion of opportunities for conflict - and thus for progress.

If that seems odd to you, then you’re in good company! It seemed odd to me too. While I’d often focused on particular tasks for extended periods, I’d never just lied in bed and thought for hours at a time. Now, I was doing it constantly, and in isolation. While I’m told some people are distressed or even destabilized when alone with their own thoughts like this, I found it delightful and exciting.

Wait. It was exciting? Shouldn’t it have been supremely boring?

Just the opposite. By setting aside all my normal responsibilities, pursuits, and entertainments, I’d created the space for an entirely different activity: open-ended exploration. That’s right - despite my sedentary state and the scarcity of Arctic sled dogs, I was exploring. Not physical environments of course, but mental, virtual ones. After all, as the physicist David Deutsch points out in The Fabric of Reality, “All reasoning, all thinking and all external experience are forms of virtual reality.”

At the close of each hour-long meditation, I’d return with eagerness to my room and those bustling, dynamic, abstract worlds. When I left them, called by the next meditation bell, it was always with some reluctance. Thinking was addictive. Compelling. Engaging. Unpredictable. The sudden appearance of an unforeseen thought might set me on a new and intriguing path to who-knows-where for who-knows-how-long. Confronted by some obstacle, I might patiently try every available tool, technique, and route until I’d overcome it. I had time.

I also had opportunity. Novel ideas are not accessible only to some privileged class of intellectuals who’ve mastered all prior thought on some subject. Every mind has the capacity to create knowledge - new knowledge. We could not learn to read, speak, or walk otherwise. Despite appearances, there is no fundamental difference between learning to say “mama” and “papa” and discovering fire, photons, and fusion. All such discoveries, however trivial or significant, are made by trial and error.

And they are ready for the taking. Our infinite ignorance dwarfs our knowledge, and one can come face to face with the unknown by simply asking “why” or “how” a few times (and rejecting the many inadequate answers on offer). Isaac Newton was acutely aware of our ignorance. Perhaps this contributed to his great scientific success, for if one feels that vast riches of understanding are available, one seeks them. He said,

I do not know what I may appear to the world; but to myself I seem to have been only like a boy playing on the seashore, and diverting myself in now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me.

In a way, none of this is surprising. It’s obvious there are things we don’t know, and that we can try to figure them out by thinking. Though simple and true, such a statement fails to capture the full scope of our situation. We know almost nothing. Ignorance, and therefore opportunities for every sort of improvement, are everywhere. The content of future discoveries, and how they will be made, are known to no one. The prime requirement for progress is creativity, which is available to everyone.

We are just beginning.