In April I lead climbed outdoors for the first time. For those who don’t know, there are several types of outdoor rock climbing. In order from least to most scary, they are:

1. Top rope climbing: Two sides of the same rope hang down from a big anchor at the top of the wall. The climber is tied to one end. The belayer threads the other through a little device. If the climber on the wall falls, she is immediately caught by the rope which links herself all the way to the top of the wall.

2. Lead climbing: There are a bunch of bolts on the wall that the climber attaches clips to as she goes. The clips are called “draws.” The climber themself starts tied to a rope, but the rope isn’t attached to anything. Instead, as the climber goes, she clips the draws into bolts on the walls and in turn clips the rope into the draws. The thing is that to get from one bolt to the next the climber must go above where she last clipped in. That means that if she falls, she falls twice as far as the distance between herself and the bolt below her. You can picture it like a person on one end of a string tied to the wall 10 feet below herself. When she falls, she falls 20 feet. It’s the same in lead climbing. This makes lead climbing scarier, but it’s also harder; the climber wastes energy pulling up the rope and clipping herself in as she goes, while holding on to whatever handholds and footholds are available for dear life. Clipping in quite scary. In the moment before relative safety, when the climber gathers her rope to clip into the draw, there is much more slack in the line than normal, and she is at the maximum distance she’ll ever be above her last clip, and if she falls she’ll fall far indeed. Beginners like myself don’t lead climb very hard routes.

3. Trad climbing: the wall is completely barren. The climber brings her own bolts, in the forms of gizmos called nuts and cams and slings and so on, that she can jam into cracks in the wall. Once she is sure they are secure, she clips into them. I won’t go into great detail on trad climbing, as it’s something I have neither done nor intend to do.

I’d climbed top rope outside and climbed lead inside—thanks to the assistance and trusty rope of CB and MW—but never climbed lead outside. As far as pure rock climbing goes, outdoor lead is about the limit of what I will ever do. I imagine a climbing skill tree, where at the very base the three branches to choose from are bouldering, top rope, and rock scrambling. Somewhere far along the paths which begin with rock scrambling and top rope one learns to mountaineer with gear, which I’m interested in, but that’s early enough along the rope climbing branch that you don’t need much trad ability. In other words, I’m uninterested in placing cams.

But lead climbing was a great next step. Here are some of the benefits of learning to lead climb (note that I say learning, and not knowing how; I am still extremely novice)

• Improved rope skills I can use in future mountaineering adventures, such as rappelling, setting anchors, and coiling rope

• Improved climbing endurance, required especially for finding and holding static positions while clipping

• Greater awareness of the body while climbing

• A new trust in myself to be cautious and safe when safety is in my own hands

• Increased appreciation for how absurd the best climbers in the world are

These benefits remain largely aspirational. I climbed lead outside only twice; first with friends, and then with two strangers from a Rhode Island climbers E-mail list. And that on the day of Spring weekend no less. They may remain aspirational for a long while yet; here I reveal that—yes!—I have decided to quit climbing. I will be taking my talents elsewhere for the time being. The reasons why are not relevant here.

May: Longest Bike

I crashed off a sled when I was 5 and broke my collar bone. My mom ran up and asked if I was ok. I replied,

“I’m painfully dead,”

and then my arm was in a sling for a few months.

I crashed off a mountain bike when I was 19 and gashed my elbow bad. When a woman named Mayim, which means water, washed out the blood I could see flecks of fat and gravel beneath my skin. I pulled or pushed my shoulder in the fall, by some means stretched it further than it was meant to go. For a month I couldn’t do pull ups and every night on the wooden slats of the shower shed I changed the sloppy bandages on my elbow, wiping away the endless oozing which would not reconstitute itself to anything other than a translucent film which slid away at the first opportunity. On the day of the injury I wrote in my journal:

“My first instinct was to get back on the bike, to go out tomorrow, because I wanted to prove to myself or someone else that I could. I really don’t want to be the sort of guy who falls off a mountain bike his first time trying and then never learns to mountain bike. But also, being injured sucks. It sucks more than I remember after the fact, and I don’t really have that much interest in mountain biking. So maybe I should just let this one lie for a while or take it slow next time I learn”

And haven’t mountainn biked since. For falling off bikes, I’ve developed more novel methods.

We biked 60 miles in Massachusetts, the longest bike ride of my life. Everyone knows about riding bikes and everyone knows what they need to know about Massachusetts (which is nothing). TLDR; I rode a bike in Massachusetts and it looked like Massachusetts. That’s not to say it wasn’t wonderful: the beachy houses so far from the ocean’ winning uphill and then getting passed by GS on the long coast down; the navy texture of misty roads and heavy trees; the wind It was wonderful! I’d like to thank GS’ roommate for the bike shorts. I’d like to thank the man who fixed my squeaking bike for free in Turner’s falls. I’d like to thank his competent hands were like a separate organism from his completely awkward manner, which I’d also like to thank. The best bit was when we got to cross the Connecticut river, a few times on our bikes. The Connecticut river is humongous. If your into humongous water, go check it out on a bike that fits you.

I’d like to thank my Cannondale R800. It deserves better than me. It deserves a boy who is five feet and seven inches tall. He’d have no complaints. Oh Cannondale R800, It’s not that you’re too small. It’s that I’m too big. My hands hurt from the weight on the handlebars. I was constantly moving them around. I’m some sort of giant freak, Cannondale R800, no bike could be big enough for me. It’s not your fault.

Sometimes when I do something just slightly risky, catch a grape in my mouth or scramble up a tall boulder, I think in my head what if this goes wrong right now will it have been worth it and what is the class of things that are this level of risk and how bad is it if it goes wrong and so on

I’m not sure whether I actually think and so on, but it’s the sort of thing I might think. Then I fold and do the risky thing.

We cleared some trees and the sun was lower in the sky. We’d biked fifty four miles and were in that final wind which follows a sluggish third quarter. Suddenly there was this gorgeous view. I wanted a photo, but there was no stopping now, and my phone was in my backpack. You see where this is going. I’m pretty proficient at biking no hands but biking no hands while taking off a small backpack is harder. I removed one strap and then the bike started to veer. Open fields and some hills in the distance, wooden fences and the road. I didn’t need push my luck for a photo and so I grabbed the handlebar and righted the ship. No issue. With the one strap removed my bag was hanging awkwardly to one side, sort of sagging off my shoulder, sort of sliding now,

ok

My pack slipped down to my elbow and swung into the spokes of the front wheel, completely jamming it. Over the handlebars I went. Crashing a bike can be quite slow, because 12 miles per hour isn’t very fast and in this case the bike stopped completely and so only I was moving. I vaulted over the handlebars as the bike unselfconsciously clattered to the ground. I watched the pavement approach, then was turned for a moment and watched the sky, and then my shoulder hit the ground and it hurt.

Thank you to GS for laughing at the whole thing. Thank you to Hashem that we were able to extract the backpack from the spokes of the front wheel, and that only two broke, and that we were only 6 miles away. And, as always, thank you to the great outdoors, you old scamp. We did it Joe!

June 31st: Wild Swim

A little silver yellow key would not fit in the lock. There was a slot for a coin. The coins had corners, rounded corners, feckless hexagon it was. Oh the coin because it was British. Get in the hole it wouldn’t go in the hole and it left a little mark on my finger from where the flesh of my thumb was flesh. So I just put my backpack in the locker without locking it and I figured no one at Docklands Aquatics Center would much care.

They say online that you need a floatie which follows you. It’s wild swimming, very wild, so there is an inflatable bobber you can buy or rent and you tie a string to your waist and the inflatable bobber floats behind you. So people can see where you are if you get lost or sink. And not kick you. Or even come get you if you crash and burn.

Bobber’s very much how it sounded when I watched England lose to Argentina—those twats how did they turn it on at the last minute—at a bar with a garden out back. This lad, and his buddies occasionally chimed in but they were the smaller voices, they were a smaller man and a woman, this lad was singing:

THERE WERE FREE GERMUHN BOBBERS IN THE EHYRE

(In the air!)

THERE WERE THFFREE GERHMUHN BOMBBERS IN THE EHYREEEEE

(In the air!)

THERE WERE THFREE GERHMUHN BOBBERS, FREE GERHMUHN BOBBERS, FREE GERHMUHN BOBBERS IN THE EHYREEEEE

ANDUH RAEH FROM ENGLAND SHAWT EM DOWN

(shot ‘em down!)

and so on. The guy, sorry the lad who was singing had an alien, robust body even though he was only regular sized and not overweight at all. He was just straining at his shirt as if he were inflated. Like the bobbers. His moustache was wet on his upper lip as he sang. He can’t have been older than 25, that’s the twist. After England lost (the football match, they won World War II) I streamed outside with the groaning crowd. I turned my shoulders narrow to slip into the stream on the street. Night air, two tiered red buses trundling along Shoreditch. Me on the sidewalk with the crowd, and these weak sauce complaints like they’d known they’d lose all along. How could they have won for a nation like that?

“Ahhhh can’t blame Jude that was absolute class from Jude”

“Well it’s just like ‘16, innit now?”

“Absolutely dirty play, not the sport at all, I hope Spain wacks ‘em”

I looked at my glowing phone on the bus.

After I went swimming in the river Thames I stopped on the dock to stretch. I’d swam about a mile. It was nice, actually. The water didn’t taste bad. I smelled like shit after, but the water tasted fine. Who knows how much I gulped down as I sputtered along, kicked my knees, turned my face to gulp in some air and some nice gray green water from the River Thames. Have you ever swam alongside a crumbling brick wall? Quite lovely. This was in Docklands, by the river on the Isle of Dogs. The Thames is pretty big, it winds its way through London and much further, but it also creeps inland. My apartment, where I’m writing this, is along a canal which comes from the river. And there are bigger canals, or inlets, or whatever they are. And where I swam was like a giant rectangle, I guess an old dock or harbor, so three sides were land and one side it went under a bridge and out to the River Thames. Both my calves cramped, that’s why I stretched on the dock. First my right leg cramped, shit you got to be kidding me just when I was enjoying myself, I was out of breath but it was hot outside and the water was real nice. There wasn’t much current, the water lapped at me like I was the dock or the crumbling brick walls. When swimming is nice it’s nice because you feel yourself moving so fast with the water rushing by you, but you’re not moving hardly at all and that’s why it’s so calm. My arms got tired so I tried to kick more, tried to have good form. BLAOW there went my right calf, so I float on my back and tread with two arms and one leg and the other leg seizing like a softball, like a knot of thick rope. Luckily I’m near a spot on that crumbling brick wall with a crumbled off hole where I can hold myself with my hand. So I grit my teeth and end up on that wall with my trembling leg. Nice young man comes up on his kayak, he says “you want to hold on to my kayak?” I say no I’m ok, perfectly good wall here. So the rest of the swimming was worse, I have to be really careful and go slow, I don’t want to set off my leg again. A middle aged woman in a wet suit passes me. The water wasn’t cold, so I could have judged, but she was fast. I wanted to chase her, I couldn’t. My calf would explode. And at the very end of the whole swim, when I decided I’d pull in to the dock and call it a day I turn to do just that and my other calf sets off. You’re joking. It’s always the one you least expect. So I ended up holding on to a different guy’s kayak until I was ok to do some kicking, and I furtively frog kicked my way back to the concrete ramp and emerged from the water like sludge, but the reality and tackiness of the concrete under my feet made my calves feel better.

And my skin was red and clean, and my body just below my skin, covered in the shit water of the River Thames.

Dearest Hacim Seeh,

Why this measured crash out?

Tell me, what else should I have done?
Doesn’t everything die at last, and too soon?
Tell me, what is it you plan to do
with your one wild and precious life?

Mary Oliver. Everyone quotes that one but it is a linguistic killshot and it is meant to be that way.

My mental model used to be something like:
There is a big chance of AI doom in relative terms: something like 5% or maybe even higher. So we should work really hard on it. But also of course 5% things never happen and we’ll be fine. So I didn’t feel actual worry. I was just playing the most responsible game.

Now I’m more worried. All the time I’m more worried. First of all, 5% was a way lowball. No more lowballs, it’s time to learn ball.

So I

Learn ball

Learn ball

Learn ball

Learn ball

Learn ball

Learn ball

and I learn ball from people too, and podcasts and conversations and incriminating google docs that people who didn’t exist yesterday show me on their phone while several frontier labs try to hire them.

Now I know ball a little. Here is how my views have changed:

1. I used to think there was about a 70% chance that with a little alignment nudging the current paradigm of AI systems would naturally be pretty constitutionally aligned, enough that they could keep an eye on and align future AI generations with the right amount of caution and so things would just go fine by default if the technical alignment work was solid. I now give this more like a 10-50% chance, depending on how hardcore and excellent our alignment and control methodologies are.

2. I used to think that Anthropic’s leadership cares so much about safety, enough that they would see very clearly and put a lot of capital into Activating a Galaxy Brain Plan when we got to crunch time. I no longer believe that, like, at all. I think the high end of what we’ll get from Anthropic is [REDACTED, THE STARS, BE GREAT]. But we won’t get much more than that. And their safety/control work will continue to be marginally useful. In the medium term.

3. I used to think that AGIs are probably pretty nearby, but ASIs would somehow take a while after that. I now think that there is about an X% chance—I should figure out X, but greater than 50—that we get ASI within two years of getting 4x research multiplier inside the labs, barring some sort of regulation. Right now I’ve heard that they are getting something like 1.6-1.8x output boost from los AIs. I don’t know. All of the numbers in this one are made up and don’t anchor on that 4x number.

4. I used to think that we’d see each AI as it arrived and become progressively more freaked out. Now I believe that an increasing tail of capabilities will be hidden inside the labs as research progress grows faster and faster and alignment grows harder and harder. See HuggingFace.

This is not what I meant to write, really. I meant to write something about priorities, or how much to ask of myself, or an excuse, or what it’s all about. The truth is, Hacim Seeh, that I know I’m still alive and playing for something other than to win the game, but I only know that because my friend got hurt somewhere in Washington and I was sad about it. So I’m playing for stuff like that. And I’ll start eating grass fed beef sometimes I think. Still doing my research.

Slow like honey, heavy with mood,

Benjamin Jacob Pomeranz

January: First Time Skiing

My parents forbade me from skiing without first learning from a professional instructor. My friend GS is on her team’s XC skiing team—count that and the foul. We skied at a hill near her home in upstate New York that one might call family friendly. Another might call it “baby town.”

In precise honesty, this wasn’t my first time skiing. I cross country skied with my family when I was eight. There was one small hill where I was thrilled and otherwise it was completely blah. My most analogous experience to downhill skiing was bombing hills on a bicycle and trail running down scree. Unlike most people, I hate the feeling of bumbling incompetence. I was nervous that I would be so lacking control on the skis that I had no fun at all; not so! After two french fry/pizza runs on the bunny hill, GS and I were cruising down a green and then a blue.

I made it a goal to fall pretty often. I wanted to learn new things like hockey stops, sharper turns, turning with my hips, and coasting along with straight knees. The fastest way, I figured, was to commit really hard to them and end up laid out on the snow, then do it better the next time. I can’t be sure, but I think this helped me improve faster. I went skiing a second time with EBW in February and felt comfortable enough to stay upright and have a total blast. What a glorious thing to learn a new kind of fun in my old age. Thanks to GS for the support.

February: Longest Snow Shoe

I snow shoed for the first time this February. It was a half mile jaunt around a lake. You scoff: 10 minutes in some sharp footwear? Has your resolution crumbled so fast as that? Is this your champion, foolish boy?

No.

The day was actually eighteen miles in snow shoes, 1.7 miles in sneakers, and .3 miles in the bed of a truck before I remembered to stop my watch.

JU and I intended to complete the entirety of the devil’s path, the big bad of the Catskills. We clad ourselves in long underwear and conviction. Some middle aged Facebook lord had completed the whole thing in 10.5 hours. We young Facebook serfs gave ourselves 12. We started at 6 AM and the sun would set a little after 6 PM. We were sure we could get the job done.

Montage

5 AM: we awoke

5:30 AM: we drove to the trailhead with grocery store fruit bags over our feet

6:13 AM: we began

8:20 AM:

9:30 AM it began to snow. Stunning. I do wonder what the hike would have been like with a clear view, but truly stunning. JU pees.

11:09 AM: Cheese

1 PM: lunchtime, and we were only about halfway through. Three out of five peaks, 4k out of 8k feet of gain, and 13 of 24 miles. Our fruit bags had failed us, or perhaps we failed ourselves. How can we blame the fruit bags? Our feet were not fruit. Then again, much like fruit, they were full of water. High spirits before lunch turned to a frigid torpor as we ate our tuna sandwiches on a frozen bench. We decided that if we did not make it to the bail out point by 3:30, we would have to call it quits and take the bailout trail to shorten the day.

1:37 PM:

2:05 PM

We were very cold and very exhausted, but we were moving. The incline was actually over sooner than we thought.

2:55 PM:

Atop the fifth peak of the day.

~3:40 PM:

We pushed on to the bail out point, but too late; our cutoff time to bail had long since passed. We didn’t have the time left for a final mountain and seven miles, nor were we excited to complete much of that in the dark. After one final “wait, should we?” JU and I took the right turn which meant we would not finish the Devil’s Path that day.

We removed our snowshoes and walked halfway back to the on road. We saw a plow that might be our savior, but it was occupied with a driveway, as plows sometimes are. Then—behold!—it was suddenly passing us, and we jumped and waved and lucky us. That was technically my first time hitchhiking and we sat in the bed of a pickup truck no less. Super fun. Somehow it didn’t feel as cold, wind and all, on our roller coaster rider finish.

Perhaps I should touch on the actual mechanics of snow shoeing, which were new to me. My prior mental image of snow shoes looked like this

but modern outdoors snow shoes are much more shiny and aggressive.

One straps in their foot, like to a moonshoe, and the foot-strapped part rotates on an axle. That way, you can go on tippy toes to dig in the crampons below your foot, without the whole shoe coming up off the ground. At first I struggled with the shoes; the ground was so frozen that when I put my foot down the stiff border of the shoe would start sliding down, before I was even able to dig in my toe. I, famously unique in my dislike of bumbling incompetence, was frustrated. Eventually I got a hang of it, and had to rely more on my toes. The Devil’s Path itself is apparently formidable in summer as well, full of rock slides and roots that one must climb up when the path is too steep to stand on. In the winter, this means steep slides of ice that we could only climb up by kicking in our bespiked toes.

It was a beautiful and new experience. Winter hiking isn’t my very favorite, but it’s better than no hiking at all and the frozen features are gorgeous. My main takeaway from this outdoor adventure is comfort with snowshoes, a wonderful tool which open up a whole swathe of the calendar to safe outdoor fun. Thanks to JU for the support.

March: Highest State High Point

My previous state high points were, in ascending height order:
1. Jeremoth Hill, RI, 812 feet

2. Mt. Greylock, MA, 3490 feet

3. Mt. Washington, NH, 6,289 feet

4. Black Elk Peak, SD, 7242 feet

5. Borah Peak, ID, 12662 feet

Now we add a 13er to the list. I flew with APF to Houston over the break, where MW attends Rice university. There, we were scooped up by MW and B, AKA The Vessel, to road trip across Texas to New Mexico. I won’t cover the driver in detail here, but suffice it to say that oil fields are what you’d imagine, windmills blink scary red lights at night, and Nonstop has some real competition for best song in Hamilton.

We landed in Carson National of northern New Mexico. This area is beautiful. In the west of the forest are the Truchas Mountains, and in the north the Sangre De Cristo mountains straddle the border with Colorado. I believe that it is an area with an interesting volcanic history as well. So there you have it.

We spent the day before our high point hike in a town called Taos, which we didn’t realize is an uber popular ski destination. It abuts Taos Pueblo, a Unesco World Heritage site and Tiwa Native American settlement which has been continuously lived in since around 1300. Taos itself was founded by Spanish colonialists in 1615, and occupied intermittently until it was established as a permanent settlement in 1795. While there are plenty of modern watering holes and Patagonia stores for skiier transplants (who, me? surely not...) the town also has a lot of historical character and architecture. The night before the big hike we had the best Tex Mex (technically New Mexican) meal of my life at Ranchos Plaza Grill. I was gonna start writing about the cool adobe church by there, but I’ve lost the plot.

Let’s return to the outdoor adventure. At 8 AM we were on the trail, just MW and I, since the others were hurting from backpacking. We agonized the day before whether to rent snowshoes at a gear shop, but the guy there was insistent that we’d be fine in strap on spikes, which we had. Thank you guy at the gear shop. Despite your sickeningly thick LA accent, you were dead on. Snow shoes would have been total overkill.

The first part of the hike was packed down snow on the oft travelled trail to Williams lake. Wearing spikes in packed snow is an incredibly satisfying combination of firmness and softness, give and stability. It’s like sleeping on a good mattress. We chit chatted and looked at the birds (identified by JU as Canada Jays).

The sun shone through air of such clarity that the eyes had to add their own blur. The peaks were spiky and brown, streaked with snow in the divets and couloirs which were shaded from the sun. Just my type.

The snowfields were mostly quite crispy. We skipped several switchbacks just high tailing it directly up them. Occasionally one of our feet would tromp down all the way through the snow, revealing that it was quite a bit deeper than our hips. We’d extricate ourselves from the so called “posthole” and continue on, and it didn’t happen enough to be upsetting rather than silly. The ground was dark, simultaneously moist and crumbly from the melting freeze. Above 12,000 feet we felt the elevation. We’d shamble up one switchback, take a breather for a minute, and shamble up the next. We shambled all the way to the ridge upon which Wheeler peaks and from there it was only a nice slice of walking to completion.

I find it quite hard to describe the exact character of mountains, what makes them distinctive. I think I lack the experience, but also the vocabulary and the science. These peaks felt pretty similar to Borah, in Idaho, but more dirty and less rocky. I believe that it was the mountains of Colorado are known for. Total dirt piles. Here is a photo:

Get off your phone, MW. Take it in.

Honorable Mentions:

- Cold plunging in a frozen pond, where a cube had been sawed away for the purpose.

- Ice skating on a pond for the first time in January, and then playing ice hockey for the first time in February, also on a pond.

- Second time skiing, and I think much harder trails

- Cold plunging in a frozen river, which was not frozen all the way to shore.

- Snowboarding for the first time, albeit it down a tiny hill on Brown’s campus.

- Coldest night of camping (on the flop backpacking trip in New Mexico)

That takes us to the end of this quarterly update. Thanks to all who have supported me in these great joys of mine. In the coming months I hope to outdoor climb, speedrun some New England high points, and maybe even try some long swimming. We’ll see, we’ll see. WOOHOO!

We have lots of AI benchmarks. They measure things like:

• Coding ability (e.g. SWE Bench, Webdev Arena)

• General Knowledge (e.g. GPQA Diamond, SimpleQA, MMLU)

• Mathematical Reasoning (e.g. FrontierMath, OTIS Mock AIME)

• Long tasks and agency (e.g. METR Time Horizons, GSO)

So, we can compare how good different LLMs and AI agents are by measuring their performance across different benchmarks. This can get messy, but still gives us a good sense of which models are most capable best at any given time.

In fact, with only a little bit of statistical fiddling and optimization (see here for details), Epoch AI integrated 37 benchmarks into their “Epoch Capabilities Index,” and this single number seems to capture which models are on top quite well.

For example, the best models right now are GPT 5.4, Gemini 3.1, and Claude Opus 4.6 (although on NumberInNameBench, OpenAI is outperforming Google and Anthropic by 74% and 17.4% respectively).

Of course, this single number is necessarily quite reductive. At the time of release, the hardest benchmarks in Epoch’s models were

1. OS Universe, “a benchmark of complex, multimodal desktop-oriented tasks”1

2. GSO Bench, “a collection of software performance optimization challenges”2

3. Factorio Learning Environment, a suite of tasks in the video game Factorio (e.g. “Build an automatic iron gear wheel factory”)

4. FrontierMath Tier 4, a set of wildly difficult math problems that postdocs and professors spent weeks writing 3

Why is it that I feel number 3 and number 4 don’t perfectly align… of course, while all these benchmarks top the charts in difficulty, they are measuring wildly different things. Indeed, one could imagine a huge improvement in computer use and long term task coherence which leads models to ace problems in Factorio and GUI manipulation while hardly improving performance on cutting edge mathematics at all.

In the era of unscaffolded chatbots, we might only have needed to measure basic reasoning and general knowledge. As models improved, it became relevant to test their coding ability as well. Now that companies are racing to build AI agents which can automate remote labor, we have a suite of tests which measure computer use and performance on long tasks. Putting all of these in the same bucket feels strange, and many people and companies have already moved on from this. In OpenAI’s announcement of GPT-5.4 they have separate sections on coding, tool use, knowledge work, and computer use/vision, each with their own benchmarks.

At first blush, it may seem obvious how to group these benchmarks: if it has SWE in the name it’s probably about coding, FrontierMath surely correlates with math olympiad performance, and so on.

Still, I wanted to cluster benchmarks using statistics. I suspected some of the results would be surprising; even to start with, I couldn’t guess whether chess puzzles naturally group in with coding ability, or math, or computer use. Will playing GeoGuessr land in a group of general knowledge benchmarks, or perhaps with those focused on visual reasoning? Their are certainly limitations to what we can find out, but I came away from my experiment with some fascinating nuggets (chess puzzles are AGI).

Methods

We want to say which benchmarks are similar without manually examining their actual semantic content. That basically leaves us with one other type of information, which is how well different models perform on these benchmarks. If the same set of models perform well and poorly on benchmark A and benchmark B, then it seems likely that the benchmarks are measuring similar things.

We use a set of benchmarks and model evaluations on those benchmarks from Epoch, once again. You can find the data here. We start with 577 models and 50 different benchmarks.

Let Z_A(m) be the Z score of model m among all models on benchmark A. We use Z scores instead of raw benchmark scores because not all benchmarks have scores in the same format, and we don’t want correlations between our benchmark data to be dominated by high variance benchmarks. My first idea was to create a vector v_A for each benchmark A where the mth index, representing the performance of model m on bench mark A, is Z_A(m):

Then, we can perform standard clustering of our data points in R^M, where M is the number of models. Unfortunately, most of the models are not tested on many of the benchmarks. I tried just plugging in 0, an average score, for Z_A(m) if model m was never evaluated on benchmark A, but this ended up clustering benchmarks based upon what models they are tested on more than anything. Specifically, we end up with benchmarks clustered together based on when they were released, and thus which models were tested on them, rather than clustering benchmarks with similar content like we intend.

The stopgap fix to this is to create a distance between any two metrics, but not embed them as vectors. We let the distance between benchmarks A and B be the average (absolute) difference in Z scores across all models that are tested on both A and B. This way, we can calculate distances between all benchmarks as long as every pair of benchmarks share at least one model which was tested on both of them. The Epoch data does not have this property to start with, but after we cut 3 benchmarks with very little data and greedily cut 20 benchmarks with very low connectivity (which tend to be older benchmarks and/or quite niche), we end up with a set of 27 benchmarks that we can analyze.

We apply three methods of clustering: hierarchical, KMeans, and spectral. Then we split into k clusters, where we test k = 2 through k = 5. When k = 2 the clusters are too large to say much across all three methods, so we restrict to k = 3 through 5. The best silhouette score, which is a measure of how self contained the clusters are, is achieved across methods when k=4, so that is what I will write about here. You can play with the data yourself at this repo.

Four Clusters

1: Economically Useful Work, and also Chess Puzzles

We can see in the figure above that the bottom right section clusters together across all three methods. That cluster is:

• apex_agents_external, which tests whether the model can complete multi-step tasks across tools and APIs

• gdpval_external, OpenAI’s benchmark of model performance on economically helpful tasks

• swe_bench_verified, a human verified subset of hard SWE problems

• terminalbench_external, where an AI is asked to manage complex tasks using only the command line

• webdev_arena_external, a human-judged benchmark of which model builds a better website from a prompt

• chess_puzzles, which is a set of chess positions and the AI must find the optimal move

My best guess for why Chess clusters with this group of benchmarks targeted more at agency and economically valuable work is that Chess puzzles, and the rest of this group, are loaded on multi step reasoning. Perhaps models which are good at considering the repercussions of some action excel at both Chess and breaking into secret files. 4

2: Hard, but not research level, reasoning benchmarks

This is the cluster in the top right of the figure, which consistently contains the following five benchmarks:

• epoch_capabilities_index, a unified measure of model performance across benchmarks

• gpqa_diamond, a set of questions on biology, chemistry, and physics at a level of PHD expertise

• math_level_5, a set of the hardest competition math problems from the AMC and AIME

• otis_mock_aime_2024_2025, competition problems which are even harder than math_level_5, but not yet research level

• geobench_external, which measures how good models are at GeoGuessr.

Why is geobench here? One would think that GeoGuessr is a task that relies more on multimodality and vision ability more than anything else, and this is backed up by the fact that Gemini consistently scores quite well on it. This might just be noise, but if anyone has theories inspired by this cluster feel free to leave a comment.

3: Factual Accuracy and Information Synthesis

• deepresearchbench, where models are tasked with gathering specific information from varied sources of data

• simpleqa_verified, a set of questions with factual answers (e.g. “On what day, month, and year was Algerian artist Mohammed Racim born?”)

• lech_mazur_writing, a rubric graded writing benchmark meant to be “a proxy for real-world writing assistance quality” 5

Once again, there is an outlier in the group. Why does a writing benchmark fit in with two benchmarks that so clearly test ability to state facts? Well, the Lech Mazur rubric emphasizes “controllability, content planning, and fluency over longer outputs,” so perhaps this sort of writing asessment is more reliant on correctly gathering data and being anti-hallucinatory than it is on style or creativity. Indeed, a major part of the grading is the inclusion of a laundry list of required elements, which makes this cluster a little more cogent. See an example in the figure below.

4: The Leftover Frontier

Our final cluster is the least unified. It has several benchmarks on high level math and reasoning:

• hle

• frontiermath

It also contains several benchmarks which are meant to be easy for humans but hard for models:

• arc_agi_1 and arc_agi_2, which ask models to follow patterns of how grids are filled in

• vpct, where a model predicts which bucket a ball dropped onto some ramps will fall into

• simplebench, a set of tricky multiple choice questions that humans find somewhat easy but models can mess up due to extraneous information and red herrings

The remaining bunch seem to focus on long term reasoning and agency, much like group 1:

• gso, the aforementioned “collection of software performance optimization challenges”

• metr_time_horizons_external, which measures the performance of models on tasks that take humans hours or more

• swe_bench_bash, where models perform on hard SWE tasks with only access to the terminal

• weirdml_external, a set of coding tasks for machine learning that don’t follow common patterns of coding tasks

We can’t naively solve this confusing group by clustering into 5 instead of 4; this cluster does indeed split, but HLE ends up apart from FrontierMath, and GSO and Metr Time Horizons split from swe_bench_bash and weirdml. I don’t have a better theory for why this is a cluster except that its the “leftovers,” and that these are all modern, difficult benchmarks which stump old models.

Grain of Salt

There are very real limitations to this method of grouping benchmarks. Most notably, for many benchmark pairs, there are only a few models which are tested on both of them. This makes our distance metric noisy, and if we had the data for more benchmark/model pairs this could make a difference. One method to test the legitimacy of the clusters in the future would be to perturb the distance between benchmarks, say as a function of the variance of their difference across different models, and see if the same clusters remain.

Moreover, we get different results with our different clustering methods. We tried to focus here on the groupings which were robust across the three methods, and as such we did not end up discussing all of the benchmarks which were actually analyzed.

This was, more than anything, an experiment. While the results are interesting, I don’t have high confidence that they are built to last. Still, I think there should be more of a concerted effort to develop quantitative methods that group benchmarks by the distinct types of AI capabilities they track. We’ve seen again and again that performance on a few cherrypicked benchmarks can be gamed. On the other hand, when we scroll through a table of everything from research level mathematics to GeoGuessr scores, we risk losing track of what exactly it is that we are measuring.

We left Hanoi on Monday morning, driving a few hours West in the early light. It was a pretty major highway, and I was shocked to realize that it was entirely moped free. All was soon made clear—I saw through the trees that a smaller moped highway ran parallel to ours. Along the drive, we stopped for coffee at a beautiful place. It was part rose garden, part cafe, part sundry goods store. It was right on the side of this big road, but I immediately saw several other white people there and so gathered it was somehow advertised towards tourists (I’ll note here that most of the traveling covered in this post, and indeed for the rest of the trip, was planned by a touring company and my dad). Since I’d had caffeine the day before, I asked about the hot cocoa instead of a delicious egg coffee. Tu Tu translated, “they grow it here!” My dad replied, “like HERE HERE?” as he gestured to the garden around us and Tu Tu clarified “No. In Vietnam.” It was pretty good hot cocoa. We ate some pomelo and jackfruit.

30 kilometers from Mai Chau, we disembarked to hop on bikes with Tu Tu and Loc. Loc is very nice, rocks at mountain biking, and has a cute 2 year old kid. She’s his screensaver. Everyone liked Loc.

Narrow roads snaked through the lush mountains. On each side of the road was a shallow rut, and on one side this rut quickly transformed into steep mountainside covered by dense greenery. Amidst the generic jungle-y flora was plenty of bamboo and banana. On the less forested side of the road there were sparse sheds, houses, and eateries. Many of the houses were in the traditional style of the Thái people. The Thái people are one of many distinct ethnic groups in Vietnam, with different languages as well. Don’t confuse them with the Thai people of Thailand, though they are related, each being part of the Tai ethno-linguistic group. So, the Thái and Thai are Tai, and thus tied. Got it?

The Thái houses were built on wooden stilts, with neatly crafted wooden abodes squatting above the open walled ground floors. Loc explained to us the various benefits of this architecture: protection from wild animals, space for cattle, and resistance to floods.

That first day of biking was full of animals and their young. At one point we heard the clucking and cook-a-ree-koo of chickens, but I didn’t see any on the road or in nearby houses. Then, I looked to my right and—lo and behold—they were poking about on the densely forested slope above our heads! More often we’d see mother hens in that rut on the side of the road, followed by their gaggle of chicks. Adorable. I’m not quite sure how anyone kept track of whose chickens were whose, but chickens were walking about all over the place.

There were also tons of cute puppies playing with each other and some dog mothers roaming about sniffing things and being neglectful parents. I haven’t always found it easy to tell the sex of a dog, but in this case it was easy: the ones with udders hanging from their stomachs were female. Not to be graphic, but I’d never seen anything like it. I’m glad everyone’s getting fed.

Speaking of which, we eventually reached our lunch stop, with a beautiful view of a river. Loc told us they call it “Ha Long Bay” on land. We didn’t get that then, and you probably don’t get it now, but we do now and soon you will.

With sore butts we were glad to see the terrain change from forested mountain to flat paths and rice terraces. There are two rice harvests every year, but we missed both. As such, the terraces are full of the dead brown stalks of rice plants past and only the specter of rice plants future.

We passed an elementary school being let out as we biked through Mai Chau, and all the happy children ran out to hop on their bikes or their parents mopeds. It was very cute. Children have much more responsibility and freedom in Vietnam than in the U.S., as you’d guess. Some of this is borne out of necessity, but I suspect much of the freedom is due to the rarity of violent crime and the strong communal trust, especially outside of big cities. Kids learn some english in school; several times now a cheeky child biking by has yelled “HELLO” and cackled as they sped away.

After dinner at the hotel there was a traditional dance show out front that we planned to skip. Unfortunately, the waiter invited us to sit and watch with our dessert. All the more unfortunately, we six were the only people there. Very awkward. My favorite dance involved a miming of the rice harvest, where the women carried baskets on their backs which were strapped by scarves to their foreheads. My least favorite dance involved the two men hopping around the women while waving batons.

We woke up the next day with even sorer butts, but with some Advil and some seat pads we were off once again. This time our ride included a 15 kilometer long, 800 meter tall climb. The only stop before the hard ascent began was a chopstick factory along the road. Bamboo is harvested from the nearby forest, then brought to an open walled shed housing a variety of chopping and shaving machines. This place brought the industrial sector straight to the forest: rotating straps, whirring metal, and noisy motors abounded. First the bambo was chopped into chopstick length tubes, then the tubes were chopped into two half tubes, then the half tubes were chopped into pair-of-chopstick length rectangles, then the rectangles were rocketed through a tube where they were split neatly into two sticks before falling into a collection bin. Bamboo dust was everywhere, and the floor was covered in debris from every stop on the bamboo to chopstick journey. The workers all wore masks. Tu Tu said, “they work very hard, very long hours. The more they work, the more money.” Still, there was a certain conviviality to the shared work. I can only imagine where so many chopsticks go.

Everyone but Sister 1 and I switched to E bikes for the climb. To be strictly accurate and please my mother, everyone but my MOM and sister 1 and I switched to E bikes for the first three quarters of the climb. Then, thinking we were only halfway there, my mom switched to an E bike. She definitely could have made it to the top on the mechanical bike if we’d known, but we’re all glad she didn’t have to. Give the athletic beast inside some rest, eh Mom? I’m a hypocrite, though. With some 500 meters left I I asked an E-biked sister to race to the top. I’m proud to inform you beloved reader that, at least this once, man beat machine.

We cruised down the mountain a ways (thank you gravity), then enjoyed a pescatarian lunch of vegetarian things and fried sardines with some buckwheat beer. According to Tu Tu, buckwheat beer is a classic Vietnamese thing. It tasted like regular beer to me, but I’ve got no alcohol differentiation skill to start with.

We said by to Loc and got back in the van to drive the rest of the way to Ninh Binh with Tu Tu and Nam, the driver. Aside on Nam: he speaks no English, wears skinny jeans and alligator shoes without socks, and he is always smiling. Aside over.

Our hotel wasn’t in the actual city center of Ninh Binh, but a touristy suburb. Ninh Binh is an extremely popular tourist destination, because it is surrounded by gorgeous limestone hills which poke up around the city. This is epitomized by Trang An, a flowing mix of water, stone, greenery, and buddhist temples. Trang An is a UNESCO world heritage site as well as a Pomeranz approved activity. The water is a gorgeous green, the trees a gorgeous green, the boats a not too shabby green. Each boat must be rowed for you by a person who works there. The women wore the palm leaf hat that jumps to mind when you think of headgear in Southeast Asia. The men wore a rounded green hat with a narrow rim, as is typical throughout Vietnam.

I had a bit of a stomach bug for the biking days and Trang An, so I have less to say about the food. Eventually, my mom recommended me the BRAT diet, which is unrelated to British Pop: bananas, rice, applesauce, and toast. The bananas, rice, and toast, were easy. I never did find applesauce, but felt better by Wednesday evening.

That night, sister 3 found a small shop where we ordered vegetarian pho and tofu banh mi. Our server had a very funny manner, despite his unusually strong English, so we all agreed he was just a strange dude. The first highlight of my night was the tofu banh mi; the sauce was sweet and tangy, softening the inside of the bread without forfeiting the crunch of the toasty crust. The tofu was just the right firmness, with a slightly fried exterior giving way to a smooth chew. We bought one to share and I tried to order a second, but we’d got their last one! I’m still grateful for the limited time me and that banh mi shared. The second highlight of my night was when my dad was leaning down to get his water bottle and almost knocked a tray of hot soup onto our waiter. He didn’t, but the waiter froze with his mouth hanging open. Then, he said

“WOW. I am stunned.”

On Thursday morning we were shuttled to Ha Long Bay, where we boarded a small cruise ship, fully equipped with 25 cabins, a treadmill, and a rooftop bar. Oo la la. There were many such ships at harbor, picking up excited tourists before heading out into the bay. At first, I thought this a wasteful and uppity use of a day. My sisters agreed, and we communally rolled our eyes at our father’s bad planning.

We turned out to be completely wrong. While the ship did have a happy hour and a photo op with traditional Vietnamese dress (we partook in a little of the former and none of the latter), the real purpose was to experience Ha Long Bay from the inside. The views were consistently spectacular. Endless islands of towering limestone contrasted tan, black, and green to the verdant aquamarine of the water. Surprisingly, we didn’t see much wildlife excluding some watery, long necked birds which my birding consultant has identified as Gray Heron.

And some cats and dogs, actually:

Cats and dogs ended up living in the middle of Ha Long Bay exactly the same way 300 people did. Put some barrels under some planks and the planks under your house and get to fishing. That lifestyle has been essentially banned since 2017, both because the children could not receive an education out in the bay and because the village trash polluted Ha Long Bay. We rowed through the village, now maintained as a tourist attraction. Several houses still seemed quite inhabited, with hanging laundry and the aforementioned pets, so I’m not exactly sure whether some fisherpeople still go back and forth and live there part time.

I wrote this on Friday, which was a travel day, and am just now getting time and service to add photos and click post. We are spending lots of time doing lots of stuff, which is doubly bad for including everything in my blog posts. Some stuff didn’t make it. Well, it will be okay. We’re in Hoi An now, Central Vietnam’s historical port turned tourist trap. More on that to come, so keep on shining and we’ll see you then.

I am in Vietnam with my family. It’s Sunday night as I write this, and we got in on Friday night. By the time we made it through immigration at the airport it was well past midnight and twenty four hours of travel, but I felt shockingly okay. I think it’s because I got prescribed nasal sprays and I’m more oxygenated now.

We were picked up at the airport by guide extraordinaire Tu Tu. That’s a nickname and the only thing he goes by and I don’t know what official name is hiding beneath because I’m not so rude as to peak under the Tu Tu. He was immediately awesome and successfully communicated—after a few failed attempts—that he hasn’t done much travel by plane recently because he is constantly having his heart broken and reconstructed by the mercurial girlfriend he wishes he wasn’t so in love with.

We couldn’t see much from the car except for highways, mopeds, and humid air, but Tu Tu suddenly asked if we wanted to stop at a flower market. It was 1 AM and we were confused. They have a flower market at 1 AM? Yes, so they can stay be bought wholesale while fresh then resold under the hot sun. I immediately said yes, then looked to my tired sister apologetically. So it was that we strolled through flowers of any kind imaginable: roses, orchids, dandelions, tulips, that’s sort of all the flowers I know but there were others. We watched old ladies strap huge bundles of them to their mopeds and trundle away. Old ladies on packed mopeds trundling away is quickly becoming a theme of Hanoi.

No one has any personal space and everyone is incredibly nice. Hanoi’s old quarter, where we’re staying, is full of people selling food, flowers, clothes, carpets, knick-nacks and gizmos, but none have yelled at us. Occasionally someone will say “hello” or a sentence in Vietnamese but its never loud or bothersome. In the flower market Tu Tu walked us right into the back of a stall, said two sentences to the guy sitting there who nodded and continued to look at his phone, and then showed us the orchids he was selling and told us about how they’re shuttled down from the mountains. Similar experiences have already repeated several times. Hanoians are mega chill.

We left pretty soon to go get some rest, but the all night flower market is highly recommended by me.

Day 1 began with a hotel breakfast of rice balls, yogurt, and two little bananas the length of my hand. I was gonna stop at 1, but I decided that I normally would eat twice that much banana and I wouldn’t let the size the banana happened to be dictate who I am. I’m gonna have to skip past in depth food descriptions if this blog post is to have any pace to it, but the food has been uniformly delicious and full of noodles and sticky rice and tofu and mushrooms and flower stems and coconut and such. Mung beans. Great sauces. Great broths.

Our first day was tour guide free, which is the preference of my sisters and I. Unfortunately, sisters 2 and 3 did not arrive until this morning, day 2, which meant only sister 1 and I (brother 1 or 4 depending how you count) got to enjoy it. Until 4 PM, it was a day of walking around and looking at things.

The city is sort of hazy, but the humidity is quite high so we weren’t certain if it was just mist. Upon returning to the hotel and wifi I ascertained that it is not mist: it is extremely high levels of tiny micron scale pollutants. Oh well. A sizeable minority of locals wear face masks for that reason.

Everything is super dense. The roads are full of mopeds. The sidewalks are full of mopeds. Many of the buildings, even, are full of mopeds. At first I thought this would make crossing the street quite hard, but it turns out its a Truman Show situation: walk where you want and the mopeds dodge you.

My favorite combinations simultaneous moped riders so far are (riders list from front to back):

1. 7ish year old girl on father’s lap, father sitting in seat, 3ish year old standing vertically on seat behind father held by mother, mother sitting on back of seat.

2. Young man on seat, two Shiba Inus on floorboard.

We checked out Hoàn Kiếm Lake. Hoàn Kiếm translates to “returned sword.” From Wikipedia,

Emperor Lê Lợi was boating on the lake when a Golden Turtle God (Kim Qui) surfaced and asked for his magic sword, Heaven's Will. Lợi concluded that Kim Quy had come to reclaim the sword that its master, a local God, the Dragon King (Long Vương) had given Lợi sometime earlier to defeat Ming China.

This is awesome to start with, but even more awesome is that there actually is an extremely rare species of pond turtle that weighs like 400 fucking pounds and has been spotted in Hoàn Kiếm.

We then attended a water puppet show. It was lit. There was music over the show played live with Vietnamese instruments I’d never before come across. My favorite was Đàn bầu, which is like if an acoustic guitar had one string and was fifty percent whammy bar. Click the link for a TikTok Hoang sent me of a guy using one to play Last Christmas on it.

My parents and Sister 1 and I got massages. It was intense. The masseuses sort of crawl all over your back like a spider and knead you. I’d never had an experience like that and I’m glad I did. The worst part was that I could hear my dad’s masseuse saying “so handsome, so handsome” and then mine said the same to me and I said “oh haha” and cringed deeply thinking of the market forces from which “so handsome” emerged. The second worst part was when I thought my hip flexor was gonna explode.

We went to bed early and I woke up feeling absolutely spritely at 6:45 this morning. Sisters 2 and 3 arrived from the airport at 8:30, meaning I had plenty of time to go for a run/walk/bob dodge and weave through the city before we all convened at the hotel. That was probably the highlight of my day. I started by heading to Train Street, which I knew we didn’t plan to see as a family. A tiny black dog happily vaulted itself along with me for a minute on my way there before I left it behind to pee on a lamp pole. There are actually several train streets in Hanoi, where railway lines are closely abutted by storefronts and cafes. They’re brightly decorated and quite pretty. The one I saw this morning had it’s very own cat which possessively cat-ed around the place.

Then I continued on to the park around Bảy Mẫu Lake. It’s a massive lake to be in the center of such a dense city, with nice paved running trails going all around it. I hadn’t seen any joggers around the city, but the park was surprisingly full of runners. Lots and lots of runners. Many in matching shirts. With bibs… I’d stumbled upon a race. I later ascertained that it was a 5k/10k fun run, and I started pushing the pace a bit more as I picked up on that energy. Because of where I was located on the course, most of the people around me weren’t too fast. There were 3 exceptions. One was a barefoot girl who couldn’t have been older than 13. The other two were her barefoot friends. They were absolutely cruising along, I clocked them at a casual 6:50 pace and they did not seem bothered in the slightest. I had a lot of fun pacing off them. Luckily, none of the local runners noticed that I was a psycho foreigner chasing their kids.

The whole park was awesome; there were old folks doing a salsa class (at 7:30 AM no less), families enjoying tea by the lake, and I even saw a giant hydraulic pole smashing up paste in a giant stone mortar. Cool.

Buddhism is the single most common religion in Vietnam, although it’s not close to a majority. Our first stop of the day with Tu Tu and us 6 Pomeri was the Trấn Quốc pagoda. It’s on West Lake, which is by far the largest lake in Hanoi. The Pagoda was quite beautiful. Tu Tu explained Buddhism with helpful anecdotes from his own life (“maybe I am eating with my parents, and I am thinking ‘they want me to get a girlfriend, they are worried about their job, they are hungry for dinner,’ but if I am with enlightened person it is just… calm”). He’s atheist but really got into it. Workers were climbing the baobab tree at the shrine to string up festive holiday lights. Catholicism is the second most common religion in Vietnam, we learned.

We next drove to a site that leaned much more Confucian, the Temple of Literature. There were lots of high school students there taking photos in fancy Vietnamese dress in honor of their graduation in the coming months. I’ll admit that I remember very little of the Temple of Literature: I’d had an egg coffee (delicious, recommended) and remembered a math problem Adin had sent me, so I mostly thought about how to expand a compact set of finite measure into an open set with less than twice the measure while I looked at the statues. My Dad made a joke about how the Confucian husband-wife relationship is swapped in our house. Bet Confucius never heard that one before.

The rest of the morning was similarly formatted around driving places then slowly peering around while Tu Tu explained stuff. He’s a great guide with a lot of history knowledge and a fire in his heart for Ho Chi Minh. Which, after today, I can’t blame him for. We went to Uncle Ho’s mausoleum, now the coolest piece of Brutalist architecture I’ve ever seen. Later, at the Imperial Citadel of Thăng Long we learned some about the Northern Vietnamese operations during the Vietnam war. This might be confusing, since the citadel is more than 1000 years old. We did read a bit about its imperial history, but soon continued on to a museum about the citadel’s expansion to bunker and headquarters for the Northern Vietnamese military. There were posters I forgot to take photos of with extremely cool block art Vietnamese women standing on felled B52 bombers. The translated nationalism/communism speak is incredible: lots of “the greatly meritorious” and “glorious determination” and stuff like that. I was surprised that the museum’s spin on the history included very little about how tragic the war was for the Vietnamese. Instead, the commitment, pride, and military success of the Vietnamese was emphasized.

We had a great lunch where Sister 2 got a cake. She was was extremely tired and jetlagged and done with everything having arrived that morning, but she was a total trooper. Happy Birthday Sister 2!

Speaking of tired, I am, so I’ll speed through the afternoon. We finally escaped the car situation and walked around much more, peering into stalls and shops. The B52 made its return on this artwork we saw on the wall of an elementary school:

and we learned firsthand what train streets do to traffic:

I’m really tired now. Future entries, if I write them at all, are sure to be much shorter. The next three days are biking outside the city. Thank you to rice noodles, Tu Tu, and you, dear reader, for making this blog post possible.

Subscribe now

The strange man asked what he studied with hardly a preamble. Mitchell didn’t even catch the guy’s name. Mitchell never said that he was a student, or that he was on a break from school. The guy just asked what he studied. They were walking up the trail, counter clockwise around Mt. Erwin, so that rows and rows of the white granite shark teeth laid out to Mitchell’s right. They were mostly bald, with the trees sheltering in the flatter valleys nestled below. Mitchell did the hike every spring

“I’m majoring in economics,”

“That’s cool. Lots of jobs from that, I bet. How much time do you spend asleep?”

The guy had huge calves with veins that showed right through his thin skin. Mitchell saw lots of older hikers like that but most wore socks. This guy didn’t have any socks on. He wore a fisherman’s shirt with two buttons undone and cargo shorts too. He was covered in pockets. Every single one looked full.

“How much time do I, well–”

“Per night. On average. Over the last year. Per night on average over the last year how much time do you sleep per night?”

“Well, probably on average around 8 hours. Give or take.”

Mitchell wondered where this was going. He liked talking to strangers and he didn’t mind the occasional oddball, but this guy had intent. It was offputting. He’d walked right up to Mitchell and said “what do you study.” One can’t help but notice that.

“Ok, and how many hours do you spend awake?”
“Well, 8 asleep means sixteen awake, right?”

“Not at all, not at all. If you spend 8 sleeping then you spend sixteen not sleeping, I’m right there with you, and this is really my fault. But I forget that people don’t know what I mean when I say awake because how could I mean anything else? You know what I’m saying? I don’t mean not sleeping. I mean,” and he snapped twice right in front of Mitchell’s face.

“Awake.”

When he snapped Mitchell saw the hairs on the back of each of his knuckles. You don’t think of hair as angular. The hair was angular. Mitchell stared at the guy’s feet. His shoes were completely flat, with almost no soles to speak of. Whatever had been there was rubbed clean off. The laces were solid green, a totally different color from the faded blue shoes. He wasn’t wearing socks. Mitchell could see the squareness of the knuckles poking through the shoe. He was awash with jealousy at the thought of the guy’s bare feet. How much time did he really spend snap-snap awake?

“I don’t know. 5 minutes, maybe. I don’t know how to tell.”

“Well I’ll take your word for it Jimbo. It’s your world. Let’s take your word for it. That makes eight hours asleep and five minutes awake and fifteen hours and fifty five minutes which are neither, which we’ll throw away for now but you should work on that, but even if we triple it and give you eight hours asleep and fifteen minutes awake, which I think is achievable for you with a few months of work, that’s ninety seven percent of your life asleep my friend. Ninety seven! I’m no mathematician, but that’s gotta matter.”

Mitchell let it wash over him. It was better as narration than conversation, better as axiom than fact. To his right he had the mountains and to his left he had a man with no socks who was apparently no mathematician. He looked to his right and listened as he walked.

“So we’ve established that you spend the vast majority of your life dreaming. When you’re not asleep and you’re not awake you can’t be much except for dead. We probably agree on that. Do you worry about your dreams, Jimbo? Do you get nightmares?”

Mitchell rarely remembered his dreams. Sometimes he’d wake up with the sweats, but he could always walk to the bathroom and pee in the dark and that made him feel better. The back of his neck cooled off while his piss hit the water.

“Not really, sir.” He smiled out at the mountains. “I don’t remember them, usually.”

The guy loaded up some phlegm and Mitchell heard it splat into a rock.

“You think I’m crazy, which is true in a few ways. If you ask some idiot what Einstein would say, you get—and I love idiots so don’t get me wrong—but you get: ‘insanity is doing the same thing twice and expecting different results.’ It’s Einstein, right, so it’s gotta be right, and that makes me insane. That’s how they think.” He snapped, and it sounded like a wood mallet on a wood bell. Mitchell counted four echoes off the granite peaks. “I’ve got no issue with that, no issue with that type of insanity at all. Einstein didn’t believe in quantum physics. He couldn’t stomach the uncertainty. People forget that, because he tripped and fell face first onto the photoelectric effect which I suppose smelled like quantum in some experimental sense. You got that for the quiz? Get a grip Jimbo. You look like you’ve never spoken English before. Particles aren’t little beans sitting there. They’re probabilites, that’s the point, chances that this hits that or that moves there, and a chance is just a chance. Schrodinger’s uncertainty principle, you ever heard of that? It’s all probabilities.. You, you’re made out of probabilities too and not the one of them is a guarantee. It’s a roll of the dice. Our guy Albert was a geometer. Solid shapes and solid colors, that’s what Einstein had figured out. It got him an A+ with honors in kindergarten and the Nobel. Well deserved, too. He was wrong about insanity, though, since it’s all color and no shape. That’s why he couldn’t kick it with quantum. Quantum physics proved him wrong about insanity and I’ve learned the lesson he didn’t. I do the same thing twice all the time, and I always expect different results. Watch this.”

He dropped to a squat without warning. Mitchell turned around two steps later to find him holding up a plum sized rock between his pointer finger and his thumb. This guy was some sort of performer for sure. Mitchell couldn’t tell if he was still talking crazy, because he’d never understood much physics. A nice boulder sat to his right, sloped towards the trail and bleached flat by the sun. He leaned right onto it with his hands behind him and took in the show with a boulder warmed ass. The guy jumped up onto one leg, wound himself up like a pitcher, and absolutely slugged it off the mountain. Mitchell lost sight of it in a blink. There were clouds, but just a few.

“Wow, man, you play baseball?”

The guy didn’t respond. He stared intently at where the rock had disappeared. After another few seconds, they heard the crack of stone on stone.

“You got that?” he asked.

“Yeah, I heard,” Mitchell said. He watched the guy breathe. As he inhaled Mitchell could see his two breast pockets moved upwards. It was another mile or so up the mountain, with rock scrambling at the end that would make for slower going. Then, what, four miles down? That would be three hours or so. Mitchell couldn’t imagine what this guy would be doing in three hours.

“Good. That was four seconds. Now can you guess what I’ll be doing next?”

Mitchell was beginning to see the game he was playing. The guy was smart, Mitchell decided, but not a genius. A smart man on a journey gone twisted. He’d throw a rock again and the sound would be different. Obviously, though, that wasn’t Einstein’s point. That’s getting the definitions all mixed up, Mitchell thought, because when Einstein said the same he really meant exactly the same. Sure, you could call both actions throwing a rock off Mount Erwin, but the direction of the throw, the currents of the wind, the shape of the stone would all be completely different.

“You’ll throw a rock again, but that’s not really the same.”

“You’re half right Jimbo. Exactly half right.”

Mitchell turned around, and there was the guy squatting with no socks. He had massive ankles, like dinosaur bones. He was holding up another rock, but

Mitchell’s vision teetered and Mitchell teetered too. He stumbled sideways and ended up on his hands and ass on a sun warmed rock like he was lying on Emma’s carpet. Some time now to be thinking of Emma’s carpet, he thought. He was sure he’d throw up. No, thank God, he wasn’t going to throw up. What had happened to the shadows? They still sat dusty blue but something had happened to the shadows for sure. The guy was already standing, already lifting his left knee like a pitcher. Plum sized rock. The vertigo played back in Mitchell’s head and he saw the shadows as they had been in that moment. Just for a centimeter, just for an instant, his shadow had walked back towards his face. Mitchell’s eyes were locked on the guy now. He knew what was coming. Same rock. Same throw. Same sun same wind same minute same second same photons on his face. Same warm ass. No way. Mitchell grasped for reality without moving, scanning his perception for any difference. He felt insane. His heart must be pounding, because he believed it now. His heart must be pounding like it hadn’t been before. He checked. It wasn’t. It hadn’t caught up to his head yet. His head couldn’t change shit. He knew for certain that whatever electricity was sparking in the gray matter in his skull was just the same as it had been before. The shadows had walked back and so had his bones and the wind and all the electricity in his brain. He wondered where his mind had gone to think these new thoughts.

The pitch was a perfect match. The guy’s elbow went so far back Mitchell thought his shoulder would dislocate. The fabric of the fisherman’s shirt bunched at his shoulder blades. One lunging step and the plum exploded out to the same point in the sky. It was gone in a blink. Mitchell couldn’t look away. The guy stared after it and began to count

“One,”

There was a cloud shaped like a pillow, sort of, where the rock had disappeared. The edges were wispy, spreading apart. Mitchell pictures rain as clouds falling apart, but it always rained from the thick, gray beasts. Why couldn’t it rain from wisps? Probably the guy knew. Mitchell considered that he could google it later, but thought it was a probable matter of fact that he wouldn’t. Probably, he thought, I’m never going home.

“Two”

Mitchell decided to close his eyes. If the fourth second came and the same crack of stone on stone echoed back to him, he wouldn’t be able to handle anything more. He didn’t know what he’d do. He loved these mountains, he told everyone that he loved nature and he loved to hike but at the end of the day in the most important instant of his life it would have to be the darkness of his closed eyelids. And the warmth on his ass. He couldn’t handle anything more.

“Three”

There wasn’t magic, and he wasn’t insane, but there could be God. He didn’t know what that had to do with this guy, but surely something. Mitchell never thought he’d meet an angel on Mount Erwin, but he never thought he’d meet an angel anywhere. Mount Erwin was probably the likeliest spot.

“Four.”

Mitchell felt a prickle in the innermost pocket of his right eye. Not prickling, but a single prickle. It was the minute beginning of a tear. The smallest bit of proof that the physical world had changed from the last time he’d lived this moment. The reason for this, though Mitchell didn’t know it, was that the plum rock was out of range. Causation can only travel as fast as the speed of light. To travel a mile, the distance to the plum rock, it takes light five microseconds. The rock would hit the ground in one microsecond, give or take. The bubble of causation which would spread inexorably through spacetime from the prickle of Mitchell’s eye would reach the rock much too late to change anything at all.

Mitchell wasn’t insane. For a millionth of a second, he awaited the crack of stone on stone.

The guy grinned at Mitchell in the complete silence. He said:

“See? Quantum.”

A note of warning: I include examples of cards from my own decks. That means they are mostly from courses which typically come after multivariable calculus and linear algebra. Following the meaning of the math hopefully won’t be necessary to get the spirit of the notes.

A penultimate note of warning: This is exactly what it sounds like. It doesn’t have much humor, there are no cute anecdotes, and if I manage to trace the shape of the human condition through my contemplation of memory and the mundane then I certainly did not notice. It’s just notes. Notes about using Anki for math.

A final note of warning: This is not about using spaced repetition to learn math. I’ve never used spaced repetition to learn math. Instead, I’ve learned a piece of math with pencil and paper and textbook and all that. Then, I’ve used Anki to forget less of it.

Why I use Anki (as opposed to nothing, not as opposed to some other software)

I am an undergraduate student studying math. About a year ago, at the start of my Junior year, I began using Anki for math classes. For that reason I remember much more about complex analysis and differentiable surfaces than I do about cryptography and multivariable calculus. That’s a shame, because now I will never be able to encrypt all the data I’ve been storing in the divergence of my vector field. On the other hand, the contrast was evidence that this spaced repetition stuff works. I don’t remember all the proofs from my post-Anki classes. I don’t even remember everything that I tried to store in my cards. Still, I remember a whole lot more than I would otherwise.

Types of cards I make

What type of thing?

When I see a map Phi in the context of Brad’s Big Theorem, it is way easier to comprehend if I instantly know that Phi is a function from the integers to the reals, or an operator that acts on simplexes, or whatever else. Cards for this that I use looks like:

where I fill in the range and domain

Often I know about some property and have a fuzzy mental model of the vibes behind it. It is super helpful to have instant recall of what exactly the property applies to.

I have several cards like the above, ensuring I instantly know what some small gear means when I encounter it in a larger piece of machinery.

Examples and Counter Examples

Knowing canonical examples of an object, examples that are perverse in some way, and snappy counter examples for plausible false statements is great. It helps me build intuitions while avoiding false assumptions. This kind of card is especially useful for propositions where the converse isn’t true, but both directions feel plausible (A—>B but not B—>A). Often, these cards don’t tell me that I will necessarily be providing an example on the front, since often counterexamples are used to answer broader questions like “is an everywhere continuous function necessarily differentiable?” Cards that I use for this look like:

Q: Prove or give a counter example that all solvable groups are abelian:

Q: Is a map with a locally isothermal chart always conformal?

A: Yes

Q: Does a conformal map always have a locally isothermal chart?

A: No

It would probably be good to add a counter example to go along with the second question as well.

Proofs

I find it difficult to memorize whole proofs. I have not memorized many whole proofs through Anki. Largely, I’ve shied away from this type of card due to the effort required. I still firmly believe they can be quite effective, which is what the aforementioned Nielsen post is mostly about. I sometimes add proofs if they are very short, or break them down into small components as best I can, or include a hint in the card. Some examples:

Q: “If for some power series we have

then the radius of convergence is 1/L. Give a heuristic of the proof.

A: The limit test yields |z-z_0|/L, so when |z-z_0|<1/L we have that the limit test is less than 1 and the function converges

Q: What is the only thing we need to show to prove that DN_p is self adjoint?

A: It is sufficient to prove that DN_p is self adjoint on a basis of the tangent space

Intuition Cards

The rare and scintillating intuition card is something of a unicorn in my math decks. I am unsure if it is helpful but am leaning towards yes. Sometimes I earn a new intuition through exercises or play, or come across someone else’s clever way of conceptualizing a piece of math-y content. “When I think of it like that, it’s obvious! I will never forget this ever again.” But then I do. So, I try making cards like:

Q: Give an intuitive explanation of a ruled surface

A: A surface that can be made by the ‘sweeping of a line.’

Q: What are two ways to think about the Augmented Homology?

A: Homology with a copy of Z removed from the zeroth homology, or homology such that the point’s homology is entirely trivial.

Short Answer Section

Short answer cards are similar to multiple choice questions on a speedy quiz. Math, famously, requires cognitive work, but many questions are almost-instantaneous-level-easy if you understand the key info. Make sure these “trivial” exercises are very, very trivial. If you make a deck full of cards that are easy after just a little thought, then a whole day’s review will require tons of thought, and that’s nem jó. Still, these quiz style cards are fun and ensure you can apply facts you’ve learned. Often, I use multiple quiz questions for one idea: the combined effect of cards “is 42 even,” “is 67 even,” and “is “-1 even” is a pretty good proxy for knowing what even means. Of course, I could just memorize the specific answers, and if I notice myself doing this I reassess those cards. Here are some examples of short answer cards in my deck:

A: Yes

If I have a good memory of what the graph of x|x| looks like, then this is obvious. If I’ve lost that mental model, it’s a tossup.

Q: How many edges are in the 3 uniform complete graph on 4 vertices?

A: 4

A: ω

It was hard to find examples of these cards. That’s not surprising, given that definitions and proofs are provided in textbooks and short form questions to quiz my understanding aren’t. Still, I love ‘em when I see ‘em so I plan to start making more.

Open Questions

1. How often should I memorize proofs?

   The rule of thumb for Anki is usually “if its worth 5-10 minutes of your time to know it forever.” With a proof, however, it requires many cards, and I might have to modify them later as I realize there are small errors, and maybe I want to change the structure of the proof because—this has become more than 10 minutes. So given a proof X, what function f(X) gives how much time it is worth to me to memorize the proof?

2. Should I spaced repeat time consuming exercises?

   If I can’t solve a problem about some material now, and I could when I learned it, then I haven’t maintained my understanding. Still, this seems like a quite strenuous practice that should be separate from general Anki memorization. So, what’s the best option? I could build a deck of exercises that I study separately when I’m extremely locked in, or let my “exercise ability” atrophy and hope I it comes back quickly when the material became relevant, or wait for Math Academy to expand their catalogue soon to do all my work for me.

3. When does information become “useless?”

   Math is known to be shockingly interconnected. I have taken classes that have been irrelevant for a semester, and then suddenly helpful in a subject I thought I would be unrelated. Being a broad mathematician is both fun and helpful. Still, these math decks can be quite heavy, a real time commitment. If I haven’t used my graph theory knowledge for 3 years, should I still be reinforcing graph theory? I don’t know how to decide this cutoff.

4. What are the best scheduled practices to playfully explore math while maintaining accountability?

   At the end of the day, the spirit of math is not memorization. I can use Anki to stabilize my recall, the framework of knowledge that I stand on. That’s good, but I don’t just want to stand on the framework; I want to jump about, do somersaults, hit my head and hang upside down. One might work through textbooks/course notes, do puzzle style problems, or pause and ponder.

Smash that like button and leave a comment if you have any thoughts to share on using SRS for math.

Staggering through premonitions of my death

I don't see anybody that dear to me”

Fleet Foxes, Tiger Mountain Peasant Song

I am finishing up this piece having moved today, and just opened my computer to see the screen is cracked and half black. Still, a pact is a pact and I did not write a blog post this month. I can’t let my friends down! With that said, this one is a brief introspection; if the valiant fight for beautiful truths of the world are your kick, tough luck and see you next time.

I.

I spent four days earlier this month hiking in the White Mountains. It was supposed to be six, but intense knee pain cut my trip short after an overeager start. The segment of the trip I completed was focused on Franconia Ridge and the Bonds, two locales I recommend highly. If you want more information about my route or advice for hiking in the Whites, you can message me and I’ll try my best. 

While on the trip, I had a lot of time to think. Since my days as a young’n watching Adventure time, I’ve seen the narrative trope of a character going to nature to introspect before coming to a major realization. If they go really far into nature and think really hard, the realization might smell like enlightenment.

This is extremely far from my experience. I do my best thinking with some combination of access to the internet, a peer to bounce ideas off of, and a writing utensil. A regular heart rate also helps. All the same, after spending so much time in the woods and I was bound to have at least a thought or two. Most of them were short lived and not impactful, but in their proportions I gleaned some information about myself.

Everyone cares a lot about people, and I already knew that I did too. Still, I hadn’t realized that friends and family would persist so heavily in my thoughts while I maintained basically no exposure to real life humans.

In stretches lacking external stimulation (let’s say five flat-ish miles with similar terrain, no people, and no view), I felt much more interested and excited by my internal monologue when it rambled on about my social life. This was unexpected; When passing the time in my head for less lengthy stints, like a spacey hour at shabbat services, I’ll occasionally find myself thinking about some basic math problem of which I have a poor understanding. Don’t tell Mom, but last week I stood in the synagogue kitchen trying to piece together how quotient rings work from my dusty memory instead of setting up the lunch. Although with helpful tools—the internet, a peer, or some paper and pen—I might resolve such questions in a jiffy, it can be a fun exercise to think them through without outside resources. It certainly passes the time. 

Yet no questions of math, physics, politics, or the social and economic state of the world came to mind (except for considering why the first ~40 hikers I saw were exclusively white). Instead, I mentally tallied my favorite person who I know well and favorite who I do not know well from every course I’ve taken in college thus far. At some point I started noting ideas that I thought would make for interesting parties. 

I think there are a few factors that pushed me in this direction. One is that I love to talk, and was starved for human contact. On the way back from my trip, waiting for pizza with a friend in New Haven, I kept losing the conversation because I could only think about food. I experienced a similar hunger for conversation on the hike. At one point, I didn’t see another human being or man made object, excluding trail, for 20 hours, which must be the longest such period in my life. As opposed to making me meditative and uncaring, a mountain hermit, it made me very eager to be with people. I really like speaking. Not much to do about it.

Another reason I thought little about anything weightier is that I was physically exhausted, and had to maintain some level of focus on hiking. When thinking hard about an interesting question, alone or more often in conversation, I’ll occasionally stare at some arbitrary fixed point or slow my walk to a crawl. This is great, because it makes me look cool and mysterious. Also, it avails me of visual or proprioceptive distractions which derail my train of thought. So, even the flatter periods of hiking were not ideal for considering anything too deep. Concern over the sharp knee pain that eventually ended my trip early was especially distracting.

Although I’ve gotten my human conact fix since the trip, I’m still very excited to see many friends who I haven’t in months as we all trickle back to school. People interest me about as much as anything else on this planet, and I am surrounded by exceptional ones. I’ll add “pass the time on Ben’s hike without being there” to their long list of skills. 

And you're not real you, is inside your real you

Oh wow, how amazing, and interesting too

But in this digital world, what can we do?

What can we do?

- Don’t Hug Me I’m Scared, The Computer Song

Topics: The Halting Problem, Paradoxes, Jesters, Busy Beaver Function

I. The Jester’s Last Laugh

There once was an aged and wizened jester, who had seen and heard much in his years. So wide were his travels and so full was his head with stories and songs that he hadn’t considered his duty to the King in quite some time. Well, he had considered it occasionally, but whenever he did he was quick to stop doing so. The Jester was old and his legs were tired, and he realized that there was no longer reason to walk the dirt road with bells on his toes. So, after seven days on foot and seven days on carriage the Jester proudly returned to the King’s court, assured by the fact that before his long waltz abroad he had been esteemed for his years of service.

Yet when the Jester crossed the Palace gates, he was seized by guards and dragged towards the King’s throne room. So feeble was his frame that the fabric of his curled shoes barely grazed the marble floor as he was hauled along.

Eventually, they passed through the imposing double doors known empire over. The Jester was thrown to the tiled floor before the King without ceremony, as a grocer tosses a sack of onions into the cellar. He looked up at the liege of his bygone youth with tears in his eyes and no words on his lips.

The King spoke without delay: “My Jester. You served me well, once. This place was a happy place, full of mirth, full of gaiety. Perhaps I owe that to you. And yet you left without a word, and I was confused. Did I not treat you well? You were counted among the most favored of my advisors, even as you danced your foolish dances and told your confounding riddles. I declared that your escape from the palace had been an exile, rather than a desertion, so that none would know how you failed my trust.”

The Jester turned away his eyes, for he knew what was to come. 

“Yes, you are an old friend, but I am a man of the law and the price of a broken exile is known to all,” continued the King. “I am sure your travels have been long, but I wish you had not returned. For breaking exile and entering the palace from which you were shunned ten and eight years ago, your punishment is death.”

The Jester still did not look up from the floor between his hands. One elbow fell to the tiles. He teetered there for a moment, but still did not speak.

“And yet you are not the same to me as another, not the same to this empire as another. For this, Jester, I grant you the dignity of choice. Return here in an hour, and speak one sentence only. If it is true, you will be beheaded. If it is a lie, you will be hanged.”

The guards lifted the Jester up, and turned to walk him out the door. The Jester continued to gaze at the floor, but what he saw could not be told.

One hour later, the Jester stood on his own two feet in front of the King, grinning.

“I see you are feeling better now, Jester! This is pleasing to me. Every man should come to terms with his fate before it arrives, and a jester’s cup should taste of laughter and not sorrow. Speak to me, then, give your final sentence and have done with it.” 

“Tonight, I will be hanged.”

The King frowned and took in a breath to tell the executioner to bring the rope. Then he stopped and shook his head, disappointed to realize that in fact the Jester had told the truth and chosen the more gruesome beheading. But no, that wasn’t right either. The King looked down at the tiles below his throne, and his mind was elsewhere as the Jester’s had been. He looked up at his old friend, who beamed at him expectantly. The King threw back his head and shook with great, booming laughs.1

II. The Halting Problem

This classic story is built upon an example of the Liar Paradox; the Jester’s last sentence is logically identical to the phrase “this sentence is a lie.” Let’s assume it is a lie. Well, in that case it is true. Fine, so let us assume it is true. In that case, it is a lie. Truth implies falsehood and falsehood implies truth, a manner of contradiction which is actually the basis of several elegant and important proofs. It is the same strain of logic I expected to see in the proof of the Halting Problem.

The Halting Problem is “the problem of determining, from a description of an arbitrary computer program and an input, whether the program will finish running, or continue to run forever.” Lets look at an example program, f(x), and try to determine when it halts.

f(x):
a = x - 10
If a < 0:  
   return a
Otherwise: 
   add 1 to a until it is less than 0

x is our input here. If x is less than 10, a will be less than 0 and the program will immediately return, or halt. Otherwise, if x is equal or greater than 10, a is greater than 0 and we start adding 1 to a until it is less than 0, which will of course never happen, and thus the program never halts. So, for example, f(2) halts, and f(11) does not.

Turing famously proved that there is no computer program, no algorithm or specific set of instructions, which can solve the halting program for all pairs of programs and inputs. Thus, the Halting Problem is “undecidable,” meaning there is no algorithm which always arrives at the correct true/false answer to it. From now on, Halts(i, x) or h(i, x) will refer to a program which always returns 1 if i(x) halts and 0 if i(x) runs forever. It is a hypothetical solution to the Halting Problem. We will prove it cannot exist.

Wikipedia gives an elegant and rigorous (to my untrained eye) explanation of Turing’s proof that Halts cannot exist, that I found I could parse as a relative layman. Before they do, however, they include the following proof sketch which comes from early computer scientist Christopher Strachey (brackets are me):

Suppose that there exists a total computable function [an algorithm which always returns true or false] halts(f) that returns true if the subroutine [algorithm] f halts when run with no inputs and returns false otherwise. Now consider the following subroutine:

def g():
    if halts(g):
        loop_forever()

halts(g) must either return true or false, because halts was assumed to be total. If halts(g) returns true, then g will call loop_forever and never halt, which is a contradiction. If halts(g) returns false, then g will halt, because it will not call loop_forever; this is also a contradiction. Overall, g does the opposite of what halts says g should do, so halts(g) can not return a truth value that is consistent with whether g halts. Therefore, the initial assumption that halts is a total computable function must be false.

Aha! Exactly the kind of thing I thought was coming, just like the riddle. A classic example of inconsistency, the computational analog of “this sentence is a lie”. Just like the Jester’s sentence evaluates to false if its true, and true if its false, our function g runs forever if it halts, and halts if it runs forever. Thus, we must not be able to compute whether it halts at all, because then this inconsistent paradoxical function would exist, and logic breaks and the rivers rain up at the sky.

But Wikipedia forges ahead, stating, “The concept above shows the general method of the proof, but the computable function Halts(x) does not directly take a subroutine as an argument; instead it takes the source code of a program. Moreover, the definition of g is self-referential.” In other words, I thought to myself as I read, the real proof will be the same thing with fancy math logic written in. And also, they need to find some cheeky way to avoid the fact that the function g references itself in its definition, which I guess logicians don’t like.

Instead, the full solution quite surprised me. I find myself disagreeing with Wikipedia2 that Strachey’s proof “shows the general method” of Turing’s. While only slightly more complex, Turing’s proof moves away from the strategy of creating an impossible function which halts if it runs forever and runs forever if it halts. With a little effort, Turing’s alternate strategy is comprehensible with no math or CS background. Here it is:

We define the halting function to be a computable two-input function h(i, x), where i is some arbitrary program3, and x is the input that goes into i. To keep that in mind, from now on we will sometimes write the function as h(i(x)), but remember that i and x are both inputs to h - it is a two input function. Recall that if i(x) halts, h(i(x)) returns 1. Otherwise, it returns 0. So, in pseudocode, h looks like this:

h(i, x):
   If i(x) halts:
      Return 1
   Otherwise:
      Return 0

Now, we will attempt to prove that for any total computable (always halts and and yields an output, can be found with an algorithm) two-input function f(a, b), any of these functions at all, f is a different function from h. This would prove that h cannot exist, because h itself is defined to be a total computable two-input function and so h cannot be different from every single total computable two-input function. That would imply h is different from itself!

Firstly, we call our arbitrary function f, like above, and keep the halting function h. To prove f and h are different, all we have to do is find two inputs a and b where f(a, b) and h(a, b) are different. To be the same function, they would have to produce the same output for every single pair of inputs, and thus this one inconsistency is all we need. Since f is totally arbitrary, meaning it can be any total computable function at all, we have no idea how playing with a and b will affect f(a, b). So, we can’t just compare the outputs of f and h on different inputs and search for an output where they differ. 

Instead, we can find a set of inputs where the output of h is dependent upon the output of f, such that if f does one thing h will do the opposite. Well, if we want h to be dependent on f, we could try passing in f as an input to h. But, if we input f to h, h will always return 1, since f is defined to be a total computable function, which means f will eventually halt. Plus, f takes in two inputs, and h only passes 1 input to the function it checks, so I’m not sure this is even allowed. That strategy is a dead end. 

Instead, it seems we’ll need an intermediate program, like a passageway between our arbitrary program f and our halting program h. Let us call this intermediary program e. Since we want h to base its results upon this program, we’ll pass it in as the first input to h. This gets us:

 h(e, x) #Checking whether "passageway program" e halts with an input x

We now need to decide x. Remember, x is the input which will go to e. How do we want e to behave? Ideally, the passageway program e will halt if the arbitrary function f(e, x) = 0, and run forever if f(e, x) =/= 0. Why is this ideal? See below for the explanation, but I recommend writing it out or thinking about it first.

If f(e, x) = 0, we want h(e, x)/h(e(x)) = 1, because we want to prove the two functions are different, which means we want e(x) to halt. We’d like e(x) to run forever if f(e, x) =/= 0, so that in that case h(e(x)) = 0. Then, for the inputs (e, x), f and h would definitely be different, which would mean they must be different functions altogether. Now we are getting somewhere! Ok, let’s write out what we have for e now.

e(x):
   if f(e, x) = 0:
      Halt
   Otherwise:
      Run forever

Uh oh. We are doing self reference again, and we know from Strachey’s idea that self reference is not allowed. Note how we explicitly mention e inside our definition of e, which is circular, and cheating, and how could we have been so blind. Shoot shoot shoot! We need that self reference to e to pass as an input to f, because we are already passing in e as the first input to h and we need to compare the functions f and h when they have the same inputs. The comparison wouldn’t show anything if we were comparing f and h across different inputs.

Aha! Here’s an idea (Turing’s, not mine):

e(x):
   if f(x, x) = 0:
      Return 0
   Otherwise:
      Run forever

See the fix? The solution is that we can sneak e into itself if we pass it in to the function as the input. Now, e no longer references itself in its definition. Instead we take the input of e(x), labelled x above, and pass it in as both inputs to f.

Then, we set the input of e to be itself. To get a visual of this, just replace every x in the code above with e. When we pass in e as the input to e (aka set x equal to e in the diagram above), we’ll be able to compare f(e, e) to h(e, e). Because e can’t reference itself, the way we are doing this is by comparing f(x, x) to h(e, x) [AKA h(e(x))], and setting x equal to e. That is a lot of e’s, but the big idea is that h(e, e) will compute its result based upon whether e(e) halts, which will in turn be decided based upon the output of f(e, e), and so we’ve connected the result of h(e, e) to f(e, e). It looks like we’re golden, but let’s walk through once more to be certain.

What happens if f(e, e) returns 0? Then, e(e) halts. So, when h(e, e) checks whether the program e halts with input e, it finds that it does, and returns 1. So, when f(e, e) = 0, H(e, e) = 1 and f and h must be different. I’ll let you check through the other possible case, where f(e, e) != 0, and if you find that you don’t get the expected result—don’t find that f and h are different—let an important person know because that is very big news.

This proof moved away quite a bit from the whole “this sentence is a lie” dynamic. If you give someone English, they can make an impossibility, a paradox, a sentence which contradicts itself, so English isn’t consistent. This doesn’t mean English doesn’t exist, but instead it means English doesn’t have an entirely logical rulebook. If you give someone a halting function, and let functions reference themselves, they can create an impossibility, a paradox, a function which contradicts itself in just the same way. That is what we did with help from Strachey way at the beginning. However, this becomes a non-issue because we don’t let functions reference themselves to begin with. So, why are we so sure there is no computable halting function now? 

We just proved why. If we imagine a total computable two input halting function exists, it can be proved to be different from all total computable two input functions, which means it is different from itself, which is impossible. Just like that, we have poofed our halting function out of existence. We know we don’t exist in a world where there exists a category of thing that is different from all things of that category; it is nonsensical. There are no cows which are different from all cows. In that, at least, our theory of logic aligns with reality. So, no Halting function. This extends too to the real world of flesh and blood and silicon and iron: no one will ever write a computable algorithm that solves the Halting Problem.

III. An Unhalting Problem

The Busy Beaver function is in the (nerd) news this month. Busy Beaver of n is defined as the longest any n state Turing machine which eventually halts will run before halting. Very broadly speaking, the number of states of a Turing machine is a limit on the complexity of the algorithms it can run. The more states, the higher the busy beaver number.

Busy Beaver, or BB, is a function that grows incredibly quickly, faster in fact than any computable function. Whatever number you can think of, written however you want, is almost certainly much smaller than BB(7). BB of 1-4 have been known since 1974, and the reason the fuzzy function is back in headlines is that BB(5) was found by a team of mathematicians and computer scientists this May. They shared their result just this month. BB(5) is now proven to be 47,176,870.

This could very well be the last Busy Beaver number humanity ever finds. It has already been proven that 4

The usual gargantuan number shtick of “ten times the number atoms in the universe” just doesn’t come close. There is, however, much more that is interesting about this function than magnitude. I considered trying my best to explain the many questions this function raises. How is it defined? How can we know that it is greater than any computable function? What does the Halting Problem have to do with this?

That all sounds quite clickbaity, but I can rest easy knowing that the click is well deserved, and the bait lures to another’s work. Ben Brubaker at Quanta Magazine wrote a phenomenal article, With Fifth Busy Beaver, Researchers Approach Computation’s Limits, that explains the BB function from the ground up (no pun intended). He illuminates the function with a much defter hand than I could. To be fair to my explanatory abilities, the article contains pictures and animations which are both quite helpful and quite pretty. The piece also takes a fun jaunt through Busy Beaver’s history. A great read. One thing that Brubaker understandably brushes over, however, is the Halting Problem. He nods to the fact that Turing proved it and then moves along. If you’ve made it this far, then that should be no problem, and I think you’ll find Brubaker’s piece all the more rewarding with an understanding of the Halting Problem.

Yet even after a first match grappling with its meaning, the incomputable nature of the Busy Beaver function can feel like arcane logical magic. So too can the grand statement that a computable Halting function is impossible. I was amazed that these confounding riddles belie a a deceivingly smooth road to understanding. Somehow, both facts fall out of the theory of computation at its most elegant, from concise, comprehensible proofs. They leave us feeling like the King—bewildered, lost, and eventually delighted.

Some people love all dogs and some people love none. I myself am a dog person, but my canine exposure is unfortunately limited, since I developed an allergy when I was 8. It was really sad. Even people who clearly love dogs are apparently confused about identifying dog quality. They turn to the dogs themselves for answers, asking over and over again: “who’s a good dog?” Fret not. I know whom.

This forbidden knowledge came to me as in a dream or perhaps a prophecy. As I walked by a dog park I peeked over at the hounds, and for a long moment I thought I was falling. My eyes flashed from Poodle to Collie to Terrier, double checking my work, but it was only a formality. I had it. The simplicity, the cleanliness of the answer was breathtaking. The closest experience I can compare it to is my first reading of the equation which defines our modern age:

While the above breakthrough is powerful, I believe my system has the potential to rewrite the entire field of Cynology, the study of dogs. With that being said, we need some definitions before we move on.

1. Good Dog: a dog that I, Ben Pomeranz, really want to hang out with, and also respect.

That is the only definition. This conjecture assumes that the dog is living with a human family, who treat it well. It thus does not apply to strays, wolves, or dogs in bad homes. This brings us to my conjecture:

In other words, a dog is a good dog (a dog that I really want to hang out with and also respect) if and only if it is nonviolent, happy to be alive, and fast. The diagram below may be illuminating.

This system is time invariant. Any dog who ever has or will exist maintains the same classification as Good Dog or not so throughout all time, although it may be hard to discern that classification before the dog exists. This is because when we classify a dog as Good or Worse, we apply our criteria to the dog in its prime. If a dog was fast in its youth, and now waddles sleepily through the house in its golden years, it is certainly still Good. Neither shall we label a dog Worse for being a sad dog after its trip to the vet. I too would be dismayed to find myself ensconced in a plastic cone.

Note that our conjecture does not state that dogs who do not meet the three criteria are bad dogs, per se. Rather, it claims only that I either don’t want to hang out with them or don’t respect them.

While some may point out that the generality of this conjecture is limited by the its subjectivity, I reply in turn with a question: does the Copenhagen interpretation of quantum mechanics not also entwine the observer in the formalities of science? That is all I have to say on that matter.

I have found that almost everyone I speak to agrees. While I am open to a disproof by counter example, I have seen none. To finish up, here is my classification of dogs that fall into each combination of our categorization12:

Disclaimer: I am not very well researched in basically anything. If I am wrong in my facts or reasoning, please correct me in the comments, on a train, or in the rain. If you enjoy the piece and think others will too, feel free to share it the world over.

I. Cat Magic

Pspspspsps works on cats. I’ve tried getting the attention of strays with plenty of other phrases. It’s a fun game which is almost always futile. “Cat! Kitty! Kitty kitty kitty! Gato! Gatito? Puh, puh, ssssssss,” useless. The feline target couldn’t care less. Her neck doesn’t turn, her paws don’t pause. Finally, I use the cheat code: “pspspsps!” The cat’s eyes dart to me, ears twitching, absolutely laser focused on my face. Sitting in a park once, I watched a cat weave around the legs of a bench. It settled in the sun for a while, then walked off behind the bench and I lost sight of it. Wondering if it would reappear, I said the magic words, and ta-da! The tiny head popped up from behind the bench, eyeing me like prey.

Lund University in Sweden has done a lot of research into “Cat-human communication,” and find a pretty convincing cause for the phenomenon—cats are much more sensitive to high frequency noises than humans, an evolutionary quirk that makes sense when you consider the rodents they hunt. Hissing–esque syllables like “sw” and “ts” and of course “ps” contain these high frequencies. This type of cat calling can be seen across languages: “ksksks” in Russian, “pisipisipis” in Turkish, and “tstststs” in Hungarian.

In fact, this is built into a common English word for cat, “pussy.” The origin of the word is closely tied pspsps effect. “Pussy” branched from the word “puss,” a very common name for pet cats since at least 1520, and probably even earlier. Many a pet owner has called their pet again and again to no avail. At some point, people realized it works much better if saying your cat’s name makes them think you sound like lunch.

Unfortunately, all of this science obscures the most important mechanism behind the “pspsps” phenomenon, which is of course magic. Perhaps not fantasy magic, but at least a sort of free lunch magic that I am not accustomed to using in the real world. It reminds me of video games like Zelda, where just learning the words to a spell or the notes to a song grants the player a new ability. Saying “pspsps” sparks that same sense of excitement. How often does “one simple trick” actually improve your life, at no cost? At least once, clearly. Let’s search for more.

II. The Cost of Motivation

Manifold is a website where users can create and bet on markets about almost anything: who will be president in 2025, when AGI will end the world, or whether they will meet their wife that month. Other users bet their fake money on the outcomes, and the markets adjust to reflect the crowd’s view of the probability of the event. These crowds live up to their vaunted wisdom; in the chart below, we can see that the markets are pretty well calibrated. Manifold users tend to over rate the likelihood of things happening, but only by a few percentage points. If a market says that a given event has an 80% chance, it happens 74% of the time.

This is great! Experts suck at predicting events, and I don’t want to hire a team of super-forecasters to tell me how likely it is that I’ll fall down the stairs this month. Plus, Manifold users bet fake money called “Mana,” which makes it legal in the United States. The markets are zero sum between users in terms of Mana, and they all seem to have fun while they make and lose fake currency betting. Then, their combined play allows Manifold to spit out accurate predictions which would otherwise be hard to come by about a whole array of events. Smells like more free lunch to me.

A few Manifold users have found a totally different way to get something out of the website. They create motivation markets.

While markets aggregate information, they also influence behavior. If there is a 50/50 market open with infinite liquidity on whether Ben Pomeranz will convert to Christianity this year, I could buy two billion yes shares and make a billion dollars by converting to Christianity. I wouldn’t, but I could. This becomes an issue with markets like “Will Mr. Smith die this week?” which is why they are banned. Motivation markets take this side effect and utilize it as a tool.

Here is how you can make your very own motivation market. First, decide upon a clearly definable thing you want yourself to do, like read 3 books in July of 2024. Then, create a market with the following conditions: if you don’t read the books, the market resolves to No. If you do, the market resolves to Yes. Then, you can literally bet on yourself, and the haters might bet against you, and that motivates you to succeed. You can show the haters and keep your money just by reading 3 books. The Brothers Karamazov is already halfway finished.

You, the motivation market maker (mmm!), want there to be lots of people betting against you, to get that real chip on your shoulder. In a regular market though, there isn’t much reason to expect people would bet against you once the probability is sufficiently low. One solution to this is to keep pouring more money into yes, driving up the probability of the market and making it worthwhile for people to bet against you. You promise that you’ll keep your reading motivation market at 95%, betting yes over and over, no matter how many naysayers bet that there is a >5% change that a book will remain unread. If you care about Mana, this should create a lot of incentive to read the books, because otherwise you will lose lots of it.

Another option is to promise that even if you succeed, you won’t take anyone’s money–everyone will just get their money back. That way, you can just bet a set amount of Mana on yes at the beginning, and there is no incentive whatsoever for anybody else to bet Yes. Lots of people will bet against you at no risk to them, so you can endanger exactly as much of your mana as you want to and still have a crowd betting against you. While neither of these setups give good information about whether you’ll read the books1, they make the stakes higher for you, both emotionally and play-financially.

How well does this work in practice? Well, of the 12 resolved motivation markets on Manifold, there were 9 where the maker succeeded in their goal and 3 where they failed. 75% is not bad, but the aspirations haven’t been very high. One guy kept his workout schedule for two consecutive weeks (each week was a successful market), and another kept the plant “Lil Basil” alive through April. In my view, the most impressive market maker was Mich, who managed to keep their total screen time on social media (including Youtube) under one hour a day for a month. I don’t know what it was before they opened the market, but I assume that was a big improvement.

This is a good system. It has public shaming and financial incentive. It is zero sum play-financially, but positive sum in terms of utility since it also gives the market maker the benefit of extra motivation. Why are there only twelve of these things on Manifold, which is the most popular site I know of for users to make their own markets?

One issue is that it is rare for the average person to be invested in play markets at all. To be motivated by Manifold, a person must either care about Mana or their reputation in the Manifold community, and there are only about 1200 weekly users on the site right now. An ideal motivation strategy would be viable for more than .0004% of America. Of course, there are plenty of people who would get into Manifold if they knew it existed, but there are many more who just would not be interested. What’s worse, even if you are into Manifold the public shame effect only really works if your friends use it too. Many accounts are anonymous, and I care a lot less about revealing myself as a failure to strangers on the internet than I do to my friends. Of course, network effects mean that if you are on Manifold, your friends are more likely to be on it too, but in a group of 1200 this can only go so far. Without the social aspect, motivation markets rely entirely on people caring about their play money.

If people could make their markets with real money, they’d definitely be invested (no pun intended), but that is illegal in the US. While I have hope for the lawyers and crypto bros who are working to make this option commonplace, right now it is a fence most people don’t jump.

And even if people had the option, most people don’t like putting their money where their mouth is—it is frightening to know your cold hard cash disappears if you aren’t the person you say you want to be. Beeminder already exists to let people do this. You create a plan, and they take your money only when you stray from it. It is very effective as a motivator, but it isn’t so popular. Beeminder themselves guess that only about 1% of the population are the types to use it. Weird things feel weird, and self imposed financial penalties are too weird for most people to touch. It doesn’t help that an application like Beeminder appears negative sum to the user: when they mess up, their money floats off into oblivion (Beeminder’s pocket).

Buying motivation is a double edged sword: the more you buy, the more likely it is to be free (more incentive = better odds of success = don’t pay a dime), but the more expensive it is if you end up paying. My guess is that most people don’t think about this in terms of the probabilities, and can’t get over the idea of being charged money, real money, for nothing. Of course, motivation isn’t nothing, but in the situation where people are charged they didn’t get their motivation anyhow. The more people have to bet, the worse the cost looks (even though it is a cost they ideally won’t have to pay), and the less likely they are to bet at all. Now the metaphors are getting mixed: there is no free lunch, unless you risk it for the biscuit.

III. Pact Lunch

Let’s try once more to find a free motivation lunch, one that hopefully isn’t too sour for the risk averse. We saw that motivation markets as they stand usually end up without a personal or social element, leaving only a financial incentive. The market maker gives the naysayers money in exchange for the utility of being more motivated; we’ll measure this utility in “Motivs.” Clearly, these Motivs have value, because the market maker is paying for them, but almost everyone struggles mentally and emotionally to convert them to and from USD. The people who are happy to calculate how much their motivation is worth go do it on Beeminder, but most find that hard. It would be great if we could switch out the Dollars or Mana with some kind of currency that people find more natural to exchange for Motivs.

Well, assume people are willing to trade motivation for motivation. What if everybody instead pays each other with Motivs? I made up a meaningless unit of motivation for no good reason and won’t let it go unused. Hell, let everyone trade Motivs with each other, just for fun. Now, the whole situation is perfectly symmetric: every person is simultaneously giving and receiving Motivs to everyone else. There is no notion of the market maker, it is just a big Motiv hub, where everyone is motivating everybody else. This is starting to smell like something we’d call emotional support, or a running club, or a pact.

Surely there is no free lunch, you say! Where is everyone getting these Motivs to give each other, now that they aren’t paying for them? If everyone is giving and receiving in equal measure, our total motivation doesn’t change. That would be perfectly accurate, if not for the fact that people care about their social standing.

Two things that I value are for my friends to respect me and also for them to have good lives ^{[source required]}. Everyone already has plenty of Motivs stored somewhere, set aside to pursue the values they really care about. By positioning some distinct task as a bridge I must cross to be Good Friend, I funnel more Motivs towards that goal. So, if task A is my goal, and task B is my friend’s goal, and we both agree, promise, swear that we will do our respective tasks by Friday, we are relying on each other and know we can help each other by doing our task. My Motivs go towards his task, and his towards mine. This means that if I fail he loses Motivs, and I’ve let him down. Suddenly, my task is a bridge on the way to helping my friend and having him respect me, and this is where I get the extra Motivs. I face the same risk that I did when I used Beeminder, except now the punishment is disappointing my friend. I don’t want to pay this social and emotional price, because I value his wellbeing and our friendship, and this creates the motivation I need to do task A.2

This lunch still carries the risk of a potential cost. However, people are much more open to exchanging social and emotional currency for motivation. People make promises all the time, and it works. While people don’t always tell the truth when they make a promise, they certainly do significantly more often. A priori, I would expect that pacts work even better than promises given their symmetry and inherently actionable nature.

After completing a few successful pacts with friends, I began to wonder why this isn’t an extremely common process. Pacts makes people better at doing difficult things. Many of mankind’s great achievements were difficult. Why don’t we see pact–like agreements in the lives of historical greats? Why didn’t Mozart and Haydn agree to make a Sonata a week? Why don’t LeBron and Steph Curry promise each other that they’ll shoot 400 threes a day? Are these giants of their fields wasting free motivs, throwing out their free lunch? Should they be even better than they are?

I think not, and the reason is kind of obvious. If LeBron thought he could improve his game by spending more time every days shooting 3s, he would. The greats don’t make pacts because they don’t have to. Some people fit neatly into the Szasz/Caplan model of preferences: their stated preferences are perfectly reflected in the actions they take, at least when it comes to what they are good at. Bron trains as hard as he can, exactly as much as he wishes of himself, given the amount of time he has. This completely devalues the motiv; he is saturated with motivation, at least with regards to getting better at basketball, and has no need for more. Even us less motivated folk can relate to this in specific domains—it would be pointless to create a pact to eat or sleep. To some impressive people, what they do is like eating.

Unfortunately, I am not one of these greats. At different times in my life I said that I wanted to avoid getting high, wanted to stretch more, wanted to spend less time on social media. I didn’t do these things, and if you asked me why I wouldn’t have been able to tell you. Then I started making pacts, and all these goals started feeling easy. I’d found the words to a new spell, an (almost) free lunch. Pacts work. If you and a friend have something that you are always meaning to do, but never doing, I highly recommend a pact. Set some ground rules and don’t let each other down. Any given one probably won’t rock your world, but they add up. Pacts have noticeably improved my life. I read a little more, my palms can touch the ground, and I found friends in college without turning to the devil’s lettuce.

Speaking of friends, I made some brand new ones this past weekend at Manifest. Four of us actually agreed to a pact of our own; we each had until the end of June to write and and publish a blog post. I said that I wanted to write this one back in April, but I never did. Luckily, I was only a few Motivs short.