It’d be difficult for a 20-year-old to explain to their great-grandparents how a bikini-clad OnlyFans creator with metastatic spinal cancer can earn $25,000 live-reading a textbook on diesel engine mechanics. Similarly, when Steve Jobs stood on stage and introduced the iPhone in 2007, no one in the audience thought, “well, there’s the end of the taxi industry as we know it!” Yet, both were true, and impossible to predict far in advance.

We’re in the same position today, trying to guess how AI is going to shape the future of work. If history is any guide, a large share of the jobs people will do 50 years from now don’t exist today. It doesn’t make much sense to try to work in a “future-proof sector” or find an “AI-safe job”. It’s much more useful to think about what skills are likely to be valuable as AIs continue to improve.

Models of Models

Below, I’ll suggest things like “learn how to solve problems with AI”; “build relationships”; “anticipate latent demand”; “do the cognitive equivalent of lifting heavy”.

An important pre-requisite to all of this is having a mental model of the kind of things that AIs are, and the kinds of things they’re good and bad at. I’ve written about these mental models here and I suggest some other recent essays at the end of this one. But theories of character writing and analogies with bicycles will only get you so far.

I advocate ubiquitous silent testing: Use AIs when you already know the answer (or will soon come to know the answer), then compare AIs to humans.

For example, before I have any kind of appointment (doctor, physio, accountant, etc.), I’ll ask some combination of Claude, ChatGPT, and Gemini all the questions I’m going to ask at my appointment. I won’t act on the answers (that’s the “silent” part), but will instead compare what the AI said to what my doctor, physio, or accountant ends up saying.

If you do this kind of thing a lot, across many contexts, you develop an intuition for the kinds of things the AIs are good and bad at, and when they can and can’t be trusted. It’s how you learn where your comparative advantage is.

From here, you’re well on your way to what Benjamin Todd, in his excellent essay “How not to lose your job to AI” highlights as his single most important piece of practical advice: learn to deploy AI to solve real problems.

AIs struggle with messy, real-world jobs. When AI companies talk about AI capabilities improving rapidly, they are referring to scores on benchmarks. But these scores don’t always translate into real-world productivity. The AI as Normal Technology argument has held up very well in this regard:

“Like other general-purpose technologies, the impact of AI is materialized not when methods and capabilities improve, but when those improvements are translated into applications and are diffused through productive sectors of the economy.”

What is less appreciated is that the bottlenecks to AI diffusion (e.g., integration costs, workflow redesign, organizational politics, regulation, etc.) are career opportunities. Almost by definition, skills related to managing these bottlenecks are in high demand. If you’re socially skilful, motivated, and have a good mental model of AI, “organizational inertia is a gift.”

Strengthen your social bundle

Nobel Prize winner Geoffrey Hinton (in)famously claimed in 2016 that we should stop training radiologists because it was “completely obvious that deep learning is going to do better.” While he was correct about the task of reading scans, he was wrong about the rest of the bundle. Radiologists only spend about a third of their time reading scans. They spend more time on strong-bundle, social tasks like teaching, consulting, or talking with patients and nurses.

Indeed, you can run this analysis on the university itself. The curriculum (e.g., lectures, textbooks, problem sets) is weakly bundled and LLMs make them free on the margin. What’s scarce, and hence, valuable, in this context? The relationships and social interactions that aren’t written down. The strong bundle is the people.

But for my money, the most valuable lesson from applying the task bundle concept is that it doesn’t really work. Not because tasks are too numerous or complex, but because most of what we do isn’t written down.

For most jobs, if you try to list out the strongly and weakly bundled tasks, you come to appreciate just how much of the job isn’t actually explicitly describable in terms of a task bundle structure. This is why work-to-rule is an effective strike tactic. When workers do only what their job descriptions explicitly require, the workplace grinds to a halt.

I started my current job during the pandemic, when every professional interaction was a scheduled zoom call. On paper, my onboarding worked: I taught courses, advised students, and sat on committees. But in practice, I was in the dark about much of my department’s history and culture, its politics and factions, and which committees actually decide things.

I lacked the tacit knowledge about how things get done around here. As the university re-opened, I slowly accumulated this knowledge through in-person social interactions that were, in many ways, the opposite of zoom meetings: unscheduled and informal chats before seminars, beers after meetings, and gossip.

For now, LLMs live in the on-paper world of job descriptions and meeting agendas. Their “back channel” tacit social knowledge is poor and likely to remain so. Therefore, Build Relationships. Develop local, un-written, last mile knowledge in your social network. “Play in terrains where there is no training data.“ Choose rooms over screens.

Where to go?

It’s easier to guess why a field will grow than it is to guess which fields will grow. Here are my three guesses.

Go where cheapness expands the market

One of the most important kinds of tacit knowledge will be about latent demand. It is well-known that every online interaction leaves a data trail. But latent demand - the things people would buy if they were cheaper - is (mostly) invisible to algorithmic systems trained on the recorded world of clickthroughs.

In some ways, students are more likely than their parents and teachers to be sensitive to local, embedded knowledge about the kinds of things people will want, and the new things people will do, when AI breaks down barriers to entry. Another comparative advantage.

Benjamin Todd has a nice example: If AI makes filing tax returns 2x cheaper, you still only file once per year. But when taxi services are made cheaper (this is what iPhone + Uber did), people spend more money on taxi services than they did before. The question to ask is: will AI’s impact on your field look more like taxes or taxis?

Go relational

In Alex Imas’s viral “What will be scarce?” essay, he argues:

“The same economic forces that moved 40% of the American workforce off farms and into factories and offices will move workers out of automatable commodity production and into what I’ll call the relational sector. By this I mean the human-intensive, provenance-rich, sometimes artisanal part of the economy where the human aspect is part of the value of the good or service itself. The economics of scarcity won’t disappear, it’ll just relocate.”

Imas shows that as incomes rise, households tend to spend a larger share on things like in-person dining, entertainment, and education. Richer people don’t buy more stuff so much as they buy more social experiences.

Here’s another directional clue. Go where the human element matters. Imas has some suggestive experiments where merely learning that AI was involved in making an artwork halved the premium people paid for exclusivity. AI-generated stuff feels easily and infinitely copyable. As such, it doesn’t have a story about where it came from, and so it can’t be a differentiator or status signal.

Of course, this story depends on transformative AI significantly reducing the cost of producing a wide range of goods and services. And it also has more than a whiff of a growing service class waiting on a wealthier one. Still, there is something actionable here which ties in with much of the advice above. Imas advises: “be the person whose involvement makes the product feel like it was made for someone, by someone.“

Be Accountable

Even under rapid AI progress scenarios where AIs quickly automate many sectors of the economy, most writers recognize there will be demand for people with standing. That is, people who can be blamed, fired, sued, praised, and promoted.

Recall the accountants from Part 1. The arithmetic and bookkeeping was weakly bundled, and the strong bundle was signing audits and carrying legal exposure. AIs have no such standing, and won’t for the foreseeable future. It took humans decades and sometimes centuries to build the kinds of ethical, legal, social, financial, and political institutions that produced this standing. And lots of unsexy, invisible labour goes into maintaining them.

My guess is that many places where a signature is meaningful (e.g., government, medicine, law, engineering, etc.) will still demand humans, long after AIs are capable of automating the underlying work. We will, however, have difficult questions to answer about the dignity of humans as rubber stampers or liability sponges for AIs.

Another question, lurking since the end of Part 1, is about the path to these positions of accountability. Here’s Todd again:

“As AI increases the value of leadership skills, it’s decreasing the value of the entry-level jobs that previously served as a training path to them.”

His practical answers are to prefer small/growing organizations over big firms for entry level jobs (because specialization makes big-firm junior roles narrower and more routine); to take on lots of side projects as leadership practice; and most importantly, to find mentorship deliberately.

I’d add that, if you take the “learn how to solve messy real-world problems with AI” advice seriously, then younger, AI-native workers (Tyler Cowen calls them the “AI Maniacs”) will be well-positioned to leapfrog the compressed entry-level into management roles (whether managing AIs or humans).

Lift heavy

The most important skill of all is meta-learning, or learning how to learn. Even if AI progress fizzles, but especially if it doesn’t, being able to quickly learn new skills is extremely valuable. AI is a blessing and a curse for this. LLMs make it very easy to learn new skills quickly, but also very easy to trick yourself into thinking you’re learning when you’re not.

The cardinal virtue of the AI era is therefore honesty. Only you can tell whether you’re putting in the work. Jasmine Sun is exactly right when she says that real work, real learning has a feeling: “there’s a cognitive equivalent to lifting heavy. Do it three or four times a week.”

I like lifting heavy (and the weightlifting analogy more generally) because barbells provide unfakeable feedback. You make the lift or you don’t.

A few months ago, I snatched my bodyweight (165 lbs). A few weeks ago, I hit a 305lb x 3 back squat followed by two 48” pvc-jump-overs. I felt prouder of these than any journal article or grant. It’s a nice reminder, as Luis Garicano says, that “the ability to derive meaning from sources other than your work is itself a form of human capital.” There’s so much uncertainty about AI and the future of work. The ideas here are hedges against that uncertainty. Perhaps the most important of all is developing a sense of self bigger than the job.

Suggested Readings

On Mental Models of AI

Guive Assadi, “Alignment Fine-Tuning is Character Writing”

Kai Williams, “The Many Masks LLMs Wear”

Rohit Krishnan, “Homo Agenticus”

On Comparative Advantage

David Oks, “Why I’m Not Worried About AI Job Loss”

Noah Smith, “Plentiful, High-Paying Jobs in the Age of AI”

Séb Krier, “The Cyborg Era: What AI Means for Jobs”

Responses to Alex Imas’s “What Will Be Scarce?”

Luis Garicano, “The Task is Not the Job”

David Deming, “Efficient Social Learning is the Human Advantage”

Molly Kinder, “The Messy Middle”

Career Advice

Luis Garicano, “A New Year’s Letter to a Young Person”

Benjamin Todd, “How Not to Lose Your Job to AI”

Jasmine Sun, “The Old World is Dying”

The simple fact of the matter is that no one has the answer to this essay’s eponymous question. Six out of ten jobs that people do today didn’t exist in 1940. The best evidence we had in 2013 suggested a 94% chance accountants could be automated, but there are more accountants now than then, and the occupation is projected to grow faster than average through 2034.

To see why it’s so hard to predict the future of work, consider: A 19th century draft horse, pulling carts or streetcars, could shit more than 50 pounds per day.

With hundreds of thousands of horses working the streets in cities like London and New York, the sheer volume led to the “Great Horse Manure Crisis of 1894.” According to Stephen Davies, “In the Times of London in 1894, one writer estimated that in 50 years every street in London would be buried under nine feet of manure.”

One moral of the story is that this “problem” ultimately solved itself because cars replaced horses. And for a while, the “Great Horse Manure Crisis of 1894” took on a life of its own as a techno-optimist meme: Don’t naively extrapolate trends because tomorrow’s innovations often dissolve today’s crises.

There’s other stories like this. Despite their name, Automated Teller Machines did not automate away the jobs of bank tellers, and the number of tellers actually increased. And for a while, the ATM story took on a life of its own as a pro-worker economics meme.

These are good stories. But the ways in which they are false are instructive for thinking about whether AI is coming for your job. The horseshit story is, well, horseshit. The Times never published an article about 9 feet of manure. It was all fake, but the vibes were good and the story spread.

And it’s true that the number of bank tellers increased for a few decades after the invention of the ATM, so it’s technically true that ATMs didn’t replace tellers. But the iPhone-enabled era of mobile banking did, and now there are far fewer bank tellers. Here, the premise was true up until about 2005, and despite the nice vibes, it’s long since expired.

These kinds of anecdotes, whether optimistic or pessimistic, tend to reach their maximum cultural relevance at the moment their epistemic warrant is weakest. We don’t yet know what tomorrow’s anecdotes about AI and jobs will look like, but by the time they’re circulating around dinner tables, we should be prepared with better conceptual tools. This essay introduces five such tools: bundle strength, comparative advantage, so-so automation, demand elasticity, and pyramid compression. Part 2 of the essay puts them to work to offer concrete, practical career guidance.

How strong is your bundle?

This essay’s title is misleading. AI almost never takes jobs. AI takes tasks from within task-bundles from within jobs. That 2013 study was scoring tasks that accountants do, like reconciling ledgers and filling in returns. And many of those could indeed be automated. But what it missed was that jobs are bundles of tasks and bundles are not automated just because their parts are.

distinguishes between weak and strong bundles. Weak bundles of tasks come apart cheaply. Consider travel agents. Searching, comparing, and booking flights used to be separate steps. Once software like Kayak could do them, this part of the travel agent’s work was unbundled. Many “bookkeeping” accountancy tasks are also bundled weakly, which is why there are fewer clerk jobs now.

Tasks like interpreting tax code, developing business strategy, signing audits, and carrying legal exposure are more strongly bundled, and these seem to explain why accounting is projected to grow. Once travel agents handed over rote booking tasks to software, the ones that remained were reinstated upmarket into curation and relationship management. Their wage growth has outpaced the private sector average since 2000, even as their numbers were halved.

It’s unhelpful to think about sectors or jobs being more or less exposed to AI. Better to ask about the task bundle strength within jobs. For many jobs, weak-bundle tasks will leave while strong-bundle tasks stick around, and even appreciate by generating new downstream tasks.

This is why the concept of comparative advantage is so important today. As long as human workers improve production somewhere - usually it will be at those strong-bundle bottlenecks - then it will often (but not always) pay to keep them around. In order for capital to fully substitute for labour, the best AI has to be better than the best-human-using-the-best-AI. That bar is much higher than AI beating the median human, and it seems likely there are a lot of jobs to do in the gap between those two.

What if a lot of automation looks like self-checkout machines?

The introduction of ATMs in the late 1960s cut the costs of running a bank branch. Banks could then reinvest those savings opening more branches. So while the number of tellers per branch decreased, the overall number of branches increased enough that aggregate teller employment also increased for decades. And like travel agents, tellers were reinstated upmarket to “relationship banking” tasks.

Self-checkout kiosks seem to automate similar customer-facing tasks, but with mostly negative results. Checkout is no faster and theft is higher. The machines don’t automate checkout so much as shift the scanning labour onto customers.

This is what economists call “so-so” automation. Good enough to (maybe) decrease the wage bill, but not good enough to significantly raise productivity or lower prices. Comparative advantage paints an optimistic picture when automation can lower prices or trigger ‘focus effects’ for strong-bundle tasks. But when automation is merely cheap, there’s no productivity gain, nothing higher-value to focus on, and nowhere upmarket to go.

Instead we get glitchy, annoying, low-quality human-replacements. A student who worked at a convenience store supervising self-checkout kiosks told me about hearing the robotic voice: “please use keypad to complete transaction” on repeat as she tried to sleep at night. Torture.

One near-term, and I think under-discussed worry about AI and jobs, is less about superintelligent AI causing mass unemployment, and more about mediocre, enshittified AI adopted at-scale anyway.

Is demand elastic?

At today’s prices, it’s not worth it for me to sue a shady landlord who pocketed my security deposit. Nor does it make sense for me to pay someone to restructure my (meagre) consulting income for tax advantages. In both cases, the fees swamp the gains. But what if legal work were 10x cheaper? Then, I would probably do both.

This is what economists call the price elasticity of demand: when the price of something falls, how much more of it do we buy? Consider food. Over the past century, farming became much more efficient and food is much cheaper. But we didn’t respond by eating more breakfasts. Demand for calories is relatively inelastic, so the productivity gains were translated into fewer farmers. Famously, in 1900, 40% of Americans worked on farms; today it’s less than 2%.

Contrast this with software. Productivity gains, especially those delivered by today’s coding agents like Claude Code or Codex, make programmers’ time go further. And the demand for software has only increased, which is why software engineering may still be a growing profession. Unlike food, demand for software is highly elastic.

Put another way, there was a lot of pent-up, latent demand for things like software, just waiting for prices to drop. And once they did, all sorts of new applications were unlocked. This is the much-discussed Jevons Paradox. It’s easy to imagine more medical imaging or diagnostic work, if it were cheaper and faster. Ditto for financial services and language translation. My landlord lawsuit and tax restructuring cases are examples of latent demand for legal services. If AI makes legal work cheaper and more efficient, the Jevons story predicts more legal work and possibly more lawyers.

Will the pyramid compress?

“Possibly” is, of course, doing a lot of work in that last sentence. The Jevons story, according to is just that: a story. “The prudent posture,” she writes, “is to treat it as one possibility among several rather than a get-out-jail-free-card.”

At big law firms, the base of the pyramid is junior associates doing many of the tasks that today’s LLMs excel at: research, summary, document review, drafting, etc. The level above them is middle-managers, with partners and executives at the top.

10x cheaper legal work might well expand the market, but one possible equilibrium is a compressed pyramid: a firm with “the same number of partners, much higher per-partner profits, and a fraction of the junior and mid-level workforce,” where the junior staff, “who would have been the next generation of partners, who would have spent ten years grinding through the associate ranks before making partner, never get hired in the first place.”

In the same way so-so automation tempers optimism about comparative advantage, pyramid compression looks like the condition under which elastic demand may not be so reassuring after all.

In a follow-up essay, Kinder notes that we don’t even need a hypothetical law firm to see this. Who knew that “office and administrative support” is the largest occupational group in the American labour force? There are nearly twice as many secretaries as software engineers. Most are women, and most don’t have a college degree. Their jobs are largely comprised of the weak-bundle tasks that AI eats first.

We are in the midst, Kinder argues, of an invisible disruption: AI could do to high-school-educated women what deindustrialization did to high-school-educated men. Except in this case, there aren’t any shuttered factories for local TV news B-roll.

What next?

When intelligence is abundant, what remains scarce? This essay introduced the concepts of bundle strength, comparative advantage, so-so automation, demand elasticity, and pyramid compression as tools that can be used to think more carefully about AI and jobs. Of course, none of these concepts tells you what to do next. That’s Part 2.

About three months ago, I became convinced that LLM capabilities were good enough that I wanted to experiment with rebuilding my reading and writing around them. This essay shows what that looks like. My goal is less to persuade than describe.

Downsides upfront

This set-up took time and effort that could have been spent reading and writing. It also requires ongoing vigilance and maintenance. It requires a number of paid subscriptions. There are non-trivial privacy risks. I often worry about a combination of sunk costs (“I’ve already invested so much time and money, I should keep going”) and imperceptibly slow cognitive erosion.

I can relate to ’s observation that LLMs make preliminary question exploration easy, but then “you spend less time grinding through arguments from first principles, a grinding that builds fluency that shows up in live exchange.”

I’ve also noticed the tools shaping when and how I work: I’m frequently handing off tasks to Claude right before going home, then checking them later, in a way that blurs my home/work boundary. I read a lot of LLM prose and would be surprised if that didn’t have some negative downstream influence on my writing and thinking. And of course there’s the downsides I don’t even notice.

These are real worries and I take them seriously. The “how I use AI” genre is also full of breathless examples of people using Claude Code and Codex to build the digital equivalent of workshops, but is much quieter about the digital equivalent of usable, durable furniture.

The setup

At the highest level of abstraction, I implement Andrej Karpathy’s LLM Wiki pattern in Claude Cowork, pointed at an Obsidian Vault that is fed by everything I read and annotate in Readwise. If that already makes sense, you can skip to the next section. If not, I’ll explain each part.

Devices
I’m migraine-prone, so I read on e-ink: Onyx Boox Note Air 5C, which runs Android, so Readwise, Obsidian, and Claude are all installed.

Apps
The foundation is Obsidian, which is a note-taking system that stores everything as plain-text markdown files (.md) in a local folder called a Vault. Markdown is highly LLM-legible and very token-efficient. My guiding principle is to get as much writing-, reading-, and teaching-relevant material into the vault as possible. I pay for Obsidian sync ($5/month) so I can access and edit the Vault from all my devices.

Everything I judge worthy, I read in Readwise Reader ($10/month). I configured the Readwise Obsidian plug-in, so that everything I read gets sent to my vault (this is the only part that required writing code, which Claude handled easily). One “clean” copy of a text gets routed to a “Full Document Sources” folder (in my experience, hallucinations are almost non-existent when working with full texts in-context). A second copy with my annotations gets routed to a triage folder where I then move it to the relevant project.

Why go through the hassle of duplicating everything I read and write?

So that Claude can read it and write about it, too. This is where Cowork comes in, which lets Claude read, write, and edit files in the vault (and on my computer more generally). It uses the same chat interface, and doesn’t require any familiarity with the terminal. When combined with Dispatch, I can, from anywhere, tell Claude what to do on my computer. To mitigate worries about cognitive erosion, I generally try to take stuff I’ve already done, and then see where AI can push it further, rather than starting with AI.

LLM Wiki

In April 2026, Andrej Karpathy described a pattern he called the LLM Wiki. One motivation was that LLMs start from scratch each time they are pointed at a collection of documents. Even with memory features, very little knowledge accumulates.

Karpathy’s insight, drawing from Vannevar Bush’s (1945) vision of the Memex, was to have the LLM compile rather than retrieve. When a new document arrives, the LLM integrates it into a Wiki: A persistent, interlinked set of Obsidian pages. Ingested once, updated frequently.

At its core, the LLM Wiki pattern is just a plain text file (CLAUDE.md) that Claude reads at the start of every session and treats as binding. Karpathy describes it as “what makes the LLM a disciplined wiki maintainer rather than a generic chatbot.”

I would argue that it’s an exercise in applied epistemology and philosophy of science. It’s my answer to the question: “what should an eager, tireless, suggestible, unimaginably well-read but sycophantic research assistant do for me?”

CLAUDE.md is a work-in-progress. I started with a handful of papers I knew well, ran them through the pattern, and iterated in light of the outputs. I have Wiki pages for all the topics in this essay series and many more besides, comprised of a couple hundred sources and growing. Here are two structural features that have emerged.

First, in light of the downsides I opened with, a provenance wall. I partition the vault into:

• RAW (source authors’ full text; for Claude, this is read-only and immutable)

• GUS (my notes, annotations, and drafts; also read-only, immutable).

• WIKI (Claude’s, freely regenerated).

My binding rule is that it must be recoverable whether a given sentence was written by a source author, by me, or by Claude. Every wiki page carries frontmatter listing its sources, generation date, and a prominent call-out box for Claude-generated text, summary, synthesis, etc.

Second, an epistemic target. Many generic implementations of LLM Wiki generate lots of interlinked summaries. I don’t want mere summaries (I’ve already read the papers!) Nor do I want forced, tidy synthesis as LLMs are wont to provide.

Instead, I ask for adversarial collaboration as my ideal: suggesting the strongest version of an argument along with strong objections. I explicitly instruct Claude to give me something to write against. The wiki pages look less like Wikipedia and more like staged debates.

I’m still wrapping my head around these growing text files on my computer. What are they, exactly? It’s a digital garden, of sorts. It also feels like an on-demand tertiary literature over my personal corpus. But I can argue with it, and it updates in light of those arguments. It seems to get better the more I use it. It is definitely more of a librarian than a library.

Ok cool, what can you do with it?

So much for the metaphysics of the workshop. Where’s the furniture?

Start downhill

I have tenure, a lot of graduate students, and a young kid at home. I probably have ~same total amount of free time to read and write as I did pre-tenure and pre-kid. But it almost never comes in 4-hour uninterrupted chunks like it used to. Now, it’s spread out over various 30-minute segments throughout the day, interspersed with drop-offs, pick-ups, driving, exercise, and many different kinds of meetings.

Lately, I seem to have my best ideas right as I’m leaving my office for said pick-up. Previously, I would use voice input to record the idea to a Trello card. Then it would sit there as an idle fragment. But with Dispatch + Cowork + Obsidian, I can now keep walking and say things like “Find the relevant sections of the papers I mentioned, add them to the notes for Chapter 3, and update the references accordingly.”

Then, the next time I sit down at the keyboard to write, I can “start on a downhill” and make the most of the 30 minutes I have.

Ask distant questions

Even within sub-fields of philosophy, what one person can read is a rounding error against what gets published. It’s tempting to have LLMs read what I don’t have time for, like Seth Lazar’s MINTY. While it’s better than nothing, I don’t find much durable value here.

My LLM wiki contains the sources I have read closely enough to annotate. But even at this scale, there are corpus-level questions that were difficult to answer pre-LLMs.

For example: “given all the sources for my AI and jobs essay, what economic perspectives are under-represented?”; “In my AI and bias essay, do I fairly represent how the sources propose constructive solutions?”; “Did I overlook any themes in my summaries of the recent anti-AI sentiment research?”

For anything I write, it’s now ~free to ask, “What’d I miss?” or “What’s the strongest objection?” from across my entire curated corpus + LLM background knowledge.

This is akin to distant reading, which is a complement to, and not a substitute for, close reading. It provides a way to see interconnections that are too big to fit in one head. The reading and annotating are the same as always. But now it’s easier than ever to view them from a higher altitude. These aren’t necessarily better questions to ask, they’re just different, second-order questions. In turn, they can help to ask different and better first-order ones.

Course Updates

Last year, I developed a large first-year course called “Digital Wisdom: How to Use AI Ethically and Responsibly.” This year, I dumped all the course materials into the vault. Claude already has a list of my “trusted sources” from things I’ve imported from Readwise. Now I can ask Claude to review those sources against the course materials, flag what’s outdated, find relevant news stories or case studies, and suggest other new updates for me to approve.

The relevant counterfactual: Maybe I can recall a few relevant news stories from last month. A few days before classes start, I’m clicking and skimming pages on the LMS, eyes glazing over, and turning a few “GPT-4”s into “GPT-5”s. I quickly get bored and mostly leave the class as it is.

Instead, with Obsidian + Cowork, I got a list of 25 targeted updates in the time it would have taken me to microwave my lunch, log-in, and locate the course material.

Conference Poster

Earlier this year, I gave a talk at the APA in Chicago on how the philosophy of childhood can inform debates about AI moral status. The same paper was accepted as a poster a few months later at SPP in Baltimore. I dumped all the papers I read, my annotations, my notes, and my conference handout into the vault. Then, I told Claude Fable in Cowork to make me a poster. Ten minutes later, the poster was ~85% done. I still had to resize some text boxes, add a few images, and make a few aesthetic tweaks. Those took less than an hour.

The relevant counterfactual: As a philosopher, I’ve only given a handful of posters, so this would have taken me a full day, if not longer: one afternoon distilling the material to the most relevant ideas and quotations. Then another afternoon fussing around in Powerpoint, adding images, arranging and re-arranging, etc.

Network Building

Two years ago, I spearheaded a new Ethics of AI specialization at the University of Guelph. The goal is to open new, non-academic pathways for philosophy grad students. In addition to courses in ethics, political philosophy, philosophy of science, etc., we require students to take classes in stats and programming.

Instead of a traditional academic thesis, students can complete an “ethics audit” where they collaborate with AI researchers in government, industry, and academia to work on ethical issues in AI deployments.

We’ve done projects on AI in hiring, brain-computer interfaces, Canada’s Medical Assistance in Dying, and more. It’s a great program, but it’s bottlenecked on developing a network of partners to work with our students. To date, it’s mostly been my research network.

Not to sound like a broken record. But I gathered all the e-mails I’d written to potential partners, summaries of past and ongoing ethics audit projects, department and program requirements, and dumped them into Obsidian.

Now, I can easily say, “I’m reaching out about a potential partnership on AI and prenatal screening. Take the materials from our AI and proteomics project, update them accordingly, and draft my intro e-mail.” Or “take the draft partnership agreement from Organization X and adapt it for Organization Y.”

I’ve also started adding programs from conferences/workshops/meetings I’ve attended. Now Claude can anchor on those to proactively search for partners for our program.

Of course, there’s no substitute for sitting down to coffee to build a new connection. That’s one of the best parts of the job and I’d never outsource it. But I have found that AI significantly reduces the “activation energy” to get started on this kind of outreach, which, in the relevant counterfactual, falls behind my research, advising, and teaching priorities.

Anything to Anything

Just like in Carlo Cordasco’s “illegible benefits” essay, I didn’t anticipate what, in hindsight, now seems like the biggest payoff of re-building around LLMs. Namely, that once something’s LLM-legible in Obsidian, it can be easily re-purposed across many modalities.

Once papers drafts or course materials are in markdown files, it’s trivially easy to output sub-sections as slides for lectures, talks, or posters; or to output html for a website; or prompts to audio, image, or video generators; or as code for interactive video demos.

Similarly, when the 2027 IEA Energy and AI report comes out, I’ll ask Claude to find everything in my research, writing, and teaching materials that cites the 2025 and 2026 reports, update them with newer figures, note the differences, and assess which scenarios were on-track.

Ditto for any other — dare I say — load-bearing sources. After I read ’s new book on AI rights, my notes will be passed through the AI Welfare wiki. When I get invited to talk to middle-schoolers about AI, I’ll custom generate a few slides from my undergrad AI Ethics course, updated with age-relevant examples, and maybe add a video game component. Then, as I’m walking back to the car, before I forget about it, I’ll have Claude update the “service” section of my CV. This set-up also affords “living research” which automatically updates as new data comes in.

More generally, our work contains multitudes. One verified essay is the basis for a conference abstract, flash-talk outline, set of slides, in-class discussion exercise, exam questions, referee-report skeleton, or a section in a grant. Or vice versa. Each is update-able and compile-able from the same versioned source using a few natural language prompts.

Pushed to the limit, the site of intellectual work is less in documents and more in ideas, arguments, and concepts.

Did I write this with AI?

So much has been written lately about AI and writing, it’s hard to know what else to say. I think sums it up pretty well:

I also liked ’s essay describing how our moral intuitions mislead with AI detectors. For what it’s worth, I feel the ick more when I use AI for brainstorming than writing. With brainstorming, it’s too easy to get anchored to the initial framing. Fluent AI prose naturally affords small, surface questions like “should this be two separate claims?” rather than deeper, critical questions like “does this claim even make sense in the first place?” With writing, Claude has all of my published essays, drafts, notes, annotations, chat history, the /editor skill I wrote, and the CLAUDE.md instructions in-context to work with. The conversations are like those I have with really smart, helpful people.

I suspect that much of the Discourse about AI and writing will fade into the background soon enough (if it hasn’t already). At the end of this essay series, I’ll return to this topic with more evidence about how it all went.

But in the meantime, did I write this with AI?

Yes and no.

Given the setup I just described, most things I write now are the outcomes of collaborations with Claude. But I rarely have Claude read or write for me because I love reading and writing. I try to free up as much time as possible for reading and writing. I want to do more of it, not less. I already spend a lot of time at the gym. I don’t know what else I’d do if I used AI to do my reading and writing.

I remember asking one of my graduate school mentors how he managed to write so much, so quickly. “Caffeine, alcohol, and rage,” he replied. When I feel really stuck in my writing, I will sometimes ask Claude — with all the context just mentioned — to draft candidate prose in different directions. Sometimes there’s a sentence or two I can’t beat (in this essay, the “rounding error” line). But LLM writing almost always sucks.

The worry is that, in writing for us, AIs will think for us. But in practice, the thing it does most reliably is generate grating prose that makes me angry enough to write better. Caffeine, alcohol, and indeed, rage.

Next up in the series: A two-part essay on “Will AI Take Your Job?”

So, today I’m announcing the Digital Wisdom essay series where I’ll be working through my views on the most important questions in AI, informed by a decade of research and teaching technology ethics.

I’ve argued before that the scarcest resource in AI today is a positive vision, and that ethicists need to re-focus on questions of “how should we live?” rather than more compliance frameworks and harm taxonomies. The series will reflect these commitments.

I’ll be writing these essays during my sabbatical, where I’ll be a Visiting Researcher at IVADO and McGill (reach out if you’ll be in Montreal!). The essays will be mostly self-contained, though I’m hoping there will be enough continuity between them that the end result will be a book.

First up, coming next week, is an essay on how I use AI. I was planning this for later in the series, but given the recent discussions about Substack x Pangram, I’ll lead with it. In short, I think Victor Kumar is right that “it’s a big loss that academics and intellectuals, afraid of being shamed, aren’t divulging constructive uses of AI in their writing process.”

So, I’ll begin by sharing how I use AI in my reading, writing, and teaching. I’ll discuss the downsides and risks, how I’m using AI to improve existing parts of my job, as well as do new stuff I couldn’t do before.

Next up will be a two-part essay, “Will AI take my job?”. Part 1 introduces five economic concepts (bundle strength, comparative advantage, so-so automation, demand elasticity, pyramid compression) that are helpful for going beyond anecdotes and thinking more carefully about AI’s impact on jobs. Part 2 then translates these concepts into practical career guidance.

After that will be my takes on, “Is AI bad for the environment?” and “Is AI biased?”

Stay tuned, share with anyone who might be interested, and let me know what else you’d like to see covered.

Like many of my colleagues, I scrapped take-home essays to protect academic integrity. But then we had to reckon with the fact that fourth-year philosophy students wanting to apply to graduate school didn’t have writing samples. They’d never written a term paper.

This is what a “solution” looks like when ethics ignores trade-offs.

In a previous essay, I argued that the field of AI ethics is one-sided, misleadingly negative, and myopically focused on harms. We’re in the midst of an AI Ethics Winter. Meanwhile, AI capabilities are advancing relentlessly. Nearly every AI policy at every level - from individual syllabi to university guidelines - evokes the ethical use of AI. What would an ethics look like that educators can actually use?

Good vs Good

At the beginning of each semester, I tell my undergraduate students that it’s quite natural to think ethics is about good vs evil. If that were true, things would be easy and I’d be out of a job. We’d just do the good things and avoid the evil things. But ethics is rarely about good vs evil. It’s almost always about good vs good. Ethics fundamentally involves trade-offs between different things that are good in different ways.

If AI was evil, and all it did was destroy the environment, produce biased outputs, and rot our brains, then things would be easy. No one would use it. Yet, one billion people use ChatGPT monthly. It’s possible this is planetary-scale false consciousness. But the much more likely explanation is that many people just find AI useful, warts and all.

This does not negate environmental impacts, bias concerns, or worries about cognitive surrender. It just shows that, like most things in life, AI is a mixed bag. It is extremely unlikely that a technology as complex and far-reaching as AI would fit neatly into an “All Good” or “All Evil” box.

The path out of the AI Ethics Winter towards policy-relevance, then, requires framing ethics less in terms of harms and more in terms of trade-offs. Everything worth talking about in AI involves trade-offs.

I can get Claude to wade through the deluge of AI papers published each week and give me summaries. Or I can download them to the black hole that is my “AI TO READ” folder. Summaries are surely better than nothing, but then again, isn’t the point of my job to read the papers?

What about using AI as an editor for writing? It’s helpful, cheaper, and more readily available than the alternatives. But people who spend a lot of time on Substack can already glimpse the homogenization resulting from Claude providing editorial direction to most of their feed.

The list goes on. What should be clear is that there’s no “solutions” to the problems posed by AI in education. It’s trade-offs all the way down. We owe it to our students to honestly and explicitly name the trade-offs involved with using and not using AI, and to model careful reasoning about them. We abdicate our responsibility as educators when we substitute this with simplistic, tribalistic, good vs evil sloganeering.

But if it really is trade-offs all the way down, how do we resolve them?

A Positive Vision

Nathan Labenz, host of the popular AI podcast, The Cognitive Revolution, often says that the scarcest resource today in AI isn’t GPUs, compute, or researchers, but a positive vision. This is as true for AI companies as it is for universities.

A positive vision is not unreflective techno-optimism, nor a manic insight about how the latest AI capabilities will upend higher ed. It is instead a slow, nuanced, deliberative account of what we’re trying to teach and why. It’s the secure base from which we can confidently explore how, if at all, AI fits with who we are and what we do.

I’m very lucky to have my first sabbatical this Fall, and in the rest of this essay, I’ll share how I’m going to be re-thinking my approach to teaching philosophy at a large, public Canadian university. Your mileage may vary.

Teaching OOD

The first component of a positive vision is a compelling answer to the question: What can students get from my class that they can’t get from AI?

It is well-known that LLMs degrade out-of-distribution (OOD): novel problems, genuinely local knowledge, fine-grained particulars, and embodied and institutional knowledge. So, one guiding ideal is to teach out-of-distribution, as much as possible.

Another way to think about it is the difference between artisanal goods and commodities. The main reason I’m willing to pay more for farmer’s market heirloom tomatoes is that they taste better. But I also like hearing from growers about micro-climates in the field and how this year’s crop compares to last. The human story and connection constitute the value. Contrast this with USDA-graded supermarket tomatoes. They’re mealy, fungible commodities abstracted away from any particular plant or field.

So, another idea is to offload the commodity-content to LLMs and focus class time and energy on the artisanal good-equivalents. Textbook accounts of consequentialism are commodities. Have the students chat to (carefully prompted) LLMs about it. That way, they can ask for examples and clarifications relevant to them. Then, class time is about my experiences using consequentialist arguments for animal welfare at the Thanksgiving table with in-laws.

New Affordances

A positive answer to the question: “what can students get here that they can’t from AI?” also needs to go beyond existing teaching and learning practices. We need to ask, what can I do now with AI that I couldn’t do before? What are the new affordances?

Andy Hall’s student-led evals project is one of the best answers I’ve seen to this question. Though not as impressive, here’s one example I trialled last semester with an AI-forward graduate student.

Say I allocate ~45 minutes per student for feedback on term papers. One thing I can do now is jot down very rough, directional comments on the paper. And instead of spending ~15 minutes polishing and editing those comments, I can now send the paper with my comments to an LLM and ask for two things: First, a polished version of my comments. Second, critical feedback on my comments, with an emphasis on my idiosyncrasies and blind spots, plus novel suggestions and counterarguments. Then, with the time I didn’t spend polishing, I can write meta-comments on the AI’s feedback. Now, the student gets three layers of feedback, plus a model of philosophical exchange.

If our duty as faculty advisors is to give students the best feedback we can, then it’s probably wrong to NOT use methods like this. Indeed, I’m quite certain that, at least for now, Gus + Claude > Gus or Claude. Not using AI has real costs.

Judgment All The Way Down

Careful readers might have noticed a tension. Teaching OOD seems to suggest a retreat: “find what AI can’t do, and plant your flag there”. New affordances seems to suggest an advance: “pick up the tool and do more”. Which is it?

Like most questions worth asking, the answer is, “it depends”. And what it depends on is what Aristotle called phronesis, variously translated as practical wisdom or judgment. It is the opposite of applying rules. It’s the capacity to see what a particular situation demands once the rules have run out. Thinkers from Aristotle to Autor have recognized that this kind of judgment resists codification and commodification.

“No AI in my classroom” and “unfettered AI access” both fail as policies because they substitute a rule for judgment. Abandoning take-home essays made the same mistake and it cost our majors their writing samples. I’ve since tied take-home writing to in-class exams, but (surprise!) it is imperfect and involves trade-offs.

When AI makes prediction cheap, the value of its complements - data and judgment - rises. When intelligence is abundant, judgment is what’s scarce. Cultivating judgment is arguably the most important goal for education in the age of AI.

We need to be honest with ourselves, and with our students. Using LLMs involves trade-offs. They tend to erode the very skills that are needed to use them well. But not using LLMs also involves-trade-offs. They are the greatest tool ever invented for learning. They are also the greatest tool ever invented for not learning. There’s nothing in the technology itself that decides which wins out. We do. One judgment at a time.

I saw this most clearly in my graduate seminar’s discussion of Valerie Tiberius’s just-published, Artificially Yours: Real Friendship in a World of Chatbots.

Tiberius rejects both Techno-Optimists who think that AI companions will solve the loneliness epidemic, and Techno-Pessimists convinced they’re nothing more than Big Tech’s latest affront to human dignity. She argues that different kinds of friendships can be valuable in different ways, some of which AI companions can deliver, some of which they can’t.

Before AI companions, many people had meaningful online relationships carried out entirely through text: Online support groups for illness and grief, MMO guilds, pen-pals, etc. One question Artificially Yours forces us to reckon with is: if people can have online, text-only relationships with AIs, wouldn’t that count as friendship, too?

When I raised this question in class, a student who had developed these kinds of online friendships (with humans, on are.na) pointed out that the novel and pressing possibility of AI friends helped to clarify something they hadn’t realized was important about their online human friendships. Namely, there was an implicit assumption that, if one of them travelled near the other, they’d meet up. If they learned an are.na friend had a meeting just a short drive away and didn’t reach out, they’d rightly feel hurt. One cannot assume this with AIs because they are not the kinds of things that can meet up in-person.

The closer AI gets to some human-like capability, the easier it is to see what’s missing.

’s recent essay argued that “AGI” was a more useful concept when we were farther away from it. The point of talking about AGI circa 2006 was “to contrast with the ‘narrow’ AI systems that existed at the time” doing single tasks like detecting credit card fraud, or filtering out spam e-mails.

Back then:

It was helpful to be able to gesture in the direction of much more capable, general-purpose systems that we might one day develop. But that’s changed. Today’s best AI systems are good enough that they’re now inside the fuzzy conceptual cloud of “AGI-ish”: that is, they’ve surpassed some people’s definitions of AGI, while falling well short of others’. As a result, talking about “AGI” is no longer a helpful way to gesture in a rough direction.

Proximity has a way of enforcing conceptual clarity.

Here’s another example:

This is half-correct. Something important is happening with false summits, when capabilities we once thought sufficient for “friendship”, “intelligence”, “reasoning”, “understanding”, or “creativity” no longer seem so, once achieved by LLMs. But the conceit that “we of course keep moving the goalposts” is too quick.

This leads to an important meta-question about AI progress: How can we tell the difference between legitimate false summits and illegitimate goalpost moving?

To maintain the belief that the Earth was the centre of the universe, Ptolemaic astronomers continually added epicycles to explain planetary motion. These ad hoc additions did explain planetary data retroactively, but they failed to predict future novel phenomena.

Contrast this with the discovery of Neptune. Astronomers noticed that Uranus’s orbit deviated from Newtonian predictions. So, they proposed an unseen planet tugging on Uranus. They calculated exactly where this hypothetical planet should be, and then Johann Galle looked through his telescope and found Neptune there.

The difference here between Ptolemaics and Newtonians is captured by Imre Lakatos’s distinction between progressive and degenerating research programs. When faced with anomalies, a progressive program modifies hypotheses in ways that generate novel predictions and insights. A degenerating program modifies its hypotheses to absorb counterevidence without producing new understanding.

This distinction provides a useful heuristic for debates about AI progress.

ARC-AGI

François Chollet’s (2019) “On the measure of intelligence” provides a positive, theoretically motivated account of intelligence as skill-acquisition efficiency. It then operationalizes it as a falsifiable instrument: The Abstraction and Reasoning Corpus benchmark. State-of-the-art models at the time scored around 5%, while humans were near-ceiling.

In late 2024, OpenAI’s o3 model hit ~80% on ARC-AGI-1, surpassing the human baseline for the first time. ARC-AGI-2 was then launched in early 2025. It took frontier models about a year to approach human-level. In March of this year, the interactive ARC-AGI-3 benchmark was released. As of this writing, base models sit near-floor (GPT-5.5: 0.43%; Opus-4.7: 0.18%).

When ARC-AGI-4 is released, will Chollet and colleagues have moved the goalposts? I don’t think so. Theirs is like a progressive research program. Each release is a positive instrument designed for the next (probably false) summit. For ARC-AGI-2 it was symbolic interpretation, compositional reasoning, and rule application. For ARC-AGI-3 it’s exploration, modelling, goal-setting, and planning. Each release does the equivalent of pre-registering what it would look like if the planet were there. A degenerating program, by contrast, moves the goalposts, bolting on denials to protect the pre-ordained conclusion “AI isn’t intelligent.”

We didn’t know that “we’d meet up if we were nearby” was partially constitutive of online friendships until the AI companions threw it into relief. Similarly, we used to think common sense factoids captured something essential about “understanding” until LLMs saturated CommonSenseQA.

AI “near-misses” often function like contrast dyes, helping us to see that implicitly held criteria were doing important work. What should we do when the next human-like capability falls to LLMs and new criteria are made visible?

Writers worth reading will have a positive account. They’ll build new instruments, specify operationalizations, update theories, and stake new predictions. But many commentators will be quick with a negative account, concocting epicycle-equivalent rationalizations to insulate pre-established conclusions about what AI can’t do. This is the difference between legitimate false summits and illegitimate goalpost moving: whether the criticism is progressive or degenerating.

“the environmental harms, the reinforcement of bias and generation of racist output, the cognitive harms and AI supported suicides, the problems with consent and copyright, the way AI tech companies further the patterns of empire, how it’s a con that enables fraud and disinformation and harassment and surveillance, the exploitation of workers, as an excuse to fire workers and de-skill work, how they don’t actually reason and probability and association are inadequate to the goal of intelligence…”

And the list goes on.

But our Martian would be excused for not knowing that AI researchers just won two Nobel Prizes, or that Indigenous researchers are using AI to preserve their languages, or that scientists are using AI to search for life on other planets, improve prediction of natural disasters, find political consensus, and improve animal welfare. Nor would they know that some people become less anxious or depressed after talking to chatbots, or that AI is detecting breast cancer earlier and more accurately than radiologists.

The “AI Winter” described a period (roughly, 1990s) when the field’s credibility collapsed because its practitioners overpromised and underdelivered. The one-sidedness of AI Ethics is putting the field on a path toward a similar collapse, an AI Ethics winter.

How did this happen?

AI ethicists often point out, correctly, that people associated with AI companies have incentives to hype AI capabilities. What is less often appreciated is that AI ethicists have their own incentives.

Peter Königs argues that AI ethicists face three structural pressures: the field is organized around commenting on new technologies, there are strong norms against purely positive commentary, and academics must publish and get grants to survive.

The result is that raising ethical concerns about AI is the most viable career path. As Königs puts it, ethicists cannot simply say to an AI researcher: “Look, what you’re doing is terrific. I don’t see any problems.” No matter how good the project, the ethicist’s job requires them to problematize it. If they don’t, they’re out of a job.

has argued that these incentive structures shape both what researchers study and how they reason. When researchers are strongly incentivized to reach these kinds of critical conclusions, they engage in motivated reasoning, lowering standards for evidence and arguments that support such conclusions and ignoring information or possibilities in tension with them.

Consider Karen Hao’s widely acclaimed Empire of AI. In a chapter framing data center water use as a continuation of colonial extraction, Hao claimed that a Google data center in Chile “could use more than one thousand times the amount of water consumed by the entire population of Cerrillos, roughly eighty-eight thousand residents, over the course of a year.”

But showed these figures were off by three orders of magnitude. It resulted from confusing cubic meters with liters. This mistake passed through teams of fact-checkers and reviewers, and major press coverage before a correction was issued.

If you don’t believe Williams’s claim about motivated reasoning, just imagine the immediate and withering scrutiny that would follow from a high-profile journalist making a claim of comparable magnitude in the other direction: “AI is saving a thousand times more water than it uses!”

These incentives and motivated reasoning tendencies conspire to produce the one-sidedness that the opening Martian parable illustrates. Williams’s analysis of news media provides a helpful analogy here. Most individual stories published by mainstream outlets are accurate, but because outlets cover a non-random sample of negative events, audiences end up radically misinformed about the state of the world:

“People develop mental pictures of reality far more negative than the objective facts warrant. They overestimate poverty, crime rates, and many other social pathologies and dangers, and believe most trends are going in the wrong direction.”

Something similar is true of AI Ethics. Individual papers are often valuable. But the one-sided market for critical content means these papers add up to a body of work that is overwhelmingly and misleadingly negative.

So What?

One obvious rejoinder here is: So what if AI ethics is one-sided? Given the AI Hype Machine, we need a counterweight!

This seems plausible at first but falls apart under scrutiny. The counterweight argument implies that, without AI Ethics, society would be pro-AI. But public opinion on AI is not very positive, and so the idea that society needs AI ethicists to provide a corrective is dubious. And even if the counterweight theory held water, Königs points out that it is very strange indeed to deliberately design a scholarly community to be biased.

Painting AI in an overwhelmingly negative light also makes it very difficult to understand why nearly a billion people use ChatGPT each week, and why 50 million pay for it. One-sidedness also distorts the space for productive debate. Pointing out the negativity bias, or suggesting that AI can be beneficial reliably invites charges of being a “tech bro” or Silicon Valley apologist.

Incentives

One might also object (without resort to ad hominems) that the incentive argument cuts both ways. Both optimists and pessimists about AI are subject to incentives that can distort reasoning. The appropriate response is to prioritize adversarial collaboration (otherwise known as “philosophy”) where both sides subject their work to scrutiny from the other. What I am arguing against here is a disciplinary monoculture where only one of these orientations is professionally rewarded.

Irrelevance

As a card-carrying AI Ethicist, I think the single biggest problem with one-sidedness is that the discipline is critiquing its way into irrelevance. AI Ethics should be an applied discipline. This means engaging with real trade-offs in real institutions under real constraints. A medical ethicist who only says, “here’s the negative side effects” and never says, “here’s how to weigh risks, benefits, alternatives” would be useless to, and eventually ignored by, healthcare providers. That’s roughly where AI Ethics stands today.

Can it be fixed?

The oldest question

Critics will be quick to point to counterexamples1 so let me state unequivocally: Work in AI Ethics has, and continues to, identify important ways in which algorithmic systems perpetuate harms and contribute to various forms of oppression. I’ve written about this. I teach classes about it. I’ve advised and supported more than 20 graduate students working on projects in this vein. Moreover, we have every reason to adopt a sharply critical posture towards the technology companies currently racing to build AGI. It’d be foolish not to.

What I’m arguing is that this is not the ONLY thing AI Ethics should do.

AI Ethics is a sub-discipline of Ethics. Ethics is about living well. And living well requires much more than identifying and avoiding harms.

Consider another parallel, this time with an (oversimplified) example from epistemology. W.K. Clifford famously argued that “it is wrong always, everywhere, and for everyone, to believe anything upon insufficient evidence.” William James countered that we have two epistemic duties: avoiding error and gaining truth. Someone who only cared about never being wrong would believe almost nothing and miss most of what matters.

An ethics that only asks, “what could go wrong?” is as incomplete as an epistemology that only asks, “how might I be deceived?” And yet, AI Ethics has become so fixated on harms that it has lost sight of the equally important task of understanding how AI might contribute to human flourishing.

On its current trajectory, AI Ethics is making itself irrelevant at precisely the moment it should be indispensable. Luckily, the path back to relevance is simple. A renewed and honest commitment to the oldest question in philosophy: “How should we live?”

Much of the recent discourse about AI progress and automation makes this mistake.

The Real Story

Benchmarks are some of the best tools we have for mapping AI progress. They provide a window into the kinds of tasks that are just-out-of-reach for frontier AIs, thereby giving them a hill to climb. An underappreciated corollary is that there are also many tasks that are so far out-of-reach that no one has bothered designing a benchmark. We tend to focus on the former and not the latter. Is anyone talking about how well Gemini 4 will score on ECEval, a benchmark designed to measure AI progress toward a drop-in pre-school teacher? Of course not. Early Childhood Educator tasks are so far out-of-reach that there’d be no training signal.

Instead, the news is that Claude Opus 4.6 beats GPT-5.2 by 144 Elo points on GDPval-AA. To be clear, this is extremely impressive. But the real benchmark story is that a decade ago, the tasks that were just-out-of-reach were identifying cats in the ImageNet database. Today, the tasks that are just-out-of-reach look an awful lot like running a business.

From this vantage point, large-scale labour displacement can feel inevitable. As Leopold Aschenbrenner says, “this doesn’t require believing in sci-fi; it just requires believing in straight lines on a graph.”

Nowhere is this more prevalent than with METR’s “Measuring AI Ability to Complete Long Tasks” benchmark. Otherwise careful thinkers are quick to infer that the straight lines on the log-graph mean we are on the cusp of fully automating software engineering. Less careful thinkers generalize to “any job that mostly involves typing on a computer.”

But has pointed out that, if you read the fine print, METR themselves admit most of the tasks in their benchmark are not representative of actual software engineering work, much less white-collar work. They are instead discrete, linear, easily measurable, algorithmically scorable stand-ins for software engineering sub-tasks. OpenAI say much the same about their GDPval benchmark.

What Dewey helps us see is that there’s actually nothing wrong with this: “Selective emphasis, choice, is inevitable…this is not an evil.” We often can’t directly measure the things we care about, so we use more easily measurable stand-ins or proxies. The problem only arises when we mistake these stand-ins for the real thing. “Deception,” Dewey continues, “comes only when the presence and operation of this choice is concealed, disguised, denied.”

Most interpretations of the METR results are deceptive in this sense. By METR’s own lights, there are a small sub-set of “messy” tasks that better represent real work done by software engineers. Witkin argues that, on these tasks, no model topped a 30% success rate, and this should have been the headline result.

Instead, we get social media posts like, “CLAUDE OPUS 4.6 HAS 14.5 HOUR TIME HORIZON!” followed by breathless commentaries about the incoming wave of automation and the “great disemboweling of white-collar jobs.”

AI progress is indeed alarmingly fast, and speeding up. It seems rational for Substack-reader-and-writer types (present author included) to worry about how and when AI will affect their livelihood. But we won’t make much progress on this question by having a bunch of mostly non-software engineers debating the merits of a blog post about crude stand-ins for software engineering sub-tasks. Like most Very Online debates, this one would benefit from touching grass.

I was recently approached by a healthcare organization, who, like most in Ontario, had an appallingly long waitlist for new patients. And like most healthcare organizations, they were optimistic that AI could deliver some much-needed efficiencies.

Indeed, the healthcare sector has been one of the most rapid adopters of AI. According to the AMA, 66% of physicians used AI in 2024, up from 38% in 2023. ChatGPT now fields 40 million health-related queries per day. Tens of billions were invested in AI-powered health tech in 2025.

AI for clinical documentation is consistently ranked as a top use case, and for good reason. It is the lowest hanging fruit on the automation tree: a repetitive, low-risk, high-volume, language-in-language-out task. Right in the wheelhouse of today’s LLMs. This made sense as a place to start on our collaboration.

The idea was that clinicians would jot down a rough “scratch note” between patient visits, and we could then use LLMs to transform that scratch note into a structured SOAP note (a standard format, consisting four sections: Subjective, Objective, Assessment, Plan). We interviewed 20 clinicians about their experience with this process.

To understand what we found, and what it means for the broader debates about AI progress and automation, we have to go into the weeds (if you want to go deeper into the weeds, you can read the paper).

Write on paper, wrong in practice: 4 themes

Heterogeneity

The clinicians in our study were mostly occupational therapists working in pediatric rehabilitation. Some worked in schools, others in the clinic. The first theme that came out during our interviews was that documentation workflows varied considerably. Some wrote scratch notes that could be easily fed to the LLMs. Others used their own shorthand for scratch notes which couldn’t easily be fed into the LLMs. School-based clinicians, working in a classroom, didn’t have the luxury of writing scratch notes at all. Some clinician’s “scratch” notes were so detailed that there wasn’t much for the LLM to do. Even within the narrow subset of clinical documentation for occupational therapists working in pediatric rehabilitation, there wasn’t a single process to be automated.

Countability

Another theme that came up repeatedly in our interviews was that “SOAP notes aren’t the problem.” Here’s two representative quotes:

“No, I think if I had to tell you what I think the problem is, I don’t think it’s our ability to write, It’s the amount of things we have to do.”

“I think I was intrigued by it at first and then the reality of it, I was like, it actually doesn’t save me time...I think our other processes are extremely inefficient.”

Even if LLMs were able to reduce their documentation burden, clinicians were skeptical that it would improve their overall situation. This echoes C. Thi Nguyen’s insight in The Score that, within organizations, “easy countability,” understood here as time spent writing notes, “automatically wins out over actual importance.”

Identity

A pervasive assumption in the academic literature is that clinical documentation is a burdensome task that should, to the extent possible, be automated away. And for many clinicians, this assumption holds. Their lives would go better if they could reduce the amount of time spent writing notes in their pajamas.

But a few clinicians in our study pointed out that note-writing can be an expression of professional identity:

“Like even in an objective way where I’m stating ‘objective’ [the ‘O’ in SOAP]...It still has your person like, it has your sound to it. And so when you’re reading and it doesn’t have your sound, like this is wild.”

Others noticed how they could guess where a clinician trained, based on their documentation style.

Many clinicians view their notes as a burden better offloaded. But some view them as an outlet for their clinical reasoning and judgment, in which case the LLM tools looked like a solution in search of a problem.

Tools

Our study and many others have found that, for some clinicians, time spent prompting, reviewing, editing, and iterating with LLMs erases productivity gains. As one of our participants said:

“I’m like this is too much effort for me to tell you what to do. I already know what I wanna do, so I would just use my own note like I would just abandon it altogether.”

Still, even if LLMs don’t save time, they can reduce burnout because many people find editing outputs to be less cognitively demanding than generating them.

In this context, perhaps our most interesting finding was that, among clinicians that didn’t outright abandon the tools, some reported adapting their own workflows by creating more AI-friendly scratch notes. Rather than tools helping clinicians, clinicians were helping the tools.

At this point, one might object that these are a bunch of cherry-picked examples from a small qualitative study of an un-representative population. What does the broader literature find?

The best evidence comes from an RCT conducted at UCLA: 238 physicians across 14 specialties, encompassing 72,000 patient encounters. Users of one tool (Nabla) found a 10% reduction - about 40 seconds per note - in documentation time. Another tool (DAX) showed smaller, non-significant reductions.

The largest longitudinal study to date is from Kaiser Permanente: over 7,200 physicians, 2.5 million patient encounters. The paper reports 15,700 hours saved collectively, with after-hours documentation reduced, on average, by a minute per appointment. 84% of physicians said AI improved ability to connect with patients and 82% reported greater job satisfaction.

A smaller longitudinal study (200+ clinicians over 180 days) found no speed up, on average. And a recent study out of UCSF showed that 86% of physicians perceived documentation time reductions when using AI tools, but these subjective impressions did not track any objective time-savings. This last point mirrors another METR study which found that software engineers subjectively felt like AI sped them up by around 20%, but it in fact slowed them down by around 20%.

So much for the weeds.

What do we make of these findings? Our qualitative study found limited benefits of LLMs for clinical documentation. The quantitative literature is mixed but seems to find real, but modest time savings and burnout reduction. Solid, but not transformative. Normal for this kind of technology, we might say.

What does this tell us about AI and automation more generally?

It’s difficult to imagine a real-world task better suited for today’s LLMs than this kind of clinical documentation. It really does seem like the lowest-hanging fruit, and so it would be easy to infer that if LLMs struggle here, we should be skeptical of claims about broader automation and job loss. While I’m sympathetic to this claim and how people like have argued for it, I think is mostly right that nobody really knows.

But if we want to make epistemic progress, there’s no substitute for going into the weeds and studying how actual people are using actual AI tools to do actual tasks within their jobs. From this perspective, one is much less likely to commit the Philosopher’s Fallacy. There’s no mistaking the stand-in for the real thing: Writing clinical notes is just one small sub-task within the broader task of “indirect” clinical work. And this “indirect” work is itself ancillary to the much broader and infinitely more complex set of meta-tasks, tasks and sub-tasks associated with “direct” patient care.

None of this is this unique to healthcare. In software engineering – arguably the ripest job for AI automation - (an actual software engineer) makes the same point: Benchmarks are not job tasks, and even if they were, jobs are still more than bundles of tasks.

And yet, quantitatively and qualitatively, AI progress continues to accelerate. The question we’re left with is whether trillions of dollars of compute can summon the real thing from the stand-ins.

When we interact with software, we’ve come to expect consistency: Excel never forgets how to sum numbers. Spell-check never loses its dictionary. When we interact with people, we expect different limitations: humans make typos, forget facts, but possess common sense.

Where do today’s LLMs fit into this scheme?

The short answer is: they don’t. They’re human-like in some ways, software-like in other ways. They’re mostly tool-like but increasingly agentic. They don’t fit neatly into existing categories. How, then, should we think of them?

Shoggoths

One popular meme depicts LLMs like this:

The shoggoth represents the base model produced through pre-training on internet-scale text data. The colourful mask (the only part we interact with) is the thin veneer of alignment achieved through post-training techniques like Reinforcement Learning from Human Feedback (RLHF): safety guardrails, instruction-following capabilities, and the cheery assistant persona that is supposed to be Helpful, Honest, and Harmless.

Despite its visceral appeal, Shoggoth overemphasizes alien otherness. If the base model is monstrous, it’s a monster constructed from human language.

Zeitgeist

Writing about GPT-3 shortly after its release, Regina Rini captured both alien otherness and familiar humanity:

GPT-3 is not a mind, but it is also not entirely a machine. It’s something else: a statistically abstracted representation of the contents of millions of minds, as expressed in their writing. Its prose spurts from an inductive funnel that takes in vast quantities of human internet chatter: Reddit posts, Wikipedia articles, news stories. When GPT-3 speaks, it is only us speaking, a refracted parsing of the likeliest semantic paths trodden by human expression. When you send query text to GPT-3, you aren’t communing with a unique digital soul. But you are coming as close as anyone ever has to literally speaking to the zeitgeist. (“The Digital Zeitgeist Ponders Our Obsolescence”)

Greg Brockman, a co-founder of OpenAI, made a similar point on a recent episode of: LLMs aren’t human, they’re humanity.

Better vibes than the Shoggoth, for sure. But still not quite right.

Gold Medal Winning Stochastic Parrots

Perhaps we should have started with a definition. The most literal answer to our question is: “next-token predictors”

During training, LLMs are shown sequences of text with some tokens masked, and they learn to predict what comes next by minimizing the cross-entropy loss between their predictions and the trillions of actual tokens in the training corpus.

It is tempting to conclude that therefore LLMs are just auto-complete on steroids, or stochastic parrots. Sure, the argument goes, they are impressive at imitating language, but they don’t understand language.

This is where things get messy. The stochastic parrot account is stronger than AI optimists want to admit, but not quite as strong as AI pessimists contend.

Here’s an example of the latter: Naïve AI pessimists argue that LLMs can’t really reason (or “understand”, or “be creative” or “think” or whatever) because they’re just next-token predictors. This is obviously wrong. Millière & Buckner dub it the “redescription fallacy”:

Its like asserting that a piano could not possibly produce harmony because it can be described as a collection of hammers striking strings, or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. (“A Philosophical Introduction to LLMs”)

Here’s an example of the former: sophisticated AI pessimists argue that LLMs model the training corpus, not the world itself. In other words, next-token prediction doesn’t reliably generate world models: internal states of the model that represent the external world.

Here, AI optimists point to examples like Othello-GPT, where a model learned to internally represent complete states of a board game. Intervention experiments showed that these representations causally drive the model’s outputs, rather than being mere correlational byproducts. A toy example, to be sure, but proof-of-concept that next-token-prediction can generate something like a world model.

But then AI pessimists counter with the example of LLMs trained on planetary trajectories. The models achieved near-perfect predictions of planetary positions but completely failed to recover Newton's law of gravitation, despite having seen it countless times in training. The LLMs learned narrow correlational heuristics, but couldn’t generalize the underlying causal model:

That is pretty damning. Given vast computational resources and all the pirated textbooks on the internet, the so-called frontier models couldn’t do much better than Kepler did in the 17^th century.

And yet, it still feels misleading to suggest that LLMs are just stochastic parrots. They won gold medals at the International Math Olympiad!

Where does that leave us?

E-Bikes

In my course “Digital Wisdom: How to Use AI Critically & Responsibly” I opt for a pragmatic model of the models: LLMs are electric bikes for the mind. I tell students: E-bikes don’t replace legs. You still need to decide where to go, steering and balancing along the way. Then, you can climb hills you couldn’t before.

I think this works pretty well, pedagogically. But it breaks down, ethically. E-bikes aren’t situationally aware, nor do they try to avoid being shut down, nor do they actively help kids die by suicide.

Shoggoths, Zeitgeists, Parrots, and E-Bikes are all catchy and convenient. But like most memes, they clarify some things and obscure others. It should be clear by now that the “right” model of LLMs is context dependent. But there is one very important contextual factor that is missing.

The Drop-In Remote Worker

I first came across the idea of a drop-in remote worker in Leopold Aschenbrenner’s influential 2024 essay (story?). They are AI agents that can be onboarded like a human employee and autonomously complete any job that can be done remotely with a computer, internet connection, and standard digital tools.

This is clearly NOT a description of today’s LLMs. But today’s LLMs are best understood as points along a trajectory whose endpoint is a drop-in remote worker.

There is, of course, much disagreement about how far along we are on this trajectory. I don’t pretend to know, though my guess is that there will be many “false summits” along the way, and that progress will be uneven across domains. Regardless of whether or when you think that endpoint is achievable, it is very difficult to understand the big picture of LLMs today without the concept of a drop-in remote worker.

At the beginning of the covid pandemic, one influential analysis suggested that 37% of occupations in the US could be performed entirely at home, and that these jobs accounted for nearly half of all wages.

A related analysis suggests that 34% of job tasks across different occupations could be completed remotely, which is to say, with a computer, internet connection, and standard digital tools.

Data from Anthropic’s Economic Index is in the same ballpark. Adding it all up, this means that on the order of trillions of dollars are paid out in wages for remote work in the US.

And that number explains today’s frantic obsession with AI agents. If the AI companies can capture even a small slice of that multi-trillion-dollar-pie by automating remote work, the hundreds of billons being invested today will have been worth it. The start-up Mechanize says the quiet part out loud.

Benchmarks are an under-appreciated resource for developing a mental model of LLMs. They often measure capabilities that are just out of reach of current systems and thereby generate a training signal to improve future LLMs.

Why is this often called the most important graph in AI today? Because it’s a benchmark for measuring progress toward a drop-in remote software engineer:

If this trend of the past six years holds for another four, then we’ll have LLMs capable of autonomously carrying out a month-long software engineering project with a 50% success rate. Other important benchmarks (e.g., SWE-bench, RE-bench, TAU-bench, Vending-bench) are increasingly focused on real-world, economically useful tasks like this. There can be little doubt about the prize that the AI companies are eyeing.

Seen in this light, Shoggoths, Zeitgeists, Parrots, and E-bikes, while useful in some contexts, miss the forest for the trees. As Anthropic co-founder Jack Clark says:

We are not making dumb tools here - we are training synthetic minds. These synthetic minds have economic value which grows in proportion to their intelligence.

I’m not arguing that just because the AI companies think about LLMs in this way, it is therefore the best model for all contexts we might care about. But like Theodosius Dobzhansky’s claim that nothing in biology makes sense except in light of evolution, I am arguing that nothing about the current AI boom makes sense except in light of drop-in remote workers.

Notes and Additional Resources

The first unit of my first-year Philosophy course, “Digital Wisdom: How to Use AI Critically & Responsibly” is about developing a mental model of LLMs. This essay reflects some of my thinking about the topic.

Here’s what I assign in the course:

• Hendrycks, Dan. "Chapter 2: AI Fundamentals." In Introduction to AI Safety, Ethics and Society. Taylor & Francis, 2024. ISBN: 9781032798028. Available at: www.aisafetybook.com

• Lee, T. and Trott, S. "A jargon-free explanation of how AI large language models work." Ars Technica, July 31, 2023. https://arstechnica.com/science/2023/07/a-jargon-freeexplanation-of-how-ai-large-language-models-work/

• I also suggest these interactive demos of the perceptron visualizer and the neural network playground.

Here’s some additional background recommendations:

• For historical context on AI generally, read ’s Substack.

• Rich Sutton’s (2019) Bitter Lesson essay captures the ethos of the present “scaling era”:

  • “We have to learn the bitter lesson that building in how we [humans] think we think does not work in the long run. The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.”

• Andrej Karpathy’s 3-hour “Deep Dive into LLMs like ChatGPT” is explicitly aimed developing a mental model of LLMs.

• “The Void” tumblr post is very weird, possibly unhinged, but somehow provides a vivid sense of LLM “phenomenology” through the lens of the “AI psychologists/psychonauts”.

• In building a mental model of LLMs, it is helpful to peek into the mental models of people who saw ChatGPT coming before everyone else. See Nathan Lambert’s annotated transcript of Ilya Sutskever’s 2015 remarks on deep learning.

In April of this year, wrote about how “surreal, dystopian, and wildly entertaining” it was to watch ChatGPT o3 guess the location of this photo:

From the simple prompt, “Guess where this photo was taken” o3 reasoned, “The pastel-colored houses and the hills in the background resemble areas like Big Sur. A license plate could offer more, but it’s hard to read.”

o3 then “leapt straight into science fiction,” autonomously running Python code to crop a binding box around a license plate in the image to zoom in and see if it says, “California”.

After six minutes, o3 offered a few guesses, one of which was correct (El Granada, California).

The privacy implications are obvious and indeed dystopian. Watching the Chain of Thought unfold was mesmerizing. The fact that AI could use tools in the Chain of Thought was a step change.

But here’s what stands out to me: “Guess where this photo was taken” is a very lazy prompt.1

o3 returns so much more than was invested. That tells us something important about how AI can short-circuit learning.

If you haven’t been following closely, here are three of the most important developments in LLMs since ChatGPT was first released, each of which contributes to the “surreal” geo-guessing capability:

1. REASONING: The original ChatGPT was only capable of System 1 Thinking. You prompted the model, it returned the first thing that comes to mind. Today’s LLMs are capable of System 2 Thinking. They can stop and think and plan and use tools before responding.

2. MULTI-MODALITY: The original ChatGPT was a narrow chatbot. You could only input text, it could only output text. Today’s LLMs take not only text as input, but also voice and other audio, images, videos, code, and they generate outputs across these and other modalities.

3. LONG CONTEXT: The When ChatGPT was first released, its context window was 6,000 tokens, or 5 pages of text. Today, Google’s Gemini 2.5 Pro context window is 1 million tokens. This means it can keep 3,000 images or 1,500 pages of text, or 45 minutes of video, or 8 hours of audio in working memory.

In practice, these developments reinforced each other and transformed LLMs from a mostly useless toy into a cognitive power tool. Still, many people don’t appreciate just how much expertise you can pack into today’s multi-modal, long-context, reasoning models. And how much better the outputs are when you do.

Compare “guess where this photo was taken” with Kesley Piper’s 1,000 word geoguessr prompt.Read it. Seriously.

This prompt enabled o3 to correctly geo-guess photos of featureless plains and random rocks. Its easy to see why Scott Alexander felt like a chimpanzee watching a helicopter.

This is superhuman stuff. And it’s unlocked through a very long, detailed prompt infused with a combination of both domain expertise (“A telephoto-looking ridge can be many kilometres away; compare angular height to nearby eaves”) and AI know-how (“You are an LLM, and your first guesses are 'sticky' and excessively convincing to you”).

Geoguessing is fun. But there are more practical applications, too.

Tools of the Trade

Everyone knows AIs have a tendency to hallucinate and oversimplify. A simple and lazy solution is to prompt the AI to “please fact check before producing your output”, or “provide necessary context”, or even pass the initial output to another LLM and ask the second LLM to fact check and contextualize. These are better than nothing.

But for contrast, read ’s Deep Background prompt. It’s too long to post here (over 3,600 words!) but worth reading as an exemplary piece of Applied Epistemology.

It’s also incredibly useful. You copy and paste the prompt (or use the prompt as Custom Instructions in a Project), then upload a question, claim or screenshot. Deep Background then generates a structured report which contextualizes and fact-checks your input.

You can edit the prompt to change how different kinds of sources are weighted. The prompt also has a bunch of interactive commands, which I illustrate here in a chat with Claude about the extended mind thesis.

Like most GenAI, Deep Background seems mind-blowingly good at first, and then slightly less impressive upon closer examination (the code for the animation is buggy, for example). But compared to relevant alternatives like google search, it’s a clear upgrade, and another powerful demonstration of combining domain expertise with AI know-how.

An example from my “Digital Wisdom” course

Here's how this dynamic played out when I was building my first-year Philosophy course, “Digital Wisdom: How to Use AI Critically & Responsibly”. The second unit is a survey of how AI is being used across the economy. Even one year ago, I would have settled for a small number of examples familiar from my own research, like AI in mental healthcare or AI in Human Resources, plus some well-worn examples like predictive policing and killer robots.

I wrote 300-word summaries for these, added a video link, plus some discussion questions. Pre-Claude 4, I would have stopped here. But now, I can use these examples as a springboard to generate more examples (“few shot prompting”). The prompt below is long, and that’s the point.2

“This module is about cutting-edge use cases of AI. I want the students to have a sense of both the amazing things people are doing with AI, but also the potential downsides and negative ethical/social implications.

Here are four examples:

[insert AI in mental healthcare summary + video + discussion questions]; [insert AI in Human Resources summary + video + discussion questions]; [insert Predictive policing summary + video + discussion questions]; [insert Killer robots + video + discussion questions]

Now, I need to draft a new section on [insert new topic here]. Each section should contain three parts. First, find a relevant video, ideally no longer than 10 minutes. Do not tell me to search for the videos, I need your recommendations, even if they are not perfect. Second, draft a ~300-word summary of this use case in the style of [examples above]. For the ~300 word summaries, ONLY engage reputable, well-known, academic or news sources. For example, some of the best sources are University news website describing the work of their faculty. It is very important that the summaries are NOT repeating promotional AI materials from commercial websites. Similarly, please ensure that the summaries are balanced and objective. For example, don’t use overly hyperbolic language about transformative potential and impacts. Be careful, measured, and critical of AI hype. Third, suggest five critical reflection questions in style of [examples above]. Before producing your final output, please double-check that all quotations are accurate verbatim, and do not contain any hallucinations or overly evaluative language. Accuracy and objectivity are paramount. Please use APA format for all citations.

It took a long time to write the prompt and to fact-check and edit the outputs. But the investment paid off. Now, the module contains not just content from my idiosyncratic research interests, but 20 examples relevant to most majors at the university.

Do something new

Most of us have domain expertise that exceeds AI know-how. Just as it took time and effort to develop the former, it takes time and effort to develop the latter (though not as much as you might think).

The three examples above illustrate how these can be combined to do things that were previously infeasible. That’s the blessing of Reasoning + Multi-Modality + Long Context.

The curse is that the models are so good now that students (or anyone, really) taking shortcuts with lazy prompts can still generate impressive-enough outputs. This disguises the need for underlying competence. Why invest in learning about telephoto ridges when “guess where this photo was taken” gets close enough?

The question, of course, generalizes to Education with a Big E.

A New Short-Circuiting Argument

One answer is that if you can get impressive-enough outputs without domain knowledge, so can everyone else. The floor’s raised, but so is the ceiling.

Here’s a better answer: When you choose to use AI to avoid learning, you’re short-circuiting the process through which you gain enough domain expertise to use AI to go beyond the kinds of outputs AI can produce better, faster, and cheaper than you.

In case you thought this was only about students, here’s the starkest answer of all, from , who “feels the AGI” more than most in Higher Education:

The usual, comfortable rhetoric about “irreplaceable” human elements of education—mentorship, hands-on learning, community building, and critical thinking—might suffice for a four-year social networking summer camp, and some parents may still value that. But in the AGI era, the only defensible reason for universities to remain in operation is to offer students an opportunity to learn from faculty whose expertise surpasses current AI. Nothing else makes sense. (“It’s Later Than You Think”).

I’m not sure if or when that AGI era will arrive. But given the headwinds currently facing Higher Education, we ignore the argument at our own peril.

This is the kind of thing AI Natives can do, I cannot, and like , it makes me want to weep. It also portends a crisis.

One recent study of over 1,600 faculty from 28 countries found that 40% of faculty feel that they are just beginning their AI literacy journey and only 17% are at advanced or expert level.

This Fall, how will the 83% of faculty lacking AI literacy address students who feel ripped off: expected to use AI professionally but not taught how? How will they answer students questioning why they should pay tuition when AI teaches better, faster, cheaper?

I’ve not seen many answers that students are likely to find convincing. That’s why I spent the summer developing a new first-year Philosophy course called, “Digital Wisdom: How to Use AI Critically and Responsibly”. The course’s message is simple:

You can choose to use AI to learn, or you can choose to use AI to avoid learning.

By now, everyone knows what it looks like to use AI to avoid learning, although the strategies are becoming more sophisticated and harder to detect.

The problem, as I see it, is that students get lots of vague and mixed messages about AI use, but very little sustained, hands-on demonstration of what it looks like to use AI to learn, rather than avoid learning.

So, what does it look like to choose to use AI to learn?

First, there’s the choice. It requires concerted, deliberate action and it doesn’t happen by default.

Persona prompting is the lowest-hanging fruit here: Rather than asking for answers, have students tell the LLM things like, “You are a biology professor who specializes in making complex concepts accessible to first-year students. Explain CRISPR using Canadian agricultural examples.”

If students learn better through concrete examples, then: “Explain [concept] by providing three real-world examples from different domains, then show me how the same principle applies in each case. Quiz me at the end to test my understanding.” And so on.

By now, everyone also knows the risks of AI in education and many judge them high enough to justify AI “bans.” I don’t think there is such a thing. Sure, we can mandate in-person exams, but does anyone honestly think students don’t use ChatGPT to prepare for them?

Reddit is full of examples. Students I trust have told me as much about my own in-person essay exams (“upload the study guide to ChatGPT, try to memorize the outputs”). Banning AI just incentivizes unguided shadow use, where avoiding learning is more likely.

We also shouldn’t forget about the risks of not using AI. It can be very difficult for some students to ask clarification questions in large lecture halls, or to admit that they don’t understand a basic concept in front of their peers.

One of the most important features of LLMs for learning is that they are patient and non-judgmental. Students can ask as many follow-ups as they want. They can ask for explanations tailored to their learning styles, or for analogies to domains they are more familiar with. Banning AI in the classroom deprives students of these learning opportunities.

New features like ChatGPT’s Study Mode, Claude’s Learning Mode, and Gemini’s Guided Learning incorporate the above ideas with the click of a button.

Of course, it’s never so simple as clicking a button.

The Hidden Curriculum of Default AI

The single biggest problem with today’s LLMs is that they are sycophantic: they tend to tell users what they want to hear and use flattering language that is inconducive to learning. Unfortunately, the default setting of LLMs seems to incentivize providing the illusion of learning, without the hard work of actually learning.

When AI constantly validates and flatters, it can create false confidence in weak work and prevent genuine skill development. In extreme cases, it can even contribute to psychotic breaks.

Thus, when it comes to using AI for learning (and AI use more generally) one of the most important prompting strategies is “anti-personas” or telling the AI what it is NOT.

By explicitly programming against sycophancy, you make it more likely that you will get the kind of honest feedback that actually promotes learning: The kind a trusted mentor would give in private, not the polite encouragement given in public.

Here are some examples I encourage students to use in my course:

The “brutal editor” persona prompt

• “You are a harsh but fair editor reviewing my work. You are NOT interested in making me feel good about my writing. You do NOT start with compliments or end with encouragement. You do NOT say things like “great job” or “you’re on the right track.” Instead, directly identify specific problems and explain why they weaken my argument. Be concise and critical.”

The “skeptical professor” persona prompt

• “You are a demanding professor who has seen thousands of student papers. You are NOT impressed by basic observations or surface-level analysis. You do NOT give credit for merely attempting something. You do NOT soften criticism with praise sandwiches. Point out exactly where my thinking is shallow, where my evidence is weak, and where my logic fails. If something is genuinely good, you'll mention it briefly, but focus on what needs improvement.” And so on.

Custom Instructions

At this point, many will object: “The temptation to just ask AI to do all the work is too great, and students won’t reliably use those prompts.”

Fair point. Students can and do choose to take shortcuts. But they can also choose to not do this, if they are shown good alternatives.

An underutilized feature in today’s LLMs is “custom instructions” (ChatGPT, Claude, Gemini). These are like “meta prompts” that automatically apply to all your conversations with an LLM.[1]

Here’s what I say to students in my course:

If you want to make it more likely that AI will help you learn rather than avoid learning, add custom instructions like:

• “When I ask for help with an assignment, respond with 3-4 targeted questions that will help me think through the problem myself, rather than giving me solutions. Only provide direct guidance after I've demonstrated my own reasoning.”

• “If I ask you to write, summarize, or analyze something for me, instead provide a structured thinking framework and ask me to work through it step-by-step, checking my reasoning at each stage.”

• “If I seem to be using you to avoid learning rather than to enhance learning, point this out.”

Interviews

A fun and thought-provoking way to write these custom instructions is to have the AI interview you, with a learning-focused prompt like this:

“Please interview me to develop a set of custom instructions for [ChatGPT, Claude Gemini]. Help me create learning-focused custom instructions by asking about: (1) how I want [ChatGPT, Claude, Gemini] to support my learning process without doing the thinking for me, (2) my preferences for direct, honest communication over excessive positivity or sycophancy, (3) my background and main use cases, and (4) specific output requirements. Focus especially on understanding when I want to be challenged, corrected, or pushed to think harder rather than given easy answers. Ask follow-up questions as needed. At the end of the conversation, please draft the custom instructions for me to review”

These aren’t silver bullets. Students can always choose to override custom instructions. But they’re no less a band-aid solution than “banning” AI and driving use into the shadows.

Podcasts and Flywheels

All the examples so far are strategies I give to students. But here’s one of my personal favourites. Lots of good podcasters choose to use AI to learn, which helps them ask better questions of experts on their podcasts, which helps me to learn about a much wider range of topics than I did pre-ChatGPT.

In turn, I use AI to extend my Zone of Proximal Development. When I’m listening to an AI researcher on Latent Space, or a biochemist on Mindscape and a technical detail goes over my head, I sometimes choose to pause the podcast, switch to the Claude app, provide a link to the transcript (which the podcaster generated with AI), ask for an explanation (using voice input which is much faster than typing), follow-up if needed, then switch back to the podcast. I do all of this from my phone, while walking around campus.

describes a similar workflow, but for Kindle books:

These exemplify the flywheel effects that are enabled by choosing to use AI to learn. They also seem quaint, relative to some AI Native workflows.

The point is: AI amplifies intent. Choose to input lazy prompts which avoid thinking, output slop. Choose to write demanding prompts affording learning, build a nuclear fusor.

None of this is to say that humanities faculty need to match HudZah’s technical sophistication. This Fall, we have to teach what we’ve always taught: How to criticize shallow thinking, how to question comfortable assumptions, how to sit with ambiguity, how to entertain opposing views charitably. One difference between my “Digital Wisdom” course and those with AI “bans” is the recognition that these skills apply at least as much, maybe more, to prompting AI as reading texts or writing essays.

[1] Beyond learning-focused applications, custom instructions are generally quite useful for getting LLMs to stop doing things you find annoying or unhelpful. I get a lot of mileage out of custom instructions like: “write at the level of a tenured academic”; “Avoid flowery, overly cheery language, sycophantic responses, or engagement-driven questions”; “Never use phrases like ‘fascinating,’ ‘great point,’ or ask follow-up questions for engagement”; “Provide comprehensive, detailed explanations without concern for length”; “Prioritize critical, analytical thinking over tone-matching or making the user feel good”; “avoid unnecessary juxtapositions of the form, “Its not X, its Y”