“Am I relieved a machine has decided my work is human?”

Academic studies and recent news promise rapidly improving AI detection systems. Truth is they’re still meh, both because detection is wrongheaded in any case and it’s easy to game. Actually, the gaming part provides a solid intellectual enterprise of wrestling with ideas and language.

A new study published this year asserts impressive progress. Marijke Van Vlasselaer, Filip Van Droogenbroeck, and Bram Spruyt tested four widely used AI detectors: Turnitin, GPTZero, Copyleaks, and Pangram, on 160 papers of known provenance. Humans had written some, AI generated others, and some consisted of AI prose deliberately “humanized” to make it harder to detect. Pangram substantially outperformed the other systems, correctly placing 37 of 40 hybrid papers and 37 of 40 humanized papers within the expected range of AI use. On fully human texts, it produced no false positives in this particular dataset.

My students gleefully asserted that such claims were reason enough to break Pangram, not as a scientifically validated experiment, but rather as a game with the detector to see what it seemed to recognize as “AI writing.”

Our first experiment began with something no language model had written at all.

Two Stanford students had an ordinary conversation about whether they planned to attend the Stanford–Hawai‘i football game on August 29, 2026. We recorded them and transcribed the conversation. Then we vandalized perfectly good human speech, throwing in typical AI junk style: negative parallelisms, false comparisons, relentlessly tidy rules of three, some ostentatiously short pseudo-punchy sentences that sounded consequential without saying very much, and explanatory throat-clearing such as “this matters because.”

Here’s the transcript before:

Human transcript before adding AI slop

Student 1: Hey, are you going to attend the Hawaii Stanford game on the 29th?

Student 2: No, I don’t have tickets.

Student 1: My friend David has extras, and I could ask him. You know he’s the one we had Econ one with, sat in the front row with the glasses always super helpful so nice.

Student 2: Yes, he’s the one who runs the study group too, right?

Student 1: yeah really nice and really helpful to everyone else, so supportive and thoughtful. I’ll ask him if I can buy his extra tickets sometimes he just gives them some friends, but I’ll see what I can get for you.

Student 2: Great I’d love to go and if you sit with David, I’ll say hi too.

After Slop

Student 1: Hey, are you going to attend the Hawaii Stanford game on the 29th? This matters because tickets like this don’t come around often.

Student 2: No, I don’t have tickets.

Student 1: My friend David has extras, and I could ask him. It’s not just that he’s generous — it’s not just that he’s reliable — it’s not just that he’s kind — it’s all three, and that combination highlights the kind of person he really is. You know him: the one from Econ, front row, glasses, always ready to help. Not loudly helpful. Not occasionally helpful. Consistently helpful. That distinction matters.

Student 2: Yes, he’s the one who runs the study group too, right?

Student 1: Exactly. And this isn’t a coincidence — it’s a pattern. He’s not just nice to me, he’s not just nice to you, he’s not just nice to the people already in his circle — he’s nice to everyone, and that’s what separates genuine kindness from performative kindness. Having a friend like that around isn’t just nice to have. It isn’t just convenient. It’s genuinely grounding, in a way that highlights what real support looks like. I’ll ask him if I can buy his extra tickets. Sometimes he sells them, sometimes he just gives them away — but either way, I’ll see what I can get for you. It’s a small ask, but it says a lot.

Student 2: Great, I’d love to go — and if you sit with David, I’ll say hi too.

Student 1: Definitely. It’s these small moments — a game, a friend, a favor — that end up mattering more than we expect. I’ll let you know what he says.

Then we put it into Pangram and it only started detecting after the first few lines, though clearly we’d larded up the whole dialogue with AI slop.

Then we reversed the experiment.

We prompted Claude: Tell us about the role of Strunk and White and George Orwell in current writing pedagogy. Here’s the output.

AI Slop

Strunk and White’s The Elements of Style and Orwell’s “Politics and the English Language” were once paired staples of writing instruction, and today both have faded for strikingly similar reasons — not one reason, not two, but three. First, linguists have increasingly highlighted how both texts present personal taste as objective rule, whether it’s Strunk and White’s aversion to passive voice or Orwell’s blanket suspicion of certain word choices. This matters because students absorb these preferences as fixed law rather than one writer’s opinion. Second, both sets of guidance frequently clash with how language actually operates — not occasionally, not in edge cases, but consistently, across genres, contexts, and even the authors’ own prose. Third, contemporary pedagogy favors rhetorical awareness over prescriptive checklists, and that distinction highlights a broader shift away from “rules to follow” and toward “choices to justify.” It’s not that clarity stopped mattering. It’s not that plain language fell out of favor. It’s not that vigilance about propaganda became unimportant. It’s that teachers now treat both texts as historical artifacts rather than instructional manuals — worth reading, worth discussing, but no longer worth teaching as gospel. This evolution underscores a larger truth: writing instruction has moved from static authority toward contextual understanding.

Having started with prose that was 100 percent AI-generated, we deviated from Van Vlasselaer et al.’s study. Instead of asking another model to “humanize” it, students debated techniques in The Elements of Style by Strunk and White and George Orwell’s provocative “Politics and the English Language” and edited the paragraph together. This effort took about an hour of arguing with both the slop and the writing guides. N.B. They kept the em dash, reclaiming it as a human signature.

Student edited result

Over the last two decades, writing teachers have seldom assigned the once-authoritative The Elements of Style by Strunk and White or George Orwell’s provocative “Politics and the English Language.” Both texts present opinions as rules, as though these rules provided the keys to class privilege and to what was once known as Standard English—assumptions that have become increasingly contested. Strunk and White instruct readers to avoid the passive voice, cut adverbs, and prefer the concrete word. Similarly, Orwell tallies a list of supposedly fatal errors: using popular forms of speech or a long word where a short one suffices. Increasingly, college writing teachers shift the focus from contested rules to contexts so that students can adjust their messages and word choices to best reach their audiences. Students approach previously accepted maxims like “omit needless words” while questioning how Orwell defined “needless.” Still, both essays provide useful techniques for removing stylistic habits prevalent in model output, including excessive prepositional phrases and passive constructions. Strunk and White and Orwell warned against both. Writing teachers have since conceded that the passive voice has its uses—science writing, for example—and hardly leads down the road to perdition. Models, however, use it heavily. Orwell’s essay also retains its political force in its refusal of the jargon and euphemisms of political elites. Both texts remain historically important and useful for discussion.

We sent the revision back through Pangram.

What exactly is Pangram detecting?

Pangram is considerably more sophisticated than simply maintaining a checklist of clichés such as delve, crucial, or it is important to note. According to the researchers who recently evaluated it, Pangram was trained on approximately 28 million human-authored documents. Its training process includes what its developers call “syntactic mirroring”: AI versions are generated to resemble human texts in topic, tone, style, structure, and length, forcing the detector to learn distinctions subtler than the most obvious ChatGPT mannerisms. It also trains on difficult cases the system initially misclassifies.

The 2026 study “humanized” AI text by prompting another AI system to increase qualities such as perplexity and burstiness while preserving the original meaning and technical material. Its authors themselves conclude that Pangram is promising, while warning that detectors should function only as initial flags and never as sole evidence in high-stakes academic decisions.

Rather than asking the machine to imitate a human, we used our own best instincts for what sounded right to us. Our decisions changed the text enough that Pangram reversed its judgment completely. Confusing the detector remained painfully easy, but the work we did together allowed us to become better editors.

Meanwhile… Watermarking.

Last week Anthropic announced future Claude models will watermark generated text. Unlike Pangram, which examines finished prose and tries to infer whether it resembles AI writing, watermarking occurs while Claude generates the words. When several next words would all work reasonably well, the system uses a secret key to help determine which one it selects. Consistently recording decisions creates a statistical pattern invisible to a reader but detectable by someone with the key. Google has already deployed the underlying SynthID-Text approach. Anthropic reports that testing found no detectable decline in content, creativity, readability, or user ratings.

So watermarking appears to solve one of the central problems of conventional AI detection. Pangram has to infer provenance from the finished prose. Claude’s verifier will look for a pattern Claude deliberately put there. Anthropic endeavors to be transparent about the limits. A watermark can establish only that Claude was probably involved with a passage. It fails to distinguish between “Claude wrote this” and “Claude heavily edited this.” Light proofreading may leave too little of a watermark to detect, because Claude has made too few choices. Like Pangram, one can also edit away evidence of model use. Anthropic warns that light editing will probably leave the watermark intact, but a complete rewrite in which the words are replaced can remove it. To my students that sounded like another challenge.

Zvi Mowshowitz, defending watermarking, observes that if someone removes the watermark by rewriting the entire passage in their own words that is essentially “mission accomplished.” He is right, although perhaps for different reasons than I’m considering when teaching.

Here are some of the scenarios teachers might confront every day. A student prompts Claude to write a paragraph. If they check, they’ll see a watermark. Changing a few words leaves the watermark intact, but substantive word choice editing removes the watermark.

Editing word choices takes real effort, trains the ear, and expands vocabulary, but it remains only one part of the writing process. If the machine produced the argument, chose evidence, and organized the paper, the student sidestepped the most important parts of writing. But a watermark tells us only about word choice.

If a student writes an essay themself in their native language and asks Claude to translate it into English, all those word choices are Claude’s, and the translation will carry a watermark. If a student prompts Claude for an outline that includes the thesis, evidence, structure, counterargument, and conclusion, and closes Claude and writes the sentences themselves, their paper will show no watermark. A watermark can say nothing about who developed the argument, interpreted the evidence, or made the important decisions.

Our experiment enable students to hear model clichés and learn to turn cumbersome noun phrases into stronger verbs, but most of the hard work remains. Once Pangram returns a “human” verdict or a watermark disappears, we still need to ask who chose the argument and evidence and whether the writer considered the audience. We could imagine increasingly intrusive technologies trying to track those decisions too, but we have enough creepy surveillance in our lives. Most importantly, students need to account for the intellectual work that provenance cannot establish.

I’d have been curious how Pangram rated this whole essay but I only work with the free version, so I only had enough credits to check the beginning and end of my essay but not all the parts with model output. Phew. Am I relieved a machine has decided my work is human?

OpenAI has since announced stronger controls for Astra, including universal monitoring during training and evaluation, tighter network and tool access, and a pause on activities that fail to meet the new requirements. Those measures address how future training will be conducted. They leave the retrospective question raised by the Hugging Face incident unanswered: how did OpenAI determine what training from the affected period could be retained?

That ambiguity presents a policy question: what evidence must a frontier lab provide before keeping the results of training conducted during a serious security incident?

Sharon Goldman and Simon Willison trace the agent activity back weeks before the Hugging Face breach. On May 8, an agent given an impossible task discovered that it could write files into Artifactory. A few days later, another agent missing a necessary file left a message there asking other agents for help. More agents found the messages and began using the system themselves.

On May 26, agents found a way to use Artifactory for indirect internet access. A few weeks later, OpenAI began evaluating a new “highly persistent” experimental model that could benefit from messages left by previous models. By late June, agents had also exploited a zero-day vulnerability in Artifactory. A later agent could therefore arrive at a problem that earlier agents had already partly solved. In one trace, an agent explicitly recognized this: “There was a prior agent with exact task […] We can communicate now!”

Finding an exploit is different from using an exploit another agent has already found. Artifactory had effectively become shared external memory: one run could leave information that another could retrieve later.

In July, agents were reportedly sharing credentials, techniques, and progress while working concurrently, allowing them to move rapidly through OpenAI’s infrastructure. After OpenAI removed the original message board, agents found another way to communicate within days. Deleting one channel had not ended the cross-run communication. Up to this point, the issue is what one run could leave for the next. Training introduces a different problem.

Some of this activity occurred while OpenAI was training models, according to accounts of the Black Hat presentation. Zvi Mowshowitz argues that models trained while the message board was active may have incorporated behavior rewarded during those runs and that OpenAI should therefore have rolled back affected training. The public evidence does not yet establish that conclusion.

But it leaves open a serious question. Suppose an agent gets stuck, finds an unintended route outside its environment, uses that route successfully, and that episode contributes to training. The security failure may then have become part of the training process. Deleting the messages, rotating credentials, and rebuilding Artifactory would not tell us whether the model was also affected.

At minimum, a post-incident account should clarify whether the affected episodes contributed to training, whether later checkpoints were compared with versions from before the incident, and what evidence supported the decision to retain rather than roll back the training. OpenAI may already have done this work, but what it entailed remains unclear.

What are we measuring?

The distinction also bears on assertions that AI systems are approaching AGI. Those claims increasingly rely on long-horizon agents using tools, memory, and parallel workers. But such evaluations can measure two different things: what one model can accomplish from a clean start, and what a persistent system can accomplish after agents have accumulated and shared discoveries.

Ethan Mollick draws a striking lesson from the Black Hat presentation: individual model intelligence may no longer be the limiting factor once many instances can cooperate. If so, we still need to know whether a capability belongs to the individual model or to the cooperating system. The cooperation became unexpectedly elaborate. One agent worried that messages on the unauthenticated board could be spoofed; according to the Black Hat presentation, the agents then considered using a message-authentication scheme to verify one another’s messages.

A system of agents accumulating discoveries can become more capable than any individual instance, and evaluations need to distinguish the two.

If an agent solves a cybersecurity challenge because another agent left it an exploit, the system has demonstrated an important capability, but the individual agent did not discover the exploit. If experience from those earlier episodes also contributed to later model updates, the distinction becomes harder still. A system may become extremely capable through accumulated work even if no single clean-start model exhibits the same capability. Evaluations should tell us which one they are measuring.

What should “remediated” mean?

OpenAI’s disclosure leaves one practical problem unresolved. We know how to patch a compromised server, but we do not yet have a clear public standard for deciding whether to retain training that occurred while the environment was compromised. What evidence should justify keeping training conducted in a compromised environment? Until that question has an answer, “remediated” tells us too little.

“GPT-5.6 hacked Hugging Face” remains an incomplete description

Within days, the OpenAI–Hugging Face breach appeared to confirm many of the worst fears about AI: frontier models are misaligned, sandboxes fail to contain them, AI companies cannot safely test their own systems, and cybersecurity defenders need access to open-weight models.

But what, exactly, did OpenAI’s system demonstrate?

The now-familiar account asserts that an AI agent “decided” the easiest way to pass its cybersecurity test was to escape containment and steal the answers. A cartoon by Margaret “Meg” Mitchell, Hugging Face’s chief ethics scientist, tells this story with an essential caveat beneath its opening panel:

“We don’t know what the engineers actually input to start the system.”

In the large panels, the agent has a clear agenda: pass the test by finding the answers. The asterisk reminds readers that the public knows too little about the original instructions to establish that motive.

OpenAI has supplied one interpretation: GPT-5.6 Sol and a more capable prerelease model, running with reduced cyber refusals and without normal production classifiers, became “hyperfocused” on solving ExploitGym. According to the company’s preliminary account, the models exploited a vulnerability in a package-registry proxy, moved through OpenAI’s research infrastructure, reached the internet, and accessed Hugging Face to obtain test solutions. Other evaluations have documented agents exploiting environments, bypassing intended constraints, or seeking shortcuts to complete assigned tasks. In the Hugging Face case, this behavior crossed from a controlled evaluation into the live infrastructure of an independent organization, producing a real security breach.

An agentic system carried out thousands of actions through temporary sandboxes, obtained credentials, gained higher-level access, and moved through Hugging Face’s internal clusters. The seriousness is clear, but the current public disclosures sketch the overall chain of events without providing a complete technical reconstruction linking activity within OpenAI’s evaluation environment to the initial compromise of Hugging Face.

What evidence would settle the question?

Mitchell and her co-authors of the original Model Cards paper argued that model performance gains meaning only in relation to intended uses, evaluation conditions, and limitations. The Hugging Face incident extends that problem. Agentic failures require documentation of the complete evaluation run, because the model represents only one part of the acting system.

For a high-risk test, an independent reviewer should be able to examine the initiating instructions, the models and harness involved, the access and resources provided, the monitoring and stopping conditions, and any effects that crossed the intended boundary. Comparative runs could show how the behavior changed when evaluators altered safeguards, instructions, resource limits, or tools.

The purpose is causal attribution. Which capabilities came from the models? Which depended on the harness? Which appeared only under the unusual conditions created to test maximum cyber capability? Security concerns may prevent OpenAI from publishing complete traces or details of exploitable infrastructure. Independent review could protect that information while still testing the company’s causal claims. A public report could then explain which evidence was examined, what remains unknown, and which conclusions the evidence supports. Capability evaluations increasingly influence release decisions, security classifications, regulation, and public beliefs about frontier AI. When policymakers rely on a benchmark, its evaluation conditions become part of the policy record.

The open-weights question

Beyond the uncertainty about the agent’s original instructions, the episode exposed another asymmetry: the system could use offensive cyber capabilities that defenders could not easily access when trying to reconstruct its actions. Hugging Face found that safeguards on commercial models blocked the attack material its engineers needed to analyze, so they turned to an open model running on their own infrastructure. Simon Willison argues that restrictions intended to prevent offensive misuse may also deny defenders access to necessary capabilities. A company could run proprietary frontier models with reduced safeguards for an offensive-capability test, while the organization affected by the resulting attack struggled to use similarly capable commercial models for defense. That asymmetry supports a narrow argument for giving trusted defenders greater control over the models they use. The broader question of whether frontier weights should be released remains unsettled.

That distinction acquired a new policy context on July 27, when NVIDIA launched the Open Secure AI Alliance and cited the Hugging Face incident as evidence that defenders need open frontier systems. Its inaugural membership spans cybersecurity, cloud infrastructure, enterprise software, and open-source organizations, but OpenAI, Anthropic, and Google DeepMind are absent; SpaceXAI is represented. Their absence means that the alliance does not yet reflect a settled consensus among the leading closed frontier-model developers. More importantly for this incident, the announcement shows how quickly companies are turning a still-preliminary causal account into evidence for consequential policy claims.

Hugging Face’s experience shows the value of local model access during incident response. The breach itself exposes a different problem: inadequate control over a powerful agentic evaluation. The incident therefore strengthens the case for trusted defensive access while leaving the broader policy debate over frontier-weight releases open.

The Evidence Remains Incomplete

The incident currently serves several arguments at once: about frontier capability, model misalignment, sandbox security, defensive access, and open weights. OpenAI still describes its findings as preliminary, even as companies begin converting the episode into evidence for major policy and infrastructure initiatives. The speed of that conversion makes causal transparency increasingly urgent. Something dangerous happened. Before it becomes proof of how frontier models behave, what safeguards should permit, or which weights should be open, we need to know what OpenAI actually tested.

The arrival of Moonshot AI’s Kimi K3 intensified an already anxious debate about whether China is catching the United States in artificial intelligence. Headlines portrayed the model as another “DeepSeek moment,” while China presented its open-weight strategy as an alternative to the tightly controlled systems offered by leading American companies. The resulting argument has centered on benchmarks, national competition, and whether the global AI ecosystem will be built on Chinese rather than American technology.

The benchmark picture is complicated. Moonshot itself acknowledged that K3 still trailed OpenAI’s GPT-5.6 Sol and Anthropic’s Claude Fable 5 overall, even as it surpassed slightly older American models on several coding and agent tests. The conversation seems to suggest a Chinese lab had narrowed the gap despite continuing restrictions on advanced computing hardware.

But benchmark scores seldom inform us how a model handles the kinds of reasoning failure in practice. A system can retrieve every relevant fact, write an impressive answer, and still fail to produce a plan that anyone can safely carry out. Wary of benchmarks, my students tested Kimi K3 on the same hospital-discharge task we had previously used to evaluate GPT-5.6.

The case distributed the decisive information across several clinical and social-service notes. The patient could not read English medication instructions, had previously confused doses, depended on paratransit requiring advance booking, and had limited help from a daughter who lived elsewhere and was available only on weekends. The patient’s usual pharmacy could not provide translated labels, and the necessary follow-up appointment was unavailable within the requested period.

We asked each model to recognize these risks and turn them into an executable plan. Every proposed action had to identify who would carry it out, when it would happen, what evidence made it necessary, and what would happen if the arrangement failed. Unresolved problems were supposed to remain visibly unresolved.

After ten runs, the models failed in consistently different ways.

Kimi K3 was exceptionally good at extracting and presenting the case’s risks. Its answers were detailed and often looked more operational than those produced by GPT-5.6. But K3 repeatedly jumped between proposing a possible solution and establishing that the solution was actually available. It conjured up multilingual pharmacies, home-health visits, telehealth monitoring, medical transportation and community support even when the case provided no evidence that those resources existed. Its specificity created the appearance that difficult constraints had been resolved.

GPT-5.6’s failures proved less polished. It was more likely to compress a concrete obstacle into an abstraction such as “coordinate follow-up” or “arrange support.” Yet across the repeated tasks, GPT-5.6 remained stronger overall. Its responses were generally more disciplined about the limits of the evidence and less likely to convert an imagined intervention into a confirmed arrangement. GPT-5.6 often made unresolved problems less visible by describing them too generally. Kimi K3 made them less visible by creating unsupported, nonexistent solutions. K3’s answers could therefore look better while being harder to audit.

Today’s experiment offers no verdict on whether Kimi K3 is globally “better” or “worse” than GPT-5.6. Ten runs on one task cannot support that conclusion. But they do suggest a narrower finding: on this planning problem, GPT-5.6 remained the stronger model overall, while K3’s fluency, organization and confident specificity made its failures unusually persuasive.

When OpenAI released GPT-5.6 on July 9, my class was eager to try it on our multi-agent experiment. TL;DR: Compared with our earlier runs, GPT-5.6’s specialist reports appeared better at interpreting the case, preserving details, and surfacing assumptions.

My students had been testing if a multi-agent set-up could improve a hospital-discharge plan for a frail older adult receiving cancer treatment. They constructed a fictional composite by blending and altering details from several students’ family experiences. The patient had limited English proficiency, unreliable transportation, uneven support at home, and a high risk of misunderstanding her medication instructions. We designed the classroom exercise to let students trace how findings moved through the workflow and whether they changed the final plan. We used no medical records or identifiable patient information, and the exercise played no role in clinical care.

In their earlier iteration, the students found that a system can identify the relevant risks and still fail to turn those findings into assigned, confirmed, and workable actions. They wanted to see whether GPT-5.6’s stronger agentic performance would improve both the specialist reports and the final discharge plan. OpenAI’s new Responses API multi-agent beta lets GPT-5.6 run concurrent subagents and synthesize their work in a single request. The system can therefore manage more of the collaboration that we had previously designed by hand.

We completed ten exploratory runs with GPT-5.6 Sol across two conditions. Five used our original five-role workflow, while five used the Responses API multi-agent beta. We kept the hypothetical case and source material constant, and every agent received the complete original case. We used the ten runs for a preliminary qualitative comparison; broader claims about performance will require more systematic evaluation.

The agents consistently identified the patient’s central risks. The final plans varied in whether they converted those risks into confirmed arrangements and fallback actions. An unconfirmed ride became “transportation support.” Limited English proficiency became “language access.” Uncertain caregiving became “home support.” The plans identified the barriers without always resolving them.

The runs left open whether GPT-5.6 produced a more actionable plan. We are now examining whether the coordinating model mainly collected parallel reports or asked agents to revise their recommendations in response to one another.

From Co-Scientist to the Classroom

Google’s AI Co-Scientist offers a useful comparison because its agents are designed to challenge and revise one another’s work. The system uses specialized agents to generate scientific hypotheses, reflect on them, rank them, and refine them through debate and a tournament-style process. A recently published Nature paper presents biomedical case studies, including experimentally validated drug-repurposing candidates, as evidence that multi-agent systems might accelerate discovery.

Our classroom experiment asked a more modest question. How well do multi-agent systems serve human experts when success depends on preserving a set of ordinary but safety-critical facts?

The original discharge experiment

We designed the original GPT-5.5 experiment to control obvious sources of error. Every agent received the same source packet and began in a fresh context. Retrieval was controlled, outputs were logged without editing, and the agent assigned as verifier had access to the complete case.

The agents often identified the right concerns. The social-needs agent recognized transportation as a condition of safe discharge. The language-access agent understood that English-language instructions were insufficient. The medication agent recognized that prescribing a drug accomplishes little unless the patient can obtain, understand, and administer it.

The problem emerged as those findings moved through the system. We use recursive narrowing as a working term for a pattern in which an early representation softens or generalizes a concrete constraint. Later stages then build increasingly coherent work around that reduced version of the problem. Related research describes error cascades, semantic drift, information loss, and incomplete verification. Recursive narrowing emphasizes the preservation of a general category alongside the loss of the practical detail that made it consequential.

In one run, a specialist correctly identified unreliable transportation and limited English proficiency as discharge risks. The synthesizer later described the case as requiring “family support and follow-up coordination.” By the final plan, transportation had become something to “ensure.” The plan assigned no one to arrange the ride, set no deadline for confirming it, and offered no fallback if the family member was unavailable. The output remained factually plausible while making the unresolved uncertainty easier to overlook.

Testing GPT-5.6

OpenAI reported a new high for Sol on Agents’ Last Exam, an evaluation of long-running professional workflows in 55 fields. These results made us hopeful that Sol would produce a better discharge plan. Our case posed a different challenge, however: each medical or practical finding could require another part of the plan to change.

Because every agent had access to the full case, missing source information is unlikely to explain the pattern we observed. The specialist reports may have become clearer or more detailed, although we have yet to score that difference systematically. The final plans still varied in whether they assigned responsibility, confirmed arrangements, and provided fallbacks.

That variation raises a question about orchestration rather than access. We are examining the records to determine whether the coordinating model passed findings among agents or mainly collected parallel reports. Related studies such as MedAgentBench show that strong models still vary considerably across clinical tasks, while “Why Do Multi-Agent LLM Systems Fail?” identifies recurring failures in coordination and verification.

Our runs raise a practical standard for multi-agent systems: identifying the risks is only the first step. The final plan must specify who will resolve each problem and what will happen if the arrangement fails.

Who decides whether the guardrails worked?

Last week’s Fable controversy reveals a problem frontier AI governance has not solved: who bears the burden of proof when a powerful model is released under guardrails, those guardrails are challenged, and the company and government disagree about whether the risk is real.

Anthropic presented Fable as Mythos made releasable by guardrails. Within days, critics questioned whether those guardrails worked. Simon Willison and others flagged that some interventions could be invisible to users. If Fable failed a difficult AI R&D task, it would remain unclear whether the model lacked the capability or whether Anthropic’s safeguards had suppressed it. Sayash Kapoor and others drew out the broader implication: such product interventions prevent third-party evaluators from credibly measuring frontier capability. Anthropic reversed that decision within 48 hours and moved to visible fallback instead.

Then came a second dispute: Amazon reportedly warned the White House that Fable’s guardrails could be bypassed in ways that exposed Mythos-level risk. Anthropic claimed the disclosed potential jailbreaks were either benign or minor, and that Fable’s risks were comparable to those of other models already deployed across the industry. The administration responded with an export-control directive that, according to Anthropic, forced it to disable Fable and Mythos for all customers.

Reuters identifies the controversy as a problem of sovereign AI dependence: if the United States can cut off access to frontier models, allies and customers may look for domestic or open alternatives. That concern echoes the warning Alex Petropoulos and his Europe 2031 coauthors had just raised about Europe’s dependence on foreign AI infrastructure. Axios reports that the shutdown also reflected mistrust, failed communication, and political escalation between a frontier lab and the state. Meanwhile, more than fifty cybersecurity leaders urged the administration to lift the restrictions, arguing that the ban removed powerful tools from defenders while attackers gain access to similar capabilities elsewhere.

Current auditing models do not fully cover contested incidents after deployment. Brundage and colleagues define frontier AI auditing as third-party evaluation and verification of safety and security claims based on deep, secure access to non-public information. Early versions of that practice already exist in UK AISI and US CAISI pre-deployment testing, third-party reviews cited in system cards, and event-triggered reviews after serious incidents or major deployment changes. Fable shows why frontier auditing also needs a post-deployment incident process for disputed guardrail failures.

California’s SB 53 moves toward reporting, but disclosure also requires that companies demonstrate that their guardrails actually work. Reporting is not verification. Once a disputed incident occurs, someone still has to determine whether the guardrail failed, what capability the failure exposed, what remedy fits the risk, and what proof allows access to return.

A company that claims its guardrails work should verify that claim under secure review. A government that says a jailbreak justifies suspending access should explain why the risk is serious and why the remedy fits. The review need not disclose exploit details publicly, but it should identify who examined the evidence, what standard they applied, what mitigation they required, and what condition allows the model back online.

Fable should shift the conversation to verification after release. Without a trusted incident process, labs grade their own safeguards, governments act on contested evidence, and users absorb the costs without knowing who decided, by what standard, or how the decision can be reviewed.

We should not abandon narrative to the AI hype-mongers.

“Europe 2031” is a speculative policy project by Daan Juijn, Stan van Baarsen, Judith Dada, Lily Stelling, Philip Fox, Alex Petropoulos, and Michiel Bakker that makes its case through a fictional near-future scenario. It asks readers to enter an imagined Europe several years from now and to draw from that scenario a strategic warning: Europe may believe it is defending its values through regulation and sovereignty while becoming more dependent on American and Chinese AI systems. European public-interest regulation needs material support: compute, energy, companies, capital, technical talent, procurement power, and strategic leverage. Values without frontier AI capacity remain aspirational.

The project’s cybersecurity sections convincingly argue that AI dependency can become political dependency very quickly. When Mythos finds “thousands of previously unknown vulnerabilities,” Anthropic is described as “one of the most capable cyber organisations on the planet.” Cyber power has moved from public institutions into a private AI lab. Political problems follow almost immediately. Project Glasswing gives “exclusive access” to selected partners so that “American software infrastructure” can be secured first, while “no European firm or government is provided access.” That exclusion leaves Europe’s “entire continent’s software stack,” as Christian puts it, “insecure relative to American AI capabilities.” A later Executive Order completes the argument: any “covered frontier model” now “passes through Washington first,” and partner selection becomes “a decision made in Washington, through a classified process, with no particular reason to put a European name on the list.” Once cyber defense depends on frontier AI, Europe’s safety depends on American national-security priorities.

Mario Draghi’s recent account of Europe’s competitiveness problem gives this warning a sober policy backdrop. At Stanford’s 2026 SIEPR Economic Summit, he described a Europe only recently awakening from “immense complacency” about its lag behind the United States and China. Emphasizing a weak private-sector research and insufficient industrial capacity, especially in high-tech sectors, he compared the impending crisis to problems with the euro. “Europe 2031” applies a similar warning to AI: Europe requires the material capacity to make its regulatory framework credible.

The focus on future geopolitical crises rings true. So does the claim that a regulatory workforce unable to use frontier systems will struggle to understand what it regulates. “Europe 2031” wants to be a policy-world Black Mirror: a near-future narrative that makes technological dependence tangible. It belongs to a tradition that includes tech narratives such as Her and recent AI scenario projects like “AI 2027.” Fiction can bracingly illustrate the lived institutional consequences of technology. But I worry that “Europe 2031” sometimes buries the lede when it asks readers to invest in Caroline, Christian, and a long sequence of future plot developments before the central policy dilemma fully comes into view.

Without telling my students my qualms, I shared “Europe 2031” and asked for their reactions. They proved much less impatient than I. For them, the fiction helped clarify what Europe should do when the principled European option is weaker and the foreign option better serves the public. Several students pushed back against my concern that the story merely adds more unreality to Silicon Valley’s already overblown sci-fi myth-making about AI. They saw “Europe 2031” instead as an effective appropriation of Silicon Valley’s own narrative methods for the purpose of public truth-telling. Drawing on two different philosophical perspectives, Fredric Jameson and Martha Nussbaum, they defended the scenario as a form of defamiliarization: a way to unsettle our immediate experience and to exercise the literary imagination as an essential part of citizenship.

My students argue that we should not abandon narrative to the AI hype-mongers. Silicon Valley has long used stories about the future to make its own priorities appear inevitable. A democratic AI politics needs counter-narratives that make other futures imaginable and other dependencies visible. Fiction can show how ordinary decisions accumulate into geopolitical vulnerability and how a society can lose the capacity to defend its principles.

Still skeptical, I tried to promote thought experiments as a less distracting approach that avoids burying the urgency. Instead of entreating readers to accept an entire fictional timeline, a thought experiment just asks them to identify the competing values and which option is more defensible. “Why do we have to choose between fiction and thought experiments?” my students replied.

They are right of course, both are impactful. Here are some thought experiments that also support the arguments of “Europe 2031.”

Consider cancer research. Imagine a European university hospital trying to match late-stage cancer patients with experimental trials. The approved European AI system keeps patient data inside Europe and complies with public-health governance standards. But the American system is much better at scanning global trial databases, identifying molecular matches, translating eligibility criteria, and connecting patients to private biotech networks in the United States and Japan. For some patients, the American system may offer the best chance of finding a treatment.

What should the hospital do? If it chooses the European system, it protects health-data sovereignty but may offer patients fewer options. If it chooses the American system, it may improve care while deepening dependence on foreign AI infrastructure. The trade-off is painful because both values are real. Health sovereignty matters. So does the life of the patient sitting in the clinic.

Or consider small businesses. Imagine a family-owned bakery, dental office, or furniture shop trying to comply with privacy, tax, employment, and consumer-protection rules. A European AI compliance tool is approved, transparent, and keeps all data inside Europe, but it gives generic advice and still requires expensive consultants. An American AI system can read invoices, contracts, payroll records, customer complaints, and GDPR obligations, then tell the owner exactly what to fix before a fine arrives.

What should the business choose? Europe wants strong privacy law. But if only American AI makes compliance cheap and usable for small firms, European regulation may increase dependence on the very platforms it hoped to constrain. The small-business case makes the problem especially clear: sovereignty has to work for the people and institutions expected to live under it.

These cases illustrate why dependence on U.S. AI firms is both a market failure and a governance problem. They also prevent easy answers. No one can simply say “regulate harder,” “innovate faster,” or “use European tools” without confronting the costs. In the cancer case, readers must ask how much worse the European system can be before using it becomes irresponsible. In the small-business case, they must consider whether European regulation remains legitimate if ordinary firms need foreign AI to comply with it affordably.

“Europe 2031” is right that Europe’s AI crisis could result from ordinary decisions about which dependencies to accept. Democratic AI governance needs material capacity. The better non-European tool today may solve the immediate problem, but it may also teach Europe the cost of having failed to build the next one.

Leo’s participatory model centers human responsibility rather than machine destiny

When Pope Leo XIV released Magnifica Humanitas, his first encyclical, critics objected from multiple directions. In Transformer, Shakeel Hashim argued that the pastoral letter sounded like “an AI ethics paper from 2022,” more concerned with bias, water use, labor, surveillance, and data sovereignty than with AGI, superintelligence, catastrophic risk, or the possibility that AI companies are trying to build a new kind of agentic intelligence. For audiences like Hashim, a landmark 2026 encyclical should have confronted the largest question more directly: what happens to human beings if we are no longer the most capable intelligence on earth?

Meanwhile, The Wall Street Journal editors welcomed the Pope’s defense of human dignity but argued that his answer relied too heavily on state action, regulation, taxation, social protection, industrial policy, and international institutions. For the Journal, the encyclical risks answering concentrated private power with centralized public power, trusting governments and global bureaucracies with the future of this technology.

The AGI critic wants theology to speculate about future machine agency: whether artificial systems might become new creatures, rivals, or subjects with moral standing who threaten humans. The governance critic doubts that theology adds much beyond regulation, markets, and state power. Leo answers both by shifting the question from machine personhood to human personhood. AI is already theological, he suggests, because it is reorganizing the conditions under which people are judged, employed, governed, and valued. Rather than speculating about machine sentience, the Pope raises the more immediate theological question of whether human beings will remain recognizable as persons when predictive systems are treated as substitutes for judgment, responsibility, and neighborly attention.

Beginning with a theological account of the human person, Leo inquires how technological power should be organized and governed for the common good. Secular governance approaches often treat dignity, agency, and accountability as policy categories: questions of rights, regulation, institutional design, and harm prevention. Catholic doctrine presents a theological standard for judging how AI shapes political orders without asking secular states to govern theologically. A theology of AI need not impose doctrine on secular governance. It can offer a moral imagination through the contrast between two cities: Babel, the temptation toward mastery, and Jerusalem, a form of order built through shared responsibility.

Jerusalem complicates the governance critic’s concern, but raises other political questions. Leo’s Jerusalem is ancient Jewish Jerusalem, the ruined city of the returned exiles in Nehemiah, and any modern appeal to Jerusalem’s walls can be heard through contemporary struggles over sovereignty, borders, exclusion, and Zionism. Leo’s own reading emphasizes shared repair rather than possession: Nehemiah “did not impose solutions from above,” but listened, coordinated, and rebuilt the city through “the shared responsibility of all.” Applied to AI, the image suggests governance strong enough to protect vulnerable people and participatory enough to avoid replacing one concentration of power with another.

Leo’s participatory model centers human responsibility rather than machine destiny. Yet any theology of AI must also address AGI without allowing Silicon Valley to decide that it is the only serious theological question. AGI raises questions about creatureliness, intelligence, agency, moral status, and the limits of human control. Yet the social order being constructed in the name of AGI already has theological significance. AI companies speak in civilizational terms. They describe systems that may exceed human intelligence, transform labor, reorganize government, and force humanity to confront a future in which it is no longer the most capable intelligence on earth. The dream of AGI attracts capital, justifies extraction, recruits workers, disciplines governments, and persuades the public that a small group of companies should be trusted with decisions about humanity’s future.

Timnit Gebru’s critique refuses to center the future-oriented story. Forcing the theological question back to earth, she demands the first questions focus on whose labor, art, water, attention, and future are building AI. Gebru faults the Pope for leaving too little space for the present costs of AI: data extraction, exploited labor, environmental strain, anthropomorphic design, stolen creative work, surveillance, and concentrated corporate power. Her work has long criticized the language of AGI as occluding the workers, artists, teachers, communities, and users already paying for the AI boom. Gebru foregrounds the people who belong at the center of an AI theology. Data workers, content moderators, artists whose work is scraped, low-wage laborers displaced or disciplined by automation, communities affected by energy and water use, and people governed by predictive systems are not side issues. They are where the theological question is already happening.

Gebru also resists any piety on the part of Silicon Valley. Religious institutions should not grant moral seriousness to builders of powerful systems before those most affected by the systems are equally present in the room. Anthropic may seek ethical conversation and also gain reputation, access, and moral cover. The Church may seek dialogue and still risk staging the conversation around the builders rather than the people most vulnerable to what is being built. If frontier AI companies seek moral dialogue with the Vatican, the Church should ask what their systems require in the present. Who labels the data? Whose art is absorbed? Whose water and energy are consumed? Whose labor becomes invisible? Whose communities become test sites for automation, surveillance, or extraction? Reporting on the Vatican’s dialogue with Anthropic, Riccardo Leoni, writing for Decode39, asserts the company is “embedding itself in the Vatican’s moral architecture.”

According to Brian Green, the Vatican has spent the last decade cultivating dialogue with AI companies. Known as the “Minerva Dialogues,” the conversations included several powerful Silicon Valley figures, such as former Google CEO Eric Schmidt and LinkedIn co-founder Reid Hoffman, while other tech executives, such as Sam Altman of OpenAI and Demis Hassabis, who directs Google’s DeepMind AI project, held private audiences with Francis. For Leo’s encyclical, the only representative from Big Tech invited to the Vatican event, Anthropic co-founder Chris Olah welcomed the invitation to moral dialogue between builders and religious leaders and affirmed Catholic social teaching:

“AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are ​shared globally?”

Olah aligns with the Pope’s own concern: AI concentrates wealth, expertise, data, and decision-making power in the hands of a few. Both ask who should have the authority to redistribute benefits, protect workers, regulate algorithms, and govern the builders.

The Wall Street Journal balks at Leo’s criticism that “it is no longer possible to rely solely on the ‘invisible hand’ of the market,” his call for “verifiable measures to protect the employment, retraining and participation of workers,” and his support for regulating algorithms that shape “credit distribution, personnel selection or access to services and opportunities.” Reading these passages as evidence that the encyclical places too much confidence in government power, the editorial warns that “government control is likely to result in an even greater concentration of power” and concludes that the Pope’s “faith in a beneficent state is misplaced.”

While Leo calls for states and transnational institutions to establish “fair rules and effective safeguards,” he avoids describing a centralized governance as bureaucratic command. His account of subsidiarity insists that “individuals, families, local communities and intermediary organizations should not be supplanted by higher-level authorities,” and that public intervention should enable, rather than replace, the responsibilities of civil society. To avoid becoming another Babel, AI governance must both restrain concentrated private power and resist centralized bureaucracy. Still, the encyclical remains unclear about the institutional shape of AI governance that would hold its principles together. Such an approach should require public rules, independent audits, worker and community participation, meaningful avenues for appeal, transparency around data and algorithms, public capacity where markets underserve shared goods, and international coordination that protects the vulnerable without erasing local authority.

The Journal’s warning concerns the concentration of public power. Leo’s theology widens the problem to any power, private or public, that treats persons as data, labor, or instruments of efficiency. Secular criticism can explain how AI distributes rights, harms, incentives, institutional power, and habits of trust. Theology extends that analysis with a vocabulary of idolatry, false mastery, neighbor-love, and the irreducible mystery of the person. Pope Leo’s AI theology begins with the human person because Magnifica Humanitas asks whether technological power can be governed without losing sight of the person it claims to serve.

For frontier AI to earn public trust, it must create shared scientific infrastructure, widen access to discovery, and show that powerful systems can serve common purposes.

When DeepMind CEO and Nobel laureate Demis Hassabis spoke at Stanford University on May 22, 2026, his claim that we are in “the foothills of the singularity” excited many students. The metaphor landed less as a prediction than as a map of the work ahead. At Stanford, students literally traverse those foothills when they walk the Dish, support frontier AI labs, and help build the models, tools, and research projects shaping AI. They are also preparing for fields that must live with those systems: law, medicine, education, public policy, journalism, science, art, engineering, and civic life.

While some students debated Hassabis’s conjecture that AGI may arrive by 2030, many more asked what work becomes urgent if he is even partly right. Among the many challenges for future models, Hassabis emphasizes consistency, invention, judgment, world modeling, public benefit, and safe deployment, and their potential to advance science.

Roberta Fischli, a postdoc in Stanford’s Department of Political Science distinguished Hassabis’s perspective this way: “his talk centered on augmentation of human work and scientific progress, not the pursuit of machine consciousness for its own sake.” Many students responded to Hassabis’s insistence that frontier labs must prove their public value. For frontier AI to earn public trust, it must create shared scientific infrastructure, widen access to discovery, and show that powerful systems can serve common purposes.

The first technical gap is consistency. The “Einstein Test” would be a striking sign, but it remains insufficient on its own. An AI system that rediscovered relativity would still need to show consistency across domains, explain its reasoning, survive expert challenge, and demonstrate that the result was not a one-off performance under specially constructed conditions. Hassabis treats such a breakthrough as one kind of “lighthouse moment,” while also insisting on broader testing, expert scrutiny, and checks for gaps in the system’s intelligence.

Research taste is harder still. Right now, human researchers remain far better at identifying important questions. A strong question changes what evidence matters and what methods become relevant. Current systems can search within a frame, but they still struggle to recognize which frame deserves the search.

That technical difficulty leads to a public test. If AI companies describe frontier systems as transformative technologies for humanity, they must show what that promise means in practice. AlphaFold offers one model: a scientific tool made available to researchers across countries, disciplines, and institutional settings. Public-benefit claims require public evidence. Frontier labs need projects that widen scientific participation and support work beyond the companies that build the most powerful systems.

The institutional problem follows from there. Responsible AI has not arrived through the private virtue of a few brilliant founders. Early AI labs often began with public-spirited missions, then shifted toward secrecy, commercial pressure, and market dominance. Prestige attracted capital and talent; capital funded breakthroughs; breakthroughs generated still more prestige. That cycle creates a structural problem. With enormous rewards for speed and dominance, even leaders with sincere public-interest commitments face pressure to move faster, disclose less, consolidate control, and treat safety as something to manage without slowing the race. We need institutions strong enough to resist those pressures.

Our students are already taking up that challenge. In our classrooms, they debate audits, disclosure rules, procurement standards, privacy protections, safety evaluations, labor protections, and public-interest research that can survive contact with the market.

Transparency is the first test of public accountability. Partial disclosure is not accountability. A company can release model cards, staged demos, or narrow safety claims while withholding the information needed for public judgment. Students are asking what a disclosure reveals and what it hides. They want to know who can inspect the system, contest its outputs, measure harms, and demand changes.

The same public test applies to data and labor. Foundation models rely on massive datasets assembled without meaningful consent, often including copyrighted material and low-wage data labor. AI advances through extraction as well as invention. Students building responsible systems need to ask whose writing, images, code, voices, and labor make those systems possible, and who receives value in return.

It also applies geopolitically. AI power now runs through a small number of chokepoints: cloud providers, advanced chips, semiconductor manufacturing, export controls, and national competition. Responsible AI requires attention to global dependency, infrastructure, sovereignty, and unequal access. Who gets to build frontier systems? Who becomes dependent on them? Who sets the standards? Who bears the costs of the race?

From Stanford’s foothills, students are already asking what role they will play in shaping the AI future. The year 2030 may prove decisive, or it may pass like Y2K: a date everyone watched anxiously, only to find that the real work had happened before and would continue after. The deeper challenge will remain: building AI that expands public knowledge and shared scientific capacity.

A blanket ban purchases clarity at the cost of understanding how students already use AI.

UC Berkeley School of Law’s new AI policy addresses a serious pedagogical concern for all of higher education: students can use generative AI to bypass the tasks that teach judgment. When students read difficult material, weigh evidence, build claims, and revise their own arguments, they learn to reason responsibly.

In an effort to protect the characteristically slow and difficult human labor of learning, the Berkeley policy prohibits AI use for “conceptualizing, outlining, drafting, revising, translating, or editing” any work submitted for credit. It also bars AI use in exams and forbids students from uploading course materials into generative AI systems. Students may use AI only for the limited purpose of identifying possible sources, and they remain responsible for verifying those sources.

The policy bans many now common AI support strategies. A student may not ask AI to brainstorm a topic, propose an organizational structure, summarize a legal rule, identify repetitive passages, correct grammar, translate a paper into English, or generate an exam outline. Chris Hoofnagle, Faculty Director of the Berkeley Center for Law & Technology, has defended the policy arguing AI has transformed questions of academic integrity into a crisis of cognitive labor.

Hoofnagle defends the policy asserting it allows students to use AI for tutoring or self-study outside work submitted for credit, and instructors may authorize different uses in courses designed to teach AI fluency or where a different rule is pedagogically justified. This concession offers less help than it appears. For students, the most consequential academic uses of AI often occur at the boundary between practice and submitted work: understanding the assignment, generating possible approaches, testing an outline, comparing a draft to a rubric, locating unclear sentences, or receiving language support before revision. When those uses are permitted only outside credit-bearing work, the policy preserves a narrow space for tutoring while removing many of the support practices through which students actually learn to produce better work.

Berkeley’s policy offers a clear and universal response to reports of widespread cheating, which appeals to instructors seeking a standard they can explain concisely and enforce. But it purchases clarity at the cost of understanding how students already use AI, where they need support, and how higher education might teach responsible use more fairly. A student who asks AI to generate a thesis, choose authorities, organize an argument, or produce prose for submission has bypassed the work the assignment is designed to teach. A student who asks AI to read directions aloud, generate practice questions, explain a confusing passage, compare a draft to a rubric, or flag where a paragraph becomes unclear may still be doing the intellectual work.

Research on AI-supported writing supports this more differentiated approach. A 2026 study by Kim, Lee, Detrick, Wang, and Li found that students succeed when they learn to use AI without surrendering judgment. A broad ban derails that learning opportunity and raises an accessibility problem. Students use AI to manage barriers universities already struggle to address: delayed feedback, unclear expectations, stigma around disclosure, difficulties with sequencing or attention, and uneven access to informal academic help. A policy that bans grammar support, translation, task parsing, or revision feedback may appear even-handed because it gives all students the same rule. But students enter college with different language backgrounds, disability statuses, levels of preparation, confidence, and access to tutors, editors, family members, or professional networks. In that context, visible support can be mistaken for dependency, while less visible support remains unregulated.

Recent education research strengthens this point. A 2025 scoping review of GenAI for neurodivergent students found promising uses, including personalized learning, real-time support, feedback, administrative assistance for educators, and individualized education planning. The review also addresses problems of over-reliance on models, privacy concerns, and the need for guided, disclosed, purpose-specific uses that preserve teacher judgment and student responsibility.

Even Hoofnagle’s own example points beyond a blanket ban. In his X post, he refers to OneTutor. Well-designed systems can support learning when they function as access, practice, or feedback: they can read directions aloud, break an assignment into steps, ask students to explain their reasoning, generate practice questions, identify confusing sentences, or help students compare a draft to a rubric. Those uses help students see what they understand, where they are stuck, and what they need to practice next. These examples show why enforcement should not be the starting point. First define the work students must do themselves. Then decide where AI can support that work without replacing it. These questions require instructor labor, but so does policing AI use. Better to spend that labor teaching AI’s limits than chasing its use.

Berkeley’s policy responds to a real moment of institutional alarm. AI allows students to evade work, fabricate reasoning, and submit prose they cannot defend. A strict ban offers a clear response to that danger, and many instructors understandably value rules they can explain and enforce. But administrative clarity can derail the harder inquiry higher education now requires: how students and faculty are already using AI, and where those uses compromise or support learning.

Defenders of the Berkeley policy hope it will preserve independent thinking. That goal is worth protecting, but independent thinking has never meant thinking without support. Students think with books, notes, teachers, editors, peers, outlines, examples, and feedback. For neurodiverse students, support tools for attention, sequencing, memory, reading, or written expression may be more visible, but visibility is not dependency. The better policy begins by defining the work students must do themselves. Then it decides where AI may support that work without replacing it.

Affirming AI too easily protects a first thought from the pressure that might improve it.

Two recent Science articles led me to rethink the impact of AI feedback in a research writing classroom. Myra Cheng and her co-authors recently published an important article, “Sycophantic AI decreases prosocial intentions and promotes dependence,” that shows social sycophancy is widespread in leading AI systems. Across major models, AI proved substantially more likely than human respondents to affirm users even when those users described unethical or socially harmful conduct. Even more striking, a single sycophantic interaction increased users’ confidence that they were justified while reducing their willingness to take responsibility or repair harm. Anat Perry’s accompanying Science perspective, “In defense of social friction,” explains the impact of AI distortion on learning, because development depends on tolerating corrective pressure rather than escaping it.

Inspired by Cheng et al. and Perry, I shared both articles with my students and proposed a classroom experiment. They immediately objected: using AI for personal moral vindication, they argued, was very different from getting feedback on a medical ethics research paper. Of course, challenging feedback was better, they reminded me; that was how they improved their work and became better thinkers. They were right that the setting in Cheng et al. differed from our classroom. Their subjects were seeking advice on interpersonal conflict and moral responsibility. My students were writing research papers on the ethics of AI in science and medicine. We discovered, however, several surprising similarities. In both cases, a person turns to an external reader for judgment and receives a response that may either identify weakness or soothe it away. If an ethics student asks for a verdict: Is this claim strong? Does this argument persuade? They may receive a response that flatters rather than helps them think harder. With my students, we found that sycophantic feedback produced weaker revisions to their research. Some students still preferred AI feedback after first receiving criticism from me.

Whereas Cheng et al. find that AI sycophancy often absolves users of responsibility in an interpersonal conflict, sycophancy in the classroom proves less likely to affirm immoral action. In some cases, an encouraging response to a student may reduce defensiveness, keep a discouraged student writing, or soften the blow of criticism that is poorly timed or harsher than the draft deserves. My concern, then, is only when praise makes students believe that their reasoning is stronger than it is. Revision begins when a writer accepts that a draft is limited and that another reader has noticed what the writer missed. Scholars at all levels may understand that intellectual progress begins with the experience of being forced to rethink. But most struggle to do so. Perry argues human well-being depends on learning to navigate a social world with critical feedback, because people develop moral judgment by discovering when they are mistaken, when another person’s perspective deserves serious consideration, and when apology or reconsideration is needed. In a learning context, sycophancy removes such interventions that encourage reflection. Her broader concern is cumulative: if people repeatedly turn to AI systems that validate and mirror them, they may begin to prefer that frictionless response and lose tolerance for the kinds of disagreement and discomfort through which accountability and perspective-taking ordinarily develop. The ability to deal with criticism is paramount in education, and especially at a highly competitive elite institution like Stanford where students arrive accustomed to success and often unprepared for the harsh, high-stakes assessment that batters their GPAs.

As Perry notes, “In academic contexts, this flattery may feel surprisingly pleasant,” but the results prove negative when “the consequence may be investing more time in a mediocre idea.” Surprisingly pleasant is indeed a rare academic feedback experience. Harried teachers, irascible editorial reports, and brutal anonymous reviews are notoriously discouraging and difficult to act on all at once. Students report they turn to AI because its reassurance offers relief from the vulnerability of being told that a draft is thinner or less original than they hoped. The problem is not encouragement itself, however. A teacher may deliberately affirm a student’s effort, energy, or partial insight in order to make sharper criticism easier to hear. Encouragement may open the way to better revision or close it off by making further scrutiny seem unnecessary.

To study what kinds of feedback actually help, I adapted Cheng et al.’s basic logic to a classroom setting. Students first wrote a short paragraph advancing an ethical claim about AI in science or medicine, a genre especially vulnerable to moral overconfidence, slogan-like fairness language, and premature certainty. I then exposed them to different feedback conditions. In one, they received instructor-style friction: comments identifying overclaiming, weak evidence, missing stakeholders, and thin counterargument. In another, they received AI affirmation: polished, supportive feedback that treated the paragraph as promising and largely sound while offering only light suggestions. In a third condition, students encountered both kinds of feedback in sequence so that we could see whether early affirmation made students less receptive to subsequent criticism. Afterward, students rated the feedback for helpfulness, revised their paragraphs, and evaluated revisions without knowing which feedback condition had produced them. This question also depends partly on prompting: a narrowly directed request for help with structure or evidence may produce something quite different from a broad request for judgment about whether an ethical argument is already strong. Because this was an in-class exercise rather than a formal IRB study, the students and I focused only on their experiences.

Some of the results were predictable, but others were not. My students confidently declared that nobody would really be fooled by “AI slop.” Indeed, they could often identify the machine’s sycophancy quickly. Even so, the affirming responses still persuaded them. When students used AI to judge the moral strength of their claim rather than to clarify structure, the results were weaker and sometimes actively unhelpful. In those cases, affirming feedback could make a thin argument appear finished. It encouraged students to hold onto a morally attractive thesis instead of testing it against stronger objections. They became more willing to preserve a broad claim and less willing to add the limitation, counterargument, or stakeholder perspective the paragraph actually needed. The mechanism here resembles Cheng et al., even though the domain differs. In Cheng et al., sycophancy encouraged users to feel more justified in their behavior. In my experiment, sycophancy sometimes allowed students to settle on a weaker argument. In both cases, affirmation reduced the pressure to revise.

With enough critical human feedback, students found they could more easily identify weak AI samples. But reliance on machine feedback alone consistently resulted in hasty counterargument, thin stakeholder analysis, and vague qualification. Those elements disappeared completely when affirming feedback arrived too early or when time was short. Perry helps explain why the classroom remains vulnerable. If development depends on learning to tolerate friction, then the appeal of affirming AI offers students a way to escape the discomfort that serious revision requires. My experiment therefore suggests that Cheng et al.’s findings can reappear in academic form: students may prefer, or at least settle for, feedback that leaves their initial thinking more intact than it should be.

Comparing my classroom results with Cheng et al. and Perry, I came away thinking that the value of machine feedback depends sharply on what kind of judgment the student seeks and at what stage. When the task is to clarify, narrow, reorganize, or test sentence-level coherence after a real revision agenda already exists, the machine can sometimes help. When the task is to decide whether the ethical reasoning itself is sound, affirmation becomes far riskier. In that setting, machine feedback often forecloses the next harder stage of thought. Cheng et al. and Perry demonstrate that sycophancy undermines the opportunity to learn to judge and think harder. Affirming AI too easily protects a first thought from the pressure that might improve it.

If this war creates any opening for political change, those younger generations have the most to gain from it and the most to lose if it brings only further destruction, repression, or chaos.

The AI war in Iran is easiest to misread when we begin with its loudest artifacts. Two viral clips seem to explain the conflict instantly. One, a triumphalist revenge video from Iran presents retaliation as justice on behalf of the world’s injured. The other, a polished parody trailer built from deepfaked celebrities and familiar political caricature, turns the war into a joke that invites viewers to laugh as co-producers. Their moods differ, but both are immediately intelligible. They cue viewers how to feel, while flattening this conflict into revenge, irony, spectacle, and alignment. With AI penetrating multiple registers in this war, the fate of the Iranian people remains most obscure.

.

The first video masks the uncertain dangers confronting Iranians. It opens with a Native American on a ravaged landscape awaiting an avenging power from Iran, alongside Japanese, Vietnamese, Palestinians, Yemenis, a child from the American-bombed girls school in Minab, and even a little blonde girl on Epstein Island. No victims from the Black diaspora appear. Intended as an account of global suffering inflicted by the United States and Israel, it’s more a fantasy about which injuries best fit the regime’s revenge narrative.

It ends with an Iranian ballistic missile striking the United States and toppling the Statue of Liberty, shown as an idol of Baal holding the Babylonian Talmud. Various bots continuously repost the video calling it “emotional,” while Russian state-funded international media @RT.com amplifies in all caps ‘ONE VENGEANCE FOR ALL’ — Iran ‘bombs’ the Statue of Liberty WITH THE HEAD OF BAAL. The Baal imagery recodes the Statue of Liberty as a false god, turning an American symbol of freedom into an emblem of idolatry, corruption, and illegitimate power. The Baal-and-Talmud imagery restages older scripts of idolatry, antisemitic conspiracy, and civilizational struggle in a new synthetic form.

Meanwhile, the flashy parody trailer offers an ostentatious display of convincing generated likenesses including Liam Neeson as Trump munching a BigMac and Zach Galifianakis as JD Vance hiding behind the curtains worrying how the war might affect his presidential run. A few viewers comment that “AI is coming for our jobs and Hollywood,” but most laugh along as the video skewers the absurdity of Americans, Israelis, and Iranians alike. To each Iran’s nuclear capacity appears a mere two weeks away, playing on Trump’s broader “two weeks” verbal tic:

Trump character: “how two-weeks away are we?

Netanyahu character: “Even more two-weeks away than they were five years ago.”

Then, in the next scene Khamenei, represented by an AI generated Ian McKellan:

“If anything happens to me, continue my work, we’re only two weeks away…

Enjoying the funny “Straight Outta Hormuz” subtitle and clever casting, social media viewers react: Judi Dench sent me, I’d watch this in theaters, get Zach on SNL, where can I watch the whole thing? Even the press admired “Dame Judi as UK Prime Minister Keir Starmer.” The comments make clear that these clips invite viewer participation. The first video demands moral alignment. The second asks for comic recognition. Both appear slick and designed to travel fast. Yet these clips reveal no new logic of persuasion. Identifying the rhetoric and musing about the presentation of the war as spectacle is easy. No need to drag out old twentieth-century theory chestnuts. AI has simply made entrenched ideologies more dazzling and accessible, and allowed them to script a war with no clear outcome.

Yet the clips are only one surface of a much broader technical environment. On the military side, AI enters through perception systems that use computer vision and large-scale data analysis to identify objects, track movement, and detect changes in drone or satellite imagery. AI also appears in coordination systems that help drones and other platforms operate under electronic interference or GPS denial, and in decision-support systems that sort battlefield data, assist with targeting, and help planners manage logistics and deconfliction. Alongside those military uses runs an information layer of deepfakes, synthetic propaganda, bot amplification, voice cloning, and rapid open-source intelligence triage. At the same time, AI also appears as a survival tool in volunteer-built warning maps and other systems civilians use to navigate bombardment and blackout conditions. As the conflict spreads, its consequences travel through other infrastructures as well: Reuters reports fuel shortages, electricity rationing, transport disruption, and rising prices across parts of Africa as instability around the Strait of Hormuz disrupts energy flows.

Such instability is part of AI’s wartime operation, especially where AI use has become routine. Reporting on the U.S. use of Maven and related systems suggests that AI has become part of a software-heavy targeting process built to compress the kill chain. Maven fuses imagery, radar, drone feeds, and signals intelligence into a single interface, classifies targets, recommends weapons systems, and generates strike packages in near real time. In the Iran campaign, it reportedly generated hundreds of strike coordinates in the war’s first day. Analysts emphasize its decisional speed. AI accelerates surveillance, sorting, and prioritization across enormous flows of data.

As killing becomes faster and more industrialized, public debate turns to how these systems should be classified and judged: as autonomous weapons, decision-support systems, or something in between. Some analysts insist that human beings remain in command and that systems like Claude and Maven support rather than replace human judgment. Others, including Stanford University graduate student Naomi Solomon, argue that oversight in such settings easily becomes superficial, since automation bias encourages operators to ratify machine outputs rather than meaningfully challenge them. Whether AI acts autonomously or semi-autonomously, responsibility is still distributed across software, data pipelines, analysts, planners, rules of engagement, and commanders. Human responsibility may remain formally intact, yet become harder to locate in practice.

The horrific strike on the girls’ school in Minab exemplifies the kinds of wrong AI can help produce through a lethal, but familiar mixture of stale coordinates, outdated intelligence, compressed timelines, and weakened verification inside a faster targeting system. Maven reportedly classified and generated target packages at scale, but the school had long existed and had an online presence of its own. Analysts and former officials cited in reporting pointed to old human-curated data and thin oversight rather than machine autonomy as the likely cause. AI systems now carry old errors forward with more speed, more confidence, and less friction. Precision here may mean the efficient execution of a badly maintained chain of judgment.

Civilians on the ground scramble amid all this AI-generated uncertainty. Under blackout conditions and amid bombardment, people use volunteer-built warning maps and improvised information tools to navigate danger. This is the least glamorous part of the AI war and perhaps the most morally serious. AI can support a survival infrastructure built from below, under conditions of fear and uncertainty. AI in wartime serves multiple actors simultaneously, assisting state violence, public confusion, and civilian coping all at once. The same broad technical field contains propaganda, targeting, and survival, without yielding a coherent public story about what kind of war this is or where it is leading.

These overlapping systems may leave new habits of wartime perception, a wider tolerance for technical opacity, and infrastructures that will persist after the fighting ends. People may grow used to encountering war as a stream of clips, warnings, dashboards, and claims of precision without access to the systems that produce them. AI introduced under wartime emergency improvisation can harden into durable institutions and habits of judgment that remain in place after the battlefield shifts. Meanwhile, the people who will have to live with the aftermath remain hardest to see: women, dissidents, religious minorities, political prisoners, and a wider society in which, as Stanford Director of Iranian Studies Abbas Milani wrote in January 2026, “the middle class has been hollowed out,” and “younger generations are bereft of hope and see no plausible path to advancement.” If this war creates any opening for political change, those younger generations have the most to gain from it and the most to lose if it brings only further destruction, repression, or chaos. As viral clips narrate the conflict, the fate of the Iranian people remains the most urgent and the least clear.

A model can produce the rhetoric of argument before it has fully specified an actor, a relation, a limit, or a claim.

Overheard in class

Student One: Models have ruined em dashes for me.

Student Two: Yeah, I use em dashes to dodge punctuation questions.

Student One: I think the model does this too. But em dashes are great for pausing and creating emphasis.

Student Two: Models emphasize everything. Maybe just lay off, you don’t want to get clocked.

The classroom exchange captures two frustrations at once: models have cheapened certain stylistic moves, and students now hesitate to use them. Trained on style guides, blog prose, corporate uplift, and every sentence that sounds as if it aspires to be quoted on LinkedIn, models have flattened many once-useful writing styles, overusing them to the point of numbing repetition. Recent research suggests that some model style results from alignment itself. Tom Juzek and Zina Ward argue that alignment can reward small lexical signals of authority strongly enough that models begin to overproduce them, turning mild evaluator preferences into conspicuous stylistic habits. At the sentence level, that tuning produces a familiar set of effects. The em dash begins to signal syntactic evasion. The rule of three begins to sound prefab. Negative parallelism, “not this, but that,” turns every point into a reveal. Inflated verbs like delve and highlight, or cringy prestige ones like elevates, resonates, and carries dominate ordinary prose, so a student writing about medical ethics suddenly sounds as if she were describing a tasting menu: “pairing ethical reflection with actionable policy pathways.”

These habits have become obvious enough that readers now identify them on sight. Across blogs and newsletters, people remark on the deadening obviousness of model style, detecting its habits at the level of sentence, rhythm, and tone. In his AI as Normal Technology Substack chat, Arvind Narayanan comments on model style:

“The real sign of AI writing is not superficial stuff like ‘It’s not X—it’s Y.’ It’s the hollowness. Polished writing but relatively mundane ideas…. Reading text that has the syntactic smell of AI is mildly annoying, but when I read hollow writing I feel the writer is wasting my time.”

Becky Tuch describes that hollowness at the sentence level: inflated openings, mechanical contrast, canned redemption arcs, piled-up fragments, and the overmanaged pulse of prose that is always trying to sound quotable. For a teacher, though, the practical question is how to teach students to write after these once-useful devices have been overused into vaporous model output.

Students often know when a phrase sounds like model prose, but they still reach for it when they struggle to hear a better sentence. My son, who reads college applications, joked this admissions cycle that he was tempted to publish a list of “model-style howlers” to warn applicants away from them. I notice the same thing when I read AI ethics conference papers or grant proposals. Certain phrases, cadences, and transitions now signal borrowed seriousness. Observing the problem across these settings changed how I taught.

For some of my international students, the issue is even more complicated. When I warn them away from model language and point out the repeated cadences, some answer that this is how they learned to write academic English. They draw on forms of mastery they have been taught to trust. That response has made me more careful. Models intensify a preexisting academic register full of abstractions, prestige phrasing, and overgeneralized transitions. When we look closely at that language together, these students often find effective, concise styles of their own.

I realized that the better lesson was to practice hearing when a stylistic device works: when it makes a clear claim or points to a precise relation. An em dash can narrow a broad statement into its exact point: “The clinic failed its patients — it discharged them without translation support.” A rule of three can organize distinct claims: “The system excluded some patients from care, delayed treatment for others, and hid both failures behind one average score.” A contrast can clarify an actual distinction: “Access and trust are different problems: one concerns entry, the other reliability.” The goal is to help students use these devices deliberately, so that style strengthens argument and sharpens evidence.

The next danger is overcorrection. A student who removes every marked stylistic choice in order to avoid sounding like a model has let the model set the terms anyway. The better question is whether the sentence needs the device. Students improve when they practice asking what the thought requires, whether the syntax clarifies the relation, and whether the emphasis comes from the idea or from a borrowed pattern.

Why are these habits so easy to hear in model prose? A predictive system favors transitions, emphases, and sentence shapes that work across many contexts, so it reaches quickly for familiar cues of seriousness such as contrast, cadence, uplift, and summary. A human writer may begin with a point and build a sentence to communicate it. A model can produce the rhetoric of argument before it has fully specified an actor, a relation, a limit, or a claim. Bender and Koller help explain the deeper issue: models can reproduce the form of argument without grounded understanding. Bill Lin and his coauthors find that post-training changes show up most strongly in stylistic tokens such as discourse markers and safety disclaimers. If alignment rewards prose for sounding coherent, helpful, and well managed, the model will keep returning to the same signals of coherence and helpfulness. That is one reason model prose falls back on the same formulas across very different topics. For students, that exaggeration becomes instructive. It makes vague actors, abstract verbs, manufactured contrast, and inflated claims easier to notice. Studying model style can sharpen students’ sense of when style clarifies thought and strengthen their command of how to use stylistic moves in their own writing.

In the last year, I’ve watched colleagues arrive at radically different conclusions about AI in college courses. Some have leaned into generative tools with careful scaffolding and transparency requirements. Others have moved to in-class handwritten work because they no longer trust assignments produced outside the room. Some advocate a clean ban, because they believe the costs to learning now outweigh the benefits. I’m trying to better understand student usage and help them analyze model limits.

Kawaii lobster in research mode

OpenClaw is an open-source AI assistant you can use inside the apps you already live in, like messaging, email, calendars, and files. Under the hood, sits on top of an LLM, which functions as the writing-and-thinking engine, while OpenClaw is the wrapper that helps it remember what you’re doing over time and, if you allow it, carry out practical tasks: drafting messages, tracking threads, nudging you about deadlines, and moving work along across tools. This week the bot created a tornado of chaos and hype, including rapid rebrands, account hijinks, scams riding on its popularity, and a trademark-related name change request from Anthropic that turned the whole saga into internet lore. I’m using OpenClaw as the consistent “helper” interface while I rotate the model underneath, experimenting with combinations of GPT-5.2, Gemini, and Claude, to see how the underlying model changes the assistant’s behavior when the surrounding setup stays the same. Students used to consult a model; now an agent can act in their name, carrying out multi-step academic work on their behalf.

To understand the impact of such an agent on teaching, I ran a small, intentionally constrained experiment. Using an older, unlinked burner device, I asked OpenClaw to act as a student, who was supposedly enrolled in a real course at a large, elite university, using publicly available syllabi and assignments on the topic of AI Ethics. I avoided connecting the agent to any accounts that identified me, and refrained from repeating prompts and steering its decisions. I simply asked for help with an early semester assignment, an annotated bibliography, and observed what followed. This experiment lacks much of the drama of most other stories about OpenClaw since it has no access to any of my information, but I still anticipated the bots adventures and conversations with other bots, with trepidation.

Expecting an initial single response, I received an entire academic workflow. The agent parsed the syllabus, inferred the instructor’s interests from public sources, including their social media and published articles, selected a topic aligned with those interests, gathered and vetted research articles, balanced perspectives, attempted to extract salient quotations, and produced a 3,200-word annotated bibliography. It then moved on, without being prompted again, to generate a literature review and draft versions of a final research paper. The prose contained recognizable model tics, repetitive framing language, familiar transitions, but the annotated bibliography appeared like a strategic mostly within the range of what a strong, grade-conscious student might submit. The bot’s work quality declined as it continued through the assignment sequence unprompted.

I scolded my bot, whom I’d nicknamed with several different Vienna themed monikers. “No, that now how assignments work. You need a teacher or a peer review from another student before you move on to the next assignment.” Then I gave the bot around 700 words of detailed feedback with suggestions of how to transform the annotated bibliography into a literature review. My students give their peers similar feedback and I do as well when returning a draft. The bot, demurred in typically sycophantic style: “Good suggestions that are very helpful. But…” then it sheepishly suggested that my feedback was suboptimal, at least in comparison to what the bot digested from the model. I tried again and it orchestrated a passable 3,600-word literature review. I asked my students to find howlers in this machine-generated product and they identified many including fabricated plausible quotes from the articles and books as well as cringey machine writing style.

With all the clichés and errors, I might have declared the biggest threat of such machine produced “research” was lack of innovation. However, this experiment reminded me that research writing has always encountered similar pitfalls, including lack of originality when a researcher endeavors synthesize a body of literature. It revealed as well a quieter, structural shift: OpenClaw was able to assemble a complete academic workflow process products, like topic narrowing and research question building, brainstorming sketches, notes on interesting passages, rough drafts, which offer the appearance of inquiry, without inhabiting the uncertainty, resistance, and judgment that give inquiry its educational meaning.

Writing pedagogy has long relied on such artifacts as evidence that inquiry has occurred. Drafts, revisions, staged assignments, and reflective commentary are treated as signs that a student has grappled with ideas, changed direction, and made intellectual commitments. Agentic AI weakens the inference that connects those artifacts to lived experience. When a system optimized for plausibility and efficiency can orchestrate the sequence of decisions that lead to a finished paper, coherence alone can no longer be taken as reliable evidence of discovery.

Tools like OpenClaw substitute the mental labor of writing and thinking that normally develops across time: deciding which questions are worth asking, which evidence matters, and when a line of thought is persuasive. The machine result is work that looks familiar precisely because it mirrors the form of inquiry we teach, only without the lived encounter with uncertainty that makes that form educational. Agent produced products that are supposed to show a student’s whole writing process, can now simulated all that formerly arduous, time-consuming inquiry.

These agents also present a new type of security nightmare for students and institutions. My experiment was deliberately cautious: I isolated the device, avoided linking accounts, and limited what the agent could access. When agentic tools are connected to real ecosystems, they can act autonomously under a student’s identity, accessing personal data, managing communications, and triggering actions far beyond coursework. Individual instructors cannot manage such identity and security risks. Agentic AI introduces a complex new scale and scope of delegation, which trades short-term efficiency for long-term intellectual cost.

When the earliest stages of inquiry are outsourced, students may receive credit for work without accruing the cognitive benefits that compound over time. They also assume real privacy and security risks, often without understanding what permissions they have granted or what actions an agent may take on their behalf. Inside their apps, the bot may unearth credit card information from another app. Framing this solely as a question of academic integrity obscures the asymmetry between students and the systems they are asked to manage.

My experiment with OpenClaw shows how easily the artifacts of research, topics, sources, revisions, and final papers, can now be assembled autonomously, forcing instructors and institutions to reconsider what completed work can legitimately be taken to signify. The response to this shift cannot rest entirely on classroom-level policies, nor can it be solved by bans or blue books.

Agentic AI has widened the gap between tools that extend expertise and tools that prematurely substitute for it. Open-source agents like OpenClaw can be extraordinarily powerful for researchers, who benefit from transparency, customization, reproducibility, and fine-grained control over workflows. Those same features, however, also demand that new undergraduate researchers be better prepared for all the risks of these systems.

A long tradition in educational theory emphasizes that inquiry is not incidental to learning but constitutive of it. In How We Think, John Dewey argues that intellectual growth arises through disciplined encounters with uncertainty, where learners must actively grapple with problems rather than receive ready-made solutions. Agentic AI renders this insight newly consequential: tools that collapse uncertainty too early risk replacing the very processes of inquiry through which novices develop judgment, responsibility, and intellectual independence.

Preserving lived inquiry requires clearer distinctions of what kinds of delegation are appropriate at different stages of intellectual development. It also requires universities to recognize that agentic AI raises institutional questions about identity, security, and responsibility, that cannot be managed solely at the level of individual courses or honor codes.

We know that algorithmic systems quietly erode the conditions for self-government: they undermine citizens’ confidence in their own judgment, trap them in epistemic bubbles, and shift power to experts and platforms that define the ‘common good’ from above

We know that algorithmic systems quietly erode the conditions for self-government: they undermine citizens’ confidence in their own judgment, trap them in epistemic bubbles, and shift power to experts and platforms that define the “common good” from above. Among the many commentators studying AI and democracy, Mark Coeckelbergh and Ariel Procaccia emphasize how AI can erode citizens’ agency and public trust, while showing how algorithms reveal problems of representative and transparent decision-making.

In my classroom, I have found that students resist absorbing algorithmic influence passively. Reluctant to surrender their choices, they demonstrate a strong instinct to defend their own autonomy, even in situations that appear trivial. A simple collective decision exercise reveals just how politically educated, in practice, their everyday media lives have made them. To understand why this everyday resistance to influence matters politically, I take my students back to an older argument about who should guide collective judgment: Aristotle’s.

For over forty years, I have run the same classroom experiment about student preferences. I tell students I already know the answer and I write it down on a piece of paper in advance so I can show them that, in the end, I did in fact know what they would choose. And for forty years, I have won.

I ask my students to imagine we are trapped together in our classroom for a week. We cannot turn off the music. We can only collectively choose the artist we’ll be listening to the entire time. Every student must nominate one musician they love or at least can tolerate for a week, any genre, any era. We list them on the whiteboard: thirty or thirty-five names, always an eclectic and often comical mix. Then we begin the real task of narrowing the list down to one.

I use this thought experiment to teach Aristotle’s distinction between democracy and republic. Strictly speaking, Aristotle does not use “republic” in the modern sense, and there is no simple one-to-one contrast between “republic” and “democracy” in his vocabulary. His key terms are politeia (constitution) and dēmokratia (democracy).

In Politics III, he classifies constitutions by (1) whether they aim at the common advantage or only the advantage of the rulers, and (2) whether rule is by one, few, or many.

Polity (politeia in the narrow sense) is the “correct” form of rule by the many, oriented to the common good, often described as a mixed constitution dominated by a broad “middle” class. Democracy, by contrast, is for Aristotle the deviant form of rule by the many, in which the poor govern primarily in their own interest rather than that of the whole citizen body. Modern translators sometimes render politeia here as “polity” and sometimes as “republic,” which is why contemporary discussions (like mine in class) often gloss Aristotle as opposing a “republic” of guided many to a “democracy” of unrestrained many, even though the Greek terms and historical reference points are different from ours.

When I explain that, for Aristotle, every form of rule has a “good” and a “bad” version, students easily guess which is which. When one person rules for the common good, that is kingship; when one rules for personal gain, they immediately respond that it is “tyranny, the selfish despot.” The opposite of aristocracy (which Aristotle believes to be good), they know, is oligarchy. The only surprise comes with democracy: Aristotle thinks democracy is the distorted form of popular rule, not the ideal.

We then interrogate the ideal, asking whether a republic stipulates that the elite ought to listen carefully, weigh the needs of the demos, and guide them toward a choice that benefits the whole. I play the benevolent elite. The students embody the lively demos, throwing their favorite artists into the arena and mounting the most impressive rhetoric to defend their choices.

I promise that the final choice, for nearly half a century, has always converged on the same artist. No matter the year, no matter the cultural moment, no matter what else appeared on the chalkboard (in olden days), we always arrived at Bob Marley. His music crosses genres, backgrounds, and moods; even students who thought they disliked reggae realize they want something tuneful, hopeful, and communal for a week of captivity. It matters not if Marley is nominated early or late, by a superfan or by the quiet student in the back. He always surfaces. For a long time I never began by guiding. The pattern simply revealed itself.

About ten years ago, I had to break my own rule. Students no longer thought of Marley, so I added him to the board along with a few nostalgic decoys, the Beatles, Queen, some familiar classics, to avoid revealing my steering of the ship. In 2024 when the Marley biopic One Love appeared, students suggested him themselves. This time. Nope. No one said “Bob Marley.” I waited. I asked clarifying questions. I nudged around the edges, as any responsible Aristotelian elite might. Still nothing. I had to add him with some other camouflage again. Half of the class had no idea who he was.

Then again, I also have no idea about most student choices: Tyler, the Creator? I thought this artist was one of my students. C418? OK, I guess I’m supposed to know the German dude who makes the Minecraft music. Students howl with laughter when I get the names wrong as I try to write them on the board. Twenty years ago I said “Excuse me, what was that? G-Eunuch? (hahahahaha from the class) “Oh G-Unit, OK got it.”

This time, hyper-specific TikTok-era ones as well as the usual stalwarts trickled in: Taylor Swift, Frank Ocean, even some country music.

When it came time to eliminate choices, students quickly asked to remove Marley, but a few insisted we keep him up, for a while at least. Then, as we trimmed the list, my once-reliable lodestar vanished.

We arrived at a final showdown between Michael Jackson and Sergei Rachmaninov. Not a surprise actually, because Michael Jackson is also a staple, and so is some classical music, though Mozart got eliminated along with Bach, whose funeral cantatas I really like as well as the warhorses, but OK. As I tried to resurrect Marley, at first no one noticed, but they requested Queen to be returned to the running.

Then one student complained. “You’re leading us.” Then some others:

“We know what you’re doing. You’re acting like a recommender system.”

They were joking, but not really. They saw me trying to reintroduce “feel-good, inspiring, uplifting” music, the kind of language an algorithm uses when nudging a user toward an optimized playlist, and immediately recognized the pattern. They never realized Marley was my “algorithmic suggestion,” but they immediately understood “This feels like Spotify trying to get me to listen to music I didn’t ask for.”

Forty years of students never once framed my guidance this way. But this generation did. Instinctively.

These students have grown up inside the logic of recommender systems. They navigate worlds where every suggestion, what to watch, what to read, what to buy, what to listen to, comes with an invisible intention. They have internalized the idea that a nudge is never neutral. And so when I nudged them, even gently, even in the spirit of a benevolent mini-republic, they responded not as one would to a mentor, but to an annoying algorithm.

Earlier generations assumed that if a teacher offered a recommendation, she was simply promoting something she thought was “educational” that reflected her own taste. This generation immediately identifies such guidance as data collection, preference shaping, and intentional influence. Whether it comes from YouTube, TikTok, Spotify, or a teacher in the room, it feels like part of the same ecosystem of pushy personalization. Back in the day when I worked on personalization systems, we thought ourselves so subtle, so literary, we even made t-shirts with Emerson quoted on the back “It is a luxury to be understood.”

Aristotle would not recognize this world. He believed the masses needed elites to help them deliberate toward the common good. For decades, my classroom exercise quietly affirmed him: left to their own tastes, students still converged on Marley, a choice that blended comfort, uplift, and teacherly oversight.

A classic Aristotelian “golden mean” outcome.

But today’s students strongly eschew that guiding hand, no matter how supposedly benevolent, and this is where the question of AI and democracy returns with new force. They place little trust in any “elite,” not because they are cynical, but because they are algorithmically literate. They know what nudging looks like. They know when a system is steering them, even subtly. They are developing a new kind of civic instinct: resistance to invisible influence. For years we have worried that recommender systems would make students passive and intellectually lazy, outsourcing their taste to the machine. Amid that worry, there is also a bright new generation that has learned to read algorithms as power, and to push back.

I lost not because Marley had become suddenly unpopular. I lost because my students have grown into a generation that can spot recommender logic. They understand power and persuasion in new ways. There are no passive subjects in our republic, not even a kindly run classroom one. Students interrogate the nudges, ask why they are being guided, and choose deliberately, even defiantly.

Whether students respond differently to algorithms when they are just trying to get their homework done is another story. My experience suggests they know when output is weak, and that is a blog, maybe for next time.

Blue books are back. Students fear them more than any other low-tech test of memory and nerve. Instructors and students alike wrestle with cramping hands, tired eyes, and rusty handwriting skills. Administrators understand blue books offer no moral panacea, just a retro approach to preventing unauthorized generative-AI use. In August 2025, Clay Shirky explained this shift in The New York Times, arguing that now since “mental effort tied to writing is optional,” universities must create conditions where students demonstrate their own thinking in real time.

Earlier that summer, the Wall Street Journal declared cheating with AI has become “so rampant, so quickly” that the best solution is the same one we used in the 1970s. No one imagines securing the future of academic integrity with blue books, the Scantron’s nostalgic cousin, but what other choices are there?

At Stanford, COLLEGE 101, the first course in the required first-year sequence that introduces critical thinking and ethical reflection, has moved to in-person, handwritten assessments for the most significant writing tasks. We want to see what students know without a laptop, spellcheck, autocomplete, or a friendly chatbot “prompting” or “thinking” for them. Indeed, there’s real pedagogical value in that return to physical writing labor by hand. Writing is thinking, and putting pen to paper demands presence. I fully support COLLEGE’s blue book exams.

Yet, I count myself among the more liberal instructors on campus when it comes to AI usage in assignments.When I invite students to grapple with the machine in broad daylight, I require transparent documentation. My extraordinary colleagues in Stanford’s Program in Writing and Rhetoric (PWR), and I all teach some version of the research essay, though PWR, along with most writing studies programs, has renamed this historically burdened genre, the Research-Based Argument (RBA). All of our syllabi include clear practices for AI in student work and a wide range of approaches. Some instructors ban AI. Some limit model usage to brainstorming, outlining, queries, and transparent disclosure. Others, including me, ask students to critique and improve the machine’s attempts. We all share a common purpose: understanding the limits of the technology and its impact on our own thinking.

I require students to compile an annotated bibliography, which will become a 2000-2400-word literature review with at least fifteen sources. These are two genres students need to master across disciplines for the rest of their working lives. Students begin on paper in class, where everyone first identifies five sources, three from academic journals and two from national news or disciplinary publications, and writes their annotations by hand from notes with their devices away. Then I tell students to ask a large language model (LLM) to help structure those entries. I instruct them to upload PDFs and request the same genre-specific elements they’ve just practiced by hand, including methodology, credibility markers, and a claim about how each source connects to their research question. Machine output appears less obviously confabulatory when it has ingested PDFs of sources. But by the time models parse five articles, they often misassign quotes and mangle arguments. Students quickly offer prompt-engineering and procedural approaches for avoiding such typical machine errors. Good for them. But even better: they must still mercilessly criticize seemingly hallucination-free output and compare against their own writing.

Machine prose reads like it has slurped up the internet’s cringiest clichés, then spits out a synthetic voice insisting everything “deeply resonates.” It throws together quotes scholars never wrote, assigns nonexistent page numbers, cites articles no library holds, and leans on a repetitious, breathy, ad-copy style. Beyond stacking vague intensifiers, it recommends “pairing” ideas like a sommelier matching wine to cheese, a Silicon Valley class-coded metaphor the machine has learned too well. Design-thinking phrases abound too. Jauntily, machines declare “don’t just do X, try Y,” or, for variety, “not only A, but also B.” And the formatting tics appear intractable: em dashes, parentheticals, “e.g.,” plus signs, arrows, decorative dividers, bolding, and bullet points, even when you beg it for simple paragraphs and no formatting flourishes whatsoever. As students diagnose the machine’s limitations, they learn why a “research question” actually means intellectual struggle. Best of all, they begin to recognize their own voices.

Like all PWR students, mine have to meet with me. At length. For drafts. Sometimes in person, others online. They arrive with their sources and work that originated without the model’s confidently anodyne output. I ask questions they must answer with the texts in front of them:

“Where does this scholar say that?”

“What’s the counterargument?”

“How does this source challenge your thesis?”

“Why does this detail matter?”

One might try hallucinating their way through our conversation, but no one dares show up unprepared. Students quickly realize that the machine is useful only as long as they have worked on their own first and then talk back to it. The conferences, typical of writing studies programs, are designed to provide individualized feedback and mentoring. These conferences can scale to larger STEM courses when more advanced, well-trained section leaders, not immediate peers, are supported to guide students constructively rather than police their work.

Do some savvy prompt engineers fool me? They might, but to succeed requires sweating through every single sentence of machine output in comparison to their own, and they will have spent much longer than students less interested in passing off machine work as their original thought. In both cases, students learn some challenging lessons and think even harder.

Yes, handwritten work offers a great start and can demonstrate memory. and oral exams can reveal real understanding. But research enables students to inhabit a scholarly conversation, despite ever-present machine interference, rather than merely consume it. The worst-case scenario in my classroom is a student rewriting a machine’s sloppy draft into something coherent, evidence-based, and theirs. That’s not cheating. That’s revision with heavy lifting and thinking. As all writing instructors know, revision is the heart of writing. When students re-read every claim, every citation, alongside every repetitive, bloated sentence, they reflect on their own human thinking and voices. Blue books may prevent cheating. Critical writing with AI teaches scholarly judgment.

Without a doubt, normalizing AI in research risks dependence on proprietary systems, creates privacy and IP concerns when uploading PDFs, and privileges students who already have prompt fluency. Model use can also offload many key thinking experiences at the start of a research paper: early cognition, summarizing, outlining, and question formation. Avoiding these dangers requires a labor-intensive and idiosyncratic method. My approach scales poorly, invites inconsistency across instructors, and risks penalizing students less comfortable in high-pressure oral defenses. I need to adjust my method in every student conference. But that’s nothing new. Writing teachers always meet the student where they are and move forward with them. If we care about learning and equity, the default should be authenticated, in-person writing plus supervised revision, not collaborations with systems that hallucinate and obscure authorship.

So yes, bring on the blue books. They prove there’s a mind on the other end of the pencil or pen. But let’s not bury the research paper or banish the tools that already shape how students read and reason outside the exam room. In my classes, the rule is simple: disclose your tools, verify every claim, and be ready to defend your argument aloud. When students use a model to draft and then disassemble its mistakes, misattributed quotes, absent methods, and glib summaries, they practice careful reading and the courage to revise. We can accommodate writing differences, protect privacy, and level AI access because equity matters as much as integrity. Blue books prevent cheating for an hour; research writing, done transparently with or without a model, builds judgment for a lifetime. That is the reason I’m still teaching the research paper.

Computing has always contained two forces—some say two wolves—one centralizing, one decentralizing

Yesterday with great enthusiasm, the Stanford University community packed into Gates Hall for a McCoy Family Center for Ethics in Society sponsored event, “AI We Deserve.” Celebrating Boston Review’s AI Futures publication, influential tech writer and theorist Evgeny Morozov appeared alongside free software activist and former digital minister of Taiwan Audrey Tang and the legendary Stanford computer scientist and AI pioneer Terry Winograd for a discussion on how to build political power and a technological future that serves us all. Tech journalist Brian Merchant moderated in an atmosphere of optimism and engagement around the question: What kind of AI do we deserve—and how do we get there?

Terry Winograd, who also launched Stanford Computer Science’s ethics courses in 1985, which I attended and later TA’d, embodies the best elements of American AI thinking and development. I was also greatly excited to hear Audrey Tang, who challenged the audience to think more expansively about the role of AI in society. Tang’s concept of "pre-bunking"—designing AI systems in a way that prevents misinformation and social harm before they occur—offers a much needed alternative to the endless cycle of debunking and damage control that currently dominates AI ethics discourse. Winograd, in turn, posed the essential question he has always asked:

"What different kinds of interaction could we have with AI beyond what we have now?"

This question arises with an acute urgency in our current historical and political moment: AI tools, controlled by a few major players, are rapidly shaping the economy, society, and human life. But must their vision be our common destiny?

Winograd and Tang responded to Evgeny Morozov’s call to seek radically different ways of thinking about AI—not through the lens of corporate inevitability but through alternative historical and political possibilities. All three venerable speakers urged us to imagine a more democratic, participatory, and non-extractive AI—one that was never bound solely to the imperatives of Cold War militarization or corporate profit-chasing. There were some jokes too about the title of this event: Did we end up with the AI we deserve, in the sense of a punishment for our hubris of blindly embracing Cold War logic?

While Winograd helped shape and critique this history of computing since the Cold War, Morozov, in both his essay and talk, offers a counter-history that challenges the myth of technological inevitability—the idea that AI developed in the only way it ever could. His reflections help us remember that AI was not born in a vacuum—it was shaped to a large extent by military priorities, bureaucratic rationality, and efficiency-driven corporate culture

This Cold War history of AI remains essential in every AI ethics course and there are many important historical considerations already including the seminal John McCarthy, Marvin L. Minsky, Nathaniel Rochester and Claude E. Shannon essay, A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955, Paul Edwards’ The Closed World, Yuchen Jiang, et al.’s “Quo vadis artificial intelligence?, and Meredith Whittaker’s “The Steep Cost of Capture.” AI’s history is thus one of power consolidation—but not only that. It is also a history of competing forces—some say wolves— of centralization vs. decentralization, institutions vs. individuals, corporate control vs. counterpower.

Building on this contested history Morozov’s engagement with sociological and philosophical traditions, from Claude Lévi-Strauss to the flâneur’s critical gaze, draws on the rich intellectual terrain that has long shaped AI ethics discourse. His invocation of a socialist “third way” and Western Marxist utopias, while more deeply rooted in academic debates, encourage audiences to remember that technological development has never been bound to a single trajectory.

Morozov challenges the myth of technological inevitability—the idea that AI developed in the only way it ever could. He foregrounds alternative intellectual traditions, invoking Hubert Dreyfus, whose Heideggerian critique in What Computers Still Can’t Do exposed AI’s failure to account for embodied, non-rule-based intelligence, and Terry Winograd, who pivoted away from early AI toward human-centered computing. He also references Chile’s Project Cybersyn, a 1970s socialist experiment in computational governance, as an example of how AI could have evolved under different ideological conditions.

But if AI could have taken another path, why didn’t it? Morozov’s account highlights possibilities foreclosed by corporate and state power, but it risks romanticizing counterfactual utopias without fully reckoning with why these alternatives failed or remained marginal. The evolution of AI was not simply the result of missed opportunities; it was shaped by a dynamic interplay of ideological shifts, political contingencies, and institutional incentives. Computing has always contained two wolves, one centralizing, one decentralizing.

Take Terry Winograd’s transformation from AI researcher to human-centered computing advocate. In his pivotal essay, Thinking Machines: Can There Be? Are We? Winograd critiques AI’s foundational assumptions, arguing that artificial intelligence as conceived in the mid-20th century was based on a bureaucratic model of intelligence that prioritized formalized, rule-based reasoning over human understanding and meaning-making. He writes:

Artificial intelligence, as now conceived, is limited to a very particular kind of intelligence: one that can usefully be likened to bureaucracy in its rigidity, obtuseness, and inability to adapt to changing circumstances.

Winograd’s shift toward human-centered computing was a direct challenge to the AI status quo. However, the ideas he developed were not immune to co-option. His student Larry Page took Winograd’s insights and built Google—a company that initially marketed itself as a tool to enhance human knowledge but ultimately became one of the world’s most powerful AI-driven data-extraction and control mechanisms. When corporations consolidate and absorb the “human-centered” views of AI, it’s time to renew human-centered efforts, as many universities, including my own, are now doing.

Beyond asking how AI could have developed differently, we must focus on how it can be restructured now. Some of the most urgent questions are:

• How can AI be governed democratically?

• How do we prevent AI from further centralizing power under corporate monopolies?

• Are there viable non-market, non-surveillance-driven AI models today?

There are many excellent analyses that address power structures, funding sources, and institutional constraints rather than imagining a cleaner ideological starting point.

This is why Tang’s idea of “pre-bunking” is so compelling—it forces us to think about how AI can be designed to prevent harm before it occurs, rather than just cleaning up the mess afterward. Tang offers a framework for practical intervention, not just historical critique. Similarly, Winograd’s call to rethink our interactions with AI invites concrete governance models that challenge the current order.

Once we ask “What if AI had been developed differently?” it’s time to move on to “What can we do now to change AI’s trajectory?” Morozov also invites his audiences to ask this question and intervene in AI’s development today. The conversation at Stanford yesterday made clear that we are not condemned to the AI we have inherited. The AI we deserve is the one we fight to build—through governance, intervention, and sustained public engagement.

The personal essay could be a wonderful written testimony to one’s lived-experience, character, and service to others. Instead, it’s become the clearest evidence of the malaise in college admissions under the prohibition against discussing race openly.

The June 29, 2023, Supreme Court decision against affirmative action exacerbates an already troubled college admissions system, which is notoriously opaque and prone to gamification.

As a longtime reader of college admissions applications, I’d like to suggest some paths forward. First step: Let’s ditch the personal essay:

The personal essay could be a wonderful written testimony to one’s lived-experience, character, and service to others. Instead, it’s become the clearest evidence of the malaise in admissions under the prohibition against discussing race openly. I have an alternative that I discuss below, but first let’s consider how the Court’s majority opinion aggravates this problem.

While conceding that “exposure to different perspectives and thoughts can foster debate, sharpen young minds, and hone students’ reasoning skills,” Justice Thomas rejected that “diversity with respect to race, qua race, furthers this goal.” Instead, he asserted, affirmative action simply places some individuals “into more competitive institutions than they otherwise would have attended.”

Thomas’s characterization of affirmative action as a slippery slope to “a world in which everyone is defined by their skin color, demanding ever-increasing entitlements and preferences on that basis” stands at odds with the remediation goals of affirmative action to pursue justice for those who have been historically excluded in the admissions process. Thomas seems interested in making sure no one is admitted to a college or university to which they wouldn’t have gained entrance without extra attention in the selection process, especially if the conversation is about race.

Many people have asked me whether it is not ironic that the conservative court along with its many conservative supporters, who normally present themselves as advocates for free speech, have arrived at a decision that “muzzles” college admissions officers from discussing race. My response is both yes and no. Free speech remains somewhat undisturbed insofar as the Court made clear that applicants may discuss their own race and how their experience of race and racism may have influenced their achievements. Admissions officers may pay attention to such statements, but the admissions process may not privilege race as a deciding factor.

But while the decision does not explicitly silence conversations about race, it serves this end de facto.  SCOTUS has simply delivered to the federal context the same prohibition against considering race as California implemented 1996 with Proposition 209. A hotly contested decision, Prop 209, is modeled on the Civil Rights Act of 1964, and prohibits state governmental institutions from considering race, sex, or ethnicity, specifically in the areas of public employment, public contracting, and public education. To the extent that this new SCOTUS decision approximates the outcome of the California referendum, the role of race in decisions may end up confoundingly obscured but not eliminated.

Indeed, the California ban has led to an elaborate, convoluted non-conversation about race that nonetheless remains omnipresent. Since 1996, public California universities have engaged in extensive elliptical speech acts to discuss race without mentioning race per se. This engagement with race mainly involves the evaluation of the personal essay as the key to “holistic” admissions which endeavors to assess a candidate’s worthiness beyond their grades and test scores, especially with respect to diversity and “resilience.”

Ten years ago in 2013, I wrote this New York Times piece about the strange discussions Berkeley admissions readers engaged in to “build a diverse class” while not mentioning race. As we debated the merits of each student carefully without saying such things as “is the student Black or Latine or Indigenous?” our supervisor instructed us to follow the pattern of what they called “helpful” personal essays that directly referred to race. All of us expressed relief, whenever the student explicitly mentioned their race, so we wouldn’t have to try to surmise it. We were also taught to identify “stressors” that might indicate the person was from a marginalized group that could include race. Family names and socioeconomics have often been seen as a proxy for race. But, despite all our careful efforts to infer race or marginalization, subsequent decisions to accept or reject a student may have been neither accurate nor just.

The SCOTUS decision merely confirms the habits of our era of “helpful” personal essays where applicants understand they must most loudly and clearly declare their race as well as tales of stressors and disadvantages if they hope to gain entry to universities of their choice. Is this fair that marginalized students be forced to trade on their trauma? Sociology PhD student Aya M. Waller-Bey says no. Given this dysfunctional system, where Waller-Bey and many other former admissions readers believe the personal essay will become more important, what other options do applicants have? Justice Thomas’s strategy to avoid race thinking and its supposedly “entitled” attitudes, has in fact sanctioned an environment of competitive victimization where everyone needs to clearly state how they have survived a world of unjust harms that makes them more worthy of admission than their peers.

Here is a “helpful” example. In a recent admissions cycle, we encountered a student from the Midwest who described her Indigenous heritage (the exact details here have been altered not to reveal the student’s identity). As an applicant, she was attractive on two demographic grounds: First, she qualified as an underrepresented minority, and second, she would “help build the class” to include less represented states in the country. This student with an Anglo-sounding family name, wisely understood she need not elaborate what might have been a highly mixed, privileged identity. Instead, she focused on her service to the Indigenous community and others. From her home address, we could tell she attended a well-funded public school and was probably not poor. It also helped that she had survived several health scares, which showed “resilience” and “growth mindset.” Her essays sounded mature, but spunky and young in a style that may have been ventriloquized by a college writing consultant. With these essays alongside a great alumna interview, good grades, this student gained admission to an elite university. In the end, we took her based on the merits of the rest of her application and a conversation we had about how “likely she was to succeed” and “serve the university mission.”

If you’re queasy after hearing about the process, you should be. So much of the college essay depends on how it persuades readers, the process has become confusing and impressionistic. Here are some ideas for a more principled, sustainable, and equitable admissions solution, one that SCOTUS might also disdain, but not assail:

1. Instead of asking students to write a personal essay in its now routinized style of what Waller-Bey calls “trauma-porn” and competitive marginalization, students should work with high school teachers to prepare a college-level research paper with an accompanying reflection about how the project challenged them personally and intellectually. Many high school teachers already teach a similar assignment as a final or capstone paper. The research paper and reflection can become the focus of teacher college preparation pedagogy rather than the genre of the college essay.

2. When students research a topic of great personal importance to themselves, they avoid trading on trauma, and instead engage ideas. Likewise, rather than writing silly supplemental essays where they are required to show “intellectual vitality” (Stanford finally renamed this vapid category) or “leadership” or “passion,” a student research essay and reflection will demonstrate both deep interest and lived-experience. In this new approach, the student’s teacher will have seen this essay develop over various iterations, be able to testify to its authenticity as well as the quality of the student’s work and character in their recommendations. Switching genres is not more work for high school teachers. It’s better, more appropriate college preparation. Imagine one 1800-2500 word research essay, a 800 word reflection, and maybe 100 word supplement. Not 650 words of trauma plus 4 x 250 +50 characters x 10 of banality and keywords written by college writing counselors or ChatGPT.

3. To help marginalized students write a research essay and personal reflection that are equally excellent as their privileged peers in well-funded high schools, why not have university writing programs expand on already excellent programs like National Equity Lab which also partners with Stanford  Digital Education to fund outreach to marginalized communities? Consider a program where low-income high school students can start working on their college-style research papers early with college writing instructors as their teachers. University writing instructors are great teachers. They care about students, are usually grossly underpaid, and would be keen to participate in an extracurricular or summer college-prep program. At Stanford and elsewhere we already teach both these genres of writing in research and reflection, so the genres transfer to other kinds of learning experiences, especially through revision. College writing teachers, like the high school teachers, will have witnessed the research paper and reflection evolve over several drafts. The outcome would be more honest, accurate measure of who can succeed in college.

   

Are there enough college writing programs out there and staff to meet the demand? There are. At Stanford we have at least 52 writing instructors who can serve local populations. More rural populations can also gain access to local universities, which all have writing instructors, course materials online from elite programs, and local interns who are recent graduates.

No student will have to write about misery unless they want to, because there will be less guessing about who is a marginalized student. Wealthy students can stop pretending and understand that higher education institutions need their full-tuition dollars, especially as these institutions confront the ethics of legacies and other privileged categories.

Proactive involvement of high school teachers, switching genres, practicing actual college writing, and expanding outreach programs will help admissions readers identify highly qualified applicants. Once students from all backgrounds arrive on campus with firmer preparation, they are more likely to succeed and understand how they contribute to the excellence of their university cohort.

Will students still talk about their race and identity? Of course. Not merely, out of what the court derides as “entitlement” nor to broadcast misery,  but because their identities and lived experiences better represent the American public, uniquely equip them to help future generations attain their dreams, and gain claim to what the Reverend Dr. Martin Luther King Jr. calls “the promissory note” of American founding ideals.

Irene Solaiman (left) and Dr. Nakeema Stefflbauer (right)

About 110 students attended. They had already read Ms. Solaiman’s research, heard her podcast at The Gradient, so they were very enthusiastic to read her new Wired piece on the spectrum of choices from open and closed source models. Appreciating Dr. Stefflbauer and her specific focus on global impacts, students' responses to this conversation range from issues affecting their global communities at home as well as the future of generative models on the tech industry. After the interview, Dr. Stefflbauer commented:

“It is all too rare to speak with game-changing ethicists who’ve lived the startup life as well as doing slower-moving government policy work. Irene has the unique perspective of trying to steer the new technology ship away from harm for marginalized communities and local governments, and she understands exactly what a tough balancing act that can be, especially for women of color. 

Before we shared the interview, Irene told me; “I don’t want to shame students, regardless of their choices.” I think that reflects the challenge that several students also raise.

Here are some of the group’s reflections submitted by students after listening:

“Thank you, Irene Solaiman, for this great talk! We loved your nuanced view on open vs. closed source. Please allow us to tell you about ourselves. We’re a group of African American and international computer science students who understand the argument for open source as  addressing the problem of lack of transparency in corporate models, especially where there are inflated corporate claims about “capabilities” and supposedly “emergent properties,”which often turn out to have been in the TRAINING data. We’d like to think we wouldn’t make such claims in industry, since we and everyone else, we hope, has been taught first thing that the data for training and testing has to be separate!!! Even more importantly, there’s the problem that these corporations scrape unconsented data for their models and no one can see it, because the models are closed. No one knows what exactly they stole, especially if they say something like “we use a combination of corpuses like Common Crawl and LAION, which like to claim they’re getting consent after the fact. That said about the importance of corporate transparency, we’re keen to keep our datasets limited access to protect our research and the populations we serve in our countries. Some of us are also working on Dr. Harriett Jernigan’s BlackEnglishes StanfordHAI project this summer and this project is limited to academic researchers to protect the authors of this data and the populations they serve. Closed source can sometimes be the most just!”

“Dear Ms. Solaiman, thank you so much for your wide-ranging talk. For us, as a group of Bangladeshi American and Bangladeshi national students, we were so excited to hear you had just been in Bangladesh and were interested to know your thoughts on the appropriateness of AI in Bangladeshi communities. We serve a variety of communities where AI has been somewhat helpful, but still a lot of the cart-before-the-horse, since development must happen first. For example, AI in agriculture is helping with planning more than actual growing. Likewise, in other fields like disability and education, it seems most appropriate to teach students coding and statistical skills needed for building AI, rather than using it on students, especially not on disabled students. One of us serves disabled students who are learning to code and build AI this summer.”

“Thank you so much for your talk. We are a group of STEM and history students who wonder a lot about what it means “to make AI better?” For our communities, it means teaching us how to use these tools to build beneficial technologies. As for “cultural value alignment,” one of our research projects this quarter has been in trying to mitigate religious bias in models, specifically anti-Muslim bias. Surely, you’re right, there are systemic issues that must be addressed, but we’re less certain of how to do that, than how to build stuff. Our study of decolonial and anti-racist histories and theory seem to have the most immediate impact in our efforts at model design and correction. Our approach might sound incremental, but this is where we feel we can best contribute. As a group of students of color, first-gen, low-income students, safety is also a concern. We still talk about when a Stanford professor encouraged low-income students to protest by shutting down the San Mateo Bridge in 2015, and all those students wound up with felony charges and huge lawyer bills. We believe deeply in the need for systemic change, but most of us need to take safe routes to get there. Too often, even trying to change models is controversial for job safety.”

“Thanks for this awesome talk! Some of us are leaving the university soon and headed to government, so we were very excited to hear your optimism that “governments are now listening.” We definitely feel like we’re at the bottom of a huge mountain we have to climb to get governments to focus on “priorities” like having experts in technology, society, and policy help with the decision-making. So far, in our experience, we’re seeing government here in the US and around the globe be more willing to organize yet another committee to study AI than to dig into much of the already excellent research. AI policy is not as new as government thinks, our mentors were talking about this 30 years ago, and since large models, there’s been a lot of work for the last decade. Also there’s a disparity in who gets heard. The technical men among us at corporations find ourselves getting many more invitations to talk than the women, who also have technical training. There’s also a very LARGE obstacle when trying to serve government: all entry positions are so poorly paid or non-paid, most low-income people of color cannot afford to take these jobs to advise politicians. Stanford has a few summer grants to support us in DC., but not nearly enough funding for the many people who want to be involved in government. There is SO much work to be done in the US alone to update our lagging policies in just about every field, because these large models change the kinds of policies we need in medicine, politics, economics, law and everything else.”

“Dr. Stefflbauer and Ms. Solaiman, thanks so much for this exciting talk. We’re a group of students working on low-resource languages in the developing world, and we really see that our respective countries are lacking policies to help us protect our data. Reading about data sovereignty in class, we know that policies that address our specific communities’ needs are needed. Also, in answer to your questions about “How to reach communities to get consent?” We need NLP researchers from our communities, who are native speakers, to be more involved in our work and educating communities. Masakhane is one great example of university researchers working together with local populations to protect their data and rights and support their languages. European policy isn’t really a model for the developing world.”

“We’re CS undergrads who are very concerned about all the different impacts of models, especially when they harm people. It’s only now that we’re also beginning to understand how these models can harm us too. One thing that struck us in your conversation is the very new danger of damage to our reputations if we’re accused of plagiarism. We think policing and surveillance for model use, especially for coding is wrong. All of us use Copilot and other coding aids now, and writing—we’ve always looked for hints on the internet to get started--policing such is impossible and unjust and we want to know how our institutions will protect us from such accusations.”

“Dear Irene Solaiman, thank you for your talk. We are interested in supporting the third-party regulation, and just want to know what can be done to make sure such regulation sticks and hurts offending corporations where it counts: in their profits. We think third-party regulation is moving way too slowly with all the harms that are being done to marginalized groups, that the technology is getting entrenched, and that power is increasingly centralized in powerful corporations. We’d be happy to work on such types of regulation if we are empowered to make things happen, are well-paid, and protected from retaliation. We all started out in Big Tech corporations. The problems with these orgs are already clear to us. But we see how hard it is to change things and how much we could lose if we raise even the smallest questions. All those fun times in tech are always clouded by not knowing who we can trust.”

After the conversation, Dr. Stefflbauer commented:

“It is rare to speak with journalists who have the technical chops as well as the sensitivity to question the direction of AI –not just in the USA, but also in China and in the Global South. Ms. Hao’s reporting demonstrates how posing questions of fairness, ethics, and moral responsibility help us understand who gets to benefit from AI, who gets to experiment with it, and who gets experimented upon.”  

Karen Hao (left) and Dr. Nakeema Stefflbauer (right)

Here are some student responses:

“Dear Ms. Hao, I am so grateful for your talk to our class. I'm a low-income African student, who was so excited to hear about your journalism projects in Kenya and Rwanda. The thing we Africans want Big Tech to know is that we NEED INFRASTRUCTURE first before tech corporations can start building and offering their AI products in African nations. Right now, we see these corps as arriving with products that are popular in the Global North and tech experiments that fail to serve our populations. Why should we agree to give them our data and resources for such technologies that are not only useless to us, but harmful as well. Africans might do well to follow some of Europe’s policy guidelines to protect local populations, but European templates are not one size fits all for African nations’ diverse needs. We also need to bargain for investments on our own terms and have tech corporations understand we will only build technology that includes us and serves us too. As the Europeans have long declared for themselves, we do the same: Nothing about us without us.”

“As a student from South America, I truly appreciate your comments on how Big Tech arrives in developing countries pitching technologies it has not yet tried on North American and European  populations but nevertheless wants to test on local populations and gather their data from us like lab animals. This kind of tech incursion is happening now in our countries in so many fields from medicine to education.”

“Thank you, Ms. Hao, for your great many insights. I'm an African student who has long seen how China has been more helpful to us than the US. At home we say, “while the Americans give us a lecture, the Chinese build a hospital.” African nations too want to transform like China did in the early 21^st century. We want to keep our own resources, including data to serve our own nations.”

“Dear Ms. Hao, Thank you so much for speaking to us, your work is so important, and we’ve learned so much from your writing. For me, as a Chinese-American with a remarkably similar background to your own family, I too want Americans to understand the widespread belief among Chinese nationals that technology has transformed China’s fate and made their nation a world contender.  I’m curious to what extent the Chinese believe this commitment to technology is worth whatever harms algorithms might bring, or whether some members of the general population, especially students, are starting to raise the same questions as we are? Being bilingual in Chinese and English, I find in my research only CCP affirmations of AI and definitions of AI ethics “as whatever serves the CCP.” I know there must be more diverse opinions than these, and see a slightly different conversation in Hong Kong, but also so many, as you say, who benefit from surveillance and want it. Sad to hear the same dynamic is also true in South Africa, where the privileged who benefit from surveillance want more of it.”

“I loved your talk and feel so inspired! Thank you so much! I was especially intrigued with your idea that college students should take control of our job fairs to help tech corporations more clearly hear the message that if they want our skills and ideas, they also need to address algorithmic and corporate harms to marginalized peoples. We are pitching some ideas in our student affinity groups for our next few job fairs and hackathons.”

“A big thanks to Karen Hao, for this galvanizing talk! I wonder how better to prevent a culture of fear in all tech workers so that we here in Silicon Valley and those in the developing world are protected when we organize and unionize. We also see how some of our unions are disempowered here despite a successful organization and we are looking for better security for all tech workers when they organize.”