A story is told about Harold Alexander, British field marshal in command of the evacuation at Dunkirk. At the end of the workday he had the habit of taking any correspondence still in his “in” tray and simply putting it in the “out” tray. When his assistant once asked about it, he said “You’d be surprised how little of it comes back.”

That sums up my approach to the winds of educational change. When a new technology appears and people are asking themselves “what about my teaching should change?” my instinct for my own teaching is “change nothing,” partly because I suspect that the excitement is overblown, and partly because I’m happy to let more venturesome people make a change first, and learn from their experiences.

But that strategy can’t be applied in the case of AI. In 2023 I, like every other educator, realized that I had to respond to student access to large language models.

Administrators have had a few years to think through how teachers should change classroom practice to meet the new reality, and many districts have posted guidelines. The policies I have seen have much in common, and I think they are wrongheaded.

(Throughout this post I’m referring to students’ independent use of AI, not to educational software nor to teachers using AI in their work.)

There seems to be widespread agreement that there’s a problem, and even on how to characterize it. Learning is a product of engaging in mental work, and students may use AI to replace that mental work and hence, not to learn. The question is what to do about it.

Almost no one advocates for “no restrictions on independent AI use” nor for “No independent AI use by students. Period.” The overwhelming majority have taken the position that students should use AI, but there should be limits.

Mashable recently ran an article summarizing the response of the US’s largest school districts. Here are the one-sentence versions, as I read them:

New York City: students are allowed to use AI for basic “research, exploration, and creative projects.” There must be educator oversight.
Los Angeles: No AI use under age 13. Over 13, students can use AI to brainstorm and to edit their text, but students must produce text. Students must cite what AI contributed.
Chicago: Students can use AI for tasks like brainstorming and summarizing. They are encouraged to use AI as a study partner.

The policy of my home county (Albemarle, Virginia) is similar. There are published categories of AI support from which teachers may choose.

In short, teachers are given a good amount of leeway to allow students to use AI or to forbid it. There is, however, little or no guidance about how that choice should best be made.

These policies share a vibe. The goal is that students use AI as an associate in their work, but the student is still very much the lead. The second goal is that students be honest about exactly what AI has contributed.

This lead/associate orientation makes sense for life after graduation. Once a student is in the workforce, they will use AI to produce products: new ideas, reports, and so on. A guiding principle of “Use AI, but let me see your work” makes sense.

This is not, however, an optimal guiding principle for students. Students produce products, but the product is seldom the point. No one wants to read the papers they write, and the math problems they solve have no practical value. Teachers have them do these tasks because they provide practice in cognitive abilities that we think matter.

Writing is an especially interesting example. Writing a substantial paper requires integrating what you know across multiple sources, formulating a detailed argument, articulating your thoughts precisely, anticipating what your reader knows and can understand, and more.

For many educators, writing papers was the only tool we had to ensure that students engage these cognitive processes. AI took that tool from us, and we don’t know of a good replacement.

This loss was unprecedented. The closest analog might be math problems and calculators or Google Translate for foreign language. But in those cases a teacher could still remind students that they would not have these tools available during in-class assessments. I can’t give my students an in-class assessment that is cognitively comparable to writing a ten-page paper.

So the point of assignments is the mental processes required to complete them, and the point of the mental processes is learning. That seems to suggest a simple litmus test for the use of AI. Artificial Intelligence tools should not substitute for tasks wherein students would benefit from doing the mental work themselves.

So…do students learn when they brainstorm? Do they learn when they edit their work? Of course they do. This principle indicates that students should use AI only when doing the work themselves would yield little learning. This guideline echoes how most of us think about calculator use. Once you’ve mastered arithmetic operations, there’s no benefit to hand calculations when you’re solving an algebra problem. But if you’re still learning algebra, don’t use the algebraic functions. The slogan might be:

“Only use AI for what you already know how to do.”

I can see two objections.

First, can’t AI scaffold learning? Part of the learning process is doing things badly. Can’t AI offer pointers for improvement?

I see two problems with this idea. The first is that the novice is not going to be a very good partner to AI in learning. The novice doesn’t know enough to ask good questions. He will give the AI system a vague prompt about the goal of the exercise whereupon AI will sharpen it for the user. Essentially, it will chivvy the user toward the polished product, and the novice will simply accept the suggestions. (Remember, I’m talking about independent use of LLMs, not about a product custom-made to provide instruction. That’s a different matter.)

The second problem I see with AI-as-scaffold is motivational. Sure, students could learn from AI by asking it to critique their work, to generate counterarguments, and so on. And I’m sure some would, some of the time. But I think it’s asking a lot of students—or better, asking a lot of human nature—to expect them to take a much harder mental path for a project in a typical curriculum, even if it is in the interest of their own learning.

A second objection to the principle might be this: even after you reach competence, you still need practice; that will deepen and refine the skill and/or knowledge. For example, if a ninth-grader can write a good paragraph, is it now okay for AI to write paragraphs for the student?

If students are competent, a teacher should ask whether doing the task themselves is still part of what students are meant to learn or practice. When students are writing, editing, reasoning, problem solving, or brainstorming (for example), the mental process is usually the thing. You don’t want the help of AI because it replaces the mental work that contributes to learning.

The alternative is that AI is replacing a task that merely supports the actual mental work you want the student to do; the replaced task was soaking up the students time and energy and providing no benefit. That would be the college student hand-calculating long multiplication problems required for her engineering problem set.

For this reason I add the word “well” to the candidate principle: “Students should only use AI for things that they already know how to do well.”

Now of course there’s a judgment to be made here. A physics teacher might think “It’s great that AI will polish my students’ writing for their lab reports“ whereas someone else might argue “your students need to learn to write in your class too.” But of course this problem is not new. We’re just seeing the issue in a new guise.

We’ve all heard “They’re going to have to learn to use AI.” But I think that slogan applies to how students will use AI at home and in the workforce, where the product is the thing. For students, learning is the thing. That’s how I come to the principle, “Students should only use AI for things that they already know how to do well.”

This education product seems to play to one the current crop of LLM’s core strengths—writing text. “Learn Your Way” is designed to rewrite textual educational content to tune the material to individual students. The published information makes it seem like it’s off to a stuttering start, but I think the idea has real promise.

The goals is that the user can upload material (such as a textbook PDF) and the system can perform the following changes

1. Rewrite it to match the user’s specified grade level

2. Rewrite it to match the user’s personal interests

3. Generate the material in multiple formats, including interactive text, narrated audio lessons, mind maps, and adaptive quizzes.

There has been one not-very-good pilot study of efficacy (more on that below) so we are mostly left to speculate about the likely usefulness of these features, and speculate as to how well Learn Your Way implements them.

Simplify: Can LLMs simplify texts while retaining meaning? It’s a capability already available on most or all of the LLM chatbots, but empirical data on how effectively they do so are sparse. Two things seem predictable. Current performance will be uneven, and performance will improve with new models.

On the one hand, making this feature easy to access may pay off. I expect there may be times that a teacher wants students to access a topic via a text and cannot find one that is not beyond the reading capabilities of most students. And it could help struggling readers and second language learners access academic content that would pass them by.

On the other hand, this feature must be used judiciously, of course. Tim Shanahan has written persuasively about the need for students to grapple with texts they find challenging and also evidence showing that too many teachers given in to the temptation to assign texts that can be read without much difficulty, instead of encouraging students to work through the problem and teaching them strategies to do so.

Alternative representations: Translating ideas into different representations is foundational to instruction; teachers are always thinking about what students should experience to make an idea click: should they hear an explanation, see a figure, execute an activity? Alternative representations certainly can help, but they can also distract and they require instruction; students must be shown how to connect one representation with the other, especially when they are novices. It’s not clear to me whether Learn Your Way will discriminate between content that would benefit from a graphic representation, a mind map, a figure, and so on, or just provide alternatives upon request. The website provides all.

(The mind maps I saw needed work. Every one used a hierarchical representation and was more like a standard set of notes than map.)

Personalization: Perhaps the most interesting feature is the ability to tune the content to a topic that interests the student. If a middle schooler is to learn about, say, the immune system, and the student is passionate about gaming, you can tell Learn Your Way to rewrite the text to incorporate the student’s interest.

Every teacher has tried to pique student’s interests by connecting to-be-learned content to topics that already interest the student; make your math problems about Taylor Swift, and students will eagerly simplify those equations! One problem in implementing that strategy is that some of your students love Taylor Swift whereas others can’t stand her. (And for some, it’s a painful subject since the release of Life of a Showgirl.) Learn Your Way solves that problem—each student can choose their own topic.

But even for students who love her, inserting Taylor Swift’s name into a standard word problem about buying watermelons is not enough to make students interested in the underlying math. Maladroitly done, the real meaning of Taylor Swift to the student is not made intrinsic to the new math material. She is mere window dressing.

Learn Your Way does not seem to solve this aspect of the problem. Consider this example of a text on “atoms and molecules” for a middle-schooler who likes books.

The original text used compared the size of atom to a spider’s web, and the Learn Your Way has changed it to the page of a book. A middle schooler who says he “loves books” didn’t say he loves books because he loves bindings and glue, and he doesn’t marvel at the slender grace of pages. He loves the excitement of learning new things, and/or the feeling of getting lost in a narrative. Learn Your Way’s example highlights an irrelevant property.

Or consider this example from the website, which offers a text to teach principles of learning to a high school student who loves skateboarding.

This example is not going to spark recognition in a student—“totally! That’s what skating is like, there’s an alarm and then you get wet!” Skating in this example is window dressing. (Worse, the example is inaccurate. This is operant conditioning, not classical conditioning.)

I wanted to try Learn Your Way on my own examples, but I can’t access it. So I asked Gemini, which I recognize is not really fair—it’s not trained for this specific function. But let’s hope that Learn Your Way is better than Gemini. I asked for about 10 examples of classical conditioning in a number of domains, and always got similarly surface-level connections between the concept and the domain. For example, when I asked for an explanation of classical conditioning that would interest a gemstone collector, the CR was a the click of a jewel case, and the US was the joy of seeing a gem. (Also strange—Gemini always provided examples of emotional conditioning.)

Still, I think the idea of having an LLM rewrite a text has promise. As we’ve all heard a million times, you get better results if you give better prompts. Learn Your Way may not be able to extract deep structure in a way that allows it to map an example to a novel domain, so you end up with a shallow example. If the user specified the deep structure, it would probably do better.

If Learn Your Way cannot be modified to appreciate deep structure and come up with better examples and analogies, it could still be useful if authors wrote skeletal texts with clear specifications of deep structure which Learn Your Way would then flesh out with examples and analogies from domains that individual users specified.

That scenario would solve another potential problem—copyright. Currently Learn Your Way allows you to upload a pdf for change. The website pretends that we’re all going to limit ourselves to open source materials. I’m no lawyer, but I wonder whether Google might not be liable for making copyright violation so easy. (Candidly, I hope so.)

Learn Your Way strikes me as in need of significant development, but a potentially useful classroom tool for teachers.

The not-very-good pilot:

Google reports a pilot study of sixty high school students—so we have just 30 students in each group. They were given an open source chapter on brain development, either the original or transformed by Learn Your Way. A content expert generated a test to be given immediately after reading. Another was emailed to students three days later, and students had had four days to complete it. (Which seems like it introduces a lot of variability in the delay, and the experimenters did not report data regarding when participants in each group submitted the test). Subjects were to take between 20 and 40 minutes reading and studying the chapter. (Study time data for the two groups also not reported.)

There was a modest but reliable difference in test outcomes.

You can try Learn Your Way here, but note that there is a waitlist, and you must be approved. I applied but had heard nothing after four days.

· A failure to bounce back from the pandemic.

· Record rates of absenteeism.

· The amount of time students spend on smartphones and social media

· Adoption of the Common Core State Standards.

· The recent increase in student mental health problems.

I am surprised that no one mentioned student conscientiousness. A viral Financial Times article reported that young adults have shown a precipitous drop in the personality characteristic over the last ten years.

The effect may be overstated in the article, but even doubters put it at d=0.40.

Other surveys indicate that student engagement is down, and teachers feel they see it in classrooms. Eighty-three percent of 3-8^th grade teachers think student reading stamina is down since 2019, and 82% of teachers think that, compared to 10 years ago, students are less able to be independent and direct their own learning.

Which of these factors is responsible for the decline in NAEP? Looking at the list of possible contributors makes me wonder whether many might be symptoms, rather than causes.

What might make students lose stamina for their work, feel less engaged by it, seek escape in their phone for hours each day, and nevertheless feel anxious and depressed?

These sound to me like the actions of students who would like to care about the world around them, to be invested in the future, but have no faith that caring matters. It sounds like the actions of people suffering a crisis of confidence.

Why would students today think that the world is a place of threat, rather than one of opportunity? And why would they feel powerless to effect change?

Consider the messages that a typical student hears from the adults in their life. Members of the opposing political party are not merely wrong, they are evil, and actively working to destroy our way of life. Children are warned about school shootings. They are warned about scams and sexual predators online. They are told that the planet is doomed due to climate change, or that politicians and scientists—people we would hope should be respected—are perpetrating a hoax. College is unaffordable, and a college degree doesn’t help anyway. If another pandemic comes, they can rest assured it will not bring the country together, but will be an occasion for further conflict.

It's not just that things are bad, children get the message that our problems are not fixable. Surveys show that the adults around them don’t trust the institutions that are supposed to help us solve these problems. We do not trust Government. We don’t trust businesses. We don’t trust the media.

And at a personal level, adults think they cannot succeed in our current system. A recent survey showed that more Americans than ever think the American Dream—the idea that anyone, of any background, can succeed through hard work—is dead, or that it never really existed.

But I think the problem is still worse. It’s not just that things are terrible, and that adults fail to express hope and guidance for a way forward. The adults don’t even seem to be in charge.

I remember an interview with pediatrician and parenting expert T. Berry Brazelton in which he pointed out a subtle but important reason for household rules. The purpose of discipline is not just to help children develop self-control (which Brazelton championed). At a deeper level, rules reassure children. Although they might say they want unrestricted freedom, children know that they don’t know what they’re doing. Boundaries show children that adults know the right thing to do, and are in control.

Children don’t want our leaders to set boundaries, but they do want to feel that the people in charge are taking their jobs seriously and seem competent.

Yet consider how our leaders responded to the NAEP score report. Tim Walberg (R-MI), Chairman of the House Education and Workforce Committee bravely averred that low scores were Joe Biden’s fault. Bobby Scott (D-VA) of the same committee said the decline only shows how “reckless” the Trump administration’s education policy is. In other words, members of the very committee in the House that is charged with taking action on educational matters, the adults whose job it is to develop a plan, disavow responsibility for doing anything. It’s someone else’s fault. (A senate committee proved no more constructive.)

What really caused the decline in NAEP scores? Obviously, we don’t know. The factors that people have pointed to (including the one I’ve described) are dynamic, they’re difficult to measure, they’re intercorrelated, and we can’t manipulate them for experiments. What’s more, performance on the NAEP is, as psychologists like to put it, “multiply determined,” meaning lots of things influence it, so it’s probable that there’s not a single reason for the NAEP decline.

But in education, we don’t have the luxury of taking action only when we are certain that we understood cause-and-effect relationships. I’m a strong supporter of bell-to-bell phone bans not because I’m certain that it will boost student achievement, but because I’m fairly certain that phones are a significant source of student distraction, and the cost of limiting student access to their phones seems minimal.

In the same way, I think students need to feel some buy-in, some efficacy in the system. They need to feel that effort is repaid. I don’t know whether restoring some sense of efficacy will help student achievement, but it might and the downside seems minimal.

This task is obviously a very long pull. Perhaps I’m grasping at straws, but I’m heartened by the fact that a candidate for Governor in my home state routinely trumpets the fact that she was voted the most bipartisan member of the House from Virginia, and she’s leading by 11 points. Maybe extreme partisanship has had its day.

While I hope for better things at the national level, I can try to remember that I can more easily have a local impact. Many of my students need a particular type of encouragement. You matter. You can make a difference in the world. I am rooting for you and I’m ready to help you.

The most glaring example is an LLM chatbot serving as a tutor, which I think is based on faulty assumptions about human motivation. But that topic requires a lengthy treatment, so I’ll leave it for another day.

But here’s another example: an LLM chatbot as a partner in the writing process, an offered in a number of places, for example, this popular guide published by Elon College. It’s sometimes framed as an exercise in realism; once out of school, students will, of course, use an LLM chatbots, so it makes sense for them to use it now, just as they will in the future. Good writers will not have LLMs write for them, but will make wise use of their capabilities.

There are assumptions in this tact. One is the facile expectation that doing the same task now is always the best preparation for doing that task later; in other words there are “working with LLMs” skills that must be burnished. The plausible alternative is that the “working with LLMs” skills are trivial, and skills that really need development relate to research, thinking, and writing. In which case students would benefit more from an assignment in which they were responsible for all aspects of the work.

Or perhaps the “partnering” strategy is a nod to the reality that students are going to use LLMs anyway, so partnering is meant to put some guardrails on that use. That assumes that this assignment will make it less likely that students will not use it in proscribed ways. Would it? I have no idea and I don’t think anyone else does either.

Here’s one more example. When educators are advised to have students partner with an LLM chatbot in the writing of papers, they are almost always told that students should cite the LLM, just like any other source. But this advice shows a narrow understanding of the purpose of citations.

The motivation behind telling students to cite an LLM chatbot is academic integrity; you cite your sources to give credit for ideas you didn’t generate on your own. You are avoiding plagiarism, and that’s a good purpose.

But you also cite sources for the sake of credibility. You are showing where your information comes from with the goal of assuring the reader that the information in your argument is trustworthy. But of course, citing an LLM doesn’t really serve this purpose, because you don’t know where the information came from. You’re citing an aggregator, the information that went into the aggregator is cloaked, and the method of aggregation is not understood by the reader. (There are LLM chatbots that cite sources, so far with mixed success.)

Another reason we cite sources is for verification. If I want to challenge the accuracy of your paper, one thing I might do is look up your sources to see if your attributions are accurate. But of course that’s a tricky matter with an LLM, as the same prompt can yield a different response.

My aim here is to point out that an LLM chatbot is a cognitive wild card. The effects of including it in assignments may be subtle and difficult to appreciate. That suggests that educators should be thoughtful, deliberate, and cautious in their use of LLM chatbots in the classroom. Which runs counter to one of the persuasive techniques of enthusiasts: “you don’t want to be left behind!” We’ve bought that argument more than once in the last twenty years, and it hasn’t served us well.

The other persuasive technique is to argue “it’s already here—deal with it.” I would rephrase that as “it’s upon us, so we must deal with it,” and there is a world of difference in that small change. I haven’t forgotten the people who made the same argument advocating that educators embrace the use of smartphones in class, one-to-one Chromebooks, and YouTube, Twitter, and TikTok as learning platforms. I remember, and I think they know I remember.

The book makes a simple point: leveled reading instruction is a bad idea. It’s not that children can’t learn anything with such instruction. The problem is that it’s inefficient, and is suppressing potential reading achievement. The children who learn the least are the ones who most need instruction and support.

That sounds, indeed, like a simple point, one that might be covered in a blog. But those familiar with Shanahan’s blog posts know that he is never satisfied with a cursory review of evidence. There are very few researchers as steeped in the literature as he, and he puts his scholarship to work in this book.

Shanahan makes a persuasive case that leveled reading has been dominant in US classrooms for decades, and shows the mistaken theories of child development and of reading that gave rise to the practice. It also has its roots in, as he puts it, “well-meaning efforts to make education attractive by making it easy, and to differentiate instruction to meet the needs of all students.”

He also reviews the (sparser than you’d expect) research literature on leveled reading, and shows that the promised advantage of matching children to their instructional level does not materialize.

I had always thought that the notion of “levels” was incoherent, as it assumes that content is irrelevant. A child at level X is presumably a level-X reader for all texts, whereas in fact the match between the subject of the text and the reader’s background knowledge is a strong contributor to comprehension. Shanahan marshals several other reasons that levelled reading does not square with theories of reading, but the most telling for me is the most painfully obvious.

Well, obvious to me only after Shanahan pointed it out. We learn when we’re challenged. If you encourage children to read texts that they can read comfortably, you limit their exposure to complex language, new vocabulary, and rich content. Yes, these books are harder to read, but you address that problem via instruction and support, not by making the task easy enough to manage without being taught anything.

Chapters 6 through 9 of the book describe Shanahan’s evidence-based views on the best way to teach challenging texts, from prereading through decoding, fluency, and comprehension. There’s also a chapter on motivation—Shanahan recognizes that it’s harder to motivate children if you are asking them to read more challenging texts—as well as a useful section that answers common practical questions.

My hope is that every elementary teacher, every reading specialist, every SPED teacher, and every administrator will read this slim little volume. Scholarly though it is, Shanahan walks the reader through the argument with concise, accessible prose. And it’s simple, powerful message should be a central theme in US reading instruction.

There is some recognition that this term is being used in different ways by different people, and that doesn’t seem like a great idea. Here’s why I think it’s a problem.

• A. There are many lenses through which to understand the world. Science is just one lens, and it can’t be applied to all aspects of our world. When it can, it’s exceptionally powerful as a way of understanding, predicting, and controlling what happens. Saying that something “is supported by science” is seen as a strong argument for whatever interpretation of the world is being offered.

• B. A is only true when science is properly deployed. I don’t just mean that experiments are well conducted, data properly analyzed and so on. Education can draw on science by different methods, and those different methods justify different conclusions.

So…what are the different ways that science can be used (specifically as it applies to reading), and what sorts of conclusions are justified?

To start, there’s an important distinction between basic science and applied science. Basic science seeks to describe the world as it is. Applied science tries to change the world, not just describe it. It entails some goal. For example, physics is a basic science that describes how things move and interact in the physical world. Civil engineering is an applied science that uses physics (and other basic sciences) in dealing with the design and construction of the built environment.

“The science of reading” can refer to:

1. the basic scientific effort to describe what is happening in the mind and brain of a proficient reader when they are reading or when they are learning to read. For example, we might note that there is abundant research on the characteristics of word representations that support reading comprehension, and conclude that we should try to build those representations in students; we take a basic science finding and use it to inspire an applied science goal.
That’s a fine idea, but it’s obviously incomplete. The research revealed something about the mental architecture of good readers, but not other components of that architecture that might interact with word representations, nor how those word representations came to be. In this scenario, basic research can be useful for inspiring goals or methods in an applied setting that we might not otherwise have considered, but there’s no guarantee those methods will work or even that the goal is useful.

2. Using scientific methods to compare the efficacy of different methods of teaching decoding, motivating students to read, improving comprehension, etc. For example, both the Pizza Hut Book It program and Accelerated Reader claim that their program boost reading motivation. These claims were not inspired by any finding from basic science—someone just thought the program was a good idea. We can evaluate the claim using experimental methods from science: we can compare the reading motivation of students who undergo Book It, Accelerated Reader, or neither.

In general, a sound conclusion from method #1 is very broad, but it’s uncertain. Basic scientific findings might (eventually) persuade you that, to be a good reader you must have robust word representations. That’s an important conclusion because it applies to all children. But it tells you nothing about what practices get you those representations.

A sound conclusion from #2 is narrow but certain. It can tell you that Book It doesn’t shift students’ reading motivation. That’s highly practical, but it applies to one use of one program.

David Daniel and I introduced another way to think about these two applications of science. Method #1 can lead to “must haves.” These are principles of learning, motivation, emotion, etc., that are so foundational you can’t ignore them. We compared these to vitamins, minerals, etc. that make up a healthy diet. Examples of “must haves” in education might be “practice is necessary for proficiency of mental processes,” or “domain knowledge is necessary for critical thinking.”

Method #2 leads to “could dos.” To continue the analogy, these might be foods that are rich in important vitamins and minerals. Nothing says you have to eat them, but they are known, proven ways to get something important. Examples in education might be using retrieval practice to commit domain knowledge to memory, or cycling between concrete and abstract examples to more quickly understand deep concepts.

So when we talk about the science of reading, it’s important to consider why a finding or claim is considered scientific. If we’re talking about basic science, it might be a “must have” but only if we’re convinced of its centrality to reading. Otherwise, the finding might be an inspiration for something to try that we hope will improve instruction.

If the scientific finding under discussion is an application—something we’re testing to see if it improves some aspect of reading—then it might tell us whether a particular practice is more useful than some comparator, in a certain context. With enough research we might be convinced that it could take the status of a “could do,” that is, a generally useful tactic for a particular purpose.

To close, let’s briefly consider what the science of reading is supposed to make people do. I’ve noted that, in my opinion, scientific findings have lead to some “must haves”—things that, given some goal, you really can’t forgo. Those must-haves typically leave flexibility of teaching practice. The “must have” is in the student’s mind, not in the teacher’s actions. For example, fluency is a “must have” for reading, and reading is essential to develop fluency. That says nothing about how to get kids to do a lot of reading, just as knowing that calcium is needed for health doesn’t commit you to eating any particular food.

At the same time, the “could dos” could be relevant here. If there is a type of teaching practice that’s been shown to be effective in moving towards a goal, then using one that has not been shown effective with the same goal in mind might be defended, but I think it would call for some justification. You don’t have to drink milk to get your calcium, but if your replacement is cotton candy, you have some explaining to do.

In the past few weeks some articles in high-profile publications have alerted people outside of education to a problem teachers have thought about nearly continuously since late 2022: students use large language models to cheat.

New York magazine published a piece titled “Everyone is Cheating Their Way Through College.” The New Yorker asked “What Happens After AI Destroys Writing?” and David Brooks devoted a column to a not-yet-published (and rightly criticized) study purporting to show that you don’t really think much if you lean on ChatGPT as you produce written work. (This is one of those instances where the study has some problems—it’s underpowered, there’s debate about whether the EEG analysis is appropriate to the use it was put—but the broad conclusion is probably right.)

Despite this concern, tech companies (with an invitation from the White House and an assist from, improbably, the American Federation of Teachers) are pouring resources into insuring that AI makes its way into schools more formally, and is not restricted to off-the-record homework assistance and furtive exam help.

This is old stuff, tech companies offering an unfounded, glittering picture of the possibilities that their products can bring to education. It’s often delivered with an undertone of threat (those who don’t sign on will be left behind). Never mind that the glittering future was not delivered with

1) One-to-one laptops

2) Smartboards

3) Massively Open Online Courses

4) Open Educational Resources

5) Flipped classrooms

I mean, we have Kahoot, but whether it’s all been a net positive is at least open to debate.

This time, tech companies are not even bothering to paint the shiny future for us. Apparently that’s not needed, but I am no more optimistic that LLMs will “change everything” given that these other innovations delivered much narrower, weaker benefits than we were promised.

So. Cheating for sure, and benefits that are unnamed and uncertain.

This all sounds pretty grim, but I see a possible silver lining here. Addressing the cheating problem created by LLMs could improve instruction. Here’s why.

I hope I’m right in thinking that forgoing writing assignments is a non-starter. ChatGPT can’t kill the school essay because it’s too useful to us.

Educators cannot directly influence learning. Learning is a product of student thinking and we can’t directly control student thought. We set particular tasks expecting that the tasks will guide student thought in particular ways, and we think we know which skills and knowledge will result. I give exams in my Introduction to Cognition class, and those exams mostly test memory for vocabulary terms and new concepts. I use this assessment because I want students to commit basic terms and concepts to memory, which is a good goal for an introductory class.

But my advanced seminar, High Level Cognition, includes no exams because my goal is not that they commit more content to memory. My students are all majors in either psychology or cognitive science, and they have a number of introductory courses under their belts. It’s time for them to conduct their own research, and to assemble ideas into a comprehensive, deep analyses. That’s an appropriate goal in many classes, and I don’t know of a task superior to writing an in-depth paper to assure that students engage in that cognitive work. That’s why I say that we can’t forgo writing assignments in school.

So how do we assure that students are actually doing the cognitive work, and not offloading it to a LLM? Software products designed to detect AI writing have not been successful. If they were, I expect students would simply tweak their output so that it evaded the detector. (And before long there would likely be another product promising to relieve students of this minor task.)

Another solution is to ask students to write using Google Docs, or another platform where the instructor can review drafts. That practice seems like no more than a nuisance for a student wanting to use an LLM. You have to tell it to write an incomplete draft with some mistakes, then type it into Google Docs, then later get the LLM to improve the draft, and so on.

I know of one way to save the essay: conferencing. Meeting with each student where they can explain and defend their ideas and their choices in how they organized and expressed those ideas, seems to me the most likely way to encourage students to generate these ideas on their own. At the very least, they must understand and be prepared to explain and defend what ChatGPT has written for them.

Before students had access to LLMs, I required conferences in some of my classes, and made them optional in others. I found that feedback is more effective because it can be more finely tuned. I can ask the student “what were you trying to do here?” And my feedback is much more carefully considered. Every instructor wonders how many students read and really process written feedback on written work, and how many glance at the grade and ignore the rest. If the student is there with me, they will not only take the feedback in, they will think about it.

In addition to providing more focused feedback that’s more likely to be attended to, you can also respond to the student’s emotional needs. You can encourage those who struggled, better explain a grade to an angry student, assure a student who thinks poor performance means they don’t belong in your class, or persuade the just-tell-me-what-you-want student to take a risk.

Conferencing does not take much more time. I do not grade the essays and then meet with students to provide feedback. I read the essay for the first time during the conference and explain to the student how I am responding as I read. (“See how the last sentence of this paragraph made me think you were going to talk about this? But then in the next sentence you talk about that….”) It’s hard to recreate that feeling 48 hours later.

I’ll also mention that the near universal response from my students is (1) they find these conferences stressful and (2) they find them extraordinarily useful.

LLMs are prompting me to make conferences required in all my classes. That’s the biggest boon to my students I’ve seen from LLMs thus far.