The idea isn’t to find contrived ways to engineer an 80% success rate. If that was my only goal, every Do Now would look like this:

I like this advice because I find it’s connected to many other areas of teaching. 80% isn’t a magic number, but striving for a high success rate pushes me to do all sorts of things in my classroom that are helpful for learning.

I want to start with a non-example: what this is deliberately different from.

It’s common to see posters in classrooms that say things like, “the power of yet: I don’t know this...yet!” or extolling the value of perseverance. And yes, perseverance is a good thing. I don’t have an issue with the signs themselves. But in many cases these posters go along with an approach to teaching: the teacher uses a curriculum, and for many students the curriculum asks them lots of questions they don’t know. So the teacher encourages students with messages about a growth mindset or perseverance in hopes of getting students to persist even though the work is clearly challenging and discouraging for students.

The high-success-rate approach says something different. In that situation, the teacher needs to make a change to increase the success rate. The primary job of a teacher is not to encourage students to persevere when things get hard. The primary job of the teacher is to engineer conditions where students are successful 80-90% of the time, inasmuch as that is possible.

What Does This Look Like?

I mentioned that what matters is not the success rate itself but all the other teaching practices that this idea has led to. Here is a rough list of ideas I’ve found helpful that are downstream of striving for a high success rate.

Check responses of all students. Teachers should ask students lots of questions, and they should also check the responses of all students as often as they can. If I’m not seeing or hearing every student’s response, I can’t judge their success rate. Ensuring every student answers is a great way to provide accountability and make sure students aren’t passively following along. There are lots of ways to do this. Some of my everyday strategies are mini whiteboards, turn and talks, choral response, and sets of written problems where I circulate around the room and look at the same 1-2 answers from every student. You might have different strategies. Lots of things work here, as long as every student has to answer and the teacher can gather a bit of data on whether students are successful.

Provide more practice. How much practice do students need? One simple answer is that students should practice until they reach a high success rate. Your curriculum will have a certain number of practice problems. That number might not be right. Providing more practice until students reach a high success rate is a great way to improve long-term retention. This can be inconvenient: practice may extend beyond the boundaries of a single lesson. Teachers have to make time and find resources. But that additional practice is often essential.

Judge a curriculum by the student success rate. Along those lines, something lots of schools wrestle with is the extent to which teachers should follow a curriculum. I’m in favor of high-quality curriculum, but fidelity to a curriculum can become a mindless march without much learning. A high success rate is my benchmark for whether a curriculum needs to be modified or supplemented. If students are not reaching a high success rate, I might need to add more scaffolding, adjust the sequencing, or make other changes. If the success rate is too high, consistently above 90%, then the curriculum is too easy.

Think of every student. Every student is an individual. Here’s one thing that is not a high success rate: give students an exit ticket, 80% of students get it right, and say that’s good enough and move on. The goal is that each individual student has a success rate of 80-90%. Now in real classrooms this is probably impossible, but it’s the goal. The goal isn’t to get 80% of the students across the finish line and ignore the rest. I need to constantly ask myself: what am I doing to support students who have not achieved a high success rate?

There will always be some students who get more questions right. The goal is an 80-90% success rate, so this is a reminder to include some challenge questions so that no student is getting every question right every day.

Similarly, there will always be some students who have a tougher time in school. This is a reminder to ask some simpler questions as an on-ramp to build confidence. I often think I’m asking easier questions early in a lesson and realize that some students are still having a hard time, which is a good moment to recalibrate.

Include lots of review. One way to do that is to include lots of review. I think this is one of the most common mistakes in teaching. Review of what students learned yesterday, last week, last month, last year. New learning always builds on something students have learned before, and those connections are clearest when students get a reminder of that prior learning. Review is good in and of itself, and it’s also a good way to increase students’ success rate.

Motivate students with success. A high success rate also plays a role in motivation. If students are constantly getting questions wrong, they are less likely to want to answer the next question. Motivation is more complicated than that, but it’s not a bad place to start, and trying other motivational strategies while students mostly get questions wrong and feel dumb in class is often futile. It’s not possible to engineer this perfectly for every student in every moment of every class, but a good rule of thumb is to start each class with a bunch of tasks that have a deliberately high success rate to build momentum.

Break complex ideas into small chunks. One piece of advice about teaching is to break complex ideas down into smaller, more manageable chunks. This isn’t bad advice, but it’s not very specific. When do you stop? At what point are the pieces small enough? My metric: if I’m struggling to get students to a high success rate, if we’re practicing and they just aren’t hitting that bar, I’m probably not breaking concepts down into small enough pieces.

If the success rate is low, change something. That last point gets at a larger idea: if I’m teaching something and students aren’t reaching a high success rate, something is wrong. I can’t always say exactly what it is, but striving for a high success rate serves as a check on my teaching. Teaching is an empirical science. Teaching is only effective if students learn something, if they can answer or produce something that they couldn’t before. Striving for a high success rate keeps me connected to that empiricism.

Prioritize. Finally, some teachers will say that all of this stuff sounds nice but they just don’t have time. They have to stay with the pacing guide or they won’t cover everything. I agree! I also do not have enough time. Here’s something I think more schools should do: make the conscious choice that, since there isn’t enough time, they will prioritize a few key skills. That’s where I started. I decided that, in 7th grade math, equation solving was the most important skill, and that I was going to spend more time on equation solving. This meant I had to cut some things, or give them a quick, superficial treatment. That’s fine. One of the lessons for me was realizing that, if I’m not achieving a high success rate with any topic, that’s a ton of wasted teaching. If I pick some things and prioritize them, I can feel much more confident that students have learned the most essential content in my course.

Another Non-Example

I want to close with another non-example. Someone might read this post and think I want to give students easy work, that I have low expectations and want to avoid challenging students. That’s not it at all. The success rate I am striving for is a ceiling as much as a floor: a success rate of 100% is almost as bad as a success rate of 50%.

This is another reason not to follow the recommendation of a certain success rate blindly, without regard for everything else that goes into teaching. My job is to teach, as best as I can, the standards as they have been set for my students.

I find this advice helpful across content areas: reading, writing, math, social studies. It’s not a magic elixir, it’s broad advice that points me in a helpful direction. Do I need more review? More practice? Break ideas into smaller chunks? Provide more challenge for my strongest students? More scaffolding for strugglers? The exact tools vary by content area, and there isn’t always an obvious solution. But the idea is simple: if the success rate is low, I need to make a change. That broad direction is a helpful one, and I think striving for a high success rate is good teaching advice.

No subject is purely one or the other. I would characterize math as being mostly a ladder, with some spiderweb qualities. I would characterize literacy as being mostly a spiderweb, with some ladder qualities.

I’m writing today about atomisation, a teaching technique best suited to ladders.1

Atomisation

Atomisation is a reasonably intuitive idea in math class. Break a math skill into small chunks, teach one chunk at a time, and build bit by bit toward larger ideas.

In literacy, most topics don’t lend themselves to atomisation. But some do, and for those it’s a helpful tool to have handy.

Our first literacy unit this year is personal narratives. On day one, the teacher guide asks me to use this definition of a personal narrative:

A personal narrative is a true story told in the first person by someone who was involved in the events being described.

This raised a question for me: Do my students know what “written in the first person” means?

The answer turned out to be no.

Atomisation starts by identifying the atom: in this case, knowing what it means to be written in the first person, as contrasted with the second and third person. Then, teach the atom on its own. Don’t overthink it: give students a bunch of examples, then practice. I use mini whiteboards. Is this sentence in the first, second, or third person? Wash, rinse, repeat.

The key insight of atomisation is that it’s hard to learn two things at the same time. If students don’t know what “first person” means, that definition up there won’t make much sense. A teacher might realize students are confused and go back to define “first person.” Not a bad teaching move. But the mess is already made, time wasted, students confused. It’s much better to start with the “atom” of first, second, and third person, before introducing the more complex idea.

If all learning were a ladder, this might be enough: focus on the atom on its own, students understand what first person means, that helps them understand the idea of a personal narrative, and we move on. But learning isn’t a ladder, and most of literacy learning is a spiderweb. In this particular case, the atom plays an important role. But what is the best way to make sure students can connect their newfound learning to different concepts in the future?

Atomise, Retrieve, Expand

What I just described is the first and most fundamental step of atomisation. Identify the atom and teach it in isolation before using it as part of a larger concept. But that’s not enough.

Next up, I want students to retrieve that atom. This is where that learning moves securely into long-term memory. The next day I give students a few more questions. Is this written in the first person, second, or third? Again, don’t overthink it: mini whiteboards, just a few questions. If students get them right, great, on to the next thing. If students get them wrong, give a few examples and practice a bit more. Then we do it the next day, the next week, the week after that. I use a little spreadsheet that automagically schedules retrievals, asking students to retrieve at longer and longer intervals.

The most important step of this whole thing is expansion. Expansion means I want to expand the contexts in which students are using this knowledge. One risk of atomisation is that breaking learning down into small steps means those steps become isolated. Students are good at naming first, second, and third person when I give them a single sentence, but they don’t think about what it means beyond that isolated atom.

One obvious place is in the definition of a personal narrative: personal narratives are written in the first person. We will read a few different personal narratives these first few weeks, and we’ll have a chance to revisit what makes a piece of writing a personal narrative. We’ll also talk about how, in a personal narrative, not every sentence is written in the first person, and there’s a subtlety where the author might describe events that happened to someone else. We’ll think about why an author might choose to write about events in the first person, and why that might be more powerful than writing in the third person. Later in the year in our poetry unit we’ll revisit the idea of perspective and talk about why, in some poems, the poet chooses to write in the first person, others in the second, and others in the third. All of those are opportunities for expansion: taking the original atom and applying it in different contexts, anchoring it with multiple connections to the larger web.

“Atomise, retrieve, expand,” is my reminder: atomisation is useful, but only if students retain the atom and apply it in lots of different ways. Atomisation alone risks turning the curriculum into a bunch of isolated, disconnected bits of knowledge. Retrieval and expansion build essential connections to everything else.

In the grand scheme of things, understanding the definition of a personal narrative and the meaning of first, second, and third person isn’t that big of a deal. Atomisation just doesn’t play that big a role in literacy. Most literacy learning is more like a spiderweb. It’s tempting to believe that, if only we could identify all the little atoms and relationships between them, we could build a machine to teach all those little atoms in the right order and make sure every student learns. That’s the dream of lots of education technology right now. Unfortunately, learning is more complicated than that. Still, in some places atomisation can play a valuable role. It’s best for small stuff, but the small stuff is worth teaching well.

I feel ambivalent about it.

On one hand, I’m enthusiastic about a lot of what often falls under that broad banner. Cognitive science has had a big influence on my teaching. I’ve learned a lot from that type of research. Cognitive load theory, retrieval practice, and more have all helped me to teach better.

On the other side, I worry that “the science of learning” overpromises. Is there, truly, a science of learning? How much do we know about learning?

One thing I find interesting: in most elementary schools across the US, teachers are aware of “the science of reading.” But “the science of learning” is almost totally unknown across all types of schools. Ask a typical teacher about it, and they won’t know what you’re talking about. Should we — people like me who have learned a lot from research often labeled as the science of learning — be spreading it, and advocating for “the science of learning” to be better-known in schools?

I’ll come back to my hesitations in a minute. First, a story.

A Story

I have a stutter. If you’ve spent a lot of time with me you’ve probably noticed it. I have decent coping strategies at this point, but as a kid it would be hard to spend more than two minutes talking to me without hearing a significant stutter.

I went to a lot of speech therapists. Most, unfortunately, didn’t know what they were doing. I have particularly vivid memories from one: a group session where we spent a bunch of time learning definitions and catchphrases. The meaning of fluency, “practice makes permanent, not perfect.” And then we were supposed to practice. It never helped much; I just spent time learning about what speech was supposed to sound like, trying to practice not stuttering, stuttering anyway, and watching the kids around me all do the same thing.

My parents, bless them, were patient, and after a bunch of failures we found someone who helped me develop strategies that worked. The strategies sound pretty counterintuitive to someone who hasn’t experienced this. Here’s one that made a big difference. I mostly stuttered at the start of a word or phrase, and in particular on words that began with consonants. Imagine I’m trying to say “cost,” and it comes out as “c-c-c, c-c-cost.” But if I try to say “accost” I don’t have any trouble at all. So a basic step is, when I start to stutter or anticipate a stutter, I add a vowel at the start of a word. This is a nice way to avoid stuttering. Unfortunately, “cost” and “accost” have different meanings, so it’s not a great long-term solution. Through speech therapy I learned to make the vowel less pronounced. It shifted to something like “(uh)cost,” and then the (uh) became totally silent, just a breath and a slight change in the cadence of my speech.

This is something I’ve found with a lot of effective teaching. It’s not a generic strategy like “break learning into small steps.” Effective teaching is rooted in an understanding of the specific content to be taught. It’s also often a bit counterintuitive, not the first thing one would try, something teachers have figured out after endless trial and error.

The Bell Curve

Many phenomena in the world seem to roughly follow a bell curve. In terms of speech skills, I fell on the wrong end of the bell curve. I went to speech therapy hoping to make a change, to move to the right. The first few rounds didn’t work. That sucked. Not fun, do not recommend.

But a bell curve isn’t the inevitable consequence of instruction. Here is Benjamin Bloom:

There is nothing sacred about the normal curve. It is the distribution most appropriate to chance and random activity. Education is a purposeful activity and we seek to have the students learn what we have to teach. If we are effective in our instruction, the distribution of achievement should be very different from the normal curve. In fact, we may even insist that our educational efforts have been unsuccessful to the extent to which our distribution of achievement approximates the normal distribution.

-Learning for Mastery

Unsuccessful instruction That’s what I experienced. The end goal isn’t a bell curve. It’s something like this:

This isn’t some fantasy land where every student is exactly equal. But there should be fewer students in the left tail, and more students shifted to the right.

I think that what Bloom called unsuccessful instruction is the norm. The bell curve is what most outcomes look like in education. This failure doesn’t mean teachers are lazy or students learn nothing. It just means that there’s this bell curve, this random outcome, and that teaching isn’t shifting the distribution. Students arrive with different affinities and aptitudes, and education reinforces them rather than changing a student’s trajectory.

I don’t think unsuccessful instruction is universal. There are some places where the education world has figured some stuff out. Phonics has been in the news in recent years, and I think phonics is a great example of an instructional tool that shifts the bell curve and skews the distribution to the right. But one reason phonics is notable is that it’s an outlier. It’s one of a handful of concrete, specific classroom practices with a strong research base that a typical school can adopt. There are others, though most are buried in the minds of experienced teachers who have figured out a great way to teach this skill or that concept. Islands of success, like the effective speech therapist I finally found. But the bulk of the research suggests that differences in teacher quality and even school quality only explain a modest portion of the variation in student achievement. I think that squares with an environment where unsuccessful instruction is the norm, and for most students school is a place where they fall in a particular spot on the bell curve and stay there.

The Blind Men and the Elephant

Here’s an Indian parable:

A group of blind men heard that a strange animal, called an elephant, had been brought to the town, but none of them were aware of its shape and form. Out of curiosity, they said: “We must inspect and know it by touch, of which we are capable.” So, they sought it out, and when they found it they groped about it. The first person, whose hand landed on the trunk, said, “This being is like a thick snake.” For another one whose hand reached its ear, it seemed like a kind of fan. As for another person, whose hand was upon its leg, said, the elephant is a pillar like a tree-trunk. The blind man who placed his hand upon its side said the elephant, “is a wall.” Another who felt its tail, described it as a rope. The last felt its tusk, stating the elephant is that which is hard, smooth and like a spear.

(source: Wikipedia)

In this story, teachers are the blind men and the elephant is the science of learning.

We know some stuff. We can describe a bit about the general shape of the thing that we might call “the science of learning.” But most of the specifics are yet to be filled in.

That’s the thesis of this post. Let’s have some humility. The education world should celebrate our successes, but also be clear-eyed about the goal and how far we have to go. I think Benjamin Bloom’s goal is exactly right: we should strive to shift the bell curve of education outcomes to the right. We should be honest that we don’t have very many well-known, widely used teaching techniques to do that.

The Risk

My experience with “the science of learning” is that it is absolutely helpful, but also woefully inadequate to the real complexities of teaching. Most of the time when I try something rooted in cognitive science or other learning research, it’s a mess on the first pass. That doesn’t mean it’s bad or useless, just that it’s missing lots of details. We have bits and pieces but the picture is far from complete.

I mentioned phonics earlier, and I think it’s a helpful example in the context of that parable. There’s one part of the elephant we know pretty well. That’s cool! We should teach kids to decode using all of the available research. We should also recognize that phonics teaches students to decode, and decoding is only one part of reading instruction. While there’s plenty of other research on the rest of reading instruction, it’s a lot muddier than phonics — and even in the world of phonics there are plenty of questions left to be answered. Kata Solow had a great recent piece about some evidence that the science of reading has caused an overcorrection toward too much phonics. That seems unsurprising to me: people focus on the part of the elephant they understand, and put less emphasis on the parts they don’t.

Math fact fluency is another interesting case. There’s a lot of correlational research showing that knowing math facts is associated with future math achievement. But there is very little high-quality, causal evidence that additional fact fluency practice leads to broader learning gains. This isn’t to say that math facts aren’t important. We have a sense of the general shape of that elephant. But most of the details are yet to be worked out.1

This leads back to my hesitation with “the science of learning.” I’m back in the classroom right now for the first week of school. I’m thinking a lot about working memory, cognitive load, retrieval practice, and more. All of that stuff plays a big role in my teaching. But the nitty-gritty practical knowledge that helps me put that stuff into practice in my classroom is full of lessons learned from trial and error, one part science and three parts empiricism.

I don’t want to go around overselling this elephant. There’s just so much we don’t know. I’m an optimist: I think there exists a future world where education works much better for many more students. But that future world will rely on a science of learning that looks very different than the one we have today. We are blind men groping around an elephant, and we need to learn about the elephant’s circulatory system, and its mating habits, and its place in the food chain.

I’ve seen both sides as a learner. I experienced unsuccessful instruction in my first few rounds of speech therapy, stuck on the wrong end of the bell curve. Then I found someone who knew what they were doing and understood the specific issue I was having, and now I’m in a totally different place on the bell curve. I’ve also been on both sides as a teacher. I’ve taught plenty of topics where the students who have been strong math students in the past get it, the students who have struggled with math in the past don’t, and I just reinforce the existing hierarchy. I’ve also found ways to teach topics here and there that truly move the needle, where those students who struggle are able to succeed and feel proud of their progress. I want successful instruction for more students. I want it desperately. But that’s not something the science of learning can offer right now.

Is the science of learning a part of that knowledge? Absolutely. But after years of trying to understand the science of learning, it is still no match for the challenges of the classroom. So I’ll share what I’ve learned, but I’m not going to go around telling people that they should adopt the science of learning, or that if only more teachers understood the science of learning we’d have a more effective profession. The science just isn’t there. We have a long way to go, and I want to be honest about that.

In some classrooms most students get right to work. In others, most don’t. Why?

Motivation is a big topic. I want to write about a framework I have that helps me think about how to motivate students in the classroom, but first I want to clarify what I mean by motivation. I’m talking about the situation above. Do students attempt the task in front of them? Do they put in effort or try to get through an assignment with as little thought as possible? Do students persevere if they don’t know something right away?

Motivation is complicated. There’s no one magic button that motivates students, no single strategy that teachers can always rely on. But that can also be overwhelming: ask teachers for advice about motivation and you’ll get lots of different answers.

The Framework

To me, the fundamental difference between motivation strategies is the time scale. Some motivation strategies are useful in the short term. These are limited: they don’t last very long, so you’ll need to keep finding more motivators, but they can also be helpful tools to build momentum. Medium-term motivators take more time to get going, but can help motivate students over multiple days and weeks with sustained effort. Long-term motivators are the real magic: self-sustaining tools that are slow to build but the most powerful. Every teacher needs multiple motivation strategies, and the best motivation strategies fit together and build on each other.

Short-Term

Short-term motivators include choice, relevance, enjoyment, and extrinsic incentives. These are all useful tools. Giving students choice helps them feel ownership over their learning. Helping students feel like learning is relevant to their lives can cultivate interest in a subject. Most humans like to have fun, and making a lesson enjoyable is better than the opposite. And while extrinsic rewards and consequences aren’t always fun to talk about, they are valuable tools.

The first problem with short-term motivators is that they are short-term. These might work well in motivating students today or tomorrow, but to motivate them next week you’ll probably need to continue designing in opportunities for choice, or relevance, or enjoyment, or extrinsic incentives. Some teachers are really passionate about these! If that’s you, go for it. But one risk is burnout: because these short-term motivators must be renewed day after day, they require continual effort and can feel like an endless treadmill.

The second problem with short-term motivators is that they often involve tradeoffs. If you are giving students choice day after day, how confident are you that those choices involve equal learning? Maybe most of your students like soccer so you use that to make learning relevant, but what about the students who don’t like soccer? If you are prioritizing enjoyment, are you compromising on learning? And extrinsic incentives offer a whole raft of tradeoffs. If you can’t be consistent with a reward or consequence it will quickly lose its power, and over time students may find ways to game the system.

This isn’t to say short-term motivators are useless. But from my perspective, their primary use is in the name: they are short-term tools to “get the ball rolling,” and move towards the medium-term and long-term motivators, or maintain momentum, rather than as the primary reason students are motivated to learn.

Medium-Term

Medium-term motivators include habits, success, and accountability. These honestly sound totally boring. Routines build habits, and a good way to get students engaging with schoolwork is to build habits through everyday consistency. If students feel successful in school they are more likely to engage with learning, and if they feel dumb they are more likely to tune out. A human in the room who communicates to students that their learning matters, and that they will hold students to account if they don’t put effort in, can nudge students in the direction of more effort.

The disadvantage of these strategies is that they don’t work right away, and they also require regular effort. Habits don’t happen overnight. The effects of feeling successful build gradually, especially for students who have often felt unsuccessful in school in the past. Students often resist accountability at first. These also aren’t strategies that a teacher can set and forget: they require constant maintenance. But, when done well, they can be incredibly powerful in helping students feel like their effort matters and that school is a place where it’s worthwhile to work hard.

The other important note about these strategies is that they work best when used consistently from the start of the year. Bad habits are harder to undo than good habits are to build. Once a student feels dumb it’s hard to make them feel smart. A student who feels like they can shirk their work without any adult caring will be even more resistant when the adult starts to hold them accountable. It’s not hopeless in the middle of the year, but it is absolutely a tougher hill to climb.

Long-Term

Long-term motivators include community, identity, and belonging.

If you talk to a veteran teacher, they often talk about these types of long-term motivators. These tools are incredibly powerful, and honestly put all the other motivators to shame.

One thing to note: these motivators can be strengthened or eroded by school culture. In some schools you get a head start: the school already creates common language and systems to support community, identity, and belonging. This is a huge leg up for teachers, especially new teachers. In other schools, it’s every teacher for themselves. It’s still possible to support community, identity, and belonging in schools like those, but it’s much harder. Many veteran teachers have struggled to transition into that second kind of school, finding that the tools that worked for them previously just don’t anymore.1

There’s a classic story in education. Rookie teacher Mr. A is struggling with classroom management and motivation. Students just don’t seem to care in his class. The principal recommends he goes to watch veteran teacher Mr. B, who teaches the same students and gets a lot more effort out of them. Mr. A goes to observe. Students do try a lot harder in Mr. B’s class. But it doesn’t seem like Mr. B is doing anything different at all. The students just sort of listen without any special teacher moves or classroom management on his part. Mr. A feels a bit dejected, like there’s some invisible secret sauce that is being withheld from him, and continues to struggle through the year.

In this story, Mr. B is an expert at building community. That’s great! It’s an incredibly powerful tool for a teacher. But it’s also a tool that is often invisible to others, impossible to imitate, and slow to develop. I’m not an expert at this: I do my best by being a weirdo and coming up with little classroom rituals.

Look, this stuff is worth trying to develop. But I would be really wary about putting all of your eggs in this basket. If the community you’re trying to build doesn’t come together, you can end up in a really tough place without some other motivation strategies to fall back on.

Final Thought

This isn’t an exhaustive collection of motivation strategies. I’m sure there’s a bunch of stuff I’m missing. I do think that stepping back to consider whether a motivation strategy is short-term, medium-term, or long-term is a useful exercise. Every teacher needs multiple motivation strategies, and every teacher should understand how those strategies fit together and build on each other.

Then I took a course sponsored by the Colorado Department of Education and it is a perfect encapsulation of this thing I dislike, so now I’m going to write about it.1

The course is titled “Powerful Practice: Evidence-Informed Math Teaching.” It’s the kind of course lots of teachers, including me, take to meet professional development requirements or move up on the salary scale. The course goal:

After successfully completing this course you will employ evidence-informed instructional routines in your classroom that support accelerated learning in mathematics for all students, especially those furthest from opportunity (multilingual learners, students with disabilities, students of color, and students experiencing poverty).

I’m a fan of evidence. I would like to accelerate learning. Sign me up.

There's a lot I agree with in the course. It has solid sections on multilingual learners, connecting multiple representations, and alternatives to keyword strategies for word problems. Good stuff! Like any PD I have some disagreements, but overall it's thoughtful and research-informed.

A Sleight of Hand

Here’s the part I don’t like. It honestly feels sneaky, like a sleight of hand. It’s not about any of the content in the course. It’s about what’s missing.

In the whole course, which is worth 25 hours of professional development, there is close to nothing about practice.

Here is the closest I could find to a statement that practice plays an important role in math learning:

While students likely need practice, the emphasis here is not drill and kill or timed repetition of math facts. Instead, we want to help students see patterns and identify pen-and-paper and mental strategies to perform calculations accurately and efficiently.

There’s a passing acknowledgment that practice is maybe necessary, and then equivocation. Don’t “drill and kill” (a phrase which is not defined anywhere in the course). Help students see patterns. Identify strategies.

Look, drill and kill is a real phenomenon. When students are asked to practice too much too soon, or when practice is excessively repetitive, that’s bad. And yes, I am in favor of having students see patterns and identify different strategies to solve problems.

But I am also unequivocally in favor of practice in math class. After learning something new, students should solve some similar problems to move that learning into long-term memory. Beyond my personal opinion, this course claimed to be “evidence-informed.” It is not possible for an honest person to read the evidence on math learning and not come to the conclusion that practice plays a critical role. If I were in charge, I would include multiple modules on structuring effective practice. Worked examples, fading supports, retrieval, interleaving, spacing? Among 14 modules, close to nothing. The attitude here seems to be that practice is something that’s grudgingly accepted as necessary, but not to be talked about.2

Practice is hard to get right. We should be talking about it! Why is practice important? Where does it happen in the learning sequence? What are some common pitfalls? There’s so much to talk about!

The Vibe

This is a vibe I get from lots of other places in the math world. I was on a curriculum adoption committee a few years ago. None of the curricula emphasized their approach to practice. They all had supports for multilingual learners, multiple representations, and routines for word problems. Good stuff! Keep all of that. But it was obvious that practice was an afterthought. Practice wasn’t mentioned in the marketing materials. The practice problems got too hard too quickly, or skipped important problem types, or were lazily written with too much repetition.

Similarly, if you attend a conference run by the National Council of Teachers of Mathematics, you’ll get the same message. Sessions on every hip and cool approach to math learning you can imagine...except practice, which will come up here and there but not in featured sessions, avoided by the vast majority of presenters.

If practice was being done well in typical math classrooms this would be fine. But it’s not! Go talk to teachers. They’ll tell you that the curriculum doesn’t provide enough practice. That they struggle to find good practice resources. That they can’t agree with their colleagues about how much practice is necessary, or even basic questions like whether most practice should happen in class or for homework, or digital vs paper and pencil.

One more thing that drives me crazy. The quote from above is in a section on balancing procedural fluency and conceptual understanding. And I get it, I think procedural fluency and conceptual understanding both play an important role in math learning. But they’ve become these two pillars of math education in a way that I think is counterproductive. In some circles you can’t talk about practice without talking about procedural fluency, then debating whether fluency means fast and accurate or also flexible, and the role of timing, and which standards call for fluency, and whether conceptual understanding has to come first.

Look, some of you are overthinking this. Practice is important because practice is how humans learn things, and math is no different. Don’t let all the fancy theory distract from the idea that, after learning something new, a bit of practice and repetition is exactly what humans need to retain what they have learned.

The medium is the message.

-Marshall McLuhan

I built a math fact practice website a few years ago. It’s still up, you can check it out if you like. I’m not a professional developer, it’s not anything too fancy. The basic idea is to adapt to the learner. The website tries to figure out which facts the student needs help with and gives them focused practice on those facts. When I’m teaching a class of 25 students, it’s hard to meet every student where they are. One advantage of technology is the ability to adapt a learning activity to each student with far less effort from the teacher.

As I tested and used the website with students I found disadvantages as well. One is “videogamification.” A typical student spends plenty of time playing video games on screens. Many spend a significant chunk of their school day playing video games as well, as those of us who are familiar with Slope and Subway Surfers can attest. In a video game, the goal is to beat the level as quickly as possible, not to learn along the way. Students bring those habits with them when screens come out in math class. In the case of my math fact website, some students would deliberately get early questions wrong in order to give themselves an easier path through the task. I tried lots of stuff to mitigate this, but I kept running into the same fundamental problem: students were trying to get through the task, not trying to learn. I ended up building a paper-and-pencil system for math fact practice to replace the website.

We could have an interesting debate here about the details. I could try to redesign the website. Maybe the benefits of personalization are worth the drawbacks of screens. I don’t want to get stuck in this specific example. Every time I’ve used screens with students, I have found videogamification to be an obstacle. It’s part of the medium. Those habits students have built elsewhere carry over to classroom technology, and that changes my decision-making when it comes to screens in school.

Technology Is Not Neutral

I’ve played a small role in the classroom technology backlash, and the backlash to the backlash is here. Among many others, Zach Groshell argues that it’s not the screen, it’s the instructional design underneath. Groshell compares the delivery of instruction, whether a screen or a human teacher, to a grocery truck. It’s not the truck that matters, it’s the food inside. “Again and again, people seem to care more about the carrier they can see than the design they cannot.” Groshell’s argument is that we shouldn’t care whether students learn from a screen or a human teacher; we should strive to deliver excellent instructional design to as many students as possible. The vehicle, to Groshell, is neutral.

I’ve named one counterexample: videogamification, the tendency for students to treat anything on a screen like a video game. And to be clear, students sometimes treat paper-and-pencil learning like that as well. My observation, based on my experience as a teacher, is that videogamification is an order of magnitude worse on screens than on paper.

That’s just one way the medium shapes the message. Another example with a broad base of research is in reading comprehension. A wide range of studies have found that comprehension is stronger when reading happens on paper compared with a screen. This is a fundamental property of the medium: humans skim and struggle to attend deeply when reading on screens.

Putting screens in front of students has advantages and disadvantages. One advantage is that the screen can look back at the student. Here’s Carl Hendrick sharing what that advantage affords online instruction:

Schooling fulfills the function of accountability. In the future, this will mean cameras on, and an AI monitoring student behavior, their latency, what websites they are looking at, their ability to focus and then producing a report for a human tutor. If you can get kids to concentrate on the apps, on the sequencing, phenomenal learning is going to happen.

If you spend time watching students learn on screens, this is totally unsurprising. It’s the logical endpoint of making screens the primary vehicle for instruction. Students are prone to rushing, to skimming, to finding the easy way out. If AI is the solution to our educational problems, of course we should have AI monitor students’ eye movements.

Carl isn’t wrong. Accountability is critical in education. One of my most important roles as a teacher is being a caring human in the room with students. That presence, our relationship, and my ability to hold students accountable for learning, are huge elements of what motivates students to learn. If (this is a big if) AI can monitor student “latency” and everything else, I have no doubt it will increase student learning in a tech-first environment. But the medium is the message. Accountability from artificial intelligence is different than accountability from a human. What message does that send to students? What kind of life does it prepare them for?

Let’s Talk About Tradeoffs

I would love to have a serious, honest conversation about the tradeoffs here. Teaching is hard. I sure would like to see a way to scale the impact of the best teachers. Teacher quality is uneven; I know well that too many students spend their school day on busywork, whether it’s delivered from a screen or a packet. Is AI the solution? Maybe.

But the starting point of that conversation is to acknowledge that the medium is not neutral. The medium is the message. The tools we use shape us. I’ve felt it. Part of the reason I moved away from classroom technology was my realization that when there’s a screen in front of every student, I became a more passive observer in the classroom, deferring to the machine to manage instruction. That decision isn’t neutral.

There are serious, smart people doing work that acknowledges those tradeoffs. Dan Meyer is building an AI tool designed to help teachers launch discussions building on student thinking, recognizing that digital classrooms risk isolating students and that teachers benefit from tools to counteract that isolation. Daisy Christodoulou is building an AI tool to radically reduce the time teachers spend assessing student writing, while keeping a human in the loop. It even works on handwritten essays, so teachers can keep screens out of the classroom while still harnessing the power of today’s technology. These are great examples of real tools that wrestle with the affordances and liabilities of today’s technology.

The internet is full of big claims about what AI can do, yet the vast majority of those claims are hypothetical: what could be, where we might end up, what is possible. I am heading back to the classroom in exactly one month. I will be making decisions based on the real resources available to me, not hypotheticals on social media. Despite some big claims, the apps that Groshell and Hendrick are developing are not yet available to boring teachers in traditional schools like mine. We can’t yet evaluate the tradeoffs of using them with students.

I’m moving down from middle school math to elementary in August. While I’m still a strong skeptic of the role of technology in the classroom, I don’t think I will be tech-free. I’ll use a bit of student-facing technology here and there, for specific purposes or as I’m required to by my school. I’ll keep experimenting with new technology, and I’m optimistic that I’ll develop more and more tools that will make planning easier for me using large language models and more. I will think hard about the advantages and disadvantages at each step of the way. That’s all any teacher can do: be open to new ideas, be clear-eyed about tradeoffs, and do the best we can with the resources we have.

I have to call a spade a spade. With the digital tools I have access to right now, paper-and-pencil classroom teaching is almost always the better option. And the vision of education that some people are selling, where technology unlocks a brave new world of efficient learning, is smuggling something sinister under the hood.

The conversations started because I’ve expressed skepticism about productive struggle in math class. A number of people have defended asking students to figure out mathematical ideas themselves. Some concerns people share are that teaching without productive struggle can lead to shallow understanding, that students become dependent on the teacher, and that students struggle to apply what they have learned in flexible ways.

A Concrete Example

Let’s get specific. One topic I teach is multiplying negative numbers. Here’s how I approach that topic.

I start pretty much every lesson by asking students a bunch of questions on mini whiteboards. My goal is to activate prior knowledge and help students connect what they know with what I want them to learn. Those questions might look like:

Here I step in. I leave those last few problems on the board, and I say hey, when we say 2 x (-3), what we mean is -3 + (-3), so 2 x (-3) is -6.

When we say 3 x (-5), we mean -5 + (-5) + (-5), so 3 x (-5) is -15.

I’ll give a few more examples, this time juxtaposing multiplying positives and negatives. What is 2 x (4)? Ok, what is 2 x (-4)? I’ll be clear about the meaning of those parentheses. Then I check for understanding and we practice a bit. This includes mixed practice — if every question is a multiplication question where the answer is negative, students aren’t thinking much, so I mix in positives, addition with negatives, etc.

I’ll do the something similar when we get to questions like -4 x 2. I start with questions like:

Then I step in. 10 x (-6) is -60, so -6 x 10 is also -60. Again, I give a few more examples, check for understanding, and we practice a bit, again with some mixed practice. I do something similar for a negative times a negative, activating prior knowledge and looking at patterns to help students see why a negative times a negative makes a positive.

Why I Choose This Approach

In each of these examples, some teachers would ask students to figure out the big idea on their own. I can imagine that working well in some classrooms. I do use that for approach some topics. Some topics lend themselves more to that type of small-step discovery, others less so. It’s not an approach I use often because what I’ve observed is some students consistently end up frustrated or discouraged when trying to make that leap. It works for some students, while for others it feels overwhelming. Using that approach day after day is likely to teach those students that they just aren’t very good at math.

This might have to do with my context. My school has persistently low math achievement. I teach a lot of students who, for years, have felt unsuccessful in math class. It’s not all of my students, but it’s a critical mass where that lack of confidence means students often tune out when they feel like the teacher isn’t supporting them, and tuning out is contagious.

Another reason I avoid having students figure new ideas out themselves is to save time. I have limited time, and I find a good way to get students solving harder problems is to build confidence early on when learning a new topic. Once students have that confidence, I ask them to apply their knowledge lots of different ways. One example is to have students look at the patterns in negatives when multiplying more than two numbers, and to figure out the pattern that three negatives multiply to a negative, four to a positive, five to a negative, and so on. Here I’m much more likely to use an exploratory approach. Students have built some confidence, and even if they don’t figure out the pattern they still get practice with their multiplication skills.

Another example is a little puzzle I stole from a Desmos activity that I now can’t find. I give students a bunch of numbers. Maybe the list is 2, -3, -2, 4, 3, 5, 1, -6. The goal is to find two subsets of numbers that have the same product. A simple solution might be 2 x (-3) and (1 x -6). Then, the challenge is to use as many numbers as possible. Can you find two groups of three numbers with the same product? This is a great way to get students practicing while also reasoning more flexibly.

The point of math class is for students to be able to apply their knowledge in lots of different ways. Taking an efficient approach to initial instruction leaves more time for this type of application.

The Criticisms

Let’s go back to those criticisms from before. My approach focuses on connecting new learning to prior knowledge, and my experience is that this helps students understand the math. I don’t think learning through productive struggle is the only way to develop deep mathematical understanding. While I give students plenty of straightforward practice to start, as soon as they’re ready I vary the questions I’m asking so students don’t become dependent on the teacher or believe they can only solve a problem if they’ve been explicitly taught a question exactly like it. I ask students to apply what they know in lots of different ways and try to ask plenty of non-routine questions.

I’m not saying I’m the perfect teacher. I’m trying to be thoughtful about lots of different pieces at the same time. I’m navigating my students’ confidence in math, their procedural skill, and their ability to apply that knowledge flexibly.

If I was tutoring a student one-on-one, I would be much more likely to ask the student to figure new ideas out on their own. I can keep a close eye on their confidence, adjust if they’re having a hard time, and make sure the process is productive. With a class of 25 students, getting that right for every student is much harder. I think it’s absolutely possible, but doing it day after day is challenging and risks discouraging students who already lack confidence. I choose to do it occasionally but it’s not my default pedagogy. Where I become even more skeptical is when curricula try to design that type of productive struggle on a daily basis. Not only is it hard to get right, but following the recommendations of a curriculum writer who doesn’t know my students, doesn’t know their strengths and weaknesses, makes success even less likely.

A reasonable person could disagree here. I’m happy to hash it out in the comments. My point is that it’s much easier to have these conversations by talking about a specific piece of teaching, rather than abstract arguments disconnected from real classrooms. There’s a stereotype that the alternative to productive struggle is just lecture and rote practice — “sit and get, then drill and kill” if you want to use the popular pejoratives. Whatever your approach to teaching, I hope this post illuminates the space between productive struggle and lecture.

Next school year I’m making a change and moving down to the elementary school in my district. I’ll be teaching 5th grade — self-contained, so teaching all subjects. I’ve spent the last year learning more about elementary teaching, earning my Colorado elementary endorsement, and doing a lot of reading about, well, reading. I’m excited for a change. I’ve learned a lot, and I know I have a lot left to learn. Plus I still get to teach math!

Three Ingredients

One of my favorite texts has been Christopher Such’s book Primary Reading Simplified. A basic question about teaching reading: what do you do in an elementary reading lesson? Such has a nice mental model for what happens in reading class that I found helpful.

Such describes three major ingredients in reading instruction.1

Fluency reading is designed to help students become more fluent readers. If a student isn’t very fluent, reading alone is not an efficient way to help them improve their fluency. The key ingredient in fluency reading is repetition: rather than reading something once and moving on or discussing it, the class reads a passage multiple times. This can involve a combination of teacher read-aloud, echo reading, partner reading, and more. The goal is simple: repetition gives students a chance to improve their fluency by getting a little bit better with each read. If students only ever read a text once, they are often practicing dysfluency. Reading multiple times, typically after modeling from the teacher, gives students a chance to practice decoding accurately and reading with fluency.

Close reading is a chance to dive deeply into a single text or part of that text. During close reading, Such recommends at least two-thirds of the time spent discussing and analyzing the reading. This is where students have a chance to think hard about the choices the author makes and meaning that might not be apparent on a surface-level read. The details will vary with the text, but the broad goal is to investigate the ways that a reader can interpret a text and what those interpretations can tell us.

Extended reading is the final ingredient, and complements the other two. If all a reading class ever did was fluency reading and close reading, they simply wouldn’t read that much. In extended reading, the goal is to get a lot of experience reading and learning from that reading. Extended reading isn’t just “ok kids, go off and read.” It might involve a mix of teacher read-aloud, partner reading, independent reading, and more. Such recommends at least two-thirds of the time spent reading, with the rest of the time discussing or supplementing the text. This is a great chance to add in vocabulary instruction, to help students build background knowledge, and more. Those can happen during fluency reading and close reading as well, but extended reading provides more “raw material” where students get experience with a lot of text.

I like thinking about reading lessons in terms of these three ingredients. A bit of focused repetition to improve fluency. Deep dives into specific sections of text to analyze and discuss. And then a lot of reading, and all the learning that comes with it. The exact balance of those ingredients will vary, depending on the needs of the class and the texts we are reading. Also, not everything that happens in a reading lesson will fit perfectly into one of those buckets. But for me, as a beginner to this whole teaching reading thing, this is a helpful mental model to get my head around what happens in reading class every day.

What About Math?

All that made me think: could those three ingredients be applied to teaching math?

A bit of focused repetition with key skills to improve fluency.

Deep dives into a few specific problems to better understand the math and how different ideas are connected.

Lots of practice solving problems across a broad range of math topics.

I want to be careful here. I’m not implying that teaching math and teaching reading are the same. But I find it interesting that this mental model makes a lot of sense to me. Repetition is a key ingredient in math fluency, and there are a number of key skills where we should prioritize repetition until students become fluent. Spending time analyzing and discussing a single problem as a class can be a great way to make connections between ideas, to understand the why behind a procedure, and to practice deeper skills of mathematical analysis. And then, students should solve lots of math problems, across a range of topics, for the simple reason that one gets better at math by doing lots of math. The mix of ingredients will again depend on the exact class, but it’s a reasonable place to start. That last piece is the one that I think is missing in lots of math classes and math curriculum: the chance to solve lots of problems that students learned about last week, last month, or last year. Solving problems is good! Not all math lessons should be laser-focused on the day’s objective. The more we review prior learning, the more prior learning is likely to stick.

There’s no one perfect lesson. Instead, we want students to have a balanced diet of the different ways of engaging with a subject. It’s not balance for the sake of balance; it’s a balance of the ingredients that are most important for learning. I think this is a reasonably broad framework for getting good at just about anything: focused practice, deep analysis, and broad exposure.

And here are two graphs from recent lessons in my curriculum:

There’s nothing wrong with these two graphs. They’re decent tools to teach the statistics standards in 7th grade. But they don’t capture a lot of the complexity of graphs that students will see in the future, and they don’t have much to teach students about the world.

Some areas of math are structured in a linear way. Addition should be learned before multiplication. Quadratics should be learned before higher-order polynomials. But not all math is nice and neat and linear, and statistics is particularly resistant to that linear structure.

Enter slow reveal graphs.

Slow reveal graphs are the brainchild of Jenna Laib, and they are brilliant.

I start by showing a slide with very little information:

Then I ask students what they notice. Super simple. I’m just looking for students to look at the graph and make a few observations. They write, then talk with a partner, then a few students share.

Next I show a bit more information:

Here I ask students what they can figure out. Maybe we try to guess what the graph represents, or just talk about some interesting features like the 24-hour time or the sharp jumps each hour. Low stress, but students often like this part the best and try to make a prediction before the full graph is revealed.

Then I show the full graph.

We talk about it a bit, answer any questions students have, and then students write to summarize the graph in their own words. This can be short, but it can also lead to some long and really fascinating conversations.

For the last six weeks of the school year, during our statistics and probability units, we are doing a slow reveal graph routine every day.1 Jenna’s website has a ton of information about how to use the lessons. What I do is tailored to my routine: we do this every day, and while I’m happy to spend time talking about the graphs I don’t want the routine to drag on forever.

Seriously, go check out slowrevealgraphs.com. There are so many graphs, and each graph has a premade slide deck that does the slow reveal part for you. It takes only a few minutes to pull graphs for a dozen lessons.

Why Slow Reveal Graphs?

Here's a graph:

I asked my students what they noticed. They made a few observations. One student said, "it's facing the wrong way."

In school math, we tend to only show students a very small slice of what graphs can look like. That's partly for practical reasons, to keep things simple and manageable. But it really seems like a shame that this student thought the graph was facing the wrong way. She had mostly seen vertically-oriented bar graphs in school, but graphs can look lots of different ways in the world beyond school.

Statistics in school often misses the forest for the trees. Or maybe a better way to say it is, focusing on a handful of the trees that are the simplest to understand and analyze, and ignoring all the other weird but cool stuff in the forest. Students should look at lots of graphs! Of all the math I teach, graphs are one of the pieces of math students will interact with most outside the math classroom. We should strive to give students a rich and broad conception of all the different ways to represent data in the world.

(This is the full version of that last graph, by the way.)

Knowing Things Is Good

My district, like a lot of districts, has deemphasized science and social studies in elementary school. That’s a shame for a lot of reasons, and one is that students just don’t know much about the world.

Teaching kids stuff is great. I enjoy teaching math, but there's a lot in the world that's worth knowing besides math. I have no illusion that doing six weeks of slow reveal graphs is a substitute for years of science and social studies. But every little bit helps, and giving students a chance to be curious and learn something a bit off-the-wall in math class is a ton of fun.

The conversations we have about slow reveal graphs have shown me how hungry students are to learn about the world. Sometimes we end up on a tangent discussing whether George Washington could have met a Greenland shark that is still alive today, or watching a video of an ostrich running at full speed that turns into a conversation about evolution, or looking at a map of Antarctica. The students who are most interested in these conversations are often students who are quiet and reserved in other parts of math class. That’s awesome! I do not care in the slightest whether a student will use their newfound knowledge of long-lived Greenland sharks in a future job. That’s not the point. The point is to learn things, because learning things is good.

How to Get Started

Go to slowrevealgraphs.com. Again, huge shout out to Jenna Laib. The collection of graphs is incredible. You can steal my routine, or look at what Jenna suggests on the website, or come up with your own. Don’t overthink it. The slow reveal is a great way to get students thinking hard without giving everything away at the start. I’m not sure the other details matter that much, as long as students get to think about a wide range of graphs and learn some new stuff about the world.

One more graph, just because graphs are fun:

Here’s a nice quote from Christopher Such about elementary reading:

In most primary schools, many pupils don’t become fluent readers because they do precious little reading in the classroom.

-source

If students don’t read very much, they are unlikely to become fluent readers.

By the same token, if students don’t practice math facts, they are unlikely to become fluent with math facts.1

There’s a naive approach that I’ve found helpful but limited: have students practice more. Yes, some students will make progress by just asking them to practice. However, putting fact fluency practice in front of students doesn’t always mean they get enough practice. Some students know so little that even in several minutes of practice they only answer a few questions. Others develop avoidance behaviors, trying to hide or find anything to do but practice their math facts.

This post is about the resources I’ve put together to help my students improve their fact fluency. These materials are designed with one overarching goal: to get reluctant math students answering more fact fluency questions.

At a basic level, what I’m sharing are a bunch of pieces of paper with math problems on them. “Worksheets,” if you prefer the pejorative. I understand the complaints about worksheets: teaching becomes an act of handing out busy work and telling students to get started. That’s not the design here. The design is to break fact fluency down into manageable pieces, to help students see the patterns in math facts, and also to provide plenty of practice so students move their math fact knowledge into long-term memory. That practice comes in the form of math problems on pieces of paper. There is no better technology for practice in math class than problems on pieces of paper. Call them “worksheets” or whatever you like.2

The Idea

In this folder are handouts to help students become fluent in addition, subtraction, multiplication, and division facts. Each operation is broken down into distinct fact families — families like x3 for multiplication or +2 for addition. Students begin by practicing each fact family in isolation. Then, they practice that fact family mixed with all of the other families they have seen so far. Then on to the next fact family and repeat.

I call this approach “incremental fact fluency,” after a research paper that helped to inspire it and the idea that it helps students to learn math facts in manageable, incremental chunks.

The best way to illustrate the approach is with a few examples.

Addition

The first round of addition practice focuses on +1s. Here is what the worksheets look like:

It’s not fancy. These are just math fact questions, broken down into fact families. These might not seem too hard. If students already know these, this is a chance to build confidence. I have found that more students than one might expect struggle with the commutative property, the idea that one can find 1 + 6 by beginning with 6 and adding 1. The focus for this round is both becoming fluent with +1s and gaining confidence, and also understanding that the commutative property can be really useful in addition problems.

The next section for addition is +2s. It has two parts. The first part is “Blocked” practice, where all questions involve +2. The first ten problems look like this:

Once students are confident with these, they can move into the “Mixed” version, where odd questions ask about +2s and even questions ask about +1s:

This pattern continues. Next up is +0s using the same structure, then doubles. Doubles begin with blocked practice:

Then, once students are confident with the doubles in isolation, they move on to mixed practice. The odd problems are doubles, while the even problems are a mix of all of the previous fact families:

This pattern continues. Introduce a new fact family, practice the fact family in isolation, then mix in all of the previous fact families. The goal is to build confidence by breaking fact fluency down into smaller chunks, while also making sure students gain fluency across all of their math facts.3

Multiplication

Multiplication uses the same basic idea. The first fact family is 2s, then 10s, then 5s, and so on, roughly in order of difficulty. For instance, the “Mixed” version of 5s starts like this:

The odd questions are all 5s, while the even questions are a random mix of the previous fact families, in this case 2s and 10s.

Bridge to Multiplication

When I tested the early version of this system, it worked great for most students and for the simplest fact families like 2s, 10s, and 5s. However, I found that some students struggled to make progress with the tougher fact families at first and often felt frustrated at the start of a new fact family. To help with this transition, I designed worksheets and accompanying lessons to help students acclimate to a new fact family. Here is an example for 2s:

The worksheet begins by having students add their way up the fact family — in this case, counting by 2s from 2 to 20 — and then subtracting back down. The rest of the handout is a random mix of those same questions interleaved together.

There are two main goals of this type of practice. The first goal is to familiarize students with the entire fact family. If a student is asked 7 x 4 and can’t retrieve the fact from memory, I want them to know that the answer must be one of the members of the 4 fact family, which narrows the possibilities considerably.

The second goal is to help students move flexibly between adjacent facts. If a student doesn’t know 7 x 4, they may know 6 x 4 or 8 x 4. Finding 7 x 4 using an adjacent math fact is far more efficient than skip-counting from zero, and also reflects the type of relational mathematical reasoning that is helpful in many other contexts.

I have found that this scaffolding makes a big difference in getting students confidently answering multiplication fact questions when they don’t already have a foothold with that fact family.

Making Connections

The main goal of this system is to get students answering as many math fact questions as possible. Time in math class is limited. Too often, students who have the fewest math facts in long-term memory also work slowly or avoid fact practice through whatever means possible, meaning they don’t get the practice they need to improve. By breaking math facts down into small, manageable chunks, and gradually asking students to practice more and more facts, students can build confidence gradually and get plenty of practice with the facts they are working on.

A secondary goal, which is happily reached by similar means, is to help students see the connections between different math facts. Connections are everywhere in math, from realizing that 6 + 1 is the same as 1 + 6, to finding 7 x 4 by beginning with 6 x 4 and adding another 4. A bedrock principle of learning is that humans learn best when they connect what they are learning to what they already know. Practice is critical to learn one’s math facts, but students need far less practice when the connections between new learning and prior knowledge are emphasized and strengthened.

An Example of How I Use These Materials

In general, I work through each sequence of fact families in order. Some fact families take longer than others: for instance, I spend much less time on multiplying by 10s than any other fact family. I circulate while students work to see their progress and decide when to move from blocked to mixed practice, or when to move on to the next fact family.

I start with more instruction when beginning a new fact family, and gradually reduce the instruction as students become more confident. Here’s an example of how I might approach the 4s fact family:

First, using mini whiteboards, I’ll ask students to skip-count by 4s up to 40. Then, again on mini whiteboards, I’ll ask students to move more flexibly between multiples of 4, for instance finding 12 + 4 or 28 - 4. After a few questions like this, I’ll have students work on one of the Bridge to Multiplication handouts, practicing those same types of questions for the 4s fact family.

Next I’ll have students pull out mini whiteboards again and ask a few questions about 4s. I’ll start with some easier ones — 2 x 4, 3 x 4, 4 x 4. As we get to tougher ones I’ll sequence them and highlight ways to remember the facts if students aren’t quickly retrieving them from memory. For instance, I might ask 10 x 4 and then 9 x 4 to emphasize finding that math fact by subtracting 4 from 40. Then I might ask 4 x 9 to emphasize commutativity. Similarly, students have already seen 5 x 4 (since the 5s fact family comes earlier in the sequence) so we can build up from 5 x 4 to 6 x 4. If students are struggling to answer these, I might put all of the facts in that family on the board and ask students to study them for a minute or two before asking some more questions.

The first time we work on a new fact family, this takes some time. but it takes less and less time on future days and students can get right into practice.

If you want to differentiate and have each student working on a different fact family, you can do that. I stick with whole-class instruction. I find working with the whole class helps me to actually teach students, rather than hoping the worksheets will teach for me. A bit of extra practice isn’t going to hurt anyone. That said, do what works for you.

Timed Fact Practice

Timed math fact practice is a perpetual flashpoint in math education. I recommend timing students occasionally using a mixed practice handout with all of the facts for a given operation. This can be a helpful data point to see if students are making progress.

I emphasize to students that the goal of math fact practice is to practice remembering the math facts, rather than having to rederive facts each time.

I don’t explicitly time students. If the amount of time is too short, some students won’t get enough practice. If the amount of time is too long, what’s the point of the timing?

Some of you love to argue about timed math fact practice. I’ll drop some more detailed thoughts in a footnote, but be warned, no one is going to like my opinion.4

In Conclusion

Here is the folder with all of the resources. Check it out! I hope it’s helpful for some people.

I’ll emphasize the same idea I started with. Too many students don’t become fluent with math facts because they don’t get enough practice. Students need practice, but just giving handouts of fact problems often doesn’t result in enough practice. There are too many new facts at once, and students either work slowly or become intimidated and avoid practice. These resources are only effective if they get students answering more questions and getting more practice. That is my overarching goal. My approach is informed by research, but the reality is that research can’t tell us exactly what to do in the classroom. The best evidence I can offer is empirical: I’ve spent the last year testing these resources with my students, and they represent the best strategy I’ve found to help students increase their confidence and improve their fact fluency.

No strategy is perfect. Over the last year my students have made a lot of progress. At the same time, that progress is often slow and inconsistent. That’s the reality of learning. When progress is slow, I fall back on two strategies. First, can I take what we’re learning, break it down into smaller chunks, and focus on one chunk at a time? And second, can I make more explicit connections between what students already know and what I want them to learn?

These materials are just pieces of paper. The design is to do those two things: break learning down into smaller chunks, and make clear connections between new learning and prior knowledge. The pieces of paper will only cause students to learn if the teacher helps students think hard about the problems and make connections.

Here’s a nice mental model for learning math facts. It’s called the “Instructional Hierarchy.”

• Acquisition: Students learn math facts by connecting new facts to what they already know. The goal is to answer accurately but using a range of strategies, without an emphasis on speed.

• Fluency: Students practice, getting more automatic over time. The goal is for math facts to be answered accurately and quickly.

• Generalization: Students apply their math fact knowledge in lots of different ways.

Let’s imagine I’m tutoring a student in their multiplication facts. First I would focus on acquisition. We would draw lots of pictures, talk about the commutative property, practice skip-counting, look for patterns in each fact family, and connect the specific math facts to prior knowledge. Then, once students have a solid foundation, we would work on fluency. Here the goal is to get faster. I would tailor the specifics to the student. We might see how many they can answer in one minute or three minutes. I would pay close attention to the student’s confidence and frustration. I would adjust the pace as we go, and figure out what motivates the student to make progress. If acquisition goes smoothly we might introduce more math facts at once. If acquisition is slow we might focus on one fact family at a time. Then, once students have some decent fluency with their math facts, we can focus on the generalization phase. As one example, I’m a big fan of the Beast Academy game All Ten, which is a daily number challenge requiring a lot of flexibility with arithmetic.

Here’s a nice graphic summarizing the Instructional Hierarchy.1

Timed Fact Practice

Timed fact practice is a perpetual flashpoint in math education. Some teachers will argue timed fact practice causes math anxiety and is unnecessary. Others are convinced it’s a crucial part of building automaticity.

The Instructional Hierarchy tries to settle this argument. Advocates for this framework would argue that the answer just depends on the stage. In the acquisition stage, timed practice isn’t appropriate. The goal is to get students calculating accurately and connecting what they’re learning to what they already know. In the fluency stage, students need to work on increasing speed to become more automatic and effortless in their calculation. Problem solved!

The issue is that in my class — like most math classes — I have some students who need work on acquisition, some students who need work on fluency, and some students who are in the generalization stage.

This is the core challenge of fact fluency. If I’m working with a student one-on-one, the instructional hierarchy is really useful! In a classroom with a wide range of students, the reality is much more complicated.

One phrase I often hear from math teachers is describing large amounts of practice as “drill and kill.” Drill and kill is excessive practice that undermines motivation and is often seen as inefficient.

I’m a huge fan of practice. That said, I think the instructional hierarchy can help us understand what teachers mean by “drill and kill” and why repetitive practice can be unproductive in some contexts. Repetitive practice is most useful when students are in the fluency stage. Asking students to practice something while they are still in the acquisition stage is going to be inefficent. I’ve seen this over and over again: students are trying to practice something but their working memory is overwhelmed, their work is slow and error-prone, the practice isn’t very productive, and the whole experience is unpleasant for students.

This type of practice often leads to avoidance behaviors. That one student always has to go to the bathroom, or the tip of their pencil breaks, or they hunch over their paper without doing much of anything. Even if the student is trying their best, they may solve problems very slowly and get less practice than their peers. And if the students who need practice the most aren’t getting very much of it, they won’t make much progress.

I have a wide range of students in the room. How can I structure a whole-class activity so that students can make progress on acquisition, fluency, and generalization, all at the same time?

Short Cycles

One solution to this challenge is differentiation. You create or adopt resources that try to meet each student where they are. Maybe one student is working on their addition fact fluency, another on multiplication fact fluency, another on long division, and so on. I don’t doubt that, when executed well, this can be a good approach. There are two challenges: first, it’s a ton of work, and second, it takes valuable time. If that system works for you, that’s great! It is just too much for me.

Here’s a simple representation of the Instructional Hierarchy:

Students are in all different places on that spectrum. Some are at the very beginning of their multiplication fact knowledge, others are ready to generalize.

Here is what I do:

Instead of treating multiplication facts like one big monolith, I break it down into smaller chunks. Practice one fact family at a time. Focus on acquisition first, then move to fluency, while mixing in lots of generalization.

Is this approach perfect? No. But it’s the best solution I’ve found to the core problem.

There’s a naive approach to multiplication facts: students don’t know their facts, so we should have them practice. My experience is that a significant group of students makes very slow progress, and often finds this process incredibly unpleasant. They are in the acquisition stage, and trying to do acquisition for all of their math facts at once is slow and demotivating. Breaking facts into smaller chunks avoids excessive, repetitive practice before students are ready for it.

I’m left with a simple indicator to figure out whether my fact fluency practice is productive. I focus on my students who I know need the practice the most. Are they getting discouraged and avoidant? Are they working slowly and not getting much practice at all? If so, I need to step back and break practice down into smaller chunks. At the same time, I mix in opportunities for generalization like All Ten. Are there some students who don’t need the practice? Sure. But quick chunks of practice won’t hurt, and I’m making sure to include some challenges appropriate to their skills. By focusing on whole-class practice rather than trying to differentiate, the whole process takes less time.

Alright this post is long enough. Next week I’ll be back with more details about my routine.

Selection Effects Are Large

Some students, regardless of teaching quality, perform better than others. This is one of the most persistent phenomena in education. Differences in achievement are not random. They are correlated with a whole bunch of other stuff, like family income. They aren’t destiny, but they are the water in which we swim, creating currents that affect everything else we do in education. These effects are persistent and large.

The most basic form of this is something I’ve heard countless times in my career: school A does thing X and they get better results than we do, therefore we should do thing X.

Thing X might be good. But simply because selection effects are large, the most likely reason school A does better is that they have a different intake of students. In the vast majority of cases, if one school performs better than another, the reason is selection, simply because selection effects are large.

Here’s a second, more subtle example. Lots of schools encourage reading by setting aside a block of time where students choose books and read on their own. This is sometimes called SSR or “sustained silent reading.” The practice follows the commonsense observation that students who read more are stronger readers.

Unfortunately, research does not support this practice. Tim Shanahan has a nice blog post summarizing the research here. The basic observation is that this is a selection effect. Strong readers read more, but this may be because they are stronger readers. Telling striving readers to read independently is not the best way to help them become strong readers. There are all sorts of reasons for this — students with fewer reading skills may choose books that are too easy, or only engage shallowly with what they read. Or it may simply be because teacher-led reading is better at improving reading skills for this group of students than independent reading.

This is why education research exists. We don’t just say, “this group does better, therefore they must be doing things right.” Research involves conducting experiments, controlling for different variables, and triangulating multiple sources of evidence. It’s not easy, because selection effects often muddy the waters. You have to understand selection effects to figure out which interventions a school should prioritize.

Intervention Effects Are Small

I’m defining “intervention” here as “something we did in an attempt to help students learn more.”

You name it. Everything we try, the effect is small. That study folks were talking about a few weeks ago about tracking (setting) in math class? Small effect. Growth mindset? Small. Reducing class size? Small.

There are plenty of studies that report large effects. What’s up with these? Many are correlational: there’s a correlation between something and student learning, but that is typically capturing selection effects rather than intervention effects. Some studies use researcher-designed measures. The researchers use their own assessments and produce a large result. Inevitably, when that research is reproduced with something tougher to game, the effect is smaller. Many large effect sizes come from experiments done in a lab setting rather than in real classrooms. Finally, there are all sorts of ways one can use statistical tomfoolery to make effects look larger than they actually are.

Another thing to watch out for is taking a bundle of different initiatives and calling it one big thing. The “science of reading” movement is a great example here. Have some states shown impressive improvement in literacy outcomes? Absolutely. But the most notable successes have involved a bunch of distinct changes, from teacher training to curriculum to phonics to accountability. Many states have struggled to replicate those successes, which underscores the fact that this is a bundle of different interventions stacked together and it’s not easy to pick out the most important active ingredients. This doesn’t mean the “science of reading” is fake or useless. It just means that it’s hard to do well, and isn’t going to cause miraculous improvement all at once.

When I say “intervention effects are small,” you might wonder what I mean by small. By small, I mean intervention effects are much smaller than selection effects. If we are interested in closing the achievement gap, or if a school that is low-performing wants to improve compared to schools that are high-performing, intervention effects are small relative to those gaps, relative to selection effects.

Small effects aren’t meaningless. There’s no one thing that will transform education outcomes, but if you add up lots of small things you can get a big thing. This changes the calculus. First, let’s be realistic about what we should expect from any one intervention. Then let’s do our best to stack interventions together. Small doesn’t mean all interventions are the same; some are small, some are very small. Let’s pick the best bets, the bets that we can scale and sustain over time.

All Things Fade

Every intervention, everything we do in schools, fades back to the status quo. The “southern surge” — states that have outperformed the rest of the US in 4th grade reading scores, attributed to the science of reading — is mostly gone by 8th grade. High-quality preschool improves outcomes for a few years but outcomes mostly slip away by about 3rd grade. All things fade, and if you want to cause sustained improvement you need a bunch of interventions throughout a student’s education.

This doesn’t mean all interventions fade to nothing. Many effects persist at a low level. But some level of fadeout is inevitable. That doesn’t mean the previous interventions were a failure. It just means that the work is endless. There is no other way.

Optimism

I’m an optimist about education. I think there is enormous potential to improve both the conditions of schooling and outcomes for students. But it’s hard work. One of the hardest problems in education research is distinguishing predictors of achievement from causes of achievement, and figuring out which causes of achievement are highest-leverage for improvement.

If you buy these three laws, and you want to improve education, here's where they lead:

Step one: do a rigorous analysis of the interventions that truly move the needle and improve student outcomes, throwing away all of the lazy correlations and selection effects.

Step two: compare the costs and benefits and implement the highest-leverage interventions to improve education outcomes, recognizing that you will need many of them.

Step three: add more interventions and sustain them over the entirety of the education system. Keep expectations reasonable. Avoid silver bullets. Stay humble, and persist.

What’s a teacher to do?

Most teachers agree students need practice. But how much practice is enough? Put a bunch of smart, experienced teachers in a room and they can give you a pretty good guess for how much practice students will need, but that guess will never be right for every classroom.

I take an empirical approach: practice until learning is secure in long-term memory.

First, I want to be clear about what I’m talking about. This is a strategy for improving retention for procedural skills. Think multi-digit subtraction, or long division, or solving a two-step equation, or finding a derivative using the chain rule. This isn’t about conceptual understanding, or how to initially introduce those skills. The problem here is that you’re teaching a procedural skill, students can do it one day, but they forget it the next.1

My Approach

I teach the skill day one. I have students practice a bit.

Day two, I ask students to try the skill again to check for understanding. I typically use mini whiteboards for this. No scaffolding, no hints, I just want to see how many kids remember. Then, I’ll model, or we’ll look at an example together, or we’ll scaffold up to the skill with some easier questions. Then practice. This isn’t a whole class period: there’s the check for understanding, then a reminder and a bit of scaffolding, then maybe five minutes of practice.

Day three, we do the same thing again. I start with a check for understanding. The key question: how many kids get it right? Let’s say on day two it was 30%. Day three, it might be 60%. The goal is to get that number a little higher each day.

Wash, rinse, repeat. My goal is to practice until students can reliably retrieve the skill the next day.2

When the success rate is low, I’m likely to choose a whole-class worked example and scaffolding. Once the success rate gets up to 80-90%, I start focusing on those final few students. Most students now know it, so I check in individually with the students who missed the check for understanding, try to figure out how to help them, and then we all practice for five minutes.

The last 10% can be stubborn. If the skill is important, I try to stick with it until we get to 100%. Are some students ready to move on? Sure. But a quick chunk of practice isn’t doing any harm, and that extra practice makes a huge difference for the students who need it most.3

Once we reach 100%, we shift into retrieval practice mode. Instead of five-minute chunks of practice, I’m mixing these questions into a Do Now or mixed practice assignment. Students should keep seeing them regularly, but one or two questions at a time is enough to keep the learning secure in long-term memory.

What’s Happening Under the Hood

I think it’s helpful to have a mental model for what’s happening in my students’ minds during this process.

Cognitive scientists delineate two different types of memory strength: retrieval strength and storage strength.

When you first learn something, retrieval strength is high. If I tell you that 18 x 8 = 144 and then ask you for 18 x 8, you will have high retrieval strength for that math fact. You won’t have high storage strength, and the learning fades quickly. Here’s what that looks like:

If I ask you for the value of π, you can probably tell me it’s about 3.14. You’ve seen it plenty of times before. You can retrieve it from memory, and that memory is durable. Rerieval strength and storage strength are both high. Here’s what that looks like:

Let’s think back to day one. Students can perform a skill. They get a bunch of questions right in a row, but they forget quickly. In that moment students have high retrieval strength, but low storage strength. They can do it in the moment, but it doesn’t quite stick. Here’s a gif to represent this situation. After practicing a bit, retrieval strength is high, but we haven’t added much to storage strength and retrieval strength fades quickly.4

The best way to improve storage strength is counterintuitive: forget a little bit, then practice again. It looks like this:

That’s the purpose of day two, and day three, and so on. Once you’ve practiced something 10 times in a row, the 11th time isn’t adding much. But if you practice 10 times, wait a day, then practice again — that does much more to improve storage strength.

High storage strength has two main benefits. First, forgetting is slower. Here’s what the model looks like when storage strength is high:

When storage strength is high, learning is more durable.5

Second, relearning is faster. All learning fades. If you don’t use it, you lose it. But if storage strength is high, you can relearn that topic quickly.6

This is the answer to the titular question. How much practice do students need? Enough that they can reliably retrieve the skill the next day. Learning is a change in long-term memory, and the best practical way to measure long-term memory is to wait a day and see what students remember. Continue practicing until students reach that threshold. Retrieval practice continues at a smaller scale after that point, and if retention slips we practice a bit more.7

Where to From Here

I know what some of you are thinking. This whole thing sounds nice, but there’s no time. I can’t spend 5-10 minutes practicing a skill from yesterday or last week every class. I have to move on to the next objective.

I hear you. To take this type of practice seriously requires a bit of instructional redesign. One approach is to set aside the first chunk of class to check for understanding and practice skills from previous days or weeks. That takes time away from your current lesson, but it averages out in the end: you’ll have less time for practice on the day’s objective, but make up for it with more time to practice on future days.

There is another option. This is where I plug my favorite off-the-wall curriculum design idea. Here is a rough sketch of a typical 7th grade sequence of units:

Six big ideas, one after another.

Here is what I did this year:

I took the toughest topics, the ones that students typically have the most trouble with, and stretched them out through the entire year. Multiple strands, taught side by side. Rational numbers means adding, subtracting, multiplying, and dividing, with negatives, fractions, and decimals. Equations and expressions includes a bunch of stuff, but the keys are finding equivalent expressions and solving two-step equations. Every day, we work a little bit on those two topics, as well as one of the other units. Sometimes it’s just a few minutes of practice on one strand. Sometimes it’s introducing a new concept. A bit of progress, every day. This structure provided time and flexibility to give students the spaced, repeated practice I’m describing in this post. It’s been a pain to plan, but it’s also my favorite change I’ve made to my teaching in the last few years.

Whatever approach you choose, you’ll need to accept that, if learning is a change in long-term memory, we can’t measure the success of our teaching in a single lesson. To figure out if students have learned, I need to check for understanding over multiple days and provide more practice when necessary. Many teachers and schools are strongly committed to the idea that every lesson must have exactly one objective, and every part of the lesson must be in service of that objective. I think we should let go of that idea. It simply is not consistent with how humans learn.

Once upon a time on Tralfamadore there were creatures who weren’t anything like machines. They weren’t dependable. They weren’t efficient. They weren’t predictable. They weren’t durable. And these poor creatures were obsessed by the idea that everything that existed had to have a purpose, and that some purposes were higher than others. These creatures spent most of their time trying to find out what their purpose was. And every time they found out what seemed to be a purpose of themselves, the purpose seemed so low that the creatures were filled with disgust and shame. And, rather than serve such a low purpose, the creatures would make a machine to serve it. This left the creatures free to serve higher purposes. But whenever they found a higher purpose, the purpose still wasn’t high enough. So machines were made to serve higher purposes, too. And the machines did everything so expertly that they were finally given the job of finding out what the highest purpose of the creatures could be. The machines reported in all honesty that the creatures couldn’t really be said to have any purpose at all. The creatures thereupon began slaying each other, because they hated purposeless things above all else. And they discovered that they weren’t even very good at slaying. So they turned that job over to the machines, too. And the machines finished up the job in less time than it takes to say, “Tralfamadore.”

-Kurt Vonnegut, The Sirens of Titan

In the physical world, humans have transcended many purposes. I no longer have to walk to the stream to get water, or start a fire to cook dinner, or perform many other mundane everyday tasks. My time is freed to serve higher purposes.

While humans have dramatically transformed the physical world, education is the realm of cognition. In the cognitive world, we are operating the same hardware that made its way out of the savannah a few tens of thousands of years ago. There are no shortcuts. The job of education is to make students smarter. That means the slow, effortful accumulation of synapses and schemas.

It’s physics. Thinking is expensive. Humans evolved to learn only with repeated effort. Those synapses only grow stronger by firing, over and over again. Encoding. Consolidation. Retrieval. Connection. Application. And in the modern world there is a lot to learn.

The magic of the human brain is that, once we have learned it, much of our own knowledge is invisible to us, and the effort expended to learn is forgotten. As long as humans exist we will want desperately to find a higher purpose, to move beyond the mundane everyday and reach for something more. We will believe that there must be an easier way, and we will try to find shortcuts for the endless grind that it takes to learn and grow intellectually.

The education world is full of “if only…” If only we could teach critical thinking. If only we could make learning more relevant. If only students had more choice. If only parents were more supportive. If only smaller class sizes. If only more school funding.

I have enormous optimism that we can produce better outcomes than what we see in today’s education system. To make that change we need to be humble, to accept that our job is to cause the slow, steady accumulation of knowledge that opens doors for young people and shows them what their minds are capable of. The volume of knowledge we need in the modern world only grows, while our brains learn at the same speed they have learned for tens of thousands of years.

Long-time readers know that I am a bit of a traditionalist and a staunch advocate for boring regular school. But the learning I’m talking about doesn’t only happen in schools, and it’s absolutely possible that we will stumble across a better model for education than the one we have. I don’t think the schools we have today work perfectly. Stop by my classroom some morning. You’ll see something a bit messy, still a work in progress. There is plenty of room to grow, to make student effort more efficient, to cut through the noise and focus on the learning that matters most.

Here is one thing I am sure of. If we do improve education, it won’t be fueled by “if only…” and attempts to bypass the humble grind of learning. Improving education will happen because of people who take the grind seriously, who dive deep into the details of the content that we want students to learn, who put the nuts and bolts together in a way that does all the little things right. That’s my purpose, my priority in my classroom. There are no shortcuts. There are a thousand little decisions every day, and my own slow, effortful learning as I figure out a little more about learning, as I help students figure out a little more about the world we live in.

One term cognitive scientists use to describe this type of thinking is “elaboration.”

The vast majority of teachers I’ve met understand this idea intuitively. We want students to think about big ideas. Walk into a random classroom and you might find teachers asking hard questions, prompting students to dig deeper.

A Non-Example

The hard part of elaboration questions isn’t the questions themselves. The hard part is getting every student thinking about the questions.

Here is a non-example.

One practice I’ve seen recommended in this general direction is for students to create their own problems. Teach a topic, then ask students to write their own problems. Maybe they trade with a partner, or we write them on the board and solve them together. The logic looks like this: generating problems involves deeper processing than solving problems, so it will cause students to think more deeply about what they are learning.

That’s probably true! Here’s the catch: when I’ve used this strategy I find that some students are able to generate their own problems, while others stare at me with blank expressions unsure of where to start.1

There are two key qualities of a good elaboration question: first, it should require effortful thinking. Second, it should cause as many students as possible to do that thinking. Below are some examples of my favorite elaboration questions. These are my favorite because, in my experience, they are the best questions for getting reluctant students to engage with this type of thinking.

Elaboration Questions

Find the pattern

Give students a sequence of problems with some sort of repetition that reveals a pattern:

• Find 10% of 200.

• Find 25% of 200.

• Find 3% of 200.

• Find 4% of 200.

• Find 6% of 200.

• Find 60% of 200.

• Find 80% of 200.

Then ask: what patterns do you notice?

We can talk about the patterns, we can make predictions about new problems using those patterns, and we can extend the pattern in different directions — maybe finding percents of 300 or 400 to see what happens next. There’s a ton of math here. Students are practicing percents, while also creating an opportunity to think hard about why percents do what they do.

Expansion

Here is a sequence of problems that get gradually harder:2

• 5 pounds of cheese cost $50. How much does one pound cost?

• 2 pounds of cheese cost $8. How much does one pound cost?

• 3 pounds of cheese cost $11.40. How much does one pound cost?

• 3 pounds of cheese cost $12. How much does one pound cost?

• 3 pounds of cheese cost $6. How much does one pound cost?

• 2 pounds of cheese cost $6. How much does one pound cost?

• 1 pound of cheese costs $6. How much does one pound cost?

• 0.5 pounds of cheese costs $6. How much does one pound cost?

• 1/2 of a pound of cheese costs $6. How much does one pound cost?

• 1/3 of a pound of cheese costs $6. How much does one pound cost?

• 2/3 of a pound of cheese costs $6. How much does one pound cost?

The goal here isn’t that every student solves every problem. The goal is to increase the difficulty gradually, figure out where students get stuck, and use that information to help students learn something new. I often find that when I sequence questions in this way, students can solve much harder problems than if I just throw out the problem cold.

What If?

Students solve a problem, or we solve a problem together. We leave the solution visible, on paper or on the board.

Then I ask what if — here’s an example.

We solve the equation: 2x + 1 = 11.

What if the equation was -2x + 1 = 11?

The exact questions here will depend on what your students know and don’t know. This is a great way to get students thinking hard if they’re reasonably confident with the first equation, confident multiplying negatives, but don’t have much experience with negatives in equations.

Non-Examples

Students solve a problem, or we solve a problem together. Then I ask a question that is a deliberate contrast with the last problem, where students might overgeneralize a rule.

First question: distribute

2(3x + 10y - 5z)

Next question: distribute

2(3x + 10y) - 5z

A non-example helps to avoid the problem of going on autopilot, copying an example without considering the structure of that example. It often leads to great conversations!

Stepping Back

The last few examples of elaboration questions all have something in common. They are all designed to get students solving one problem or several problems accurately before doing some deeper thinking. That’s a great general strategy to get students thinking hard about elaboration questions: build confidence with a few things students know how to do, and then use that confidence as a springboard to tackle tougher questions. The details here will depend on your students. The initial questions need to be ones students can solve confidently, and the leap you’re asking students to make needs to be accessible but not too easy.

Ok, back to a few more examples.

Sentence Completion

Writing in math class can feel like a pain. I could say a lot more about this, but for this post I have one go-to teaching move to get students writing about math.3 Here’s what it looks like:

Complete the sentence:

To find the surface area of the prism, Lin found the areas of the three rectangles, added them, and multiplied by 2. She multiplied by 2 because…

This is one of those things students end up doing without really understanding. “Oh yea, I multiply by two for those problems…” Sentence completion is the best way I’ve found to get students thinking and writing about the why, without getting a bunch of blank stares and blank papers in response.

Numberless Word Problems

Word problems are a pain. This could also be a much longer post. But the short version: a good way to get students thinking hard about word problems is to take away the numbers. Without numbers, students can’t just grab numbers, smush them together with an operation, and move on. You can do this by covering up the numbers in a problem you already use, or writing your own. An example might look like this.

A runner jogs one lap around a circular track. How far does the runner jog?

Then, prompt students. What do you need to know to answer the question? What would you do with that number if I gave it to you? What if I gave you the radius instead?

Elaboration Questions

You can call these whatever you like. I’m partial to “elaboration questions” because it’s concise but you might like something different. You also might have lots of other strategies that work well! Let me know what works for you.

The key idea I want to emphasize: the goal of elaboration questions is not to get a handful of students thinking hard. The goal is to get every student, or as close as possible, thinking as hard as possible. Some of the examples I gave might seem simple. That’s often the case! We are teachers. We know a lot more than our students. It’s easy to overestimate what students know, ask a bunch of hard questions, and get a lot of blank stares in response. My priority when I ask elaboration questions is to get as many students thinking as I can. These strategies are the best tools I’ve found to do so. Even questions that seem simple on the surface can work well when we scaffold and sequence them in ways to get every student thinking.

Whether you use the Marzano model or something else, this is common advice in schools. Post the learning goal. Reference the learning goal throughout the lesson. Make sure students can explain the learning goal.

I think this is unhelpful advice.

Working Memory

Memory is the residue of thought.1 One of the core challenges of teaching is that the place where humans think, our working memory, has limited capacity. To maximize learning, teachers need to be deliberate with what students are thinking about. To use the language of cognitive load theory, extraneous load on working memory makes learning less likely.

My argument in this post is that all the hullaballoo about learning goals can end up being extraneous load on working memory. The idea that teachers should constantly reference learning goals and that students should, at any point in the lesson, be able to state the learning goal, is simply not reasonable given human cognitive architecture. Students cannot hold all of that information in their minds while also thinking hard about the stuff we want them to learn.

One qualifier here. I’m not saying that we should avoid telling students what we’re learning, or that we should march students into class and give them a series of tasks with no purpose or explanation. My argument is that the way we communicate those ideas with students should be conscious of the limits of working memory.

Circumference

Ok here’s example one. Let’s say I’m teaching students to find the circumference of a circle. I might start the lesson by saying, “Today we’re going to learn how to calculate something new as part of our work with circles.” I would avoid mentioning circumference. Students don’t know what that means yet. Then, the first thing I do in this lesson is to practice working with radius and diameter of circles. Draw a circle with a radius of 3 inches. If a circle has a radius of 10 inches, what is its diameter? Etc. I would tell students that we’re beginning with a review of radius and diameter because they’ll need to be able to convert between them later in the lesson. Here I am laser-focused on radius and diameter. I’m not introducing anything extra. I’m trying to make sure students are good to go with those ideas, checking for understanding and addressing any issues that come up.

Then I would introduce the idea of circumference with a bunch of examples and non-examples. Again, laser focus here: my goal is to help students understand what the word circumference means and why it can be useful. I’ll communicate that thinking about circumference is our main goal for the day, but I don’t want to get hung up on questions about how to find circumference yet. I’m not worried about whether a student can state the full learning objective for the lesson. I’m trying to make sure students understand the meaning of the word “circumference.”

Later in the lesson I’ll show students how to find circumference and we’ll practice a bit. This is hard! Students need to remember pi, they need to convert from radius to diameter if necessary, they’re using a calculator, they need to remember to write the units. Students are often focused on the calculation and lose track of the bigger picture. That’s totally normal. If you walk into my lesson and ask students what they’re learning, they might say they aren’t sure. That’s a reality of the human mind: students’ working memory is busy with the calculations, and they’ve lost the bigger picture.

To some teachers this is mindless calculation or memorization. I disagree. Part of learning circumference is a bit of practice to get the basic muscle memory for the calculation. We should expect that students will lose the forest for the trees, focusing on the calculation rather than what it represents. Once they gain a bit of confidence with the calculation, it’s my job to help everyone step back, think about what that calculation means, and connect it to everything we were doing earlier in the lesson. This is the part where I emphasize our learning goal and help students see the bigger picture.

I want to be clear: students should understand the learning goal by the end of the lesson, and I should also communicate to students the purpose of each part of the lesson along the way. Where I disagree with the conventional wisdom is that I don’t want to spend a bunch of time reading the learning goal at the start of the lesson, restating it multiple times, and making sure students can articulate the goal at any point during class. All of that feels like a distraction from the math I want students thinking about along the way.

Two-Step Equations

Here’s another example. Let’s say I want students to solve two-step equations, like 2x + 1 = 11. At the start of class I’ll tell students that we’re learning about a new type of equation today. I won’t stress about “two-step equations” — students might not know what those are yet. Often a goal for this lesson would be written something like “solve two-step equations of the form ax + b =c or ax - b = c where a, b, and c are whole numbers.” That’s a ton of extraneous information. Students don’t need to know the definition of whole numbers or parse what ax + b = c means before launching the lesson.

Again, I would start the lesson with some review and practice of prior knowledge. We would practice equations like 2x = 10 and x + 1 = 11 and lots of other similar equations. We would also practice evaluating expressions like 2x + 1 where x = 5. I would explain to students that we’re practicing these skills because we’ll need them to solve more complicated equations, but that’s it. I don’t want students worried about what those equations look like or what the larger objective is, I want to focus on one-step equations.

Then I introduce two-step equations. Again, I’m not worried about stating the learning goal here. What I want students thinking about is the connection between the one-step equations we were just solving and the new equation I’m showing them. How is 2x + 1 = 11 similar to x + 1 = 11? What happens after I subtract 1 from both sides of each equation? What’s left, and how do we solve that equation? Ok, what if the equation is 2x + 3 = 11? What’s different? I want student thinking laser-focused on my questions, and the connections between one problem and the next. I do not care if they can state the learning goal. That is extraneous load.

Later in the lesson we’ll look at equations with subtraction. Again, I don’t want to say, “hey students, our goal is to solve two-step equations of the form ax + b = c or ax - b = c. Look, we’re about to meet our objective!” This part of the lesson is tricky. Why do we sometimes add and sometimes subtract when solving equations? That’s a hard concept. I want students thinking about the meaning of those operations, not trying to parse the meaning of the learning goal.

As the lesson nears the end, I will step back and help students see what they’ve learned and name these equations as “two-step equations.” But I want to wait until students get some fluency with the math, so they have space in their working memory to see both the forest and the trees.

The Pitfalls of Learning Goals

In any class there will be some students who are more confident with the relevant prior knowledge than others. Some students will arrive to class already fluent with diameter and radius, or proficient with one-step equations. Others will be a bit shaky. Those more fluent students have extra space in their working memory. They can hear a learning goal at the start of class and repeat it back to you throughout the lesson. The students who don’t have that fluency simply do not have the cognitive capacity to do so. Their working memory is busy thinking about what we want them to learn, as it should be.

It’s easy to get this backwards. You might observe that stronger students can keep track of the learning goal, and think that you need to put more emphasis on the learning goal so that all students can keep track of it. That gets the causation backwards. Students aren’t learning because they can keep track of the learning goal. They can keep track of the learning goal because they already have a lot of the skills we want to teach.

I understand where all the emphasis on learning goals comes from. It’s easy as a teacher to lose track of the goal ourselves and end up planning a bunch of activities that don’t have a clear purpose. Every part of a lesson should contribute to student learning.

Also, we should absolutely tell students what they are going to learn. If nothing else, it’s a basic courtesy. My basic framework is to tell students what they’ll be learning at the beginning of the lesson, but use student-friendly language rather than stating a formal learning goal. During the lesson, I’m focused on managing working memory load, connecting new learning to prior knowledge, and making sure students understand what I want them to learn from each chunk of the lesson. By the end of the lesson, the learning goal should be clear to students.

There are all sorts of pitfalls of an emphasis on learning goals. Posting learning goals can become an exercise in compliance: the principal comes around and makes sure the learning goal is written on the board because that’s an easy thing to observe for, never mind whether or not students are learning. I’m not required to do that in my current job but I have been in the past. It always feels like a bit of theater where we go through the motions without any emphasis on whether students are learning. There’s opportunity cost here: all the energy a school spends chasing teachers around and telling them to write the learning goal on the board could be spent on something else.2

Another pitfall is that not every element of a lesson has to connect to a single learning goal. I start and end every class with a bit of mixed retrieval practice. Twice a week we practice fact fluency. Those aren’t tied to the day’s learning goal and that’s fine. I do have clear goals for what I want to accomplish during those times, and I communicate those goals to students. It just isn’t part of one big goal for the lesson.3

My real issue is that, in my experience, an emphasis on learning goals focuses on surface features rather than the substance of learning. Thinking drives learning. Everything I do in the classroom is meant to get students thinking. At best, all of this stuff about learning goals feels like a distraction from thinking. At worst, it can actively interfere with student thinking by overloading working memory at the wrong time.

I’ve Moved On

It’s funny these articles came out now because I did the experiment in January. I feel like I’ve moved on. It’s not an experiment anymore. It’s just the way I teach. I radically reduced technology use in my class, I’m happy with that, and I’m thinking about a lot of other aspects of my teaching these days. I emphasize this because there’s a growing movement advocating for less technology in schools. I think that’s a good thing! But classroom technology is just one of many, many things that matter for student learning. Some of the big claims I’ve seen about technology use harming learning don’t seem in line with the evidence. Let’s reduce classroom technology use, sure, but let’s be realistic about the impact that will have on student achievement, and also work on the dozens of other things that can help students learn.

Just a Tool

A few responses said something along the lines of, “Sure, technology can be overused, but computers are a tool just like any other. Why is it such a big deal not to use student-facing technology at all? Why not only use it when it benefits learning?”

Before my experiment, I would have said that I’m someone who doesn’t use technology very much. I’d estimate we used Chromebooks about 20% of class time. Going cold turkey for a month was a helpful way to see that even that 20% was way too much. I’m also not completely zero-tech now. We’ve used Chromebooks twice in the last three months, both for narrow practical purposes.1 I’d recommend teachers give a tech-free experiment a try, just to see how they feel about it.

A related point here is that schools spend a ton of time, energy, and money on student technology. Something like 90% of US schools provide one-to-one devices to students. On Monday, when I asked students to pull out their Chromebooks, one looked like this:

This is the type of stuff teachers and other staff deal with when students carry Chromebooks around all day every day. The practical question here is whether schools should be providing one-to-one devices. Maybe schools should have a computer lab or shared Chromebook cart instead.

Assessment

A final response I saw a few times was surprise that I was giving quizzes and tests on Chromebooks before my tech-free experiment. I get it. That was dumb. In retrospect, it seems obvious students should be assessed on paper. I started giving assessments online during covid for practical reasons, and kindof just never thought twice about it again. They were easy to grade and there was no paper to keep track of for absent students. Students take most high-stakes assessments online now, so there’s a common logic in schools that students should practice taking other assessments online. There’s also a bit of cachet that comes with doing things with technology. I remember one time a few years ago the superintendent was doing a classroom walkthrough and I was giving a quiz. She complimented how nifty one of the interactive online questions was. They can be pretty nifty! That doesn’t mean it was good teaching, but there’s an assumption in education that using technology is innovative or whatever. This is a long-winded way of saying: trying a tech-free experiment can help teachers look at their practices from a new perspective, and maybe realize that they’re using technology for the wrong reasons.

There’s a lot of talk these days about how students are addicted to their phones. I think teachers can just as easily get addicted to classroom technology. The classroom tech I used before definitely made my life a bit easier. I work a bit harder now. That’s fine with me. I think it’s worth the trade. Students work a bit harder too! Those are exactly the types of conversations I hope schools start having about their technology use.

Here is the practice that my current curriculum provides for solving two-step equations:

This is not enough practice. (These are “High Quality Instructional Materials,” by the way.1)

Let’s step back for a second. In 7th grade, students learn to solve two-step equations with rational numbers. Negatives, fractions, decimals are all fair game. In 6th grade, students learn to solve one-step equations.

This is a tough standard! If you haven’t been a 7th grader for a long time, it might seem easy. It is not.

Let’s go back to that page above. That’s the first lesson in the unit on equations. Students do two activities to learn how to solve two-step equations. Then, this practice. The next page has a bunch more word problems, and that’s it.

A good place to start with two-step equations is to work on equations with whole numbers and whole number solutions. No fractions, no negatives. There are two (two!!!!) equations that fit that description to start this set of problems. Then a problem with a fractional answer, a problem with a negative answer, and on to word problems.

If I’m lucky, most students get through the first two problems before hitting an absolute wall at #3 or #4 and giving up. That’s the best-case scenario if I use these materials as intended.

For the vast, vast majority of 7th grade students, this is completely, utterly, woefully insufficient practice.

The Solution

A key principle of teaching is to provide plenty of practice, and to carefully control the complexity of that practice over time.

A broad strategy to help students solve equations like these:

• Start with one-step equations using positive numbers. Students learned them last year, but they would benefit from some review and practice.

• Then two-step equations with whole numbers that work out nicely.

• Then two-step equations with larger numbers, decimals, fractional answers, etc.

• Then one-step equations with negative numbers.

• Then two-step equations with negative numbers.

• Then two-step equations with negatives, fractions, decimals, the works.

(I’m oversimplifying a bit here, but this gives a sense of all the math that’s compressed into a handful of problems in the curriculum I’m given.)

At each stage, students should practice and get confident before moving on.

But I have these materials I’m supposed to use, and they fall short.

My solution, right now, is to create additional practice materials using generative AI. Here’s an example:

Simple, straightforward. All the answers are whole numbers. Everything works out nicely. Students can gain some confidence with these before moving on to tougher problems. I can generate this handout and similar handouts in under a minute. I generate lots of little worksheets like this one, increasing the difficulty bit by bit, checking for understanding along the way, and helping students build confidence with each skill.

If you’re interested in the workflow, I’ll drop it in a footnote.2

Stepping Back

This is the completely boring part of teaching. Students need practice, the materials I have don’t provide enough practice, so I generate more. It’s also the part of teaching that often gets skipped. It takes time and effort to create materials like this. The default option for lots of teachers, and for past versions of me, is to say hey, this is what the curriculum says, so that’s what we’ll do. Having a tool that reduces the time and effort to generate more practice makes it more likely I’ll take the time to do it. And it’s not just tacking on more practice to each lesson. I can get really granular, creating an on-ramp to tough tasks by helping students practice the constituent parts before the tough stuff or increase the difficulty more slowly than the curriculum prescribes.

To some people, the promise of generative AI is being able to vibe-code interactive learning apps on the fly. Maybe! I don’t know. I think a typical math teacher would benefit more from being able to quickly generate practice at a granular level for students in small chunks with very little friction. We all have a tendency to focus on the flashy, fun, exciting parts of teaching, and to ignore the nitty-gritty details that seem boring but matter more for regular everyday learning.

Another observation I have here is that I shouldn’t need generative AI to do this. It is the year 2026. My curriculum should come with a tool to generate this type of practice, and ideally a ladder of recommended skills to step students through. I’m not optimistic that curricula in the US will suddenly start providing more paper-and-pencil practice anytime soon, so this is my reality. Still, it wouldn’t be very hard for a curriculum to build a set of worksheet generators like mine.

My final observation, and the one that worries me the most. I use this tool all the time. Most classes, I give students at least one AI-generated handout to practice something — maybe a bit of practice with a prerequisite skill, or to solidify a topic from the day before, or to review something relevant for an upcoming lesson. As I’ve used AI more, I’ve noticed a change. Using AI to generate a worksheet is easy, so I gravitate more toward AI-generated materials. I’m less likely to look at other resources or generate something myself. Creating my own materials might be a better fit with what my students need, but it takes more effort.

This is the trend of technology. Once a machine can do something, it’s tempting to turn that job over to the machines entirely. I do still create some of my own resources from scratch — you can check out examples of expansion sequences or because, but, so activities. But as the year has gone on, I can feel the pull. That worries me! I want to do the intellectual work of responding to my students’ needs, not just going with whatever is easiest. The danger isn’t that AI-generated materials are bad. Often they’re totally fine, and the materials they’re replacing are much worse. The risk is that I start outsourcing the thinking that should be at the center of my job. If I’m not careful, I end up on autopilot. Lesson planning becomes printing out a few AI-generated worksheets, hoping for the best, and calling it a day. Right now, this is the best option I have, but the more I use AI, the more nervous I feel about it.

2

Here’s the basic workflow for my AI-generated worksheets:

I created a Gemini Gem. You can check it out here. It’s basically a way of pre-loading the LLM with examples of my preferred format and style. I open the gem and request a specific type of worksheet.

The worksheet pops out. I click the little copy button on the code snippet.

Next, you’ll need a free account at Overleaf. It’s easy, if you have a Google account you can link it and be set up in seconds. Start a project. Paste the code into the center pane. Click “Recompile” and the worksheet will show up in the right pane. You don’t need to know any of the code! I didn’t when I started. Over time I’ve figured out some stuff so I can make simple edits rather than having to reprompt. Once the worksheet is ready, there’s a little button next to Recompile to download a pdf. Print, copy, good to go.

The usual LLM caveats apply. I need to be very specific in what I ask for. Sometimes it takes multiple prompts to get what I want. Most of the time it works well, but sometimes the LLM does weird nonsensical stuff and I have to close and start again. Diagrams are a mixed bag, some are good but some are a mess.

This does take a bit of effort to learn. There are tools out there that can generate AI-created worksheets with fewer steps. I prefer this one because it uses a consistent format that I like and it’s more customizable than other tools. I bet a typical teacher could figure out this workflow in five minutes or less.

Fierce Feats of Problem Solving

I shared a classroom with an English teacher over a decade ago who did something he called “Fierce Fridays.” I don’t remember the details, it was some type of close reading routine. And to get students excited, each Friday he would show a picture of something “fierce.”

Fast forward to today.

I adapted the idea into what I call “Fierce Feats of Problem Solving.” I wrote a few months ago about how I incorporate math puzzles, games, and problem solving activities into my classes. Before we do these, I frame the activities as “Fierce Feats of Problem Solving” and show students a picture of something fierce.

The first fierce picture of the year is always this one:

I use all sorts of pictures. Humans, animals. Whatever. If it looks vaguely fierce, it’s in.

Over time, students started contributing. They send me pictures of their pets, or their younger siblings, or weird memes. I’ve accumulated a massive collection of fierce photos. Now I sometimes get stuff like this in my email inbox:

If You’re Not Cracking...

Like many teachers, I love lame little catchphrases. Something I say sometimes is “Let’s get cracking,” as in “Let’s get started.” I think that’s a pretty normal phrase. Check me for a second here. Am I right? Does “get cracking” mean “get started” in this context?

My students this year told me that “cracking” actually means “to do crack.”

Now let’s be clear. My students are wrong. That is not what “cracking” means. I will not let a bunch of 12-year-olds change the meaning of a cherished phrase.

So I turned it into a little call-and-response. As class starts and students sit down to begin their Do Now, I say, “Let’s get cracking! If you’re not cracking...” and students (reluctantly) respond, “you’re lacking!”

Bootytickled

Ok this one is really weird. Brace yourself.

My students recently started saying “bootytickled.” Bootytickled seems to be a synonym for “bothered,” so “Bro why you so bootytickled” would mean something similar to,“Hello friend, why is this bothering you so much?”

Anyway, students started saying that word. I requested they not say it. Seems reasonable, right? This is math class. We should be talking about math, not booties.

A student wondered aloud why I was so bootytickled by the word bootytickled. I observed that it was, in fact, a bit ironic. Turns out many of my students don’t know what “ironic” means, so I taught a little impromptu lesson about irony. I thought I was being clever. Teachable moment, amirite?

This was a mistake. By engaging sincerely with the idea of being bootytickled, I gave the word legitimacy. Now I can’t eradicate it.

Like many teachers I get frustrated or annoyed on a pretty regular basis. Word has spread. When Mr. Kane gets annoyed, make a joke about him being bootytickled.

Happily, this faded after a few weeks and students went back to making 6-7 jokes. But for a while, any time I was visibly annoyed with something, a student would comment that I was bootytickled. For me, it became a kindof weird little reminder. I would get annoyed at someone flipping their water bottle. A student would say, “Mister why you so bootytickled?” And I would say to myself, “Hey, I’m not going to let this bother me.” I would take a little moment to find serenity, push down the annoyance, take the water bottle, and keep teaching.

Be Weird

I don’t recommend copying what I do. Every teacher is weird in their own way. Let your personality shine through in whatever way works for you.

I do think there’s a lesson here. Lean into weirdness. Be human.

One of my perpetually unpopular opinions is that school is good. Age-graded classrooms, one-size-fits-all curriculum, and factory-model schools are easy to hate. Hate them if you like! School is far from perfect, but it’s the best we have. It’s the worst way of educating ever invented…except for all the others.

I think one key reason is the weirdness. A bunch of kids are required to come to my class every day and I give them some math to do. For 50 minutes they’re stuck with me and I’m stuck with them. All the inside jokes and weird little moments are what change an obligation into a ritual. Am I a complete loser in my students’ minds? Absolutely. But I’m a complete loser who students are, more often than not, willing to work hard for. That sounds pretty cool to me.

In the real world where I receive professional development and have random education consultants tell me what to do, the things I hear as evidence-based are...interesting.

Over time I’ve developed a kind of bullshit detector for evidence. There’s no one thing that can tell you if an idea is or is not evidence-based. But looking across a number of dimensions, teachers can make better-informed decisions based on the evidence we’re shown. This post is a kind of crash course in how to think critically about evidence in education.

Effect Size

The first thing to understand about research in education is the meaning of an effect size. Below is a normal distribution. The vertical lines represent one standard deviation. At a basic level, an effect size measures the difference between two groups in standard deviations. An effect size of 1.0 means that the difference is approximately one standard deviation — the distance between vertical lines on the graph below. This means a 50% percentile student moves to the 84th percentile, an 84th percentile student moves to the 98th percentile, etc.

A helpful comparison here is to height. American men are, on average, about 5 feet 9 inches tall (175 centimeters) and the standard deviation is about 3 inches (7.6 centimeters). An effect size for height of 1.0 means the effect is, on average, 3 inches (7.6 centimeters). That’s a big difference! The vast majority of effect sizes in education are less than 1. A rough “average effect size” is in the neighborhood of 0.4, which is about 1.2 inches (3 centimeters). Imagine, two groups of adult men. One group is, on average, about 1 inch taller than the other group.

Height and education are different in many ways, but one thing they have in common is that they are the product of lots and lots and lots of little variables. The average adult height is about 4 inches (10 centimeters) taller today than it was a few hundred years ago. Height isn’t some number that’s programmed into us from birth. But there also isn’t one silver bullet that makes humans taller. Humans are taller today because of lots and lots of little changes in our lives. The effect size of any one intervention to make people taller is small, but collectively those effects add up to a large change. That’s a decent mental model for thinking about education.

Ok, let’s talk about research. We’ve got effect sizes. Lots of folks in education wield “research” as a way to reinforce their ideas. What are some things to watch out for? Preview: there are a lot of things to watch out for!

Nonsense

There’s a lot of nonsense out there. In a real PD session last year, a very highly-paid consultant told us that “current research shows student attention span in minutes is their age, plus or minus two.” That’s nonsense. A quick Google search will reveal a wide range of answers depending on a wide range of factors. I don’t know what else to say here. Learning styles, the learning pyramid, and lots more fall into this category. If something “research says” sounds too neat and tidy, do a bit of research. Check the citations. Follow the evidence. It might be nonsense.

Correlation vs Causation

One piece of “research” that I’ve seen multiple times in my professional development is this John Hattie meta-analysis on “collective teacher efficacy.” This collection of studies says that, when teachers believe in their students, those students learn more. Hattie references an effect size of 1.57, which is massive! The catch is, none of these studies are experimental. This is a correlational result. We know that teachers beliefs in students and high levels of student learning are associated, but we don’t know which is causing the other, or if some other factors are causing both. My hypothesis: high levels of student achievement cause teachers to believe in students. If you work in a school where students do well, it’s easy to believe in those students. If you work in a school that’s struggling, it’s much harder to believe in your students. When a result is correlational, it’s hard to figure out what the result means and easy to draw simplistic conclusions.

Correlational research isn’t all useless. There are lots of places in education where we can’t run experiments for practical or ethical reasons, or we rely on quasi-experiments that aren’t fully randomized. Correlation can give us valuable insights or help triangulate other results. But correlation will always be a limited tool, and it’s easy to be misled. Correlation vs causation is probably the most common issue I see in interpreting education research, so I’m going to drop a bunch more examples in a footnote.1

Bias

Another consultant came to my school and told the English teachers that they needed to use the i-Ready digital platform for at least 45 minutes each week. Research showed, they said, that students who used i-Ready for 45 minutes each week made more progress than their peers.

Look, I am so cynical about stuff like this. If you Google the result, you’ll find something on the Curriculum Associates website (the company that produces i-Ready) saying that research supports 45 minutes blah blah blah. The overview starts with, “Curriculum Associates analyzed data…” Let’s just stop there. The company that produced the product analyzed the data for us. How kind of them! It seems like every curriculum commissions some sort of study like this. We should probably just ignore them all. Unless I am 100% convinced a study on a commercial product was done by a disinterested third party, I’m going to assume there’s some bias involved.

Researcher-Designed vs Standardized Measures

I taught in a summer school program as part of my student teaching. During that program, I was required to give the same test at both the start and the end of summer school. My students grew from an average of 31% to an average of 82%. The standard deviation was 15%, meaning my teaching had an effect size of 3.4. 3.4!!! Even higher than collective teacher efficacy! Am I the best teacher in the world? (No.)

The reason my results looked so good is that my teaching was very tightly aligned to the outcome assessment. In this case, I was teaching word problems. The school had me introduce a very structured approach to annotating word problems where students got points for underlining the question, circling key information, etc. When they took the assessment at the beginning of summer school, students didn’t know how to annotate the problems correctly so they got lots of points off even if they knew the math. My student teaching wasn’t a research study, but if it were we would call this a researcher-designed assessment. In general, it’s much easier to show growth on a researcher-designed assessment because the intervention can focus efforts on narrow measures of success. It’s much harder to demonstrate growth on a broader measure like state standardized tests or university entrance exams. Even less reliable measures like grade point average are much harder to influence at scale than a narrow researcher-designed measure.

One more example of this phenomenon: Dr. Jo Boaler ran an 18-day math summer camp focused on mindset and open-ended tasks. The study reported an effect size of 0.52. That’s pretty big! But it wasn’t measuring a standardized result. The assessment was four open-ended tasks from the MARS project. What does that mean for regular school learning? It’s hard to say. It’s not surprising that a program focused on open-ended tasks caused an increase in scores on open-ended tasks.

Cost-Benefit Analysis

Here’s an interesting study. Researchers ran a randomized trial of a growth mindset intervention, so some students received the intervention and others did not. The intervention was short: two 25-minute sessions. The result was an effect size of 0.11 on grade point average.

That’s a small effect size. In a lot of contexts, we might be pretty dismissive of an effect of 0.11. But here, the intervention is tiny and easy to implement. Two 25-minute sessions. That’s nothing. If education could find a dozen different interventions with an effect size of 0.11 that take 50 total minutes to administer, that could add up to a substantive change in outcomes. Let’s compare that to a recent study on Khan Academy. Following Khan Academy’s recommendations for 30 minutes per week of computerized learning resulted in an effect size of 0.08. And that’s for the students who spent 30 minutes on the platform, which was only 10% of the total! As any teacher will tell you, finding time for two 25-minute lessons on growth mindset isn’t too hard. Lots of schools have a homeroom or advisory block where they can squeeze in this intervention. Getting students to spend 30 minutes every week on Khan Academy: surprisingly hard, and has a smaller effect size.

This isn’t a full-on endorsement of growth mindset. There’s a lot of variability in growth mindset results, and the researchers always emphasize that growth mindset interventions are challenging to design. The point here is that, if the research is rigorous and randomized and the cost is small, even small effect sizes can matter.

Comparison Group

Rigorous research makes a comparison. If we want to know how well something works, we need to ask: what is it being compared to? Schools have finite time and resources. Teachers constantly navigate tradeoffs.

The World Bank published a paper last year on an AI education intervention. The end result was a respectable effect size of 0.21 on final exam scores. Here’s the catch: the experimental group received a six-week, twice-a-week, 30-minute after-school intervention using AI. The control group received…nothing. As Michel Pershan writes, they “only proved that their program is better than nothing.”

To give one more simple example of this phenomenon: the SAT is an American university entrance exam. Students often take the test multiple times, and on average do better the second time. I found a few different sets of numbers, but we can ballpark the effect size of taking the exam twice at around 0.15-0.20. That’s a significant effect from literally no intervention at all. Students learn things and get better at stuff all the time. If the study compares the same group of students before and after an intervention, we should expect to see some growth even if the intervention doesn’t do anything at all.

The most rigorous education research compares two serious interventions. If we want to know how well a curriculum works, we compare it to another high-quality curriculum. If we want to know how well tutoring works, we compare it to another plausible use of that same time and resources, maybe small-group instruction or extra teacher planning time. The question isn’t whether something works. Most interventions work! The question is, what is the best use of our time and resources?

Replication

One education research result that just will not die is what’s often called Bloom’s 2-sigma study. Benjamin Bloom wrote a paper summarizing two dissertation studies on one-to-one tutoring. The headline result: an effect size of 2. That’s massive, and has been repeated over and over again as evidence that one-to-one tutoring is the gold standard in education.

There are a bunch of issues with this result, but the one I want to talk about here is simple: when other researchers have tried to replicate it, the results have fallen dramatically. There are tons of studies we could look at. One study looked at a large tutoring program in Nashville. The result? An effect size in the range of 0.04 to 0.09 on literacy test scores. There was no effect on math test scores, or on course grades in either subject.

This doesn’t mean tutoring doesn’t work. Plenty of studies find significant effects. The reality is, there’s a ton of research out there. If you want to say something is research-based, you can probably find a study that you can reference. The most valuable research replicates — it has been repeated in different contexts over and over again, giving us some evidence the result is robust. If a result relies on a single study, we should be suspicious.

Heterogeneity

People love to average things in research. Average results, average effect sizes of a bunch of different studies. Those averages can hide heterogeneity. In a classic meta-analysis by Kluger & Denisi, the researchers found an average effect size of 0.41 for feedback. The catch: huge heterogeneity. One-third of the studies they looked at had a negative effect size: providing feedback decreased performance! The lesson here isn’t about whether feedback is good or bad. It depends!

A few more examples: the growth mindset intervention above showed larger results for lower-achieving students. The effect size of 0.08 for the Khan Academy intervention was limited to the 10% of students who spent the recommended amount of time on the platform. That heterogeneity is important!

Learning is complex. Results depend on all sorts of things. Averaging results together can hide important context.

Lab vs Real World

Many cognitive science phenomena are studied in a lab setting, where researchers can control lots of variables that are hard to control for classroom teachers. Results that are produced in a lab don’t always translate to real classrooms. One example: the interleaving effect is a popular cognitive science result. Interleaving question types (mixing different question types within a study session) typically leads to more durable learning than “blocked” practice where students focus on one question type at a time. That result shows up consistently in lab studies, but has been less consistent in studies conducted in real classrooms. Here is an example of a study in which researchers struggled to replicate the phenomenon in a classroom-based context. This doesn’t mean interleaving is useless. I’m a big fan of interleaving! There is a ton of research on interleaving. Plenty of studies find a positive result in classroom contexts, but those results are often smaller than in a laboratory. These results suggest that interleaving is tricky to get right.

In general, research results are smaller and less consistent in real classrooms. This is common across education research, and we can often learn a lot more from classroom contexts than lab studies.

Months of Learning

If you see someone claim that a teaching practice results in a growth of 4 or 6 or 8 months of learning, you should ignore that number entirely. It’s nonsense. Researchers are typically converting effect sizes into months of learning through some very dubious math. First, no one agrees on what a month of learning is equal to. Sometimes a school year is equal to an effect size of 1, or 0.4, or something else. Second, I hope I’ve convinced you through everything else in this post that effect sizes aren’t that simple. Let’s take the result of my summer school teaching from before: an effect size of 3.4. Did I cause more than three years of additional learning in my three-week summer school class as a student teacher? (No, I did not.) Does taking the SAT a second time result in 4 months of learning? (No.) This is not a serious way to measure learning. We should ignore these claims entirely.

Don’t Average Effect Sizes

Finally, averaging effect sizes. Every time I have been in some sort of PD session and someone brings up John Hattie, I just tune out. Hattie is famous for putting together what’s probably the largest synthesis of education research ever. His website is an interesting reference for meta-analyses of lots of different topics. However, Hattie also distilled his synthesis by publishing an average effect size for over 100 different topics in education research. You can think of this entire post as an argument that averaging effect sizes doesn’t make any sense. Those averages then get taken out of context by people who want to make a particular claim about a particular area of education research. Hattie’s work can be a helpful starting point: if you want to learn about a topic, his website collects a lot of research on lots of different topics. The average effect sizes we should mostly ignore.

In Summary

Teaching is hard. It would be great if there was some magic spell I could cast on my students to make them learn more. Unfortunately, there isn’t. You can maybe tell from my tone in this post that I have a lot of disdain for the ways research is typically used in education. I both believe in the value of evidence-informed teaching, and I mostly see research used in simplistic or opportunistic ways to justify the fad du jour. If we are serious about improving education, that means finding lots and lots of things that make a small difference, and adding up those small differences to make a substantive change in the quality of the education students receive. Finding lots and lots of things that make a small difference means being rigorous in how we interpret evidence.

No research is perfect. The best evidence has been replicated in multiple contexts and demonstrates converging evidence across different types of studies. There are a bunch of studies I’ve mentioned in this post that fail half the criteria or more. Let’s start by ignoring those, and focus on what’s left.

There are two big traps in education research. The first is oversimplification. It’s tempting to pick out some large, flashy effect and claim that this one thing is the answer to all of our problems. It isn’t. Teaching is hard. There’s no one idea coming to save us.

The second trap is dismissing small effects. In reality, all effects are small. Large effects are usually the result of shoddy research. This is tricky! Small effects can be spurious. We need to think rigorously about which small effects are real, which are worth the effort, and what we need to do to execute them effectively. That’s all hard, and that’s the core work of improving teaching and learning.

Very few studies will meet all of the criteria I laid out above. We don’t need to dismiss everything. Serious ideas to improve education come from multiple lines of evidence. That evidence is never going to be perfect. There’s no one silver bullet to improve education, and there’s no one secret thing that guarantees a study is robust and useful for teachers. There are no shortcuts. The goal should be to read broadly, evaluate evidence rigorously, and do the slow, careful work of figuring out what really works.