I’m grateful to Fordham for giving me the opportunity to respond to Curriculum Associates CEO Kelly Sia’s defense of i-Ready.

You can read Sia’s original piece here and the original Fordham publication of my response here.

Curriculum Associates CEO Kelly Sia recently defended her company’s flagship i-Ready platform to Fordham readers. The program, used by millions of students, has come under fire from teachers, parents, and students as part of an ed-tech backlash sweeping the nation.

To set the record straight, Sia stated that Curriculum Associates “welcome honest scrutiny of our research.” She then pointed to a large correlational study out of Georgia State University that evaluated i-Ready use in one large southeastern metro district during the post-pandemic recovery as proof their approach “works.”’

The study behind the claim

She’s right to build her case from it. This appears to be the only independent, peer-reviewed study directly evaluating i-Ready Personalized Instruction outcomes in the program’s 15-year history. So let’s take her up on the offer and scrutinize it. When you read the fine print, a very different picture emerges than the one she’s selling to us.

Start with the number she would like us to focus on. To show her product lifts the achievement of traditionally underserved students, Sia quotes the study directly, and the quote sounds impressive: students who were “Black, male, FRLM-eligible, or with intellectual disabilities experience(d)…up to 14 weeks of instructional growth.”

What the headline leaves out

But look at what she leaves out. Read the actual study and you learn that the 14 weeks applied only to the intellectual disability group, only in reading, drawn from a sample of around 900 students. For every other named group, the effect was 2.2 to 3.5 weeks. Four groups were bundled together, and the biggest result was associated with all of them.

This is why we can’t trust a for-profit ed-tech company to interpret research on their own platform or write their own review. It’s in their interest to cherry-pick the quote that sells the product.

And it’s important to understand what 2.2 to 3.5 weeks actually means, because converting achievement into “months of learning” can be misleading and mathematically messy. A cleaner approach is to look at percentile growth. Translate the effect sizes and you get a 1-to-2-point percentile bump over an entire school year. In plain terms: a student at the 50th percentile moves to the 51st or 52nd by June. The result is statistically significant, yes. But a result can be statistically significant and still be practically minuscule.

There are other facts in the study that Sia omits and that any administrator considering this program deserves to know.

The most important one: the study found that the top third of students in prior achievement derived virtually no statistically significant benefit from the platform. They spent hours on i-Ready’s software, and their math achievement didn’t budge, showing only a tiny, borderline blip in reading.

The top third of students saw virtually no measurable benefit from the platform.

And consider this: the district deliberately prioritized the platform for its lowest-performing schools. So this sample’s “top third” aren’t necessarily all high-achievers. Many are likely closer to the national median. This means in a typical district mandating many hours of screen time, the group experiencing no meaningful effect could in fact be much greater than one third. As a father of two high-achieving children who are mandated to use this program every day, I watch this “zero effect size” play out with my own children. It’s frustrating to know they are spending valuable instructional time in front of a screen receiving no measurable benefit.

Correlation is not causation

Then there’s the finding about teacher involvement. Sia notes that “outcomes improved further when teachers assigned lessons in addition to computer-assigned ones.” That’s true. Researchers found a strong positive correlation when teachers manually assigned lessons. These teachers did so 16 to 19 percent of the time, which translates to nearly one in every five lessons in math and one in six in reading. As a Title I teacher who has used i-Ready for seven years, I can tell you that’s not a trivial amount of work. It requires paying close, sustained attention to the students in front of you.

Sia wants you to read this as proof the program works and works even better with a teacher’s help.

But this is a correlational study, and this explanation is simply her interpretation of the results. Here’s an equally plausible explanation: the “added benefit” has nothing to do with the software. Instead it is showing us the impact on student achievement from the kind of teacher who watches student data closely and is likely a more effective instructor overall. Was the benefit from the teacher or the program? With a correlational study, you can’t tell. It’s messy and open to myriad interpretations.

Even the researcher whose framework Curriculum Associates uses to market the program’s effectiveness, Matthew Kraft of Brown University, says that correlational estimates should be viewed with heavy skepticism. He cautions that “correlational relationships are, on average, substantially larger than causal effects” and that “effect sizes from studies based on correlations or conditional associations do not represent credible causal estimates.” This means, by the company’s own chosen measuring stick, the 1-to-2 percentile bump is likely an overestimate of the true effect. Even taken at face value, it falls below the 0.10 threshold the company’s own efficacy page calls meaningful.

The researchers were candid about this in a manner Sia was not. She says the study shows i-Ready “works.” The authors say: “Our analyses are descriptive versus causal as we cannot fully account for endogenous variation.” They directly admit that they are “unable to collect data on or account for other recovery interventions occurring during the same period,” and are “unable to fully understand or account for selection into treatment.”

In other words, they couldn’t separate i-Ready from everything else happening during the chaos of post-pandemic recovery. We can’t tell if the software was driving the results or was just along for the ride.

None of this proves the program doesn’t work. It proves we don’t actually know.

To make a strong causal claim here, Curriculum Associates needs a randomized controlled trial or a much stronger causal design. That’s the kind of evidence Curriculum Associates has avoided across the product’s fifteen-year history.

If Curriculum Associates truly believes in the efficacy of their product, they should prove it: fund an independent, pre-registered RCT and commit to publishing the results no matter what they show. As one of the largest and best-funded ed-tech companies on the planet, they are well positioned and have the resources to do it.

The real cost is instructional time

None of this is trivial. While the program license might be considered cheap by some at roughly $30 per student per year, the true cost is paid with instructional time. Just consider how running i-Ready with fidelity eats up classroom minutes before a single online lesson begins: there’s the logistical process of distributing devices, managing logins, untangling headphones, and navigating websites, only to reverse this process minutes later. Even with a very conservative estimate of 5 minutes for this entire transition, daily use would add at least 25 minutes of lost instructional time per week. And that’s assuming kids stay logged in, focused on their work, and ignore the temptation of the entire internet that is only one tab away. Add that to the 30 to 49 minutes the company recommends, and you are realistically looking at a minimum of 55 to 74 minutes per week per subject. That is the equivalent of an entire core instructional block lost to an algorithm every week.

All that for a likely inflated 1-to-2 percentile boost for the neediest students and no measurable benefit for a large fraction of the kids who are mandated to complete this daily screen time ritual. Such a mandate should require a heavy burden of proof.

So is the juice worth the squeeze?

Sia says she welcomes honest scrutiny. Good. Fund an independent RCT and publish the results, whatever they are. Until then, the honest answer is the one the fine print already gives us: not enough for the squeeze they’re asking schools to accept.

As the teacher circulates, a boy raises his hand and tells her, “I have to subtract 137 from 205, but I don’t know how.”

At this point in the lesson, nobody has taught him how to do this explicitly. The target strategy hidden in the teacher’s guide is subtracting by adding on, and hasn’t yet been revealed. For now, he is supposed to struggle.

His teacher has been told in professional development that the appropriate response to a student’s plea for help at this point isn’t to provide instruction or to suggest possible strategies. That would amount to “taking over student thinking” and communicating a lack of confidence in a student’s ability to solve the problem on his or her own. Instead, it recommends that she ask a question to orient his thinking, and then to walk away.

So she replies, “Think about a way you could represent the problem. What tools have you used before when subtracting?”

And before he can respond, she’s gone.

This vignette is reproduced closely from a whitepaper on productive struggle written by the authors of the popular K–8 math curriculum i-Ready Classroom Mathematics. i-Ready occupies a unique space within modern educational curriculum in that all four named authors have been at the helm of one of the two major organizations that shape mathematics teaching in the United States: the National Council of Teachers of Mathematics and the National Council of Supervisors of Mathematics. Three of the four authors served on NCTM’s board of directors, and the fourth is the current president-elect of NCSM. The whitepaper, written by two of the four authors, serves as a philosophical justification for their vision of productive struggle and as a practical guide for how to implement it in the classroom.

It’s a script I followed closely earlier in my career. I was trained by i-Ready and have taught the curriculum in grades 3, 4, and 5 for the past seven years in a Title I school. Over that time I’ve walked away from more students than I am comfortable admitting, and it took me far too long to realize the damage I was causing by doing so.

That’s because, as other educators like Linda Diamond of the Evidence Advocacy Center have clearly articulated, the success of this entire approach depends entirely on the learner in front of you. While it may empower well-prepared learners with sufficient prior knowledge, for students lacking foundational skills it regularly leads to isolation and frustration. By mandating this instructional method regardless of a student’s readiness, we don’t teach resilience. The lesson the most vulnerable learners often take away from repeated exposure to this approach is that math is an impossible barrier and that no help is coming.

To be clear, I am not arguing here that opportunities for problem-solving should be abandoned altogether. I agree with researchers Hiebert and Grouws, the originators of the term productive struggle, that challenging tasks can lead to robust learning—in the right circumstances. I also agree with Hiebert when he wrote a response to an article by education reporter Holly Korbey, why schools are rolling back ‘productive struggle (in which I was a featured teacher) and stated that productive struggle tasks “should be appropriately calibrated” and matched “to students’ current levels of understanding.”

In fact, that’s the exact point I want to make.

Paracelsus, a Swiss-born physician, supplied the foundational principle of the field of toxicology: only the dose makes a thing a poison. Yet he left it to the physicians who came after him to supply a second related principle. It’s not just how much—but for whom. The dosage that is medicinal for one patient can be toxic to another.

The Daily Dose

In i-Ready and many other inquiry/discovery math curricula, tasks that are high-cognitive demand and call upon students to struggle aren’t just occasional opportunities to apply hard-earned knowledge, but serve as a daily instructional focus and one of the primary means of acquiring that foundational knowledge in the first place.

These tasks are often prescribed for every student, every grade, and across the overwhelming bulk of core instruction. They progress on a rigid schedule with little verification that students actually have the requisite prior knowledge to successfully engage with the task. In practice, this means that many students receive huge doses of uncalibrated productive struggle.

And teachers and parents would be rightly shocked by the incredibly thin evidence behind doing this way of structuring a curriculum.

What the Authors Actually Tell Teachers to Do

In his response to Korbey’s piece, Hiebert criticized a lesson in which the curriculum asked me to teach multi-digit subtraction to a group of students who weren’t prepared. He (and fellow author Ronald Gallimore) argued that this represented “failures of instructional judgment rather than failures of productive struggle.” I agree, though I want to be clear whose instructional judgement failed. The point I was trying to make is that for students in schools not situated in highly educated and privileged communities, this failure is one of curricular design—one that gives students a single week of subtraction exposure in 3rd grade at the beginning of the year, exposes them to no less than 5 strategies over that time, and then doesn’t build in sufficient practice of any of those strategies to gain mastery. When the same students come to me in 4th grade, their unpreparedness in subtraction is due to instructional choices made by the curriculum authors. I make sure to adapt the lesson and build the necessary background knowledge, but not all teachers are as comfortable straying off district-mandated pacing guides.

I also want to underline, highlight, and draw flashing red arrows to other essential clarifying points Hiebert and Gallimore make (emphasis mine):

“The key to making struggle productive is teacher planning and scaffolding assistance. Hiebert and Grouws did not envision students working in isolation while teachers stand aside. Making struggle productive requires careful task selection, monitoring student thinking, providing hints and representations, asking questions, and helping students connect emerging ideas. The teacher’s role is active, not passive. A student’s frustration is a call for teacher action.”

This doesn’t match the description of productive struggle and advice from i-Ready’s authors in their paper describing how to support students in productive struggle. Here’s what they tell teachers to do at the very moment of difficulty: pose a question and “walk away” (p. 15), withholding the hint, representation, or strategy on the grounds that to “explain how to subtract with regrouping or suggest another subtraction strategy” would be “you doing the thinking” (p. 15).

Nor do they recommend that teachers should lead students through questioning. They admonish this as funneling and instead tell teachers to ask “a question that acknowledges that thinking and reorients it” (p. 8), after which the teacher again is told to step away. In other words, they abandon the student at precisely the point they are struggling the most.

Hiebert and Gallimore also point out above another key factor for successful productive struggle: careful task selection. Note that this is challenging to provide in a curriculum with a set pacing guide and expectations by districts of curricular fidelity. All students, regardless of preparedness, must tackle the same difficult task in a set order regardless of readiness. Teachers aren’t being asked to carefully select tasks; the tasks are already selected for them.

Hiebert and Gallimore are also right to assert in their response to Korbey’s article that:

“The issue, then, is not whether students should struggle, but how that struggle is structured and supported. Mathematics instruction requires balance….Students need explicit instruction and deliberate practice. They also need opportunities to struggle productively, to grapple with important ideas, make conjectures, test strategies, and learn from mistakes.”

That is a fair assertion. But contrast that to what i-Ready’s lead author, himself a former NCTM board member and a contributor to Principles to Actions, says about explicit instruction in a webinar for i-Ready’s parent company:

“The trouble with this (I do, we do, you do) when applied to learning mathematics is that it does not take into account cognition. Reasoning and sense-making are absent.”

He goes on to create a strawman of high-quality explicit instruction by conflating it with a passive teacher-led routine. To be clear, he places a red X across “I do, we do, you do” and characterizes the routine as a behaviorist relic that doesn’t support student cognition. Rather than position productive struggle and explicit instruction as complementary practice, he presents them as competing methods with only one supporting student thinking. This is a false choice, because well-designed explicit instruction provides students with the very knowledge that make inquiry and reasoning possible.

(Source: What Happens When Teachers Talk Less and Students Talk More, Dr. Mark Ellis, lead author i-Ready Classroom Mathematics)

So when Korbey is talking about schools abandoning productive struggle, what I think needs to be clarified is that it’s not that teachers and leaders are calling for the abandonment of all challenging instruction or opportunities for problem-solving. Instead it’s a call to abandon the practice of teaching all content through high-demand cognitively challenging tasks that rely on productive struggle that many or most students aren’t often ready for, and to reassert the importance of explicit instruction and time for deliberate practice.

And while the focus of this piece is i-Ready, due to the deep relationship between the curriculum and NCTM and NCSM, it shouldn’t be evaluated as just another textbook. It is, in many ways, a direct commercial translation of a particular productive-struggle philosophy and what good math teaching looks like. Millions of children move through its routines daily. So if that philosophy collapses in real classrooms, it raises serious questions about the structure beneath it—not only i-Ready, but the discovery-first foundation shared by many modern math curricula.

Their Own Evidence Argues Against Them

With that said, let’s turn to the evidence cited in i-Ready’s whitepaper on productive struggle, because even these don’t serve as a justification for their daily dose, but rather a warning label against it.

Take the empirical claim that “Students who regularly interact with high-demand tasks demonstrate greater learning gains compared to those who do not (Stein 2009).” This reference is a case study for professional development drawn from an earlier study (also cited) of the QUASAR project in the 1990s. QUASAR was a $10 million project to show that reform mathematics could successfully improve outcomes in low-achieving urban middle schools. In the correlational study, carried out in conjunction with the University of Pittsburgh, teachers were provided with extraordinary levels of support in the form of coaching and professional development in how to implement cognitively demanding tasks. Even with support that most schools could only dream of, the failure rate of productive struggle “doing mathematics” tasks far outweighed their success. In fact, of the 58 tasks initially set up at the highest cognitive-demand level, 62% failed to remain at the “doing mathematics” level:

• 8 declined into “using procedures without connection to meaning” (became rote)

• 11 declined into “unsystematic exploration” (students floundering without direction)

• 10 declined into “no mathematical focus” (students was no longer focused on the intended mathematics)

• 7 were mixed in a way that didn’t classify cleanly

In other words, even with significant support, professional development, and coaching, teachers were only able to maintain “doing mathematics” with middle schoolers approximately 38% of the time. Let that sink in. Middle school students are not developmentally equivalent to elementary students. The former’s executive functions, while still maturing, are far enough along to make sustained, self-directed struggle more approachable with heavy support. i-Ready takes that model and pushes it down to kindergarten through fifth grade, onto children whose working memory, cognitive flexibility, inhibition, and meta-cognition are all still under construction.

Even the researchers who carried out the cited analysis (Stein, M. K., Grover, B., & Henningsen, M., 1996) cautioned that task appropriateness and students’ prior knowledge were major factors in success or failure with high-cognitively demanding tasks (agreeing with Hiebert and Gallimore). Further, this analysis offers no direct support for daily, every-lesson use of high-demand tasks where students are meant to discover mathematics on their own.

An important insight from their work is that carrying out such tasks effectively is highly challenging for teachers to pull off—one that requires careful planning and preparation. In other words, not something your average elementary teacher tasked with teaching multiple subjects daily can easily or reasonably do well.

This is echoed in the other citation used to support the whitepaper’s claims used to justify a daily dose of struggle with high-demand tasks: Kapur’s 2010 work on productive failure, used to justify the claim that “when teachers embrace productive struggle, it leads to long-term benefits, with students able to apply their learning to new problem situations (Kapur 2010).”

Kapur, whose productive failure is distinct from “productive struggle” as defined by i-Ready’s authors, nevertheless warns against daily engagement with cognitively demanding discovery tasks. When asked about the frequency that teachers should engage students in these tasks in an interview for an Edutopia article, he couldn’t have been more clear:

“That doesn’t mean students should be doing it all the time, right? In my work with schools and universities, I say: If you’re going to adopt productive failure in math or science or language, don’t do it every week. Instead, think about where productive failure is really helpful in terms of understanding and transfer. So in a semester, there are probably three to five big ideas you want students to really deeply understand. For those ideas, design productive failure activities; use it surgically where there are important ideas to learn.”

To be clear, both Kapur’s Productive Failure and i-Ready’s productive struggle Try-Discuss-Connect routine end in teacher-facilitated consolidation. But Kapur’s suggestion about frequency is crucial.

The Research Base

Then we must consider i-Ready’s research base. In what would be shocking for any teacher or parent to learn, i-Ready’s entire research base contains just one randomized controlled trial focused on mathematics. That would be alarming on its own, but consider this: the study in question (Doabler et al. 2019) was studying the effects of a high-quality explicit instruction intervention for developing number sense and raising achievement of kindergarten English Learners. Apparently 20th century methods haven’t gone out of style, because the results showed explicit instruction was highly effective.

Their Own Analogies

Finally, their own analogies unwittingly undermine their arguments. The whitepaper attempts to justify daily productive struggle by pointing to areas where students thrive through challenge, such as music, sports, and video games. They write:

“In many other contexts, such as playing video games, sports, or learning an instrument, students already have growth mindsets. They understand that making mistakes is natural, and they learn from those mistakes. They realize that sustaining hard work yields improvement, and they see the fruitful results. It’s time to bring the same productive struggle that students apply to other situations to the mathematics classroom!”

But these are all areas in which explicit instruction and deliberate practice play foundational roles. A student learning piano doesn’t discover hand position through productive struggle. Teachers show them, model it, and correct it in real time. Then students must engage in sustained and frequent deliberate practice. Sports work the same way. A soccer coach doesn’t throw kids into a game and hope they’ll figure things out. They drill, directly teach technique, and coach. Even video games are built using million-dollar budgets and rely on constant scaffolding and immediate feedback. Every action produces instant information about whether the player succeeded or failed. None of these domains operate the way productive struggle is prescribed by the authors. That is, none of them mandate struggle comes first, without foundational instruction. None of them remove the teacher’s role or prescribe practices that allow little time for practice. The authors cite their own counterexamples without recognizing it. The very domains they invoke to justify their approach actually demonstrate why their approach is backwards.

The One-Way Pipeline of Implementation

I want to be transparent about my perspective. I write this as four things at once: a teacher who has run this curriculum daily in a Title I school, a father whose children have been taught by it since kindergarten, an educator who has spent countless hours auditing the evidence behind it, and someone who has seen district-level teacher feedback gathered by our teachers union documenting overwhelming concern about the curriculum in practice.

That last piece is important, as it’s imperative that districts gather data about what is and isn’t working from teachers directly. This is because curriculum implementation functions largely as a unidirectional pipeline: NCTM strongly shapes the teaching frameworks, curriculum publishers design products that align with them, programs recieve “HQIM” labels based on more on alignment than demonstrated efficacy, states and districts treat those labels as signals of overall quality, and then teachers are often told to teach these programs with fidelity.

When this fails to produce the desired outcomes, instead of listening to teachers, they are often the ones left to shoulder the responsibility for that failure and are blamed for not implementing the approach with fidelity.

To this end, I wrote a letter to my curriculum director expressing my concerns about our inquiry-first approach to teaching math. She forwarded my writing to Curriculum Associates to get their response. When the company responded, they defended their approach by pointing to their inspirations: John Hattie and NCTM’s Principles to Actions. The latter amounts to a closed feedback loop: two of the lead authors were in leadership at the time Principles to Actions was written, and are explicitly listed in the document as providing valuable feedback on it. The former is yet another example of citing experts who directly contradict their approach. Hattie has commented that:

“…if you don’t teach the surface and the content, you’ve got nothing to inquire about. The reason it [inquiry] comes out very low on the chart is because most teachers introduce it far too early.”

Equity

In the end, when practitioner feedback is ignored and the system continues to prescribe an instructional approach that relies on strong prior knowledge, the truth is that not all kids fall behind equally.

The difference almost always comes down to whether there is anyone there to help when much of the actual teaching and practice gets outsourced to the kitchen table. When schools don’t deliver the explicit instruction and practice students need to be successful with a daily dose of productive struggle tasks, parents must pick up the slack by providing the missing instruction themselves or paying for private tutoring. When the latter happens, success gets privatized. This is called shadow education, a parallel system that runs alongside public school and privileges some students over others. Kids whose parents work multiple jobs, or didn’t have access to strong math instruction themselves are the ones who fall further behind.

What Needs to Happen

We need to stop adopting and mandating curricula that treat struggle-first as the default mode of instruction, and we need to stop demonizing explicit instruction, sustained deliberate practice, and committing knowledge to memory—not as approaches opposed to inquiry, but as the foundation that makes inquiry possible.

As literacy researcher Pamela Snow argues, how we educate children is a public health issue. Math and reading both predict future outcomes in life to a remarkable degree. She correctly points out that in education we engage in “uncontrolled experimentation with children” by adopting trends before there is solid evidence to back them up. This wouldn’t be acceptable in medicine or any other areas of field of public service, and it shouldn’t be acceptable in education either. The shocking lack of evidence behind the way we are currently teaching math is deeply unfair to teachers and parents, and it’s devastating to the children who pay the price.

This isn’t a call to get rid of challenging tasks or opportunities for students to productively struggle in its original definition. It’s a call to sequence and dose it appropriately based on the students in front of us. Because right now we’re poisoning kids with a daily dose of struggle that the evidence doesn’t support.