I’ve rarely seen schools make explicit choices among different ways of delivering Tier 2 AII to below average students who may be at risk. Here are the choices I try to make explicit in my consultation and MTSS design activities. With advantages and disadvantages.

1. GE Teacher DOES MORE Within Their Class

   1. Advantage(s): No additional resources required. Easiest to do.

   2. Disadvantage(s): More work. Assumes teacher knowledge and AAI materials and high quality classroom behavior support.

2. GE TEACHERS Do Flexible Skill Grouping (FSG) Within the Reading/Language Arts Block Across Classes within a Grade Aligned to Different Levels of a Supplemental Core Program or AII Program

   1. Advantage(s): No additional resources required. Relatively easy to do. Students may transition to different classrooms aligned to their instructional needs.

   2. Disadvantage(s): Different. Assumes teacher knowledge and AII materials and high-quality classroom behavior support especially for transitioning between classrooms. Assumes placement tests for differentiated intervention. Must differentiate between FSG and histories of tracking. Requires scientifically sound progress monitoring data (e.g., Strategic Monitoring) at least monthly for readjusting groupings.

3. GE TEACHERS Do FSG Across Classes Across Grades

   1. Advantage(s): No additional resources required. Relatively easy to do. Students may transition to different classrooms of different grades aligned to their instructional needs. Increased ability to reach more students when there is serious academic diversity of need.

   2. Disadvantage(s): VERY Different. The old “country school model.” Assumes teacher knowledge and AAI materials and high-quality classroom behavior support especially for transitioning between classrooms. Assumes placement tests for differentiated intervention. Must differentiate between FSG and histories of tracking. Best with scientifically sound progress monitoring data (e.g., Strategic Monitoring) at least monthly forreadjusting groupings.

4. School Provides Before or After School Intervention

   1. Advantage(s): No additional resources required of teachers during the reading block. Has been a successful model with trained grandparents, for example, in the Juniper Gardens Project.

   2. Disadvantage(s): Staffing personnel before and after school AII. Assumes the interventionist has knowledge and AAI materials with placement tests and high quality classroom behavior support. Best with scientifically sound progress monitoring data (e.g., Strategic Monitoring) at least monthly.

5. School Provides Computerized Interventions During the Core Instructional Block or At Other Times of the School Day Within the Classroom

   1. Advantage(s): No additional resources—in theory- required of teachers during the reading block.

   2. Disadvantage(s): All computerized interventions require high student motivation, high levels of engagement, and self management. Can be costly and high quality implementation requires adult attention for monitoring student engagement consistent with positive behavior support. Not all computerized interventions are AII and may not be cohesive with core instruction.

6. School Creates and Staffs Universal Intervention Periods. Schools reduce allocated time to some blocks or eliminate curriculum content to create an intervention period (e.g., 30 min) where all or some students receive intervention and all teachers/educators are interventionists.

   1. Advantage(s): An easy sell. Not usually seen as “more work.”

   2. Disadvantage(s): Assumes the interventionist has knowledge of the intervention area (e.g., reading) and access to an AII program. Interventionist time may be “diluted” by providing “intervention” to students who are achieving satisfactorily who may not need intervention. Often, practical time for intervention within the intervention period can be less than necessary due to not enough time to implement the AII with integrity and lost time due to issues in transition.

7. Central Decision Makers BUILD Coordinated Remedial Resources. Tiered intervention is provided not by GE core teachers but by Title I teachers, Reading Specialists, ML/EL Teachers, Highly Trained Paraprofessionals.

   1. Advantage(s): Seen by GE teachers as supporting their role, increasing their students’ success, making the difficult job of teaching a little easier. Allows for cohesive implementation if good decisions are made about an AII program and delivery of high quality AII if good choices are made.

   2. Disadvantage(s): Requires strong leadership and a commitment to thinking differently, risking territorial battles and changes in roles. Assumes the interventionist has knowledge of the intervention area (e.g., reading) and access to an AII program.

Comments and criticism are welcome.

The early years:

· When the idea that not every problem was a special education problem,

· Not every student who was in some way different was a person with a disability,

· When new thinking, “problem-solving,” was required, and

· Where a new data system driven by student performance and progress ruled.

I’ve lived through and contributed to the expansion of a Problem-Solving model beyond an innovative rural special education co-op with a passionate, brilliant leader willing to challenge current belief systems. After observing implementation in the Pine County Cooperative, we began implementation in a large urban district with a high proportion of low-income students, developing local norms for special education decision making with an emphasis on Problem Identification (i.e., a “first gate screening) and quality IEP goals and frequent progress monitoring. These early efforts contributed to a huge collaborative effort that formed a powerful and long-lasting statewide model that opened the doors to nationwide innovations. The unique Iowa collaboration of local leaders partnering with visionary state leadership and higher education faculty led to a non-categorical model of special education and demonstrated to our country that school improvements were possible. I’ve seen first-hand the change in language from a Problem-Solving model, to Response to Intervention (RTI) to Multi-Tiered Systems of Supports (MTSS). I’ve seen the impact of radical changes in the IDEA of 2004 that implicitly preferred an identification model based on RTI versus the prevailing and disproven practices of using presumed measures of cognitive ability to somehow judge whether a student’s achievement was consistent with that score. I’ve consulted in 45 states, from our nation’s 3^rd largest school district to very, very small schools and communities in the farthest regions. Perhaps equally important, I’ve been fortunate to be informed by the best of the best, from researchers to school leaders.

Here are my own set of misunderstanding.

1. The unit of change, the instrument of change, is the classroom general education (GE) teacher, often disguised as team members.

I witness MTSS implementation efforts where GE teachers:

a. collect data MTSS data, usually three times per year;

b. are expected to interpret screening data to determine WHO receives more appropriate;

c. use data to design and delivery appropriately intensive intervention (AII) to enable more individualized instruction;

d. deliver not only core (i.e., Tier 1) instruction, but also to deliver Tier 2 AII;

e. collect and use frequent progress monitoring data to evaluate progress and, if necessary, identify changes in the current intervention and deliver modified instruction a formative approach.

Teachers, whether at the individual level or in building, grade level or problem-solving teams are tasked with knowing what to change in core instruction as well as what are AII.

Given the our collective knowledge gaps regarding the science of reading, instructional design, positive behavior support, and our teachers’ documented “assessment illiteracy,” that seems this focus seems like one designed to fail. MTSS appears to be built on the backs of teachers and if all this doesn’t seem to teachers to “be more work,” I’d be surprised. I believe it to ask too much.

In my observation, the solution is not to have the unit of change be teachers, singly or in teams, but to be those persons who manage the contingencies regarding the tools in the toolbox teachers use. In short, the curricular programs. True, sometimes knowledgeable teachers have the capacity to influence these decisions about which programs to use, and ideally, teachers should have a voice.

But when I look at the history of successful MTSS interventions, and specifically, when there is meaningful growth in reading achievement, it’s always leadership who makes informed choices about which appropriately intensive core program is used and who provides the necessary supports all of us need to do new, important things well. Leadership that:

· Selects core reading programs that are research-based, with emphasis on direct, explicit instruction, programs that are carefully sequenced and actually field tested with necessary revisions. Leadership that ensures AII programs in Tiers 2 and 3.

· Develops and delivers focused professional learning and ongoing coaching support that enables teachers and teams to get good at delivering AII with engagement and passion.

· Looks at extent achievement data and about their students (i.e., low-income status and English learning needs) to determine the appropriate intensity of their core program. In more high-risk communities, core instruction may need to look like what is delivered in Tier 2 or even Tier 3 elsewhere. In other words, knowing how to increase the dosage of instruction aligned to the needs of students. That may mean supplementing the core or even supplanting the core.

· Supports cohesion of instruction across tiers. To me, Tier 2 should be a program that explicitly addresses the weakness(es) of core instruction. Tier 3 should be an intervention that is explicitly designed as a catch-up program to reduce the gap.

· Embeds special education explicitly in MTSS implementation with cohesion in AII and data systems, reducing separateness and incorporating wherever possible (e.g., Tier 3) special education and related services personnel in intervention delivery.

In my experience, putting some much of the onus for MTSS implementation on teachers (and teams) to implement high-quality MTSS (e.g., invent and deliver more individualized core and tiered AII) is near impossible. Obviously, we do the best we can but we must be very careful to avoid MTSS implementation built on the backs of teachers. I’m firmly convinced that our system of supports be ones of supports for teachers as well as students.

In future Substack posts, I hope to address misunderstandings of:

· Assessment practices. Too much data, not using data as intended, and not enough attention to progress monitoring. Mission creep.

· Tiers, Risk status screening decision making. Confusing a heuristic with a road map, and why too often risk status confuses who receives AII.

· Use of screening data tied to risk status that may not be aligned to the needs of students and available resources. Trying to provide AII in ways that don’t add load to general education teachers whenever possible.

· The Where’s Waldo of special education in MTSS implementation, failing to see the synergy of improved special education services and MTSS success.

As always, I welcome comments and criticisms.

Decades ago, it was clear that basic reading skills measured by R-CBM/ORF ceilinged at Grades 6-7. The only way to make was difficult probes was to shift from narrative to informational text. Doing that tapped into more than basic reading skills, like vocabulary and content knowledge. Plus, even if creating more difficult graded probes beyond Grades 7-8 was possible, developing norms, especially local norms, would be very difficult. Given the ceiling of ORF at Grades 6-7, Benchmarking for screening norms or for assessing growth and development (i.e., universal progress monitoring) doesn’t make any sense.

Secondary MTSS—and special education eligibility, annual and 3-year re-evaluations will not make sense and not be implemented well without a clear framework about WHO we serve that is different than a Normative approach. Consistent with IDEA-2004, I recommend a Standards-Based approach. My inserted comments are in BOLD into Section 300.309 of the IDEA.

• Section 300.309 (Determining the existence of a specific learning disability) has been revised, as follows:

(1) Paragraph (a) of § 300.309 has been changed (A) to clarify that the group described in 300.306 may determine that a child has a specific learning disability if the child does not achieve adequately for the child’s age (a normative approach) or to meet State-approved grade-level standards (legitimizing a standards-based approach) in one or more of eight areas (e.g., oral expression, basic reading skill, etc.),

To determine WHO may be appropriate candidates for MTSS tiered intervention and perhaps when that student may need intervention beyond the reach of general education (i.e., need for specialized instruction), the Performance Discrepancy would be determined by comparison to a STANDARD (e.g., end of Grade 6 oral reading score that is below average).

Students screened who perform below the Standard would be good candidates for TREATMENT (direct, explicit instruction using appropriately intensive intervention). Students who are struggling in content area classes but not below the Standard would not be abandoned. Instead, should would provide SUPPORT. We know how to provide this support. Help content area teachers become better teacher, an established problem given the lack of quality preservice training on explicit, research-supported teaching strategies. See the amazing work from the University of Kansas on Content Literacy Curriculum (CLC), Strategic Instructional Model (SIM), Anita Archer’s work, or Substack articles by Hendrick or Groshell. Supports like explicit vocabulary instruction, effective note taking, teaching from a “big ideas” perspective.

A Standards-based approach was actually put into statewide practice. Alan Coulter and I had a contract with the Tennessee Department of Education in 2015 to help them develop and roll out RTI2 for secondary. An important outcome was that we finally came up with a label for a Standards-Based Performance Discrepancy, one that actually had its roots in 1982!. Together with Teresa Nichols, then the Tennessee state director of Special Education, we labeled the Performance Discrepancy as one from Minimum Basic Skills Proficiency (MBSP). That discrepancy from an end-of-year Grade 6 ORF score is what defined a problem that required more intensive intervention. Performance in the Standard assessment material (e.g., Grade 6 ) is also what identified the goal (i.e., reading at least as well as the MBSP criterion) and the assessment material in which the response to the intervention (RTI) would be determined. For students in special education with IEP goals, MBSP became the Standards-Based goal and the way progress would be evaluated.

Like many of today’s implicit, if “below the surface” MTSS concepts, this approach to defining a problem that warranted more intensive intervention or special education need was derived from the model implementation principles of Deno’s Data-Based Problem Modification (DBPM; 1977). DBPM into practice system wide by Gary Germann in Pine County, a rural cooperative that served as a primary site for Curriculum-Based Measurement (CBM) development and the prototype for a Problem Solving model, in 1982. Imagine, 52 years ago a Performance Discrepancy from a STANDARD, not a NORM defined the need for more intensive intervention for secondary students.

What, then, defines the Standard of MBSP today? Like many concepts derived from a Problem-Solving model—that evolved into RTI—and is deeply embedded in MTSS, that Standard should be defined locally. What is MBSP in a particular situation (i.e., that particular community)? I always recommend that school districts to define their reading (or other basic skill) Standard that serves as the MINIMUM for success in secondary classes.

Based on early federal standards for “readability” of federal documents decades ago, which at the time was 6th grade, to me, that’s a good place to begin discussion. An even better way is to collect data on secondary students currently with a reading IEP goal. Given that regardless of how these students were previously determined eligible, how these students perform now provides a valuable perspective of what the local community implicitly considers to be a significant problem that warrants more intensive intervention.

Once schools establish a Standard, below which requires more appropriately intensive intervention, we have identified the WHO. Only then can schools more clearly identify the WHAT. Schools have now identified which students need TREATMENT (more intensive intervention). In other words, a direct and appropriately powerful intervention. And equally, we have identified that students who, while above the MBSP Standard who need SUPPORT for success in content area classes. Now, we have a clear rationale about what we need to do to provide that support, an areas that to me has not be a clear—and compelling—priority. With the exception of states like Pennsylvania that has provided extensive training in this area.

Consider an End of Year Standard like reading as well as an end of year student on a simple oral reading test, R-CBM/ORF. Start looking through the lens of Who and What? The lens of WHO we TREAT—and HOW. WHO we SUPPORT—and HOW. If our Treatments are weak—or maybe even no existent, our needs are more clear. If our Supports are weak—or maybe even no existent, our needs are more clear and MTSS will improve.

Deno, S. L., & Mirkin, P. K. (1977). Data-based program modification: A manual. Reston, VA: Council for Exceptional Children.

Deshler, D. D., Ellis, E., & Lenz, B. K. (1996). Teaching adolescents with learning disabilities (2nd ed.). Denver, CO: Love.

Lenz, B. K., Deshler, D. D., & Kissam, B. R. (2003). Teaching content to all: Evidence-based inclusive practices in middle and secondary schools. Boston, MA: Allyn & Bacon.

Lenz, B. K., Ehren, B. J., & Deshler, D. D. (2005). The Content Literacy Continuum: A school-reform framework for improving adolescent literacy for all students. TEACHING Exceptional Children, 37(6), 60.

Marchand-Martella, N. E., & Martella, R. C. (2010). Read to Achieve: Comprehending Content-Area Text and Comprehending Narrative Text. In. Chicago, IL: McGraw Hill Education.

Marchand-Martella, N. E., Martella, R. C., Modderman, S. L., Petersen, H., & Pan, S. (2013). Key areas of effective adolescent literacy programs. Education and Treatment of Children, 36, 161–184.

Shinn, M. R. (2008). RTI at the secondary level. In S. L. Fernley, S.D., Norlin, J. (Ed.), What do I do when...The answer book on RTI. Horsham, PA: LRP Publications.

Shinn, M. R., Windram, H. S., & Bollman, K. A. (2016). Implementing RtI in Secondary Schools. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of Response to Intervention (pp. 563–586). New York: Springer.

Tindal, G. A., Shinn, M. R., Walz, L., & Germann, G. (1987). Mainstream consultation in secondary settings: An evaluation of the Pine County Model. The Journal of Special Education, 21, 94–106.

For each of these Practice Statements, I have identified Ineffective Practices, that must be discarded not only because of their lack of benefit, but in many cases (e.g., the Ability Achievement discrepancy model for identifying students with learning disabilities), to prevent harm as well as waste of valuable resources.

I recognize that academic outcomes for students with disabilities are uniquely influenced by sets of procedures either established by institutional understandings of best practice and/or law (e.g., MTSS Benchmarking, IDEA-2004). Therefore, my Practice Statements for Special Education will be nuanced to reflect those influences, however redundant some of the Statements may be for those derived for typically developing students.

Special Education Screening Practices for Students with Reading Difficulties

Assessment is a process of collecting data to make a variety of decisions (Ysseldyke, 2023). In Special Education in particular, we must recognize that assessment is not just testing students to identify a disability and determine eligibility for special education.

No longer are there as clear boundaries regarding where assessment for special education purposes begins. Now it is recognized that there are evidence-based ways where early identification through academic and behavioral screening assessment and appropriately intensive, evidence-based intervention promotes positive development and circumvents the need for special education.

Among the most common general educational assessment decisions is Screening, a process that determines if a student’s performance is sufficiently discrepant from an expected level of performance to warrant intervention or further assessment; in MTSS language, this is a Performance Discrepancy, one that requires more intensive intervention.

Reading Screening Evidence-Based Practice Statements

1. Special education screening should be infused into the standard assessment practices (i.e., Benchmarking) of Multi-Tiered Systems of Supports (MTSS).

Ineffective Practices. Special education screening, including screening for dyslexia, should not be seen as a separate, stand-alone process, but instead incorporate the valid performance discrepancy data collected as part of MTSS. There is no evidence that multiple reading tests, composite scores, or rating scales are required to identify or support early intervention. These Ineffective Practices consume valuable time and funds that would be better served for more powerful intervention.

2. Systematic Universal Screening for reading difficulties, including dyslexia, should occur once per year, if efforts are not made to increase the use of these data for Universal Progress Monitoring. Regardless of Universal Screening frequency, Grades K-5 should use a Multiple Gating Approach.

Ineffective Practices. Reading screening has been promoted erroneously as universal screening three-times per year process as part of general education MTSS Benchmarking. This Benchmarking process often extends beyond Grade 5 to Grades 9 and above. This interpretation has resulted in significant over-testing of individual students who are not at risk.

Furthermore, this interpretation has distracted attention from Benchmarking’s original intent, to Universal Progress Monitoring to ensure successful reading growth and development. Multiple screening tests, within and across years are Ineffective Practices.

Routinely collecting specific reading skill data more useful for intervention planning on all students should be avoided and used only as a second gate when an intervention program placement test is unavailable or when students are not progressing.

3. Screening for reading difficulties should be limited to a single evidence-based measure appropriate in Multiple Gating with the K first gate being Letter Names in Fall K and Letter Sounds thereafter, adding an ORF measure consisting of highly decodable passages as an end-of year screener. For Grades 1-5 (e.g., any published, valid Reading Curriculum-Based Measure (R-CBM), Oral Reading Fluency (ORF) should serve as the first gate.

Ineffective Practices. Published commercial tests for use in MTSS and special education decision making now contain a number of reading subtests, including compositive scores, and rating scales for Universal Screening. There is no evidence, however, that more than a single screening measure is necessary to identify students at risk for reading difficulties. Many of these additional reading tests may be useful for instructional planning, but are not necessary for Universal Screening nor Universal Progress Monitoring.

4. Screening cut scores to identify reading difficulties could be carefully considered to align to the reading intervention needs of the general education population.

Ineffective Practices. Cut scores that accompany reading screening tests may not align to the existing reading needs of a school. In a school with an already known high population of at risk readers, a screener will also identify a high proportion of students as at risk, confirming the obvious and making intervention decisions difficult. Reading risk proportions (e.g., 25%) are best used for program evaluation with changes in Risk Status % serving as an indicator of group reading programs’ effects across tiers. Screening cut scores must be adjusted to align to the school’s intervention resources.

REPRESENTATIVE ARTICLES

Fletcher, J. M., Francis, D. J., Foorman, B. R., & Schatschneider, C. (2021). Early Detection of Dyslexia Risk: Development of Brief, Teacher-Administered Screens. Learning Disability Quarterly, 44(3), 145–157.

Shinn, M. R., & Walker, H. M. (2026). Increasing Universal Reading Screening Using Multiple Gating to Increase Efficiency. Unpublished paper.

VanDerHeyden, A. M., & Burns, M. K. (2018). Improving Decision Making in School Psychology: Making a Difference in the Lives of Students, Not Just a Prediction About Their Lives. School Psychology Review, 47(4), 385–395. doi:10.17105/SPR-2018-0042.V47-4

VanDerHeyden, A. M., Burns, M. K., & Bonifay, W. (2018). Is More Screening Better? The Relationship Between Frequent Screening, Accurate Decisions, and Reading Proficiency. School Psychology Review, 47(1), 62–82. doi:10.17105/SPR-2017-0017.V47-1

Ysseldyke, J. E., Chaparro, E.A., VanderHeyden, A.M. (2023). Assessment in Special and Inclusive Education (14th ed.). pro-ed.

Special Education Eligibility Practices for Students with Reading Difficulties

GUIDE:

Assessment for special education eligibility for students with reading and mathematics difficulties most frequently is based on Specific Learning Disabilities, (SLD), which includes dyslexia. Significant changes in special education law in 2004 enabled schools to use a process of Response to Intervention (RTI), a more promising approach than practices based on the Ability-Achievement model. This later approach has been criticized repeatedly for its lack of evidence of reliability and validity from psychometric and social justice perspectives.

Evidence-Based Practice Statements

1. Special Education Eligibility Evidence-Based Practice should prioritize assessment of (a) a Significant Reading Performance Discrepancy, that (b) fails to respond to appropriately intensive general education intervention, a Progress Discrepancy, using basic skill achievement tests validated for these purposes, plus evidence of the Need for special education.

Ineffective Practices. A well-documented Response to Intervention process, with an ecological approach to answering questions about IDEA inclusionary, determinant, and exclusionary factors is currently the “best evidence” approach.

Eligibility practices based on the Cognitive Ability-Achievement Discrepancy model are Ineffective Practices and must continue to be abandoned due to more than 35 years of accumulated psychometric and of disconfirmatory validity research evidence. Furthermore, other approaches centered around students’ causal cognitive ability deficits such as Patterns of Strengths and Weaknesses (PSW) are Ineffective Practice and should not be used due to evidence of lack of reliability within and among models and validity of any particular PSW model.

2. Need for Specially Designed Instruction should be documented clearly with assessment evidence that the IEP provides an appropriately intensive intervention to reduce the achievement gap (i.e., the Performance Discrepancy) that exceeds the resources provided by the student’s general education program.

Ineffective Practices. Determining special education eligibility without explicit data regarding Special Education Need is inconsistent with IDEA-2004 and is Ineffective Practice.

3. Required observations for SLD identification should include systematic classroom observations of the student’s current general education reading program to determine that an Appropriate, research-based intervention has been delivered.

Ineffective Practices. Observing only student behavior using rating scales or observational tools (e.g., on task) without systematic attention to the appropriateness of general education reading curricular and instructional features is Ineffective Practice. Too often, required observations serve as procedural requirements and are not useful in making decisions about appropriate instruction. The latter is too frequently assessed only using consensus teams opinions using a procedural checkbox creating pressures against dissenting opinions. This is Ineffective Practice.

REPRESENTATIVE ARTICLES

Dombrowski, S. C., McGil, R. J., Farmer, R. L., & Kranzler, J. H. (2021). Beyond the rhetoric of evidence-based assessment: A framework for critical thinking in clinical practice. School Psychology Review. https://doi.org/10.1080/2372966X.2021.1960126

Fletcher, J. M., & Miciak, J. (2016). Comprehensive cognitive assessments are not necessary for the identification and treatment of learning disabilities. Archives of Clinical Neuropsychology, Advance Access Published December 8, 2016.

Fletcher, J. M., & Miciak, J. (2019). The identification of specific learning disabilities: A summary of research on best practices. In (pp. 1-32). Austin, TX: The University of Texas System/Texas Education Agency.

McGill, R. J., Dombrowski, S. C., & Canivez, G. L. (2018). Cognitive profile analysis in school psychology: HIstory, issues, and continued concerns. Journal of School Psychology, 71, 108-121.

Miciak, J., Fletcher, J. M., Stuebing, K. K., Vaughn, S., & Tolar, T. D. (2014). Patterns of cognitive strengths and weaknesses: Identification rates, agreement, and validity for learning disabilities identification. School Psychology Quarterly, 29, 21-37.

President’s Commission on Special Education Excellence. (2002). A new ERA: Revitalizing special education for all children and their families. In. Washington DC: US Department of Education.

Stanovich, K. D. (2005). The future of a mistake: Will discrepancy measurement continue to make the learning disabilities field a pseudoscience. Learning Disabilities Quarterly, 28, 103-106.

Stuebing, K., Fletcher, J. LeDoux, J., Lyon, G.R., Shaywitz, S., Shaywitz, B. (2002). Validity of IQ discrepancy classifications of reading disabilities: A meta-analysis. American Educational Research Journal, 39, 469-518.

Vaughn, S., & Fletcher, J. M. (2020-2021). It’s Time to Act on a 20-Year-Old Consensus. American Educator.

Assessment for Special Education Reading IEPs and Progress Monitoring

GUIDE:

Assessment for special education eligibility for students with reading difficulties, (i.e., SLD) in IDEA-2004 requires information on Present Level of Academic and Functional Performance (PLAFP) to enable IEP goals to be written and progress toward those goals to be reported to parents at least as often as non-disabled students. Without observable and measurable goals and frequent progress monitoring, interventions that are failing to reduce a student’s Performance Discrepancy may continue longer than necessary. Improving the quality of IEP goals and frequent progress monitoring is one of the most solvable problems in special education.

Evidence-Based Practice Statements for IEPs and Progress Monitoring

1. A student’s annual IEP goal for a reading Performance Discrepancy should be based on reducing the achievement gap on a general outcome measure (GOM) validated for progress monitoring of students with disabilities (e., published members of the Curriculum-Based Measurement (CBM).

Ineffective Practices: Too many IEPs contain many goal statements based on specific skill deficits with a criterion of 80%. At best, foundational skills require a substantially higher criterion of success. At best, these goals would lead to progress monitoring using Mastery Monitoring (MM), with its advantages (e.g., high instructional utility) and disadvantages (e.g., requiring development of reliable and valid measures for each goal.

2. A student’s Progress Discrepancy measured by the IEP goal for a reading should be assessed at least every 2 weeks on the GOM validated for progress monitoring of students with disabilities.

Systematic IEP goal progress monitoring remains infrequent and unsystematic. This lack of progress monitoring enables ineffective special education interventions to be continued when they need to be modified for student progress.

Ineffective Practices: Using teacher observation of goal attainment in the absence of either Mastery Monitoring or General Outcome Measures and a failure to revise the IEP inconsistent with the IDEA requirement when there is a lack of progress.

REPRESENTATIVE ARTICLES

Bailey, T. R., & Weingarten, Z. (2019). Strategies for Setting High-Quality Academic Individualized Education Program Goals. In. Washington, DC: National Center on Intensive Intervention, Office of Special Education Programs, U.S. Department of Education.

Jenkins, J. R., & Fuchs, L. S. (2012). Curriculum-Based Measurement: The paradigm, history, and legacy. In C. A. Espin, K. McMaster, S. Rose, & M. Wayman (Eds.), A measure of success: The influence of Curriculum-Based Measurement on education (pp. 7-23). University of Minnesota Press.

Fuchs, L. S., & Fuchs, D. (1988). Curriculum-Based Measurement: A Methodology for Evaluating and Improving Student Programs. Diagnostique, 14(1), 3–13.

Shinn, M. R. (2013). Progress monitoring methods and tools for academic performance. In H. M. Walker & F. M. Gresham (Eds.), Handbook of evidence-based practices for emotional and behavior disorders (pp. 173–191). Guilford.

The first thing is our inability to understand General Outcome Measurement (GOM) and how it is complementary, but different from Mastery Monitoring. GOM requires a validated measure of a larger construct. What is labeled Oral Reading Fluency (ORF), in commercial products like aimsweb, Acadience, DIBELS, etc. is based on research on Reading Curriculum-Based Measurement (R-CBM). Decades of research validates a short oral reading sample, collected under standard conditions for 1 minute, is an indicator of general reading ability. That is, knowing how many words a student reads correctly can tell us a lot about how well a student reads.

Unfortunately, a label like ORF and how some commercial products treat ORF as a subskill and/or bury it is a reading composite score misleads too many of us astray. ORF is not a (sub)skill of reading. It should, in my very humble opinion, cannot be “taught.” It is not be confused with automaticity with the code, a critical skill that can be taught. ORF is the sum of a number of reading skills, not the least of which is automaticity, but also includes comprehension, effective strategies for unfamiliar words, etc.

This week, I used AI to redo my website (markshinn.org) and prompted me to scan in and update the 2 edited books I did on CBM that are out of print and for which I now own the copyright. I’ve scanned them in an offer them for sale. I edited 2 CBM books, Curriculum-Based Measurement: Assessing Special Children and Advanced Issues in Curriculum-Based Measurement that included chapters by Stan Deno, Lynn Fuchs, Jerry Tindal, Scott Baker, and Doug Marston among others.

Shinn, M. R. (Ed.) (1989). Curriculum-based measurement: Assessing special children. New York: Guilford.

Shinn, M. R. (Ed.) (1998). Advanced applications of Curriculum-Based Measurement. New York: Guilford.

Some of the chapters are dated. Others, however, are great reads and can remind us—or help us understand the accomplishments that CBM has enabled us to achieve in MTSS and even dyslexia screening. “Old” isn’t necessarily bad.

I’ve copied an except of the first chapter of the Advanced Issues book as an example.

From Chapter 1. Shinn, M. R., & Bamonto, S. (1998). Advanced Applications of Curriculum-Based Measurement: “Big ideas” and avoiding confusion. In M. R. Shinn (Ed.), Advanced applications of Curriculum-Based Measurement (pp. 1–31). New York: Guilford.

Curriculum-Based Measurement (CBM) is a set of standard simple, short duration fluency measures of reading, spelling, written expression, and mathematics computation. CBM was developed to serve as Dynamic Indicators of Basic Skills (DIBS; Shinn, 1995) or general outcome indicators (Fuchs & Deno, 1991) measuring “vital signs” of student achievement in important areas of basic skills or literacy. In other words, they were designed to function as “academic thermometers” to monitor students’ growth in important skills domains relevant to school outcomes. CBMconsists of the following core testing strategies:

1. In reading, students read aloud from basal readers for 1 minute. The number of words read correctly constitutes the basic decision- making metric.

2. In spelling, students write words that are dictated at specified intervals (either 5, 7, or 10 seconds) for 2 minutes. The number of correct letter sequences and words spelled correctly is counted.

3. In written expression, students write a story for 3 minutes after being given a story starter (e.g., “Pretend you are playing on the playground and a spaceship lands. A little green person comes out, calls your name, and ... “). The number of words written, spelled correctly, and/ or correct word sequences is counted.

4. In mathematics, students write answers to computational problems via 2- to 5-minute probes. The number of digits written correctly is counted.

CBM is used by individual special education and general education teachers and school psychologists in public schools throughout the United States. It is used system wide in a Problem-Solving model in a number of schools and school districts in this country. In one state, Iowa, CBM forms a cornerstone of special education systems reform using a Problem-Solving approach. See Reschly and Grimes (1991) and Tilly and Grimes (Chapter 2, this volume) for information on this statewide use of CBM.

This chapter, like this book, is not intended for the novice reader to acquire basic information about CBM and its use in a Problem-Solving model. Instead, that reader is referred to Shinn (1989) or a variety of professional articles on CBM (e.g., Deno, 1986; Fuchs & Deno, 1991; Shinn, 1995). This chapter is intended for the reader who is generally familiar with the measurement tools and the basic framework for the Problem-Solving model as conceptualized by Deno (1989). Its purpose is to provide a brief history of CBM and review the “big ideas” in its use. The chapter also is intended to review these “big ideas” in the context of some of the criticism directed at CBM as a measurement instrument and as part of a change in the role of school psychologists.

BRIEF HISTORY OF CBM

CBM has evolved through a number of phases and names for its testing strategies, beginning with Stanley Deno’s attempts in the early 1970s to provide his special education student teachers with efficient yet accurate ways of assessing the effects of their instruction. Building on the accumulating body of knowledge from behavioral assessment of social behaviors in the budding field of applied behavior analysis, Deno sought to give his teacher trainees academic measures that could be (1) collected daily, if necessary; (2) graphed; and (3) evaluated for evidence of student learning within short periods of time. See Deno ( 1992) for more information on this early history.

Deno’s development and field-testing efforts in public school special education programs with Phyllis Mirkin led to the first formal conceptualization of CBM as Data-BasedProgram Modification ( DBPM; Deno & Mirkin, 1977).

The validation and enrichment phase of CBM began in earnest in 1978, when Deno was awarded a contract from the then U.S. Bureau of the Educationally Handicapped (BEH) as co-principal investigator of the University of Minnesota Institute for Research on Learning Disabilities (IRLD). One of the principal aims of the IRLD was to develop and validate simple methods of writing Individualized Education Plan (IEP) goals that could be used formatively to assess student progress. Graduate students, including Lynn Fuchs, Jerry Tindal, Doug Marston, Steve Robinson, Caren Wesson, and Mark Shinn were instructed to search the professional literature for potential academic or social behavioral tasks that showed promise as simple measures of student improvement. The potential measures were vigorously examined with school-aged children, with and without disabilities, regarding their technical properties (e.g., reliability, concurrent validity) as well as their logistics features (e.g., ease of administration).

Although the core measures included some of the tools of DBPM (e.g., oral reading fluency), the IRLD research identified other technical tools such as fluency-based spelling and written expression assessment. See Deno, Mirkin, and Chiang (1982) as an example of these early validation efforts. The validated tools were referred to as “simple” measures.

After extensive investigations of technical adequacy, the use of the simple measures in educational decision making became the focus of study. In the Pine County Special Education Cooperative under the direction of Gary Germann, teachers were trained, students’ progress was assessed over time, and decision-making outcomes were evaluated (e.g., Germann & Tindal, 1 S85). The use of the descriptor “simple measures” to represent the emerging measurement technology proved unsatisfactory, in large part because the term only described one broad and perhaps less relevant feature (i.e., the fact that the measures were not complicated) of the measurement system.

Data-Based Assessment was the next descriptor that was used. It tapped into a more salient feature about the measurement system in that educational progress decisions were to be data based, but eventually this term was discarded in preference to Curriculum-Based Assessment (CBA) around 1983 (e.g., Marston & Magnusson, 1985).

This descriptor appeared to represent one of the measurement technology’s more important features and was responded to favorably by educators. Interest in the general topic of CBA exploded in the mid-l 980s with a decrease in confidence in prevailing assessment practices in special education and school psychology (i.e., test-and-place using commercial, nationally norm-referenced tests) and an increased interest in seeking alternative testing approaches. CBA as a viable alternative was promoted by the publication of a special issue of Exceptional Children on the topic edited by Tucker (1985), and an edited book by Idol, Nevin, and Paolucci, & Whitcomb (1986). These summary articles and book chapters also were accompanied by a rapidly expanding professional literature with research-based investigations of features of CBA.

However, it was clear when reading the mid- l 980s professional literature that there were a number of different approaches to assessing student performance with curriculum materials. CBA was not a unified method of testing and, in fact, the approaches grouped together under this general term differed considerably. In an effort to set its testing practices apart from other CBA models, a consistent use of the descriptor “Curriculum-Based Measurement” (CBM) was undertaken around 1986 in Deno’s seminal article in School Psychology Review on formative evaluation (Deno, 1986).

The differences in CBA models were laid out explicitly in a special issue of School Psychology Review (Shinn, Rosenfield, & Knutson, 1989). Two of the most salient differences among the CBA models were the (1) purposes of assessment, and (2) research support for testing practices and decision making. CBM from the late 1980s and through today has seen continued support from the U.S. Department of Education for preservice graduate training and research and development. CBM has also served as a dependent measure in a number of federally funded research projects. See Chapter 4 by Fuchs and Chapter 9 by Powell-Smith and Habedank-Stewart in this book and Shinn, Powell-Smith, and Good (1996) for examples of this federally funded research.

The general acceptance of CBM as a viable assessment tool is evidenced in a number of other ways, including its prominence in task force documents on assessment and special education reform. Among the most visible federal documents in recent years was The National Agenda for Achieving Better Results for Children and Youth with Disabilities (U.S. Department of Education, 1994). The document, compiled by assessing interdisciplinary focus groups of educators, parents, and advocates, emphasized the role of CBM by name and the type of measurement characterized by CBM (e.g., continuous progress measures, measuring student results, assessments to maintain and support appropriate teaching) as potential solutions to the accepted problems of conventional “test-and-place” assessment activities. Similar content was included in the National Association of School Psychologists document (1994) entitled Assessment and Eligibility in Special Education: An Examination of Policy and Practice with Proposals for Change.

As mentioned earlier in this chapter, adoption of CBM and using it in a Problem-Solving model continues at a high-level pace. An interesting shift in focus has occurred since the mid-l 990s, however. Although one of the inherent messages in using CBM in a Problem-Solving model is the linkage to what is taught in general education classrooms and the need to make special education decisions in the context of general education expectations, most adoption historically has been stimulated by special education assessment reform interests. In particular, alternative assessment implementation using CBM has been spearheaded by school psychologists. It appears that adoption now is being stimulated much more frequently by general educators, most typically principals who are seeking ways of documenting the progress of their students in the basic skills areas. We see this avenue as the most promising for facilitating the improvement of basic skills and literacy for all children, including students who receive special education.

Congratulations on reading my Substack this far. I hope that it has clarified some confusions that exist and may pique your curiosity to go back in the Time Machine and read some more, especially by Stan Deno and Lynn S. Fuchs.

In 1998, in a contract with the David and Lucille Packard Jr. Foundation, I was asked to help evaluate its California project that supported implementation of the Open Court reading program in the primary grades in multiple sites. Mr. Packard wanted more than just evaluation data from commercial tests, typically insensitive for K and Grade 1. At the recommendation of Doug Carnine, Mr. Packard contacted me to use more direct and sensitive measures for beginning reading. I asked a (former) colleague to help me combine some K-early Grade 1 measures (i.e., early DIBELS) with my work in Curriculum-Based Measurement (CBM).

The results of our evaluation were positive. K and G1 students were “reducing the normative gap!” on our combined early DIBELS and CBM measure (Reading CBM or ORF). Shortly thereafter, this combination became the core reading components of a number of commercial products, DIBELS that became Acadience, aimsweb, DIBELS 8th ed., Fastbridge.

In 1980 local CBM norms development began to operationalize Stan Deno and Phylis Mirkin’s Data-Based Program Modification (DBPM) individual problem solving model. By thee mid 1980s, teacher interest in collecting reading (and other CBM data) on ALL their students was increasing rapidly from just the results from SOME of their students as part of the development of local norms. multiple times (3x per year), Fall, Winter, Spring.

The practice of testing ALL the students was 1 of a number of options for developing local (e.g., subsamples of class, school, district) norms, but given teacher interest in data on all their students, an option was given to develop norms on ALL their students. We (jokingly) called this norm development option “sheep dipping” based on the practice of providing preventative treatments for everyone.

Clearly, sheep dipping was an inappropriate name for a process of collecting REPEATED achievement data on all students. The term Benchmarking—or Benchmark Assessment was chosen.

Benchmarking enabled schools to do 2 things. First and foremost, Benchmarking enabled Stan Deno’s vision of providing teachers information on Growth and Development on all their student to come to fruition. Universal Progress Monitoring! Additionally—and secondarily—Benchmark data could be used to identify students with Performance Discrepancies. Students sufficiently different from normative performance that might indicate the need for potential more appropriately intensive intervention or more intensive evaluation could be identified. In short, Universal Screening.

The timing of the specification of Benchmarking (not coincidentally) coincided with a number of federal reading reports, policies and practices, most notably No Child Left Behind that required reading screening in funded Reading First sites and the development of commercial products. DIBELS, aimsweb, etc. make collection of repeated reading data on students easier to collect, organize and report.

Fast forward a quarter century. Response to Intervention (RTI), subsequently Multi-Tiered Systems of Support (MTSS) and most recently dyslexia screening seems to have made multiple within (and across) years standard educational practice. But distorted. Universal screening became the focus. Universal progress monitoring to support ALL students reading growth and development all but disappeared as a first priority. The screening priority became bloated by the expansion of what originally intended as Deno’s “simple measures” or “indicators” of reading general outcomes to include a variety of mastery monitoring reading subskills for purposes of teacher instructional planning.

As a result, it seems like today, across 50 states and Washington DC all students (with and without risk for reading difficulties) are screened on multiple measures, including most that are not screeners, multiple times, within and across years. A proficient reader in K , then 1, then 2, then 3 (and often higher within MTSS) is screened!

Teachers are overwhelmed with subskill reading data with the presumption of developing differentiated instruction.

Time for a different approach. Resurrecting Benchmarking for universal growth and development. Using a Multiple Gating (MG) approach for universal reading screening.

Recently, Hill Walker, Ph.D., one of education’s icons in the field of behavioral health and disorders, and I were asked to write a white paper on Multiple Gating for a prominent organization. Attached you will find the white paper.

Some References

Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232.

Germann, G. (2010). Thinking of Yellow Brick Roards, Emerald Cities, and Wizards. In M. R. Shinn & H. M. Walker (Eds.), Interventions for achievement and behavior problems in a three-tier model, including RTI (pp. xiii–xxxv). Bethesda, MD: National Association of School Psychologists.

Germann, G. (2012). Implementing Data-Based Program Modification Big Ideas. In K. L. M. C. Espin, S. Rose, Miya Miura Wayman (Ed.), A measure of success: The influence of Curriculum-Based Measurement on education (pp. 79–87). Minneapolis, MN: University of Minnesota Press.

Germann, G., & Tindal, G. (1985). Applications of direct and repeated measurement using curriculum based assessment. Exceptional Children, 51(2), 110–121.

Shinn, M. R. (1988). Development of curriculum-based local norms for use in special education decisionmaking. School Psychology Review, 17, 61–80.

Shinn, M. R. (2010). Building a scientifically based data system for progress monitoring and universal screening across three tiers including RTI using Curriculum-Based Measurement. In M. R. Shinn & H. M. Walker (Eds.), Interventions for achievement and behavior problems in a three-tier model, including RTI (pp. 259–293). Bethesda, MD: National Association of School Psychologists.

The paper

https://tinyurl.com/MGATE2026

A webinar I did recently. I can’t claim it is any good. First time through new content is usually very rough (for me-and the attendees).

https://us06web.zoom.us/rec/share/b4IPHr8s0fGenA9gLCKcCXlFvD4mpQxVVs0KiA-bEE20iOSPj6XwwqCquVcNDrNh.ovV0QXg613jTqEKK

Unfortunately, we spend more time teaching people how to determine the Performance Discrepancy (i.e., whether a student is sufficiently different from a norm or standard to require an appropriately intensive intervention) through practices like universal or individual student screening, than on determining the Progress Discrepancy and whether it is being reduced.

Historically, there are two relatively obscure approaches to interpreting academic PM data in a time series, (a) an experimental approach that employs a number of methods, some of which are useful (changes in slope, variability, change in level at the point of intervention) and some that are less useful...changes compared to a “baseline.” I find Baseline troublesome in academics because (a) there is rarely a no treatment condition and (b) it requires a wait to fail perspective. The other approach to evaluating PM graphs (b) is called a therapeutic approach. A student’s current performance level now is compared to where the student would be expected to be based on an expected rate of progress. For example, if a student is predicted to be reading (about) 75 Words Read Correctly (WRC) by December 15th and they are now reading (around) 80 WRC, I would consider the intervention to be reducing the gap. Reading (around) 60 WRC, i would consider evaluating the current intervention. To me, this is the preferred method.

The therapeutic approach is closely tied to a Problem-Solving model (Deno, 1989, 2002, 2012), where the discrepancy between what is expected and what is occurring is measured on an ongoing basis. For example, is the student’s Expected ROI equal to or greater than the Actual ROI. This approach, of course, means spending some time thinking about what the Expected ROI would be!

I worry that too many people are relying on “normative” growth rates, that are derived from a range of typically achieving students, most of whom are NOT receiving “distinguished” (i.e., research-based, appropriately intensive) intervention. What does improving at 2 WRC per week mean for a Grade 2 student who has a severe/significant Performance Discrepancy? We need to know that!

I will start with a major premise that guides my decision making.

Does the proposed intervention reduce the achievement gap?

A Grade 2 student who performs at the 1st percentile at the beginning of intervention who performs at the 15th percentile at the end of the year HAS significantly reduced the achievement gap! The student still, needs intensive intervention, but great growth! A student who is at the 15th percentile before intervention who improves at the same rate of typically developing students and is at the 15th percentile at the end of intervention has indeed grown. But not enough to reduce the achievement gap! To me, this intervention is not effective.

ROI is one way to judge whether the “gap” is being reduced, but we all know that slope, in an of itself, can be problematic. It is based on a number of assumptions:

1. that growth is linear,

2. that probe to probe difficulty is nearly perfectly equivalent, and

3. that student, teaching, and curriculum variables are perfectly equivalent day to day!

I doubt these assumptions are true. Calculating slope typically includes all the data points and to me, many of them are irrelevant. Data gets old--no longer relevant, but is included in the calculation. If my goal is to lose 30 pounds, do I need to know how how well I’m doing now based on my starting weight? The rate at which I lost weight? I have lost weight? Or should I look at where I am NOW relative to my goal of losing 30 pounds by my goal date?

I like ROI as a floor below which our goal should never fall. But the goal we select--and our expected—ROI MUST reduce the achievement gap. The aimline and trend line serve as heuristics. Am I on target to be losing 30 pounds by X date?

It’s a here and now approach within the Therapeutic model. Originally conceived by people like Owen White, Norris Haring, and Kathleen Liberty from the Precision Teaching fame of the 1970s and 1980s--and often used as a decision rule in Lynn and Doug Fuchs’ research, 3 days below the aimline is a trigger to consider whether the intervention is effective.

I find the 3-day rule too prescriptive, but the idea that we judge “where are we now compared to where we want to be” is a good one. That estimate of the current Performance Discrepancy IS reliable and to me, makes sense. If you are worried about lack of progress, get a new, high quality data point. Give 3 probes from the goal material and compared that value to where the student would be expected to perform, given the ROI for expected rate of progress. If the data points are below the aimline think about evaluating the current intervention.

I must also comment more on ROIs. Most of the commercial CBM products are not especially clear about how ROIs are calculated. I find ROIs calculated at specific points in the distribution of student scores (e.g., ROI for students at the 10th percentile) misleading at best and potentially dangerous if used to set goals. That is, thinking that students at the 10th percentile learn at a different rate than students at the 50th percentile. Doesn’t that sound like (false) perceptions of cognitive ability? Like ROI is a characteristic of the student?

Remember this. ROI is a characteristic of the intensity of intervention!

Decades ago, school psychology graduate students in a summer practicum teaching students identified as LD and eligible for summer school based on their lack of progress during the school year, were obtaining ROIs greater than 3-4 WRC per week. Of course, the students, although novel to teaching reading, were using Reading Mastery, an intensive reading intervention, and were carefully supervised to deliver the program well.

Finally, time to abandon the label ambitious for goals. The label was first used by Fuchs, Fuchs, Hamlett, Walz, & Germann (1993) who calculated ROIs using reading probes that were not controlled well for difficulty with students who were receiving unremarkable instruction. In my experience, the label scares educators. I mean, ambitious must mean difficult to achieve!

Bottom line. Progress monitoring practices need more professional learning, avoiding goal setting without thinking about the needed intensity of intervention. If we expect more—and we should—we need to deliver more. More appropriately intensive, research-based, field tested, interventions designed to reduce the gap.

The some of the references below are old, but reflect some needed new thinking when talking about PROGRESS.

You can find many of them in my Dropbox.

https://bit.ly/MarkProLearn

Shinn, M. R. (2013). Progress monitoring methods and tools for academic performance. In H. M. Walker & F. M. Gresham (Eds.), Handbook of evidence-based practices for emotional and behavior disorders (pp. 173-191). New York, NY Guilford.

Fuchs, L. S. (1989). Evaluating solutions: Monitoring progress and revising intervention plans. In M. R. Shinn (Ed.), Curriculum-based measurement:Â Assessing special children (pp. 155-183). NY: Guilford.

Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L., & Germann, G. (1993). Formulative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22(1), 27–48.

Tindal, G. (1982). Visual analysis of time series data. (Unpublished doctoral dissertation), University of Minnesota, Minneapolis,

White, O. (1971). A pragmatic approach to the description of progress in the single case. (Unpublished doctoral dissertation), University of Oregon, Eugene, OR,

White, O. (1987). Some comments concerning “The quantitative synthesis of single subject research.” Remedial and Special Education, 8, 34-39.

White, O. R. (1972). Methods of data analysis in intensive designs. Paper presented at the Paper presented at the annual meeting of the American Educational Research Association.

White, O. R. (1974). Evaluating educational process. In: Seattle: University of Washington, Child Developmental and Mental Retardation Center, Experimental Education Unit.

White, O. R., & Haring, N. G. (1980). Exceptional teaching (2nd ed.). Columbus, OH: Merrill.

They seem to be a prescription about what MTSS “looks like” and the basis for schools’ evaluation of their current status. I hear “we have too many kids in the red (i.e., Tier 3). We should only have 5%!

These %s were never intended to be a prescription nor a tool to evaluate current MTSS implementation. BTW, knowing the % of students with academic performance discrepancies and/or who need more appropriately intensive intervention should be “today’s news.” Prior to MTSS, knowing how well our students are doing relative to achievement standards and risk should have been known for a long time. Schools are/were data rich!

The %s are heuristics, ways to promote thinking! Dr. Walker and I attempted to clarify what these #s “mean” in theory, but more importantly in practice in the first chapter of the third edition of our NASP Interventions book (2010). I’m not trying to hawk the book (you can find it in my collection of professional learning article at https://bit.ly/MarkProLearn), but, in theory, the Tier 1 core program should meet the needs of most of your students. 80% is an arbitrary number and it is an aspiration, not reality. The idea is that if sizable proportions of students are not meeting achievement standards and may be at risk, then the core, Tier 1 intervention needs to be increased in its intensity.

The #s in Tier 2, suggest, in theory, that no matter how effective your core program is, there will be some students who will need more intensive intervention? Why 15%? Forget the value. It’s that some students will need more.

The #s in Tier 3 suggest, in theory, that no matter how effective your core program is, no matter how effective Tier 2 intervention might be, there will be some students who will need the most intensive intervention? Why 5%? Forget the value. It’s that SOME students will need MOST.

More detail on this topic is provided in the Walker and Shinn chapter, but remember, this heuristic is something that supports our thinking about what we want to accomplish and what we may need to do to intensify intervention, potentially across all tiers. In too many cases in our Country, Tier 1 in one community may need to look like Tier 2 or even Tier 3 in other communities. Do we think, do we believe we can do that? Are we willing to do that?

The #s help us understand the underlying concepts. Continuously—or contemporarily—is really kind of silly. For example, if one defines meeting state standards as “reaching” students, then probably most schools’ core programs are “effective.” But we need to be wary. Too many state standards tests have low cut scores and worse labels for student performance (i.e., “Basic, Below Basic”). They are not real measures of student proficiency. Just compare % meeting state standards to NAEP results. Or international comparisons.

Finally, think just a minute about the % in Tier 3. If those #s reflected reality, there are either 0% of students in SE, or if Tier 3 is synonymous with SE (which, although there might be some overlap, it is not), our SE population would be slashed in half!

In brief, avoid the #s. Think. Don’t let the #s get in the way. Is the core meeting the needs of most students? If not, increase the dosage/intensity. Lots of kids at risk? Increase intensity of instruction across the board! Simultaneously! In proactive design!

In the webinar—and in a previous Substack—I attempted to communicate that we can “do less” MTSS testing and dyslexia screening. We don’t need 3x per year screenings with multiple reading subskills, tested over multiple years, often at least three years or more.

Finding poor readers at risk for reading difficulty is remarkably simple. Have a student read aloud reasonably controlled difficulty grade-level passages. I prefer 3 to avoid the impact of a single passage that may be of higher or lower interest or less relevant to the student. I also prefer the median because it is not unduly influenced by a single unusual score. I can sound like I know more than I do by saying the median is the least biased estimator of central tendency when there is a small sample size or non-normal distribution. I also appreciate multiple reading samples at a given time because it enables me to attend to the qualitative features of reading, the additional datum that brings richness to oral reading fluency.

Post webinar, I received a comment about how, despite what we know, about how we can do less assessment (and ideally, more appropriately intensive instruction), practitioners are often constrained by legal practices such as mandated dyslexia screening.

I am struggling with the business/product nature of education. For me, there is legislation in my state that requires reading screening (great!) but then also required the use of <test name deleted> by all districts.

Advocates, especially parents, often are not “sophisticated” when it comes to assessment. That said, the same goes for legislators who listen to advocates. Sadly, the same complaint can be made about many, too many, educators, including leadership. And, even test publishers.

The latter are understandably in the business of selling things, giving the customer what they believe the customer wants. I’ve been part of developing commercial products. It’s a good thing. But often giving the customer what they presumably wants makes mistakes. A case in point is information obtained by asking “focus groups.” Steve Jobs of Apple never believed in focus groups. He wanted to deliver a product that they NEED, not what the customer WANTS.

If you want sales, you provide what mandatory dyslexia screening mandates. Or you try to flesh out what good MTSS assessment practices look like, often on the basis of modeling common practices. As a result, today we have the bastardization of Benchmarking. Where universal screening is the principal, if not sole purpose of testing 3x per year, to determine which students need more appropriately intensive intervention. Where the intended Benchmark purpose of universal progress monitoring for growth and development of all students is given short shrift. As the wise Gary Germann said, adults will always be more interested in screening because we’ve always been interested in finding “broken” kids. Progress monitoring, and universal progress monitoring, the intended primary goal of Benchmarking, is finding out whether what the ADULTS do “makes a difference.” I swear, too often we don’t really want to know because (we) the adults would have to do something different.

I learned this year when I implemented ORF screening in my 4/5 school. We learned <I’ve deleted the commercial product’s name> does not populate graphs for overall trends at the school or district level unless you administer both the ORF and MAZE to calculate the composite.

To me, composite scores are an example of extremely talented psychometricians, statisticians, and university professors seeking new things to publish using “dark magic” to make the simple more complex—and more complicated. Thanks to Dr. Chris Schatschneider for introducing me to this term.

Composite scores combine multiple measures with the intent of increasing predictive validity, sensitivity and specificity. I wrote a Substack about my concerns in prediction models and the composite scores that increase “predictive accuracy.” How? The statistics are far too complicated for nearly everyone I know to understand. Me, included. But published research articles demonstrate that ORF/R-CBM is a validated method for differentiating students’ reading needs as well as student progress.

I don’t need complicated “dark magic” to know that THIS KID, NOW, needs more appropriately intensive reading instruction. Reading composite score that “bury” R-CBM with multiple other reading measures and even pseudo-scores like accuracy, just make things less time and cost efficient, but worse, more complicated than necessary.

We can do less. Use ORF (R-CBM) as the primary tool in MTSS and dyslexia screening Grades 1-5, but ensure principal use for universal progress monitoring to ensure all students are moving toward the goal of literacy. Do less, but obsess. Get really good at it. And be judicious when additional test data are needed, for whom, and when.

Relatedly, let’s take Student B who was not identified as AR at Fall screen who was identified as AR at the end of year. This instance also considered an error in statistical prediction methods. A result like this lowers a screening test's specificity. The student was a false negative, “missed” in the Fall screening.

Outcomes like these seem likely. Student achievement is not fixed, predetermined. Student achievement dynamic, and dependent on a number environmental variables. One variable is early and powerful reading intervention to increase AR students’ achievement? Intervention to reduce the reading achievement gap? Isn't the point of early screening to provide an appropriately intensive intervention to Student A so they are longer AR?

When positive status change happens, that's great news, not a screening error! What about Student B who was not AR at the beginning of the year whose end of year screening status was AR? Is that, too, a failure of the screener? Unlike Student A who was identified as AR and presumably provided an intervention, perhaps Student B didn’t achieve much during the academic year, for which there are numerous possible explanations. How about poor attendance? Poor core or Tier 1 instruction? Perhaps there were issues with behavioral health or classroom climate?

The point is that unlike screening statistical prediction models where risk is more “static” (e.g., smoking and cancer risk) student learning is dynamic. There are many potential intervening variables that can affect achievement. The change in Student B’s Risk status is NOT a screening error.

Unfortunately, statistical prediction models that evaluate screening tests with criteria like Sensitivity and Specificity form the basis for major decisions about the quality of a reading screening tool. You will see these evaluative standards used on federally funded research sites like the National Center on Intensive Intervention screening tool charts (https://charts.intensiveintervention.org/ascreening). No two reading screening tests create their AR cut scores the same way. They differ in the "dark math" used (i.e., the statistical models), differences in each screener’s predictor variable (e.g., single word reading) and the predicted outcome criterion (e.g., a state Grade 3 test), resulting in judgments of Sensitivity and Specificity, we may misjudge some useful reading screening tests. It seems we must “trust the experts.”

How about changing how we evaluate screeners?

1. Does the screening test identify students who need early intervention, the most discrepant readers, reliably? A screening should identify the same students if the screener is repeated.

2. Does the screening test lead to an appropriately intensive intervention? If it doesn't--and this is more complicated--then it's a poor screener.

3. Does the screener result meaningful student achievement that REDUCES THE GAP for students identified as AR, a Big Idea that has formed the core of my practice for more than 40 years?

These latter two criteria are more complicated and, of course, beyond the control of the screener. Yet, considering the two variables would be a way of judging Consequential Validity, a screener’s test use. Does the screening test use result in meaningful achievement gain for students AR?

One of my early career influences, the late Dr. Maynard Reynolds, a professor of special education at the University of Minnesota said our job is to"not to observe the heavens and make predictions. Our job is to CHANGE those predictions.”

Quality screening and appropriately intensive intervention go hand in glove. Don’t screen if you don’t intervene, as early, accurately, and powerfully as possible. And let’s carefully consider the use of predicted At Risk status cut scores as the primary basis for selecting a reading screener. We need to look carefully at the dark magic process.

Subscribe now