Reading Mastery and Students with Learning Disabilities: A Comment on the What Works Clearinghouse Review NATIONAL INSTITUTE FOR DIRECT INSTRUCTION Technical Report 2012-1 Jean Stockard and Timothy W. Wood July, 2012 Technical Report 2012-1 Reading Mastery and Students with Learning Disabilities A Comment on the What Works Clearinghouse Review A recent report by the What Works Clearinghouse (2012) examined two studies of the use of Reading Mastery with learning disabled students and concluded that it had “no discernible effects on reading comprehension and potentially negative effects on alphabetics, reading fluency, and writing.” This conclusion is in stark contrast to dozens of studies of Reading Mastery and other elements of the Direct Instruction (DI) corpus of material. This research has consistently found strong positive effects of the programs on academic achievement for students of all ability levels. The Office of Research and Evaluation at the National Institute for Direct Instruction examined the WWC’s report and the research articles used to develop its findings and found very serious problems with its conclusions. The WWC analysis was based on only two articles. One (Cooke, Gibbs, Campbell, & Schalvis, 2004) compared two very similar Direct Instruction programs, Reading Mastery and Horizons, and found that students in both programs made gains over the academic year that were significantly greater than those made by students in national and state level populations. Gains in both programs were similar, leading the WWC to conclude that Reading Mastery was no better than its comparison program. They ignored the fact that students performed significantly better than the national or state norms or that the comparison program was similar on all but a very few characteristics, essentially a modified version of Reading Mastery. The second article reviewed (Herrera, Logan, Cooker, Morris, & Lyman, 1997) involved two groups of students, both of whom appear to have received Reading Mastery as part of the schools’ “usual and customary school day curriculum.” One group of students also received 45 minutes of supplemental phonemic related instruction, from their regular classroom teachers, that involved motor activities to accompany practice of skills. Not surprisingly, the group receiving the additional instruction had significantly larger gains than those who did not have additional learning time. Despite these differences in exposure and the fact that both groups appear to have had Reading Mastery as their usual reading curriculum, the WWC used these results to suggest that Reading Mastery could have potentially negative effects. The first section of this report provides additional details on each of these studies. The WWC uses what can be termed an “exclusive approach,” putting a great deal of emphasis on “internal validity” and focusing only on studies that incorporate a select set of characteristics, involving the nature of the design, subject assignment, testing procedures, setting, and time of publication. In short, the WWC appears to be searching for a perfectly designed experiment, implicitly suggesting that a perfectly designed experiment will give the 1 Reading Mastery and Students with Learning Disabilities best results. Not surprisingly, very few studies pass its screening procedures. In contrast, meta-analyses and literature reviews often take a much broader view of the literature, looking at the totality of information that is available. While a WWC report may find only one or two studies that meet its criteria, the meta-analyses and other reviews of Direct Instruction typically report on dozens of studies. The second section of this document reports the results of several meta-analyses and literature reviews of DI programs, focusing only on those that report results separately for special education students. Over the last four and one-half decades many studies have appeared regarding the efficacy of Direct Instruction. The National Institute for Direct Instruction is compiling and abstracting this vast bibliography. The final section of this document provides citations to and abstracts of many individual studies that pertain to the content of the WWC report. The full NIFDI bibliography is available on-line at www.nifdi.org/research. The WWC Report A valid test of the efficacy of any intervention requires that the treatment received by the control group and the treatment group be distinct. The WWC clearly includes this criterion in its list of reasons that a study may fail to meet WWC standards, noting that a study “may fail to meet WWC evidence standards” if “the measures of effect cannot be attributed solely to the intervention—the intervention was combined with another intervention” (What Works Clearinghouse, 2011, p. 15). The WWC conclusions regarding Reading Mastery (RM) and learning disabled students were based on analyses of two studies, both of which should have been excluded from analysis based on this criterion. This section summarizes the designs of these studies and explains why they do not provide a valid test of the efficacy of RM. A Comparison of Two Direct Instruction Programs The first study reviewed by the WWC (Cooke, et al., 2004) is titled “A Comparison of Reading Mastery Fast Cycle and Horizons Fast Track A—B on the Reading Achievement of Students with Mild Disabilities.” It examined gains in reading achievement over the school year of learning disabled students studying with one or the other of these programs and compared these gains with national norms and with each other. Reading Mastery and Horizons are both part of the DI corpus of curricular material and were developed and authored by S. Engelmann and colleagues. The first three pages of the Cooke, et al. paper describe the close alignment of the programs, stating that “Horizons shares many important design features with Reading Mastery” (p. 140). Seventeen very specific points of design similarity are cited (p. 140, citing Engelmann, 2000), and only five points of difference are described (pp. 140-141). The major difference between the programs is the prompts used to help students learn the relationships between letters and sounds, primarily involve differences with orthography at the early stages. In other words, Horizons may be seen as a modified version of Reading Mastery. Thus the study reported by 2 Reading Mastery and Students with Learning Disabilities Cooke and associates did not compare Reading Mastery with another type of reading program. It compared Reading Mastery with a slightly altered version of the same program. The authors report results using norm-based standard scores, comparing students’ standing to other students in their grades at the testing time. Such a norm-based comparison allows the calculation of effects relative to the total population, for, as explained by the authors, “if students progressed at an average rate across the course of the study, they would show no change in these scores” (p. 146). They found that, relative to other students in the nation and state, students in both groups made statistically greater gains over time. These results appeared on all measures used. Effect sizes ranged from .09 to .36 for sub-tests of the WJ-R and .71 to .78 for subtests of a state assessment. (Effect sizes of .25 or higher have traditionally been seen as educationally important (Tallmadge, 1977).) The gains were similar for students in both Horizons and Reading Mastery. The WWC report focused on the lack of difference between Horizons and Reading Mastery, rather than a) students in both programs had significantly stronger gains than students in national or state level comparison groups and b) the programs were essentially equivalent in design. An appropriate reading of the article would be that it provided confirmation of the efficacy of the Direct Instruction approach in teaching reading. Instead, the WWC chose to report the study as showing “no discernible effects” of Reading Mastery. A Comparison of Reading Mastery and Additional Intervention The second article reviewed by the WWC (Herrera, et al., 1997) compared growth over time in achievement of students who received only Reading Mastery with those who received an extra 45 minutes daily instruction involving “sensorimotor, procedural activities patterned to phonemes, graphemes and phonographs” (p. 71). Reading Mastery was described as “the usual and customary prescribed curriculum for the school day” (p. 78). Six classrooms of students receiving only Reading Mastery were treated by the WWC as the intervention group for their review. The intervention group in Herrera, et al.’s analysis was comprised of students in five other randomly selected classes within the same district. The teachers in these classrooms were trained, in a 60-hour program, to use a sensorimotor procedure to teach reading “for forty- five minutes per school day as a supplement to the usual and customary school day curriculum.” An appendix to the article describes sample activities in the approach, which is termed the “Stabilized Learning System.” It describes how “visual representations of words are simultaneously created and expressed via movement” (p. 85). Students were given explicit instruction in various phonemic elements accompanied by associated movements, designed to help “ensure stabilization of the movement and represented processes” (p. 85). Movements involved hand motions or activities on floor grids, where students would “perform all of the above [phonemic] activities in simultaneous-oral fashion (saying the 3 Reading Mastery and Students with Learning Disabilities word, then the letter names while ‘moving’ them or ‘stepping’ them, then saying the word at the end) in simultaneous-sounding fashion.” The authors described this approach as “implicit,” because of the centrality of sensorimotor functions rather than emphasis on the rule based elements of learning to read. While described as implicit, the program appears to be very different than the “implicit” learning approach used by the so-called “whole language” movement, for it used explicit reinforcement of learning related to “phonemes, graphemes, and phonographs” (p. 71). In addition, all of the teachers assigned to the program used Reading Mastery in their classrooms as part of the district’s “usual and customary prescribed curriculum” (p. 78), and it would be expected that the skills and procedures used there would extend to their work in the sensorimotor component. Teachers were well trained in the intervention, and were reported to have “received sixty hours of instruction involving these [sensorimotor] procedures” (p. 78). For their analysis the WWC treated this intervention group as the control condition and the group that did not receive Stabilized Learning (SL), only having Reading Mastery, as the intervention or treatment condition, implicitly assuming that the two treatments were independent of each other. However, a careful reading of the article casts doubt on this assumption. As noted above, Reading Mastery was “the usual and customary prescribed curriculum for the school day” (p. 78). The authors reported that the sensorimotor intervention “was implemented for forty-five minutes per school day as a supplement to the usual and customary school day curriculum” (p. 78). While the SL program “was integrated into the school day with no extra time allotment required” we have not been able to find any place within the article that explicitly states that the students using the SL system were not exposed to RM. If SL were indeed, as stated, a “supplement to the usual and customary prescribed curriculum,” they would have also received Reading Mastery. Thus, the analysis appears to have compared gains in reading skills of students exposed to Reading Mastery to those exposed to Reading Mastery plus an additional 45 minutes of instruction in phonics-related skills. In short, the design does not appear to provide a test of the efficacy of Reading Mastery but, instead, a test of the extent to which 45 minutes of additional teaching in phonemic related matters can supplement the gains made by Reading Mastery. As would be expected, the students receiving this additional instruction had stronger gains over time. An internet search using Google Scholar found no other scholarly studies of this program. If, in fact, the Stabilized Learning System did provide superior results to Reading Mastery there should be other tests and reports of this finding. Conclusion When examined carefully, neither of the studies cited by the WWC as demonstrating the ineffectiveness of Reading Mastery actually does so. In one study (Cooke, et al., 2004), students studying with Reading Mastery had significantly stronger gains than those in 4 Reading Mastery and Students with Learning Disabilities national and state populations. The WWC ignored these results and focused instead on the lack of difference between Reading Mastery students and those in another Direct Instruction program, claiming that this demonstrated a lack of effectiveness. The other study appeared to use nonequivalent interventions, with both groups exposed to Reading Mastery as part of the usual curriculum and the comparison group having significant extra instructional time designed to reinforce the phonemic understandings gained in the schools’ usual and customary reading curriculum, Reading Mastery. A large body of literature has documented the efficacy of Direct Instruction curricula, including many studies that looked at special education populations, including learning disabled students. The next section describes summaries of this work. Meta-Analyses and Literature Reviews The literature related to Direct Instruction is very large, with publications appearing over the last fifty years. A number of authors have reviewed the literature and summarized the major findings. These summaries build upon the epistemological tradition found in the classic literature on experimental design (Campbell and Stanley, 1963; Cook & Campbell, 1979; Shadish, Cook, & Campbell, 2002) and a very large methodological literature within the social sciences that stresses the importance of repeated studies, or replications, of analyses (See Stockard, 2011 for a summary). Writers in this epistemological tradition emphatically state that there can be no “perfect” experiment and that it is essential to examine effects of interventions over a broad range of settings and conditions to amass knowledge. The authors of the literature reviews and meta-analyses described below have systematically reviewed dozens of studies of Direct Instruction conducted in many different locales and with a wide variety of subjects. Four of the reports were systematic literature reviews; four were quantitative meta-analyses. All of the reviews concluded that Direct Instruction curricula are highly effective. Their methods and conclusions are summarized below. Only reports that included specific mention of results pertaining to special education students are included. When results regarding Reading Mastery and/or learning disabilities or special education students were reported separately, they are noted. The systematic literature reviews are listed first, followed by the meta-analyses. In each section reports are listed alphabetically. Systematic Literature Reviews Systematic literature reviews provide narrative summaries of the literature, detailing the findings and noting key similarities and differences in the results. 1) Kinder, D., Kubina, R., & Marchand-Martella, N. (2005), Special education and Direct Instruction: An effective combination. Journal of Direct Instruction, 5 (1), 1-36. Also 5 Reading Mastery and Students with Learning Disabilities distributed by SRA/McGraw-Hill as Special Education and Direct Instruction: An Effective Combination. This article summarizes 37 studies that used Direct Instruction materials with students with high-incidence disabilities, ranging in age from pre-school to high school, and 8 studies that used DI materials with students with low-incidence disabilities. Of the 45 studies, over 90 percent found positive effects for the Direct Instruction programs. Seven of the studies involved school-aged populations with high incidence disabilities using Reading Mastery or DISTAR (the precursor to Reading Mastery) as compared to another program. Learning disabilities are typically considered a high incidence condition and, thus, these studies are most relevant to considering the conclusions of the recent WWC report. (They were Branwhite, 1983; Kuder, 1990; Marston, Deno, Kim, Diment, & Rogers, 1995; O’Connor, Jenkins Cole, & Mills, 1993; O’Connor & Jenkins, 1995; Richardson, DiBenedetto, Christ, Press, & Winsberg, 1978; and Stein & Goldman, 1980). Only one of these studies (Marston, et al., 1995) found that the comparison group outperformed the students in the DI program. One of the studies (Glang, Singer, Cooley, & Tish, 1992) looked at the impact of Reading Mastery for students with low incidence disabilities using a single subject design (two students with traumatic brain injury). Results with this study, as well as with the other 7 studies of students with low incidence conditions, indicated positive effects. 2) Przychodzin, A.M., Marchand-Martella, N., Martella, R.C., & Azim, D. (2004). Direct Instruction Mathematics Programs: An Overview and Research Summary. Journal of Direct Instruction, 4 (1), 53-84. This article reviews 12 studies of DI math programs published since 1990. All but one of the studies showed positive results. Four of the studies looked at students with disabilities (Glang, Singer, Cooley, & Tish, 1991; McKenzie, Marchand-Martella, Martella, & Moore, 2004; Sommers, 1991; and Young, Baker, and Martin,1990), and all showed results in favor of Direct Instruction. 3) Przychodzin-Havis, A.M., Marchand-Martella, N., Martella, R.C., Miller, D.A., Warner, B.L., & Chapman, S. (2005). An Analysis of Corrective Reading Research. Journal of Direct Instruction, 5 (1), 37-65. This article reviewed 28 published studies of the Direct Instruction Corrective Reading program. Twenty-three of the studies used teachers to deliver instruction (4 in general education classrooms, 12 in special education, and 7 in alternative education settings). Five studies looked at implementations by paraprofessionals or peer instructions in general education or special education settings. Twenty six of the 28 studies (92.8%) found positive results. Only one study found greater gains with another intervention. Similar results 6 Reading Mastery and Students with Learning Disabilities appeared with different types of assessments (e.g. standardized tests or curriculum-based measures), in different settings, with different types of instructors, and with different research designs. 4) Schieffer, C., Marchand-Martella, N.E., Martella, R.C., Simonsen, F.L., & Waldron-Soler K.M. (2002). An analysis of the Reading Mastery Program: Effective components and research review. Journal of Direct Instruction, 2(2), 87-119. This article reviewed twenty-one studies of Reading Mastery that compared its use to that of another program. Four of these studies looked at general education populations, eight focused on general education remedial readers, and nine looked at special education students. Results in fourteen of these studies (67%) favored RM, other programs were favored in three (14%), and there were no differences in the remainder. Interestingly, they included the Herrera, et al. (1997), study noted above as one of the three studies that favored other programs. Meta-Analyses Meta-analyses summarize study results using quantitative methods and, usually, more restrictive criteria for selecting studies than the literature reviews described above. Meta- analyses typically use an “effect size” metric to summarize the results, averaging the effects over numerous studies. The effect size is simply the difference between two average scores divided by the common standard deviation. In other words, it describes a difference between two groups in standard deviation units (a standard metric). A number of years ago Tallmadge suggested that an effect of .25 standard deviations was educationally important (Tallmadge, 1977). All of the meta-analyses of Direct Instruction programs have found average effect sizes far surpassing this level. 1) Adams, G.L. & Engelmann, S. (1995). Research on Direct Instruction: 25 Years Beyond Distar. Seattle, WA: Educational Achievement Systems. Adams (who conducted the meta-analysis) limited his review to studies that met the following criteria, which are similar to those used in other analyses:  The study included a comparison group.  Pretest scores were available and showed no significant differences between the groups.  Descriptive statistics were provided for both groups (means, standard deviations, and sample sizes). 7 Reading Mastery and Students with Learning Disabilities  The study lasted for more than one session.  The DI intervention was not combined with another, incompatible program.  More than one subject was in each group no single-subject designs).  The study involved an entire program, not separate components of DI.  The study used Direct Instruction programs developed by Engelmann and associates (not the so-called “di”). Thirty-seven studies were found that met these criteria, and these studies included 173 comparisons of mean scores. Of these comparisons, 87.3% favored the Direct Instruction condition, and 12% favored the comparison program (50% would be expected by chance). Sixty-four percent of the comparisons were statistically significant in favor of DI, while 1.2% were statistically significant in favor of the comparison program (5% would be expected by chance). Of the 173 comparisons, 101 involved students in special education. The average effect size across these 101 studies was .76. When the effect sizes were aggregated to the level of each study, the average effect size across the 21 studies including special education students was .90. 2) Coughlin, C. (2011). Research on the Effectiveness of Direct Instruction Programs: An Updated Meta-Analysis, Poster presented at the Annual Meetings of the Association for Behavior Analysis International, May, 2011, NIFDI Technical Report 2011-4. Couglin limited this meta-analysis to randomized control studies. Twenty studies with 95 separate comparisons were identified. Nine of the studies involved students with disabilities. Over a third of the studies examined Reading Mastery or its precursor DISTAR. The average effect size across all the studies was .66. Effect sizes were similar for studies of general education students (.69) and those with special education students (.71). They were slightly smaller, on average, for reading and other programs (.56 and .54, respectively) than for language (.81) and mathematics (1.03). The effect sizes were regressed on methodological characteristics of the studies including sample size, whether or not fidelity was monitored, the length of the intervention, the type of test used to measure achievement, and the year the study was conducted. The only statistically significant relationship was with sample size, with smaller effects found for larger samples. 3) Hattie, John A.C. (2009). Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. London and New York: Routledge. Hattie used a meta-analysis technique to review the results of over 800 meta-analyses of research studies related to student achievement. He summarized the results of four meta- analyses that included DI, incorporating 304 studies, 597 effects and over 42,000 students. He found that the average effect size associated with DI was .59 and noted that the positive results were “similar for regular (d=.99) and special education and lower ability students 8 Reading Mastery and Students with Learning Disabilities (d=0.86), … [and] similar for the more low-level word-attack (d=.64) and also for high-level comprehension (d=.54)” (pp. 206-207). No other curricular program showed such consistently strong effects with students of different ability levels, of different ages, and with different subject matters. 4) White, W.A.T. (1988). A meta-analysis of the effects of Direct Instruction in special education, Education and Treatment of Children, 11 (4), 364-374. Twenty-five studies of the effectiveness of Direct Instruction programs with special education students were examined using quantitative meta-analysis techniques. The analysis was limited to studies with a comparison group of students assigned to a non-DI treatment prior to the start of the intervention. Fifty-three percent of the outcomes significantly favored DI (5% would be expected by chance). The average effect size across all comparisons was .84 standard deviations. Similar effects appeared with different conditions, ages, skill areas, and type of outcome measure. Scholarly Studies of Direct Instruction This section includes citations to and abstracts of a large number of individual research studies regarding the impact of the Direct Instruction curricula on students’ reading achievement. The first section focuses only on studies that examined the impact of Reading Mastery, or its precursor DISTAR, on the reading achievement of students with disabilities, thus paralleling most closely the focus of the WWC report. The second section provides summaries of a number of articles that focus on students with disabilities using other elements of the Direct Instruction body of curricula. The third section examines published data from school districts around the country that have used Reading Mastery with special education students, reporting the magnitude of change using effect sizes. Reading Mastery and Disabled Students This section only includes studies that report results of the use of Reading Mastery and outcomes with special education students. Frankhauser, M. A., Tso, M. E., & Martella, R. C. (2001). A comparison of curriculum- specified reading checkout timings and daily 1-minute timings on student performance in Reading Mastery. Journal of Direct Instruction, 1(2), 85-96. This study gathered data on words read per minute and number of errors on curriculum- specified reading checkouts; in addition, the number of readings needed to reach criteria on these checkouts with and without the addition of a daily timing was recorded. Four second- and third-grade students with reading disabilities who received reading instruction using Reading Mastery II or Fast Cycle (Engelmann & Bruner, 1995) participated in this study. 9 Reading Mastery and Students with Learning Disabilities Comparisons were made of each student’s progress across phases. Results showed no overall change in mean words read per minute, errors per minute, or number of timings to meet curriculum-specified criteria at reading checkouts for all students. These results suggest that the systematic practice and curriculum-specified reading checkouts within the Reading Mastery lessons provide the structure needed for students in need of reading remediation to make consistent progress in reading. Gersten, R. M., & Maggs, A. (1982). Teaching the general case to moderately retarded children: Evaluation of a five year project. Analysis and Intervention in Developmental Disabilities, 2, 329-343. This study examined the effects of a five-year program for pre-adolescent students with low- moderate intelligence levels that utilized the Direct Instruction programs, DISTAR Language and DISTAR Reading. Students were administered the Stanford-Binet as a pretest and posttest measure and student gains were compared to the standardization sample of the Stanford-Binet. Additionally, the Peabody Picture Vocabulary Test was administered as a posttest to examine concurrent validity. Results indicate that students who received the DISTAR program made statistically significant gains when compared to the norm sample. Kuder, S. J. (1990). Effectiveness of the DISTAR reading program for children with learning disabilities. Journal of Learning Disabilities, 23(1), 69-71 The reading achievement of students with learning disabilities (N=24) who received DISTAR instruction was compared to that of similar students using basal reader materials. No statistically significant differences were found between groups in overall reading scores, but the DISTAR group outperformed the basal reading group on the Word Attack subtest. O’Connor, R. E., Jenkins, J. R., Cole, K. N., & Mills, P. (1993). Two approaches to reading instruction with children with disabilities: Does program design make a difference? Exceptional Children, 59(4), 312-323. This study analyzed 2 approaches to reading instruction for children with disabilities in relation to reading achievement. The study compared Direct Instruction Reading Mastery I and II, and Addison Wesley’s Meet the Superkids and The Superkids’ Club. Results indicated no significant difference between the instructional programs at the end of treatment or one year later. The only notable difference was for children who made advanced progress, where the Direct Instruction group registered larger reading gains on the California Achievement Test (CAT) sound recognition subtest and the Peabody Individual Achievement Test (PIAT) reading recognition and spelling subtests. 10 Reading Mastery and Students with Learning Disabilities Riepl, J. H., Marchand-Martella, N. E., & Martella, R. C. (2008). The effects of Reading Mastery Plus on the beginning reading skills of students with intellectual and developmental disabilities. Journal of Direct Instruction, 8(1), 29-39. This study explored the effects of "Reading Mastery Plus" in grades K-2 at a Title 1 school with six students with intellectual and developmental disabilities (IDD). Two students in each of grades kindergarten, 1, and 2 participated. "Diagnostic Indicators of Basic Early Literacy Skills (DIBELS)" spring fluency probes were used to assess changes in students' beginning reading skills and risk status. Results indicated students made large overall gains on fluency measures. Additionally, there were several improvements in the "DIBELS" risk status categories. Somerville, D. E., & Leach, D. J. (1988). Direct or indirect instruction?: An evaluation of three types of intervention programme for assisting students with specific reading disabilities. Educational Research, 30(1), 46-53. Three intervention programs (psychomotor, self-esteem enhancement, and Direct Instruction) were conducted with 40 children who had reading difficulties. The direct instruction program resulted in the greatest gains. Post-intervention questionnaires completed by subjects, parents, and teachers indicated that perceived success differed significantly from measured success. Stein, C., & Goldman, J. (1980). Beginning reading instruction for children with minimal brain dysfunction. Journal of Learning Disabilities, 13(4), 52-55. This study examined the differential effect of two reading programs, DISTAR and Palo Alto Reading Program, on early elementary aged children diagnosed with minimal brain dysfunction. Results indicated that the DISTAR program was more effective in producing larger mean gains in reading achievement, as measured by the Peabody Individual Achievement Test (PIAT). Stockard, J. (2008). Reading Achievement in a Direct Instruction School and a “Three Tier” Curriculum School, NIFDI Technical Report 2008-5. Eugene, Oregon: National Institute for Direct Instruction This report examines data from two schools within the same Oregon school district. One school adopted the Reading Mastery Direct Instruction program as the core reading curriculum for all primary children, while the other used a “three tiered” model, occasionally employing DI for students that teachers felt would benefit from the instruction. There were 11 Reading Mastery and Students with Learning Disabilities almost equal numbers of students from each school and there were no significant differences between the schools in the students’ eligibility for free or reduced lunch, their racial-ethnic characteristics, or their special education designation. A slightly modified version of DIBELS was used to assess achievement gains over the time span of the study. Data were available for 2 cohorts of students who were in the schools from kindergarten through third grade. Students in the Direct Instruction School had statistically significant higher gains in Oral Reading Fluency (ORF) from first through third grade than students in the Control School. These differences were especially marked for students in special education. By the end of third grade, the effect size for ORF for the total group of students was .42, while the effect size for special education students was .73. By the end of the study the reading scores of special education students in the DI school were essentially equivalent to those of the general education students in the non-DI school. The odds of a child in the DI school having levels of oral reading fluency that would indicate a possibility of academic failure were less than half that of a child in the control school. Similar results occurred with the measures of onset recognition fluency, phonemic segmentation fluency, and nonsense word fluency, consistently favoring the Direct Instruction students. Zayac, R. (2008). Direct Instruction reading: Effects of the Reading Mastery Plus – Level K curriculum on preschool children with developmental delays. Dissertation from Auburn University. This study investigated the effects of the Reading Mastery Plus - Level K program on preschool children with developmental delays. Findings indicated that preschool-aged children both with and without developmental delays were able to acquire beginning reading skills. While the research design inhibited the identification of any functional relationships between the Reading Mastery Plus - Level K program and the participants' reading gains, the data showed that young children with developmental delays can acquire skills that are necessary to begin reading. The author suggests that this is an important finding, especially considering the fact that the number of children with autism spectrum disorder is increasing. Studies of other DI Curricula and Students with Disabilities While Reading Mastery is typically used with students in grades K-3, Corrective Reading is the program prescribed for older students. Thus, many of the studies of Direct Instruction curricula with older special education students involve Corrective Reading. Some also involve exposure to other programs. This section includes citations to and abstracts of a number of studies involving special education students who were exposed to Direct Instruction curricula other than Reading Mastery. Because all of the Direct Instruction programs are built using the same techniques and criteria for effective instruction, results 12 Reading Mastery and Students with Learning Disabilities from all of these studies are important to consider. As with the set of works reviewed above, results were overwhelmingly in favor of the Direct Instruction curricula. Anderson, D. M. (2002). Using Reasoning and Writing to teach writing skills to students with learning disabilities and behavioral disorders. Journal of Direct Instruction, 2(1), 49-55. This study examined the effectiveness of the Direct Instruction writing program, Reasoning and Writing, with 10 fourth and fifth grade students with learning disabilities and/or behavioral disorders. Instruction with Reasoning and Writing occurred for six weeks, and pretest and posttest scores on the Test of Written Language-2 (TOWL-2) were used to indicate program effects. As a group, results indicated that students made both statistically and educationally significant gains overall on the TOWL-2. Benner, G. J. (2007). The relative impact of remedial reading instruction on the basic reading skills of students with emotional disturbance and learning disabilities. Journal of Direct Instruction, 7(1), 1-15. This study investigated the effects of a Direct Instruction reading program, Corrective Reading, on the reading achievement of elementary and middle school students with emotional disturbance (ED) and learning disabilities (LD). Comparison students receiving non-DI instruction were selected based on demographics. Results indicated that students in the Corrective Reading program made significantly greater gains than comparison students; additionally, students with ED were more responsive to Corrective Reading than students with LD. Benner, G. J., Kinder, D., Beaudoin, K. M., Stein, M., & Hirschmann, K. (2005). The effects of the “Corrective Reading Decoding” program on the basic reading skills and social adjustment of students with high-incidence disabilities. Journal of Direct Instruction, 5(1), 67-80. The purpose of this study was to examine the effects of the "Corrective Reading Decoding B1" program on the basic reading skills, social adjustment, and treatment responsiveness of elementary and middle school students with high-incidence disabilities (N = 51). Students were provided an average of 3 40-45 minute lessons per week over the course of nearly 4 months. Statistically and educationally significant improvements were found between students who received "Corrective Reading Decoding Level B1" (n = 28) and those in the comparison condition (n = 23) on measures of basic reading skills and social adjustment. A large percentage of students who experienced below average basic reading skills (i.e., nonresponders) at pretest performed in the average range at posttest (i.e., responders). 13 Reading Mastery and Students with Learning Disabilities Booth, A., Hewitt, D., Jenkins, W., & Maggs, A. (1979). Making retarded children literate: A five year study. The Australian Journal of Mental Retardation, 5(7), 257-260. The Distar model of reading and language instruction was shown to be effective in developing the reading and language skills of 12 retarded Australian children beyond the usual expectations for such children, and in fostering mastery of most of the basic literacy skills. A battery of measures (e.g., Peabody Picture Vocabulary Test) were administered to 12 subjects. At the beginning of the study they averaged 10 years of age and had measured IQs ranging from 35 to 55. Instruction occurred with the Distar language program, and involved all levels of the Distar language and reading programs. Results indicated that students demonstrated mastery of most of the basic literacy skills. Bracey, S., Maggs, A., & Morath, P. (1975). Teaching arithmetic skills to moderately mentally retarded children using direct verbal instruction: Counting and symbol identification. Australian Journal of Mental Retardation, 3, 200-204. This study analyzed the impact of Direct Instruction’s arithmetic program and behavioral techniques to teach arithmetic skills to a group of 6 institutionalized moderately mentally retarded children (IQs approximately 35-50). The aim of the study was to determine whether the subjects made significant gains in 4 types of arithmetical skills: object counting, making lines from numerals, the meaning of plus, and increment additions. Results revealed significant improvement ( p < .05) in each of the skill areas. Campbell, M. (1981). A study of Corrective Reading as an effective and appropriate program for reading-disabled, learning-handicapped secondary students. Report presented to Faculty of School of Education. San Diego, CA: San Diego State University. The study showed that the Corrective Reading Program was successful with upper elementary children identified as having mild learning disabilities and junior high school underachievers. Carnine, D., Engelmann, S., Hofmeister, A., & Kelly, B. (1987). Effects of instructional design variables on vocabulary acquisition of LD students: A study of computer-assisted instruction. Journal of Learning Disabilities, 20, 206-213. This study compared two computer-assisted instructional vocabulary programs used with 25 learning disabled high school students. Results indicated that the Direct Instruction program, which utilized smaller teaching sets and cumulative review exercises, was more 14 Reading Mastery and Students with Learning Disabilities effective in helping students achieve mastery than the program using a large teaching set and no cumulative reviews. Carnine, D. (1989). Teaching complex content to learning disabled students: The role of technology. Exceptional Children, 55, 524-533. The article documents the use of Direct Instruction procedures with technological adjuncts to teach learning disabled students. Examples are given of effective technology use to teach earth science/chemistry, fractions, health, reasoning skills, and vocabulary, and findings from several studies are briefly reviewed. The article documents how such a comprehensive intervention reduces performance differences between students with learning disabilities and their peers, while using technology to minimize, or even reduce, the demands placed on the teacher. Clark, D. S. (2001). Components of effective reading instruction for reading disabled students: An evaluation of a program combining code- and strategy-instruction. Unpublished doctoral dissertation, University of Toronto. This study randomly assigned 44 disabled readers, seven to ten years of age to an experimental treatment intervention, Phono-Graphix, or to the Direct Instruction program, Corrective Reading. Both programs placed emphasis on decontextualized word identification training with instructional focus at the level of the phoneme. Each group received a mean of 10.5 hours of formal, one-to-two instruction and an equivalent number of hours of assigned homework over a 7.5-week period over the summer. No reliable pretest-posttest gains were demonstrated for either program on any of the standardized measures of word identification. A large treatment effect was found for the Phono-Graphix group, however, on an experimental measure of word identification. Further analysis revealed that this treatment effect was attributable to large, reliable improvement in reading orthographically irregular words. Cole, K. N., Mills, P. E., & Dale, P. S. (1989). Comparison of the effects of academic and cognitive curricula for young handicapped children one and two years post program. Topics in Early Childhood Special Education, 9(3), 110-127. Follow-up testing of students who had participated when ages 3-7 in two early intervention programs, Direct Instruction and Mediated Learning, was conducted at 1 year and 2 years post-intervention. Children from both programs generally maintained or increased cognitive and academic skills following intervention. Trends suggesting differential effects for subscales of the Peabody Individual Achievement Test and the Stanford-Binet (fourth 15 Reading Mastery and Students with Learning Disabilities edition) were noted in the first-year follow-up. The differential effects decreased by the second year post-intervention, although the overall benefits of early intervention appeared to continue. Dale, P. S. & Cole, K. N. (1988). Comparison of academic and cognitive programs for young handicapped children. Exceptional Children, 54(5), 439-447. Two highly contrasting models of preschool education for mildly handicapped children were compared. Direct Instruction led to greater gains on the Test of Early Language Development and the Basic Language Concepts test. Mediated Learning led to greater gains on the McCarthy Verbal and Memory scales and Mean Length of Utterance measure. Darch, C. & Kameenui, E. J. (1987). Teaching LD students critical reading skills: A systematic replication. Learning Disability Quarterly, 10, 82-91. Contrasted 2 approaches to teaching learning disabled (LD) students 3 critical reading skills, using 25 LD students in Grades 4–6. Subjects were randomly assigned to treatment groups using Direct Instruction or discussion with a workbook. The former featured specific rules and strategies to detect instances of faulty arguments, whereas in the latter group subject involvement was encouraged through discussions on how to use critical reading skills. Both groups received 40-minute lessons for 12 consecutive school days. Three dependent measures designed to assess students’ knowledge of the three critical reading skills were administered. Subjects in the direct instruction group were found to significantly outperform their counterparts in the discussion/workbook group on each measure. Darch, C. & Simpson, R. G. (1990). Effectiveness of visual imagery versus rule-based strategies in teaching spelling to learning disabled students. Research in Rural Education, 7(1), 61-70. Among 28 upper elementary learning-disabled students in a summer remedial program, those that were taught spelling with explicit rule-based, Direct Instruction, strategies out- performed students presented with a visual imagery mnemonic on unit tests, a posttest, and a standardized spelling test. Darch, C., Eaves, R. C., Crowe, D. A., Simmons, K., & Conniff, A. (2006). Teaching spelling to students with learning disabilities: A comparison of rule-based strategies versus traditional instruction. Journal of Direct Instruction, 6(1), 1-16. 16 Reading Mastery and Students with Learning Disabilities This study compared the effects of the Direct Instruction program, Spelling Mastery, and traditional spelling instruction on spelling achievement of special education elementary students receiving services for a learning disability. Unit tests, a standardized spelling test, a sentence-writing test, a transfer test, and a maintenance test were administered to all students. Instruction occurred daily, for 30 minutes, for four consecutive weeks. Four different word types (i.e., regular, morphological, spelling rule, and irregular) were introduced as instruction progressed. After receiving instruction in one of the instructional groups, the students were compared on scores from unit tests, a standardized test, a sentence-writing test, a transfer test, and a maintenance test. Results indicate that students in the Spelling Mastery group significantly outperformed comparison students on words of high predictability (regular, morphological, and spelling-rule words). Darch, C., & Gersten, R. (1985). Effects of teacher presentation rate and praise on LD students’ oral reading performance. British Journal of Educational Psychology, 55, 295- 303. Four learning disabled students were taught beginning reading with a Direct Instruction program. The teacher systematically varied the rate of instructional presentation (rapid pace vs. slow pace) and frequency of praising (praise vs. no praise). A modified reversal design was used to evaluate both the isolated and interactive effects of the individual instructional components. Experimentation lasted 25 school days. The results indicated that increases in teacher presentation rate and increased use of praise led to improvement in levels of on- task behavior and correct student responding. Each of the variables in isolation had an effect. Praise appeared to be more powerful than presentation rate. The combination of the two variables had the most powerful effect. These findings extend the research of earlier studies to learning disabled students and confirm other studies regarding the presentation style used in Direct Instruction programs. Darch, C. (1990). Comprehension instruction for high school learning disabled students. Research in Rural Education 5, 43-49. This article presents an instructional approach for helping learning disabled high school students comprehend content area instruction in regular classrooms that combines the use of Direct Instruction and advanced organizers. It also identifies and discusses general methods teachers can use to teach learning disabled students successfully in regular classrooms. Flores, M. M., & Ganz, J. B. (2007). Effectiveness of Direct Instruction for teaching statement inference, use of facts, and analogies to students with developmental 17 Reading Mastery and Students with Learning Disabilities disabilities and reading delays. Focus on Autism &Other Developmental Disabilities, 22(4), 244-251. This study investigated the effects of a Direct Instruction (DI) reading comprehension program implemented with students who had developmental disabilities, including autism spectrum disorders (ASD) and reading delays. There is little research in the area of reading comprehension for students with ASD and no research as to the effectiveness of reading comprehension DI. A multiple-probe-across-behaviors design was employed. A functional relationship between Direct Instruction and reading comprehension skills and behaviors was demonstrated across all behavioral conditions and across students. Results and their implications are discussed. Flores, M., Shippen, M., Alberto, P. & Crowe, L. (2004). Teaching letter-sound correspondence to students with moderate intellectual disabilities. Journal of Direct Instruction, 4(2), 173-188 This study examined the efficacy of systematic and explicit instruction in phonic decoding for 6 elementary students with moderate intellectual disabilities. A multiple probe across behaviors with embedded changing conditions design was employed in order to analyze the effect of Direct Instruction on teaching the following skills with regard to consonant-vowel- consonant (CVC) words: letter-sound identification, continuous sound blending, sounding out, and decoding (sounding out then telescoping). Results indicate that the majority of students demonstrated an understanding of letter-sound correspondence and decoding basics following intervention. Franklin, M. E., Little, E., & Teska, J.A. (1987). Effective teaching strategies used with the mildly handicapped in the mainstream. Focus on Exceptional Children, 20, 7-11. The article explores evidence of the benefits and limitations of four instructional strategies (Direct Instruction, classroom management, cooperative grouping, and metacognition) on the successful integration of mildly handicapped children in the mainstream. Ganz, J. & Flores, M. (2009). The effectiveness of Direct Instruction for teaching language to children with autism spectrum disorder. Indentifying materials. Journal of Autism and Developmental Disorders, 39, 75-83. This study examined effects of the Direct Instruction program, Language for Leaning, on improving language ability of elementary students with autism spectrum disorders. A single- subject changing criterion design was employed. Probes were given 2-3 times a week to measure students’ ability in identifying items made from different materials. Percentage of 18 Reading Mastery and Students with Learning Disabilities non-overlapping data was calculated to supplement visual analysis when determining the effectiveness of the intervention. Results indicate that a functional relation was demonstrated between the implementation of Language for Learning and an increase in correctly naming presented items; additionally, the percentage of non-overlapping data was at least 90% for all participants, suggesting that the intervention was effective at increasing the number of correct responses over time. Gersten, R. (1985). Direct Instruction with special education students: A review of evaluation research. Journal of Special Education, 19, 41-50. Studies evaluating the effectiveness of direct instruction curricula and teaching procedures are reviewed and, in some instances, critiqued. The six studies indicate that direct instruction tends to produce higher academic gains for handicapped children than traditional approaches. They also suggest that some of the more subtle principles of direct instruction—such as insistence on complete (rather than partial) mastery of each step in the learning process—are important. The author contends that future research should more carefully measure program implementation and begin to look more precisely at specific instructional variables. Suggestions for appropriate research designs are presented. Hempenstall, K. (1997). The effects on the phonological processing skills of disabled readers of participating Direct Instruction reading programs. Australian Digital Theses Program, RMIT University Library. This thesis examines the effects of two Direct Instruction reading programs on the phonological processes of students with teacher-identified reading problems in nine northern and western Melbourne primary schools. In the first study students (mean age 9.7 years years) were assigned to the treatment condition or to wait-list comparison groups. Based on the results of a program placement test of rate and accuracy, students were assigned to one of two entry points into the Corrective Reading program (A, B1). The students in the intervention group received 60-65 lessons (in groups of five to ten students) from teachers at their schools, or, for some students, at a resource centre for surrounding schools. An additional study, with younger (mean age 8.8 years) less advanced readers involved a similar design and teaching approach. The program, Teach Your Child to Read in 100 Easy Lessons, was presented to thirteen students in two settings. When compared with a similar cohort of wait-list students, the students in each program made statistically significant and educationally important gains in such phonologically-based processes as word attack, phonemic awareness, and spelling; and, statistically significant gains, of at least moderate effect size, in phonological recoding in lexical access and phonological recoding in working memory. 19 Reading Mastery and Students with Learning Disabilities Hicks, S. C. (2011). Effects of direct instruction on the use of and response to prepositions by students with an intellectual disability. Unpublished doctoral dissertation, University of North Carolina at Charlotte. Hicks, S. C., Stevenson, K. M., Wood, C. L., Cooke, N. L., & Mims, P. (2011). Effects of direct instruction on the acquisition of prepositions by students with intellectual disabilities. Journal of Applied Behavior Analysis, 44, 675-679 Students with an intellectual disability often struggle with significant language delays or impairments. Although this population of students can acquire language skills, they often require methods of explicit instruction of language skills to do so. Direct Instruction (DI), a system of explicit and systematic instruction, could be one of these methods. The purpose of this study was to investigate the effects of DI on the use of and response to prepositions by three elementary school students with an intellectual disability. A multiple baseline design across prepositions was used in this study with replication across students. The researcher used DI to model examples and nonexamples (i.e., "This is above," "This is not above.") of three prepositions (e.g., above, behind, beneath) to the students. In addition to the instructional sessions, students participated in three generalization activities. Results of this study showed a functional relationship between Direct Instruction and students' use of and response to prepositions. Students demonstrated the ability to use and respond to prepositions consistently after receiving DI on each of the three target prepositions. Furthermore, all three students maintained the skill up to 56 days from instruction on each of the prepositions. These findings are important to this population of students because of the need for explicit and systematic instruction of language skills; it has been demonstrated that DI is an effective instructional tool in teaching these skills in an efficient and effective way. Horner, R. H., & Albin, R. W. (1988). Research on general-case procedures for learners with severe disabilities. Education and Treatment of Children, 11, 175-188 This paper presents an overview of the Direct Instruction principles and guidelines that define general-case programing and then reviews recent research findings related to general-case instruction with learners with severe disabilities. Infantino, J. & Hempenstall, K. (2006). Effects of a decoding program on a child with Autism Spectrum Disorder. Australasian Journal of Special Education, 30(2), 126-144. This case study examined the effects of a parent-presented Direct Instruction decoding program on the reading and language skills of a child with high functioning Autism Spectrum 20 Reading Mastery and Students with Learning Disabilities Disorder. Following the 23 hour intervention, reading comprehension, listening comprehension and fluency skills improved to grade level, whilst statistically significant improvements were also noted in receptive language skills. There were no significant changes in phonological and decoding skills for which various possibilities are explored, including the student's prior double deficit in phonological skills. The findings are consistent with research suggesting that increased intervention intensity, along with greater emphasis on phonological skills may be necessary to advance the decoding skills of children displaying a double deficit. The reading and language outcomes were sufficiently promising to warrant further studies employing methodologically sound group designs with this population. Kelly, B., Carnine, D., Gersten, R., & Grossen, B. (1986). Effectiveness of videodisc instruction in teaching fractions to learning-disabled and remedial high school students. Journal of Special Education Technology, 8, 5-9. This study compared the effectiveness of an interactive videodisc curriculum incorporating principles of instructional design with a traditional basal curriculum to teach basic fraction skills to 28 high school students, including 17 mildly handicapped students. The videodisc curriculum's superiority at posttest and at two-week maintenance were attributed to better instructional design. Kelly, B., Gersten, R., & Carnine, D. (1990). Student error patterns as a function of curriculum design: Teaching fractions to remedial high school students and high school students with learning disabilities. Journal of Learning Disabilities, 23, 23-29. This study compared the relative effectiveness of a Direct Instruction curriculum, which incorporated three empirically derived principles of curriculum design, with a basal approach in teaching basic fractions concepts to students with learning disabilities and other low performing students in high school remedial math classes. Results indicated that, although both programs were reasonably successful in teaching the material, the Direct Instruction was significantly more effective. Mean scores on a curriculum-referenced test were 96.5% for that group and 82.3% for the basal group. Secondary analyses of item clusters revealed that areas of weakness in the performance of the basal group could be directly linked to hypothesized flaws in its curriculum design. Kitz, W. & Thorpe, H. (1995). A comparison of the effectiveness of videodisc and traditional algebra instruction for college-age students with learning disabilities. Remedial & Special Education, 16(5), 295-307. 21 Reading Mastery and Students with Learning Disabilities This study compared the effectiveness of two algebra programs, the Direct Instruction-based Mastering Equations, Roots, and Exponents delivered via videodisc and a traditional text- based program. 26 college-age students with learning disabilities were randomly assigned to receive instruction with either the videodisc or traditional program. All participants were administered assessments from both programs. Results indicate that students who were in the videodisc group significantly outperformed students receiving traditional instruction not only on the Mastering Equations, Roots, and Exponents post-test, but also on the Project Success Algebra Placement Test (tied to the text-based program), despite the control group’s higher pre-test scores on this test and daily practice with the material assessed. Additionally, students in the videodisc group received significantly higher grades in their first algebra class in the semester following the completion of instruction. Kroesbergen, E. & Van Luit, J. (2003). Mathematics interventions for children with special educational needs: A meta-analysis. Remedial and Special Education, 24(2), 97-114. This article presents the results of a meta-analysis of 58 studies of mathematics interventions for elementary students with special needs. Interventions in three different domains were selected: preparatory mathematics, basic skills, and problem-solving strategies. The majority of the included studies described interventions in the domain of basic skills. In general, these interventions were also the most effective. Furthermore, a few specific characteristics were found to influence the outcomes of the studies. In addition to the duration of the intervention, the particular method of intervention proved important: Direct instruction and self-instruction were found to be more effective than mediated instruction. Interventions involving the use of computer-assisted instruction and peer tutoring showed smaller effects than interventions not including these supports. Leiss, R. H., & Proger, B. B. (1974). Language training for trainable mentally retarded: Annual project report – second year – ESEA Title III. ERIC 097 789, 83p. During the 1973-74 school year, 230 trainable mentally retarded (TMR) children (ages 7 to 14 years) were exposed to one of two language training conditions: Distar or Peabody. A population of 116 students who continued from the first year of the project and 114 new entries were assigned in as random a fashion as possible to either Distar or Peabody. Subjects were divided into low IQ (21-43) and high IQ (44-53) groups. Sex was built into the design, as was pretest-posttest and new entries versus continuees. Thus, a five-factor, 2 x 2 x 2 x 2 x 2 repeated-measures design was subjected to analysis of variance for each of three basic criteria: Peabody Picture Vocabulary Test, Illinois Test of Psycholinguistic Abilities, and Mecham Verbal Language Development Scale. Seven children were selected randomly from each of the 16 between-factor cells to yield a total of 112 children. Longitudinal analyses were also conducted on just the continuing students with pre- and posttest data from the 22 Reading Mastery and Students with Learning Disabilities three basic measures from both years of the project to yield a treatments-by-IQ-by-Sex-by- Measures (2 x 2 x 2 x 4) design. While no significant differences emerged for the high-IQ children, the low-IQ children were aided more by Distar than by Peabody. In the 5-way designs, gain in the total sample was not marked. However, when one considers only the continuing students, significant gains in language functioning did occur. Leiss, R. H., & Proger, B. B. (1977). Language training for trainable mentally retarded children: ITPA, Peabody, and DISTAR techniques. ERIC 140 527, 39p. To determine the most effective language training activities for trainable mentally retarded (TMR) children, the variables of degree of previous language training, IQ, Peabody language treatment program versus Distar language treatment program, pretest versus posttest, and sex were examined with 122 TMR subjects (7 to 14 years old). Results of the Peabody Picture Vocabulary Test, the Illinois Test of Psycholinguistic Abilities, and the Mechann Verbal Language Development Scale indicated that subjects were significantly better aided by the Distar program than by the Peabody program, and that only the children previously exposed to language training showed any significant growth in language. A second study examined in depth the nature of previous language experience on Distar performance. Maggs, A., & Morath, P. (1976). Effects of direct verbal instruction on intellectual development of institutionalized moderately retarded children: A 2-year study. The Journal of Special Education, 10(4), 357-364. This study compared the effects of two programs on the language gains of students with moderate to severe mental retardation across two years of implementation: DISTAR Language and the Peabody Language Kit. Results indicate that students receiving instruction with DISTAR Language achieved significantly greater gains on all assessment measures than the control group. Additionally, DISTAR students progressed at approximately the same rate as typically developing children, gaining 22.5 mental age months across 24 months, whereas control students only gained 7.5 mental age months in the same period of time. Maggs, A., & Morath, P. The effects of direct verbal instruction on intellectual development of institutionalized moderately retarded children: A two-year study. Journal of Special Education. 1976, 10, 357-364 This study examined the intellectual development of 28 moderately retarded children (IQ's approximately 30-55), using Distar Language 1 Kit, together with positive reinforcement, modeling, and shaping techniques, in a two year intervention project. Results revealed 23 Reading Mastery and Students with Learning Disabilities significant differences in favor of the experimental treatment conditions on all dependent variables (i.e., the Basic Concept Inventory, Reynell Verbal Comprehension, Stanford-Binet (L-M) Intelligence, Piaget's Class Inclusion, Piaget's Seriation, and Bruner's Matrix) related to the subjects’ performance. Subjects seemed to be able to maintain approximately a "normal" rate of intellectual development over the 2-yr period. Marston, D., Deno, S. L., Kim, D., Diment, K., & Rogers, D. (1995). Comparison of reading intervention approaches for students with mild disabilities. Exceptional Children, 62(1), 20-37. This study examined effects of six different reading instructional strategies for students with mild disabilities: peer tutoring, reciprocal teaching, computer-aided instruction, effective teaching principles, Direct Instruction using Corrective Reading, and direct instruction principles applied to a basal reading series. Additionally, a comparison group was included. Student achievement and instructional ecology were assessed with curriculum-based measures and two observation systems. Results indicate that, after 10 weeks of implementation, students receiving computer-aided instruction and instruction that applied direct instruction principles to a basal reading series made significantly greater gains in reading achievement than the comparison group. Owens, S. H., Fredrick, L. D., & Shippen, M. E. (2004). Training a paraprofessional to implement Spelling Mastery and examining its effectiveness for students with learning disabilities. Journal of Direct Instruction, 4(2), 153-172. This study examined: 1) the effectiveness and efficiency of training a paraprofessional to implement Spelling Mastery, 2) the effectiveness of Spelling Mastery with students with learning disabilities, and 3) the maintenance and generalizability of spelling skills. Results indicate that students were able to learn, maintain, and generalize spelling skills following instruction with Spelling Mastery; additionally, paraprofessionals were able to effectively implement Spelling Mastery. Polloway, E. A., Epstein, M. H., Polloway, C. H., Patton, J. R., & Ball, D. W. (1986). Corrective Reading program: An analysis of effectiveness with learning disabled and mentally retarded students. Remedial and Special Education 7(4), 41-47. The Corrective Reading Program (CRP) was used with a group of learning disabled (LD) and educable mentally retarded (EMR) adolescents with data collected on the achievement of these students in the domains of reading recognition and comprehension. When compared to reading progress made in prior years, both groups showed significantly greater 24 Reading Mastery and Students with Learning Disabilities improvement. LD students experienced larger achievement gains than EMR students in both recognition and comprehension with differences in the former domain being statistically significant. Proger, B. B & Leiss, R. H. (1976). Language training for TMR children: Third-year results and comparison with first two years: The Peabody, DISTAR, and ITPA programs. Norristown, PA: Montgomery County Intermediate Unit 23 Special Education Center. Effects of the different components of the Distar language program were tested on 48 trainable mentally retarded (TMR) children. Three designs were used comparing IQ, sex, continuation status, selected standardized test results, and replicates. Results indicated that the degree of previous formal language training (continuation status) did not have any effect on current functioning. The Distar program produced steady, acceptable progress of the TMR children throughout the school year. In addition, the Distar program proved more effective when compared with the Peabody Language Development program received by 40 TMR children. Scruggs, T. E., Mastropieri, M., & Levin, J. R. (1985). Vocabulary acquisition by mentally retarded students under direct and mnemonic instruction. American Journal of Mental Deficiency, 89, 546-551. Twenty EMR junior-high-school students learned the definitions of unfamiliar English vocabulary words under either direct instruction or mnemonic instruction in a crossover design. When in the mnemonic-instruction condition, students remembered 50% more vocabulary definitions than when they were in the direct-instruction condition. Only three of the 20 students failed to exhibit superior performance when in the mnemonic instruction condition. Supplementary analysis of the response data revealed distinctly different error patterns in the two instructional conditions. Shippen, M. E., Houchins, D. E., Calhoon, M. B., Furlow, C. F., & Sartor, D. L. (2006). The effects of comprehensive school reform models in reading for urban middle school students with disabilities. Remedial and Special Education, 27(6), 322-328. The authors compared the effects of two CSR models ("Success for All" and "Direct Instruction") in reading for urban middle school students with disabilities who were performing 2 or more years below grade level in reading. The results indicated that students with disabilities showed little or no reading skill gain from either comprehensive school reform model and remained markedly behind. 25 Reading Mastery and Students with Learning Disabilities Somerville, D. E., & Leach, D. J. (1988). Direct or indirect instruction?: An evaluation of three types of intervention programme for assisting students with specific reading disabilities. Educational Research, 30(1), 46-53. Three intervention programs (psychomotor, self-esteem enhancement, and Direct Instruction) were conducted with 40 children who had reading difficulties. The Direct Instruction program resulted in the greatest gains. Post-intervention questionnaires completed by subjects, parents, and teachers indicated that perceived success differed significantly from measured success. Stephens, M. A. (1993). Developing and implementing a curriculum and instructional program to improve reading achievement of middle-grade students with learning disabilities in a rural school district. ERIC 359 492. A program was developed and implemented to improve the reading achievement of 56 learning-disabled (LD) students in grades five through eight in a rural school district in South Carolina. Only 12% of the students were meeting the state standard in reading. Baseline test data indicated that reading achievement was from two to six grade levels below grade placement. Problems identified were absence of written basic skills curriculum aligned with the tests used to measure achievement; lack of consistent, effective instruction in the basic skills areas; and the need for more parental support and involvement in the education of students with special needs. Solution strategies for improving reading achievement of LD students included the development of functional basic skill curriculum guides, training for teachers in the use of direct instruction techniques, field testing the curriculum guide, inclusion of the basic skills objectives in the individualized education programs of the students, implementation of a direct-instruction model in the teaching of reading, use of the Corrective Reading Program, and involvement of parents. As a result of intervention strategies, 25% of the targeted students met the state standard on basic skill tests. Over 55% of the targeted students gained two or more grade levels in reading. The discrepancy between the students' estimated ability and reading achievement was reduced by 41%. Parent involvement increased from 10% to 75%. Viel, K.A. (2008). The Effects of Direct Instruction in Writing on English Speakers and English Language Learners with Disabilities, Unpublished PhD Dissertation, Georgia State University. Viel-Ruma, K., Houchins, D., Jolivette, K., Fredrick, L., & Gama, R. (2010). Direct Instruction in written expression: The effects on English speakers and English Language Learners with disabilities. Learning Disabilities Research & Practice, 25(2), 97-108. 26 Reading Mastery and Students with Learning Disabilities Students with disabilities often struggle with writing tasks. In order to improve the written expression performance of high school students with deficits in written expression, a Direct Instruction writing program was implemented. The participants were six high school students in programs for individuals with learning disabilities. Three of the six students were served in programs for students who are English Language Learners. Using a multiple-probe across- participants design, the effect of the writing program was examined. The intervention was implemented over a 5-week period with maintenance checks conducted 2 and 4 weeks after the termination of instruction. Results were variable, but there appeared to be a positive trend in student writing performance as measured by correct word sequence, length of text, and the TOWL-3. Implications for practice and future directions are also provided. Walker, B., Shippen, M. E., Alberto, P., Houchins, D. E., & Cihak, D. F. (2006). Using the Expressive Writing program to improve the writing skills of high school students with learning disabilities. Journal of Direct Instruction, 6(1), 35-47. The complex nature of written expression presents difficulty for many students, particularly those with learning disabilities (LD). The purpose of this study was to investigate the effects of the DI writing program, Expressive Writing, for high school students with learning disabilities using a single-subject design methodology. Results indicated that the Expressive Writing program improved the writing skills of the students in this study. Students also were able to generalize and maintain the writing skills learned during intervention. Young, M., Baker, J., & Martin, M. (1990). Teaching basic number skills to students with a moderate intellectual disability. Education and Training in Mental Retardation, 25, 83- 93. Performance of 5 students (ages 8 and 10) with moderate intellectual disability was compared as they received the DISTAR Arithmetic instructional intervention and a Discrimination Learning Theory (DLT) intervention not requiring a fast-paced verbal student response. Academic engagement and skill mastery were greater during the DLT intervention. Analyzing Case-Study Data from Schools SRA, the publisher of Reading Mastery and other Direct Instruction programs, routinely posts information from schools regarding changes in standardized test scores after they began to use Reading Mastery and other Direct Instruction programs. This section describes results given by SRA from four different districts regarding changes in reading test scores of special education students after using Reading Mastery, sometimes in conjunction with Corrective Reading for students in older grades. All reports are available from the SRA research report website (http://mheresearch.com/product_info.php?segID=4&subID=21&proID=38&catID=7). 27 Reading Mastery and Students with Learning Disabilities In a recent article, Stockard (2011) explained how the comparisons in these reports conform to the tenets of a cohort control group design as described by the classical literature on research design (Campbell and Stanley,1963; Cook & Campbell, 1979; Shadish, et al, 2002). The cohort control group design is an internally valid design especially appropriate for settings such as schools and provides good external validity. The article also showed how the data presented in the SRA reports could be used to calculate effect sizes, similar to those used in the meta-analyses described in an earlier section. Recall that Tallmadge (1977) suggested that effect sizes of .25 or larger be considered educationally important. All of the effect sizes calculated from the data in these reports and reported below far surpassed this level. (See Stockard (2011) for similar analyses for a large number of other districts with data regarding general education and total student populations.) SRA/McGraw-Hill. (2006). Exceptional education and regular education students excel with Direct Instruction. The Iredell Statesville School District in North Carolina has approximately 21,000 students in grades pre-K – 12. Almost three-fourths are Caucasian, 18% African American. Thirty-five percent qualify for free or reduced lunch. In fall 2003, teachers in the special education department adopted SRA/McGraw-Hill’s Direct Instruction programs Reading Mastery and Corrective Reading for intervention in Grades K–12 because they felt students with disabilities were falling further and further behind their regular education peers. The students responded so well that the programs were implemented with at-risk students in regular education the following school year (2004–2005). They too responded well, and reading scores increased for both groups. The percentage of special education students attaining adequate yearly progress changed from 43 percent in 2002-03, before implementation of RM and CR, to 66% in the 2005-06 academic year. This represents an effect size of .46. SRA/McGraw-Hill. (2006). Reading proficiency more than doubles among Putnam County special education students. The Putnam County School District in Cookeville, Tennessee serves about 10,000 students. The student population is 91% Caucasian, and 47% qualify for free or reduced lunch. Reading proficiency among special education students in Grades 3-8 jumped from 37% in 2003, before implementing Reading Mastery and Corrective Reading to 85% in the 2005- 2006 school year, corresponding to an effect size of .98. School officials reported that socioeconomic status had no influence on reading proficiency, reporting that 100% of special education students in one of the poorest schools in the district read proficiently in 2005. A school district researcher found statistical significance in spelling and 28 Reading Mastery and Students with Learning Disabilities comprehension abilities among students exposed to Corrective Reading. In addition, greater mean gains were achieved by those same students in all decoding, spelling, and comprehension tests. SRA/McGraw-Hill. (2008). Direct Instruction reduces special education referrals in Louisiana school district by half. The Rapides Parish school district in Louisiana serves approximately 24,000 students, about half of whom are Caucausian and half are African American. Sixty percent classify for free or reduced-price lunch. At the start of the 2006-07 the district introduced Reading Mastery and Corrective Reading in three targeted schools due to concerns with high numbers of students being referred for special education. Once this early intervention began the number of referrals for special education evaluations dropped by 50% and the number of students who qualified for special education after referral dropped from 100% to 42%, an effect size of 1.28. SRA/McGraw-Hill. (2008). Special education students at California elementary school achieve AYP with Direct Instruction. Virginia Primrose School in Fontana, California, serves slightly more than 800 students in grades Pre-K to 5. The school’s population is 80% Hispanic, all students qualify for free or reduced-price lunch, 87% are transient, and 55% are English learners. The school’s principal introduced Reading Mastery in grades K-3 and Corrective Reading in grades 4-5 after finding that the majority of fifth graders read at only the second grade level. After introducing these curricula scores on state assessments rapidly improved. The percentage of special education students achieving AYP moved from 21.2% before implementation to 37.2% by the end of the second year of implementation, an effect size of .35. 29 Reading Mastery and Students with Learning Disabilities References Branwhite, A. B. (1983). Boosting reading skills by Direct Instruction. British Journal of Educational Psychology, 53, 291-298. Campbell, D.T. & Stanley, J.C. (1963). Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally. Cook, T.D. & Campbell, D.T. (1979). Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally. Cooke, N. L., Gibbs, S. L., Campbell, M. L., & Shalvis, S. L. (2004). A comparison of Reading Mastery Fast Cycle and Horizons Fast Track A-B on the reading achievement of students with mild disabilities. Journal of Direct Instruction, 4(2), 139-151. Engelmann, S. (2000). About reading – A comparison between Reading Mastery and Horizons. Effective School Practices, 18 (3), 15-26. Glang, A., Singer, G., Cooley, E., & Tish, N. (1991). Direct Instruction: Applications with students with brain injury. ADI News, 11(1), 23-28. Glang, A., Singer, G., Cooley, E., & Tish, N. (1992). Tailoring Direct Instruction techniques for use with elementary students with brain injury. Journal of Head Trauma Rehabilitation, 7(4), 93-108. Herrera, J.A., Logan, C.H., Cooker, P.G., Morris, D.P., & Lyman, D.E. (1997). Phonological awareness and phonetic-graphic conversion: A study of the effects of two intervention paradigms with learning disabled children. Learning disability or learning difference? Reading Improvement, 34(2), 71-89. Kuder, S. J. (1990). Effectiveness of the DISTAR reading program for children with learning disabilities. Journal of Learning Disabilities, 23(1), 69-71. Marston, D., Deno, S. L., Kim, D., Diment, K., & Rogers, D. (1995). Comparison of reading intervention approaches for students with mild disabilities. Exceptional Children, 62(1), 20-37. McKenzie, M. A., Marchand-Martella, N. E., Martella, R. C., & Moore, M.E. (2004). Teaching basic math skills to preschoolers using Connecting Math Concepts Level K. Journal of Direct Instruction, 4(1), 85-94. O’Connor, R. E., Jenkins, J. R., Cole, K. N., & Mills, P. (1993). Two approaches to reading instruction with children with disabilities: Does program design make a difference? Exceptional Children, 59 (4), 312-323. O'Connor, R. E., & Jenkins, J. R. (1995). Improving the generalization of sound/symbol knowledge: Teaching spelling to kindergarten children with disabilities. The Journal of Special Education, 29(3), 255-275. 30 Reading Mastery and Students with Learning Disabilities Richardson, E., DiBenedetto, B., Christ, A., Press, M., & Winsberg, B. G. (1978). An assessment of two methods for remediating reading deficiencies. Reading Improvement, 15(2), 82-95. Shadish, W.R.. Cook, T.D. & Campbell, D.T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin. Sommers, J. (1991). Direct Instruction programs produce significant gains with at-risk middle school students. Direct Instruction News, 11(1), 7-14. Stein, C., & Goldman, J. (1980). Beginning reading instruction for children with minimal brain dysfunction. Journal of Learning Disabilities, 13(4), 52-55. Stockard, J. (2011) “Merging the Accountability and Scientific Research Requirements of the No Child Left Behind Act: Using Cohort Control Groups” Jean Stockard, Quality and Quantity: International Journal of Methodology, electronic version available on-line, December, 2011. Tallmadge, G. K. (1977). The Joint Dissemination Review Panel Ideabook. Washington DC: National Institute of Education and U.S. Office of Education. What Works Clearinghouse (2012). WWC Intervention Report, Reading Mastery and Students with Learning Disabilities. Washington, D.C.: Institute of Education Sciences What Works Clearinghouse (2011). Procedures and Standards Handbook, Version 2.1. Washington, D.C.: Institute of Education Sciences. Young, M., Baker, J., & Martin, M. (1990). Teaching basic number skills to students with a moderate intellectual disability. Education and Training in Mental Retardation, 25 (1), 83-93. 31 Reading Mastery and Students with Learning Disabilities