AN ANALYSIS OF SPECIFIC LEARNING DISABILITY EXCLUSIONARY CLAUSE by PAMELA A. LYBARGER A DISSERTATION Presented to the Department of Educational Methodology, Policy, and Leadership and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Education June 2017 ii DISSERTATION APPROVAL PAGE Student: Pamela A. Lybarger Title: An Analysis of Specific Learning Disability Exclusionary Clause This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Education degree in the Department of Educational Methodology, Policy and Leadership by: Charles Martinez, Ph.D. Chairperson Joanna Smith, Ph.D. Core Member Keith Hollenbeck, Ph.D. Core Member Laura Lee McIntyre, Ph.D. Institutional Representative and Scott L. Pratt Dean of the Graduate School Original approval signatures are on file with the University of Oregon Graduate School. Degree awarded June 2017 iii © 2017 Pamela A. Lybarger This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs (United States) License. iv DISSERTATION ABSTRACT Pamela A. Lybarger Doctor of Education Department of Educational Methodology, Policy and Leadership June 2017 Title: An Analysis of Specific Learning Disability Exclusionary Clause The purpose of Individuals with Disabilities Education Improvement Act (IDEIA) is to protect the rights of students with disabilities and to assure that all students receive equitable access to a free and appropriate public education; yet there are explicit exclusions written in the law for students who may have experienced environmental, cultural, and economic disadvantages. An explanatory sequential mixed methods design was used to study Section Four of the exclusionary clause of Specific Learning Disability (SLD) that states students must be excluded from identification if their learning difficulties are primarily the result of environmental, cultural, or economic disadvantage. The Contextual Interaction Theory (CIT) was borrowed from water governance policy analysis to provide a theoretical framework for better understanding the complexity of context and interaction processes in special education policy implementation in Oregon. Survey (n = 100) and interview (n = 6) results showed that 87% of Oregon school psychologist respondents demonstrated some level of non-compliant behavior on Section Four implementation. Barriers to implementation included unclear state and federal guidelines, lack of measurable terms, unclear roles and responsibilities, external pressures to find students eligible, and lack of confidence that Section Four can be applied in every case for 69% or more of the sample. Recommendations for future research include v defining the Section Four terms, developing a measurement tool with cut offs, and training to improve implementation; however, there is concern that this may be an impossible task. Recommendations were made to develop a state level Task Force to begin the dialogue; however, future research is needed to gain a deeper understanding of validity of CIT, definitions of Section Four terms that are measurable, identification and definitions of the mitigating mechanisms in the determination of Section Four for appropriate and possible implementation. A supplemental spreadsheet file included with this dissertation was used to illustrate the coding and mapping of Ochoa, Rivera, & Powell (1997) factors onto CIT categories. vi CURRICULUM VITAE NAME OF AUTHOR: Pamela A. Lybarger GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene, OR Western Oregon University, Monmouth, OR Oregon State University, Corvallis, OR DEGREES AWARDED: Doctor of Education, Educational Methodology, Policy, and Leadership, 2017, University of Oregon Master of Education, Special Education, 2001, Western Oregon University Bachelor of Biological Science, Ecology, 1997, Oregon State University AREAS OF SPECIAL INTEREST: School Administration (K-12) Special Education Educational Policy & Equity PROFESSIONAL EXPERIENCE: Behavior Programs Coordinator, McMinnville School District, 2015-present Special Programs Coordinator/Director, Sheridan School District, 2013-2015 Special Education Teacher, North Santiam School District, 2007-2013 PUBLICATIONS: Rochelle, J. and Lybarger, P. (In press). First at the Finnish Line: The Origins and the Future of Finnish Education Reform. In Y. Zhao & B. Gearin (Eds.), Dreams and Nightmares: Global Education Reform on the Brink of the Fourth Industrial Revolution. New York, NY: Routledge. North Santiam School District. (2008-2011). Response to Intervention Manual. Stayton, OR: Student Services. vii ACKNOWLEDGMENTS There have been many people who have helped me in this process. First, I would like to thank Dr. Charles Martinez for his leadership and support. His equity work is inspiring. I would also like to express my sincere appreciation to Drs. Joanna Smith and Keith Hollenbeck for their assistance and countless hours in the preparation of this manuscript. Next, I would like to thank Dr. Laura Lee McIntyre for her expertise on my committee. Most importantly, special thanks are due to Mr. Dan Lybarger, whose support and encouragement made this undertaking possible. viii This is work is dedicated to the special educators in the state of Oregon for their constant pursuit of excellence in the identification and support of students with disabilities. It is a difficult job with resources that are often less than adequate. Thank you for all you do! ix TABLE OF CONTENTS Chapter Page I. INTRODUCTION .........................................................................................................1 II. LITERATURE REVIEW ..............................................................................................4 Empirical Studies on Exclusionary Clause ............................................................. 4 Implementation of SLD Identification Policy......................................................... 7 Policy Reviews of the Exclusionary Clause ........................................................... 9 History of Policy Implementation Research ......................................................... 13 Theoretical Framework: Contextual Interaction Theory (CIT) ............................ 15 Definition of Terms......................................................................................... 17 Interaction Process .................................................................................... 17 Arena ......................................................................................................... 18 Key Characteristics ................................................................................... 18 Motivation. .......................................................................................... 19 Cognitions. .......................................................................................... 20 Resources of Capacity and Power....................................................... 20 Specific Context ........................................................................................ 21 Structural Context. .................................................................................... 21 Wider Contexts. ........................................................................................ 22 Governance Regime. ................................................................................. 23 Theory’s Assumptions. ................................................................................... 24 CIT Process Model. ........................................................................................ 24 Arena ......................................................................................................... 25 x Chapter Page Interaction Process. ................................................................................... 25 Contexts. ................................................................................................... 26 Specific Context .................................................................................. 27 Structural Context ............................................................................... 27 Wider Context ..................................................................................... 27 Application of CIT Framework to Study of SLD Identification Practices. .... 28 Purpose of the Study ............................................................................................. 29 III. METHODS ..................................................................................................................30 Research Design.................................................................................................... 30 Phase One........................................................................................................ 30 Phase Two. ...................................................................................................... 31 Setting ................................................................................................................... 31 Participants ............................................................................................................ 32 Survey Sample. ............................................................................................... 33 Interview Sample. ........................................................................................... 33 Time Aspect .......................................................................................................... 35 Unit of Analysis .................................................................................................... 35 Data Collection Instruments ................................................................................. 36 Survey Instrument. .......................................................................................... 36 Work Setting and Demographic Items. ..................................................... 37 District Size as a Proxy for Urban and Rural Status. .......................... 38 xi Chapter Page Socio-economic Status (SES) Item. .................................................... 38 Factors Influencing Compliance. .............................................................. 38 CIT Context and Factors Items. ................................................................ 40 Interview Instrument. ...................................................................................... 40 Procedures ............................................................................................................. 41 Survey ............................................................................................................. 41 Interviews ........................................................................................................ 43 Data Analysis and Interpretation .......................................................................... 44 Survey Analysis. ............................................................................................. 44 Interview Transcription Analysis. ................................................................... 45 Combined Analysis and Interpretation. .......................................................... 45 Validity ................................................................................................................. 46 Triangulation. .................................................................................................. 47 Methodological Triangulation. ................................................................. 47 Data Triangulation. ................................................................................... 48 Analysis Triangulation. ................................................................................... 49 Investigator Triangulation. .............................................................................. 49 Theoretical Triangulation................................................................................ 50 IV. RESULTS ....................................................................................................................51 Descriptive Statistics of the Sample ..................................................................... 51 RQ1: Extent and Ways of Compliance ................................................................. 52 Findings on the Extent and Ways of Compliance From Survey Results ........ 52 xii Chapter Page Findings of Ways of Compliance From Interview Results. ............................ 54 Consideration of Section Four .................................................................. 54 Questioning if Section Four Should Remain in the Law. ......................... 55 RQ2: Factors of Implementation in Oregon ......................................................... 56 Findings on Implementation Factors From Survey Results. ........................... 56 Implementation Barriers to Compliance. .................................................. 56 Definitions of Environmental, Cultural, and Economic Disadvantage ..... 58 Ochoa Categories Used in Determining Section Four .............................. 59 Attendance as the Most Influential Factor ................................................ 60 Findings on Implementation Factors From Interview Results ........................ 62 Barriers to Compliance ............................................................................. 62 Pre-Referral Interventions (RTI or MTSS) ............................................... 65 Attendance as an Influential Factor .......................................................... 66 Definition of Environmental, Cultural, and Economic Disadvantage ...... 66 Acculturation............................................................................................. 67 Pre-Service Preparedness .......................................................................... 69 Contexts .................................................................................................... 70 Personal Beliefs ........................................................................................ 71 RQ3: Description of Influences to Implementation Using CIT ............................ 71 Findings on Implementation Factors Using CIT From Survey Results .......... 71 xiii Chapter Page Findings on the Implementation Factors Using CIT From Interview Results ............................................................................................................. 72 Summary of Key Results ...................................................................................... 74 RQ1: Extent and Ways of Compliance. .......................................................... 74 RQ2: Factors of Implementation in Oregon ................................................... 75 RQ3: Description of Influences to Implementation Using CIT ...................... 77 V. DISCUSSION ..............................................................................................................78 Discussion of RQ1: Extent and Ways of Compliance .......................................... 79 Implementation ............................................................................................... 79 Barriers to Compliant Implementation ........................................................... 79 Discussion of RQ2: Factors of Implementation of Section Four in Oregon......... 81 Barriers to Compliant Implementation. .......................................................... 81 Discussion of RQ3: Description of Influences to Implementation Using CIT ..... 82 Discussion of Levers to Improve Implementation ................................................ 83 Limitations of the Study........................................................................................ 86 Non-response Bias .......................................................................................... 86 Self-response Bias ........................................................................................... 86 Researcher Bias. .............................................................................................. 89 Construct Validity ........................................................................................... 87 Policy Implications ............................................................................................... 88 Pre-Service Institutions. .................................................................................. 90 Practicing School Psychologists and Educators ............................................. 90 Chapter Page xiv ODE and Policy Actors ................................................................................... 91 Federal Policymakers and Researchers ........................................................... 92 Key Future Policy Directions and Research ......................................................... 92 Publication Action Plan ........................................................................................ 94 APPENDICES .................................................................................................................. 96 A. CONTEXTUAL INTERACTION THEORY....................................................... 96 B. SURVEY PROTOCOL ........................................................................................ 97 C. SEMI-STRUCTURED PHONE INTERVIEW GUIDE .................................... 106 D. PRE-NOTICE EMAILTO SPECIAL EDUCATION DIRECTORS ................. 111 E. INVITATION EMAIL TO PARTICIPANTS .................................................... 112 F. FOLLOW-UP EMAIL TO PARTICIPANTS .................................................... 113 G. EXCEL SPREADSHEET OF CODED SURVEY DATA ................................. 114 REFERENCES CITED ................................................................................................... 121 SUPPLEMENTAL FILES EXCEL SPREADSHEET: CIT CODED SURVEY RESULTS xv LIST OF FIGURES Figure Page 1. Contextual Interaction Theory (CIT) Map .................................................................. 17 2. Explanatory Sequential Mixed Method Design .......................................................... 41 3. Survey Results CIT Key Characteristics..................................................................... 73 xvi LIST OF TABLES Table Page 1. Setting and Population of Survey as Dependent Upon School District Size .............. 32 2. Demographic Counts and Percentages of Survey Sample .......................................... 34 3. Survey Questions by Item Category and Research Question ..................................... 37 4. Oregon School Psychologist’s Level of Compliance with Section Four .................... 53 5. Why Oregon School Psychologists Ignored or Circumvented Section Four .............. 57 6. Descriptive Statistics for the Level of Endorsement for Defining Characteristics of Section Four ................................................................................... 58 7. Endorsement of Criteria Used by School Psychologists in the Determination of Section Four Exclusionary Clause in Oregon......................................................... 61 8. Notable Quotes Regarding the Barriers to Compliance as Identified by Oregon School Psychologists .................................................................................................. 64 9. Responses to the Interview Question: How Would You Define the Exclusionary Clause? .................................................................................................. 68 1 CHAPTER I INTRODUCTION It has been over 40 years since The Education for All Handicapped Children Act (P.L. 94-142) was passed in 1975. This initial federal policy has been reauthorized several times; the last time was over a decade ago as the Individuals with Disabilities Education Improvement Act (IDEIA; Individuals with Disabilities Education Improvement Act of 2004, 2005)1. The U.S. Department of Education (USDOE) reports that the four purposes of the policy are to assure (a) that all children receive a free and appropriate public education, (b) the protection of parents’ and students’ rights, (c) the assistance to states and localities to provide for children with disabilities, and (d) that the efforts to educate children with disabilities are effective (2011). Implementation and monitoring of this complex social policy is a huge undertaking and is represented by the large number of students that are served in special education across the nation. According to the National Center for Educational Statistics (2015), the number of children ages 3-21 receiving special education services in 2012-13 was 6.4 million, or about 13 percent of all public school students. The largest single category in special education has been Specific Learning Disability (SLD) and it generally represents nearly half of all students in special education (National Center for Educational Statistics, 2015). 1 Although the reauthorization changed the name of this policy, those in the field continue to refer to the policy as Individuals with Disabilities Education Act (IDEA), which I will do for the purposes of this study 2 SLD identification has historically been the center of a long research debate (Chalfant, 1989; Dombrowski & Gischlar, 2014; Fletcher, Coulter, Reschly, & Vaughn, 2004; Fuchs, Deshler, & Reschly, 2004; Hallahan et al., 2007; Lyon, 1996; McDermott, Goldberg, Watkins, Standley, & Glutting, 2006; Scanlon, Boudah, Elksnin, Gersten, & Klingner, 2003). Chennat and Singh’s (2014) literature review concluded that SLD may be the “most contested and elusive concept within special education” (p. 2) and is the only disability category (out of 13 disability categories) whose criteria for diagnosis are defined by law. The legal definition of SLD is “a disorder in 1 or more of the basic psychological processes involved in understanding or in using language, spoken or written, which disorder may manifest itself in the imperfect ability to listen, think, speak, read, write, spell, or do mathematical calculations” (United States Government Publishing Office, 2011, pp. 856–857). The statute also lists eight components of the clause, specifying that “such term does not include a learning problem that is primarily the result of visual, hearing, or motor disabilities, of intellectual disabilities, or emotional disturbance, or of environmental, cultural, or economic disadvantage” (2011, p. 857). This last part of the exclusionary clause (i.e., “environmental, cultural, or economic disadvantage”) is referred to as Section Four. All 50 states report using all eight components of the exclusionary clause (Reschly & Hosp, 2004), including Section Four. The first five components of the exclusionary clause refer to other disabilities as being the primary cause of the learning difficulties. Learning difficulties that are primarily the result of another disability are clear exclusionary factors for also being identified as SLD, as their needs are met in special education in other ways. Researchers have studied each of these other factors from various perspectives; though few have 3 studied Section Four of the exclusionary clause (Harris, Gray, Davis, Zaremba, & Argulewicz, 1988; Ochoa, Rivera, & Powell, 1997), which refers to the exclusion based on environmental, cultural, or economic disadvantage. There is not a clear understanding of what it means to be environmentally, culturally, or economically disadvantaged (Chalfant, 1989; Lyon, 1996), yet the law requires the exclusion of students with learning problems that may have resulted from being disadvantaged in these ways. The purpose of this dissertation is to better understand the implementation of a complex social policy using Section Four of the SLD exclusionary clause as a case study. The study of compliance with and implementation of Section Four in the state of Oregon allows for the application of a theoretical framework from outside of education to inform future studies. In this dissertation, I will review current literature that includes eight studies of compliance, implementation, and policy review on the exclusionary clause. Then, I will briefly discuss the history of policy implementation research. Next, I will introduce a theoretical framework for the study. Then, I will discuss the methods to answer my three research questions. Then, discuss results. Finally, I will draw conclusions, make recommendations, and discuss the limitations of the study. 4 CHAPTER II LITERATURE REVIEW In the last 40 years, few scholars have published work explicitly on the exclusionary clause (Chandler, 2014; Fletcher & Navarrete, 2003; Harris et al., 1988; Lyon, 1996; MacMillan & Siperstein, 2001; Mellard, Deshler, & Barth, 2004; Ochoa et al., 1997; Ryan, 2013), with even fewer empirical studies on the topic (Chandler, 2014; Harris et al., 1988; Ochoa et al., 1997). First, I will discuss the three empirical studies (Chandler, 2014; Harris et al., 1988; Ochoa et al., 1997), then transition to the implementation studies (Fletcher & Navarrete, 2003; Mellard et al., 2004) and scholarly paper (MacMillan & Siperstein, 2001) mentioning the exclusionary clause, and finally, the two policy implementation reviews (Lyon, 1996; Ryan, 2013) on the topic to complete the body of research identified in this literature synthesis. Empirical Studies on Exclusionary Clause The seminal empirical study by Harris et al. (1988) established that nearly half of the school psychologists who responded to a national survey (n = 74) reported ignoring the exclusionary factors for environmental, cultural, and economic disadvantage. Although the sample size was small for a national study, and the survey response rate was only 24%, results indicated that the number of students actually excluded based on Section Four was relatively small (M = 2.3, Median = 0.6, n = 74). Of the study’s respondents, 47.3% reported that they regularly tried to comply with Section Four, 37.5% routinely ignored or circumvented the regulation, and 9.7% sometimes complied and sometimes did not. Results also indicated that males complied more often than females (X2 (1, n = 67) = 3.95, p < .05) and those serving four or fewer schools were less likely to 5 comply with the law. No causal inferences were made in the study and are yet to be discovered through research. This study introduced a potential policy implementation problem with Section Four, as there appears to be inconsistency with implementation and compliance. Half of the sample reported that they did not follow the law; which would clearly impact consistent policy implementation across the nation. Ochoa et al. (1997) extended the Harris et al. (1988) study by investigating factors used in the decision-making process of inclusion or exclusion that were used to comply with Section Four of the law, especially when evaluating students identified as bilingual or with Limited English Proficiency (LEP). Ochoa et al. (1997) used a single open-ended survey item to identify 37 decision-making factors, with the authors identifying 17 additional factors from research and professional experience, which were all categorized into six themes: (a) family and home factors, (b) language instruction and language- related factors, (c) assessment instrument and procedural safeguards, (d) educational history factors, (e) general educational factors, and (f) other miscellaneous or unidentifiable factors. There were many limitations of the study that were acknowledged by the authors: exclusive use of self-report, one open-ended question, possible fatigue in answering a single question (i.e., the respondents may not have listed every factor that they use), factors not included due to non-recognition or obviousness or levels of automaticity, and a low survey response rate of 29% (n = 74). Not all responses were able to be categorized and resulted in 22% of the responses being categorized as other. There is also introduced bias with the authors including factors that they state were overlooked by the study participants. Although this study may not provide an exhaustive list of possible factors used in the decision-making process of inclusion or exclusion based on 6 Section Four, it did provide an initial list of factors that may be used to comply with Section Four. While this study built off the Harris et al. (1998) study, Ochoa et al.’s (1997) generalizability was limited and did not identify factors that influenced the limitations of compliance with Section Four. The compliance findings of both studies were similar, as Ochoa et al. stated that “the extent to which many of these factors are used, however, appears to be low” (p. 163). The sample size of the Ochoa et al. study was small, resulting in less than 17% (n = 74) of the population of school psychologists in the eight states studied, which illustrated that additional research is needed to confirm this list of factors and categories for generalization and to determine how these factors are used in the implementation and compliance of Section Four. The most recent empirical study of the exclusionary clause was completed by Chandler (2014), which focused on the SLD identification process when implemented in a high poverty rural school district. She used a qualitative case study approach to interview, observe, and review documents to examine what teachers believed about poverty in a poor rural school district and how their assumptions influenced their decisions on SLD eligibility. Findings suggested that the middle-class backgrounds of the study’s teachers likely influenced their attitudes and beliefs about poverty and “significantly impact[ed] their ability to understand the context from which their students come” (p. 37). This means that teacher attitudes of poverty and their middle-class lens influenced the context of their interactions in the decision-making process of SLD identification. Findings also demonstrated a mismatch between SLD process in the study and mandated law, specifically around the exclusionary clause, with Chandler (2014) noting a “lack of RTI [response to intervention] implementation and the avoidance of the 7 discussion regarding exclusionary factors” (p. 36). This study addressed several gaps in prior research by examining the overlapping topics of poverty, rural education, and SLD identification; however, this was a case study with potential teacher response bias and selection bias that threatens the internal validity of the study. The methods for this study included convenience sampling of 11 participants of a single district and conducting semi-structured interviews. As with many case studies, the generalizability was not strong, but Chandler’ findings suggested that the context in which policy actors operate matters in the policy implementation process of SLD identification and Section Four. Implementation of SLD Identification Policy The paucity of empirical research on SLD identification practices created a need to analyze the implementation of the policy. Mellard, Deshler, and Barth (2004) proposed that SLD identification was not simply about “building a better mousetrap;” but ensuring better policy implementation at multiple levels with various policy actors (p. 229). The three empirical studies, described above, all concluded that the exclusionary clause has been ineffectively implemented or even ignored by school personnel (Chandler, 2014; Harris et al., 1988; Ochoa et al., 1997) when considering policy implementation from a micro- or street-level perspective (Weatherly & Lipsky, 1977). The Mellard et al. (2004) policy implementation study noted that the influences of stakeholders in the SLD identification process was unique as compared to other disability categories. Respondents reported a “broad array of factors beyond a student’s performance on formal and informal assessments influence ultimate decisions made about a student’s eligibility for learning disability services” (Mellard et al., 2004, p. 229). The primary conclusion of their study was that “the assignment of school personnel to implement SLD identification practices 8 seem[ed] to be influenced by the prevailing cultures, values, power relationships, perceived roles (of self and others) and so on" (Mellard et al., 2004, p. 241). In other words, contextual variables have a strong influence on the decision-making process and policy implementation for SLD. Although the study had a small sample size (N = 113) for national generalization of findings, it did highlight the possibility that the identification criteria may be secondary to the implementation practice of the policy at the street-level. In another SLD policy implementation study, Fletcher and Navarrete (2003) described the troubling trend of over identification of Hispanic students in special education under the category SLD. Fletcher and Navarrete proposed that SLD is a socially constructed disability, rather than an internal or distinct disability. They reported that the exclusionary clause could be critically reducing the number of all students referred to special education and acknowledged that the Harris et al (1988) and Ochoa, et al. (1997) studies were the only two studies conducted on the exclusionary clause prior to their study. Fletcher and Navarrete (2003) reminded the reader that federal mandates are in place that requires nondiscriminatory assessment for special education and suggested contextual factors may be influencing implementation practices of IDEA and student learning. A similar finding was found by Chandler (2014). Although Fletcher and Navarrete (2003) did not conduct a full literature review, the wide body of research cited provided strong evidence to support their argument that a new assessment paradigm may assist in the drive for more equitable practices in special education. At around the same time, MacMillan and Siperstein were invited to speak at the Learning Disabilities Summit in Washington, DC in August, 2001, at which they 9 presented a paper regarding the operational definition and trends of SLD. The paper clarified the authoritative definition of SLD from the law, used citations to show that research has demonstrated that teacher referral (for special education evaluation under the category of SLD) may influence the evaluation process with subjective judgment and local norms, and, using assumption from other findings and informal discussions with school personnel, the authors concluded that school personnel have ignored exclusionary factors and processes to justify SLD classification, and documented that contextual factors were influencing the eligibility deliberations. They concluded that the increasing numbers of students identified as SLD were due to breakdowns in the identification and evaluation process and indicated that “schools have opted to ignore the ‘exclusionary criteria’ … in order to serve students in need” (p. 6). They stated “[i]n truth, the research does not inform practice because the database derives from a population of ‘LD’ students who only vaguely resemble school-identified ‘LD’ students. We contend that the researchers studying subjects with LD and the practitioners serving students with LD do not agree on who is LD” (p. 8). They suggested that research was not informing practice in this regard and that identification practices were primarily based on the needs of schools to help low performing students. They called for a refinement of eligibility categories to address the “unhealthy schism between research and practice” (p. 8). Policy Reviews of the Exclusionary Clause In addition to the empirical studies, implementation studies, and scholarly paper, two policy reviews are relevant to the review of the SLD exclusionary clause. Lyon (1996) published a policy review of SLD identification and implications on basic reading skills. The literature review was extensive with 71 articles cited and a historical account 10 of SLD origins that is often cited by other researchers (573 citations of this article are reported by Google Scholar). In the review, Lyon (1996) reported that Congress established separate programs for impoverished students (e.g., Title One) and it was Congress’ intent to exclude students whose underperformance was primarily attributed to poverty; however, “this distinction is difficult or impossible to draw, and no empirical data exist to support this exclusionary practice” (p. 56). He argued that SLD “is not a distinct disability, but an invented category created for social purposes” (p. 60). Lyon suggested that parents and teachers are more comfortable with the diagnosis because it was less stigmatizing and does not imply low intelligence. These sentiments were echoed 17 years later in Ryan’s (2013) policy review that focused exclusively on Section Four of the SLD exclusionary clause. Ryan’s (2013) policy review also included a historical perspective of the components of environmental and economic disadvantage and suggested the connection between law, policy, and neuroscience research on the brain development of children in poverty. Ryan claimed that new neuroscience research indicates that economic disadvantage results in internal processing differences that meets the criteria for SLD and argued that the clause should be removed from the law, citing it as an “unjustified barrier to special education” (p. 1503). Ryan’s historical account of the origins of the exclusionary clause are noteworthy and indicate that the inclusion of environmental and economic disadvantage as exclusionary factors stemmed from opponents’ worry “that the category would funnel special education funds to poor students” (p. 1465). He cited the book, Identification of Learning Disabilities, Research to Practice, in which Martin (2002), former USDOE Assistant Secretary for Special Education and Rehabilitative 11 Services substantiated the claim that powerful policy actors were successful in including the exclusionary clause into P.L. 92-142 due to fears that the SLD category “would open the door for ‘handicapped funds’ to be used for minority and other economically disadvantaged children” (p. 87). Others have argued that the SLD category provides a disability category for middle-class white children whose learning was not commensurate with their peers, but presented with average or above average intelligence which was more palatable to middle and upper class parents (Chennat & Singh, 2014; Colker, 2011). Although the P.L. 94- 142 was adopted in 1975, there was no clear definition of SLD (Colker, 2011; Hallahan & Mercer, 2002; Lyon, 1996, 1996). In fact, the adopted definition was intended to be provisional (Colker, 2011) and five research institutes were funded by the United States Office of Education (USOE) (Hallahan & Mercer, 2002) to clarify the definition, SLD characteristics, and effective interventions at the time of adoption of the law. The largely unstudied SLD category was likely included in P.L. 94-142 due to influential stakeholders in the policy arena with strong advocacy from parents (Hallahan & Mercer, 2002; Martin, 2002). Some of the greatest influence in the policy adoption process for SLD identification was borne from fear of the economic impact of including SLD as a special education category. Section Four was specifically included to exclude minorities and economically disadvantaged students from accessing handicapped funds (Hallahan & Mercer, 2002; Lyon, 1996; Martin, 2002; Ryan, 2013). Several authors concluded that there was no empirical data to support Section Four (Lyon, 1996; Ryan, 2013), yet it remains part of the mandate today. Lyon (1996) argued that SLD “is not a distinct 12 disability, but an invented category created for social purposes” (p. 60), supported by Ysseldyke and colleagues at the University of Minnesota (one of the five institutes created by USDOE), who argued that [S]LD can best be defined as ‘whatever society wants it to be, needs it to be, or will let it be’ at any point in time” (Hallahan & Mercer, 2002, p. 52). The general purpose of the IDEA was to ensure more equitable practices in the identification and service provision for students with SLD (and other disabilities). IDEA embedded mandates for comprehensive and nondiscriminatory evaluations for all students, yet also included an exclusionary clause based on the status of environmental, cultural, or economic disadvantage. Regardless of the reasoning behind the language of the clause, there is a need to know how policy actors, schools, districts, and states implement it or if they circumvent it. Consideration of this issue from a policy implementation perspective would provide additional insights into the SLD identification process while providing a framework for future policy analysis of detailed implementation of large complex social policies. Policy relevance is the primary criterion for the worth of policy implementation studies and can be beneficial to policymakers if the implementation study is “pertinent, sound, and timely” (Williams, 1982, p. 182). With the SLD category comprising nearly half of all students in special education, this clause has the potential to dramatically change the face of special education if removed or changed. To this end, IDEA is due for re-authorization again. Policymakers will need valid and reliable policy implementation research findings to adequately prepare for the re-authorization process. This policy review will hopefully ignite interest in the Section Four implementation and provide a 13 theoretical framework for future studies. There are equity issues embedded in the exclusionary clause for students that may be influenced by environmental, cultural, or economic disadvantage factors. More research is needed to fully understand the effectiveness and implications of the implementation of Section Four of the exclusionary clause as an aspect of IDEA policy. Application of a comprehensive theoretical framework is needed to adequately capture all of the components of such a large and complex social policy through various contexts and interaction processes from the federal, state, and local levels. History of Policy Implementation Research The history of policy implementation research has been described in generational terms by Fowler (2013). The first-generation research (studies published in the early 1970s or earlier) focused on the difficulties of policy implementation. The second- generation of policy research began in the late 1970s and continued throughout the 1980s and explored the characteristics of successful and unsuccessful policy implementation efforts. The third-generation of policy implementation research began in the 1990s and continues in the present day, with a primary focus on the implementation of increasingly complex policies. Third-generation policy research has focused less on implementation outcomes and more on implementers as learners and scaling up to expand effective educational reform (Fowler, 2013). Elmore (1979) described a process of forward and backward mapping to affect the implementation process and the outcomes of policy decisions. This process required that complex educational reform has been studied from the perspective of the street-level bureaucrat (Weatherly & Lipsky, 1977) or at the micro-level (Berman, 14 1978) , which considers policy implementation from the bottom-up perspective. The concept of backward mapping considers the implementation process from the implementer’s perspective, at the point “which administrative actions intersect private choices” (Elmore, 1979, p. 604). The same process for developing better policies based on the knowledge about determining whether policy decisions could be implemented prior to implementation could be applied to policy analysis as well. Elmore (1979) explained the process as: Only after that behavior is described does the analysis presume to state an objective; the objective is first stated as a set of organizational operations and then as a set of effects, or outcomes, that will result from these operations. (p. 604) Research has identified the need to consider policy implementation from the implementers’ perspective, including the social infrastructure for implementation (Fowler, 2013). Then “in the final state of analysis the analyst or policymaker describes a policy that directs resources at the organizational units likely to have the greatest effect” (Elmore, 1979, p. 604). Miles and Louis (1990) suggest five essential elements to move knowledge to action: (a) clarity, (b) relevance, (c) action images, (d) will, and (e) skill. These elements are described as clear knowledge that is not “fuzzy, vague, or confusing;” that is “meaningful” and “connected to one’s normal life and concerns;” and allows the implementer to have an image of “what to do to get there” (p. 58). To move this knowledge into action, the implementer must also have the “motivation” and “skill” or capacity that allows for “behavioral ability to do the action envisioned” (p. 58). A detailed theoretical framework is needed to better understand these elements from the bottom-up perspective. 15 Theoretical Framework: Contextual Interaction Theory (CIT) The process of policy analysis for a complex social policy such as IDEA can become overwhelming and difficult to contextualize and describe. This dissertation utilized an environmental policy theoretical framework for use in educational social policy that assisted in description of compliance and implementation factors for Section Four and provided a framework for a meta-analysis of future studies on the topic. Implementation of IDEA requires analysis of various levels of context from the federal, state, and local levels; as well as individual policy actor interactions that influence implementation and decision-making factors at the local school level. The multi-level and multi-actor aspects of environmental policy analysis made Contextual Interaction Theory (CIT) an ideal candidate for a transfer to educational social policy analysis (O’Toole, 2004). In order to move social policy analysis of the SLD exclusionary clause forward, a complex framework was needed to capture the multi-level and multi-actor aspects of IDEA policy implementation. This complex theoretical framework allowed consideration of influences of various levels of contextual factors and multiple layers of social interaction processes at the same time. O’Toole (2004) proposed the use of CIT for more general policy applications in which policy instruments adopted by governments can feed into social processes between implementers and the target groups they seek to influence. After searching literature for all studies mentioning or using CIT in ERIC, Google Scholar, and a general search of the University of Oregon library database, it was determined that Zehavi (2011) was the only researcher to answer the call to use CIT in the context of educational policy research. He modified CIT to compare Australian and Israeli attempts to regulate non-governmental 16 schools. Originally, CIT was developed at the University of Twente in the Netherlands for providing insights into the influence of contextual factors on the water governance processes (Bressers, 2007, 2009; Bressers & de Boer, 2013; De Boer, Kruijf, Özerol, & Bressers, 2013); however, the usefulness of CIT in educational policy analysis was yet to be fully discovered. Figure 1 shows a theoretical framework of CIT that allowed for consideration of the contextual factors of multi-levels and multi-actors from individual interactions, specific contexts, structural contexts, and a wider context (also available in Appendix A). CIT allowed for a level of complexity that provided a framework for federal policy implementation to be considered at various levels of implementation with descriptions of necessary elements to move knowledge into action (Miles & Louis, 1990). CIT also captured interaction processes within each level of context and individual policy actor interaction that influenced the implementation and eventual outcomes of the policy. This web of interaction and influence becomes a conceptual model of the complex policy network that can be analyzed at various layers of interaction, context, and influence for a complete policy analysis. The following steps will be used to unpack CIT: (a) the major terms will be defined; (b) the theory’s assumptions will be identified; (c) the process model will be divided into parts and explained (e.g. interaction process, arena, and contexts); and (d) the framework will be adapted for use in the educational policy context. 17 Figure 1. Contextual Interaction Theory (CIT) map—a multi-level and multi-actor network of various social contexts with an unknown number of conceivable factors that might influence the course of the interaction process. Figure adapted from Bressers (2009). Copyright 2009 by Presses Polytechniques et Universitaires Romandes. Definition of terms. Figure 1 introduces many new terms that will be needed to understand the CIT (see Appendix A). The definition of terms will begin on the right of the figure and move to the left. In this case, policy implementation is considered at the individual policy actor level, which takes place at the interactive process loop level. After the elements of the interactive process loop have been defined, definitions will continue to the elements in the circles to the left of the Figure. Interaction process. The interaction process is a flow of key characteristics (i.e., motivation, cognitions and power and capacity) of the policy actors within a given arena that ultimately influence outcomes of human interactions, including those created by policy implementation (Bressers, 2009). The interaction process is the internal process that influences the individual policy actor’s decisions and actions. In the case of SLD identification, these individual policy actors are the Individual Education Plan (IEP) team 18 members that are part of the evaluation team (e.g., school psychologists or qualified evaluators of SLD, special education teachers or case managers, general education teachers, administrators or district special education directors serving as district representatives, other specialists or counselors serving the student, parents, and sometimes students who are active members of the decision-making process). The interaction process also occurs during the evaluation process as team members discuss the student’s situation and characteristics as well as during the IEP meeting when the eligibility determination is made. Arena. The environment or setting that the ultimate process occurs in. It will typically represent the interaction process of several policy actors (Bressers, 2009). In SLD identification, the arena could be a particular evaluation meeting, the processes within a school building or district, or could also represent the interaction processes in a professional development workshop or policy committee meeting. The arena consists of a group of policy actors, at any level, that engage in work that involves the development or implementation of the policy. Figure 1 illustrates an arena on the right side of the figure, which incorporates the interaction process of several policy actors. This social interaction or exchange is the interaction process and where (the environment) that process occurs is considered the arena. The arena is most proximally influenced by the specific context of the interaction. Key characteristics. Policy actors’ motivation, cognitions, and capacity and power flow through influences within an interaction process that, in turn, influences the interactions within an arena that ultimately results in the outcomes of policy implementation. Key characteristics are the internal personal and social structures that 19 influence the observable behavioral outcomes of individuals within the network (Bressers, 2009). The key characteristics are the strengths and weaknesses of each person that influences his or her behaviors, these internal interactions are rarely identified and may be impossible to discern. Each key characteristic is defined below with examples. Motivation. Policy actors’ learning is dependent upon motivation and observed by behaviors that are influenced by their own goals, values, self-interests, external pressures, and the de-motivational effect that can occur when an actor perceives their preferred behavior is beyond their capacity (Bandura’s self-effectiveness assessment; Bressers, 2009). In other words, each policy actor has their own goals, values, self-interests, and external pressures that influence his/her behavior and motivation. Motivation is influenced by the perceptions of the person regarding their ability to achieve desired outcomes. For example, a general education teacher has expectations for having every student in their class make progress and meet Common Core State Standards (CCSS). The teacher may develop a high motivation to have a student identified as SLD to relieve pressure and reconcile his/her belief that they are a good teacher. A de-motivational example may be that the teacher has exhausted his/her skills and knowledge on how to help the student learn and have given up on believing that they can make a difference for the student. Motivation influences other interaction components through relevance of resources for intended action and focused attention. It is also influenced by availability of resources for intended action and by opportunities and threats. An example of this impact could be the Social Exchange Theory that states that the costs of action are greater than the benefits (Homans, 1958). In the case of SLD identification, if the discomfort of telling a parent that the environmental or economic (home) conditions are the primary 20 factor in a student’s lack of ability to perform commensurate with their peers is greater than the support resources available to the person, there will likely be a lack of motivation to have that difficult conversation. Cognitions. Bressers (2009) defines cognitions as the actors’ process of filtering observations, learning processes, experiences, etc. Sometimes cognitions are the interpretations of reality that are held to be true. Cognitions influence opportunities, threats, and strategic value; and are influenced by the focusing of attention, data search, and processing capacity of the individual. An example in SLD identification could be the professional understanding of the IDEA law, how it is applied, and his or her interpretation of the contextual factors and interactions of others. Resources of capacity and power. Power is often the result of attribution by others and capacity is dependent upon availability and accessibility of resources. There can be formal rules, money, skills, time, and consensus impacts on capacity. Dependency on resources shapes the balance of power. Power and capacity influence motivation and cognitions through availability of resources for intended action and data search and processing capacity. In turn, power and capacity are influenced by relevance of resources and the strategic value of cognitions (Bressers, 2009). Again, this can be thought of as the analysis of cost/benefit of compliance. For example, the school psychologist suspects that environmental and economic disadvantage has influenced the learning of a student, but the student needs special education services to make adequate progress toward grade level standards. The school psychologist is influenced by the fact that past practice is that students with similar characteristics and disadvantage have been routinely identified in the past. The psychologist ignores or deemphasizes the impact of attendance and lack of 21 early vocabulary development in the evaluation process to ensure that the student is found eligible. She knows that she wants to get along with those that she works with and qualification and identification as SLD is the outcome that the IEP team expects. The school psychologist has been influenced by the power of those that she works with to make this decision, which has been framed over time by the lack of resources to help all students, regardless of eligibility status in special education. Specific context. This is the most proximal context to the interaction process that influences the arena through previous decisions and the specific circumstances of the case. Various actors may bring different specific contextual factors to their individual interaction process within the arena or situation (Bressers & de Boer, 2013). This is case specific and can be at the individual, building, or district level. For SLD identification, the specific context is influenced by previous decisions, past practice, and specific case circumstances. The IEP team dynamics aide in the development of the specific context, so strong personality types or individual convictions within the team dynamic can alter or influence the interaction process through the specific context. For example, a strong parent advocate who demands an evaluation and threatens the district with legal action if the evaluation is not completed could cause a strained relationship between the evaluation team and parents who could influence the case specific context, which does not allow for adequate data collection regarding Section Four factors. Structural context. CIT (Bressers & de Boer, 2013) refers to structural context as governance with levels and scales of the interaction, the various networks and actors involved, perspectives and goal ambitions at play, the strategies and instruments, as well as the responsibilities and resources for implementation. This context is not actor 22 specific, but is common among many different specific contexts, although implications sometimes differ. This level of context is likely the state or district level of policy implementation. In the case of SLD, the structural context also involves the teacher and school psychologist licensure and preparation programs that helps frame the individual specific contexts that ultimately influence the individual actors’ key characteristics. This level of context also includes the state level guidance, compliance monitoring, and resources for implementation. Wider contexts. The wider context is the most distal context to the interaction process and consists of the problem, political, economic, cultural, and technological contexts that make up this wider societal layer of the theory (Bressers, 2009). In Bressers (2009), the wider context represents the federal level of implementation with some influences at the state level. This context refers to the overall purpose of IDEA to provide a free and appropriate public education to all students. It involves the political, economic, social, and cultural contexts of the development and implementation of IDEA. There are also technological aspects that refer more to the methods and strategies used to implement, enforce, and monitor federal law and policy. The technological context refers to the white papers and regulations that help implement the policy. In Oregon, an example of a technological process is the System Performance Review and Improvement (SPR&I) process that requires that reports of who is identified as SLD, when they were identified, and where. Each school district reports this information to the state; the state then requires a sample of files be reviewed each year to ensure compliance standards are met. These compliance standards correlate to key requirements of IDEA law for SLD evaluation and identification procedures. There are technical assistance papers and 23 mandated professional development opportunities that are developed as a result of the data collected from SPR&I processes. This system performance and review process also identifies school districts that are not complying with compliance standards. Districts that are not complying must demonstrate improvement or face the loss of federal funding for special education. Governance regime. This term refers to the concept of multi-level and multi-actor interactions. Actors at every level can simultaneously influence interactions at every level of society to various degrees. Essentially, governance regime refers to the network aspect of the theory (Bressers, 2007; Bressers & de Boer, 2013). Governance regime as presented in the case of exclusionary clause would present from macro- to micro-level as: IDEA mandates the evaluation of students suspected with learning challenges and outlines both inclusion and exclusionary criteria for eligibility determination; the state of Oregon receives federal funds for incentives for compliance of the law and Oregon transfers the federal law into state level administrative rules (OARs); Oregon Department of Education (ODE) provides the regulations, eligibility documents, procedural guidance and training, compliance and monitoring activities to ensure general compliance with SLD evaluation and eligibility; Oregon eligibility documents include a statement that indicates that lack of achievement is not primarily due to exclusionary factors; Oregon school districts develop strategies and procedures for evaluation and documentation of evaluation and eligibility, the school district provides various levels of resources including training and personnel to implement state and federal policy; individual personnel at the district and building level act as individual policy actors in the process of evaluation until the specific contextual factors of previous decisions and specific case 24 circumstances influence the interaction processes in the school or team arena that ultimately results in a team decision as the outcome of evaluation and eligibility determination for individual students. Theory’s assumptions. There are six main assumptions of this theory (Bressers, 2009): (a) policy processes are actor-interaction processes; (b) policy actor characteristics of motivation, cognitions, and capacity and power (boxes in right side of process Figure 1) are influenced by numerous factors from within and outside the process; (c) motivation, cognitions, and capacity and power influence each other and all must be considered to avoid losing important insight. Every change in one key actor characteristic influences the other two; (d) characteristics of the actors shape the process, but are also influenced by the course and experiences in the process and can therefore gradually change during the process; (e) characteristics of actors can be shaped from outside influences (external contexts found on the left side of Figure 1); and (f) the context is also influenced by outside contexts of political, socio-cultural, economical, technological, and problem contexts. Their influence on the actor characteristics may be both direct and indirect through governance regime. CIT process model. As illustrated in Figure 1, CIT has two main components to the theory model. The left side of Figure 1 are multiple layers of contexts that influence other layers of context and influence the arena of interaction processes. The right side of Figure 1 illustrates the multiple layers of interaction processes for individual policy actors in the arena. The model is generally read from right to left, as the theory focuses on the arena with the multiple interaction processes that are influenced by multiple levels of contexts. The model also illustrates an infinite network of interactions and influences as 25 the model supports multiple layers of interactions, influences, and contexts. It involves an arena of interaction processes for any number of policy actors. These interactions are influenced by wider circles of contexts that grow more distant with further reaching influence than those that are more proximal. The entire theoretical framework will be divided and described by its components (i.e., arena, interaction process, and contexts). Arena. The CIT process typically begins from the street-level bureaucrat perspective (Weatherly & Lipsky, 1977) also known in the policy literature as a micro- level (Berman, 1978). The arena is the setting of the interaction process that consists of a web of interactions and represents the right side of the CIT model in Figure 1. This is representative of the group dynamics of several policy actors interacting in a team membership. This of course would vary depending upon the situation and number of policy actors involved and in the case of SLD identification may include school psychologists or qualified evaluators, special education teacher or case manager, parent(s), student, general education teacher(s), administrator(s), and other specialists. Interaction process. Each layer of interaction process represents only one policy actor, as represented on the right side of Figure 1. A policy actor is likely an individual stakeholder of the policy. In the case of special education, the policy actors are likely Individualized Education Plan (IEP) team members. These IEP members become policy actors, as they implement IDEA. The policy actors could be school psychologists, special education teachers or case managers, general education teachers, administrators or school district representatives, parents, and other specialists or team members. Each policy actor engages in their own interaction process given the components below and their interactions. Each of the policy actors then influence and are influenced by others as they 26 interact to make evaluation and eligibility decisions involved in the identification process for each individual student. Each interaction is influenced by the key characteristics of each policy actor. Motivation, cognition, and capacity and power all interact and influence each other. As one of these characteristics influences another, there is a ripple effect on each of the other characteristics until there is action by the person through their behavior. The complexity of these interactions by practitioners in the field (aka: policy actors) is best understood with a simple model of fifteen contextual factors that each only have two possible values which would result in 30,000 possible combinations (Bressers & de Boer, 2013). It is clear that this theoretical model is impractical for isolating the impact of a single factor. The resources of the actors provide them with the capacity to act and power in relation to others. Their cognitions influence their interpretations and perceptions of social interactions and other forms of cognitive input (e.g., data, policies and procedures, prior knowledge, assessment results, etc.). And, their motivation within the interaction is influenced by internal and external pressures that, when interacting with cognitions, also create productive or non-productive settings for the process and future interactions with others. All of these actor characteristics work together to explain the social interaction process that ultimately result in outcomes and decisions about how the policy is interpreted and implemented at the individual local level. Contexts. All social interactions are influenced by the contexts in which they occur. The left side of Figure 1 illustrates the levels of contextual factors that exist in complex social policy. The contextual factors are overlapping and have variable levels of influence on the interaction process and individual decision-making. 27 Specific context. The Specific Context is the case specific context and involves the geographical setting of the interaction, including case specific circumstances, previous decisions, and historical and cultural norms for the particular environment or institution. This is the transfer of policy to practice at the local level. Structural context. The Structural Context generally has a broader influence on the Interaction Process and is more stable in the policy transfer. There may be a lack of interconnectedness and cohesion between the Specific and Structural Contexts. This is the level for administrative rules, legislation, compliance monitoring, and policy transfer of processes with some national scale associated. There are perceptions of the problem and goal ambitions with multifaceted strategies and instruments for addressing them. The resources and responsibilities for implementation assume a complex multisource basis for implementation. This is typically at the state or regional level of a complex social policy. Wider context. These contexts are very distal to the specific situation or setting, but may have some direct influences on the interaction. This is the interaction of the political system, judicial system, sociocultural, economic, and technological factors that may be national or international in their influence. The social and cultural foundations of education merge with the societal norms and expectations. These are also highly impacted by the political and economic systems, which in turn often result in judicial system influence with laws and court findings that ultimately influence other factors and contexts. These many external change agents and policy actors in other areas of society will also influence various aspects of the Wider Contexts. Policy analysis has also demonstrated that other contexts and interaction processes can also become influential 28 across multiple levels and broader contexts to influence the Wider Context policy agenda setting process. Application of CIT framework to SLD identification practices. Using CIT for a study of SLD identification practices can help conceptualize the complex multi-layered governmental influences on Section Four of the IDEA exclusionary clause. This is a federal mandate in the form of an education act that includes inducements to gain compliance through the various contexts of implementation. The various layers of government and regulation then transfer this federal mandate to statutes, administrative rules, policies, and procedures from a top-down approach. The Structural Context brings the policy to more of a state level as the responsibility for compliance and management of the federal mandate and inducement dollars for special education are transferred to the state and regional levels. States offer their guidance through administrative rules, compliance and monitoring, training, and support to the regional and local levels. The local Specific Context then interprets the broader context and develops policies and procedures that influence daily practice of local policy actors. At each level there have been policy actors that have influenced the contextual layer and ultimately the local level of implementation, which include the pre-service teacher preparation programs and licensure requirements. Each layer of influence then merges the interaction process of each individual on a multi-disciplinary team. There will be specialists, parents, and other professionals that may have various experiences and perceptions about the exclusionary clause. The level of coherence with intent of the exclusionary clause through the multi-level and multi-actor process can be captured with CIT. This framework can allow many aspects of the IDEA SLD exclusionary clause to be 29 identified and mapped. Context matters in policy implementation and CIT offers a complex method of categorizing factors about policy actor characteristics, contextual factors, and their interactions in a different way. Study Purpose The purpose of this dissertation study is to explore the implementation of Section Four of the SLD Exclusionary Clause of IDEA (Individuals with Disabilities Education Act) stating that students must be excluded from SLD identification if their learning problems are primarily the result of environmental, cultural, or economic disadvantage. I have examined how this policy has been implemented in the state of Oregon with a focus on the various contexts and interaction processes at play. Creswell (2014) suggests a mixed method design as a means to neutralize the weaknesses of quantitative and qualitative methods when conducted in isolation. This study used an explanatory sequential mixed method design in the form of a web-based survey with a mix of closed- and open-ended questions followed by semi-structured phone interviews to answer the following research questions: 1) To what extent and in what ways do Oregon school psychologists comply with Section Four of IDEA’s SLD Exclusionary Clause? 2) What factors influence local policy actors’ implementation of the exclusionary clause in Oregon? 3) How can Conceptual Interaction Theory (CIT) be used to describe the multi- levels of context and factors that influence the implementation of Section Four of IDEA’s SLD Exclusionary Clause? 30 CHAPTER III METHODS The methods section will first describe the research design used in the study, then the design will be unpacked by describing the various aspects of setting, participants, time aspects, unit of analysis, and the data collection instruments used. The data collection instruments will be broken down further into the individual sections that will address the three research questions presented in Chapter II. Then, the procedures that were used to administer the data collection instruments will be described. Next, the data analysis and interpretation methods used will be described. Finally, the methods section will conclude with a discussion of validity thru the triangulation method for mixed methods design. Research Design Creswell (2014) suggests that a mixed methods integrated approach is designed to provide a more comprehensive analysis of the research problem(s) and is often informed by a theory. The overall research design for this study utilized a multi-phase approach to mixed methods as described by the following design notation (Creswell, 2014): (QUANT + qual) QUAL Phase one. The first phase, was a concurrent mixed methods approach to build upon the previous findings in literature on Section Four of the exclusionary clause (Harris et al., 1988; Ochoa et al., 1997). Both of the two previous studies on Section Four used a survey approach with Harris et al. focusing on quantitative methods and Ochoa et al. focusing on qualitative methods. These two methods were combined for this study to better describe the implementation of local policy actors’ compliance efforts given 31 various factors and influences in the environments within the contexts of their situations using the CIT. Qualitative results were used to explain and gain insights into quantitative aspects of the survey data. Phase two. Phase Two consisted of qualitative follow-up interviews with six volunteer respondents that helped explore the reported factors that influence policy implementation at the local level. The follow-up interviews allowed for the investigation of the reported individual policy actors’ interaction processes and contextual factors through a semi-structured phone interview. Surprising survey results were also explored during the interviews. Setting The setting for this study was the state of Oregon. Oregon has been a relatively impoverished state with 21% of the population using food stamps, which is the third highest percentage of food stamp use in the nation (Oregon, 2015). The unemployment rate has remained above the national average for over 17 years which indicates that there are a significant number of students in public schools that are likely environmentally, culturally, or economically disadvantaged in the state of Oregon. With a fifth of the Oregon’s population school age (U.S. Census Bureau, 2015), it was important to answer questions about how disadvantages may have affected their ability to access and receive educational supports. School aged students are served in Oregon’s public K-12 education system that is divided into 19 Education Service Districts (ESD) in five regional areas. The five regions: metro, central valley and north coast, southern, central, and eastern are represented by 19 ESDs. The ESDs support and serve 197 public school districts in the 32 state. Some school districts are established to serve one part of a city, whereas, other school districts serve multiple cities or an entire rural county. Participants The participants for this study were all licensed school psychologists in the state of Oregon currently working and registered with the Teachers Standards and Practices Commission (TSPC; n = 435) at the time of the study. The demographic information about the population is limited, as only contact information was provided by TSPC. In Oregon, school psychologists are employed by individual school districts or at the ESD level; usually depending upon the size and location of the district. Smaller, and often more rural, school districts generally purchase psychological services from the ESDs in the region. Larger school districts, generally those with more than 4,000 students, employ the majority of the school psychologists in the state (Table 1). The conditions for participation in this study differed for the two phases and are discussed below. Table 1 Setting and Population of Survey as Dependent Upon School District Size No. students in school district No. school districts No. school psychologists % of population 4,000 or greater 34 308 70.8% 1,500 – 3,999 24 39 8.9% 500 – 1,499 10 12 2.8% 499 or less 0 0 0.0% ESDs 13* 76 17.5% Total 81 435 100.0% Note. *Not all ESDs or school districts employ school psychologists. 33 Survey sample. The population of Oregon school psychologists (n = 435) were invited to participate in the phase one (survey) to answer this study’s research questions. Although 119 Oregon school psychologists responded to the survey with an overall response rate of 27%, only 100 respondents were included in the survey sample (sample response rate of 23%). Of the 119 respondents, 16 respondents had not completed SLD evaluations within the last year and 3 respondents initiated the survey, but only answered the first question. The survey was anonymous. There were eight questions that targeted the descriptive statistics for school psychologists and the settings in which they worked (Table 2). There were 100 respondents that made up the sample. Due to the potentially sensitive nature of some of the questions, especially those regarding personal demographic information and compliance, respondents had the option to skip any question within the survey. Not all respondents answered every question. Table 2 shows the demographic information available for the survey sample. Eight-one percent of the respondents reported working in at least one urban school or district (n = 99). And 96% reported serving at least one designated Title One school or district (n = 99). Most respondents worked in four or fewer schools (79%; n = 100) and completed a mean of 41 SLD evaluations per year (n = 96). The sample included 8% with three or fewer years of experience, 33% with 4-8 years, 16% with 9-12 years, 12% with 13-16, and 30% with 17 years or more experience. Interview sample. The phase two interview sample was a purposeful sample that represented Oregon school psychologists from across the state (n = 6). Interviewees represented both rural and urban settings from each of the five regions identified above. 34 The central valley and north coast region was represented with two participants that also shared experiences in the metro region. Three of the six interviewees represented urban settings. Two of which worked at ESDs and another worked for a school district serving more than 4,000 students. The other three interview participants represented rural settings in Oregon. Two worked for ESDs and one worked for a school district serving between 1,500 and 3,999 students. The interview sample is considered a purposive sample that represented the population of interest. Table 2 Demographic Counts and Percentages of Survey Sample Demographic marker n % of sample Setting items Worked in ≥ one district with ≥ 4,000 students 81 82 Served at least one Title I school or district 95 96 Evaluate bilingual students 94 94 Number of schools worked in: 4 or fewer 79 79 5 or more 21 21 Individual items No. of years licensed experience evaluating SLD 0-3 8 8 4-8 33 33 9-12 16 16 13-16 12 12 17 or more 30 30 Gender Male 17 22 Female 60 78 Ethnicity Hispanic or Latino 1 1 Not Hispanic or Latino 77 77 Race White 73 95 Non-White 4 5 35 Time Aspect This study aimed to describe policy implementation in Oregon based on data collected from the experiences of practicing school psychologists. Data were collected at a single point in time, making it a cross-sectional study. Although the study design has a sequential element of two phases, the time gap between survey and follow-up interview was minimal. Generalizations about social life are more difficult with cross-sectional designs, due to the nature of having data from a single point in time (Babbie, 2013). This study has built upon prior research as a way of revisiting the phenomena of Section Four implementation by school psychologists and increasing the generalizability of findings to Oregon specifically. Unit of Analysis A unit of analysis is defined by Babbie as “the what or whom being studied” (2013, p. 97). The individual school psychologist was the unit of analysis for this study. Babbie (2013) explains that researchers “must anticipate what conclusions she or he wishes to draw with regards to which units of analysis” (p. 98). Survey and interview responses were used to capture school psychologists’ individual influences, factors used, and experiences. The intent of this study is to consider Section Four implementation from the bottom-up perspective, which was described in Chapter II. Oregon school psychologists provided this perspective as a local policy actor. School Psychologists, the unit of analysis, were used to answer all three research questions. To what extent and in what ways school psychologists are complying with Section Four (RQ1) was analyzed using individual responses that represented the population of Oregon school psychologists. The individual unit of analysis of quantitative 36 and qualitative questions was used to identify factors that influence implementation to answer RQ2. Individual responses were used to map onto the CIT to describe the various levels of context and interaction factors that influence implementation to analyze RQ3. Data Collection Instruments There were two data collection instruments in this study: (a) web-based survey questionnaire and (b) semi-structured interview guide. Both instruments were developed using CIT and results from the Harris et al. (1988) and Ochoa et al. (1997) studies as guidelines. Although the exact protocols from these two previous studies were not available for replication, careful attempts were made to build upon their results to compare their findings to Oregon, confirming and generalizing or noting contrasts, as appropriate. Dissertation committee members knowledgeable about the development of survey questionnaires and interview protocols assisted in the final development of the data collection instruments to adequately address the research questions. Survey instrument. The survey (see Appendix B) had 25 questions with item categories to address each of the three research questions as outlined in Table 3. The first question of the survey asked participants if they have completed initial SLD evaluations in the last year. This was a quality control question and discontinued the survey for any participant that had not completed SLD evaluations in the last year (n = 16). The purpose of this exclusion was to reduce measurement error due to lack of recent experience with the SLD evaluation process. The other 24 questions addressed the three research questions of the study (Table 3). 37 Table 3 Survey Questions by Item Category and Research Question Item category RQ1: To what extent and in what ways do Oregon school psychologists comply with Section Four of IDEA’s SLD Exclusionary Clause? RQ2: What factors influence local policy actors’ implementation of the exclusionary clause in Oregon? RQ3: How can Conceptual Interaction Theory (CIT) be used to describe the multi- levels of context and factors that influence the implementation of Section Four of IDEA’s SLD Exclusionary Clause? Current work setting and demographic 2, 3, 4, 6, 7, 8, 22, 23, 24 Factors for complying with Section Four 20 12, 13, 17, 18, 19, 21 15, 21 CIT context and factors 9 14, 16 5, 10, 11, 12, 13, 14, 15, 16 Work setting and demographic items. There were several work setting and demographic items included in the survey. Descriptive statistics were reported to provide information for applicability and generalizations for stakeholders and describe the study sample. Participants were asked questions about the number of schools they worked in, the size of district or ESD, and if least one of the schools or districts is considered Title One to gather information about the setting(s) the participant worked. They were also asked if they had completed bilingual evaluations in the last year and personal demographic questions reported in the Participant section of this Chapter. Two item categories need further explanation and are described below. 38 District size as a proxy for urban and rural status. Respondents were asked if they serve at least one district of 4,000 or more. This question was used to determine rural versus urban designation. Although a crude proxy for urban versus rural, it is one that is commonly used in the state of Oregon (Confederation of Oregon School Administrators, 2015) and will provide a reference for generalizing to other settings. Oregon is unique state with urban and rural districts intermixed in the central valley. Rural and urban definitions used by the Confederation of School Administrators (COSA) were used for this study. COSA uses the number of students enrolled in the district to define urban and rural. Districts with 4,000 or more students are considered urban settings and those serving below 4,000 students are considered rural settings. This designation of rural and urban may not accurately reflect the community services available. Rural settings were also coded by the size of the district, as shown in Table 1. ESDs were also coded separately depending upon the school district sizes served. Socio-economic status (SES) item. The impact of background and current setting was addressed with consideration of a question about whether respondents were currently working and completing SLD evaluations in Title One schools. The economic status of the individual respondent has little impact on the results of this survey so no questions were asked regarding personal socio-economic status of participants. Although there are many other variables that can impact income and SES experience, it is believed that school data is most relevant to the study and was asked in terms of Title One status. Factors influencing compliance. There were many questions asked to gather data about the factors that school psychologists use to comply or not comply with Section Four. These factors were influenced by the context in which the school psychologist 39 works and in the situational interactions that he or she experiences. Better understanding of these factors was the essence of research question three (RQ3) and CIT was used to categorize, frame, and describe the data for analysis. Participants were asked about their general level of compliance with Section Four; the factors they used in their decision-making process; level of pressure to find students eligible; beliefs about their role, application of Section Four in every case, if aspects of Section Four are measurable, pre-service preparation, and if Section Four should remain in the law; level of confidence that Section Four is measurable and can be applied in every case, the role of previous decisions on current decisions, and factors that may have been used to ignore or circumvent Section Four in the past. Previous studies have identified factors using open-ended questions that were then coded to identify categories of factors used in compliance. Phase One of the current study (i.e., survey questionnaire) utilized those currently identified factors to allow participants to check those that apply and allow for open-ended entry of other factors not previously identified by Harris et al. (1988) and Ochoa et al. (1997) studies. Ochoa et al. (1997) reported that 22% of their responses were unclassifiable into 1 of the 36 categories identified in their research. They cited that the responses were not discernable because they were too vague. The current study attempted to classify more of those responses using CIT as a contextual and interactive process model to describe the responses more completely. A limitation of reported by Ochoa et al. (1997) was fatigue in answering the open-ended question. The use of lists and nominal scales were also an attempt to strengthen this study by improving methods to reduce participant fatigue to 40 obtain more complete answers from respondents by providing them with the most common factors used and allow them to fill in other factors not listed. CIT context and factors items. Many of the questions asked in about the section Factors Influencing Compliance were also used to map onto the CIT theoretical framework to describe the multiple-levels of context and factors that influence the implementation of Section Four to answer RQ3. Those questions asked about pressures influencing decisions and confidence levels of application and ability to measure factors in every case. The last question of the survey is an invitation to volunteer for a follow-up interview for the second phase of the study. Interviews allowed for deeper questioning about CIT contexts and factors. See “Interviews” section below for additional details. Interview instrument. The results from the survey were analyzed for surprising results to frame follow-up questions for the semi-structured phone interview guide (Appendix C). This guide began with a statement (as approved by the Internal Review Board [IRB]) that outlined the study, described their consent options for participation and recording, the fact that their responses would remain confidential, and what would be done with the data gathered. There were three primary questions that were asked of each participant, which included two scenario questions used to gather data about characteristics associated with CIT interaction processes (i.e., motivation, capacity and power, and cognitions) and Ochoa et al. (1997) factors, as influences on decision-making about Section Four. These scenarios included a situation in which the participant was asked how they would proceed with a hypothetical evaluation case and another in which the participant is asked to recommend a candidate with potentially questionable evaluation practices. Each of these scenarios had multiple follow-up questions that could 41 be asked to clarify responses and gain further insights based on responses. The last section of the interview was based on survey results. Five questions were asked, unless respondent already addressed the content of the question in prior responses. These questions asked participants to describe the process used to comply with Section Four, what makes Section Four difficult to apply, what could make Section Four more measurable, what is getting in the way of compliance, and concluded with an open-ended question to elaborate on anything that the participant wanted the researcher to know that may not have been asked or previously addressed. Procedures The sequential mixed method design had two distinct phases (Figure 2). The first phase was the anonymous web-based survey with primarily quantitative data collection with some qualitative open-ended questions. The second phase was qualitative interviews of a purposeful sample of volunteers. The procedures for each phase are described below. Quantitative and Qualitative Data Collection Initial Analysis Qualitative Data Collection and Initial Analysis Combined Analysis and RQ Implications Figure 2. Explanatory sequential mixed method design. Survey. Dillman, Smyth, and Christian (2014) suggest that electronic questionnaires are the fastest growing form of surveying, not only in United States, but in the world. Although electronic surveys can pose some challenges for researchers, due to limitations with the population’s or sample’s access to email and their familiarity with computers (Dillman et al., 2014), these challenges were mediated by the fact that all study participants are professionals that use email and computers in their daily 42 professional practice. In fact, all participants have accessed their professional licensure through email and web-based applications. The email survey was administered from November 13, 2016 to December 16, 2016 in the state of Oregon. Each Oregon Director of Special Education at Education Service Districts (n = 19) and School Districts (n = 197) received an email introducing this study and requesting that they encourage their school psychologist to complete the survey (see Appendix D for pre-notice email). Participants received their first direct email request and link to the survey on November 13, 2016 (see Appendix E for invitation email). Because all participants have provided an email and have actively used web-based programs for completing applications and submitting documentation to TSPC, a web-based survey is an appropriate survey mode for this population. Some of the participants provided multiple emails (e.g., home and work) for TSPC; surveys were sent to all email addresses provided TSPC for the maximum amount of coverage. An email reminder was sent the end of the first week (see Appendix F for follow-up email example), thanking those that had participated and encouraging others to complete the survey. It was intended that each school psychologist received three requests or reminders: (a) direct request to complete the survey, (b) a request from their supervisor, and (c) an email reminder. It is possible that some individuals could have received additional reminders or requests if they provided multiple email addresses or if supervisors or colleagues discussed the survey. The survey was closed after five weeks with an overall response rate of 27% (n = 119); however, 16 respondents did not complete SLD evaluations in the last year and three did not complete survey questions beyond the first question. The response rate used 43 for the study is 23% (n = 100), which included only the number of responses that answered substantive questions. Participants were not required to answer questions and had the choice to skip or not answer any question. As a result, the number of respondents was reported for each question. As illustrated in Figure 3, survey results were analyzed prior to phase two and were used to develop the interview protocol as described below. Interviews. A sequential mixed methods approach helped identify the types of qualitative questions to ask participants in the second phase (Creswell, 2014). As shown in Figure 2, the semi-structured interview guide was developed after the first phase was complete and data had been analyzed. Bogdan and Biklen (2003) suggested this interview method to get comparable data across subjects, while allowing the subjects a chance to shape the content of the interviews. Questions were asked to address specific CIT and phase one derived questions, but allowed the subjects to talk freely about the contextual and interaction factors that were specific to their experiences. The interview guide consisted of three main open-ended questions that had numerous follow-up questions to allow for deeper discussions regarding the experiences of each interviewee (Appendix C). Not all questions were relevant to every interviewee; therefore, the frequency of responses to each question was identified in the results section. Survey participants were given the opportunity to volunteer for follow-up interviews and were asked to follow a link to complete contact information. The link was used to ensure that their survey data remained confidential and anonymous. There were initially 11 volunteers. An email invitation to participate in interviews was sent to all 11 volunteers on January 17, 2017 with three responding with scheduled times for interviews. A follow-up email was sent on January 25, 2017 to the other seven volunteers 44 and three additional school psychologists responded. Recorded phone interviews were conducted between January 20, 2016 and February 6, 2017 at the convenience of the interviewee. Interviews lasted about one hour each. There were a total of six interviews completed (n = 6). Data Analysis and Interpretation The mixed methods approach required data to be analyzed separately as survey and interview data (phases one and two, respectively). The data was then combined for analysis and interpretation to draw conclusions and gain a richer understanding of the results. The separate analysis and interpretation of survey and interview data will be described first, and then the general description of the combined data analysis and interpretation will be described. Survey analysis. The data collected from the survey (n = 100) was first analyzed using a Qualtrics report that provided descriptive statistics. These results were then coded, using this researcher’s judgment and knowledge about Ochoa factors and CIT framework, to map results onto a spreadsheet of Ochoa et al. (1997) categories and factors in rows and the CIT contextual factors and interactive process loop characteristics in columns (included as a supplemental file with this dissertation). The process of coding involved placing each of the survey results, or percentage of endorsement of each survey item, onto the spreadsheet according to the various possible categories and sub-categories of CIT characteristics. For example, Educational Opportunity was coded in Specific Context and nine categories and sub-categories within the interactive process loop (see the supplemental file included with this dissertation). The following categories and sub- categories were identified as representing the influence of Educational Opportunity: (a) 45 Motivation that included External Pressure and Self-Effectiveness Assessment and interactive sub-categories that represents the possible movement to Motivation through Availability of Resources for Intended Action and Opportunities and Threats; (b) Capacity and Power category that included Attribution by Others and interactive category of Strategic Value; and (c) Cognitions that included Frames of Reference and Observations of Reality and the interactive category of Data Search and Processing Capacity. This coding process was used for each of the survey results then totals were reported for each key characteristic of the CIT framework (i.e., Motivation, Capacity and Power, and Cognitions). Averages were also calculated and reported for each of the three key CIT characteristics and those equal 100%. This spreadsheet allowed percentages to be calculated for each of the CIT contextual and interactive process loop factors using the level of endorsement of each survey item in each column using their totals and subtotals. Interview transcription analysis. All interviewees (n = 6) gave verbal consent to have the interview recorded. Those recordings were transcribed verbatim and coded into CIT and Ochoa et al. (1997) categories. Transcripts were also analyzed and coded for common themes that may not have been adequately captured by other previously identified categories. Key words or phrases were searched and topics were coded or analysis. Results were used to better understand and describe survey results. Combined analysis and interpretation. Quantitative data was analyzed using Qualtrics and Excel spreadsheets with descriptive statistics. Those descriptive statistics were shared in this dissertation using tables and narrative to compare and contrast results with previous findings of Harris et al. (1988) and Ochoa et al. (1997) to answer the first two research questions. 46 The qualitative results from the survey and interviews were coded and categorized based on established categories from CIT and Ochoa et al. (1997), for this reason a single coder was appropriate with reliability of coding confirmed with triangulation of findings and multiple searches and queries using Microsoft Word and Excel. Creswell (2014) suggested developing a qualitative codebook that can be used in the study that defines the categories and includes quotes to illustrate the code. Data was collected and organized using word processing and spreadsheet computer programs (e.g., Word or Excel). A spreadsheet was developed with quotes coded into Ochoa et al. (1997) categories, key words, and comments. Transcripts were used and queried in two different ways. There was the file of all transcripts in one file used to find key words or phrases and analyze responses from single interviewees to ensure accurate interpretation. Transcripts were also coded using color and divided into themes by topics of interest. Data were cross- referenced using each of these analysis tools for comparison and interpretation. The results were reported using tables, a figure, and narrative in the final dissertation report. Validity The mix of both qualitative and quantitative methods provides a better understanding of the research problem than either type by itself, according to Plano-Clark and Creswell (2010). The mixed method approach offers both controls and potential threats to validity (Creswell, 2014). Some researchers approach the methodology of a mixed methods design as two distinct methods of quantitative and qualitative and address threats to validity in a compartmentalized manner (Creswell, 2014). Another way of approaching study validity is to embrace the strength of mixed methods and use between- and within- types of triangulation (Hussein, 2009) to address the validity. “The between- 47 method triangulation has been used for the aim of achieving convergent validity and testing the degree of external validity” (Hussein, 2009, p. 3). The within-method of triangulation is defined as the “crosschecking” for internal consistency Denzin (1978) in (Hussein, 2009). This is the multiple perspectives in data collection and analysis towards “increasing internal credibility of the research findings” (Hussein, 2009, p. 3). The approach to validity, in this study, does not assume that the quantitative and qualitative methods are mutually exclusive. There are strengths and weaknesses of each and by combining them, the weaknesses can be neutralized and benefits strengthened to improve the overall validity of the study through multiple forms of triangulation. The specific limitations include internal, external, statistical conclusion, and construct validities for this study. Triangulation. There are five types of triangulation identified by Hussein (2009) that include: methodological, data, analysis, investigator, and theoretical triangulation. “Triangulation is a validity procedure where researchers search for convergence among multiple and different sources of information to form themes or categories in a study” (Creswell & Miller, 2000, p. 126). This study includes all triangulation types as a method for controlling potential threats to validity and to strengthen study. Methodological triangulation. Methodological triangulation is the use of more than two methods in a single study of the same phenomenon (Creswell, 2014; Hussein, 2009). There are three considerations that are at the heart of mixed methods design, which have been used to strengthen this study: a) Concurrent timing to provide data for a more complete understanding of the research questions by gathering both quantitative and qualitative data from the same participants and the same time (Plano-Clark & 48 Creswell, 2010); b) The use of a qualitative priority to explain quantitative results (see “data triangulation” for further explanation) and contextual factors with qualitative data collection and analysis (Creswell, 2014); and c) Data triangulation that allows for a mix of the two datasets to utilize multiple data sources to develop a more complete understanding of the research questions. Creswell (2014) identifies potential threats to validity using a convergent approach from unequal sample sizes, different concepts or variables, and lack of follow- up on conclusions. These potential threats to validity are also addressed with the study design of a single survey. Babbie (2013) states that “survey research is probably the best method available to the social researcher who is interested in collection original data for describing a population too large to observe directly” (p. 253). The mixed methods approach will provide equal sample sizes through single administration of both open- and closed-ended questions in the same survey. The concepts and variables are cross checked throughout the survey to provide a “’mixing’ or blending of data, it can be argued, provides a stronger understanding of the problem or question than either by itself” (Creswell, 2014, p. 215). The qualitative questions will provide additional perspectives on compliance through open-ended questions regarding factors, influences, and barriers to implementation. Data triangulation. As in methodological triangulation, data is collected in a single point in time from the same participants that allows for quantitative data to be explained by the qualitative items. In this study, the qualitative data had a descriptive role in digging deeper into the contextual factors that is influencing the interaction processes of policy implementation at the local levels. The qualitative and quantitative datasets will 49 be mixed in analysis for a more comprehensive understanding of the research questions. There will be some within-triangulation of responses to validate results. There are also between-triangulation that will generate a distribution of the population’s results for aggregated responses for generalization of findings (Hussein, 2009). There is potential for measurement error of the data on sensitive questions (i.e., questions regarding the respondents level of compliance to the law). This study utilizes multiple methods to reduce measurement error as recommended by Dillman et al. (2014), that includes: (a) confidential responses, (b) the opportunity for mixed-modes (i.e., survey and interview), and (c) the ability to answer sensitive questions in written format. Analysis triangulation. Methodological triangulation methods and the three considerations noted above also utilizes analysis triangulation (Hussein, 2009) for validation purposes. Considering quantitative and qualitative data in the same study allows the two methods to promote the validation and completeness of the dataset for analysis. I will also compare results of some of the data analysis to prior studies for a cross-method of triangulation (Hussein, 2009) utilizing the strategy of convergent validity and testing the degree of external validity with prior studies from the literature review (Harris et al., 1988; Ochoa et al., 1997). Investigator triangulation. Investigator triangulation is typically the use of multiple investigators at any stage of the study (Hussein, 2009). In the case of a dissertation, there are oversight investigators through the use of proposal approval and defense. The other use of investigator triangulation within the study design is the use of committee experts in the design of the survey protocol and in the consistency coding checks during data analysis. The close collaboration of an external reviewing committee, 50 (e.g., a dissertation committee) adds credibility to the study when the external reviewing process occurs throughout the study (Creswell & Miller, 2000). Theoretical triangulation. The final type of triangulation is theoretical, meaning the use of multiple theories for the purpose of supporting or refuting findings and seeing the problem through multiple lens (Hussein, 2009). The CIT theory is a combination of multiple theories to produce a complex theoretical framework for better understanding complex social policy. I would argue that this is evidence of theoretical triangulation. 51 CHAPTER IV RESULTS This dissertation study aimed to answer three questions. RQ1 evaluated the extent and ways Oregon school psychologists complied with Section Four of IDEA’s SLD Exclusionary Clause. RQ2 assessed factors that influenced local policy actors’ implementation of the exclusionary clause in Oregon. Finally, RQ3 appraised how Contextual Interaction Theory (CIT) could be used to describe the multiple levels of context and numerous factors that influenced the implementation of Section Four of IDEA’s SLD Exclusionary Clause. The Chapter begins with descriptive statistics of the samples for both the survey (phase one) and interview (phase two) of the study. Next, the key results are reported as they pertained to each of these three questions. The chapter concludes with a summary of results. These study results explicitly aim to build off of earlier prior research (Harris et al., 1988; Ochoa et al., 1997) and notations to these seminal works are made as appropriate. Descriptive Statistics of the Sample Oregon school psychologists were surveyed (n = 100) and interviewed (n = 6) to obtain the results reported in this chapter. There were 119 responses to the survey (n = 435, 27% response rate). Of the 119 responses, 100 respondents had completed initial evaluations for SLD in the last year and were included in the current study’s results. Respondents had the choice not to answer individual questions on the survey at their discretion; as a result, not all questions have 100 responses. There were 11 volunteers for follow-up interviews; six were conducted. 52 Of the 100 survey respondents, 81% reported working in at least one urban school or district with 96% serving at least one designated Title One school or district. Most respondents worked in four or fewer schools (79%) and completed a mean of 41 SLD evaluations per year. The sample included 8% with three or fewer years of experience, 33% with 4-8 years, 16% with 9-12 years, 12% with 13-16, and 30% with 17 years or more experience. Interviewees represented the five regions in Oregon (i.e., metro, central valley, coast, southern, and eastern. Both ESDs and individual school districts were represented in urban (3 interviewees) and rural (3 interviewees) settings. RQ1: Extent and Ways of Compliance RQ1 evaluated the extent and the ways Oregon school psychologists complied with Section Four of IDEA’s SLD Exclusionary Clause. Survey results describe the extent of compliance (Table 4), including the frequency of exclusion. Survey and interview results describe the ways used to comply that include: (a) method of evaluations used, (b) consideration of Section Four, and (c) opinion of whether Section Four should remain in the law. Findings on the extent and ways of compliance from survey results. The primary result that answers RQ1 was the level of compliance from survey results. The study survey asked respondents about their general level of compliance with excluding students based on Section Four. Of the respondents, 87% rarely or sometimes complied and 12.7% of school psychologists reported meeting the expectation with 100% compliance (Table 4). Respondents estimated that they completed approximately 3,956 SLD evaluations in the last year with mean of 41 per school psychologist (n = 96). The survey asked 53 Oregon school psychologists how many times in their career they (or another team member) had communicated to parents that their child’s learning problems resulted primarily from one of the exclusionary factors of Section Four with the following means: environmental disadvantage, 3.74; cultural disadvantage, 4.03; and economic disadvantage, 0.87. The most common response in all categories was they had never communicated exclusion based on Section Four. With a combined mean of 8.64 for all three factors, Oregon’s frequency of exclusion is similar to the national mean that Harris et al. (1988) reported; however, the majority of responding Oregon school psychologists have never excluded a student based on Section Four. The method of evaluation, discussed below, helps explain the ways that IEP teams are attempting to comply. Table 4 Oregon School Psychologist’s Level of Compliance with Section Four General level of compliance % endorsing response choice (n = 79) Always 12.7 Sometimes 43.0 Rarely 29.1 Never 15.2 Regularly tried to comply 55.7* Routinely ignored or circumvented the law 44.3** Never ignored or circumvented the law 49.4*** Note. Italicized levels of compliance categories reported by Harris et al. (1988). * Author derived percentage from the “always” and “sometimes” categories. ** Author derived percentage from the “rarely” and “never” categories. *** Survey question 25 (n = 77). 54 The most common method of evaluation reported on the survey was Pattern of Strengths and Weaknesses (PSW), with 94% of the sample using either primarily PSW (64%) or a combination of intervention data and PSW (30%). The other 6% of the sample reported using a form of Response to Intervention (RTI) or Multi-Tiered Systems of Support (MTSS) to evaluate students without PSW as a component of the evaluation. Regardless of method, the IEP team considers the data collected during the evaluation process at an IEP meeting, where the determination of Section Four is conducted. Findings of ways of compliance from interview results. Interviews primarily focused on the ways of compliance, rather than on the extent of compliance of Section Four. The six school psychologists interviewed discussed the team process used in consideration and determination of Section Four and their thoughts about whether Section Four should remain in the law. These two key findings are described below. Consideration of Section Four. The team process for consideration of Section Four was a common theme in the experiences of the school psychologists interviewed. None of the interviewees discussed experiences of excluding students based on exclusionary clause or having discussions during an IEP meeting that could possibly lead to exclusion (n = 6). The IEP meeting process and team discussion of the exclusionary clause were described as: “They read through it and they go ‘the student’s lack of achievement is primarily the result of’ and they’ll say ‘it’s not because visual, hearing, motor impairment’ and they usually stop right there. They don’t really even finish the rest of it.” Another description was “I quickly read off the factors and check no…. I would say that the team is considering this is not the primary reason that the child is not achieving. That’s how it would go.” Another said, “In my experience, there has not been 55 a mention of attendance, language, culture, lack of instruction; none of that. It’s not in the report and it doesn’t get mentioned. Five out of six of interviewees (n = 5) described that a multi-disciplinary team was present, but when asked whether they felt the determination of the exclusionary clause was made by the team, one revealed that “I guess if you mean it’s truly a team decision in the sense that they have actually read that entire statement and considered every one of those phrases? I would say no.” Another school psychologist described the Section Four determination as a “check no and go process” and was asked if she had ever excluded a student with learning difficulties based on exclusionary factors, the response was “No, I don’t think I’ve ever, ever, ever marked that.” Questioning if Section Four should remain in the law. This study asked participants about their opinions on whether the clause should remain in the law and Oregon’s SLD Eligibility Statement. The survey results indicate that 52% of the respondents agreed or strongly agreed that Section Four should remain as a factor in the decision-making process for SLD eligibility. Generally, those interviewed also had mixed emotions about whether it should stay in the law or not. They all agreed (n = 6) that the concept of consideration of these factors were important. One interviewee summed up the dilemma that Oregon school psychologists currently face: I guess I would be wary of taking that out of it, because I do think it’s an important factor but in practice, it’s not being utilized very well. If nothing is going to change, I’m not sure it’s serving the purpose it’s intended. A general consensus was evident in the interviews—they want a better and more consistent way to define and measure the exclusionary clause factors. One recommendation was to “consider separating those different factors out, and having clear guidance on thresholds, more explicit, like this is what we mean and this is what we 56 would consider.” He felt that it “should be within our skill set,” but acknowledged the need for additional training on the clause at all levels of implementation to make the law more compliable. Respondents reported a pervasive practice of ignoring Section Four. RQ2: Factors of Implementation in Oregon RQ2 assessed the factors that influenced local policy actors’ implementation of the exclusionary clause in Oregon as defined by Ochoa et al.’s (1997) seminal survey of 648 school psychologist in 8 states. There were 37 key factors identified by school psychologists in five categories (as represented in Table 7). These factors and categories will be referred to as Ochoa Factors and Ochoa Categories. Key results are reported in the areas of: (a) implementation barriers to compliance, (b) operational definitions of the exclusionary clause, and (c) factors that are used in Oregon to determine Section Four. Several additional key factors that influence implementation not uncovered in Ochoa et al.’s study were found: (a) level of acculturation, (b) pre-service, (c) context of implementation, and (d) personal beliefs of the local policy actor. Findings on implementation factors from survey results. There are many factors that influence the implementation of Section Four in Oregon. They key findings of the survey include: (a) five barriers to compliance for more than half of the respondents; (b) eight defining characteristics of environmental, cultural, and economic disadvantage with 75% or greater respondent agreement; and (c) 12 factors used by 80% or more of the sample to determine Section Four. Each of these key findings are discussed below. Implementation barriers to compliance. The survey offered further detail as to why school psychologists are not complying with Section Four. Table 5 shows that the 57 lack of clarity of state and federal guidelines is the greatest barrier to implementation. Four other barriers were identified by more than half of respondents. Table 5 Why Oregon School Psychologists Ignored or Circumvented Section Four Survey responses n % endorsed item Believe state and federal guidelines are unclear 87 94.3 Don’t think it is measurable 88 85.8 Unclear about role and responsibility in determining Section Four 87 77.0 Often or sometimes feel pressure to find kids eligible, even though Section Four exclusion should be applied 81 74.1 Not confident that an IEP team can apply Section Four in every case 82 69.5 Past practices often or sometimes influence current decisions 79 45.6 Lack access to the resources to adequately measure the factors 77 39.0 School/District procedures make application difficult 77 33.8 Lack adequate training to apply Section Four in all situations 77 33.8 Think Section Four is discriminatory 77 22.1 Don’t completely understand Section Four or how to apply it 77 16.9 Definitions of environmental, cultural, and economic disadvantage. The survey proposed 17 characteristics that may be used to define Section Four as presented in Table 6. Respondents were given the opportunity to add characteristics; however, none were added. Eight of the seventeen characteristics had averages above 75% in the agree or strongly agree categories and fewer than 10% of the respondents reported being unable to determine if they agree or disagree with the statement. This level of endorsement suggests that these characteristics are generally accepted as a defining characteristic for each of the terms and are highlighted in bold for emphasis in Table 6. 58 Table 6 Descriptive Statistics for the Level of Endorsement for Defining Characteristics of Section Four Section Four characteristic n M* SD % SA/A % DK Environmental disadvantage includes: 1. Lack of early development/social childhood experiences 80 3.20 0.91 91.25 5.00 2. Inadequate health care 80 3.08 0.89 88.75 5.00 3. Lack of appropriate instruction 80 2.98 0.97 78.75 5.00 4. Circumstances such as medical conditions, neglect, abuse, violence, mobility, etc. 80 2.93 1.17 83.75 11.25 5. Inadequate parenting or supervision 80 2.96 1.15 82.50 10.00 6. Exposure to toxins (i.e., lead, water pollutants, etc.) 79 2.57 1.15 70.88 12.66 Cultural disadvantage includes: 1. Belief, customs, etc. of a particular society or group that differs from the majority group and impacts the learner’s perspective about the benefits of learning or attitude toward reading, writing, math, listening, or concentrating 80 2.86 1.08 82.50 8.75 2. Results from language barriers 80 2.89 1.04 76.25 6.25 3. Differences in social capital 80 2.36 1.33 60.00 18.75 4. Experiences that results inadequate access to educational readiness or a delay in development 79 2.87 1.16 82.30 11.39 5. Results from differences in religious beliefs and customs 80 2.28 1.17 47.50 12.50 Note. Bold characteristics have less than 10% of the respondents unsure of the definition. Means based on a four-point scale, where: 4 = strongly agree (SA), 3 = agree (A), 2 = disagree (D), 1 = strongly disagree (SD); DK = Didn’t Know. 59 Table 6 (continued) Descriptive Statistics for the Level of Endorsement for Defining Characteristics of Section Four Section Four characteristic n M* SD % SA/A % DK Economic disadvantage includes: 1. Poverty 82 3.39 0.49 100.00 0.00 2. Social class 82 3.01 0.96 85.37 6.10 3. Combination of variables that include parents’ occupation, level of education, income, wealth, and place of residence 81 3.17 0.84 86.00 2.47 4. Social marginalization 81 2.78 1.25 76.55 13.58 5. Negative social segregation 81 2.70 1.27 74.08 14.81 6. Institutional discrimination 80 2.59 1.34 70.00 16.25 Note. Bold characteristics have less than 10% of the respondents unsure of the definition. Means based on a four-point scale, where: 4 = strongly agree (SA), 3 = agree (A), 2 = disagree (D), 1 = strongly disagree (SD); DK = Didn’t Know. All six of the characteristics for both Environmental and Economic Disadvantage had averages of endorsement of greater than 70%; however, only three items each had less than 10% that didn’t know if the item would be a defining characteristic (Table 4). Cultural Disadvantage had two of the five characteristics that had lower endorsement and greater variance. Respondents endorsed social capital and religious factors less frequently as defining characteristics of culture. It appears that Oregon school psychologist respondents endorsed eight characteristics for Section Four terms. Ochoa Categories used in determining Section Four. Results reported from the survey demonstrate that in practice, responding Oregon school psychologists used the Ochoa et al. (1997) factors in their evaluations to different degrees: 26% used Language Instruction and Language Related Factors, 22% Assessment and Procedural Safeguards, 19% Educational History Factors, 18% Family and Home Factors, and 15% General 60 Education Factors. The Survey asked respondents to rank order these six factor categories in order of importance in making the determination of environmental, cultural, or economic disadvantage as the primary factor in the student’s learning challenges, and 70% of survey respondents selected Language Instruction and Language Related Factors as highest ranking. Interestingly, General Education factors, which includes attendance, was rated as least important or useful category with 9% of the respondents giving it the highest or second highest ranking. However, this is inconsistent with frequency of use and importance that is placed on it in practice as described below. Table 7 also shows that there are 12 Ochoa et al. (1997) factors that are endorsed as criteria used to determine Section Four by at least 80% of the Oregon School Psychologists who responded to this study. There were four Family and Home Factors and four Language Related Factors that were identified as specific criteria used in the determination of Section Four. Only one of the eleven factors identified by Ochoa et al. (1997) were endorsed as useful by at least 80% of respondents under the category of Assessment and Procedural Safeguards. Most of the school based criteria as determined by Educational History and General Education Factors had percentages of endorsement in the 70s or above; however only three made the threshold of greater than 80%. Attendance as the most influential factor. Respondents referred to attendance 88.31% of the time, making it the most used factor in the survey. School attendance is part of the General Education Factors category (Ochoa et al., 1997), which in comparison to the other factors was the least selected overall (15% General Education Factors), as shown above. 61 Table 7 Endorsement of Criteria Used by School Psychologists in the Determination of Section Four Exclusionary Clause in Oregon Factors for determining Section Four % endorsed criteria (n = 77) Family and Home Factors (18%)* Review of sociological information and family history 87.01 Frequency of moves / migration 87.01 Parent interview / info 85.71 Length of time in U.S. 85.71 Sibling comparison 66.23 Home visit 16.88 Received free lunch at school 16.88 Language Instruction / Language Related Factors (26%)* Years of English instruction 87.01 Received instruction in bilingual program 84.42 Years in ESL instruction 83.12 Progress in ESL instruction 80.52 Language dominance/ proficiency assessments 75.32 Home language 72.73 Comparison to students of similar backgrounds 70.13 Assessment and Procedural Safeguards (22%)* Observation of student 84.42 Performance section/nonverbal IQ 72.73 Comparison of scores in English and native language 71.43 Team decision process 64.94 Adaptive behavior of student 62.34 Bilingual evaluator 62.34 Interview student 61.04 Assessment and procedural safeguards 59.74 Student's work samples 59.74 Use of interpreter 50.65 Severe discrepancy in both English and native language 42.86 Note. Ochoa et al. (1997) identified the factors and categories shown. Individual factors within each Ochoa et al. category are presented in order of percentage (highest to lowest) employed by Oregon school psychologists in their SLD evaluations to answer Section Four. *Percentages were calculated as counts of the level of endorsement from the overall responses by coding their distribution across both Contextual and Interactive Process Loop factors of the CIT, calculating category subtotals, and then dividing each category subtotal by the overall factor totals. 62 Table 7 (continued) Endorsement of Criteria Used by School Psychologists in the Determination of Section Four Exclusionary Clause in Oregon Factors for determining Section Four % endorsed criteria (n = 77) Educational History Factors (19%)* Educational history/ Educational opportunity 87.01 Time in American schools 79.22 Pre-referral interventions 79.22 Extent of schooling in native country 76.62 History of difficulties (even in native country) 66.23 General Education Factors (15%)* School attendance 88.31 School records 84.42 Teacher report/ interview 77.92 Professional judgment 74.03 Physical/Medical data 72.73 Processing deficit present 50.65 Note. Ochoa et al. (1997) identified the factors and categories shown. Individual factors within each Ochoa et al. category are presented in order of percentage (highest to lowest) employed by Oregon school psychologists in their SLD evaluations to answer Section Four. *Percentages were calculated as counts of the level of endorsement from the overall responses by coding their distribution across both Contextual and Interactive Process Loop factors of the CIT, calculating category subtotals, and then dividing each category subtotal by the overall factor totals. Findings on implementation factors from interview results. The six school psychologists interviewed differed in school or district context and background, but described similar experiences that could be coalesced around several key themes: (a) five barriers to compliance that influence implementation of Section Four in Oregon; (b) the use of pre-referral interventions in the evaluations, regardless of evaluation method used; (c) multiple uses of attendance data; (d) definitions of Section Four; and (e) other identified factors that influence Section Four implementation. These five key themes from the interview findings will be discussed in the following sections. Barriers to compliance. When asked about what hinders/prevents compliance with Section Four, interviewees (n = 6) identified issues with operational definition of the 63 clause, measurement, desire to help kids overriding exclusion, uncertainty of role and responsibility, and external pressure as barriers to compliance (Table 8). These barriers to compliance and implementation were identified from coding the interview transcripts into common themes. The coding structure resulted from both the themes that emerged from analyzing transcripts, but was also informed by survey results. Interviewees were asked, what do you think is getting in the way of compliance?” At least four of the six interviewees discussed aspects of each of these themes and notable quotes are shown in Table 8. Measurement Issues were the most prevalent with all six interviewees mentioning this as a barrier in implementation of Section Four. Lack of Operational Definitions and Desire to Help Kids were themes that were identified by four of the six respondents. The last two themes (i.e. Uncertainty of Roles and Responsibility and External Pressure) were discussed by five of the six interviewees. One of the school psychologists discussed the issues and recommendations shared by the majority of the interviewees (Table 8) regarding the clarity of state and federal guidelines as the overall barrier to implementation: The whole problem with SLD is that it is done so inconsistently when thinking of Patterns of Strengths and Weaknesses; and how do we measure social/emotional and background piece? I think that at least coming up with a more uniform type of consistency policy about what the criteria is…maybe with a little bit of flexibility for teams to make decisions with their professional judgment. I think that is important. I think that if they are too strict…that could be equally frustrating, but I almost feel that right now there is very little legal guidance out there. It is very loosy goosy and can be subject to all sorts of interpretation in every single case. So, I think I’d be looking for consistency in that. But how to turn that stuff into something that is measurable? I’m not sure if that is possible. 64 Table 8 Notable Quotes Regarding the Barriers to Compliance as Identified by Oregon School Psychologists Barrier to compliance Notable quotes Lack of operational definitions The lack of our understanding of what the definition is…. …the lack of knowledge of each separate factor…or how to define each factor …it would not be difficult to apply if somebody would define them with cut-offs Measurement issues …how do you measure that? Once you do measure it, how to do you interpret that? The entire team is still making guesses… I get uncomfortable with those questions because we don’t have good measures for it. …lack of way to measure it adequately. …coming up with a more uniform type of consistency policy about what the criteria is… Desire to help kids It’s like we’re just going to qualify them because they need the help. …we’ll get help to the kids no matter what we call it …the will of the group to make the kid eligible …most people are aware of cultural, environmental, and economic disadvantage in their minds, they’re like…’well, that’s why that kids needs special ed. Even more’ Uncertainty about roles and responsibilities …really depends on the school …different types of rules in each district…it’s not like I get to decide on what I think is best practice …political sticky wicket …I really try to guide teams …people really want a black and white decision. That probably stems from a lot from fear of lawsuits Pressure for ignoring Section Four …a lot of pressure from teachers and administers because a lot of times special education is the only game in town when it comes to helping students …so much pressure from a team ....very uncomfortable for an individual or a team …The pressure to get them to qualify is high because there aren’t a lot of alternatives for support and help …they’re just ignoring it or they’re going out of their way not to research it and figure it out 65 The interviews revealed 100% of the school psychologists (n = 6), identified the need for a survey or measure that would define and quantify the factors used to determine Section Four. Some school psychologists suggested a survey or rubric with three levels of impact to frame the factors and influences for the IEP decision. All six of the school psychologists recognized the pitfall of having another measure with cutoffs that might take away from professional judgment, but also welcomed the idea of having something to help quantify and frame the discussion. Pre-referral interventions (RTI or MTSS). Five out of six Oregon school psychologists referred to some form of pre-referral intervention to “compare how the student is performing compared to similar peers,” especially when considering Language Related Factors. One interviewee stated, “I think we're putting more time into it in the pre-referral process,” when discussing the consideration of culture and language. The use of the pre-referral process with interventions was referred to 48 times by interviewees during the six interviews conducted. One school psychologist implied that the Response to Intervention (RTI) process could be used as a mitigating factor for determining the impact of disadvantage: You can't control what goes on in that student's environment when they're not at the school. You can't change a lot of that stuff. What you can change is the instruction curriculum and the environment. Even though family and economic disadvantage is important for understand and maybe developing empathy especially for teachers that might have a hard time, or are having a challenging time. Those are the things that you can impact the least. What you can impact is the instruction curriculum and environment. There was agreement that the interventions the student received at school was important; another school psychologist said, “As far as environmental and economic disadvantage I would say that's being pushed down by Oregon RTI. You know, focus on the Intervention not the Learner.” Progress monitoring and intervention was mentioned 66 often in the interviews, even though the school psychologists predominately used PSW as the model of evaluation; they discussed it as a method for determining the impact of disadvantage in the area of attendance as well. Attendance as an influential factor. Five out of the six interviewees indicated that attendance was the most influential factor in considering the exclusionary clause. School psychologists use attendance as an indicator for multiple purposes with wide implications: for example, one psychologist stated that “even if a kid had missed a lot of school, if the student needs special education, then they get it.” Another interviewee reported using attendance differently, stating: I think that the attendance piece with Specific Learning Disability is huge because for every day that the miss, it takes a couple of days to catch up and so if you are missing even 20 days the potential impact is more than just those 20 days. All school psychologists interviewed (n = 6) use attendance as a measure of access to education and three of the six also use it as a measure of dysfunction in the home. An interviewee described the use of attendance this way: I think one of our first go to's [sic] here is probably attendance. I think when I filled out your survey to begin with I think it was so interesting because it's been so hard to quantify the family home life scenario. I think that's a huge gap in our identification process and that goes right along with attendance. If a student is really having a difficult time at home with social emotional issues at home and then but their attendance is really low and it's low because of those issues I wouldn't make that necessarily an exclusionary factor. It just really depends on the student and how they learn and how we can try to kind of parcel out, is this a lack of exposure to education or is this something deeper? I mean I think that's super challenging. I don't think we'd ever be 100% right in any way that we assess and make it all work. Definition of environmental, cultural, and economic disadvantage. Defining Section Four was a challenge for many of the school psychologists during the interviews. Two school psychologists interviewed stated that they didn’t know how to define the 67 exclusionary clause. When prompted, one of these interviewees defined the exclusionary clause as the following: Other factors that would suggest that the challenges we're seeing at school, whether it's academic growth or behavior challenges, but I guess specific to SLD, this is academic challenges might be explained or might be the kind of a result of economic or cultural disadvantage versus a true organic learning disability. The remaining three interviewees provided a list of factors that could be used to compare students to another group (Table 9). Responses were matched to the defining characteristics identified in the survey (those factors with 75% or greater respondent agreement displayed in Table 6) and are shown bolded in Table 9. However, none of the school psychologists interviewed defined Section Four in measurable terms that captured all three aspects of the exclusionary clause. Interviewees discussed previously unidentified factors that could impact Section Four implementation. Coded interview transcripts were analyzed for themes that matched factors (as described above) to explain further the survey results. In addition to the Ochoa et al. (1997) factors and acculturation (an additional factor identified in survey responses), other themes were identified as possible influences to Oregon implementation of Section Four. School psychologists discussed: (a) advice they would give pre-service institutions to better prepare school psychologists to apply Section Four, (b) contextual factors, and (c) personal beliefs that influence understanding and implementation. Acculturation. Acculturation was mentioned by three of the six school psychologists interviewed. Several school psychologists suggested using an acculturation scale as a measurement tool to better quantify Section Four components of environmental and cultural disadvantage, one interviewee stating, “I think that Collier has tried with their acculturation scales and levels and they have tried to quantify it a little bit better. 68 Table 9 Responses to the Interview Question: How Would You Define the Exclusionary Clause? Section Four characteristic Quotes Environmental disadvantage includes: Lack of early development/social childhood experiences …if they have been privileged and they have had access to things like libraries and museums…family uses a lot of language…exposure to culture and places Inadequate health care …health issues Lack of appropriate instruction …access to adequate instruction …being absent…lack of instruction Circumstances such as medical conditions, neglect, abuse, violence, mobility, etc. …chaos in the kid’s background …abusive home…TVs on all the time, even late at night …basic needs met Inadequate parenting or supervision NA Exposure to toxins (i.e., lead, water pollutants, etc.) NA Cultural disadvantage includes: Belief, customs, etc. of a particular society or group that differs from the majority group and impacts the learner’s perspective about the benefits of learning or attitude toward reading, writing, math, listening, or concentrating …how they use language at home Results from language barriers …how strong they are in their native language Differences in social capital NA Experiences that results inadequate access to educational readiness or a delay in development …they come with thousands and thousands more words and vocabulary to start …before they are even able to access anything academic and learn they need to have some sort of a meaningful social emotional steadiness Results from differences in religious beliefs and customs NA 69 70 Table 9 (continued) Responses to the Interview Question: How Would You Define the Exclusionary Clause? Section Four characteristic Quotes Economic disadvantage includes: Poverty … haven’t had a lot of stress and trauma in their lives because their parents haven’t been stressed trying to pay their bills…exposure to stressful environments that literally change the brain …home is one main room Social class …fewer tools in the toolbox Combination of variables that include parents’ occupation, level of education, income, wealth, and place of residence …parent level of language use Social marginalization …economics and environmental often go together Negative social segregation …behavior of the kid …our schools are more segregated than we like to think…the majority of your students look like that but they’re only identifying a certain percentage Institutional discrimination NA Sometimes it led to more questions than answers. It would be overwhelming, all the information.” Another interviewee suggested that the state could use ELPA21 (Oregon state assessment of English learner student performance) to “come up with some sort of cohort that we can compare using things like the acculturation quick screen” as a measure of the level of acculturation. The other interviewee suggested that in larger districts they have enough students to develop cohorts for comparison and acknowledged that it is “more the linguistic side of it that I think we're more in tune with” and able to measure. Pre-service preparedness. Interview respondents were asked what advice they would give their pre-service institutions to better prepare new school psychologists to 71 answer the Exclusionary Clause. One interviewee felt that with the support of Oregon RTI, through training and guidance, we should “keep doing what you're doing” and focus on “the mission to help the kids and put the learner at the very end.” He felt that by putting the students’ response to interventions as the priority and considerations of environmental, cultural, and economic disadvantage last, we could continue with status quo. Two other school psychologists suggested focusing more time on exposure to differences to reduce bias and increase experience with “every type of learning environment…in every single grade level, many different districts, many different socioeconomic status environments.” Individual advice to other pre-service school psychologists from one interviewee was to: really educate yourself on whatever cultural background you have [at your school] and any others so if things come up you can kind of say that makes sense because that is the culture that this student came from or no it really doesn’t make sense in either culture. The other three interviewees suggested more in depth instruction and time dedicated to factors and determination of Section Four. Contexts. The wider and structural contexts of Section Four make local implementation more difficult. The operational definitions of environmental, cultural, and economic disadvantage are absent from the federal law and state regulations. There are no readily available state level trainings or white papers on how to apply Section Four. A veteran school psychologist in the field stated, “I was in graduate school so long ago, and I’d never seen any kind of workshops or conferences or presentations on this at all.” The Specific Contexts also matter, according to interviewees. Survey respondents report that they often or sometimes feel pressure to find a student eligible, even with exclusionary factors present (74%) and that past practice influences current decisions 72 (46%). When asked about the resources that are available to them to apply Section Four, four out of six interviewees reported collaborations with other school psychologists as the only resource they have, one identified accessing Oregon RTI project, and one identified the ability to contact ODE if they had a question. Personal beliefs. Interviewees stated the reason that they believe cultural aspects are more measurable than the others are because they use language-related factors as proxy measures of culture. One school psychologist explained it this way: “I think the cultural disadvantage is a broad area, and the others are more family based. We have less to look at when we look at impacts. Yes, it is a bit of stereotyping.” Another interviewee described cultural factors as language development when considering Section Four: We're just trying to see, did they get some sort of comparable instruction, then as far as cultural, looking at the language development, access it, supports for that language development. In my practice, again, I feel like we just don't dig into it. There's a really, in my opinion, a low bar to even mark that box, which happens so rarely. There's no really quantitative consistent way of looking at that. I would hypothetically answer it the way I would in a practical world, which is unless there's something so glaring, the bar so low, we don't do it in what I would call an equitable way. RQ3: Description of Influences to Implementation Using CIT RQ3 asks how can CIT be used to describe the multiple levels of context and factors that influence the implementation of Section Four of IDEA’s SLD Exclusionary Clause. Results from the survey and interviews were coded using the established characteristics and components of CIT (Bressers, 2007). Contextual Interaction Theory was used as a theoretical framework to allow for better understanding of the interaction process and connections between Ochoa et al. (1997) factors. Findings on implementation factors using CIT from survey results. The survey results were mapped onto CIT contextual and interactive process loop categories 73 and factors as show in the supplemental file included with this dissertation. Each of the key characteristics—Cognitions (41.92%), Motivation (35.84%), and Capacity and Power (22.24%)—was reported with associated supporting components within the interactive process loop. The percentage of use reported by Oregon school psychologists on the survey and influencing sub-categories within the CIT interactive process loop and contexts are illustrated in Figure 3. Figure 3 indicates that Cognitions (41.92%) had a higher level of endorsement and possible influence than other characteristics. Cognitions is the policy actors’ process of filtering observations, interpretation of reality, and prior knowledge / experiences to make sense of current information. Motivation, which is a characteristic of one’s own goals and values, external pressures, and assessment of self-effectiveness, has a level of endorsement of 35.84%. Capacity and Power characteristics had the lowest level of endorsement on the implementation of Section Four (22.24%), according to survey results. The most highly endorsed individual interactive characteristic was Interpretation (10.96%) and the least endorsed characteristic was Resources Available and Accessible (3.43%). Focusing of Attention was endorsed at 7.0%, and was the most highly endorsed sub-category linking key characteristics to each other. Key RQ3 findings on the implementation factors using CIT from interview results. Interview responses about influences to the implementation of Section Four at the local level differed from survey results. Five out of six interviewees described influences of Capacity and Power as the most influential factors. The perception of how decisions are made about Section Four were primarily that they rarely occur. A statement 74 that was representative of the interviewees’ experiences in IEP team meetings and the decision-making process was: The will of the group to make the kid eligible. A lot of times their minds already made up. Especially if the numbers are all there. If you're doing pattern of strength and weakness. I guess if you mean it's truly a team decision in the sense that they have actually read that entire statement and considered every one of those phrases? I would say no. Figure 3. Survey results CIT key characteristics—determined by level of endorsement of Ochoa et al. (1997) factors from Oregon school psychologist’s survey results. Five of the six interviewees indicated there was no active engagement of IEP team members in considering Section Four during the IEP meetings. There were mixed responses to the question of who truly holds the power in the IEP team decision-making 75 process: (a) whole team (n = 1); (b) school psychologist (n = 2); (c) school staff, including general/special education teachers (n = 2); and (d) district administrator (n = 1). One school psychologist described the pressure involved in making recommendations about Section Four, even when he holds the power to do so, as: If I were just a fire brand I could stand up and go no we got to consider this and but that could possibly mean losing my job or having that district go hey we don't want [so and so] to come be our school psych anymore because every time we're at a meeting he brings up this part about environmental, cultural, and economic disadvantage. You know? During the interviews with school psychologists, several reported that they believe in the teaming process, but just don’t believe that IEP teams in Oregon understand this clause, how to measure it, how to apply it, or the advantages of excluding when students need assistance. Summary of Key Results IDEA’s SLD Exclusionary Clause states that students must be excluded from SLD identification if their learning problems are primarily the result of environmental, cultural, or economic disadvantage. I have examined how this policy has been implemented in the state of Oregon with a focus on the various contexts and interaction processes at play using an explanatory sequential mixed methods approach. This study endeavors to answer three research questions to better understand the implementation of Section Four in Oregon. Third generation policy analysis “contributes important information about implementers as learners” for implementing complex policies (Fowler, 2013, p. 254).The main findings of this study are summarized below. RQ1: Extent and ways of compliance. Oregon school psychologists who responded to the survey have an extremely low compliance rate for the Exclusionary Clause with 87% rarely or sometimes complying, which suggests inconsistent 76 implementation of the Section Four mandate in Oregon. The majority of the school psychologists in this study sample, reported not having excluded a student based on Section Four, although the overall combined mean of 8.64 is close to national average (Harris et al., 1982). The most common method of SLD evaluation is PSW (94%); however, most school psychologists prefer to include some progress monitoring or intervention data whenever possible (83%). Interview results suggest that measurement issues and external pressure are barriers to compliance. There are mixed feelings by survey participants about whether the Section Four should remain in the law (52% agreed or strongly agreed), but there was agreement among interviewees that “it’s not begin utilized very well.” RQ2: Factors of implementation in Oregon. As outlined in Table 7, there were 12 factors that 80% or more of the sample of Oregon school psychologists used as criteria for determining Section Four. There were factors in each of the following categories: Family and Home (4), Language Instruction and Language Related (4), Assessment and Procedural Safeguards (1), Educational History (1), and General Education (2). Attendance was the single most highly used factor (88.31%) and will be discussed in more detail below. In practice, survey respondents used Language Instruction and Language Related Factors most often (26% of all factors used) in the determination of Section Four. The evaluation process utilized the pre-referral process and interventions to measure impact of Language Related Factors (83%). The single most often used factor was attendance (88.3%), although respondents ranked the importance of this category the lowest in the final determination (only 15% of school psychologists rated importance in the highest or 77 second highest category). Additional factors were identified in the study as influencing Section Four implementation, beyond those used by Ochoa et al. (1997), which included: acculturation, pre-service preparedness, contextual factors, and personal beliefs of local policy actors. This study identified several key barriers to implementation as well. The greatest barrier identified in the study was the belief that state and federal guidelines are unclear for Section Four (94.3%). Other barriers to compliance were identified in the study Survey: unclear about role and responsibility in determining Section Four (77.0%); pressure to find kids eligible, even though Section Four exclusion should be applied (74.1%); lack of preparedness to answer Section Four (73%); lack of confidence that Section Four can be applied in every case (69.5%), don’t think Section Four is measurable (51.9%); and that past practices often or sometimes influence current decisions (45.6%). These same barriers were echoed in the study’s interviews with one out of six school psychologists indicating that it is truly a full IEP team decision and wide variability in the role and responsibility of the school psychologist in the decision-making process. There appears to be great pressure to find students eligible, because “special education is the only game in town when it comes to helping students.” The most talked about barrier in interviews (noted in all six interviews) was the difficulty with measuring Section Four factors. This was mirrored in the survey, with fifty-two percent of the school psychologists in the study reporting that they do not believe Section Four is measurable. This leaves practicing Oregon school psychologists asking questions about Section Four; such as, “how do you measure it? Once you do measure it, how do you interpret that?” Those interviewed suggested additional guidance 78 on the operational definitions of Section Four terms, how to measure them, with comparative norms for a clearer determination of Section Four. There were eight characteristics identified that survey respondents had at least 75% agreement on as defining characteristics of Section Four. Environmental Disadvantage was defined as the lack of early development/social childhood experiences, inadequate health care, and lack of appropriate instruction. Cultural Disadvantage characteristics included the belief, customs, etc. of a particular society or group that differs from the majority group and impacts the learner’s perspective about the benefits of learning or attitude toward reading, writing, math, listening, or concentrating; and results from language barriers. Economic Disadvantage was defined as having characteristics of poverty, social class, and a combination of variables that include parents’ occupation, level of education, income, wealth, and place of residence. RQ3: Description of influences to implementation using CIT. The last research question for the study was how the CIT theoretical framework could be used to describe the implementation of a complex policy to better understand the multiple layers of context and factors that influence implementation. Figure 3 represents the survey responses for key characteristics of the CIT theoretical framework. Cognitions (41.92%), which includes interpretation, frames of reference, and observations of reality was the most predominately used characteristic in the decision-making process loop. Results indicated that Specific Context, which is the most proximal to the interactive process loop, had the most influence (11.39%) on the decision-making process. Findings also suggest that Motivation (35.84%), which involves the person’s own goals and values and 79 the external pressures, may be influenced by the Focusing of Attention on the Cognitive aspects of the decision-making process (7.00%). 80 CHAPTER V DISCUSSION In this chapter, the results of the quantitative and qualitative findings are discussed by research question, and include a discussion of the contextual factors that impact the implementation of Section Four and a comparison of results to prior seminal research. Following the discussion of findings, study limitations and policy implications are presented. At the conclusion of the chapter, an action plan is proposed, that describes the steps necessary to turn the dissertation into a ready-to-publish policy analysis as required for the policy analysis D.Ed. dissertation option. This study aimed to explain the implementation of Section Four in the SLD identification process as a means to shed light on a continuing problem of practice in special education in Oregon through the analysis of policy implementation. For over 40 years, the Exclusionary Clause has remained untouched in the IDEA law with little research or investigation in literature (as described in Chapter 1). This dissertation study proposed a theoretical framework to explain the implementation in Oregon in a way that allows stakeholders to observe the interaction that happens during the implementation of Section Four. It identified which factors are being used in Oregon to determine Section Four and how CIT could be used to better understand the decision-making processes in future research. Previous researchers have contributed to the discussion regarding Section Four. Their work established the foundational work on potential factors used to determine Section Four (Harris et al., 1988; Ochoa et al., 1997), suggested routine non-compliance (Chandler, 2014; Harris et al., 1988; Ryan, 2013), and boldly questioned its place in 81 IDEA law (Ryan, 2013). This study generalized findings to Oregon and allows policy actors and stakeholders to better understand implementation, barriers to compliance, suggests some levers to improve implementation, and revives Ryan’s call for consideration of the removal of Section Four if changes cannot be made to remove the identified barriers to improve the potential for equitable implementation. Discussion of RQ1: Extent and Ways of Compliance The extent of implementation of the IDEA mandate is expected to be 100% across the nation. In the case of Section Four compliance in Oregon, there is no exception to the federal mandate. The extent and ways in which Oregon school psychologists and IEP teams are complying with Section Four was evaluated by school psychologists’ responses to a survey (n = 100) and follow-up interviews (n = 6). Implementation. Currently, 12.7% of Oregon school psychologists from this study’s sample reported that they always comply with the implementation of excluding students based on environmental, cultural, or economic disadvantage. We know that Oregon has been a relatively impoverished state with a significant number of students in public schools that are likely environmentally, culturally, or economically disadvantaged. (Oregon, 2015; U.S. Census Bureau, 2015). It is important to answer questions about the implementation of a policy (i.e., Section Four) that may be affecting Oregon students’ ability to access and receive educational supports. Barriers to compliant implementation. Results of this study indicate that implementation of Section Four in Oregon is highly inconsistent with federal law. School psychologists and IEP teams are required to consider factors of disadvantage, determine if learning challenges are primarily the result of disadvantage, and exclude them from 82 special education eligibility if they are. Survey responses from Oregon school psychologists show that 12.7% always comply, 43.0% sometimes comply, 29.1 % rarely comply, and 15.2% never comply. These are combined into two categories of regularly tried to comply (55.7%) and routinely ignored or circumvented the law (44.3%). In comparison, Harris et al.’s (1988) small national study reported 47.3% of school psychologists and IEP teams who completed their survey regularly tried to comply, 37.5% routinely ignored or circumvented the regulation, and 9.7% sometimes complied and sometimes did not.2 Oregon school psychologists in this study demonstrated the similar lack of compliance as found in the Harris et al. (1988) study. With the clause being ignored so frequently, as shown above, it is surprising that 52% agreed or strongly agreed that Section Four should remain in the law. Interviews confirmed a low level of meaningful consideration and engagement with the question of exclusion during IEP meetings when the decision is made. The lack of true consideration was a common theme in the experiences of school psychologists interviewed, indicating that during the IEP meetings, the Section Four determination has become a “check no and go process.” Most described a multi-disciplinary team that is present and utilizes a Pattern of Strengths and Weaknesses (PSW) model (94% of the sample used PSW), but who does not actively engage in the decision-making process of Section Four determination as described by five out of six interviewees. 2 Transforming data to match more closely with Harris’ categories would produce: 12.7% always tried to comply, 44.3% routinely ignored or circumvented the law, and 43.0% sometimes complied and sometimes did not; however, 49.4% report that they have never ignored or circumvented the law (Table 4). Regardless of how one argues the combination of findings to compare to Harris et al. (1988), it is clear that Oregon lacks 100% is not compliance with Section Four of IDEA exclusionary clause. 83 Discussion of RQ2: Factors of Implementation of Section Four in Oregon Ochoa et al. (1997) identified decision-making factors within six themed categories: (a) family and home factors, (b) language instruction and language-related factors, (c) assessment instrument and procedural safeguards, (d) educational history factors, (e) general educational factors, and (f) other miscellaneous or unidentifiable factors. This study quantified the use of the Ochoa et al. (1997) factors in Oregon to determine that 12 factors were used by 80% or more of the sample: attendance, educational history/educational opportunity, review of sociological information and family history, frequency of moves/migration, years of English instruction, parent interview/information, length of time in US, received instruction in bilingual program, observation of student, school records, years in ESL instruction, and progress in ESL instruction. The Language Instruction and Language Related category was used most often (26% of all factors used) in consideration of Section Four; however, attendance was the most used single factor (88.31%). Barriers to compliant implementation. There were many barriers to implementation, as shown in Table 7, but the overall barrier identified by Oregon school psychologists in this study was that the state and federal law is so unclear that it makes compliance nearly impossible (94.3%). More than half of this study’s school psychologists reported the following barriers to implementation: lack of operational definitions of the terms used in Section Four, lack ways to measure the factors, unclear roles and responsibilities of school psychologists in the process of determination, pressure to find students eligible, lack of understanding and preparedness to answer Section Four, and the lack of confidence that Section Four can be applied in every case. 84 They reported a “low bar to even mark that box,” on the eligibility form. Half of those interviewed reported that IEP teams usually do not have any documented evidence supporting their decision. One school psychologist summarized the overall barrier in Oregon with this statement: I'd say the biggest barrier for [school psychologists] to overcome is they need more information, they need more in-depth information, a better understanding of what this all means because in my mind the reason that person can't make a decision is because they just don't understand it at a level that's deep enough. School psychologists report a political sticky wicket because they want to do the right thing, but clear guidelines and procedures for measuring Section Four factors are lacking (94.3%) of school psychologists believe that the state and federal guidelines are unclear). And only about half of respondents stated that they believe Section Four is even measurable (51.9%). Measurement of Section Four factors was by far the greatest barrier to implementation during discussions with school psychologists. The difficulty in measurement and the uncomfortable position that school psychologists are placed in when leading an IEP team in the decision-making process was evident in the top two reasons that the law may not be complied with. School psychologists were nearly begging for some assistance on understanding the definition of Section Four, identifying criteria and tools for measuring it to relieve some of the pressure that is involved in the decision- making process of Section Four determination. Discussion of RQ3: Description of Influences to Implementation Using CIT Findings suggest that CIT, as a theory derived from assessing water governance, is applicable to education research. CIT can be used to describe the contextual and interaction factors involved in the implementation of the exclusionary clause (Figure 3). The Ochoa et al. (1997) factors were easily mapped onto the CIT model (Appendix G) to 85 analyze the frequency of use of quantitative and qualitative measures by Oregon school psychologists. Results suggest that Oregon school psychologists, as represented by this study’s sample, rely heavily on their understanding to make Section Four decisions (Cognitions 41.92%). Cognition is a key characteristic of CIT’s Interactive Process Loop, consisting of interpretations, frames of reference, and observations of reality. The perspectives and motivations of respondents, including their own goals and values, impacts of external pressure, and evaluations of self-effectiveness were influenced through the Focusing of Attention (7.0% of the time). This means that the potential influence through Focusing of Attention from Motivation to Cognitions was the most common influence when school psychologists were given scenarios that required them to analyze information and make a determination about Section Four. Attribution by Others and the Accessibility and Availability of Resources (Capacity and Power) was the lowest endorsed CIT characteristic on decision-making in scenarios and frequency of use of factors in Section Four determinations (22.24%). According to survey/interview data, the specific context, which is the local environment in which the school psychologist works, has more influence than their knowledge or understanding of state or federal factors during the actual decision-making process. Discussion of Levers to Improve Implementation In third generation policy analysis, as described in Chapter 1, policy implementation research has moved to understanding policy from the implementers’ perspective (Fowler, 2013). The CIT allows for implementation to be considered from the perspective of the street-level bureaucrat (Weatherly & Lipsky, 1977). With this in mind, 86 levers of action can be considered in our attempts to first understand the situation and making recommendations for the future. The greatest lever in this policy analysis is Oregon school psychologists themselves. One psychologist stated that “people that go into school psychology go into it because, for the most part, they’re really good people and they really care about kids.” If they were given operational definitions, ways and tools to measure the factors, norms or cut-off scores for comparisons, Oregon could improve implementation significantly according to this school psychologist who stated: It would not be difficult to apply if somebody would define [the factors] with cut- offs. That goes back again to that whole thing if you could make it black and white. You specify this many years of English. You specify this language level. You specify this many days in school uninterrupted. This many years in school uninterrupted. If you make it very clear like that...then people in the field will do it…I'm saying that if it's very yes, no, this score, that score, then they'll do it. This study identified eight characteristics with at least 75% agreement that can be used to begin defining the Section Four terms. Environmental Disadvantage was defined as the lack of early development/social childhood experiences, inadequate health care, and lack of appropriate instruction. Cultural Disadvantage characteristics included the belief, customs, etc. of a particular society or group that differs from the majority group and impacts the learner’s perspective about the benefits of learning or attitude toward reading, writing, math, listening, or concentrating; and results from language barriers. Economic Disadvantage was defined as having characteristics of poverty, social class, and a combination of variables that include parents’ occupation, level of education, income, wealth, and place of residence. The final lever that can be used to create change is the knowledge about CIT and possible uses in future studies to determine if the results from this study are reliable 87 measures of what Oregon school psychologists actually do in the decision-making process involved in implementation of Section Four. Results suggest that the framework would provide a theoretical structure for the data and provide researchers a model to map both quantitative and qualitative data about social interactions. This data could then be analyzed using a variety of research methods. The CIT framework may provide information about potential levers for policy change and implementation science for complex social policies, such as those found in education. These levers for policy implementation follow the premise that the law will continue and that compliance will be required. Since the provisional definition and Section Four have been in the law for over 40 years without review, it is a logical premise. However, without definitions and valid ways to measure Section Four the equity concerns raised by prior researchers are likely to continue (Chandler, 2014; Fletcher & Navarrete, 2003; MacMillan & Siperstein, 2001; Mellard, Deshler, & Barth, 2004). To address equity concerns, Gartner and Lipsky (1987), propose a single system of special education that reduces the use of biased measures for determining eligibility. They challenge IDEA and state that “attitudes and assumptions about the disabled and disability require change” (Gartner & Lipsky, 1987, p. 368). Mellard, Deshler, and Barth indicate that values and biases attribute to the decision making process and that LD identification must be more than trying to “build a better mousetrap” (2004, p. 229). Limitations of the Study Although the mixed method study design provides a rich process for triangulation for strengthening a study, there are always limitations. There are several limitations identified for this study that include that introduces threats to internal and external 88 validity: (a) non-response bias, (b) self-response bias, (c) researcher bias. There were also threats to construct validity with the use of an unproven theoretical framework. Non-response bias. The greatest limitation to this study was potential non- response bias. The entire population of school psychologists in the state of Oregon was invited to participate and threats to validity and reliability are possible with differences in response rates by a variety of demographic factors that could alter represented perceptions. This study had a response rate of 23% on the quantitative survey. It was an anonymous survey and there is no way to determine if the respondents of the sample fully represent the entire population of school psychologists in Oregon. Because it is unlikely that the sample if fully representative of the population it makes generalizing findings to the population of Oregon school psychologists or to school psychologists in general problematic. Ninety-four percent of the sample reported using a form of Pattern of Strengths and Weaknesses model of evaluation. These results may not be reflective of those using a Response to Intervention model and the generalizability to the entire population of school psychologists It is unclear how the method of evaluation impacts Section Four determination (Chalfant, 1989; Fuchs et al., 2004; Reschly & Hosp, 2004). Self-response bias. Participation was voluntary in this study. As with any self- administered study, there is the possibility that participants will over generalize their responses or will lack the detail needed to fully understand the questions. The topics in this study, especially the compliance questions, can be sensitive in nature. Participants may have answered in a way that meets the perceived expected response, rather than their actual feelings and opinions. This is a compliance law that participants are required by licensure to comply with in a professional manner. Responses could have been biased 89 given this potential conflict. The methods of the study, including making questions voluntary and anonymous were used to reduce potential response bias. Researcher bias. Creswell (2014) describes researcher bias for any qualitative study. The researcher’s bias can introduce error during the process of coding and interpretation of results. Creswell also states that “good qualitative research contain comments by the researchers about how their interpretation of the findings is shaped by their background, such as their gender, culture, history, and socioeconomic origin” (2014, p. 202). As the primary investigator on this dissertation project, my background as a special educator with experience in SLD evaluations using both PSW and RTI methods of evaluation led me to question Section Four determinations as a problem of practice. I continue to work directly with and supervise school psychologists in Oregon. My background as an equity leader also shapes my interpretation of findings. Construct validity. There are also threats to construct validity with an untested theoretical framework. Contextual Interaction Theory was introduced to educational policy analysis, but is yet to be proven as a reliable way to represent the data. The methods also relied on researcher coding of criteria used in Section Four determination before mapping onto the theoretical framework. These may not be valid or reliable codes, which threatens construct validity for Research Question Three findings. Policy Implications Spillane and Callahan “argue that a cause of implementation failure, rarely examined in the literature, concerns the ways in which local implementers miss or misconstrue the intent of policy” (2000, p. 402). They go on to explain that: implementation problems are a product of implementers’ efforts to ignore, sabotage, or adapt interventions to fit their local agendas and preferences. Another 90 is that implementation failure results because local implementers lack the know- how or capacity to carry out policy makers’ proposals. This policy analysis makes no causal inferences, but does describe the behaviors of Section Four policy implementers in the state of Oregon (Elmore, 1979). The outcomes of implementation are abysmal for a policy that mandates 100% compliance (12.7% of study respondents report this level of compliance). This analysis provides a framework and data to describe implementation of Section Four in Oregon and allows for forward and backward mapping to improve policy implementation (Elmore, 1979). Ryan (2013) called for Section Four of the SLD exclusionary clause be removed from the law as a result of his policy analysis; however, study respondents have mixed emotions about this recommendation. When asked if Section Four should remain as a factor in the decision-making process for SLD eligibility, 51.8% of survey respondents agreed or strongly agreed that it should; however, there is consensus that the state and federal guidelines were unclear (94.3%). If Section Four remains in the law, some very serious work needs to be done to clarify the terms and how to measure them. The four purposes of the federal IDEA policy, that includes Section Four, are to assure (a) that all children receive a free and appropriate public education, (b) the protection of parents’ and students’ rights, (c) the assistance to states and localities to provide for children with disabilities, and (d) that the efforts to educate children with disabilities are effective (2011). Each state has the obligation to ensure implementation to meet these objectives. Currently, barriers to implementation appear to be so great, that it may be unrealistic to expect compliance and effective implementation. If implementation were the only option, ODE is encouraged to consider creating a task force to investigate 91 Section Four implementation more fully and oversee the following five recommendations: 1. Operationally define Section Four terms 2. Develop a measurement tool with cut-offs or norms for comparison 3. Provide enhanced professional development opportunities through: a. Pre-service learning guidelines for educational institutions training Oregon school psychologists; and b. In-service training opportunities for those school psychologists already implementing Section Four 4. Monitor implementation of Section Four through the SPR&I (System Performance, Review, and Improvement) process for IDEA implementation 5. Make federal recommendations for clarifying IDEA for all states. If the above recommendations are implemented by ODE, the following stakeholders may be interested in this study’s findings: first, pre-service organizations may be interested in the findings to better prepare Oregon school psychologists for answering the exclusionary clause in their practice with SLD evaluations; second, Oregon school psychologists may be interested to self-evaluate their own practice and participate in and ODE task force on Section Four; third, school districts and ESD special education administrators may be interested in better understanding implementation of Section Four to improve local practice; fourth, ODE may be interested in levels of compliance and implementation of Section Four to improve compliance and monitoring for the state to improve implementation across the state; and finally, policymakers and researchers may be interested in the use of CIT for future educational policy analysis. The current study is 92 the first to use CIT as a framework for complex educational policy. The implications for each of these stakeholder groups will be discussed below. Pre-service institutions. Universities and other pre-service organizations providing training to Oregon school psychologists and educators responsible for conducting evaluations and making Section Four determinations may use this study to better understand the experiences of Oregon school psychologists. This study identifies a problem of practice for Oregon school psychologists with regards to IDEA compliance. Regardless of the potential mandate to change pre-service requirements about Section Four training from ODE, pre-service organizations may want to answer the call from school psychologists for additional support and training regarding Section Four implementation. Stakeholders may review the “advice” that was provided by school psychologists for applicability to their services. Stakeholders may consider the factors used in making determinations for comparison with their instructional goals for their students. Finally, stakeholders may review the CIT for applicability in describing the interactive experiences to better prepare school psychologists for the use of a complex social policy, such as IDEA in their daily professional practice. Trainers of school psychologists may be looking for specifics about improving the practice and implementation rates in Oregon. Unfortunately, this study raises more questions about the policy than local policy actor’s ability to implement. Without additional work regarding definitions, measurability of factors that relate to the defined terms and specific procedures with decision-making rules; it is possible that this policy has little hope of being effectively implemented. 93 Practicing school psychologists and educators. Practicing school psychologists and other educators may review findings to learn more about the application of Section Four by their colleagues in Oregon. They can review and compare their practices against those represented in this study for personal professional development and consideration. The current study is intended to be a description of implementation of Section Four in Oregon with no causal inferences. School psychologists’ impressions and experiences are reflected, but in no way reflects the practices of any single individual. Results of this study may also be used by this stakeholder group to request additional professional development by their employer, licensing board, or governing body; such as ODE. ODE and policy actors. This is the primary stakeholder target for this study. It is hoped that this researcher will have an opportunity to discuss her findings and analysis of this policy with stakeholders at ODE. There is a rich source of information to be mined from this study to inform policy implementation, compliance monitoring, and future professional development on the implementation of Section Four in Oregon. It is unlikely that Oregon is significantly different than other states trying to implement this federal policy; however, Oregon may use this dissertation study as a way to increase awareness on this topic and move forward with the above recommendations. Future research would be needed to make any true causal relationships or decisions, but it is the hope that interest in such research could be an outcome of this study. Federal policymakers and researchers. Federal policymakers may be interested in these findings if ODE implements recommendations or considers research on IDEA implementation during reauthorization. Policy researchers may be interested in the expansion of CIT to study a complex social policy, such as IDEA. O’Toole (2004) 94 proposed the use of CIT in more general policy applications. Stakeholders in policy research may be interested in this attempt at expanding the use to education. There is promise that CIT may be applicable to meta-analysis research in the future, as the framework encompasses multi-levels of contextual and interaction processes with multiple policy actors. This researcher humbly calls other policy researchers to consider the use of CIT for other replications and case studies to gain a deeper understanding of this complex, but worthy topic. Key Future Policy Directions and Research The overall premise that was presented here was to improve implementation. The purpose of this particular premise was that this study only described the current conditions and implementation in Oregon within the limitations of the study. Part of the limitations is the Wider Context of policy implementation that includes an active mandate of IDEA to comply with Section Four. The premise to improve implementation may be faulty given the current state of the policy. It is entirely possible that Section Four is impossible to effectively implement. Future research is needed to verify CIT as a valid theoretical framework for the interactive decision-making process. An empirical design is needed to research the contextual factors and decision-making processes to better determine the interaction process that lead to the decisions made with the definitions and measurement tools identified by the recommended Task Force. Additional research is needed to adequately determine the mitigating factors of things that can be measured both in the home and school environments. It is possible that with such a subjective determination of conditions that impact the internal processing of a student is impossible to adequately measure; making Section Four policy implementation also impossible. 95 If future research does determine that through empirical studies that potential definitions and factors to measure them are too elusive for environmental, cultural, and economic disadvantage terms; then serious consideration must be given to removing Section Four from IDEA. Currently, school psychologists in Oregon, and across the nation, are placed in a position to ignore or circumvent a policy because of barriers beyond their control. This also places the students that they serve in jeopardy of inequitable circumstances of being included or excluded based on subjective criteria that society has been unable to adequately define, yet require professionals to exclude them from special education services based on these disadvantages. At the same time, we require school psychologists to be ethical decision-makers. It appears that more research is needed to determine if these two expectations are at odds with one another. It is the hope of this researcher that this dissertation will ignite interest in the topic of Section Four and the exclusionary clause of IDEA. It has been over 40 years since the call for research was made when SLD was defined in the law, yet it appears that we are no closer to actually understanding SLD and what should be included or excluded in the law. There is no guarantee that no child is wrongly being excluded or included in special education as a result of Section Four; yet there is no room in the pursuit of equity for all to have such a potentially harmful clause to remain in the law after 40 years of searching. Disadvantage can be framed in different ways depending on implicit and explicit bias. The current criteria identified as useful by Ochoa et al. (1997) and the findings of this study may not adequately explain Section Four terms. There are no valid and reliable ways to measure the criteria for determining Section Four nor are the mitigating mechanisms understood. These concerns revive the question raised by Lyon (1996) and 96 Ryan (2013) about the appropriateness of Section Four in IDEA law. If no significant changes can be made to reduce the barriers to effective implementation, serious consideration should be made to the removal of Section Four from policy. Publication Action Plan Publication of this dissertation is not required as part of the D.Ed. Policy Analysis dissertation option; however, this section will outline the elements that would be required to make this a publishable policy analysis. In order to make this more approachable for a policy audience, I would need to make further refinements to the section reviewing literature and theoretical framework. This chapter of the dissertation is lengthier than commonly found in policy briefs and more academic in language and tone. The literature review/theory would be shortened and simplified, with more details referenced in appendices. To create a policy brief from this work, I would also condense the methods section to allow for understanding of the methods used, but not provide the level of detail presented here; appendices could be provided for replication of the study. Additionally, a policy brief likely would customize the findings to the specific audience. There were several questions asked in this study that could be highlighted to address the concerns of the particular stakeholder group as the target audience. For example, the purpose for which each of these elements (namely, level of compliance, influential factors, and use of CIT) may be different for different audiences. Pre-service institutions may be less concerned with the use of CIT as a policy analysis tool so RQ3 could be simplified to shorten the overall analysis for their use, whereas, ODE may be more interested in the levels of compliance (RQ1) and the contextual factors and implementation barriers (RQ2 97 and RQ3), but less interested in the actual factors used by school psychologists in Section Four determination. Lastly, I would target discussion and implications to the specific policy audience. This is a broad and complex federal policy implemented by states that affects several stakeholder groups differently. Summarizing key points for their particular interest would make this policy analysis more applicable to each stakeholder group and would likely increase the probability of it being read and of use to them. For example, key recommendations were provided for ODE that impact all stakeholder groups in this discussion. If a policy brief were written specifically for school districts and local policy actors (i.e., school psychologists, special educators, and IEP team members) as implementers of Section Four, specific action steps and recommendations for entries in special education evaluation manuals and handbooks would be most appropriate. Local policy implementers are most interested in the process of implementation and would most likely benefit from a how to guide of implementation. The purpose of such a detailed information regarding the barriers to implementation would be to provide background for local policy development or adaptation to improve local implementation (i.e., process and procedures during IEP team meetings and SLD evaluation processes). 98 APPENDIX A CONTEXTUAL INTERACTION THEORY (CIT) 99 APPENDIX B SURVEY PROTOCOL 100 101 102 103 104 105 106 107 108 APPENDIX C SEMI-STRUCTURED PHONE INTERVIEW GUIDE 109 110 111 112 113 APPENDIX D PRE-NOTICE EMAILTO SPECIAL EDUCATION DIRECTORS 114 APPENDIX E INVITATION EMAIL TO PARTICIPANTS 115 APPENDIX F FOLLOW-UP EMAIL TO PARTICIPANTS 116 APPENDIX G EXCEL SPREADSHEET OF CODED SURVEY DATA 117 118 119 120 121 122 123 REFERENCES CITED Babbie, E. (2013). The practice of social research (13th ed.). Belmont, CA: Wadsworth, Cengage Learning. Berman, P. (1978). The study of macro and micro implementation of social policy. The Santa Monica, CA: Rand Corporation. Bogdan, R. C., & Biklen, S. K. (2003). Qualitative research for education: an introduction to theory and methods (4th ed.). Boston, MA: Allyn & Bacon. Bressers, H. (2007). Contextual Interaction Theory and the issue of boundary definition: Governance and the motivation, cognitions and resource of actors. The Netherlands: Institute for Governance Studies at University of Twente. Bressers, H. (2009). From public administration to policy networks: Contextual interaction analysis. In S. Nahrath and F. Varone (Eds.), Rediscovering public law and public administration in comparative policy analysis: Tribute to Peter Knoepfel (pp. 123-141). Lausanne: Presses polytechniques et universitaires romandes. Bressers, H., & de Boer, C. (2013). Contextual interaction theory for assessing water governance, policy and knowledge transfer. In C. L. de Boer, J. Vinke-de Kruijf, G. Özerol & H. T. A. Bressers (Eds.), Water governance, policy and knowledge transfer: international studies on contextual water management (pp. 36-54). London: Routledge. Chalfant, J. C. (1989). Learning disabilities: Policy issues and promising approaches. American Psychologist, 44, 392–398. doi: 10.1037/0003-066X.44.2.392 Chandler, R. (2014). Teachers’ beliefs about poverty and the impact on learning disabilities identification in a poor, rural school district. The Rural Educator, 35(3), 31–39. Chennat, S., & Singh, A. (2014). Labeling children with specific learning disabilities (SLD): A critique. International Journal of Advance Research, 2(2), 1–19. Colker, R. (2011). The learning disability mess (Working Paper No. 156). Columbus, OH: Center for Interdisplinary Law and Policy Studies, Ohio State University Moritz College of Law. Confederation of Oregon School Administrators. (2015). COSA 2014-2015 survey: Salaries and economic benefits for adminstrators in Oregon school districts and ESDs. Salem, OR: COSA. Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage Publications, Inc. 124 Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory into Practice, 39(3), 124-130. doi:10.1207/s15430421tip3903_2 De Boer, C., Kruijf, J. V., Özerol, G., & Bressers, H. T. A. (2013). Water governance, policy and knowledge transfer: International studies on contextual water management. London: Routledge. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method (4th ed.). Hoboken, NJ: John Wiley & Sons, Inc. Dombrowski, S. C., & Gischlar, K. L. (2014). Ethical and empirical considerations in the identification of learning disabilities. Journal of Applied School Psychology, 30(1), 68-82. doi:10.1080/15377903.2013.869786 Elmore, R. F. (1979). Backward Mapping: Implementation Research and Policy Decisions. Political Science Quarterly, 94, 601-616. doi:10.2307/2149628 Fletcher, J. M., Coulter, W. A., Reschly, D. J., & Vaughn, S. (2004). Alternative approaches to the definition and identification of learning disabilities: Some questions and answers. Annals of Dyslexia, 54, 304–331. doi:10.1007/s11881-004-0015-y Fletcher, T. V., & Navarrete, L. A. (2003). Learning disabilities or difference: A critical look at issues associated with the misidentification and placement of Hispanic students in special education programs. Rural Special Education Quarterly, 22(4), 37-46. Fowler, F. C. (2013). Policy studies for educational leaders: An introduction (4th ed.). Upper Saddle River, NJ: Pearson Education Inc. Fuchs, D., Deshler, D. D., & Reschly, D. J. (2004). National Research Center on Learning Disabilities: Multimethod studies of identification and classification issues. Learning Disability Quarterly, 27, 189-195. Gartner, A., & Lipsky, D. K. (1987). Beyond special education: toward a quality system for all students. Harvard Educational Review, 57(4), 367-395. Hallahan, D. P., Keller, C. E., Martinez, E. A., Byrd, E. S., Gelman, J. A., & Fan, X. (2007). How variable are interstate prevalence rates of learning disabilities and other special education categories? A longitudinal comparison. Exceptional Children, 73, 136-146. Hallahan, D. P., & Mercer, C. D. (2002). Learning disabilities: Historical perspectives. In R. Bradley, L. Danielson, & D. P. Hallahan (Eds.), Identification of learning disabilities: Research to practice. London: Routledge. 125 Harris, J. D., Gray, B. A., Davis, J. E., Zaremba, E. T., & Argulewicz, E. N. (1988). The exclusionary clause and the disadvantaged: Do we try to comply with the law? Journal of Learning Disabilities, 21(9), 581-583. doi:10.1177/002221948802100911 Homans, G. C. (1958). Social behavior as exchange. American Journal of Sociology, 63, 597–606. Hussein, A. (2009). The use of triangulation in social sciences research: Can qualitative and quantitative methods be combined? Journal of Comparative Social Work, 4, 1-12. Individuals with Disabilities Education Improvement Act of 2004, Pub. L. No. 108–446, § 601 et seq., 118 Stat. 2647 (2005). Lyon, G. R. (1996). Learning disabilities. The Future of Children, 6(1), 54–76. doi:128.223.174.87 MacMillan, D. L., & Siperstein, G. N. (2001). Learning disabilities as operationally defined by schools: Executive summary (Vol. ED 458 759). Washington, DC: U.S. Department of Education. Martin, E. W. (2002). Response to “Learning disabilities: Historical perspectives”. In R. Bradley, L. Danielson, & D. P. Hallahan (Eds.), Identification of learning disabilities: Research to practice (pp. 85-89). New York: Routledge. McDermott, P. A., Goldberg, M. M., Watkins, M. W., Standley, J. L., & Glutting, J. J. (2006). A nationwide epidemiologic modeling study of LD: Risk, protection, and unintended impact. Journal of Learning Disabilities, 39, 230-251. doi:10.1177/00222194060390030401 Mellard, D. F., Deshler, D. D., & Barth, A. (2004). LD identification: It’s not simply a matter of building a better mousetrap. Learning Disability Quarterly, 27, 229-42. doi: 10.2307/1593675 Miles, M. B., & Louis, K. S. (1990). Mustering the will and skill for change. Educational Leadership, 47(8), 57-61. National Center for Educational Statistics. (2015). Children and youth with disabilities. Washington, DC: National Center for Educational Statistics, U.S. Department of Education. Ochoa, S. H., Rivera, B. D., & Powell, M. P. (1997). Factors used to comply with the exclusionary clause with bilingual and limited-english-proficient pupils: Initial guldelines. Learning Disabilities Research and Practice, 12(3), 161–167. O’Toole, Jr., L. J. (2004). The theory–practice issue in policy implementation research. Public Administration, 82, 309-329. doi:10.1111/j.0033-3298.2004.00396.x 126 Plano-Clark, V. L., & Creswell, J. W. (2010). Understanding research: A consumer’s guide. Upper Saddle River, NJ: Pearson Education, Inc. (Merrill). Reschly, D. J., & Hosp, J. L. (2004). State SLD identification policies and practices. Learning Disability Quarterly, 27, 197-213. Ryan, J. E. (2013). Poverty as disability and the future of special education law. The Georgetown Law Journal, 101, 1455-1503. Scanlon, D., Boudah, D., Elksnin, L. K., Gersten, R., & Klingner, J. (2003). Important publications in the field of LD in light of imminent topics. Learning Disability Quarterly, 26, 215-224. Spillane, J. P., & Callahan, K. A. (2000). Implementing state standards for science education: What district policy makers make of the hoopla. Journal of Research in Science Teaching, 37, 401-425. doi:10.1002/(SICI)1098- 2736(200005)37:5<401::AID-TEA2>3.3.CO;2-4 United States Government Publishing Office. Individuals with Disabilities Education Act, Pub. L. No. 94-142, § 1401, 20 U.S.C. 2011 Chap.33 Sect. 1401 Title 20 - Education 853 (2011). U.S. Census Bureau. (2015). Oregon QuickFacts from the U.S. Census Bureau. Retrieved November 7, 2015 from http://quickfacts.census.gov/qfd/states/41000.html U.S. Department of Education. (2011, June 10). Thirty-five years of progress in educating children with disabilities through IDEA [Pamphlets; Publicity]. Retrieved August 10, 2015 from http://www2.ed.gov/about/offices/list/osers/idea35/history/index_pg10.html Weatherly, R., & Lipsky, M. (1977). Street-level bureaucrats and institutional innovation: Implementing special-education reform. Harvard Educational Review, 17(2), 171- 197. doi:10.17763/haer.47.2.v870r1v16786270x Williams, W. (1982). The study of implementation: An overview. Chatham, NJ: Chatham House Publishers, Inc. Zehavi, A. (2011). Regulating non-government schools: Explaining success and failure. Regulation & Governance, 5, 446-464. doi:10.1111/j.1748-5991.2011.011112.x