THE IMPLEMENTATION OF A SOCIAL-EMOTIONAL LEARNING CURRICULUM FOR TARGETED STUDENTS: EVALUATING STRONG START AS A TIER II INTERVENTION by MICHAEL CHRISTOPHER SCHWARTZ A DISSERTATION Presented to the Department of Special Education and Clinical Sciences and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Philosophy September 2016 ii DISSERTATION APPROVAL PAGE Student: Michael Christopher Schwartz Title: The Implementation of a Social-Emotional Learning Curriculum for Targeted Students: Evaluating Strong Start as a Tier II Intervention This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Philosophy degree in the Department of Special Education and Clinical Sciences by: Dr. Laura Lee McIntyre Chairperson Dr. S. Andrew Garbacz Core Member Dr. Roland Good III Core Member Dr. Gina Biancarosa Institutional Representative and Scott L. Pratt Dean of the Graduate School Original approval signatures are on file with the University of Oregon Graduate School. Degree awarded September 2016 iii © 2016 Michael Christopher Schwartz iv DISSERTATION ABSTRACT Michael Christopher Schwartz Doctor of Philosophy Department of Special Education and Clinical Sciences September 2016 Title: The Implementation of a Social-Emotional Learning Curriculum for Targeted Students: Evaluating Strong Start as a Tier II Intervention Social-emotional learning (SEL) is an accepted way of promoting an individual’s resilience, which is the ability to overcome challenging life circumstances to lead a fulfilling life. SEL is traditionally taught in schools as a universal, classroom-wide intervention. However, there is currently a gap in using SEL curricula as secondary, Tier II interventions. Strong Start, a SEL program, was evaluated as a Tier II intervention with 35 2nd grade students using a randomized-control trial design. Data were collected from students on their content knowledge of social-emotional constructs as well as from teachers on ratings of student problem behaviors and prosocial behaviors. Findings suggest improvements in student knowledge, teacher ratings of problem behaviors, and teacher ratings of prosocial behaviors for all students over time, with no significant group by time interaction. Limitations and future directions are discussed. v CURRICULUM VITAE NAME OF AUTHOR: Michael Christopher Schwartz GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene Colby College, Waterville, Maine University of Otago, Dunedin, New Zealand DEGREES AWARDED: Doctor of Philosophy, School Psychology, 2016, University of Oregon Bachelor of Arts, Psychology, 2010, Colby College AREAS OF SPECIAL INTEREST: Mental Health Prevention and Promotion in Schools Behavioral, Social-Emotional, and Academic Intervention in Schools Early Interventions with At-Risk Populations PROFESSIONAL EXPERIENCE: Teaching Assistant, Department of Family and Human Services, University of Oregon, Eugene, Oregon, 2014 – 2015 Parent Coach and Skills Coach, Oregon Social Learning Center, Eugene, Oregon, 2013-2015 Research Assistant, Center on Teaching and Learning, University of Oregon, Eugene, Oregon, 2011 – 2014 Research Assistant, Project EARLY Leadership Training Grant, University of Oregon, Eugene, Oregon, 2010 – 2011 Practicum Student, Child Development and Rehabilitation Center, Eugene, Oregon, 2014 – 2015 Practicum Student, Child and Family Center, Eugene, Oregon, 2012 – 2014 Practicum Student, Springfield Public Schools, Springfield, Oregon, 2011 – 2012 vi GRANTS, AWARDS, AND HONORS: University of Oregon, Department of Special Education and Clinical Sciences, School Psychology Program, Travel Grant Award ($500.00), 2015 University of Oregon, College of Education, Dissertation Research Award ($1688.00), 2014 University of Oregon, College of Education, Department of Family and Human Services, Graduate Teaching Fellowship, Teaching Assistantship, 2014 – 2015 University of Oregon, College of Education, Center on Teaching and Learning, Graduate Teaching Fellowship, Middle School Intervention Project, 2011 – 2014 U.S. Department of Education, Office of Special Education Programs (OSEP) Leadership Training Grant, University of Oregon, Project EARLY Fellowship, 2010 – 2011 Cum Laude, Colby College, 2010 vii ACKNOWLEDGEMENTS I would like to express my sincere appreciation for the support and guidance of Dr. Laura Lee McIntyre for this dissertation study. With her guidance, advising, and mentorship, she has helped developed my idea for a study to what has become the final version of this manuscript. I am forever grateful for her willingness to take on this project and for her supervision of my research and professional experiences. I would also like to express my thanks to the members of my dissertation committee: Dr. Andy Garbacz, Dr. Roland Good, and Dr. Gina Biancarosa. The members of my committee provided me with invaluable ideas and advice throughout the study. Thank you for participating on this dissertation with me. I would also like to express my gratitude to the College of Education for helping to support this study through the Dissertation Research Award. I would also like to express my appreciation for the students, teachers, and parents that participated in this research study. Finally, I would like to thank all of my fellow students that helped me conduct this study: Kenya Makhiawala, Tom Cariveau, Alondra Canizal, Bethany West McCarter, and Jake Mahon. I would not have been able to do this without your support. viii To my family, who have constantly supported me in everything that I have done, and to the memory of Dr. Ken Merrell. ix TABLE OF CONTENTS Chapter Page I. INTRODUCTION ....................................................................................................... 1 Limitations of the Traditional Mental Health Service Delivery Model .............. 1 Mental Health Prevention and Promotion ........................................................... 3 Public Mental Health Framework and the Application to Schools ..................... 5 Resilience and Social-Emotional Learning ......................................................... 7 Applications of SEL Programs ........................................................................... 9 Strong Kids – A Social-Emotional Learning Curriculum ................................... 11 II. METHOD ................................................................................................................... 16 Research Design .................................................................................................. 16 Participants and Setting ....................................................................................... 16 Recruitment ......................................................................................................... 17 Procedures ........................................................................................................... 20 Measures ............................................................................................................. 23 Assessment Procedure ........................................................................................ 28 III. RESULTS ................................................................................................................. 30 Analytic Approach .............................................................................................. 30 Descriptive Statistics and Baseline Comparisons ............................................... 30 Correlations Among Variables of Interest .......................................................... 32 Effect of Strong Start on Student Content Knowledge ....................................... 34 Effect of Strong Start on Ratings of Student Problem Behavior ........................ 35 Effect of Strong Start on Ratings of Social-Emotional Skills ............................ 36 x Chapter Page At-Risk Student Analyses ................................................................................... 37 Post Hoc Power Analysis .................................................................................... 39 IV. DISCUSSION ........................................................................................................... 40 Summary of Implementation .............................................................................. 40 Summary of Evidence of Efficacy of Strong Start ............................................. 42 Limitations .......................................................................................................... 46 Implications for Future Research ........................................................................ 48 Conclusions ......................................................................................................... 49 APPENDICES ................................................................................................................ 51 A. TABLES ......................................................................................................... 51 B. TEACHER CONSENT FORM ...................................................................... 58 C. SYSTEMATIC SCREENING OF BEHAVIOR DISORDERS .................... 60 D. PARENT/GUARDIAN CONSENT FORM .................................................. 64 E. STUDENT ASSENT FORM ......................................................................... 70 F. STRONG START LESSONS AND TOPICS .................................................. 71 G. STRONG START SAMPLE LESSON ........................................................... 72 H. STRONG START FIDELITY OF INTERVENTION CHECKLIST ............. 75 I. STRONG START CONTENT KNOWLEDGE ASSESSMENT ..................... 76 J. SOCIAL SKILLS IMPROVEMENT SYSTEM ............................................. 81 K. SOCIAL EMOTIONAL ASSETS AND RESILIENCY SCALE ................. 83 REFERENCES CITED ................................................................................................... 86 xi LIST OF FIGURES Figure Page 1. Mean scores on the Strong Start Content Knowledge assessment ............................. 34 2. Mean scores on the SSIS – PB subscale ..................................................................... 36 3. Mean scores on the SEARS-T form ............................................................................ 37 xii LIST OF TABLES Table Page 1. Student demographic characteristics by intervention or control group ....................... 51 2. Means and standard deviations of dependent variables by intervention or control group before and after intervention participation ........................................... 52 3. Correlations between variables of interest ................................................................... 53 4. Two-way, mixed effects analysis of variance summary table for the effects of intervention and time on Strong Start Content Knowledge .................................... 54 5. Two-way, mixed effects analysis of variance summary table for the effects of intervention and time on SSIS-PB ratings ............................................................... 55 6. Two-way, mixed effects analysis of variance summary table for the effects of intervention and time on SEARS-T Ratings........................................................... 56 7. Office discipline referral (ODR) rates at the end of the school year across the Participating School District ....................................................................... 57 1 CHAPTER I INTRODUCTION Positive mental health is essential to a child’s academic and social development and impacts the child’s ability to live in a fulfilling manner (Merrell, 2008). Additionally, the mental health of a child can have an impact on the community or an environment in which the child lives (Power, 2003). For example, Merrell and Gueldner (2010) note that students with anxiety disorders may have interpersonal relationships that are negatively affected by their anxiety. Further, suicidal behaviors may be associated with other disorders such as depression or conduct disorders. Thus, students who have more mental health or behavior problems may likely require the coordinated help of professionals within school and community settings (Merrell & Gueldner, 2010). Limitations of the Traditional Mental Health Service Delivery Model While the importance of promoting children’s mental health is recognized globally, there is evidence that children’s needs are not adequately addressed (World Health Organization, 2004). Some estimates suggest that approximately 20% of children will experience the symptoms of a mental disorder, and 5% of children will experience significant life impairments as a result of a mental disorder (Massey, Armstrong, Boroughs, Henson, & McCash, 2005). Other studies suggest that one in five children will require mental health services during their academic careers, but only 30% of such children will actually receive the necessary services while they are in school (Albers, Glover, & Kratochwill, 2007; Chafouleas, Volpe, Gresham, & Cook, 2010). Thus, the number of children requiring mental health supports exceeds the number of children who are likely receiving intervention. 2 Further, these figures may be differentiated for white and minority children or children from a variety of socioeconomic (SES) status backgrounds, thereby creating health disparities. Individuals from low SES or minority backgrounds are less likely to receive mental health services. Access to care may be affected by a lack of health insurance, a lack of trust of mental health service delivery system, stigma associated with mental health, or a real or perceived lack of responsiveness to the cultural background or needs of families (Nastasi & Varjas, 2008). This is complicated by additional risk factors that ethnic minorities or individuals from low SES backgrounds may be more likely to face in their daily lives. Individuals from ethnic minority backgrounds are more likely to be at a disadvantage by living in distressed neighborhoods with substandard housing or higher crime rates, by attending low-quality schools, or by having fewer social supports or resources (Brooks, 2006). These stressor-filled environments are often associated with lower academic achievement and poorer development of social skills in children (Elias & Haynes, 2008). As the number of at-risk conditions grow due to increasing poverty levels and the changing of demographics across the United States, children are more likely to enter school without essential prosocial behavior. Children who are equipped with fewer social skills and competencies may be at greater risk for experiencing later negative outcomes like school failure, peer rejection, and the development of mental health problems (Bagdi & Vacca, 2005). This, in turn, creates economic and social burdens to which society needs to attend (Doll & Lyon, 1998). For instance, Greenberg, Domitrovich, and Bumbarger (2001) estimate that mental illnesses and disorders cost the United States nearly 75 billion dollars. 3 Despite the growing need to provide mental health services to children, there are limitations to the traditional mental health service delivery model. Wakefield (1997) argues that mental health services have long adhered to a medical model, which views emotional and behavioral difficulties as stemming from within-person limitations, thereby ignoring any ecological or environmental factors that contribute to an individual’s pathology. The process of assessing an individual’s limitations not only overlooks the factors that contribute to successful academic or social functioning, but may also contribute to the stigma associated with receiving mental health services (Jimerson, Sharkey, Nyborg, & Furlong, 2004). The consequence of such a service delivery model is that it promotes a reactive approach to mental health; services are provided to individuals that have developed severe mental health problems, making it more costly, more intensive, and more expensive (Power, 2003). Mental Health Prevention and Promotion Positive psychology provides a counter to the reactive, medical model of mental health treatment. The field of psychology has historically focused on assessing the negative factors that an individual faces, rather than including the positive aspects or strengths of an individual (Seligman & Czikszentmihalyi, 2000; Suldo & Shaffer, 2008). Positive psychology offers an opportunity to buffer against the difficulties that individuals encounter in life. The positive psychology movement, coupled with proactive approaches to improve mental health (e.g., Cowen, 1994), provides the foundation for mental health prevention and promotion. Cowen (1994) delineated the importance of psychological wellness, including preventing dysfunction (or maladaptation) and promoting psychological health. Cowen’s 4 model assumes that psychological wellness benefits from early inoculants, which serve to prevent later problems. Psychological wellness could be enhanced through early attachments, acquiring competencies, promoting healthy settings, fostering empowerment, and acquiring the skills necessary for navigating the difficulties of life. Thus, instead of adopting a reactive model of service—one that focuses on the negative aspects of an individual, waits for an individual to require services in response to a crisis, or that relies on referrals from a “wait-to-fail” approach (Doll & Cummings, 2008)—the concepts of mental health promotion and prevention have been adopted with a specific emphasis on focusing on health as more than just an absence of illness (Kobau et al., 2011). The fields of mental health prevention and mental health promotion have garnered attention nationally and globally. Mental health promotion is intended to develop the positive mental health of an individual, as it focuses on enhancing an individual’s psychological wellness, developing age appropriate competencies, and increasing an individual’s resilience (Cowen, 1994; Masten & Coatsworth, 1998; Suldo & Shaffer, 2008). Mental health promotion, similarly to positive psychology, concentrates on improving an individual’s subjective well-being (Seligman & Czikszentmihalyi, 2000). Rather than focusing on dysfunction or what is wrong with an individual, there is a greater concentration on developing positive emotions and improving functioning (Kobau et al., 2011). Promoting mental health likely results in increases in social control, health gains, positive behaviors and attitudes, and may be associated with a reduction in inequalities (Peters, 1988). 5 The field of mental health prevention bases interventions on the reduction of risk factors and the enhancement of protective factors, thereby buffering against risk (Greenberg et al., 2001; World Health Organization, 2004). Prevention services are provided in a proactive manner, such as before the onset of a mental health disorder. Nastasi and Varjas (2008) suggest that the practices of mental health prevention programming within the public health model include: providing services to the population being served, making use of evidence-based practices, conducting screenings for mental health concerns, and conducting programs that improve functioning while reducing risky behavior. Public Mental Health Framework and the Application to Schools Doll and Cummings (2008) and Merrell and Gueldner (2010) apply the concepts of a public mental health framework to a school-based three-tiered prevention model. The three-tiered model is typically portrayed as a triangle with three levels, providing a continuum of services at each level. At the bottom level is the primary or “universal” level of support, serving approximately 80% of the school’s population. At the universal level efforts for mental health prevention are geared toward serving all students within the school (Mills, Stephan, Moore, Weist, Daly, & Edwards, 2006). Doll and Cummings (2008) note that efforts at the universal level include evidence-based curricula, social problem solving strategies, and school-wide bullying prevention plans. Despite the primary prevention efforts, there may be students who do not respond. Secondary or “targeted” supports are provided to students who require additional services and who may be at-risk for developing emotional, behavioral, or mental health problems. The targeted level of support is geared toward a smaller percentage of a school, approximately 15% 6 (Merrell & Gueldner, 2010). Finally, the top of the triangle depicts the tertiary or “indicated” level of support. This reflects the proportion of the population, approximately 5% of students, who do not respond to the first two levels of support. These are the students that have significant needs requiring individualized intervention supports. Merrell and Gueldner note that students at this level have historically consumed most of the school’s resources, as they often require significant accommodations or supports, reflecting the crisis intervention or “wait-to-fail” approach. Positive behavior interventions and supports (PBIS) is one type of school-based three-tiered model. PBIS consists of creating a positive and predictable school environment for students and teachers (Ross & Horner, 2009). Similar to the mental health model described above, PBIS consists of three distinct levels of supports. At the primary level, there are defined behavioral expectations for the school, with explicit instruction on how students are expected to behave in different locations around the school. Behaviors are supported through the use of positive reinforcement strategies, such as teacher positive praise or token economies (Reinke, Herman, & Stormont, 2013). For students that are in need of additional supports, structured, easy-to-implement evidence- based interventions are used (Anderson & Borgmeier, 2010). Examples of secondary interventions include Check-In/Check-Out (e.g., Simonsen, Myers, & Briere, 2011) and First Step to Success (e.g., Walker et al., 1998). These interventions involve careful monitoring of student behavior and providing praise and reinforcement for appropriate behaviors. If students are unresponsive to secondary interventions, tertiary supports, including a functional behavior assessment and an individualized behavior plan, are provided (Debnam, Pas, & Bradshaw, 2012). 7 PBIS has been increasingly adopted in schools in the U.S. and Canada (McIntosh et al., 2006) and there have been demonstrated effects for the system-wide intervention. PBIS implementation has been associated with the reduction of student suspensions and office discipline referrals (Bradshaw, Mitchell, & Leaf, 2010). PBIS, much like a tiered- system of providing mental health supports, fits within a school-wide application. Greenberg and colleagues (2003) argue that schools are currently expected to provide more than academic services. Instead, schools are expected to teach academics, behavioral expectations, and help develop their students to be productive citizens. Thus, schools have the opportunity to provide programming and interventions that may serve to enhance school-based mental health by developing the social and emotional abilities of children (Doll & Lyon, 1998; Greenberg et al., 2003). Schools have the distinct potential of creating and modeling supportive environments, which can lead to future mental health benefits (McMahon, Parnes, Keys, & Viola, 2008). Additionally, schools can build on the creation of healthy environments while also seeking to develop individual student assets and competencies (Power, 2003). One of the ways that schools may seek to develop student assets is through the promotion of resilience and related social-emotional learning (SEL) skills. Resilience and Social-Emotional Learning Resilience is the ability to overcome adverse situations to lead a fulfilling life (Doll & Lyon, 1998; Jimerson et al., 2004; Masten & Coatsworth, 1998). Masten and Coatsworth (1998) note that society has provided increasing obstacles for children and adolescents to overcome to become successful adults. These societal factors, coupled with adolescent risk behavior (Mettrick, Lever, Burke, Mills, & Ghunney, 2008), 8 prompted research on the factors that lead to children developing into productive adults. Resilient children are likely to have positive temperaments, close peer relationships, strong self-efficacies, and close relationships with caregivers or other caring adults (Doll & Lyon, 1998). Resilience may be promoted by providing services that help children develop friendships, social engagements, and connection with opportunities that allow for participation and engagement with organizations (Doll, Jones, Osborn, Dooley, & Turner, 2011). Kobau et al. (2011) provide a description of how such skills may be developed within- and between-people as well as at an ecological level: 1) individual coping and optimism strategies may be developed at the within-person level, 2) support for social interactions may be developed at the between-person level, and 3) the development of a supportive environment with a variety of supports to help all children succeed may be developed at the system or school level. Brooks (2006) suggests that schools may function at two of these levels. Schools may enhance within-person social competencies through instruction (e.g., social problem solving, self-regulation skills) as well as develop protective factors at the ecological level (e.g., developing supportive environments, providing opportunities for involvement, and facilitating). Social-emotional learning (SEL) is an increasingly accepted and empirically supported way to promote resilience. The Collaborative for Academic, Social, and Emotional Learning (CASEL) defines SEL as involving “the processes through which children and adults acquire and effectively apply the knowledge, attitudes, and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions” (CASEL, 2012, p. 6). CASEL promotes five critical components 9 in SEL instruction: self-awareness, self-management, social awareness, relationship skills, and responsible decision-making (CASEL). In other words, CASEL defines resilience to include the behaviors and competencies that promote positive outcomes, individual success, and fulfillment. The constructs that are consistent with SEL—whether they are within- and between-person abilities—may be taught at the school level, learned by students, and measured with assessments (Merrell, Carrizales, Feuerborn, Gueldner, & Tran, 2007). Traditionally, SEL learning curricula have been conceptualized as fitting into the multi- tiered system of support as a universal or primary prevention, targeting SEL-related competencies and preventing future problem behaviors (Albers et al., 2007). Universal SEL curricula aim to aid the social, emotional, cognitive, and behavioral development of all students within a classroom or a school (Becker & Domitrovich, 2011). There have been demonstrated improvements in SEL knowledge and competencies when universal programs have been implemented (Harlacher & Merrell, 2010; Merrell et al., 2008), and a recent meta-analysis demonstrates the positive benefits of teaching SEL-related skills (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). Applications of SEL Programs Durlak and colleagues (2011) conducted a meta-analysis of 213 school-based, universal SEL programs, the first such large scale-meta-analysis of programs geared toward improving students’ social-emotional development. Durlak et al. explored the effect of SEL programs on a variety of outcomes (e.g., social and emotional skills, positive social behaviors, conduct problems, and academic performance) in over 270,000 kindergarten to high school students. Results of the meta-analysis suggested that there 10 were benefits of implementing universal SEL programs for both behavioral and academic outcomes. There were improvements in students’ attitudes about themselves or their schools, increased ratings of prosocial behaviors, and reduced conduct and internalizing problems (Durlak et al., 2011). Additionally, the meta-analysis found an 11 percentile point gain in student academic achievement for students who received SEL programs in comparison to control students. Thus, universal SEL curricula have the potential to improve student behavior and academic performance. Specific types of universal interventions may be targeted to different age ranges of students, but all aim to reduce emotional and behavioral problems while promoting social-emotional competence. Promoting Alternative Thinking Strategies (PATHS) is a teacher-led curriculum that is provided to students once per week for 30 weeks, targeting emotion awareness, self-control, positive social interactions with peers, and problem solving (Kusché & Greenberg, 1994). The preschool version of the curriculum was delivered to three and four-year-old students using a randomized-control trial (Domitrovich, Cortes, & Greenberg, 2007). Students in the intervention condition were rated as having improvements in emotion knowledge, self-regulation, and social skills in comparison to control participants, as rated by their teachers and parents. Second Step is another SEL intervention that has been rigorously evaluated using a randomized control trial design. Grossman and colleagues (1997) evaluated Second Step, a universal aggression prevention program, on improving student prosocial behavior and decreasing aggressive behavior in urban and suburban elementary schools. The 30 lesson curriculum primarily focuses on teaching students social skills. Grossman et al. found that Second Step was associated with a decrease in students’ aggressive 11 behaviors as well as an increase in their neutral or positive behaviors. Multimethod and multi-informant assessments were used, representing observations, teacher reports, and parent reports of child behavior. Most of the intervention effects maintained at a six month follow up. Despite the demonstrated effectiveness of these SEL programs, there are some limitations of these interventions. The interventions described can be time-consuming— both in terms of teacher training time or teacher implementation time—and some are expensive for schools to purchase. Elias, Zins, Graczyk, and Weissburg (2003) note teacher time and resources may be barriers to effective implementation and sustainability of SEL curricula within schools. Elias et al. (2003) note that staff turnover, poor fidelity with intervention implementation, and underestimations of intervention time and costs can prevent the use of SEL programming, despite the need for schools to focus on social- emotional growth. With these considerations in mind, Merrell and colleagues (2007) developed the Strong Kids series. Strong Kids – A Social-Emotional Learning Curriculum Strong Kids is designed to be a social-emotional learning program to teach students coping skills, social-emotional skills, and individual strengths or assets related to the skills associated with a student’s resilience. Strong Kids is purposefully designed to be a teacher-implemented universal SEL intervention able to be implemented at low cost and with low levels of training (Merrell, 2010; Merrell et al., 2007). Specific curricula are designed for different age ranges: preschool and kindergarten through second grade (known as Strong Start), third through fifth grade and sixth through eighth grade (Strong Kids), and ninth through twelfth grade (Strong Teens). There are some demonstrated 12 benefits to the implementation of the series across different grade levels; however, the focus of the following review will concentrate on the impact of Strong Start on first and second graders’ social-emotional skills. There are three peer-reviewed journal articles examining the outcomes of Strong Start implementation in elementary schools. Caldarella, Christensen, Kramer, and Kronmiller (2009) studied the impact of Strong Start on the social-emotional competencies of 2nd grade students (N = 26) using a quasi-experimental design. Strong Start was implemented in one of two 2nd grade classrooms. Dependent variables included teacher ratings of student internalizing and externalizing behaviors on the Social Skills Rating System and teacher ratings of student prosocial behaviors on the Social Skills Behavior Scales (SSBS). Caldarella et al. (2009) found an improvement on the student prosocial behaviors scale of the SSBS (d = .59) as well as an improvement on the ratings of internalizing behaviors on the SSRS (d = .38) in favor of the intervention group. Findings suggest moderate effects in improving students’ peer-related prosocial behaviors and decreasing internalizing behaviors in students who received Strong Start. Kramer, Caldarella, Christensen, and Shatzer (2010) examined the effects of Strong Start for 67 kindergarten students in four kindergarten classrooms employing a quasi-experimental time-series design. Teachers and parents completed the behavior rating scales twice before the intervention’s implementation (with a six week interval between assessments) and twice after the intervention’s completion (with another six week interval between assessments). Dependent variables included the teacher completed peer relations subscale of the SSBS, the parent completed Home and Community Social Behavior Scales (HCSB) (a parent version of the SSBS), and the internalizing subscale of 13 the SSRS. Comparing pre-intervention means with post-intervention means, Kramer and colleagues (2010) found statistically significant differences of teacher ratings of prosocial behaviors on the SSBS (d = 1.39), parent ratings of prosocial behaviors on the HCSBS (d = .44), and teacher ratings of internalizing behaviors on the SSRS (d = .48). Kramer et al. (2010) did not find statistically significant differences for parent ratings of internalizing behviors on the SSRS. Consistent with the Caldarella et al. (2009) results, findings suggest improvements of prosocial behaviors and decreases in internalizing behaviors as a result of Strong Start implementation. Whitcomb and Merrell (2012) evaluated the effect of Strong Start on 83 first grade students’ emotion-related knowledge and teacher-rated student behaviors and emotions. Strong Start was implemented using a quasi-experimental time-series design in four classrooms across two schools. There were three data collection periods: at the beginning of the school year, prior to intervention implementation in January, and after intervention implementation in April. Students completed a content knowledge assessment and the Assessment of Children’s Emotion Skills (ACES), while teachers completed the Peer Relations subscale of the SSBS and the Problem Behavior subscale of the SSRS. As a result of Strong Start implementation, there was an improvement on content knowledge (d = .35), an improvement on the ACES (d = .47), improved scores on the Peer Relations subscale (d = .31), and a decrease on teacher ratings on problem behaviors (d = -.19) from January, prior to intervention implementation, to April, after the intervention was completed. These results suggest that Strong Start has moderate effects on children’s emotion knowledge and skills and small effects on problem behavior. 14 Even though there have been demonstrated benefits of implementing Strong Start with kindergarten to second grade students, there is a common limitation with both studies. Caldarella et al. (2009), Kramer et al. (2010), and Whitcomb and Merrell (2012) did not employ experimental designs with their studies, as they did not use a random assignment of participants or classrooms to condition. The internal validity of findings is therefore limited. Thus, it is not clear if the results found within the studies are due to implementation of the Strong Start curriculum or if there were other factors that impacted the results. Furthermore, most studies do not distinguish the effects of Strong Start on students that may be most in need of SEL-related competencies. Calderalla et al. (2009) monitored five treatment group students based on an at-risk category on the Peer Relations subscale of the SSBS. Teacher ratings on the SSBS and SSRS suggested that there were larger effect sizes for at-risk students in comparison to low-risk students. There was an effect size of 1.75 for at-risk students as defined by the Peer Relations subscale. There was an effect size of -.88 for at-risk students as defined by the Internalizing subscale of the SSRS. There was an effect size of -.21 for at-risk students as defined by the Externalizing subscale of the SSRS. Despite the fact that the at-risk students benefitted more from the intervention than their low-risk counterparts, evaluating the efficacy of the program for Tier II or at-risk students was not a stated goal of the study. To date, only one study (a dissertation) has been identified that used Strong Start as a Tier II intervention. Sicotte (2012) purposefully screened for students at-risk for emotional and behavior disorders and examined the resulting impact of Strong Start on 15 problem behaviors and emotion knowledge using a quasi-experimental with between- subjects and within-subjects comparisons. Sicotte (2012) did not find statistically significant differences in ratings of problem behaviors pre- and post-intervention assessments between intervention and control students; additionally, there were no differences between groups in levels of emotion knowledge. Limitations to the study included the small sample size of the participants (N = 24) and that four of the students received behavioral supports during Strong Start implementation that were not received prior to the Strong Start implementation in the classroom. Additionally, Strong Start was delivered as a universal, classroom intervention and may not have provided the participants (at-risk students) sufficient opportunities to practice the behaviors embedded within the curriculum. Further, Strong Start may not have provided sufficient intensity of an intervention (McIntosh, Campbell, Carter, & Dickey, 2009). The proposed study seeks to address a number of the gaps identified in the literature. Specifically, this study proposes to evaluate the efficacy of Strong Start as a Tier II intervention, using a purposeful screening of students for emotional and behavior disorders and a random assignment to either an immediate intervention condition or a wait-list control condition. SEL programs have been successfully embedded at the universal level for students, but there is still a gap in using SEL programming as a targeted intervention (Fox & Hemmeter, 2009; Stoiber, 2011). This study seeks to determine how a Tier II implementation of Strong Start impacts emotion knowledge, teacher ratings of problem behaviors, and teacher ratings of social-emotional assets for at-risk first and second grade students. 16 CHAPTER II METHOD Research Design The study investigated the application of a social-emotional learning curriculum as a Tier II intervention on student’s social-emotional knowledge, teacher ratings of student social-emotional assets and resilience, and teacher ratings of student problem behavior. Participants were rank-ordered based on teacher ratings of internalizing and externalizing problems. Teacher ratings were gathered using the first gate of the Systematic Screening of Behavior Disorders (SSBD) (Walker & Severson, 1990). Participants were paired and randomly assigned to the intervention or wait-list control condition. Thus, the study employed a 2 x 2 mixed-effects design with one between- subjects effect (intervention or wait-list control) and one within-subjects effect (time; pre- and post-intervention). Pre- and post-intervention data regarding social-emotional knowledge, ratings of social-emotional assets, and student problem behavior were collected, and these measures served as dependent variables in the analyses. Participants and Setting This study was conducted in an elementary school located near a mid-size city in the Pacific Northwest region of the United States. The school consisted of 516 students as of the 2013-2014 school year (Oregon Department of Education, 2014). The ethnic demographics consisted of 65% White students, 27% Hispanic or Latino students, 5% multiracial students, 2% Asian students, 1% American Indian or Alaska native, and less than 1% Black or African American students. 18% of students were characterized as English learners, 69% of students were considered economically disadvantaged, and 15% 17 of students had disabilities for which they were receiving services (Oregon Department of Education, 2014). Study participants recruited to participate included 38 students (n = 20 boys) from three 2nd grade classrooms at the elementary school. Over the course of intervention implementation, three students left the study (two students moved away from the school; one student’s parents asked to leave the study after the first lesson). This resulted in a final sample of 35 participating students (n = 18 boys). The range of participant ages was 7.58 years to 8.56 years (M = 8.07 years, SD = 0.30). Table 1 provides the demographic characteristics of students assigned to the intervention and control conditions. Recruitment Recruitment was initiated after approval was obtained from the University of Oregon’s Office of Research Compliance Services and the institutional review board of the participating school district. The principal investigator (Schwartz) collaborated with the school district leader responsible for coordinating school-based research initiatives. The school district leader initiated contact with the school principal and second grade teachers (n = 4) from a local elementary school and assessed interest in participating. The principal investigator met with the second grade teachers to explain the nature of the study, to discuss any potential modifications, and survey interest in participating in the study. The teachers recommended modifications to the wording of the parent consent form as well as creating a Spanish translation of the parent consent form. Upon modification of the consent forms (and subsequent re-approval from the University’s Office of Research Compliance Services), three of the four teachers agreed to participate 18 in the study and provided informed consent (Appendix A). Teachers then completed the first stage of the SSBD (Appendix B). The SSBD is a validated, three-stage gating procedure used to identify students with internalizing and externalizing behavior disorders (Lane et al., 2009; Walker & Severson, 1990). Stage 1 of the SSBD involves teacher nominations of students who exhibit internalizing and externalizing symptoms. Stage 2 involves teacher completion of two rating scales, the Critical Events Index and the Combined Frequency Index, for each student in the classroom who has passed through the first screening gate (stage 1). Stage 3 of the SSBD requires well-trained observers to perform observations of academic engaged time and peer social behavior for students who have passed through the second screening gate (stage 2). The present study used stage 1 of the SSBD to identify at-risk students. With the information obtained from teacher completion of stage 1 of the SSBD, the principal investigator sent home active parent consents to the top 10 internalizing and top 10 externalizing students from each classroom to inform caregivers of the opportunity for their child to participate in the study (see Appendix C). The letters contained information detailing the purpose of the study, the assessment procedures used pre- and post-intervention implementation (e.g., student self-report measure on social emotional learning knowledge, teacher ratings of student behavior), and that student information would remain confidential. Additionally, contact information of the investigator was provided on the letter should parents be interested in obtaining more details about the study. Parents who agreed to have their child participate in the study were asked to sign and return the consent form to the teachers, who then provided the consents to the investigator. In total, 60 consent forms were sent to parents. 51 active responses were 19 returned to the principal investigator (for a response rate of 85%). Thirty eight responses provided agreement for participation in the study and 13 declined participation in the study. After parents returned consent forms permitting their children to participate in the study, student participants were read the student assent form by the principal investigator (Appendix D). All students provided assent to participate in the study, and teachers were asked to complete assessments (described below) for each participating student. Student participants were listed by three separate characteristics before being randomly assigned to condition: student sex, teacher identification of student as “internalizing” or “externalizing” as defined by the SSBD, and the rank-order on the internalizing or externalizing dimension. Students were paired by their placement on each list, with one member of the pair being assigned to intervention by a coin flip. An a priori statistical power analysis was conducted to determine a sample size needed for the study using the G*Power 3 software program (Faul, Erdfelder, Lang, & Buchner, 2007). The power analysis used a repeated-measures, within-between interaction design, assuming a moderate effect size (.40), a power value of .80, and an alpha value of .05. The analysis recommended a minimum sample size of n = 16 (with eight students per group) to detect the expected effect size. At the beginning of the intervention, 18 students were assigned to the intervention condition and 20 students were assigned to the control condition. After student attrition was accounted for, 16 students completed the intervention, and 19 students remained in the control condition for a total of 35 students. Thus, a sample size of 35 appears to be an adequate sample. 20 Procedures Intervention (Strong Start). The intervention condition consisted of the implementation of Strong Start: Grades K – 2. Strong Start is designed to be a social- emotional learning program that teaches students the skills to promote resilience, to increase coping skills, and to learn social-emotional skills and assets (Merrell et al., 2007). The curriculum consists of ten, 30 min weekly lessons. The curriculum is designed to have an impact on students’ cognitive, affective, and social functioning in a short period of time (Merrell et al., 2007). Lessons broadly consist of describing and understanding feelings, problem solving strategies, clear thinking, thinking positively, and proactive ways of dealing with stress (Appendix E). Each lesson consists of a review of the previous lesson’s topics, practice opportunities to role play current lesson topics, feedback for students on their performance (in role plays or providing answers to questions), and instructor modeling opportunities. The curriculum emphasizes introducing topics, coupling lessons with classroom behavior management techniques, directly teaching skills, and providing practice opportunities for the skills that are taught. Additionally, Strong Start was designed to be low cost in terms of time expenditure for training and implementation and requires minimal costs for materials. These intervention features make Strong Start a viable option for a Tier II intervention. Trained graduate students (training described below) in the University of Oregon’s School Psychology program implemented Strong Start to five to six students twice per week for five weeks (e.g., 20-30 min per session) in the elementary school’s cafeteria in Spring 2014. The second grade teachers identified two 30 min periods (Monday and Wednesday) for intervention delivery. One of the weeks during 21 intervention implementation was postponed a week due to a school function (class musical), which required students to practice during the scheduled intervention sessions. Thus, the five week intervention was completed over the span of six weeks. The interventionists completed each lesson and provided students with opportunities to answer questions embedded within the lesson, to role-play examples, and participate in the ways that are specified within the curriculum. A sample lesson plan is provided in Appendix F. Lesson plans were developed by the principal investigator using the scripted parts of the Strong Start curriculum. Control group. Approximately half (n = 19) of the students were assigned to the wait-list control condition. Students in the wait-list control condition continued to receive education services as usual. All teacher-reported and self-reported assessments were collected pre- and post-intervention for control students following the same assessment schedule as the intervention condition. The intervention was offered to students in the control group following post-assessment data collection. Due to time constraints with the end of the school year, only 60% of the lessons were provided to the wait-list students. Intervention training. Trained graduate students in school psychology taught Strong Start individually twice per week for five weeks. Implementers received one initial 1-hour training session on Strong Start prior to the first week of implementation of the curriculum, with 30 min sessions provided each subsequent week for the following lessons. The initial training session included an overview of the study, effective procedures for curriculum implementation, and an introduction to the types of lessons that were covered in Strong Start. Each training session covered key components of each lesson, which were described and modeled. Interventionists were given opportunities for 22 role-playing implementation as well as performance feedback (Felver, 2012). Additionally, training criteria for each lesson were developed, with interventionists demonstrating 100% mastery of curriculum and instruction through role-playing key components of each lesson. Treatment integrity. Self-report and observation measures were used to assess fidelity of implementation. These measures consisted of component checklists for each session (see Appendix G for an example). Each checklist listed the interventionist’s name, the school, start and stop time for the session, and the intervention components for that session. These checklists were provided to the interventionists during training. The interventionists completed the checklist following each lesson. To calculate fidelity of implementation, the number of completed intervention components was divided by the total number of components for each lesson. Self-reported data suggest high fidelity of implementation (m = 99.87%). Additionally, interventionists completed a quality of session summary item describing how well each session went for them. Quality of session summary items were a one question, 6 point Likert scale response (1 = poor, 6 = well). The range of mean ratings for sessions between the three interventionists ranged from 5.2 to 5.4. Interobserver agreement. In 30% of the sessions, the principal investigator served as an observer and completed a fidelity of implementation checklist which was compared to the interventionist’s self-report. Total agreement was calculated by dividing the number of agreements by the sum of the number of agreements plus number of disagreements. In total, interobserver agreement was 98.88%. Additionally, the observer 23 provided a quality of session summary rating for each lesson observed conducted by an interventionist. The mean quality of session summary rating was 5.1. Measures Assessment overview. The study used one measure, the SSBD, to identify at-risk students appropriate for a Tier II intervention. There were three quantitative dependent variables: the student self-report Strong Start Content Knowledge Assessment (SSCK), the teacher form of the Problem Behavior subscale of the Social Skills Improvement System (SSIS), and the teacher form of the Social Emotional Assets and Resiliency Scale (SEARS). Screening. The Systematic Screening of Behavior Disorders (SSBD; Walker & Severson, 1990) was used to identify students at-risk for behavior problems. The SSBD is a multiple-gating screening device for the identification of externalizing and internalizing behavior disorders (Severson, Walker, Hope-Doolittle, Kratochwill, & Gresham, 2007; Walker & Severson, 1990). There are three components to completing the SSBD. The first stage of the SSBD consists of teacher nominations of students on their class rosters. Teachers are presented with characteristics of externalizing and internalizing behaviors and are asked to rank-order ten students in their class that match the profiles of such behaviors. The second stage of the SSBD consists of teachers completing the Critical Events Index (CEI) and the Critical Frequency Index (CFI). The third stage of the SSBD consists of observations of academic engaged time in the classroom and social behavior on the playground (Severson et al., 2007). For this study, the first gating procedure was used to identify potential student participants. 24 Walker and Severson (1990) discuss the variety of reliability and validity studies that have been conducted for the multiple gates of the SSBD. The interrater reliability of the first gate, as measured by correlations of student ratings between pairs of teachers, ranged from .89 to .94 for externalizing behaviors and .82 to .90 for internalizing behaviors. Test-retest reliability, which consisted of teachers completing the gate again within ten day to one month, led to a range of scores from .81 to .88 for externalizing behaviors and .74 to .79 for internalizing behaviors. The criterion validity of the SSBD was explored using factor analyses: it was determined that a three factor structure (needs assistance, disruptive, and low achievement) explained 61% of the variance (Walker & Severson, 1990). The discriminant validity studies determined that using the first gate of the SSBD led to correct classification for internalizing and externalizing behaviors 84.69% of the time. The SSBD also has predictive validity: the researchers asked teachers to rate their students as either “externalizers,” “internalizers,” and “non-ranked students.” At a follow-up assessment, the researcher found that 69% of “externalizers” were still rated as externalizers, and 52% of “internalizers” were rated as internalizers. Thus, the SSBD was selected for the present study given its utility in identifying students at-risk for emotional and behavioral problems. Content knowledge. Strong Start Content Knowledge Assessment (SSCK; Whitcomb, 2009) is an experimental assessment developed Whitcomb, an author of Strong Start K – 2. The assessment consists of 18 items, with students receiving one point for each correct answer on 16 items and two points for each correct answer on two items (Appendix H). The assessment attempts to gauge the emotion knowledge that is taught 25 within the curriculum, which includes the identification of emotions, the recognition of others’ emotions, and problem solving strategies. Sample items include circling the appropriate emotion face out of three pictured possible responses (e.g., “Circle the surprised face”) or being presented a scenario and asked to circle the corresponding emotion or emotions that someone in the scenario experiences (e.g., “Henry was lying in bed and heard a loud noise. Circle the faces that show the feelings that he might have”). The assessment is a 10 min task and was administered to students out loud. Students completed the assessment with paper and pencil. No reliability or validity data currently exist for this experimental assessment (Whitcomb & Merrell, 2012). This measure was selected given its face validity in measuring content associated with the Strong Start intervention. Problem behavior. The Problem Behavior Subscale of the Social Skills Improvement System (SSIS; Gresham & Elliott, 2008) was used to assess students’ problem behavior. The SSIS is a revision to the Social Skills Rating System and includes updated national norms, additional subscales, and improved psychometrics (Gresham & Elliott, 2008; Gresham, Elliott, Vance, & Cook, 2011). The SSIS is a multi-informant assessment system that is designed to measure a student’s social skills, academic competence, and problem behaviors. For this study, the teacher version of the Problem Behavior scale was used to assess student problem behaviors. The problem behavior subscale was designed to assess problem behavior for students aged 3 to 18. Teachers or other educators familiar with the student may complete the SSIS. The problem behavior subscale consists of five subdomains: Externalizing, Internalizing, Bullying, Hyperactivity/Inattention, and Autism Spectrum. There are 46 items on the problem 26 subscale, and each item uses a 4-point Likert scale (0 = never, 4 = almost always) (Appendix I). Sample items include: “disobeys rules or requests” and “acts anxious with others” (Gresham, Elliott, Cook, Vance, & Kettler, 2010). The internal consistency of the teacher form of the SSIS Problem Behavior subscale range from .76 (Bullying) to .89 (Externalizing) for the combined five- to twelve-year-old norming population. The internal consistency for the combined Problem Behavior subscale is .94, suggesting that scale scores are relatively free from random error influence. The test-retest reliability scores, gathered by two repeated teacher assessments of 144 five- to twelve-year-old students ranged from .75 (Bullying) to .85 (Autism Spectrum), with a combined Problem Behavior subscale test-retest score of .81. Interrater reliabilities ranged in their correlation scores on the Problem Behavior subscales from .46 (Bullying) to .69 (Autism Spectrum), with an overall correlation of .57 (Gresham & Elliott, 2008). Gresham and Elliott (2008) used a variety of methods to demonstrate the SSIS’s validity. Bivariate Pearson-product correlations, conducted with a sample of five- to twelve-year-old students, among items on the Problem Behavior subscales range from .39 to .89. These results suggest that the items on the SSIS Problem Behavior subscale measure represent the constructs for which they were meant to correspond. Moderate levels of convergent validity evidence was obtained by examining how different raters rated scores of a similar trait: correlations across teacher and parent ratings of a sample of students of five- to twelve-year-olds ranged from .11 to .36. Moderate discriminant validity evidence was obtained by examining the relationships between the scores SSIS rating scales with other measures, as completed by different raters. Moderate to high 27 correlations between teacher ratings were found on the SSIS and the SSRS, the Behavior Assessment System for Children (Second Edition), and the Vineland Adaptive Behavior Scales (Second Edition) (Gresham & Elliott, 2008). Social-emotional skills. The Teacher Form of the Social Emotional Assets and Resiliency Scale (SEARS-T; Merrell, 2011) was used to assess students’ social and emotional skills. The SEARS is a multi-informant, strengths-based assessment system designed to measure emotional and behavioral skills that lead to satisfying relationships, promote social and academic functioning, and improve responses to adverse or stressful situations (Merrell, 2011; Nese, Doerner, Romer, Kaye, Merrell, & Tom, 2012). The SEARS assesses several domains critical for development: self-regulation, social competence, empathy, and responsibility (Merrell, 2011; Romer & Merrell, 2013). The teacher form of the SEARS (SEARS-T) consists of 41 items used for measuring social-emotional skills for children in kindergarten through twelfth grade. Each item uses a 4-point Likert scale (0 = never, 4 = almost always) (Appendix J). Sample items include “Works independently on assignments” or “Is respected or looked up to by other students.” The internal consistency of the SEARS-T subscales range from .91 (Empathy) to .95 (self-regulation and responsibility). The internal consistency for the total score is .98. Test-retest reliability consisted of teachers from two elementary schools in Washington completing the SEARS-T two weeks after they originally completed the form (Merrell, 2011). The test-retest reliability scores range on the subscale scores range from .84 (Empathy) to .92 (Social Competence and Responsibility), with a .94 score for the total score. 28 Merrell (2011) conducted exploratory and confirmatory factor analyses to determine the internal structure of the measure. The analyses confirmed a four-factor model, which explained 63.19% of the total variance. Bivariate Pearson-product correlations within the SEARS measures ranged from .61 to .79. Merrell (2011) also explored convergent construct validity. The SEARS-T correlation to the parent rating form of the SSRS rating subscales ranged from .39 to .79; the SEARS-T correlation with the Peer Relations scale of the School Social Behavior Scales-2 (SSBS-2) ranged from .76 to .87. Assessment Procedure Assessment schedule. Data were collected in two waves: pre-intervention and post-intervention. During the pre-intervention wave, teachers were given packets for each student that included the SSIS-PB and the SEARS-T. Teachers were given two weeks to complete each assessment for each student prior to intervention implementation. Teachers were given $80.00 for their efforts. The principal investigator conducted the SSCK assessment with students over two days during the initial wave of pre-intervention assessments. The principal investigator was blind to condition as random assignment had yet to occur when pre-intervention assessments were conducted. After the intervention was completed, teachers were again given packets for each student that included the SSIS-PB and the SEARS-T and were given two weeks to complete the assessments. Teachers were given $80.00 for completing the post- assessments. Two trained graduate students in school psychology, separate from the interventionists and blind to condition, conducted the post-intervention SSCK. The assessors were given an introduction to the measure that involved reading the items of the 29 measure and tips for general assessment procedures. The assessors conducted the SSCK over a period of two days after the intervention’s completion. Scoring procedures. Student and teacher responses on the quantitative dependent measures were scored, tabulated, and entered into SPSS 21.0 for Mac. The principal investigator served as the primary scorer for all dependent measures. 30 CHAPTER III RESULTS Analytic Approach This section includes a description of the analyses used to address research questions. Descriptive statistics of the student demographic sample, as well as the means and the standard deviations of the dependent variables of interest, are described. Baseline comparison analyses were used to explore differences between the intervention and control groups. Correlation analyses were used to explore relations between variables of interest to determine if covariates were necessary to include in subsequent analyses. Two- way, mixed effects analyses of variance (ANOVAs) were conducted to examine the effects of the Strong Start curriculum on students’ knowledge of social-emotional content knowledge, teachers’ reports of students’ problem behaviors, and teachers’ reports of students’ social and emotional assets and resilience. Further two-way, mixed effects ANOVAs were conducted to see if there were students that were at-risk, as determined by their standard scores on the SSIS-PB, to see if students that may have needed the intervention the most specifically benefited from Strong Start. A post hoc power analysis was conducted to determine the sample size of future experiments to find desired effect sizes. Descriptive Statistics and Baseline Comparisons Descriptive statistics were obtained for demographic variables of the sample. Baseline comparisons were conducted on student demographic variables to determine if there were significant differences between intervention and control groups. 31 Descriptive statistics for the student demographic sample are described in Table 1. Baseline comparisons between the intervention and control groups were conducted using an independent samples t-test to compare the age of the students and chi-square tests of independence were used to compare the percentage of male students in each group, the percentage of ethnicity of white students in each group, and the percentage of students with Individualized Education Plans (IEP) in each group. Students in both groups were an average of 8 years old. No significant difference in age between the intervention (M = 8.02, SD = 0.33) and the control groups (M = 8.12, SD = 0.28), t(33) = .91, p = .37 was found. A chi-square test of independence was used to compare the percentage of boys assigned to each condition, χ2 (1, N = 35) = .70, p = .40. A chi-square test of independence was used to compare the percentage of white students assigned to the intervention or the control condition. There was not a significant difference in whether white students were more likely to be assigned to the intervention condition, χ2 (1, N = 35) = 2.69, p = .10. A chi-square test of independence was also used to compare the percentage of students with an IEP as they were assigned to the intervention or control condition. There was not a significant difference in whether students with IEPs were more likely to be assigned to the intervention condition, χ2 (1, N = 35) = 2.76, p = .10. It is important to note that these findings may be limited due to the fact that there was a small overall sample size. Descriptive statistics for the each of the dependent variables between intervention and control groups across the two periods of assessment (pre-intervention and post- intervention). The mean scores and standard deviations for the three dependent variables (Strong Start Content Knowledge, Social Skills Improvement System-Problem Behavior, 32 and Social Emotional Assets and Resilience Scale) are listed in Table 2. From pre- intervention to post-intervention, the overall scores for the SSCK increased for both the intervention and the control group. Higher scores are reflective of increased student emotional knowledge on the measure (Whitcomb, 2009). Raw scores on the SSIS-PB were converted to standard scores, and descriptive statistics were derived for the standard scores (Gresham & Elliott, 2008). Across the two waves of assessment, standard score means for both groups decreased, suggesting that teachers rated their students lower for problem behaviors post-intervention. Raw scores on the SEARS-T were converted to T- scores, and descriptive statistics were conducted on the standard scores (Merrell, 2011). From the time pre-intervention assessments were conducted to post-intervention assessments were completed, standard score means for both groups increased, suggested that teachers rated their students higher for having skills such as self-regulation, empathy, social competence, and resilience. Correlations Among Variables of Interest Prior to running statistical analyses to address the main research questions of the dissertation, Pearson bivariate correlation coefficients were calculated to determine the relations among variables of interest. The correlation coefficients are displayed in Table 3. Student IEP status was negatively correlated with SEARS-T Pre-intervention scores, r(33) = -.53, p = .001, and with SEARS-T Post-intervention scores, r(33) = -.45, p = .003. These relations suggest that student IEP status was associated with lower social skills, self-regulation skills, empathy behaviors. Additionally, student IEP status was positively correlated with SSIS-PB Pre-intervention scores, r(33) = .56, p = .001, and with SSIS-PB Post-intervention scores, r(33) = .60, p = .001. These associations suggest that students 33 with an IEP were more likely to be rated higher for problem behaviors by their teachers. When student IEP status was included as a factor in the analysis of the main research questions, there were no differences in the statistical outcomes. The standardized teacher-rated assessments, the SEARS-T and the SSIS-PB, were associated with each other pre-intervention and post-intervention. The pre-intervention SEARS-T was positively associated with the SEARS-T at post-intervention, r(33) = .86, p = .001, while it was negatively associated with the SSIS-PB at pre-intervention, r(33) = -.86, p = .001, and at post-intervention, r(33) = -.83, p = .001. Students rated highly on positive behaviors at pre-intervention were likely to be rated similarly at follow-up; students rated highly on positive behaviors at pre-intervention were likely to be rated as having fewer problem behaviors pre-intervention and post-intervention. The post- intervention SEARS-T was negatively associated with the SSIS-PB both at pre- intervention, r(33) = -.74, p = .001, as well as at post-intervention, r(33) = -.81, p = .001. Students that were rated highly on positive behaviors post-intervention were likely to be rated as having fewer problem behaviors pre-intervention and post-intervention. Students that were rated as having more problem behaviors pre-intervention were associated with having higher problem behavior scores at post-intervention, as the pre-intervention SSIS- PB was positively associated with the SSIS-PB post-intervention, r(33) = .92, p = .001. The Strong Start Content Knowledge assessment, an unstandardized measure, was only positively associated with itself pre-intervention and post-intervention. SSCK scores at pre-intervention were positively associated with SSCK scores post-intervention, r(33) = .51, p = .002. Students that scored highly on the SSCK prior to intervention were likely to score highly on the SSCK after the intervention. 34 Effect of Strong Start on Student Content Knowledge The research question that students receiving the Strong Start curriculum as a Tier II intervention will lead to improved student SEL knowledge as measured by the SSCK was tested with a two-way mixed-effects ANOVA. The within-subjects factor was Time (Pre-intervention and Post-intervention) and the between-subjects factor was Intervention (Intervention or Control). The Intervention * Time interaction effect was not significant, F(1, 33) = 1.64, p = .21, indicating there was no significant difference between intervention and control groups over time. Consequently, consistent with conventional ANOVA logic, main effects of class and time were examined. The main effect of time was significant, F(1, 33) = 14.10, p = .001, η2partial = .30, indicating there was a significant increase on student responses on the SSCK over time (Table 4). Figure 1 displays the difference in SSCK scores across time as a result of participation in the intervention and control conditions. Figure 1. Mean Scores on the Strong Start Content Knowledge Assessment 15.5 16 16.5 17 17.5 18 18.5 Pre-Intervention Post-Intervention M e a n T o ta l S co re o n t h e S S C K Control Intervention 35 It is important to note the possible ceiling effects that occurred with the measure. The maximum score that could be obtained on the SSCK was 20, and the mean scores on the SSCK were 17.79 and 18.06 for the control and intervention groups, respectively. At the pre-intervention assessment one student (2.90% of the sample) correctly responded to all items. At the post-intervention assessment, nine students (25.70% of the sample) correctly responded to all items. This suggests that the measure may not have been to fully capture variation or growth in individual scores as both groups responded with high rates of accuracy to the measure. Effect of Strong Start on Ratings of Student Problem Behavior The research question that students receiving the Strong Start curriculum as a Tier II intervention will lead to decreased ratings of student problem behavior as measured by the SSIS-PB was tested with a two-way mixed-effects ANOVA. The within-subjects factor was Time (Pre-intervention and Post-intervention) and the between-subjects factor was Intervention (Intervention or Control). The Intervention * Time interaction effect was not significant, F(1, 33) = .81, p = .38, indicating there was no significant difference between teacher’s ratings of intervention and control groups over time. Consequently, consistent with conventional ANOVA logic, main effects of class and time were examined. Additionally, there was no main effect of time, F(1, 33) = 2.88, p = .10, η2partial = .08, (Table 5). Figure 2 displays the difference in SSIS-PB scores across time as a result of participation in the intervention and control conditions. 36 Figure 2. Mean Scores on the SSIS – PB Subscale Effect of Strong Start on Ratings of Social-Emotional Skills The research question that students receiving the Strong Start curriculum as a Tier II intervention will lead to increase ratings of student prosocial behavior as measured by the SEARS-T was tested with a two-way mixed-effects ANOVA. The within-subjects factor was Time (Pre-intervention and Post-intervention) and the between-subjects factor was Intervention (Intervention or Control). The Intervention * Time interaction effect was not significant, F(1, 33) = 1.25, p = .27, indicating there was no significant difference between teacher’s ratings of intervention and control groups over time. Consequently, consistent with conventional ANOVA logic, main effects of class and time were examined. The main effect of time was significant, F(1, 33) = 13.98, p = .001, η2partial = .29, indicating there was an increase of teacher ratings on the SEARS-T over time (Table 6). Figure 3 displays the difference in SEARS-T scores across time as a result of participation in the intervention and control conditions. 104 105 106 107 108 109 110 111 112 113 Pre-Intervention Post-Intervention M e a n S ta n d a rd S co re s o n t h e S S IS - P B Control Intervention 37 Figure 3. Mean Scores on the SEARS-T Form At-risk Student Analyses Separate two-way, mixed effects ANOVA analyses were conducted for students that were 1.5 SD above the mean and 1 SD above the mean, as rated by teachers on the SSIS-PB subscale. These analyses were conducted to determine if there were differences in student performance or student ratings over time for those that may have needed the intervention (in comparison to students in the average range of problem behavior). Eight students (four in the intervention group, four in the control group) were rated with standard scores on the SSIS-PB of over 122. Two-way, mixed effects ANOVAs, with intervention condition as the between-subjects factor and time as the within-subjects factor, were conducted to determine if there were impacts of Strong Start on student content knowledge, ratings of problem behaviors, and ratings of prosocial behaviors. 60 61 62 63 64 65 66 67 Pre-Intervention Post-Intervention M e a n T -S co re s o n t h e S E A R S -T Control Intervention 38 The Intervention * Time interaction on the student content knowledge measure was not significant, F(1, 6) = .28, p = .61. Consistent with ANOVA logic, main effects of class and time were examined. In both instances, no main effects were found. The Intervention * Time interaction on the ratings of student problem behavior was not significant, F(1, 6) = .37, p = .57. Consistent with ANOVA logic, main effects of class and time were examined. No main effect of time was found, F(1, 6) = 5.43, p = .06, η2partial = .48. The Intervention * Time interaction on the ratings of student prosocial behavior was not significant, F(1, 6) = 1.33, p = .29. Consistent with ANOVA logic, main effects of class and time were examined. The main effect of time was statistically significant, F(1, 6) = 33.33, p = .001, η2partial = .85. All students, regardless of assignment to condition, improved on ratings of prosocial behaviors over time. Fourteen students (five in the intervention group, nine in the control group) were rated with standard scores on the SSIS-PB of over 115. Two-way, mixed effects ANOVA, with intervention as the between-subjects factor and time as the within-subjects factor, were conducted to determine if there were impacts of Strong Start on student content knowledge, ratings of problem behaviors, and ratings of prosocial behaviors. The Intervention * Time interaction on the student content knowledge measure was not significant, F(1, 12) = 0.00, p = 1.00, η2partial = 0.00. Consistent with ANOVA logic, main effects of class and time were examined. There was no main effect of time, F(1, 12) = 3.51, p = .09, η2partial = 0.23. The Intervention * Time interaction on ratings of student problem behavior was not significant, F(1, 12) = .003, p = .96, η2partial = 0.00. Consistent with ANOVA logic, 39 main effects of class and time were examined. The main effect of time was statistically significant, F(1, 12) = 5.07, p = .04, η2partial = .30. All students, regardless of condition, improved on ratings of student problem behaviors. The Intervention * Time interaction on ratings of student prosocial behavior was not statistically significant, F(1, 12) = 3.55, p = .08, η2partial = .23. Consistent with ANOVA logic, main effects of class and time were examined. The main effect of time was statistically significant, F(1, 12) = 30.91, p = .001, η2partial = .72. Post Hoc Power Analyses Post hoc power analyses were conducted to determine the power obtained using SPSS for each of the dependent variables. For the Strong Start Content Knowledge Measure, the power analysis used a repeated-measures, within-between interaction design with the effect size found (.05), an alpha value of .05, a sample size of 35, with two groups, two measurement occurrences, and a correlation between repeated measurements of .51. The measured power for the SSCK was .24. For the problem behavior subscale of the SSIS, the power analysis used a repeated-measures, within-between interaction design with the effect size found (.02), an alpha value of .05, a sample size of 35, with two groups, two measurement occurrences, and a correlation between repeated measurements of .92. The measured power for the SSIS-PB was .14. For the teacher form of the SEARS, the power analysis used a repeated-measures, within-between interaction design with the effect size found (.04), an alpha value of .05, a sample size of 35, with two groups, two measurement occurrences, and a correlation between repeated measurements of .86. The measured power for the SEARS-T was .19. 40 CHAPTER IV DISCUSSION This section discusses the main findings of the study in terms of the treatment implementation integrity and the subsequent impact of Strong Start K-2 on the dependent variables of interest. Limitations of the study are examined and the impact of this study’s findings on subsequent future research will be explored. The purpose of this dissertation study was to explore the efficacy of Strong Start K-2 as a selected, Tier II intervention for students that were screened for being at-risk for internalizing or externalizing behaviors using a randomized-controlled trial design, a rigorous method of intervention evaluation. The effects of Strong Start were measured with three dependent variables: Strong Start Content Knowledge, to measure emotion knowledge that is covered in the curriculum; the teacher form of the problem behavior subscale of the Social Skills Improvement System; and the teacher form of the Social Emotional Assets and Resiliency Scale, to measure the social and emotional skills of students. Treatment integrity was measured with a combination of interventionist self- report and direct observation. Summary of Implementation Treatment Integrity. Treatment integrity data were collected across 100% of treatment sessions, and 30% of the sessions were observed for interobserver agreement of fidelity of implementation. Treatment integrity data consisted of self-report measures corresponding to the critical components of each lesson, and self-reported data suggested a high level of treatment integrity (99.87%). The range of self-reported content completion across all lessons was 99.63% to 100.00%. For each session, a quality of 41 session summary item was also completed by each interventionist, with the mean ratings of sessions across the entire intervention implementation ranging from 5.2 to 5.4 on a 6- point scale. The combination of treatment integrity and quality of session ratings suggest that Strong Start was implemented fully and implemented well. Interobserver agreement, comparing the principal investigator’s observations of sessions with the interventionist self-report, was also found to be high (98.88%), confirming the high fidelity level with which Strong Start was implemented. McIntyre, Gresham, DiGennaro, and Reed (2007) note that documenting treatment integrity is a critical component of determining the functional relationship between the implementation of the independent variable and subsequent changes in related dependent variables. Thus, the high level of treatment integrity suggests that a high level of internal validity was obtained with Strong Start implementation. This high level of treatment integrity compares favorably to the three peer- reviewed articles involving Strong Start. In two studies (Calderella et al., 2009; Kramer et al., 2010), Strong Start was implemented strictly by teachers, and 92 to 95% of lesson content was completed by teachers. In the other study featuring Strong Start (Whitcomb & Merrell, 2012), a counselor and a teacher implemented the curriculum in separate classrooms, with lesson components covered ranging form 80 to 100% of specific lessons covered. However, it is difficult to directly compare the mean ratings of content covered in this study and the peer-reviewed, published articles, as no mean ratings of the amount of content of lessons completed were provided in each of the studies. Yet, the implementation of Strong Start in this study is consistent with other studies that have 42 used the curriculum, suggesting that this intervention can be implemented accurately by a range of change agents in schools. Summary of Evidence of Efficacy of Strong Start Impact on Social-Emotional Knowledge. Results from this study suggest that all students, regardless of treatment condition, improved on their knowledge social and emotional constructs that are embedded within the Strong Start curriculum from pre- intervention to post-intervention. These results differ slightly from the other peer- reviewed article that has used the SSCK: Whitcomb and Merrell (2012) utilized an interrupted time-series design in which four first grade classrooms to evaluate the impact of Strong Start on the SSCK. The effect size demonstrated in that study (0.35) is comparable to the effect size found for time in this study (0.30). However, all students in the Whitcomb and Merrell (2012) study received the intervention with no comparison control group. The current study included a more rigorous evaluation of Strong Start but including a control group. Unlike Whitcomb and Merrell (2012), findings from the current study suggest that student changes in social emotional knowledge are not primarily due to exposure to the curriculum given that students in the intervention and control conditions both improved. It is unclear, however, if students in this study improved on social and emotional knowledge concepts as a result of time or being exposed to the measure for a second time. Impact on Problem Behaviors. Results from the current study suggest that there were no differences for students improving their problem behaviors before and after the intervention. Additionally, there was no statistically significant difference between the intervention and the control group. Other studies that have implemented Strong Start 43 have found improvements for problem behaviors as rated by teachers or parents. Whitcomb and Merrell (2012) found an improvement between pretest and posttest on teacher ratings of the problem behaviors subscale of the Social Skills Rating System; Kramer et al. (2010) and Calderella et al. (2009) found statistically significant decreases on internalizing subscales of standardized measures. However, the limitation with the above studies is that they relied on quasi-experimental methods such that there are no equivalent comparison groups to compare possible related changes of behaviors. For instance, Crean and Johnson (2013) implemented a clustered randomized controlled trial to evaluate an alternative SEL curriculum (PATHS) and found a curvilinear change in aggression and conduct problems for students from third to fifth grade, with intervention students demonstrating less problems over time. Although this study did not demonstrate differences between intervention and control students, it adds to the knowledge of SEL curricula that have implemented a more rigorous methodological design. Impact on Social-Emotional Assets and Behaviors. Results from this study suggest that all students, regardless of treatment condition, improved their prosocial behaviors between the pre-intervention and post-intervention teacher assessments. Quasi- experimental studies that have implemented Strong Start have found that teacher or parent ratings of prosocial behaviors have improved as a result of intervention implementation (Calderella et al., 2009; Kramer et al., 2010; Whitcomb & Merrell, 2012). However, all of these studies have used the peer relations subscale of the School Social Behavior Scales (Merrell, 2002), while this study implemented the SEARS, a more global measure that also includes items related to self-regulation, social competence, empathy, and responsibility (Merrell, 2011). Despite similarities in findings 44 with this study to other studies that have used Strong Start, it does not account for why students in the control condition did not differ from students in the intervention condition. Strong Start was not efficacious in improving behavioral and social-emotional outcomes in the current study, given that there were no differences between the intervention and control students on each of the dependent variables. However, the current study was powered to detect moderate effects, so it is possible that Strong Start may demonstrate small effects if a follow-up study is appropriately powered. There may be a couple of reasons why there were no interactions for time and intervention condition on the outcome measures. The students were attending an elementary school that was utilizing School-Wide PBIS (SWPBIS). Bradshaw, Waasdorp, and Leaf (2012) found in a multilevel analysis of randomized controlled effectiveness trial that schools implementing SWPBIS decreased children’s behavior problems, increased social-emotional functioning, and increased prosocial behaviors. The participating elementary school were practicing Tier I intervention supports, with components of their token economy implemented in the dissertation study. It may be that all the students had benefited from the supports embedded within the school and the teachers did not distinguish between control and intervention students on the basis of their problem behaviors and prosocial behaviors. Direct follow-up with the participating school district demonstrated that participating school used major and minor office discipline referrals (ODRs) at a daily rate per month that was fewer than the other schools in the school district that collected ODR data. ODRs are a means of describing the volume of problem behaviors that occur within a school setting (McIntosh et al., 2006). As the teachers in the school were using 45 ODRs less than their school district counterparts, this may considered another source of evidence that the school was benefitting from the SWPBIS supports that were already embedded within the school. Another reason that there may not be an interaction between may be the amount of time dedicated to the intervention. Although the Strong Start and the Strong Kids series have not been evaluated previously using a true experimental design, other SEL curricula have. McCormick, O’Connor, Cappella, and McLowry (2015) evaluated the INSIGHTS curriculum as it was applied within a cluster randomized controlled trial over a two year period for kindergarten and first grade students. McCormick et al. (2015) found intervention effects for students with high maintenance temperaments (e.g., high motor activity, negative reactivity to demands or feedback, and lower on-task behaviors). The Conduct Problems Prevention Research Group (2010) conducted a separate cluster randomized controlled trial for the Fast Track PATHS curriculum. Over two years, first graders received 57 lessons and second graders received 46 lessons. At the end of third grade, the authors found increased prosocial behavior, reduced aggressive behaviors, and improved academic engagement for students that received the intervention. It may be that Strong Start, delivered over a span of six weeks and only ten total lessons, may not have been a large enough dose with sufficient reinforcement and practice to lead to differentiation between intervention and control students. Although an empirical question, Strong Start may be more appropriate for use as a Tier 1 intervention. Finally, providing intervention to a small group of at-risk students may have had the opposite effect on student behavior. Mean changes on the SEARS and the SSIS were larger for control students than intervention students (Table 2). Shytenberg and 46 colleagues (2014) noted that affective responses to positively or negatively valenced stimuli are intensified with group coattention. Thus, students who were exposed to the content of Strong Start may have reinforced each others’ reactions, subsequently impacting their behaviors back in the classroom. Dishion and colleagues have discussed the phenomenon of peer contagion when a deviant peer group is brought together for intervention. Ironically, this type of approach may be counterproductive given that peers may reinforce each other’s’ problem behavior (Dishion, McCord, Poulin, 1999) Limitations There are a number of limitations that should be weighed when examining the results and considering the generalizability of the study. The discussion of the limitations includes the participant sample, testing effects, lack of blindness to condition, and the curriculum usage. Sample. This study used a small sample of students from a one elementary school in the Pacific Northwest. A majority of the students (77.14%) were in the “average” range of problem behavior (within 1.5 standard deviations from the mean), despite the attempt to screen students that may have had more externalizing and internalizing behaviors. Garner, Mahatmya, Brown, and Vesely (2014) note that students from suburban schools have different chances to develop social and emotional competencies than students in urban schools, with low-income children generally rated as having less prosocial skills or competencies than more middle-income or upper-income peers. It may be that Strong Start may be more beneficial to a population more “at-risk” than this study used. 47 Testing Effects and the Use of Experimental Measure. One of the limitations of the study was the use of the SSCK, an experimental measure that does not have reliability or validity evidence. Internal consistency measures pre- and post-intervention were low (.48 to .63, respectively) at both time points, and improvement for all students may be related to re-exposure to the measure a second time at post-intervention. Another limitation of the SSCK is that there may have been a ceiling effect for scores on the measure. Due to the high correct rates of response at the post-intervention assessment (just over a quarter of the sample correctly responded to all of the test items), there may not have been flexibility to measure or monitor the changes in student emotion knowledge. It is likely that the SSCK is not appropriate for a second grade population as it was used in this study. Alternative measures, such as the Assessment of Children’s Emotion Scales (ACES) (Schultz & Izard, 1998) or standardized measures of social- emotional comprehension (McKown, Allen, Russo-Ponsaran, & Johnson, 2013) may have been stronger measures for monitoring student change of emotion knowledge as a result of intervention implementation. It may also be worthwhile to consider developing measures that tap into deeper social-emotional constructs for older elementary school students. This may allow researchers to be able to more accurately measure changes in social-emotional knowledge as a result of SEL intervention implementation. Lack of True Blindness to Participant Assignment to Condition. Repeated measurements of prosocial and problem behaviors pre-intervention and post-intervention may have led to a response bias. The teachers may have been sensitive to the items presented pre-intervention, leading to improved ratings at post-intervention and inflating the amount of behavior change for all students. Further, it may be possible that teachers 48 perceived the intervention students to be at-risk and subsequently view them more negatively than control students (Montague & Rinaldi, 2001; Safran & Safran, 1985). Curriculum. The three previous peer-reviewed studies that implemented Strong Start implemented it as a universal intervention, the stated purpose of the intervention’s application. This study sought to use and evaluate Strong Start as a selected, Tier II intervention, given the short-term nature of the curriculum, ease of use and implementation, and the embedded practice and feedback features written into the curriculum. It may be that the best use for Strong Start is as a universal intervention. Power. Button et al. (2013) discuss the limitations of studies that have low statistical power. They note that studies with low statistical power are less likely to find effects that genuinely exist, have lowered probabilities of reflecting true effects (if the effects reach statistical significance), and that effects that are found may be overstated. The range of statistical power for each of the dependent variables in this study was .14 to .24, suggesting that a low amount of power was obtained to find true effects. Implications for Future Research The current study provides direction for future research on Strong Start, the Strong Kids series, and SEL curricula. This study used the SSBD as a tool for screening students and guiding the recruitment of participants before randomly assigning participants on the basis of gender and screening category. Future studies should consider using the SSIS (or a similar measure) before random assignment to condition to ensure that student participants have a more accurate label of “at-risk” status. It may be that by selecting all students that assented to participate may have minimized the number of truly at-risk students, thereby minimizing the desired targeted sample. 49 It is important to continue studying SEL curricula using experimental methods. This study did not distinguish improvements between students in the control and intervention conditions—experimental methods can help establish the efficacy and effectiveness of SEL programming. Additionally, this study used a direct student measure of emotion knowledge as well as teacher ratings of behavior. Future studies may want to consider including parent ratings (e.g., Kramer et al., 2010), direct observations (e.g., Felver, 2012), and academic outcomes (e.g., Schonfeld et al., 2014) to determine the full benefits of implementing SEL curricula. Future studies of the Strong Kids series, as well as other SEL curricula, should consider measuring the prevention benefits and long-term outcomes of implementing SEL interventions in school settings both in schools using PBIS and schools not using this multi-tiered framework of interventions for improving student social and behavioral functioning. Suldo, Gormley, DuPaul, and Anderson-Butcher (2014) note that mental health and academic outcomes are interrelated areas of student functioning. Future studies should seek to compare student behavior and academic outcomes as well as the long-term effects and benefits of receiving SEL programming (e.g., McCormick et al., 2015) by comparing students that have received SEL programming to students that have not received such programming. Conclusions Despite not finding interactions between intervention and time for the outcome measures in this study, this study contributed to our understanding of implementing SEL curricula in schools as a selected intervention. This study was the first study to implement any SEL curricula as a Tier II intervention and the first of the Strong Kids series to use a 50 true experimental design with random assignment to an intervention or control condition. The external validity of the study is limited, given the use of trained graduate students as intervention implementers and the small sample of student participants that may not generalize to the school population at large. Yet the internal validity of the experiment was strong, given that the intervention was implemented with a high level of fidelity and the use of a true experiment to evaluate student outcomes. This study evaluated the implementation of a SEL curriculum and applied the study within the context of a PBIS framework. We have limited knowledge of Tier II applications of SEL and mental health promotion curricula within a PBIS framework, but this study included key features of applications of Tier II interventions, such as the use of a behavior screener, an easy to implement curriculum, and a group-delivered intervention (Sulkowski, Joyce, & Storch, 2012). School based SEL and mental health interventions are important tools as schools become predominant locations for aiding in mental health related treatment (Franklin, Kim, Ryan, Kelly, & Montgomery, 2012; Suldo et al., 2014). This study evaluated a SEL curriculum experimentally within the context of schools for students that may have needed it most. 51 APPENDIX A TABLES Table 1 Student Demographic Characteristics by Intervention or Control Group Demographic Intervention n = 16 Control n = 19 t or Χ2 Mean Age in Years (SD) 8.02 (.33) 8.12 (.28) t = .91 Gender - % Male (N) 43.75 (7) 57.90 (11) Χ2 = .70 Ethnicity - % White (N) 87.50 (14) 63.16 (12) Χ2 = 2.69 % Latino (N) 6.25 (1) 26.32 (5) % Black (N) 6.25 (1) 0.00 (0) % Asian (N) 0.00 (0) 5.26 (1) Special Education - % IEP (N) 25.00 (4) 5.26 (1) Χ2 = 2.76 Note. All t or Χ2 values p > .05. 52 Table 2 Means and Standard Deviations of Dependent Variables by Intervention or Control Group Before and After Intervention Participation Pre-Intervention Post-Intervention Variable Intervention M (SD) Control M (SD) Intervention M (SD) Control M (SD) SSCK 16.56 (1.86) 17.05 (1.55) 18.06 (2.41) 17.79 (1.36) SSIS-PB 112.44 (23.85) 110.58 (14.94) 111.44 (16.12) 107.32 (13.87) SEARS-T 63.88 (8.21) 62.63 (8.29) 65.75 (6.84) 66.11 (6.47) 53 Table 3 Correlations between Variables of Interest Variable 1. 2. 3. 4. 5. 6. 7. 8. 9. 1. Age in Years - 2. Gender (Female) -.00 - 3. Student IEP .22 -.07 - 4. SSCK Pre- Intervention total -.01 -.11 -.06 - 5. SSCK Post- Intervention total -.30 .08 -.07 .51** - 6. SEARS-T Pre- Intervention total -.07 .12 -.53** .33 .22 - 7. SEARS-T Post- Intervention total -.08 .04 -.55** .27 .04 .86** - 8. SSIS-PB Pre- Intervention total .13 -.03 .56** -.26 -.21 -.86** -.74** - 9. SSIS-PB Post- Intervention total .13 -.06 .60** -.19 -.12 -.83** -.81** .92** - Note. **Correlation is significant at .01 level (two-tailed). 54 Table 4 Two-Way, Mixed Effects Analysis of Variance Summary Table for the Effects of Intervention and Time on Strong Start Content Knowledge Source df SS MS F p η2partial Between subjects Intervention 1 .21 .21 .04 .84 .00 Error between 33 164.14 4.97 Within subjects Time 1 21.73 21.73 14.10** <.01 .30 Intervention * Time 1 2.53 2.53 1.64 .21 .05 Error within 33 50.84 1.54 ** p < .01 55 Table 5 Two-Way, Mixed Effects Analysis of Variance Summary Table for the Effects of Intervention and Time on SSIS-PB Ratings Source df SS MS F p η2partial Between subjects Intervention 1 155.32 155.32 .24 .63 .01 Error between 33 21628.77 655.42 Within subjects Time 1 78.93 78.93 2.88t .10 .08 Intervention * Time 1 22.24 22.24 .81 .38 .02 Error within 33 905.84 27.45 t p = .10 56 Table 6 Two-Way, Mixed Effects Analysis of Variance Summary Table for the Effects of Intervention and Time on SEARS-T Ratings Source df SS MS F p η2partial Between subjects Intervention 1 83.47 83.47 .11 .75 .00 Error between 33 26183.40 793.44 Within subjects Time 1 124.24 124.24 13.98** <.01 .29 Intervention * Time 1 11.10 11.10 1.25 .27 .04 Error within 33 293.24 8.89 ** p < .01 57 Table 7 Office Discipline Referral (ODR) Rates at the End of the School Year across the Participating School District Schools April May June Participating School 3.51 2.81 0.80 Schools 2 - 5 8.25 8.21 3.86 58 APPENDIX B TEACHER CONSENT FORM Dear Teacher, My name is Michael Schwartz and I am a graduate student in the school psychology program at the University of Oregon. Your school has expressed willingness to consider adopting Strong Start for 1st and 2nd grade classrooms. Strong Start is a social emotional learning curriculum designed for kindergarten through third grade. During the 2013-2014 school year, I am planning on conducting a dissertation study to evaluate the effects of this curriculum with at-risk 1st and 2nd grade students. The study will investigate the outcomes of the Strong Start curriculum, where the goal for the program is to teach resiliency skills to children, skills that help children identify emotions, handle stressful situations, and respond to social situations positively. The purpose of this research study is to determine if Strong Start helps increase children’s knowledge of emotions, decrease problem behaviors, and increase social-emotional skills. Participating students will be randomly assigned to participate in Strong Start or be assigned to a wait-list control group and be offered Strong Start at the end of the study. Strong Start lessons will occur two times per week for 20 minutes over the course of 5 weeks. These lessons will be conducted by a trained graduate student in the school psychology program at the University of Oregon, will include approximately 6 1st and 2nd grade students, and will take place in a separate room on school campus during a time that is least disruptive to your classroom schedule. Prior to the implementation of the Strong Start curriculum, you will be asked to use a brief screening assessment to identify students in your classroom who may benefit from additional instructional support concerning social and emotional learning. The initial screening assessment should take no longer than 10 minutes for you to complete. Parent consent will be collected from parents of students who you identify as needing additional support. Parents will consent to having for their child to participate in Strong Start and participate in the brief assessment pre- and post-intervention. The pre- and post-assessment will take approximately 20 minutes to complete per child and involves teacher completion of two questionnaires—a problem behavior measure and a social-emotional skills measure. You will be compensated with a $75 gift certificate for your involvement in the study. Participating students will be asked to complete a short measure (administered as an interview) that assesses their emotion knowledge. This assessment will take about 5-10 minutes for students to complete and will be administered outside of your classroom. Your participation in this study is voluntary. Your decision to participate will not affect your job and you will not be evaluated for employment purposes. In order to maintain 59 confidentiality throughout the study, data collected will be marked with a code number and your name and the name of the student will be removed. In agreeing to participate, you are expressing that you are willing to support our implementation of social-emotional lessons for students in your classroom two times per week as well as to participate in assessment activities. If you have questions about this project, please contact me at (650) 766-2777 or at mschwart@uoregon.edu, or my advisor, Dr. Laura Lee McIntyre at (541) 346-7452 or at llmcinty@uoregon.edu. Sincerely, Michael Schwartz Your signature indicates that you have read and understand the information provided above, that you willingly agree to participate, that you may withdraw your consent at any time and discontinue participation without penalty, that you have received a copy of this form, and that you are not waiving any legal claims, rights, or remedies. Print Name: _______________________________ Signature: _______________________________ Date: ___________________ 60 APPENDIX C SYSTEMATIC SCREENING OF BEHAVIOR DISORDERS Instructions Step One: Carefully study the definitions and examples of externalizing and internalizing behavior problems presented on pages 3 and 4. Step Two: Select an externalizing and internalizing student group from the students in your class. Get a copy of your class roster. For both externalizing and internalizing dimensions, review the characteristic behavior patterns of ALL students in your class and select the 10 students who most closely match each of the behavior profiles. Second, write the names of the 10 students on each of the forms (internalizing and externalizing) whose characteristic behavior patterns most closely match those behavioral dimensions in Column One. The order or magnitude of the behaviors is not important at this point. The goal is simply to identify the groups of internalizing and externalizing students.  It is very important that you select the externalizing and internalizing groups according to how they actually behave (i.e., what they say and do) and not according to either presumed intent of their behavior or what you infer they may be thinking and feeling. The definitions and examples of externalizing and internalizing dimensions should be the sole criteria used to form the externalizing and internalizing groups (n = 10 each).  Even if you feel you do not have 10 students in your class who match the behavioral descriptions, it is important that you go ahead and make that difficult judgment. It is essential that 10 students each be identified for the two dimensions in order to assure that all students are adequately screened for the two behavior patterns.  Students in the Externalizing and Internalizing lists must be mutually exclusive. That is, a single student can appear on only one of the lists—not both! The authors recognize that occasionally a pupil will exhibit the behavioral characteristics associated with both externalizing and internalizing behavior patterns. When this occurs, simply judge the student on the dimension (i.e., externalizing or internalizing) which seems to best characterize her/his overall behavior pattern. The accuracy of the screening is often adversely affected by having a student’s name on both lists. If a teacher is concerned about a student’s behavior problems, then that student is likely to be high ranked on either the externalizing or internalizing behavioral dimension and be eligible for further screening.  Do not include the names of any students you have known less than one month on either the externalizing or internalizing groups. Step Three: Rank order each of the students on each of your externalizing and internalizing lists. 61 Use Column Two to rank order the 10 students listed in Column One who manifest externalizing and internalizing behavior problems according to the degree or extent that their behavior matches the definition of each of the respective dimensions of behavior problems. The student in your class assigned the rank of number one is the individual who, in your judgment, most exemplifies the externalizing or internalizing behavioral profile described below. The student who receives the rank of 10 is the one who least exemplifies this behavior profile. Rank order students based on your observations and interactions with them during the past month or longer. Rank Ordering on Internalizing Dimension Internalizing refers to all behavior problems that are directed inwardly (i.e., away from the external social environment) and that represent problems with the self. Internalizing behavior problems are often self-imposed and frequently involve behavioral deficits and patterns of social avoidance. Non-examples of internalizing behavior problems would be all forms of social behavior that demonstrate social involvement with peers that facilitate normal or expected social development. Examples include: Non-Examples include:  Having low or restricted activity levels  Initiating social interactions with peers  Not talking with other children  Having conversations  Being shy, timid, and/or unassertive  Playing with others, having normal rates or levels of social contact with peers  Avoiding or withdrawing from social situations  Displaying positive social behavior toward others  Preferring to play or spend time alone  Participating in games and activities  Acting in a fearful manner  Resolving peer conflicts in an appropriate manner  Not participating in games or activities  Joining in with others  Being unresponsive to social initiations by others  Not standing up for one’s self Instructions: 1) Review the definition of internalizing behavior and the list of all students in your class. 2) In Column One, enter the names of the 10 students whose characteristic behavior patterns most closely match the internalizing behavioral definition. 3) In Column Two, rank order the students listed in Column One according to the degree or extent to which each exhibits internalizing behavior. The student who 62 exhibits internalizing behavior to the greatest degree is ranked first and so on until all 10 students are rank ordered. M o st E x em p li fy in g I n te rn al iz in g B eh av io r Column One L ea st  I n te rn al iz in g  M o st Column Two List Internalizers Rank Order Internalizers Student Name Student Name 1 2 3 4 5 6 7 8 9 10 Rank Ordering on Externalizing Dimensions Externalizing refers to all behavior problems that are directed outwardly, by the child, toward the external social environment. Externalizing behavior problems usually involve behavioral excesses (i.e., too much behavior) and are considered inappropriate by teachers and other school personnel. Non-examples of behavior problems would include all forms of adaptive child behavior that are considered appropriate to the school setting. Examples include: Non-Examples include:  Displaying aggression toward objects or persons  Cooperating, sharing  Arguing  Working on assigned tasks  Forcing the submission of others  Making assistance needs known in an appropriate manner  Defying the teacher  Listening to the teacher  Being out of seat  Interaction in an appropriate manner with peers  Not complying with teacher instructions or directives  Following directions  Having tantrums  Attending to task  Being hyperactive  Complying with teacher requests  Disturbing others  Stealing  Not following teacher or school imposed rules 63 M o st E x em p li fy in g E x te rn al iz in g B eh av io r Column One L ea st  E x te rn al iz in g  M o st Column Two List Externalizers Rank Order Externalizing Student Name Student Name 1 2 3 4 5 6 7 8 9 10 Instructions: 1) Review the definition of externalizing behavior and then review a list of all students in your class. 2) In Column One, enter the names of ten students who characteristic behavior patterns most closely match the externalizing behavioral definition. In Column Two, rank order the students listed in Column One according to the degree or extent to which each exhibits externalizing behavior to the greatest degree is ranked first and so on until all 10 students are rank ordered. 64 APPENDIX D PARENT/GUARDIAN CONSENT FORM Dear Parent or Guardian, Your child’s school has adopted a curriculum called Strong Start for the 2013-2014 school year and has agreed to be a part of a research study to evaluate the effectiveness of this program. This curriculum teaches children skills that help them identify emotions, handle stressful situations, and interact positively with others. The research study is being conducted by Michael Schwartz, a school psychology PhD student at the University of Oregon, and supervised by Dr. Laura Lee McIntyre, a professor at the University of Oregon. We wish to invite your child to participate in the Strong Start curriculum evaluation. Teachers have identified students in their classroom who may enjoy and benefit from a small group instructional setting for the development of prosocial behaviors and relationship skills. The curriculum will be implemented in small groups of students and involves two 20-minute social skills lessons per week over a 5-week period. We estimate that 25 to 30 students across 4 classrooms will participate in the opportunity. Trained graduate students from the University of Oregon will serve as instructors for participating 1st and 2nd grade students. The purpose of this research study is to understand if Strong Start helps to increase student knowledge of emotions, increase student social-emotional skills, and decrease student problem behaviors. Several assessments will be used to evaluate the effectiveness of the Strong Start program. Teachers will be asked to rate student social-emotional skills and behavior before and after the implementation of Strong Start. In addition, students will be asked questions about their knowledge of emotions in social situations. These questions will be read to students and will also appear in a written and picture format. The student assessment will take your child approximately 5-10 minutes to complete and will be conducted by graduate students from the University of Oregon. After each assessment, your child will have the opportunity to select a small prize (like a sticker or an eraser). The assessments are intended to be fun and enjoyable, but your child is not required to participate. Children will not be penalized if they do not participate. The Strong Start lessons encourage children to think generally about their feelings and friendships. Students could potentially experience feelings of discomfort, such as stress or embarrassment when they talk about friendships. Most children enjoy the Strong Start lessons, but if completing the assessments or lessons is upsetting to your child in any way, he/she can stop answering the questions and can talk to his/her teacher or me. Assessment forms will not have children’s names on them. The forms will have an identification number that will correspond to a class list that I will keep. This list will be password-protected. Once all of the assessments have been completed, I will destroy the 65 class list. At that point, code numbers will make it so that I cannot link any data collected back to individual children. I will also be asking your child’s teacher to rate each individual student’s general emotions, social behaviors, and problem behaviors at two different points in time. Again, your child’s code number will be used so as to maintain his/her confidentiality. Your child’s participation in the project will help us better understand if Strong Start helps enhance the social competence and skills of young children. If you are interested in looking at the types of questions being asked or lessons being delivered in this project, I will leave copies of the assessments and curriculum with your child’s teacher. Additionally, your child will be asked if he or she is willing to participate in the project and it will be noted that their participation is also voluntary. We anticipate that there will be benefits to your child, and by extension, your home, as a result of your participation in this study. Strong Start seeks to improve self-regulation and prosocial skills across a variety of environments, including school and home. Additionally, by participating in this study, it will add knowledge for providing effective supports for students in schools. There may be potential risks to participation for your child, but we believe these risks are minimal. Students will be participating in small groups outside of the classroom, so there is a potential risk for a breach of confidentiality. We will go to great lengths to preserve your confidentiality as well as your students’ confidentiality. Additionally, there may be a stigma of being associated with a small group. In previous studies of Strong Start, students reported the experience was positive and fun, and activities are geared towards being interesting and informative. If you do not wish to have your child participate, there will be no negative consequences. Your choice to have your child participate in this project will not affect your child’s educational experience, your relationship to your child’s school, teacher, or with the University of Oregon. Additionally, you may still withdraw your consent and stop your child’s participation or stop your participation with the project at any time without penalty. If you have questions about this project, please contact me at (650) 766-2777 or at mschwart@uoregon.edu, or my advisor, Dr. Laura Lee McIntyre at (541) 346-7452 or at llmcinty@uoregon.edu. If you have any questions about your child’s rights as a research participant, please contact the Research Compliance Services, University of Oregon at (541) 346-2510 or ResearchCompliance@uoregon.edu. Please indicate if you DO or DO NOT give consent for your child to participate in this study by checking one of the boxes below and return this form to your child’s teacher by Nov. 1st, 2013. □ YES, I do give consent for my child (name)__________________________ to 66 participate in this study. □ NO, I do not give consent for my child (name)___________________________ to participate in this study. Your signature indicates that you have read and understand the information provided above, that you willingly agree to participate, that you may withdraw your consent at any time and discontinue participation without penalty, that you have received a copy of this form, and that you are not waiving any legal claims, rights, or remedies. Print Child name: _______________________________ Date: _____________ Print Parent/Legal Guardian name: __________________________________ Parent/Legal Guardian Signature: ___________________________________ 67 CONSENTIMIENTO DE PADRE/GUARDIÁN PARA LA PARTICIPACIÓN EN UN ESTUDIO DE INVESTIGACIÓN Universidad de Oregon Estimado Padre o Guardián, Esta es una oportunidad para su estudiante de participar en un curriculum llamado Strong Start en el año escolar 2013-2014. Los maestros de la escuela estuvieron de acuerdo de ser parte de un estudio de investigación para evaluar la efectividad de este programa. Este curriculum le enseña a los niños destrezas que ayudan a identificar emociones, manejar situaciones estresantes e interactuar positivamente con otros. Michael Schwartz, un estudiante de PhD en psicología escolar en la Universidad de Oregon, dirigirá el estudio de investigación. El estudio será supervisado por la Doctora Laura Lee McIntyre, una profesora en la Universidad de Oregon. Deseamos invitar a su niño/a a participar en la evaluación del curriculum Strong Start. Los maestros han identificado estudiantes en el aula que podrían disfrutar y beneficiarse de un grupo pequeño de instrucción para el desarrollo de comportamientos pro-sociales y destrezas para relacionarse. El curriculum será implementado en grupos pequeños de estudiantes que involucrarán dos lecciones a la semana de 20 minutos cada una acerca de destrezas sociales por un periodo de 5 semanas. Estimamos que de 25 a 30 estudiantes de 4 aulas diferentes participarán en el programa. Estudiantes de posgrado de la Universidad de Oregon serán entrenados para ser los instructores para los estudiantes de 1er y 2do grado que participen. El propósito de este estudio de investigación es entender si Strong Start ayuda a los estudiantes a incrementar el conocimiento de emociones, destrezas socio-emocionales, y disminuye los problemas de comportamiento estudiantil. Las evaluaciones serán utilizadas para evaluar la efectividad de el programa Strong Start. Le preguntaremos a los maestros que tomen un índice de las destrezas socio-emocionales y comportamiento del estudiante antes y después de la implementación de Strong Start. Adicionalmente, le preguntaremos a los estudiantes preguntas acerca de su conocimiento acerca de emociones en situaciones sociales. Le leeremos las preguntas a los estudiantes y también serán presentadas de forma escrita y en formato de foto. La evaluación de estudiantes le tomará a su niño/a aproximadamente de 5 a 10 minutos para completar y será realizada por un estudiante de posgrado de la Universidad de Oregon. Después de cada evaluación, su niño tendrá la oportunidad de ganarse un premio pequeño (como una etiqueta o un borrador). Nuestra intención es que las evaluaciones sean divertidas, pero su niño/a no está obligado a participar. Los niños/as no serán penalizados si no participan. Las lecciones de Strong Start motivan a los niños/as a pensar generalmente acerca de sus sentimientos y amistades. Potencialmente, los estudiantes podrían experimentar sentimientos de incomodidad, como estrés o vergüenza al hablar acerca de sus amistades. La mayoría de los niños/as disfrutan las lecciones de Strong Start, pero si la participación en las evaluaciones o en las lecciones es desagradable para su niño/a de cualquier manera, 68 el/ella puede parar de responder las preguntas y puede hablar con su maestro/a o conmigo. Las hojas de evaluación no tendrán los nombres de los niños/as. Las hojas tendrán un número de identificación que corresponderá a la lista de la clase que yo tendré. Esta lista estar protegida con contraseña. En cuanto todas las evaluaciones se hayan completado, yo destruiré la lista de la clase. En ese momento, los números del código no me permitirán conectar los datos recolectados con los nombres de los niños. También, le preguntaremos al maestro de su niño/a que tomen un índice emociones generales, comportamientos sociales y comportamientos problemáticos de cada niño/a en dos puntos diferentes de tiempo. Otra vez, el número de código de su niño/a será utilizado para mantener la confidencialidad de el/ella. La participación de su niño/a en el proyecto nos ayudará a entender si Strong Start ayuda a aumentar la competencia social y destrezas de niños/as jóvenes. Si está interesado/a en ver el tipo de preguntas que se harán o las lecciones que se van a dar, yo dejaré copias de las evaluaciones y el curriculum con el/la maestro/a de su niño/a. Adicionalmente, le preguntaremos a su niño/a si el/ella quiere participar en el proyecto y notaremos si su participación es voluntaria. Anticipamos que habrán beneficios para su niño/a, y en extensión, su hogar, como resultado de su participación en el estudio. Strong Start busca mejorar capacidades de auto-regulación y destrezas pro-sociales a través de una variedad de entornos, incluyendo la escuela y el hogar. Puede que haya un potencial riesgo en la participación de su niño/a, pero creemos que estos riesgos son mínimos. Los estudiantes participarán en grupos pequeños fuera del aula, por esta razón no habrá riesgo de incumplimiento de confidencialidad. Nosotros haremos mucho para preservar la confidencialidad de usted y la de su estudiante. Adicionalmente, hay una estigma de estar asociado con un grupo pequeño. En previos estudios de Strong Start, los estudiantes reportaron que la experiencia de positiva y divertida, y las actividades están creadas para ser interesantes e informativas. Si no desea que su niño/a participe, no habrán ningunas consecuencias negativas. Es su decisión de dejar que su niño/a participe en este proyecto y no afectará la experiencia educativa de su niño/a, su relación con la escuela, maestra/o, o con la Universidad de Oregon. Adicionalmente, usted puede retirarse de su consentimiento y parar la participación con el proyecto en cualquier momento sin sanción. Si tiene preguntas acerca del proyecto, por favor contácteme por teléfono al (650) 766- 2777 o por correo electrónico mschwart@uoregon.edu, o a mi aconsejador, Dr. Laura Lee McIntyre al (541) 346-7452 o pro correo electrónico a llmcinty@uoregon.edu. Si tiene alguna pregunta acerca de los derechos de su niño/a como participante de investigación, por favor llame al Research Compliance Services, Universidad de Oregon al (541) 346-2510 o por correo electrónico a ResearchCompliance@uoregon.edu. 69 Por favor indique si usted NO QUIERE o QUIERE dar consentimiento para que su niño/a participe en este estudio al marcar uno de los cuadros de abajo y devuelva esta forma a la profesora, a través de su niño/a, Nov. 1, 2013 a mas tardar. □ SI, le doy consentimiento a mi niño/a (nombre)__________________________ para que participe en este estudio. □ NO, no le doy consentimiento a mi niño/a (nombre)________________________ para que participe en este estudio. Su firma indica que ha leído y entendido la información provista en esta forma, que usted voluntariamente está de acuerdo con participar, que usted puede retirarse de su consentimiento en cualquier momento y descontinuar su participación sin ninguna sanción, que usted tiene una copia de esta forma, y que usted no está cediendo una reclamación legal, de sus derechos, o de remedios. Nombre del Niño/a: _______________________________ Fecha: _____________ Nombre del Padre/Guardián Legal: __________________________________ Firma del Padre/Guardián Legar: ___________________________________ 70 APPENDIX E STUDENT ASSENT FORM Dear Student, My name is Michael Schwartz and I am a student at the University of Oregon. I am working on a project that will help me to learn more about how children in elementary school understand feelings and friendship. Sometime soon, a teacher from the University of Oregon will begin teaching lessons twice a week from a book called Strong Start to help you learn about understanding feelings and being a good friend. Today, and on two other days several weeks from now, I will also be asking you to answer some questions. These questions will each take about 5-10 minutes to complete. Please answer the questions the best you know how. You will not get a grade on these questions. Your teachers will not see your answers, and I will not write your name on this paper. Instead, I am going to give you your own number. When I look at these papers later, I will not know who answered the questions. In other words, your answers will be kept secret. If the questions I read make you feel uncomfortable in any way, you can talk to me, your teacher, or your parents about it. You do not have to participate in these activities if you do not want to, and you will not be in trouble if you decide not to participate. Also, you can ask me any questions at any time that you might have. If you choose to participate in these Strong Start activities, you will have the opportunity to choose a small prize when the project is over. There may be other benefits as well— you may be able to help understand how to help other children better, and to be more successful with your emotions as well as your friends. If you would like to participate, write your name here: ____________________________ Thank you, Michael Schwartz 71 APPENDIX F STRONG START LESSONS AND TOPICS Lesson number and title Main purpose 1. The Feelings Exercise Group Introduce students to the Strong Start curriculum 2. Understanding Your Feelings 1 Teach students to name basic emotions 3. Understanding Your Feelings 2 Teach student appropriate ways to express positive and negative feelings 4. When You’re Angry Teach students to deal with their anger and helpful ways of managing anger 5. When You’re Happy Teach students to understand and express happiness and make connection between happy feelings and the use of positive thinking 6. When You’re Worried Teach students to manage stress, anxiety, and worries 7. Understanding Other People’s Feelings Teach students how to identify others’ feelings and how to empathize 8. Being a Good Friend Teach students basic social skills and communication skills 9. Solving People Problems Teach students to solve problems and conflicts with others 10. Finishing UP! Review of major concepts in the Strong Start program Note. Adapted from Merrell et al. (2007). 72 APPENDIX G STRONG START SAMPLE LESSON Underline = Fidelity of Implementation Normal font = Statement by Interventionist Highlight = Change based on specifics Italics = Interventionist Action Strong Start Lesson 4 I. Previous Lesson a. During our last meeting, we discussed how to understand our feelings and okay ways for showing them. Raise your hand if you can tell me an okay way of showing one of the feelings. i. Wait for student response and provide feedback appropriately. 1. E.g., Telling someone you are sad is an okay way to show your feelings. b. How about a not okay way? i. Wait for student response and provide feedback appropriately. 1. E.g., You are right! Using your hands against someone when you are sad or angry is not an okay way to show your feelings. c. We also learned about good feelings and not good feelings. Raise your hand if you can remember a good feeling. [Pause] How about a not good feeling? II. Introduction a. Today we will talk about a feeling called anger. Anger is a normal feeling and everybody feels angry sometimes. We will learn what anger looks like and when it might happen. We will also learn ways to deal with our anger so that we don’t hurt ourselves or others. III. Read a Book on Feelings a. We will start today’s lesson by reading [book title] by [author]. I want you to focus on which characters feel angry, if it’s a good or not good feeling, what the character looks like and what the character did when they were angry. b. Point out actions or behaviors of characters. c. These incidents will be underlined in the book that you read. When you come up to one of these situations you may ask: i. Which character was angry? ii. Do you think it was a good or not good feeling? iii. What did the character look like when he or she was angry? iv. What did the character do when he or she was angry? IV. Different Forms of Anger a. Use Supplement 4.1 73 b. This is angry. Angry is generally not a good feeling. What does angry look like in this picture? Raise your hand if you’ve ever felt angry. What did your body look like or feel like? i. Have students share what their bodies felt like. It might include feeling hot, having tight muscles, feeling shaky or near tears. Provide feedback as appropriate. ii. Engage in Think/Pair/Share Activity 1. Now I want for all of you to think of a time that you felt angry. When you are ready, I want you to turn to your partner and share your idea. iii. Wait for 20 – 30 seconds. 1. Thank you for sharing your experiences with each other. Can I have two people share their experiences with the whole group? iv. Pull out sheet of paper for the group. 1. I heard different ways and words of how people think of being angry. What are some other examples of words that make you think of anger? 2. [If no response] Some words that I might think of relating to anger are angry, mad, furious, or upset. Are there others that you can think of? V. Ways to Help and Ways to Hurt a. Today we’ve been talking about anger. All people feel angry sometimes, and it’s all right to feel angry. Most of the time, something happens to make us angry. This is called a spark. Something sparks our anger. b. Like the kind of sparks that start a fire, there are things we can do to stop anger and keep it from spreading, and there are things we can do that spread the anger and hurt ourselves and others. c. One time, Henry really wanted to go to his friend’s house but his mom said no. When this happened, his muscles got tight and he began to feel hot. i. What sparked Henry’s anger in this situation [his mom saying no]? ii. How did his body feel [muscles were tight and body was hot]? iii. What do you think Henry did next? d. Take out Supplement 4.2 after student feedback. e. There are two ways that you can deal with your anger: “Ways that Help” and “Ways that Hurt.” i. Henry felt really angry when he couldn’t go to his friend’s house. Since this happened a long time ago, Henry didn’t know about “Ways that Help” and “Ways that Hurt” when handling his anger. ii. In this situation, Henry stuck out his tongue, stomped his feet, and slammed the door to his room. When he was alone in the room, he kicked the wall. iii. This kind of behavior is what I call “Ways that Hurt.” Henry stayed mad and wasn’t acting nicely. When he got older and the same thing happened, he knew how to make himself feel better. 74 iv. He learned a special trick called Stop, Count, In, Out. f. Stop, Count, In, Out takes four steps: i. Stop = When you feel a spark, stop what you are doing. ii. Count = Count to 10. iii. In = Take a deep breath in. iv. Out = Breathe out. g. Let’s practice it together. Like Henry, let’s say our mom or dad says we can’t go over to someone’s house to play. i. First step is to stop. Let’s stop what we are doing [have everyone freeze]. ii. Now, let’s count to 10 either out loud or to yourself. [count to 10 with students]. iii. Good. Now we take a deep breath in [model to students]. iv. [Breathe out] And we finish by breathing out! Great job! VI. Assessment of Anger a. Let’s do some practice. i. Go over “sparks” and include Henry and what Henry did following the spark. Assess for student understanding by asking: “Is it a way that helps or a way that hurts” for each. On those that hurt, ask students how Henry can use a “Way that Helps” while referring to the handout. VII. Think, Pair, Share a. You guys are doing great work today! To end the session, I want you think about the time that you shared with your partner. Tell your partner if you handled your anger in a way that helps. b. Wait for 20 to 30 seconds. c. Thank you for sharing with your partner. May I have two examples of what you guys did when you were angry? d. [Optional based on time remaining] Have students draw the even that they shared with their partner. VIII. Conclusion a. Thank you for providing such thoughtful responses! Everyone feels angry sometimes, and there are many ways that we can handle our anger. It is important to use a “Way that Helps” so that we don’t hurt ourselves or others. 75 APPENDIX H STRONG START FIDELITY OF INTERVENTION CHECKLIST Observer Name: __________________ Teacher Name: _________________ Date: _____________________ Start Time: _________ End Time: _________ Directions: Check each component that the interventionist covers while teaching the lesson. Complete the Summary at the bottom and return to the investigator. Lesson 4: When You’re Angry ________ 1) Reviewed previous topic and main ideas from last lesson (appropriate and inappropriate ways of expressing emotions). ________ 2) Introduced the topic to be learned today. ________ 3) Read a book and used the questions to prompt discussion on angry feelings ________ 4) Used the supplement to show different feelings and expressions of anger ________ 5) Explain the “Ways to Help” and “Ways to Hurt” when angry and introduce “Stop, Count, In, Out.” ________ 6) Use examples provided in curriculum to assess understanding of how to respond appropriately when angry ________ 7) Engage the class in a think, pair, share activity about how student handled a situation when they were angry ________ 8) Review the lesson objectives Summary Components Completed: # of components (____ /8) x 100 = ______% of intervals Quality of Session Summary – How well did the session go? Poor Well 1 2 3 4 5 6 Notes: 76 APPENDIX I STRONG START CONTENT KNOWLEDGE ASSESSMENT Student Name: ________________________ Participant ID: ________________________ Date: _________________________ Pre-Assessment Post-Assessment Strong Start Content Knowledge Assessment 77 78 79 80 81 APPENDIX J SOCIAL SKILLS IMPROVEMENT SYSTEM Student Name: ________________________ (completed by teacher) Participant ID: ________________________ Date: _________________________ Pre-Assessment Post-Assessment Directions: Please read each item and think about this student’s behavior during the past month. Then decide how often this student displays the behavior. Problem Behaviors: Never Seldom Often Almost Always 1. Acts without thinking. 0 1 2 3 2. Is preoccupied with object parts. 0 1 2 3 3. Bullies others. 0 1 2 3 4. Becomes upset when routines change. 0 1 2 3 5. Has difficulty waiting for turn. 0 1 2 3 6. Does things to make others feel scared. 0 1 2 3 7. Fidgets or moves around too much. 0 1 2 3 8. Has stereotyped motor behaviors. 0 1 2 3 9. Forces others to act against their will. 0 1 2 3 10. Withdraws from others. 0 1 2 3 11. Has temper tantrums. 0 1 2 3 12. Keeps others out of social circles. 0 1 2 3 13. Breaks into or stops group activities. 0 1 2 3 14. Repeats the same thing over and over. 0 1 2 3 82 15. Is aggressive toward people or objects. 0 1 2 3 16. Gets embarrassed easily. 0 1 2 3 17. Cheats in games or activities. 0 1 2 3 18. Acts lonely. 0 1 2 3 19. Is inattentive. 0 1 2 3 20. Has nonfunctional routines or rituals. 0 1 2 3 21. Fights with others. 0 1 2 3 22. Says bad things about self. 0 1 2 3 23. Disobeys rules or requests. 0 1 2 3 24. Has low energy or is lethargic. 0 1 2 3 25. Gets distracted easily. 0 1 2 3 26. Uses odd physical gestures in interactions. 0 1 2 3 27. Talks back to adults. 0 1 2 3 28. Acts sad or depressed. 0 1 2 3 29. Lies or does not tell the truth. 0 1 2 3 30. Acts anxious with others. 0 1 2 3 83 APPENDIX K SOCIAL EMOTIONAL ASSETS AND RESILIENCY SCALE Student Name: ________________________ (completed by teacher) Participant ID: ________________________ Date: _________________________ Pre-Assessment Post-Assessment Directions: To be completed by a teacher or other school personnel for students in Grades K to 12 (ages 5 to 18 years). Never Some- times Often Always 1. Likes to do his/her best in school 0 1 2 3 2. Feels sorry for others when bad things happen to them 0 1 2 3 3. Knows when other students are upset, even when they say nothing 0 1 2 3 4. Is good at understanding the point of view of other people 0 1 2 3 5. Works independently on assignments, without help 0 1 2 3 6. Is comfortable talking to many different people 0 1 2 3 7. Makes friends easily 0 1 2 3 8. Expresses disagreement with other people without fighting or arguing 0 1 2 3 9. Tries to understand how other students feel when they are not doing well 0 1 2 3 10. Is a good listener 0 1 2 3 11. Other students ask him/her to hang out with them 0 1 2 3 84 12. People think she/he is fun to be with 0 1 2 3 13. Is well-liked by teachers and other students 0 1 2 3 14. Is good at solving problems 0 1 2 3 15. Is good at starting conversations 0 1 2 3 16. Understands how other people feel 0 1 2 3 17. Makes good decisions 0 1 2 3 18. Is good at settling disagreements of other students 0 1 2 3 19. Is comfortable telling other people how he/she feels 0 1 2 3 20. Asks others for help when she/he needs it 0 1 2 3 21. Understands how people could feel different about the same thing 0 1 2 3 22. Stays in control when he/she gets angry 0 1 2 3 23. Cares what happens to other people 0 1 2 3 24. Thinks before she/he acts 0 1 2 3 25. Is comfortable being in large groups 0 1 2 3 26. Other people see him/her as a leader 0 1 2 3 27. Is respected or “looked up to” by other students 0 1 2 3 28. Is dependable, someone you can rely on 0 1 2 3 29. Thinks of her/his problems in ways that help 0 1 2 3 30. Accepts responsibility when she/he needs to 0 1 2 3 31. Is good at telling stories and jokes 0 1 2 3 32.Stands up for herself/himself 0 1 2 3 85 33.Knows how to calm down when stressed or upset 0 1 2 3 34. Knows how to identify and change negative thoughts 0 1 2 3 35. I trust her/him 0 1 2 3 36. Works well with other students on group projects 0 1 2 3 37. Can figure out whether or not negative thoughts are realistic 0 1 2 3 38. Can identify errors in the way he/she thinks about things 0 1 2 3 39. Knows how to set goals for what she/he wants in life 0 1 2 3 40. Is able to handle problems that really bother other students 0 1 2 3 41. When life is hard, doesn’t let things get to him/her 0 1 2 3 86 REFERENCES CITED Albers, C.A., Glover, T.A., & Kratochwill, T.R. (2007). Where are we, and where do we go now?: Universal screening for enhanced educational and mental health outcomes. Journal of School Psychology, 45, 257-263. Anderson, C. & Borgmeier, C. (2010). Tier II interventions within the framework of school-wide positive behavior support. Behavior Analysis in Practice, 3, 33-45. Bagdi, A., & Vacca, J. (2005). Supporting early childhood social-emotional well being: The building blocks of early learning and school success. Early Childhood Education Journal, 33, 145-150. Becker, K.D., & Domitrovich, C.E. (2011). The conceptualization, integration, and support of evidence-based interventions in schools. School Psychology Review, 40, 582-589. Bradshaw, C.P., Mitchell, M.M., & Leaf, P.J. (2010). Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. Bradshaw, C.P., Waasdorp, T.E., Leaf, P.J. (2012). Effects of school-wide positive behavioral interventions and supports on child behavior problems. Pediatrics, 130, 1136-1145. Brooks, J.E. (2006). Strengthening resilience in children and youths: Maximizing opportunities through the schools. Children and Schools, 28, 69-76. Button, K.S., Ioannidis, J.P.A., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S.J., & Munafo, M.R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews: Neuroscience, 14, 365-376. Caldarella, P., Christensen, L., Kramer, T.J., & Kronmiller, K. (2009). Promoting social and emotional learning in second grade students: A study of the Strong Start curriculum. Early Childhood Education Journal, 37, 51-56. CASEL (2012). 2013 CASEL Guide: Effective social and emotional learning programs: Preschool and elementary school edition. Retrieved from casel.org/casel- guide/download-the-2013-guide Chafouleas, S.M., Volpe, R.J., Gresham, F.M., & Cook, C.R. (2010). School-based behavioral assessment within problem-solving models: Current status and future directions. School Psychology Review, 39, 343-349. 87 Conduct Problems Prevention Research Group (2010). Effects of a multiyear universal social-emotional learning program: The role of student and school characteristics. Journal of Consulting and Clinical Psychology, 2, 156-168. Cowen, E.L. (1994). The enhancement of psychological wellness: Challenges and opportunities. American Journal of Community Psychology, 22, 149-179. Crean, H.F., Johnson, D.B. (2013). Promoting Alternative Thinking Strategies (PATHS) and elementary school aged children’s aggression: Results from a cluster randomized trial. American Journal of Community Psychology, 1-2, 56-72. Debnam, K.J., Pas, E.T., & Bradshaw, C.P. (2012). Secondary and tertiary support systems in schools implementing school-wide positive behavioral interventions and supports: A preliminary descriptive analysis. Journal of Positive Behavior Interventions, 14, 142-152. Dishion T.J., McCord J., Poulin F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, 755–764. Doll, B., & Cummings, J.A. (2008). Best practices in population-based school mental health services. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (1333-1347). Bethesda, MD: National Association of School Psychologists. Doll, B., Jones, K., Osborn, A., Dooley, K., & Turner (2011). The promise and the caution of resilience models for schools. Psychology in the Schools, 48, 652-659. Doll, B., & Lyon, M.A. (1998). Risk and Resilience: Implications for the delivery of educational and mental health services in schools. School Psychology Review, 27, 348-363. Domitrovich, C.E., Cortes, R.C., & Greenberg, M.T. (2007). Improving young children’s social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. The Journal of Primary Prevention, 28, 67-91. Durlak, J.A., Weissberg, R.P., Dymnicki, A.B., Taylor, R.D., Schellinger, K.B. (2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. Elias, M.J., & Haynes, N.M. (2008). Social competence, social support, and academic achievement in minority, low-income, urban elementary school children. School Psychology Quarterly, 23, 474-495. Elias, M.J., Zins, J.E., Graczyk, P.A., Weissburg, R.P. (2003). Implementation, sustainability, and scaling up of social-emotional and academic innovations in public schools. School Psychology Review, 32, 303-319. 88 Faul, F., Erdfelder, E., Lang, A.G., & Buchner, A. (2007). G*Power 3: A flexible statistical analysis program for the social, behavioral, and biomedical sciences. Behavioral Research Methods, 39, 175-191. Felver, S. (2012). A pilot study of Strong Start: Preliminary evidence of feasibility and efficacy of social and emotional learning in preschool. (Unpublished doctoral dissertation). University of Oregon, Eugene. Fox, L., & Hemmeter, M.L. (2009). A program-wide model for supporting social emotional development and addressing challenging behavior in early childhood settings. In W. Sailor, G. Dunlap, G. Sugai, and R. Horner (Eds.), Handbook of Positive Behavior Support (pp.177-202). New York: Springer. Franklin, C.G.S., Kim, J.S., Ryan, T.N., Kelly, M.S., & Montgomery, K.L. (2012). Teacher involvement in school mental health interventions: A systematic review. Child and Youth Services Review, 34, 973-982. Garner, P.W., Mahatmya, D., Brown, E.L., & Vesely, C.K. (2014). Promoting desirable outcomes among culturally and ethnically diverse children in social emotional learning prorams: A multilevel heuristic model. Educational Psychology Review, 26, 165-189. Greenberg, M.T., Domitrovich, C., & Bumbarger, B. (2001). The prevention of mental disorders in school-aged children: Current state of the field. Prevention & Treatment, 4, 1-62. Greenberg, M.T., Weissberg, R.P., O’Brien, M.U., Zins, J.E., Fredericks, L., Resnik, H., & Elias, M.J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466-474. Gresham, F.M, & Elliott, S.N. (2008). Social Skills Improvement System: Rating Scales Manual. Minneapolis, MN: Pearson, Inc. Gresham, F.M., Elliott, S.N., Cook, C.R., Vance, M.J., & Kettler, R. (2010). Cross- informant agreement for ratings for social skill and problem behavior ratings: An investigation of the social skills improvement system—rating scales. Psychological Assessment, 22, 157-166. Gresham, F.M., Elliott, S.N., Vance, M.J., & Cook, C.R. (2011). Comparability of the social skills rating system to the social skills improvement system: Content and psychometric comparisons across elementary and secondary age levels. School Psychology Quarterly, 26, 27-44. 89 Grossman, D.C., Neckerman, H.J., Koepsell, T.D., Liu, P.Y., Asher, K.N., Beland, K., et al. (1997). Effectiveness of a violence prevention curriculum among children in elementary school: A randomized control trial. Journal of the American Medical Association, 277, 1605-1611. Harlacher, J.E., & Merrell, K.W. (2010). Social and emotional learning as a universal level of student support: Evaluating the follow-up effect of Strong Kids on social and emotional outcomes. Journal of Applied School Psychology, 26, 212-229. Jimerson, S.R., Sharkey, J.D., Nyborg, V., Furlong, M.J. (2004). Strength-based assessment and school psychology: A summary and synthesis. The California School Psychologist, 9, 9-19. Kobau, R., Seligman, M.E.P., Peterson, C., Diener, E., Zack, M.M., Chapman, D., & Thompson, W. (2011). Mental health promotion in public health: Perspectives and strategies from positive psychology. American Journal of Public Health, 101, 1-9. Kramer, T.J., Caldarella, P., Christensen, L., & Shatzer, R.H. (2010). Social and emotional learning in the kindergarten classroom: Evaluation of the Strong Start curriculum. Early Childhood Education Journal, 37, 303-309. Kusché, C.A., & Greenberg, M.T. (1994). The PATHS curriculum. South Deerfield, MA: Channing-Bete Co. Lane, K.L., Little, M.A., Casey, A.M., Lambert, W., Wehby, J., Weisenbach, J.L., & Phillips, A. (2009). A comparison of systematic screening tools for emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders, 17, 93-105. Massey, O.T., Armstrong, K., Boroughs, M., Henson, K., & McCash, L. (2005). Mental health services in schools: A qualitative analysis of challenges to implementation, operation and sustainability. Psychology in the Schools, 42, 361-372. Masten, A.S., & Coatsworth, J.D. (1998). The development of competence in favorable and unfavorable environments: Lessons from research on successful children. American Psychologist, 52, 205-220. McCormick, M.P., O’Connor, E.E., Cappella, E., & McClowry, S.G. (2015). Getting a good start in school: Effects of INSIGHTS on children with high maintenance temperaments. Early Childhood Research Quarterly, 30, 128-139. McIntosh, K., Campbell, A.L., Carter, D.R., & Dickey, C.R. (2009). Differential effects of a tier two behavior intervention based on function of problem behavior. Journal of Positive Behavior Interventions, 11, 82-93. 90 McIntosh, K., Chard, D.J., Boland, J.B., & Horner, R.H. (2006). Demonstration of combined efforts in school-wide academic and behavioral systems and incidence of reading and behavior challenges in early elementary grades. Journal of Positive Behavior Interventions and Supports, 8, 146-154. McIntyre, L.L., Gresham, F.M., DiGennaro, F.D., & Reed, D.D. (2007). Treatment integrity of school-based interventions with children in the Journal of Applied Behavior Analysis 1991-2005. Journal of Applied Behavior Analysis, 40, 659- 672. McKown, C., Allen, A.M., Russo-Ponsaran, N.M., & Johnson, J.K. (2013). Direct assessment of children’s social-emotional comprehension. Psychological Assessment, 25, 1154-1166. McMahon, S.D., Parnes, A.L. Keys, C.B., & Viola, J.J. (2008). School belonging among low-income urban youth with disabilities: Testing a theoretical model. Psychology in the Schools, 45, 387-401. Merrell, K.W. (2002). School social behavior scales (2nd ed.). Baltimore: Paul H Brookes Publishing. Merrell, K.W. (2008). Helping students overcome depression and anxiety: A practical guide (2nd ed.). New York: The Guilford Press. Merrell, K.W. (2010). Linking prevention science and social emotional learning: The Oregon Resiliency Project. Psychology in the Schools, 47, 55-70. Merrell, K.W. (2011). Social and emotional assets and resilience scales (SEARS). Lutz, FL: Psychological Assessment Resources. Merrell, K.W., Carrizales, D., Feuerborn, L., Gueldner, B.A., & Tran, O.K. (2007). Strong Kids Grades 3 – 5: A social and emotional learning curriculum. Baltimore, MD: Paul Brookes Publishing Co. Merrell, K.W., Parisi, D.M., & Whitcomb, S.G. (2007). Strong Start: A social and emotional learning curriculum for students in grades K-2. Baltimore: Paul H. Brookes Publishing. Merrell, K.W., & Gueldner, B.A. (2010). Social and emotional learning in the classroom: Promoting mental health and academic success. New York: The Guilford Press. Merrell, K.W., Juskelis, M.P., Tran, O.K., Buchanan, R. (2008). Social and emotional learning in the classroom: Evaluation of Strong Kids and Strong Teens on students’ social-emotional knowledge and symptoms. Journal of Applied School Psychology, 24, 209-224. 91 Mettrick, J., Lever, N., Burke, R., Mills, C., & Ghunney, A. (2008). Persistently dangerous communities and schools in the United States: How might school mental health help? Advances in School Mental Health Promotion, 1, 53-63. Mills, C., Stephan, S.H., Moore, E., Weist, M.D., Daly, B.P. (2006). The President’s New Freedom Commission: Capitalizing on opportunities to advance school-based mental health services. Clinical Child and Family Psychology Review, 9, 149- 161. Montague, M., & Rinaldi, C. (2001). Classroom dynamics and children at risk: A followup. Learning Disability Quarterly, 24, 75-83. Nastasi, B.K., Varjas, K. (2008) Best practices in developing exemplary mental health programs in schools. Best practices in school psychology (5th Edition). Bethesda, MD: National Association of School Psychologists. Nese, R.N.T., Doerner, E., Romer, N., Kaye, N.C., Merrell, K.W., & Tom, K.M. (2012). Social emotional assets and resilience scales: Development of a strength-based short- form behavior rating scale system. Journal for Educational Research Online, 4, 124- 139. Oregon Department of Education (2014). School and district report cards. Retrieved from http://www.ode.state.or.us/data/reportcard/reports.aspx Peters, R.D. (1988). Mental health promotion in children and adolescents: An emerging role for psychology. Canadian Journal of Behavioral Science, 20, 389-401. Power, T.J. (2003). Promoting children’s mental health: Reform through interdisciplinary and community partnerships. School Psychology Review, 32, 3-16. Reinke, W.M., Herman, K.C., & Stormont, M. (2013). Classroom-level positive behavior supports in schools implementing SW-PBIS: Identifying areas for enhancement. Journal of Positive Behavior Interventions, 15, 39-50. Romer, N., & Merrell, K.W. (2013). Temporal stability of strength-based assessments: Test-retest reliability of student and teacher reports. Assessment for Effective Intervention, 38, 185-191. Ross, S.W., & Horner, R.H. (2009). Bully prevention in positive behavior support. Journal of Applied Behavior Analysis, 42, 747-759. Safran, S.P., & Safran, J.S. (1985). Classroom context and teachers’ perceptions of problem behaviors. Journal of Educational Psychology, 77, 20-28. 92 Schonfeld, D. J., Adams, R. E., Fredstrom, B. K., Weissberg, R. P., Gilman, R., Voyce, C., Tomlin, R., & Speese-Linehan, D. (2014).Cluster-randomized trial demonstrating impact on academic achievement of elementary social-emotional learning. School Psychology Quarterly, Online First Publication, December 8, 2014.http://dx.doi.org/10.1037/spq0000099 Schultz, D., & Izard, C.E. (1998). Assessment of children’s emotions scales (ACES). University of Delaware, Newark. Seligman, M.E.P., & Csikszentmihalyi, M. (2000). Positive psychology: An introduction. American Psychologist, 55, 5-14. Severson, H.H., Walker, H.M., Hope-Doolittle, J., Kratochwill, T.R., & Gresham, F.M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193-223. Shytenberg, G., Hirsh, J.B., Apfelbaum, E.P., Larsen, J.T., Galinsky, A.D., & Roese, N.J. (2014). Feeling more together: Group attention intensifies emotion. Emotion, 6, 1102-1114. Sicotte, J.L. (2012). Effects of Strong Start curriculum on internalizing, externalizing behaviors, and emotion knowledge among kindergarten and first grade students. (Unpublished doctoral dissertation). University of Massachusetts, Amherst. Simonsen, B., Myers, D., & Briere, D. (2011). Comparing a behavioral Check-In/Check- Out (CICO) intervention to standard practice in an urban middle school setting using an experimental group design. Journal of Positive Behavior Interventions, 13(1), 31-48. Stoiber, K.C. (2011). Translating knowledge of social-emotional learning and evidence- based practice into responsive school innovations. Journal of Educational and Psychological Consultation, 21, 46-55. Suldo, S.M., Gormley, M.J., DuPaul, G.J., & Anderson-Butcher, A. (2014). The impact of school mental health on student and school-level academic outcomes: Current status of the research and future directions. School Mental Health, 6, 84-98. Suldo, S.M., & Shaffer, E.J. (2008). Looking beyond psychopathology: The dual-factor model of mental health in youth. School Psychology Review, 35, 167-180. Sulkowski, M.L., Joyce, D.K., & Storch, E.A. (2012). Treating childhood anxiety in schools: Service delivery in a response to intervention paradigm. Journal of Child and Family Studies, 21, 938-947. 93 Wakefield, J.C. (1997). When is development disordered? Developmental psychopathology and the harmful dysfunction analysis of mental disorder. Development and Psychopathology, 9, 269-290. Walker, H.M., Kavanagh, K., Stiller, B., Golly, A., Severson, H., & Feil, E.G. (1998). First step to success: An early intervention approach for preventing school antisocial behavior. Journal of Emotional and Behavioral Disorders, 6, 66-80. Walker, H.M. & Severson, H.H. (1990). The Systematic Screening for Behavior Disorders: A multiple gating procedure. Longmont, CO: Sopris West. Whitcomb, S.A. (2009). Strong Start: Impact of direct teaching of a social emotional learning curriculum and infusion of skills on emotion knowledge of first grade students. (Unpublished doctoral dissertation.) University of Oregon, Eugene. Whitcomb, S.A., & Merrell, K.W. (2012). Understanding implementation and effectiveness of Strong Start K – 2 on social-emotional behavior. Early Childhood Education Journal, 40, 63-71. World Health Organization (2004). Prevention of mental disorders: Effective interventions and policy options: A summary report. Geneva, Switzerland: World Health Organization Department of Mental Health and Substance Abuse.