STIMULUS FADING WITHIN CHECK-IN/CHECK-OUT by AMY LYJ\ll\lE KAUFFMAN A DISSERTATION Presented to the Department of Special Education and Clinical Sciences and the Graduate School of the University of Oregon in partial fulfillment ofthe requirements for the degree of Doctor ofPhilosophy September 2008 11 University of Oregon Graduate School Confirmation of Approval and Acceptance of Dissertation prepared by: Amy Kauffman Title: "Stimulus Fading within Check-in/Check-out" This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Philosophy degree in the Department of Special Education and Clinical Sciences by: Cynthia Anderson, Chairperson, Special Education and Clinical Sciences Richard Albin, Member, Special Education and Clinical Sciences Robert Horner, Member, Special Education and Clinical Sciences Bruce Stiller, Member, Not from U of 0 Thomas Dishion, Outside Member, Psychology and Richard Linton, Vice President for Research and Graduate StudieslDean of the Graduate School for the University of Oregon. September 6, 2008 Original approval signatures are on file with the Graduate School and the University of Oregon Libraries. © 2008 Amy Lynne Kauffman 111 An Abstract of the Dissertation of Amy Lynne Kauffman for the degree of in the Department of Special Education and Clinical Sciences to be taken Title: STIMULUS FADING WITHIN CHECK-IN/CHECK-OUT Approved: Dr. Cynthia Anderson IV Doctor of Philosophy September 2008 To address the growing academic and social needs of students, schools must utilize efficient and effective methods of behavior support. Targeted interventions support students who are at-risk for developing more severe problem behaviors. These interventions are implemented similarly across a group of students, and thus serve as an efficient and cost effective method of behavior support. Check in Check out (CICO) is a targeted group intervention that has a growing research base supporting its efficacy (Crone & Horner, 2003; Todd, Campbell, Meyer, & Horner, 2008). Although there is a growing research base, no studies have examined maintenance of reductions in problem behavior upon fading. The present study examines (l) if a functional relation exists between CICO and reductions in problem behavior, and (2) which components of CICO can be successfully vfaded with reductions in problem behavior maintaining. In addition, this study examines if teacher attention varies across phases of the study, and predicts successful fading procedures. A reversal design was used to evaluate the efficacy of CICO and subsequent fading phases in four elementary school aged boys. Results indicated that CICO was functionally related to reductions in classroom problem behavior in all four participants. In addition, a functional relation was established with CICO and increases in academic achievement in three participants. Fading procedures were mildly successful, with only some intervention components being removed with reductions in problem behavior maintaining. The amount of adult attention did not vary across study phases, however adult attention became less dependent on problem behavior during the CICO and first fading phase. Clinical and conceptual implications, as well as future research will be discussed. CURRICULUM VITAE NAME OF AUTHOR: Amy Lynne Kauffman GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon: Eugene Oregon Grand Valley State University: Allendale Michigan DEGREES AWARDED: Doctor of Philosophy in School Psychology, 2008, University of Oregon Bachelor of Science with an emphasis in Psychology, 2003, Grand Valley State University AREAS OF SPECIAL INTEREST: Applied Behavior Analysis Targeted Interventions PROFESSIONAL EXPERIENCE: State Trainer, Michigan's Integrated Behavior and Learning Support Initiative, 2007-Present Research Assistant, Educational and Community Support, University ofOregon 2005 - 2007 Graduate Teaching Assistant, School Psychology, University of Oregon, 2004- 2005 Program Assistant, Statewide Autism Resources and Training Network, Allendale Michigan, 2002-2003 Direct Care Worker, Pine Rest Community Mental Health, Grand Rapids, Michigan, 2000-2002 VI I • Vll GRANTS, AWARDS AND HONORS: Effective Educational Practices Research Award, University of Oregon, 2007 College of Education Dissertation Award, University of Oregon, 2007 Outstanding Student in Psychology, Grand Valley State University, 2003 Phi Kappa Phi, National Honors Society, 2001-2003 Psi-Chi, National Honors Society in Psychology, 2001-2003 Laura Wile Miller Education Scholarship, Berrien Community Foundation, 1998 PUBLICAnONS: Mcintosh, K., Campbell, A., & Russell, D. (in press). Differential effects of a tier two behavioral intervention based on function of problem behavior. Journal of Positive Behavior Interventions. Todd, A., Campbell, A., Meyer, G., & Homer, R. (2008). The effects of a targeted group intervention on problem behavior: Elementary school implementation of Check in Check out. Journal ofPositive Behavior Interventions, 10, 46-55. Henderson-King, D., Henderson-King, E., Bolea, B. Koches, K. & Kauffman, A. (2004). Seeking understanding or sending bombs: Beliefs as predictors of responses to terrorism. Peace Psychology: Journal ofPeace and Coriflict, 10 (1), 67-84. Vlll TABLE OF CONTENTS Chapter Page 1. INTRODUCTION AND LITERATURE REVIEW............................................. 1 Continuum ofBehavior Support 4 School-Wide Positive Behavior Support 5 Universal Systems.......................................................................................... 6 Intensive Individualized Interventions........................................................... 7 Secondary Intervention Systems 8 Check-in/Check-out 9 Check-iniCheck-out Research 14 Maintenance................................................................................................... 18 Statement of the Problem and Research Questions 19 II. METIIODS............. 21 Setting and Participants , 21 Measurement.................................................................................................. 26 Design and Procedures.. 32 Data Analysis...... 37 III. RESULTS 39 Problem Behavior and Academic Engagement 39 Office Discipline Referrals . 48 Teacher Ratings 50 Check-iniCheck-out Points 50 Relation of Adult Attention and the Efficacy of CICO and Fading 53 Fidelity of Implementation 57 Contextual Fit................................................................................................. 58 Chapter IX Page IV. DISCUSSION 59 Summary of Findings............................................ 59 Study Limitations 65 Implications for Research and Practice 69 APPENDICES 77 A. CHECK-IN/CHECK-OUT SELF ASSESSMENT 77 B. CONSENT FORMS 79 C. FUNCTIONAL ASSESSMENT CHECKLIST FOR TEACHERS AND STAFF (FACTS) 85 D. CHECK-IN/CHECK-OUT POINT RECORDS..................................................... 89 E. CHECK-IN/CHECK-OUT FIDELITY MEASURE 92 F. TEACHER PERCEPTION QUESTIONNAIRE.................................................... 94 G. SELF-ASSESSMENT OF CONTEXTUAL FIT................................................... 96 H.CHECK-IN/CHECK-OUT AGREEMENT 101 1. DAILY POINT CARD 103 J. CHECK-IN/CHECK-OUT HOME REPORT.......................................................... 105 REFERENCES 107 LIST OF FIGURES Figure Page 1. Check-in/Check-out Cycle 11 2. Kyle's Results for Problem Behavior and Academic Engagement 40 3. Mike's Results for Problem Behavior and Academic Engagement 42 4. Nick's Results for Problem Behavior and Academic Engagement 45 5. Paul's Results for Problem Behavior and Academic Engagement 47 6. Average Office Discipline Referrals 49 7. Average Rating of Student Problem Behavior and Teacher Effort 51 8. Check-in/Check-out Points 52 9. Intervals Scored with Adult Attention 54 10. Behavior-Based Conditional Probabilities 55 11. Event-Based Conditional Probabilities 56 x Xl LIST OF TABLES Table Page 1. Response Definitions 28 2. Average (range) Interobserver Agreement 30 1CHAPTER I INTRODUCTION AND LITERATURE REVIEW Educators face a growing challenge meeting the needs of students. Students come into schools with diverse backgrounds (cultural, socioeconomic status, parent education level) and have a large variety of instructional and behavioral needs. Addressing the behavioral needs of all children is particularly important for educators due to the No Child Left Behind Act of 2001 (NCLB, 2001). Schools that are labeled "persistently dangerous" under NCLB are at risk for losing funding, staff, and students. More than ever, schools are in need of effective methods of providing support to students who exhibit behavior problems. Schools often respond to students exhibiting problem behavior by implementing interventions designed for a specific student. These interventions are typically developed by individual teachers or teams of teachers and support staff at schools. There are a large variety of strategies that are implemented, ranging from token economies to individual counseling sessions. Research on the efficacy of individual strategies varies greatly based on the type of intervention however, generally speaking, strategies that emphasize consistent changes in situations that evoke problem behavior (antecedents) and how problem behavior is responded to (consequences) tend to be effective (Sugai, Homer, & Sprague, 1999), whereas counseling alone has not been shown to have an effect on problem behavior (Dryfoos, 1990). 2One common intervention strategy with empirical support is the behavior report card. The behavior card strategy typically involves (1) monitoring the student's daily behavior on a card, (2) providing a reward for desired behavior, and (3) sending the card home to the student's caretakers. Research on the use of behavior cards (also called daily behavior report cards) has been present for decades. Studies have demonstrated that behavior report cards positively impact problem behaviors, academic engagement, assignment completion, and assignment accuracy (Chafouleas, McDougal, Riley-Tillman, Panahon, & Hilt, 2005; Chafouleas, Riley-Tillman, & McDougal, 2002; Davies & McLaughlin, 1989; Dougherty & Dougherty, 1977; Fairchild, 1983; Drew, Evans, Bostow, Geiger, & Drash, 1982; Schumaker, Hovell, & Sherman, 1977). For example, Schumaker et al (1977) examined effects of the behavior report card on school conduct, daily assignment grades, and teacher satisfaction with three middle school aged boys using a concurrent multiple baseline across participants design. The behavior report card was a classroom based system that relied on parents providing home privileges contingent on improved school behavior. Results demonstrated marked improvement in problem behavior, assignment grades, and teacher satisfaction. Similarly, Dougherty & Dougherty (1977) evaluated effects of a classroom based daily report card on work completion and talk outs with two participants using a multiple baseline across behaviors design. The teacher provided daily ratings, and parents were encouraged to provide rewards for appropriate school behavior. Results demonstrated reductions in talk outs and increases in work completion following implementation of the 3behavior report card. Results from these studies indicate that a behavior report card intervention using home-based rewards is effective in improving school performance, Limitations ofTypical Behavior Report Cards. Although these results are promising, there are several limitations to the typical implementation of the behavior report card. Limitations fall into two broad classes, limitations resulting from home-based contingencies, and limitations due to the individualized nature of the program. First, in most cases rewards are provided by parents. Ifparental participation is not possible, systems relying on home-based contingencies may not be effective in improving school performance. Similarly, many children who are in need ofa targeted intervention live in chaotic home environments (e.g., substance abuse, mental health problems, extreme poverty), and the rewards may not be consistently delivered as planned (Walker, Irvin, & Sprague, 1997). Further, the latency from the occurrence of the desired behavior to the delivery of a reward may be too great for some children. Another problem with exclusively home-based rewards is that, because rewards are not delivered at school, there is little opportunity for positive outcomes to be paired with school; such pairing might result in a student viewing school more positively. Early research on behavior report cards also may be limited as, in these studies, the behavior report card was either a classroom or individualized intervention. Although this may be effective for the specific student targeted, it typically requires a large amount of resources as a program must be developed and implemented each time a student is in need, thus requiring a large amount ofresources. Further, creating an individual behavior card program for all of the students who need additional support may result in several 4different interventions in the same school, or even classroom. This may cause confusion among teachers and staff, which could result in poor fidelity of implementation. An alternative to a piecemeal approach to discipline is a comprehensive, systematic discipline program. What follows is a brief overview ofa three-tiered model for implementing comprehensive supports within a school, and an overview of School Wide Positive Behavior Support (SWPBS). Finally, I discuss how the behavior report card program fits into SWPBS. Continuum ofBehavior Support Current efforts call for a proactive continuum of behavior support, in place for all students in a school (Homer, 2003; Mayer & Leone, 1999; Skiba & Peterson, 2000; Scott & Eber, 2003; Walker et aI, 1996). Proactive systems of behavior support focus not only on the remediation of problem behaviors, but also on prevention. Thus, a continuum of behavior support ranges from universal strategies that aim to prevent the occurrence of problem behavior in all students in the school, to highly structured, individualized interventions for students who display severe problem behaviors (Walker et aI, 1996). The level of the intervention should match the severity of the problem behavior. Walker et al (1996) developed a continuum of behavior support based on the public health model delineating primary, secondary, and tertiary prevention (Gordan, 1983). Following this logic, a school implements a proactive, universal system across all students. This primary prevention system will provide enough support for 80-85% ofthe school population. A secondary prevention system is in place to support students who 5are at-risk for developing more serious problem behaviors, and do not respond to the universal system alone. This is typically successful for 10-15% of the school population. Finally, a highly individualized tertiary prevention system is in place for the students who display the most serious problem behaviors, and do not respond to the universal or secondary levels of prevention. This is typically in place for 3-5% of the student population. School-wide Positive Behavior Support One example of such a continuum of behavior support is school-wide positive behavior support (SWPBS). School-wide positive behavior support is a system of support that aims to prevent the occurrence of problem behaviors, as well as provide support to students who display more severe problem behaviors (Homer, Sugai, Todd, & Lewis- Palmer, 2005). This system is based on the application of behavioral science, valued outcomes, empirically-validated procedures, and systems changes to the whole school context (Homer et aI, 2005). School-wide positive behavior support integrates four key features: student outcomes, research validated practices, systems, and data use (Homer et aI, 2005). Student outcomes refer to the focus on defined and measurable standards of success. The focus of SWPBS is to provide behavioral support that promotes students achieving success socially and academically. The second feature, research validatedpractices, refers to the curriculum, classroom management, rewards, instructional procedures, and contingency management procedures implemented in the school. Practices are determined individually by each school, but they must be (1) related to student outcomes, 6(2) efficient and feasible, (3) cost effective, and (4) based on sound educational theory (Homer et aI, 2006). Systems refers the administrative leadership, team structures, staff training, and organization of the school that are needed to effectively implement and sustain SWPBS (Homer et aI, 2005). Data use refers to the ongoing collection, and use of discipline and academic data for decision making (Homer et ai, 2005). Integrating the previously mentioned key features across all levels of support, SWPBS follows the behavior support continuum logic by implementing a universal system that is in place for all students, secondary interventions for students who are at- risk for developing problem behaviors, and intensive, individualized interventions for students who display serious problem behaviors. Universal Systems To create a predictable and positive school climate, a universal system is implemented in all settings, and across all staff in the school. Key features of the SWPBS universal system include: (1) defining 3-5 positively stated school-wide expectations, (2) directly teaching the expectations in all settings, (3) a reward system that encourages desired behaviors, (4) a clear, fair, and consistent system for discouraging problem behaviors, and (5) collecting and using data for on-going decision making (Homer et aI, 2005). In addition to the specific practices in the universal system, several systems level features are present to ensure the efficacy and sustainability of SWPBS. These features include (1) team-based implementation, (2) administrative support, (3) a documented commitment from administration and educators, (4) adequate personnel and time allocation, (5) a budget, and (6) an infonnation systems to gather and summarize discipline infonnation (Homer et aI, 2005). Several studies have documented that this universal approach reduces office discipline referrals, suspensions, and expulsions (Colvin, Kameenui, & Sugai, 1993; Nelson, Martella, & Marchand-Martell; 2002; Nelson, 1996; Lewis, Sugai, & Colvin, 1999; Taylor-Greene et aI, 1997). Further, studies have found that schools that are implementing this universal strategy have increases in attendance and academic gains, when research-validated instruction is provided (Colvin et aI, 1993; Kellam, Mayer, Rebok, & Hawkins, 1998; McIntosh, Chard, Boland, & Homer, 2006; Taylor-Greene et aI, 1997). Although these systems are effective in preventing many problem behaviors, some children require more behavioral support. Intensive Individualized Interventions A large body of research has focused on providing individual students behavior support plans for the 3-5% of students with the most significant needs (March & Homer, 2002; Scott & Eber, 2003). These students may be provided special education services under the label of emotional disturbance (Lewis & Sugai, 1999). These interventions are typically designed based on functional assessment data, comprehensive in nature, and require adequate personnel time and resources. Studies have found that function-based individualized student interventions are highly successful in reducing the problem behaviors of children with the most intense behavioral needs (Carr, 1977; Scott & Eber, 2003). However, many ofthese interventions take a significant amount oftime in planning, monitoring, and implementing. As a result, schools implementing SWPBS 7 8have secondary interventions to meet the needs of students who are not supported by universal systems but may not require intensive, individualized interventions. Secondary Intervention Systems Secondary or targeted group interventions are intended for children with whom the universal system is inadequate, but who do not require an intensive individualized intervention. Children who may benefit from targeted group interventions typically have received multiple office referrals and are at-risk for developing more severe problem behaviors due to poor peer relationships, poor relationships with adults, and/or academic failure (Lewis & Sugai, 1999; Walker et aI, 1996). Typically, these children do not need an intensive, individualized intervention. Instead, they may benefit from a "group" intervention such as social skills training, anger management programs, behavior report cards or Check-in/Check-out (CICO) systems, and homework clubs (Hawken & Homer, 2002). For example, social skills training programs may provide explicit instruction to students on positive interactions with peers. Check-in/Check-out systems may increase adult attention and monitoring of students. Academic targeted group interventions may provide more practice in certain skill areas (e.g., reading fluency, math facts). The term "group" does not indicate that the intervention is always delivered in a group format; rather it is that the procedures and practices are implemented similarly across a group of students. This eliminates the need for schools to recreate systems each time a student presents with behavioral challenges. Instead, a system for responding is already in place; the student can simply be referred into that system. As a result, the student may begin to receive intervention earlier (time is 9not needed to develop an intervention, it already exists), the intervention may be implemented with greater fidelity as educators are familiar with it, and data are more likely to be collected and used to guide decision-making as ongoing data collection is part of the school culture in SWPBS. Although targeted group interventions may differ in the method of intervention and the skills taught, they are all (l) efficient in delivery, and (2) cost effective (Hawken & Homer, 2003). To be efficient in delivery, these interventions should take no more than 10-min per day for a teacher to implement, students should be able to access the intervention shortly after being identified for needing support, and require little assessment prior to implementation. In addition, the intervention must be cost-effective, so that it can support multiple students with few resources. Although several studies have evaluated effects of targeted interventions within SWPBS (e.g., Fairbanks, Sugai, Guardino, & Lathrop, 2007; Hawken & Homer, 2003; March & Homer, 2002; Todd, Campbell, Meyer, & Homer, 2008), this type of intervention has received the least amount of attention in the literature when compared with universal and individual interventions. One targeted intervention with growing empirical support when implemented within a comprehensive system ofbehavior support is the Check-infCheck-out program. Check- in/ Check-out Check-infCheck-out is a common targeted intervention that goes by several different names (e.g., Behavior Education Program, Hello Update Goodbye program). Importantly, these programs share critical features and are based on the logic of the 10 previously discussed behavior report cards. This logic was expanded to create an intervention system that schools can efficiently and effectively implement across multiple students. In contrast to the behavior report card interventions described earlier (e.g., Dougherty & Dougherty, 1977; Schumaker et aI, 1979), the CICO program is implemented within a school-wide system of positive behavior support. Further, CICO is a systematic intervention that is implemented similarly across students in the school. The systematic implementation allows schools to provide support shortly after the student is identified, ensures clear and consistent procedures for implementation across all settings and staff in the school, and requires minimal time and effort from teachers. Components of the CICO program include: (l) student identification, (2) the daily cycle, (3) continuous monitoring and evaluation, and (4) system features. Figure 1 displays the basic CICO cycle. Student Identification. Students are identified for the CICO program through a variety ofmethods. Office discipline referrals (ODRs) are a common method for identifying students who may benefit from the program. Office discipline referrals can identify the types of problem behaviors the student engages in, the setting, the time, and also the frequency of the problem. Students who show a consistent pattern of mild problem behaviors that occur in several settings and with different staff are typically good candidates for CICO (Crone et aI, 2004). Students who engage in high frequency, or high intensity, dangerous problem behaviors may need a more intensive individualized plan. Students may also be identified by teacher nomination. Regardless of the method 11 Student Recommended for C CO CIC is Implemente Check In Check Out (CICO) Morning Check-in CICO Coordinator Summarizes Data ------.... For Decision Making Bi-weekly SST Meeting to Assess Student Progress Figure 1. Check-ill/Check-ollt Cycle Regular Teacher Feedback Revise Program Exit Program 12 of identification, there is a clear process for requesting additional support and accessing the intervention (Crone et aI, 2004). Daily Cycle. Once identified for the program, the student is provided with a brief overview of CICO, including the behavior expectations to be met. The behavior expectations are typically tied to the individual school's behavior expectations. For example, the school expectations may be "Be Safe, Be Responsible, and Be Respectful." The student's card will list the three behavior expectations, and daily points are earned for following the expectations. For example, during morning check in, a student would earn points under "responsible" for coming to check in and bringing back the parent report. Points are earned under "respectful" for waiting quietly until it is the students tum to check in. Finally, points are earned under "safe" for walking in the room. The behavior expectations are operationalized for the student in all settings (CICO room, classroom, playground, etc.). When the program commences, students will "check in" with a school staff member each morning. During this morning check in, the staff member reviews the behavior expectations, provides a verbal prompt for appropriate behavior (e.g., "remember to be responsible during class and work quietly"), gives the student the daily behavior card, and collects the previous day's home report (described next). The students carry the card with them all day. Teachers provide verbal feedback and record points (described next) on the student's behavior card at several pre-identified times during the day. Points are provided to the student for each of the identified behavior expectations. For example, 13 students may receive three points for doing a "great job", two points for "okay", and one point for "hard time" with a specific expectation. The teacher rates the student for all of the behavior expectations, thus the student has the opportunity to earn a total of nine points for that particular time. At the end of the day, the student "checks out" with a school staff member. At this time, the staff member collects the card, records the number of points the student earned, provides verbal feedback regarding the student's behavior, and completes the home report. The home report typically indicates whether or not the student met his or her goal for percent of points earned, and requires a parent signature. Parents are encouraged to provide rewards if their child meets his or her goal, and to refrain from implementing negative consequences for not making his or her goal. Thus, in contrast to the previously discussed behavior report card interventions, parents are not solely responsible for feedback. Each day, the number of points the student earns is recorded. The points may be used to "purchase" certain items or privileges at school. For example, students may get pencils, stickers, or other small tangible items. Students may also purchase special activities such as extra recess or lunch with a teacher. Continuous Monitoring and Evaluation. Student progress data are collected and summarized on a regular basis. Typically, these data include a graph of the percentage of points earned each day, and office discipline referrals. These data are used to determine whether the CleO program is effective for the particular student. A separate school-based team may be developed for this purpose, or an existing team may take on the 14 responsibility (Crone et aI, 2004). Data also are used to determine if the program needs to be modified for the student and could be used to determine ifthe intervention can be removed (i.e., it is working well and is no longer needed). Systems Variables. The CICO program is not an individual or classroom intervention. The intervention is built into the school-wide discipline system. One key feature is that all staff members are aware of how to access and implement the intervention (Crone et aI, 2004). This encourages consistent implementation across different staff members and settings in the school. Because the intervention program is embedded within the school-wide system, it is consistently available and students thus are able to begin the intervention almost immediately after being identified for needing more support (Crone et aI, 2004). Check-iniCheck-out Research Expanding on the behavior report card literature, seven recent studies have utilized the behavior report card logic in a school-wide system of support. Two descriptive studies on effects ofCICO at the level of the entire school (not individual student outcomes) examined (1) impact ofCICO on office discipline referrals, and (2) whether schools were able to implement the program with fidelity (Hawken, 2006; Filter et aI., 2007). Results ofboth studies indicated that upon implementation of CICO, office discipline referrals decreased. In addition, Filter et al (2007) found that schools were able to implement the program with fidelity. A study by Hawken and Homer (2003) examined effects of the Behavior Education Program, a CICO program, on problem behaviors and academic engagement 15 in four sixth grades boys who attended a rural public middle school. A concurrent multiple baseline across participants design was used to evaluate intervention outcomes. In contrast to previous studies on behavior report cards, the intervention was implemented within a school-wide system of behavior support, rather than simply as an independent, individualized intervention. In addition, and as described above, this study focused on increasing opportunities for positive attention and reinforcement in the school setting, rather than relying on caregivers to provide rewards. Results indicated a reduction in problem behavior, and an increase in academic engagement for all four participants. Todd, Campbell, Meyer, & Homer (2008) replicated the previous study, except the intervention was applied with four elementary school aged boys. A concurrent multiple baseline across participants design was used to evaluate intervention effects. Results indicated reductions in problem behaviors across all four participants, ranging from 15% to 19%. In addition to evaluating the efficacy of CICO, researchers have also examined factors that predict individual's success on the Behavior Education Program (a Check- in/Check-out program). March & Homer (2002) conducted a descriptive analysis that examined effects of the BEP on office discipline referrals for 24 middle school students. In addition, the perceived maintaining function ofeach participant's behavior was assessed using teacher interviews. Results indicated that 80% ofthe students whose behavior was perceived to be maintained by adult attention displayed reductions in office discipline referrals. In contrast, only 27 % and 62.5 % of students whose behavior was 16 perceived to be maintained by escaping academic demands or peer attention displayed reductions in office discipline referrals upon implementation of the BEP program. Following the descriptive study, March & Homer (2002) examined adapting the CICO program for three middle school students who did not respond to the standard intervention. A concurrent multiple baseline across participants was used to examine effects of the adaptation. The adaptation was based on a descriptive functional assessment which revealed that the behavior of one participant was maintained by escape from tasks, the behavior ofone participant was maintained by peer attention, and the behavior of the third participant was maintained by escaping from difficult tasks and peer attention. This information was used to alter the intervention to match the function of problem behavior. For example, the adapted plan for a participant whose behavior was perceived to be maintained by peer attention included: (1) a seating change away from peers, and (2) BEP points earned may be used for basketball time with friends. A range of 11 % to 26% reductions in problem behavior were observed upon implementation of the adaptation. Fairbanks, Sugai, Guardino, & Lathrop (2007) conduct a two part study examining the CICO program across 10 elementary students. Part one of the study utilized case study to examine the impact of CICO on student problem behavior. Results indicated reductions in problem behavior in four participants. Part two of the study implemented adaptations to the CICO program in four participants who did not respond to the standard program. A descriptive functional assessment was conducted, and the problem behavior of two students was hypothesized to be maintained by peer and adult 17 attention, one was hypothesized to be maintained by peer attention and escaping academic tasks and one was hypothesized to be maintained by adult attention. Adaptations to the program were based on the assessment information. For example, for students whose behavior was hypothesized to be maintained by peer attention, recess and reward time privileges were earned through CICO points. A concurrent multiple baseline across participants design was used to evaluate intervention effects. Results indicated an average of 16% reductions in problem behavior upon implementation of the adaptations. Similarly, Campbell, Anderson, and Todd (under review) worked with two participants who did not respond to CICO. A functional assessment suggested that the problem behavior of both participants was maintained by peer attention. When CICO was modified to address this function-the participants could gain access to peer attention when they met their point criteria each day-reductions in problem behavior were demonstrated. To summarize existing research, CICO interventions seem effective as part of a comprehensive system of behavior support. Further, existing studies on nonresponders suggest that CICO may be more effective-as a standard intervention-for students whose problem behavior is maintained by adult attention. Although research has suggested that CICO systems are effective in initially reducing problem behavior in students, little it know about the durability of the behavior change. Effective interventions must not only produce initial effects; they must also produce changes that are durable and generalizable across multiple situations (Stokes & Baer, 1977). Further, for targeted interventions to be efficient over time, they should be able to be gradually 18 faded while reductions in problem behavior maintain. To this date, no studies on CICO have examined maintenance of effects while the intervention is fully in place, let alone after components have been faded. Maintenance To date, maintenance of intervention effects has been sorely neglected in the behavioral literature (Carr et al., 1999). The majority of research that does address maintenance typically includes a follow-up phase 6-8 weeks after the intervention is in place (Carr et aI., 1999). Although it is beneficial to demonstrate that results maintain over time---and we need more such demonstrations-research is needed as well to evaluate whether intervention components might successfully be removed at some point. The rationale for this is two-fold. First, implementing an intervention requires an allocation of resources that might be put to other use if the intervention is no longer needed. Second, students will move on to new settings (e.g., different schools) where CICO may not be available; thus the intervention should be faded in a systematic way whenever possible. Unfortunately, documenting effects of intervention fading is somewhat difficult. In acquisition studies, researchers attempt to demonstrate that the participant behavior will change systematically upon implementation, and removal of an intervention. For example, ABAB reversal designs are commonly used in acquisition studies. In contrast, researchers examining maintenance aim to demonstrate that behavior changes upon initial implementation of the intervention, but does not changes after a withdrawing the intervention. It is difficult to demonstrate functional control over a dependent variable 19 that does not change in level when the intervention-independent variable-is manipulated. Researchers examining fading often fade the intervention using traditional single subject designs but functional control in not gained over maintenance (Rusch & Kazdin, 1981). Statement ofthe Problem and Research Questions Check-in/Check-out is an emerging targeted intervention for providing efficient and effective support to students who are at-risk for developing more serious problem behaviors. Although a research base supporting this intervention is emerging, there are currently several gaps. The present study strived to expand the literature on CICO program. Demonstration ofEffects ofCheck-iniCheck-out. To date, a total of seven studies have examined effects of CICO on problem behavior within a school-wide system of positive behavior support. However, only two of the studies experimentally evaluated the effects ofthe standard CICO program (Hawken & Horner, 2003; Todd et aI, 2008). More replications are needed to further establish CICO as an effective program. This is particularly important due to the recent call from the federal government for schools to use "evidence based practices". More research is needed on CICO to establish it as an evidence-based practice. The present study experimentally evaluated effects of CICO across four elementary school students. Evaluation ofFading Check-iniCheck-out. Currently, no studies have examined the maintenance of effects of CICO, let alone evaluating fading procedures. In order for CICO to be truly efficient and cost effective to implement, the intervention must be able 20 to be faded successfully. In addition, because CICO has several components, it is important to determine which components can be successfully faded. Overall, few studies in the behavioral literature have examined fading individual components separately. The present study examined stimulus fading across four students. Research on Factors that Predict Successful Fading. Children in schools have diverse experiences and backgrounds. Even children who attend the same school and are in the same classroom are exposed to differences in environmental contingencies. Given the heterogeneous nature of schools, one would not expect that the same fading procedure would be equally effective across several different children. Research is needed on environmental variables that may predict the successful fading ofan intervention. To begin to address this research gap, the present study examined the role of adult attention in the efficacy ofCICO and successful fading. To summarize, the present study strived to increase the existing literature on targeted group interventions and maintenance effects by addressing the following research questions: (l) Is there a functional relation between CICO out and reductions in problem behavior? (2) Which components of CICO can be faded successfully (Le., such that problem behavior remains suppressed and the occurrence of pro-social behavior is maintained)? (3) Is adult attention predictive of success on CICO and fading? 21 CHAPTER II METHODS Setting and Participants Setting. The present study took place in a suburban elementary school (K-5) located in the Pacific Northwest during the 2006-2007 school year. The school had 310 students with 56% identifying as Caucasian, 33% identifying as Latino, 7% identifying as American Indian, 2% identifying as African American, and 1% identifying as Asian or Pacific Islander. During the 2006-2007 school year, 72% of students qualified for free or reduced lunch services. On state standardized assessments, 77 % ofthird grade students were proficient in reading, and 70% ofthird grade students were proficient in math. For fifth grade students, 48% were proficient in reading, and 35% were proficient in math. The school had been implementing School-wide Positive Behavior Support for approximately 5 years. This included: establishing and teaching behavior expectations (Be Safe, Be Responsible, Be Respectful), a school-wide token economy for rewarding appropriate behavior, and a continuum ofresponses to problem behaviors (major and minor office discipline referrals). The school had met criteria,(80/80) for four consecutive years on the School-wide Evaluation Tool (Sugai, Lewis-Palmer, Todd, & Homer, 2001), a measure that assesses the fidelity of SWPBS. 22 The school had been implementing the CICO program for one year prior to the study. On the Check-in/Check-out SelfAssessment (Homer, Todd, & Dickey, 2005), the school had 16/17 key features (94%) in place. The item that was rated as "not in place" was that the school did not have a team that managed with CICO program and reviewed the data on a regular basis. Although data was reviewed, it was the sole responsibility of the CICO coordinator and the building principal. A copy of the assessment tool is available in Appendix A. In the previous school year, the school had approximately twelve students on their CICO program. During the year of the study, approximately eight students were on CICO at any given point in time. All direct observations took place in general education classroom settings. The specific activity (e.g., large group reading instruction, math independent work) was determined individually for each participant based on the results of the functional behavior assessment (described below) that was conducted prior to the beginning of the study for each participant. Participants. A total of four participants completed the study. Participant selection occurred in several steps. First, office discipline referrals were examined, and students with three to five referrals were identified as potential participants. An email was sent to the student's classroom teacher to determine ifhe/she would like to have the student participate. In addition, teachers could also nominate a student to participate if that particular student did not have office discipline referrals. Upon identifYing students through office 23 discipline referrals and/or teacher nomination, parental and teacher consent was obtained. Appendix B displays copies of the consent forms. Second, a brief descriptive functional assessment was conduced. A selection criterion for the study was that problem behavior appeared to be sensitive to adult attention and thus, prior to conducting the study proper, a functional behavior assessment consisting ofan interview (Functional Assessment Checklist for Teachers and Staff; March, Homer, Lewis-Palmer, Brown, Crone, Todd & Carr (2000) and confirmatory observations were conducted with all participants. A copy of the Functional Assessment Checklist for Teachers and Staff (FACTS) is available in Appendix C. The purpose of the FBA was to identify students whose behavior was likely to be sensitive to adult attention and so the FBA was not part of this experiment but rather served as a screening for participants. Results of the FBA for each participant are summarized below. Kyle. Kyle was a second grade Caucasian student who qualified for special education services under the category ofleaming disability, specifically in reading. In addition, he was diagnosed with attention-deficit hyperactivity disorder-combined type and was taking methylphenidate. Dosage information was not available. He received 50% ofhis reading instruction in a general education setting, with the remaining time spent in small group instruction in a resource room setting. He qualified for free and reduced lunch services through the school. He was referred to the study due to disruptive behavior. 24 The FACTS interview with Kyle's teacher indicated that he engaged in disruptive behavior during large group reading instruction. More specifically, she reported that he talked to peers, made noises, made faces at peers, and put his head down when large group reading was occurring and students were taking turns reading a story aloud. Further, results of the FACTS interview suggested that disruptive behavior might be maintained by adult and peer attention. COIlfmnatory direct observations indicated that du.ring large group instruction, his disruptive behavior was typically resulted in adult attention. All data collection took place during large group reading instruction when students were expected to read aloud or complete another activity as a large group. His classroom had 26 students and one teacher. During reading instruction, there was a parent volunteer present approximately 3/5 days of the week. Mike. Mike was a fifth grade Caucasian student who received all instruction in a general education setting. Although he did not qualify for special education, he received additional small group reading instruction from and instructional assistant (within his large group reading class). Results of the FACTS interview suggested that problem behavior occurred most often during math instruction, and included disruption and failure to complete independent work. This was most likely to occur when assignments took more than 10- min to complete. (e.g., daily math facts worksheets, solving problems out the textbook, or reviewing previous math concepts). Further, results suggested problem behavior might be 25 maintained by adult attention and/or task avoidance. Confirmatory direct observations suggested that adult attention was the consequence that followed problem behavior most often, as when he was off task or disruptive his teacher typically provided verbal reprimand and redirected him to complete the task. All data collection took place during independent math work, specifically during the 20-min portion ofclass devoted to daily math facts worksheets. There were 27 other students and one teacher in the room during this activity. Mike was in the same class as Nick and Paul. Nick. Nick was a fifth-grade Latino male who qualified for special education services under the category of speech and language impairment. He received all academic instruction in the general education setting, but also received 30-min of speech services each week. In addition, Nick was diagnosed with Attention-Deficit Hyperactivity Disorder-Combined Type by his family's physician and was taking amphetamine and dextroamphetamine, Adderall. Information on the specific dose was not available. Nick was referred to the study by his classroom teacher due to disruptive and off-task classroom behavior. Results of the FACTS interview indicated that Nick's primary problem behaviors were out ofseat, off-task, and talking with peers. These problem behaviors occurred during independent math instruction and were likely maintained by adult attention. Confirmatory direct observations suggested that when Nick engaged in problem behavior he was likely to receive adult attention (e.g., adult reprimand). In addition, he was also 26 able to escape the academic task until his teacher provided attention. All direct observations took place during independent work during math. Paul. Paul was a fifth grade Caucasian male. He qualified for special education under the category of learning disability, specifically in the area ofreading. He received small group reading instruction from the special education teacher, but all other instruction was delivered in the general education setting. He was referred to the study by his teacher because of disruptive and non-compliant behavior. Results of the FACTS interview indicated that he talked to peers, make noises, and verbally refused to complete independent assignments during large group instruction and independent work in math. Typically, during large group instruction he would talk to peers and make noises, and during independent work he would verbally refuse to complete the task. Conftrmatory direct observations indicated that the majority ofhis problem behavior was followed by adult attention. All direct observation for the study took place during large group math instruction. Measurement The primary dependent variable in the present study was student problem behavior. The following dependent measures were used in this study: direct observations of student problem behavior and academic engagement, offtce discipline referrals, points earned during CICO, and teacher ratings. The independent variables, CICO and fading procedures, were measured using a fidelity checklist. Finally, contextual fit were assessed using a survey. 27 Data Collection. Direct observations were conducted 3-5 days per week. Data were collected using pen and paper across 15-min observations using a 5-s partial interval recording system. All direct observations of student behavior took place in a classroom setting during relevant academic activities as reported above. Definitions of problem behavior, academic engagement and teacher attention are provided in Table 1. Although definitions of specific problem behaviors are provided, observers coded problem behavior overall. Interobserver Agreement. Interobserver agreement was assessed for 32% of observation sessions. During these sessions, two observers independently collected data. Total agreement, occurrence agreement, and nonoccurrence agreement were calculated. Total agreement was calculated by dividing the number of intervals that both observers agreed a response did or did not occur by the number of total intervals and multiplying by 100 for that particular observation. Occurrence only agreement was calculated by dividing the total number of intervals both observers agreed a response occurred by the number of intervals either observer scored a response and multiplying by 100. Non- occurrence agreement was calculated by dividing the total number of intervals both observers agreed a response did not occur by the total number of intervals either observer did not score a response and multiplying by 100. For problem behavior, total agreement averaged .94 (range = .89 to 1.0), occurrence only averaged .92 (range = .86 to 1.0), and nonoccurrence only averaged .95 (range = .91 to 1.0). For academic engagement, total agreement averaged .92 (range = .87 Table 1. Response Definitions Response Disruption Inappropriate Placement Noncompliance Negative Verbal or Physical Academic Engagement Adult Attention 28 Definition Making a noise or physical action irrelevant to the task that other individuals can see or hear (talking out, talking to peers, etc.) Student's body in not in expected location during an activity (out of seat, in another desk, etc.) Verbally or non-verbally refusing to follow adult direction within 10-s of request Any form of physical aggression, or negative verbal statement directed to adults or peers (e.g., "this sucks," "you are stupid") Following teacher requests within 10 s, eyes oriented toward teacher or relevant materials for academic task, and completing in-class tasks as requested by the teacher. Adult physically or verbally responds to the student 29 to 1.0), occurrence only averaged .91 (range = .85 to 1.0), and nonoccurrence only averaged .95 (range = .91 to 1.0). For adult attention, total agreement averaged .95 (range = .91 to 1.0), occurrence only averaged .96 (range = .92 to 1.0), and nonoccurrence only averaged .96 (range = .91 to 1.0). Table 2 provides interobserver agreement for each participant. Office Discipline Reftrrals. Major and minor office discipline referrals were defined as instances of problem behavior reported by school staff. Minor referrals were defined as instances ofproblem behavior that are major rule violations, but do not need to be handled by the office staff. For example, disrespect toward an adult, teasing peers, and engaging in unsafe behavior on the playground may result in a minor office discipline referral. Major office discipline referrals were defined as instances ofproblem behavior that are typically handled by administration. Examples included physical aggression, continued disruption of instruction, and non-compliance. Each referral resulted in a formal report and was entered into an electronic database. The average number of office referrals (both major and minor) per day was calculated by dividing the number of referrals by the number of schools days. Check-iniCheck-out Points. The total number of points the student earned on the CICO card was recorded daily by the coordinator. A percentage of points was calculated by dividing the number of points earned by the number of points possible. In addition, the percentage ofdays in which the student met his or her goal was calculated by dividing the numbers of days the Table 2. Average (range) Interobserver Agreement 30 Total Agreement Occurrence Only Non-occurrence Only Kyle Problem Behavior .97 .91 .93 (.91 - 1.0) (.88 - 1.0) (.91 - 1.0) Academic Engagement .94 .92 .97 (.89 - .97) (.85 - .97) (.93 - 1.0) Adult Attention .96 .96 .98 (.94 - 1.0) (.92 - .98) (.94 - 1.0) Alex Problem Behavior .95 .90 .93 (.89 ~ 1.0) (.86 ~ 1.0) (.91 ~ 1.0) Academic Engagement .90 .89 .94 (.89 - .95) (.85 - .91) (.92 - 1.0) Adult Attention .92 .96 .92 (.91 - .94) (.92 - 1.0) (.91 - .94) Alton Problem Behavior .91 .94 .98 (.90 - .95) (.86 - 1.0) (.94- 1.0) Academic Engagement .95 .93 .96 (.90 - 1.0) (.87 - .96) (.94 - 1.0) Adult Attention .97 .96 .97 (.93 ~ 1.0) (.91 - 1.0) (.93 - 1.0) Corey Problem Behavior .93 .91 .94 (.89- 1.0) (90 - 1.0) (.91 - 1.0) Academic Engagement .90 .89 .91 (.89 - .93) (.85 - .93) (.91 - .98) Adult Attention .95 .94 .96 (.92 - 1.0) (.92 - .96) (.91 - 1.0) 31 goal was met by the total number of days. The data were gathered by examining the CleO coordinator's record (see Appendix D for a copy of the record fOnTIs). Fidelity ofImplementation. The primary independent variables were implementation ofClCO, and the fading components ofthe intervention. Fidelity of implementation was measured on 27% of days the students participated in CICO. Fidelity was assessed by directly observing check-in, classroom rating times, and check- out. A fidelity checklist (see Appendix E) was completed based on direct observations. The checklist contained twelve items that assessed the presence ofkey features across program. Interobserver agreement was assessed for 25% of fidelity observations. Agreement was calculated by dividing the total number of items that were scored the same by the total number of total items. Agreement averaged 97%, with a range of 91 % to 100%. Teacher Perception. Teacher perception of student problem behavior and ease of implementation was assessed approximately one to two times per week throughout the study. Teachers completed a 2-item questionnaire (See Appendix F). Teachers provided a rating on the target student's problem behavior compared to other students in the class using a 5-point Likert scale, with a rating of 1 must better than other students, a 2 indicating better than other students, a 3 similar to most other students, a 4 indicating worse than other students, and a 5 indication much worse than other students. Teachers also rated the amount of effort put into managing student behavior compared to other students in the class using a 5-point Likert scale, with a rating of 1 32 indicating little or no effort, a 3 indicating some effort, and a 5 indicating a tremendous amount of effort. Contextual Fit. Contextual fit was assessed during the initial CICO phase, and at the completion of the study. The classroom teacher for each participant completed the Contextual Fit Questionnaire (Homer, Salantine, & Albin, 2003) for each participant. The Contextual Fit Questionnaire assessed school staffmember's perceptions on the ease of implementation, amount of effort needed to implement the intervention, and whether the effects of the intervention were worth the effort (See Appendix G). Design and Procedures Design. A reversal design was used to (1) demonstrate functional control over CICO, and (2) to evaluate fading procedures. First, an ABAB reversal design was used to demonstrate functional control over CICO. The reversal design not only allowed for demonstration of functional control of CICO, it also provided a means for evaluating whether effects of CICO maintained when the intervention is simply removed-or whether it is necessary to fade the intervention systematically. Second, reversal designs were used during fading to determine which features ofCICO could be systematically removed. There were three phases in the study, baseline, CICO, and fading. The fading phase included three steps as the intervention components are removed systematically (description of fading phases listed below). Baseline. During baseline, participants partook in the school's typical school-wide discipline system. This included office discipline referrals (major and minor), loss of 33 privileges, and verbal reprimands for problem behaviors. In addition, they may have received school-wide rewards and verbal praise for appropriate behaviors. Additional methods of behavior support (e.g., individualized interventions) were not in place for any participant. Check-in/Check-out. Check-in/Check-out consisted of the follmving components, each described below: training, check-in, CICO card, classroom routines, check-out, home report, and incentives. Prior to starting the CICO program, each participant attended two 30-min training sessions with the CICO coordinator and the primary investigator. These sessions focused on (1) teaching the child the routines ofthe CICO program, (2) providing examples and non-examples of desired behaviors, and (3) providing practice checking in and checking out in a specific location. The first training session focused on explaining the program to the student, describing the daily routines, describing the CICO card and points, and finally brainstorming possible rewards for the student to work toward. After the first session, the student and the CICO coordinator signed a contract that explicitly described responsibilities for CICO (See Appendix H). The focus of the second training session was to practice the CICO routines. The student practiced going to check-in, soliciting feedback from the classroom teacher, and going to check out. After completion of the above-mentioned training sessions, the students began CICO program within two school days. Each morning, before class started, the students "checked in" with the CICO coordinator at the school. During this session, students received their daily point card (see 34 sample in Appendix 1), turned in the previous day's parent report form (see sample in Appendix J), and engaged in a short, positive interaction with the CICO coordinator. For example, the CICO coordinator greeted the student by saying "1 am glad that you are here today" or "Thank you for coming down to see me this morning". The coordinator then asked for the child's home report, and provided the new card. Typically, a verbal prompt was also provided (e.g., "Have a great day"). This interaction was brief, typically lasting 2 to 5-min for each student. The CICO daily report card was a 4 x 5 inch piece of cardstock paper. On each card, there were 5 opportunities for the student to earn points for appropriate behavior: check-in, mid-morning, lunchtime, mid-afternoon, and check-out. Each teacher determined the exact time for ratings based on natural transitions during the day (e.g., before reading starts, after recess, before lunch, after math). Each participant could earn up to three points for each separate target behavior. Target behaviors were tied to the school's behavior expectations (safe, responsible, respectful). Points were assigned using a three point scale with a 1 indicating the student had a "hard time", and 2 indicating the student did '''okay'', and a 3 indicating that the student did "great". Therefore, students may have earned up to 45 points per day. Each day, the students had a goal ofearning 80% ofthe possible points. After the morning check in, the students took the card and went to their classroom. At the three specified times during the day the student approached the classroom teacher and elicited feedback. The teacher rated the student's behavior using the 3-point scale described above. The teacher typically provided 35 additional verbal feedback and an explanation of the ratings (e.g., "Great job, you earned a three because you completed all of your work", "I rated you a two for responsibility because you were talking during reading time"). In addition, the teacher provided a pre- correction for the next opportunity to earn points. If the student did not approach the teacher to elicit feedback, the teacher approached the student, provided a reminder and point feedback. The student repeated this process at each specified time of the day. At then end ofthe day (typically 5 to 10-min prior to dismissal) the student walked to the specified check out location and gave the card to the coordinator. The coordinator (l) recorded the number of points earned, (2) completed a parent report, and (3) provided feedback to the student regarding his behavior. If the student met his goal of percentage of points the coordinator provided verbal praise. If the student did not meet his goal, the coordinator provided neutral feedback (e.g., "Lets try to meet the goal tomorrow"). The parent home report was an 8 ~ by 11 inch piece ofpaper. This report contained the student's name, the date, whether or not the student met his goal for percentage ofpoints earned, a place for comments, and a parent signature line. The CICO coordinator completed the form, and gave it to the student to take horne. The student was expected to give the report to his parents and obtain a signature. ,The student then returned the previous horne report to the coordinator the following morning. Parents were encouraged to provide incentives at horne, and refrain from punishment if their child did not meet his goal; however, no data were collected to determine whether this occurred. 36 Each day, the students could earn a specified number ofpoints based on the percentage of points earned on the card (e.g., 10 points for 100%, 9 points for 90%,8 points for 80%). Students had the opportunity to spend points one time each week. Students could purchase certain tangible items and social privileges. The point value for items ranged from relatively small (e.g., 10 points for a pencil, 20 points for a notepad) to large (e.g.,100 points for 5-min additional recess, 300 points for lunch with an adult). Fade ClCO 1. During this phase, the number oftimes that participants received feedback and points from his teacher for appropriate behavior was reduced. Participants continued to check in and check out with the CleO coordinator. Feedback was reduced from three to two times per day; once mid day and once at the end of the day. As described above, points toward reinforcers were based on the percentage earned. Therefore, participants were still able to earn the same amount ofpoints each day, despite the reduced opportunities for teacher feedback Fade ClCO 2.Participants continued to check in and check out with the CleO coordinator. The number of times the participants received feedback from his teacher was further reduced from two to one time per day. Thus, they received feedback and points for appropriate behavior during check in, at the end of the day, and check out. Similar to previous phases, the total number ofpoints that may be used toward rewards was based on the percentage of points earned. Fade ClCO 3.During this phase, participants no longer carried the CleO card, and thus, did not receive teacher feedback during the day. They continued to check 37 during the morning and check out after school with the CleO coordinator. The participants received points for appropriate behavior during check-in and check-out. Students continued to earn points to obtain rewards based on the percentage of points earned. In addition, the parent report was no longer sent home each day. Data Analysis Visual analysis was used to examine observational data. Level, trend, variability, and the immediacy of effects were analyzed to interpret results of the single subject study. In addition to effects of CleO on problem behavior, we examined the relation between adult attention and the efficacy of ClCO and fading. Although it was not experimentally evaluated (adult attention was not manipulated directly), adult attention was observed during all phases of the study. Conditional probabilities were calculated to explore the extent to which adult attention followed pro-social behavior and problem behavior. Two formulas were used, a behavior-based proportion and an event-based proportion. For both calculations, the numerator was the same, intervals scored with problem behavior followed by intervals scored with adult attention in the same or next consecutive interval. For the behavior-based proportion, the denominator was intervals scored with problem behavior. For the event-based proportion, the denominator was intervals scored with adult attention. Behavior-based proportions identify the proportion of responses followed by a specific consequence. Proportions closer to 1 indicate a richer schedule of consequence delivery. Event-based proportions identify the proportion of intervals that a consequence followed a response. Proportions closer to one indicate a stronger dependency of the consequence. To avoid an inflation of the probability based on infrequent occurrence of adult attention or problem behavior, conditional probabilities were calculated only for observations during which the respective denominator was scored during at least 5 intervals. 38 39 CHAPTER III RESULTS The present study investigated (1) whether there was a functional relation between CICO and reductions in problem behavior, (2) whether CICO could be systematically faded while reductions in problem behavior maintained, and (3) whether adult attention predicted the success of CICO and fading procedures. First, data are presented addressing the efficacy of CICO and fading procedures. Second, results on the relation between adult attention and the success of CICO and fading are presented. Problem Behavior and Academic Engagement. Results for problem behavior and academic engagement are depicted in Figure 2 for Kyle, Figure 3 for Mike, Figure 4 for Nick, and Figure 5 for Paul. For both figures, the top panel depicts results for intervals scored with problem behavior and bottom panel depicts results for intervals scored with academic engagement. Kyle Problem Behavior. During baseline, intervals scored with problem behavior averaged 37%, with a range of26% to 49%. Upon implementation ofCICO, an immediate reduction of problem behavior was observed, with problem behavior occurring in an average of 12% of intervals (range = 3% to 24%). There were no overlapping data points across baseline and this phase. 40 a am a Fa:e1 Fa:e2 I 100 , I J :5 I 80-'. , .~ 0 , UI .- J - > oo~ I'" ro~.r: IOJ OJ Iem , :: E • I Problemo OJ 40- I Behavior -:0 I c 0 ,OJ ~ : ...... '!:'a. 20 • I,. IOJ , 'a. , I ...\ " .... I , .. •0 , , .... 0 5 10 15 20 25 2D 35 40 45 50 Sessions El.. ClCX) El.. ClCX) Fa:fe 1 Fa:fe2 100 - • ..... ! • • ., .r: _ • .... , ••00.::= C ,.~ OJ +" , ..... .....UI E 80 ., , j- OJ ,roo>~ '" ,OJ 0> , - c 60 • , .!Ow , - u , o .- , - E ,c OJ 40 , AcademicOJu ,~ ~ I Engagement c'f« ,l20 , I , , 0 , 0 5 10 15 20 25 30 35 40 45 50 Sessions Figure 2, Kyle's Results for Problem Behavior and Academic Engagement 41 After a brief return to baseline to establish functional control over problem behavior, CICO was reinstated, and an immediate reduction of problem behavior was again observed. Problem behavior occurred only rarely during any observations, with an average of 3% of intervals (range = 0% to 9%). When fading was introduced, no change in the level or variability of problem behavior was observed. Problem behavior occurred in an average of 5% of intervals (range = 0% to 25%). Although Kyle continued to emit problem behavior only rarely during the second fade, data collection ceased due to the end of the school year. Academic Engagement. During baseline, intervals scored with academic engagement averaged 63%, with a range of 44% to 83%. Academic engagement immediately increased upon implementation of CICO, averaging 83% of intervals with a range of 61 % to 94% During the return to baseline, academic engagement continued to be scored in most intervals. When CICO was re-instated, the level ofacademic engagement increased and variability decreased, averaging 96% of intervals with a range of 91% to 100%. Academic engagement continued to occur in the majority of intervals during both steps of fading although a small increase in variability was observed. Mike Problem Behavior. Problem behavior occurred during an average of 28% of intervals during baseline (range=8% to 40%). Implementation ofCICO resulted in an immediate reduction of problem behavior, with problem behavior occurring an average of 11% of intervals (range = 3% to 30%). 42 100 - a. aco a. aco Fade 1 Fade 2 Fade 3 Fade 2 £ 0 0 ,, , -ji 00 , , , ,~.~ , , , ,0 , , , ro ro 0 , 0 ,~.r 00 , , , , , Q) Q) , I , , , , cO) , , , , , , E , , , , , 15 , , , I , , Q) , , ~:D 40 ~ ., , , I , , Problemc 0 • / , , , , ,Q) ~ , , " , , , Behavior20.. / • , , , , , , Q) 20- , , , , , • , 0.. , /\ , , \ , , ,, , , , ,, , , , , , •, -, ., ,, • , \' .- , ~0 , ,- -----_.. , ,. "1·'----- 0 5 10 15 20 25 30 35 40 45 Sessions Fade 2 Fade 3 Fade 2 .r~ c -;:: Q) if) E - Q) ro Ol~ ro Q) Ol ~ C~w - 0o --~E C Q) Q)u t? ~cr« 100 00 00 40 20 o o • a. 5 10 a. , , , , , , , , , :.~: I , , , , , , , , , , , , , , , , 0 , 0 o 0 o 0 , , 15 QCX) 20 , , , , , ., , , , , , , , , , , , , o . o o , 25 Fade 1 30 , , :.. ;" , , , , , , , , , , 35 , , , , :. , , , , , , , , , , o o , o o -'. • 40 Academic Engagement 45 Figure 3. Mike's Results for Problem Behavior and Academic Engagement 43 A brief return to baseline was implemented to establish functional control over problem behavior, and resulted in an immediate increase in problem behavior. Due to the intensity of his behavior and teacher request, the reversal was limited to two data points. Upon return to CICO, problem behavior again was scored in relatively few intervals, averaging 9% of intervals, with a range of2% to 23%. When the fading was introduced, intervals scored with problem behavior remained low and variability was reduced with an average of 5% and a range of 3% to 9%. Intervals scored with problem behavior remained low upon the second fading phase with an average of 7% (range = 6% to 8%). Upon the third fading phase, problem behavior increased slightly and due to teacher request, the second fading phase was reinstated. This resulted in a reduction in problem behavior to levels observed in the prior phases of CICO. Academic Engagement. During baseline, intervals scored with academic engagement averaged 64%, with a range 45% of 70%. Upon implementation of CICO, academic engagement increased, averaging 72% of intervals (range = 54% to 92%). During the reversal to baseline, academic engagement occurred in the majority of intervals, however a small reduction was observed. Academic engagement immediately increased when CICO was re-implemented, averaging 85% of intervals with a range of 70% to 96%. Intervals scored with academic engagement remained similarly high during the first and second fading phases. During the third fading phase, intervals scored with academic engagement slightly decreased, and due to teacher request the second fading phase was re-instated. Re-instatement of the second fading phase resulted in an 44 immediate increase in intervals scored with academic engagement similar to levels observed in previous phases. Nick Problem Behavior. During baseline intervals scored with problem behavior averaged 28% with a range of 10% to 37%. Upon implementation ofCICO, an immediate reduction in problem behavior was observed, occurring an average of 8% of intervals (range = 3% to 8%). A brief return to baseline to establish functional control over problem behavior was implemented, and problem behavior slightly increased. Due to teacher request, CICO was re-instated, and problem behavior immediately decreased, occurring an average of 6% of intervals (range = 10% to 18%). Problem behavior remained low when fading was initiated, with an average of 4% of intervals scored with problem behavior (range = 0% to 7%). Intervals scored with problem behavior continued to remain low during the second fading phase. Upon implementation of the third fading phase, problem behavior slightly increased, and due to teacher request, the second fading phase was re-instated. This resulted in a reduction in problem behavior similar to levels observed in previous phases. Academic Engagement. During baseline, intervals scored with academic engagement averaged 58%, with an average of 43% to 71 %. An immediate increase in intervals scored with academic engagement was observed, with an average of 79% and a range of 61 % to 90%. 45 100 a a Fade 1 Fade 2 Fade3 Fade2 80 40 20- o o • 5 10 ..-. ,. 15 :...... , \ 20 .. • 25 , o o ---' 35 40 Problem Behavior 45 Sessions Fa:Je2 Fa:Je 3 Fa:Je 2 Academic Engagement • / I :. :. : Vi , , , , , , , o o o , , Fa:Je 1 , , '.,, , , , , , , , , , o , , , , , , .,-------'--- QCX) •• • QCX)a o 80 20 40- 100 £ ~ ~ :ii VI E - '" '" CJ)~ '" '" CJ) - c:Ew - uo --~ E c: '" "'-0 f:! '" '" ua.<{ o 5 10 15 20 25 35 40 45 Sessions Figure 4. Nick's Results for Problem Behavior and Academic Engagement 46 Following the return to baseline, a small reduction in intervals scored with academic engagement was observed. Academic engagement increased upon re- instatement of CICO, averaging 89% of intervals with a range of 70% to 100%.. Intervals scored with academic engagement remained high during the first two fading phases. During the third fading phase, a small reduction in intervals scored with academic engagement was observed, and due to teacher request, the second fading phase was re- instated. Upon returning to the second fading phase, intervals scored with academic engagement increased to levels observed in previous phases. Paul Problem Behavior. During baseline, intervals scored with problem behavior averaged 23% with a range of 2% to 55%. Upon implementation of CICO, an immediate reduction was observed, with problem behavior occurring an average of 16% of intervals (range = 7% to 33%). Following a short return to baseline, problem behavior increased. Due to teacher request, CICO was re-instituted resulting in a reduction in the level and variability of problem behavior. Problem behavior rarely occurred, averaging 8% of intervals with a range of 1% to 18%. The level and variability of problem behavior remained low when fading was initiated, averaging 3% of intervals (range = 0% to 7%). This pattern continued during the second fading phase. During the third fading phase, problem behavior increased, and due to teacher request, the second fading phase was re-instituted. This resulted in a reduction in problem behavior similar to levels observed in previous phases. 47 100 00 00 - 40 - 20 o o 5 10 15 , , , , · , , , · , , , , , , , , · ,. \ 20 • • 25 Fa:le 1 Fa:le 2 Fa:le3 Fa:le2 , , , , , , , , , · , , , , , , , , , , , · , , , . , · , , , , , , . , , , Problem. , , , , , Behavior, · , , , , , , • , , , /. , . , ;., . , ...........:.~L '''- :n 35 40 45 Sessions 45 Academic Engagemenl 4035 Fa:le 1 Fa:le2 Fa:le 3 Fa:le2 , , :. / , .' • 100 - :. .....r .c _ 00- ......'~ c I::;; OJ- 0.25 Major Referralsro 0 iii 02 o Baseline0. - 0.25· Minor Referrals ro 0 o Baseline iii 02 0. ID OeD - 0.25 -, Major and Minor Referrals ro 0 i 0 Baseline.iii 02- 0. OeD « 0.5 - 0- ...... Kyle Mke Nck Pall Participant 5- Teacher Effort 4.5 OJ 4c .....- o Baseline ~ 3.5 -0:: I!JCXD OJ 3 - - o Fade 1 .<: u 2.5 -m o Fade2([) -- I- 2 .....1 ~ o Fade 3([) -,- -OJ 1.5m OJ > 1« r0.5-0 Kyle Mke Nck Pall Participant Figure 7. Average Rating of Student Problem Behavior and Teacher Effott 52 100 - Percent of Points Earned I pr---I-e-- ,---80 ,-- - f----- - if) II J f-----C0 oOCD[L 604- 0 o Fcde 1 C (l) o Fcde2~ 40 -(l) [L o Fcde3 20 a Kyle Mke Nck Paul Participant Percent of Days Goal Met 100 ,--~ 80 - if) >- l\l 600 4- 0 c 40(l)~ (l) [L 20 a Kyle Mke Nck ,-- - f-- Paul oOCD I!J Fcde 1 o Fcde2 o Fcde3 Participant Figure 8. Check-in/Check-out Points 53 During CICO, the average percentage of points earned across all participants was 83%. On average, participants continued to earn a high percentage ofpoints (above 80%) during all fading phases. During CICO, participants met their goal an average of 89% of days. The percent of days with the goal met remained high (above 80%) during the first and second fading phases. During the third fading phase, the average percent of days with the goal met reduced to 74%. Relation ofAdult Attention and the Efficacy ofClCO and Fading We assessed the relation between adult attention delivery and problem behavior to detennine if the amount of attention delivered varies and whether teachers might be less likely to attend to problem behavior during CICO and fading. Because similar patterns were observed across participants, results are depicted and discussed across participants. Thus, intervals scored with attention are in Figure 9, behavior-based calculations are in Figure 10, and event-based calculations are in Figure 11. For each figure, the top panel depicts results for Kyle, the second panel for Mike, the third panel for Nick, and the bottom panel for Paul. Across participants, adult attention occurred rarely and changes in delivery of adult attention were not observed across phases (Figure 9). During baseline, intervals scored with attention averaged 7%, with a range of0% to 25%. Attention remained similarly low during CICO and all fading phases. Figure 10 depicts behavior- based conditional probabilities and, across participants, the majority of intervals scored with problem behavior did not result in adult 54 a CXD a aD Fade 1 Fade 2 100 rn 80 m ~ 60(I) E '0 40 - C Kyle~ 20 (I) •Cl. 0 -- ---- 0 5 10 15 20 25 30 35 40 4S 50 Sessions a aD a aD Fade 1 Fade 2 Fade 3 Fade 2 100 . rn 80 -ro ~ (I) E 00 - '0 c 40 , Mike(I) , ~ (I) 20 - • ,Cl. , , , 0 ,- , - 0 5 10 15 20 25 30 35 40 45 Sessions a aD ElL Fade 1 Fade 2 Fade 3 Fade 2 100 - rn 80m ~ :~ 60- '0 40 NickC ~ . & 20 0 0 5 10 15 20 25 30 35 40 45 Sessions bL UUJ t1.. UUJ race " ....aae'" t-aoe '" t-aoeL 100 80 • rn I m ~ 002 c '0 40 C Paul(I) 20 -;~ (I) Cl. 0 0 5 10 15 20 25 30 35 40 45 Sessions Figure 9. Intervals Scored with Adult Attention 55 ElL OCD ElL OCD Fade 1 Fade 2 08 (/l , ""iii c: 0.6 - ,Q) ·c · Kyle · , '0 ·0.4 ,. c . , ~0 :. ,....-.....t , . ~ , • ..' \0 02 .~ . I0. •e • ~ • ! ,. •D.. • ..: \ 0 .' .. _.-....-.-..- .- • 4o. .J. • 0 5 10 15 20 25 30 36 40 45 50 Sessions ElL OCD ElL OCD Fade 1 Fade; Fade 3 Fade 2 .: (/l 0.8 Ii""iii ,c: .ill 0.6c '0 Mike c 04 I0t0 , •g. 0.2 . , .. .. I ,D: • I'. , • • .. ••• ' ~0 -40 - .'_........--...-. • • '. , • 0 5 10 15 20 25 30 35 40 45 Sessions ElL cx:o E'l... OCD Fade 1 Fade 2 Fade 3 Fade 21 0.8 I(/l ""iii t c: 06 •.ill c .\ '0 . •04 , • •c Nick 0 •t • it8. 02 • .: •2 • 10:D.. 0 ., • ...... 0 5 10 15 20 25 30 36 40 45 Sessions cx:o ElL OCD Fade 1 Fade 2 Fade 3 Fade 2 (/l 0.8""iii c: Q) E 0.6 '0 \c Paul0 04 -t 0 •0. '.e 0.2· • ; \D.. • 4o' ,. / 0 • • ~_. .' ...... • . _... .'- ...- 0 5 10 15 20 25 30 35 40 45 Sessions Figure 10. Behavior-based Conditional Probabilities 56 a... (XX) 6L (XX) Fade 1 Fade 2 1 (/) 08 Cii c: 06- •2 ·c ·"- ·0 0.4 · ·c , Kyle 0 , :e ·0 0.2 , .. /,0- .. ,. e ., a. 0 -------'-- "-- ... . .... 0 5 10 15 20 25 30 35 40 45 50 Sessions a (XX) a aco Fade 1 Fade~ Fade 3 Fade 2 ,e (/) 0.8 - \ Cii c: e, •2 0.6c "- : 0 .. c 0.4 •0 :e .. , Mike, 0 :. :-0- 0.2 •e - -a. 0 - • --- ---.' 0 5 10 15 20 25 30 35 40 45Sessions a... aD a... aD Fade 1 Fade 2 Fade 3 Fade 2 (/) 0.8Cii -, •c: 2 -0.6 • , .£: e "- . 0 . c , Nick0.4 , I0 , • :"-E ,0 : .. 0- e 0.2 a. -,e /, 0 • '. • 0 5 10 15 :zcs 25 30 35 40 45essions a... aD a... aD Fade 1 Fade 2 Fade 3 Fade 2 (/) 08 Cii c: :-2 0.6 c '0 c 0.4 - 0 Paul:e -:00- 0.2e - , a. .- :- -, - •0 .. • '. • ,--- 0 5 10 15 20 25 30 35 40 45 Sessions Figure 11. Event-based Conditional Probabilities 57 attention. During baseline, with an average conditional probability of .11. This pattern did not change upon implementation of CICO, with an average conditional probability of .11. This pattern remained consistent during all fading phases. Event-based calculations (Figure 11) reveals that, with the exception of results obtained with Paul, adult attention delivery was fairly dependent on problem behavior during baseline for all participants, with an average conditional probability of .42. When CICO was implemented, this dependency was reduced to an average conditional probability of .19, suggesting that teacher's were attending to behaviors other than those targeted for reduction under CICO. Although adult attention appeared somewhat more dependent on problem behavior once fading began across participants, this conclusion should be tempered by (a) the variability in the data and (b) the low occurrence of attention overall. Fidelity ofImplementation Fidelity of implementation was assessed 27% of sessions using the checklist described previously. Overall fidelity was high, with an average of 97%, and a range of 83% to 100%. For Mike, fidelity averaged 97% with a range of91 % to 100%. For Kyle, fidelity averaged 98%, with a range of 91 % to 100%. For Paul, fidelity averaged 96%, with a range of 91 % to 100%. For Nick, fidelity averaged 95%, with a range of 83% to 100%. On the day that fidelity was moderate (83%), Nick's card was not scored at the correct time, however his teacher reported that she provided feedback at a different time during the day. 58 Contextual Fit Contextual fit was assessed two times for each participant; during the initial implementation ofClCO, and at the end of the study. Classroom teachers were asked to complete the 16-item questionnaire for each participant (therefore the same teacher completed the checklist for three participants). Overall, results indicated that the ClCO intervention rated high on contextual fit during initial implementation ofClCO (M = 98%). Mike, Nick, and Paul's teacher rated contextual fit as 97% across all three participants. More specifically, all items were rated as a "6", except that the school provided the time to implement interventions, which was rated as a "4". Kyle's teacher rated overall contextual fit as 98%, with all items rated as a "6", except that the school administration is invested in designing and implementing effective behavior interventions, which was rated as "5". At the completion of the study, both teachers rated contextual fit at 100% for all participants. 59 CHAPTER IV DISCUSSION In this study we used a reversal design to examine the efficacy of stimulus fading within the CICO program. Check-iniCheck-out has an emerging research base supporting its utility in reducing problem behavior and increasing academic engagement in students. Although CICO has been shown to be effective, no studies to date have documented methods for fading or removing the intervention when students are successful. Thus, this study examinedl) whether CICO can be faded while reductions in problem behavior maintaining, and 3) if adult attention delivery predictive of success on the CICO program and fading. In this chapter findings are discussed in relation to the previous research questions. In addition, the chapter is broken down into three sections: summary of findings, study limitations, and implications. Summary ofFindings In this section, results are summarized and discussed in relation to the previously stated research questions. Problem Behavior. This research question was evaluated using an ABAB reversal design with two primary phases: baseline and CICO. Overall, CICO was functionally related to reductions in problem behavior across all participants. 60 Other more indirect measures of problem behavior suggested as well that CICO resulted in improvement in student behavior. Teacher ratings were generally lower during CICO phases when compared with baseline ratings. A reduction in office discipline referrals was evident upon implementation of CICO, with no participants receiving any major referrals after implementation. Finally, on average, participants earned greater than 80% of daily points on the CICO card, and met their individual goal more than 90% of days. Academic Engagement. Although academic engagement was not the primary dependent variable in this study, it appeared that CICO also increased engagement. A moderate functional relation was established for Nick and Mike, with academic engagement varying as a result of CICO. In fact, for Nick, CICO appeared to have stronger control over academic engagement than over problem behavior. This is not surprising given that Nick's teacher initially reported that off-task behavior was a concern. CICO had relatively weak control over Kyle's and Paul's academic engagement. Although it appeared to increase upon initial implementation of CICO, it did not reverse back to baseline level. Importantly, academic engagement was not the dependent measure upon which phase changes were made; had changes been contingent upon academic engagement, it is feasible that different patterns would have emerged for these two participants. Interestingly, academic engagement increased significantly after CICO was re- instated for all four participants. This may indicate that CICO needs to be in place for a certain amount of time before it impacts student academic engagement. 61 Overall, results indicated that CICO is effective in reducing problem behavior and increasing academic engagement for all four participants. The multiple measures of problem behavior (i.e., direct observations, teacher ratings, office discipline referrals, CICO points) all indicated similar results. In addition to the efficacy of CICO, the high ratings of social validity should be noted. When CICO was implemented, all teachers rated the intervention with high contextual fit for all participants. Teachers also rated that it took less effort to manage the participant's behavior compared with baseline ratings. This is an important finding because implementing an intervention increases the amount of time the teacher may devote to a particular student. It is promising that although teachers may have put in more time to implement the intervention, the amount of effort managing problem behavior decreased. Fading. Overall, fading procedures were mildly successful for participants. Although fading was successful to a certain point for all participants, the intervention was never completely removed during fading. In addition, participants never stopped checking in and out with the CICO coordinator during the fading phases, and therefore conclusions about fading that component of the intervention cannot be made. This is a particular concern because schools often report that the time spent checking in and out with the coordinator is often the most resource laden portion of the intervention. Reductions in Kyle's problem behavior maintained during the first and second fading phases, where he was receiving teacher feedback one time per day. Fading was 62 discontinued due to the end of the school year, and therefore no assumptions can be made regarding the success of future fading phases. In addition, Kyle's teacher requested that the fading phases remain in place for a longer period of time than originally planned; therefore restricting the amount of fading that could be completed prior to the end of the school year. Nick, Mike, and Paul maintained reductions in problem behaviors across the first two fading phases. During the third fading phase, where scheduled classroom feedback was eliminated, slight increases in observed problem behavior, teacher perceptions, CleO points, and decreases in academic engagement were present. Although increases in problem behavior were not dramatic, particularly with Nick, the teacher requested that the students move back to the second fading phase. The school year ended shortly after the second fading phase was re-implemented. Therefore, based on the limitations due to timing of the school year and requests made by classroom teachers, the present study does not demonstrate that CleO can be systematically faded in a socially significant manner. Although portions of the intervention, particularly teacher feedback, can be reduced, it is unknown whether other portions of the intervention can be systematically faded. Despite the inability to fade all components of the intervention, teachers continued to rate the intervention with high contextual fit. All teachers increased their ratings upon study completion, indicating that CleO has contextual fit that maintains over time. In addition, teachers continued to rate the amount of effort put into managing problem behavior lower than average baseline ratings. 63 Adult Attention. In addition to effects of CICO and fading on problem behavior, we were also interested in examining adult attention. More specifically, we were interested in whether adult attention changed as a result of CICO and fading. When the results of adult attention are taken together, it did not appear that the success of CICO and fading was related to attention delivery. The amount of attention delivered was relatively low, and conditional probabilities were limited because of this. Overall, adult attention was low across all participants. This may be largely due to the specific setting and activities where observations took place. For all participants, observations took place during large group activities, with at least 20 other students present. It could be anticipated that adult attention would be relatively low during this time when compared with settings with fewer students present (e.g., small group instruction, individual instruction). In addition to low adult attention, it did not appear that problem behavior varied as a function of attention. No patterns emerged for any participants based solely on the amount of attention. Behavior based conditional probabilities were calculated to examine the proportion of intervals with problem behavior that were followed by adult attention. Minimal to no changes in probabilities were observed for all four participants. For Kyle, Nick, and Mike adult attention did not follow problem behavior for a majority of intervals across sessions, although there was large amount of variability across sessions. For Paul, it appeared that conditional probabilities increased in level during the fading 64 phases. However, it should be noted that Paul displayed relatively low problem behavior during fading and therefore 14% of sessions were not included in these calculations. It is not surprising that behavior based conditional probabilities did not change across study phases because CICO does not include an extinction component. Therefore, we would not expect teachers to have provided less attention to problem behavior. In fact, one might expect that teachers may be more attuned to student behavior during CICO given that they were providing feedback several times each day. Results should be interpreted with caution due to the number of sessions that were eliminated due to low levels of problem behavior. Overall, an average of 14% (range = 2% to 20% of sessions) of sessions were not included due to low levels of problem behavior. This is of a particular concern when interpreting the patterns of conditional probabilities during CICO and fading, as problem behavior was low during these phases. Event based conditional probabilities were calculated to examine the proportion of intervals with attention preceded by problem behavior, or the dependency of adult attention on problem behavior. Overall, it appeared that the dependency of adult attention on problem behavior decreased during CICO and the fIrst fading phase; however results varied within participants. In addition, variability was high across all phases of the study. For Kyle, Mike, and Nick adult attention was less dependent on problem behavior during CICO, particularly when CICO was re-instated after the return to 65 baseline. For Paul, no significant change was observed, however the dependency of adult attention on problem behavior was relatively weak during baseline phases. During fading, event based conditional probabilities slightly increased for all participants. For Kyle, an increasing trend during the second fading phase was observed. Although the dependency of the adult attention increased, it did not appear to effect problem behavior or academic engagement. Similarly for Mike, the dependency of adult attention on problem behavior increased during the first and third fading phases, however variations in direct observations of problem behavior did not appear to be related. For Paul, event-based conditional probabilities increased during the third fading phase. In addition, there was a slight increase in problem behavior and a reduction in academic engagement during this phase. Proportions remained relatively high when the second fading phase was re~instated,but problem behavior was low during this phase. These results should be interpreted with caution as 44% of sessions were not used due to low amounts of adult attention. This was a particular problem during the fading phases, as only 5 sessions were used to calculate event based conditional probabilities. Conditional probabilities for Nick increased during the third fading phase, where a slight increase in problem behavior and a reduction in academic engagement was also observed. However, when the second fading phase was reinstated, problem behavior decreased where as event~based conditional probabilities remained high. Study Limitations The present study utilized a reversal design to control for threats to validity. This design allowed us to evaluate whether CICO was effective, while controlling for possible 66 external influences. To evaluate fading, reversal designs were used to detennine the extent to which fading was successful. Although the design controlled for several threats to validity, there were several threats to internal and external validity. Threats to internal and external validity, as well as other limitations are described below. Internal Validity. Although the use of a reversal design controlled for external factors that may have influenced study results, the length of the study and variation among participants presented some limitations of the results. Results should be considered in conjunction with the limitations presented below. One possible threat to internal validity is participant reactivity to being observed. The presence of an additional person in the classroom may have influenced behavior, however reactivity effects likely decreased over time as observers in the classroom became a regular occurrence. The fading phases occurred at the end of the school year, which may have resulted in possible setting events. For example, there were several special activities such as assemblies, field day, school-wide rewards, and a decrease in academic instruction. These events may have made adult attention and/or incentives provided from CICO less desirable to participants, and therefore resulting in an increase in problem behaviors. Anecdotally, expectations for appropriate behavior during classroom instruction were lower at the end ofthe year. For example, teachers provided fewer reprimands to the entire class for engaging in minor problem behaviors. 67 External Validity. Due to the low munber of participants, results may only be generalizable to similar individuals (i.e., Caucasian males). One particular concern in this study is that the participants were all male. It is unknown whether female participants would display similar results. In addition, the majority of participants were Caucasian. Caution should be used when applying results to other ethnic groups. The majority of research on CICO has included Caucasian males as participants, and future research should attempt to include female and/or racially diverse participants. In addition, the problem behavior of all participants appeared to be maintained by, or at least sensitive to adult attention. Previous studies have established that the standard CICO program may not be effective for participants whose behavior is maintained by peer attention or escaping tasks (e.g., Campbell, Anderson, & Todd, under review; March & Homer, 2002). Although modifications to the program have documented success, results of the fading procedures cannot be generalized to participants who engage in problem behavior to obtain peer attention or escape tasks. Mike, Nick, and Paul were all in the same classroom, and had the same teacher. Therefore, results are limited to teachers who have the similar skills in classroom management. The present study took place in a suburban elementary school. Generalizing results to either urban or rural settings is questionable. Another potential concern is that the school had been implementing School-wide Positive Behavior Support with fidelity for approximately four years. It is unknown whether CICO is effective 68 without a school-wide system of behavior support in place. This is a particular concern when examined in conjunction with other studies examining CICO. All previous studies took place in schools that had a universal discipline system in place. In addition, CICO is a multi-component intervention that may vary across different settings. For example, schools may have a different system of acknowledgement; have more or less times for teacher feedback, and different behavior expectations. Although these components are aligned with the CICO procedures, results of this particular study are only generalizable to schools that implement CICO with similar features as the present school. Additionally, observations were only IS-min and occurred 3 to S days per week. Check-in/Check-out is implemented across the entire school day, and the direct effects of the intervention during other times are unknown. This is of particular concern because the participants were reported to engage in varying levels of problem behavior across the entire day, and observations only took place when problem behavior was most likely. Although this is certainly a limitation of the current study, other measures (e.g., teacher ratings, CICO points, office discipline referrals) provided similar results for all participants. Other Limitations. There are several other limitations in the present study, particularly the measures. One significant limitation is the teacher ratings of student behavior. Classroom teachers were asked to rate the target student's problem behavior in relation to other students in the classroom twice per week. However, teachers did not complete ratings as 69 often as requested. On average, teachers provided ratings more frequently during the beginning of the study and decreased particularly during fading. When completing ratings only occasionally, and not on a pre-determined schedule, teachers may have only provided a rating when the students behavior was particularly challenging or above average. Results from the behavior ratings should be interpreted with caution. This also applies to teacher ratings ofthe amount of effort he/she spent managing the student's behavior. The teacher may have provided a rating only on days that took an abnormal amount of effort. Additionally, results from office discipline referrals should be interpreted with caution. Office referrals are often subjective, and reporting behavior may vary across school staff members. No direct information (other than principal verbal reports) was obtained regarding the consistency of reporting across staff and settings. Implications for Research and Practice In this, section implications for practice, specifically the use of targeted interventions, fading procedures, and the role of adult attention in intervention success will be discussed. In addition, implications for future research will be discussed. The present study documented that the secondary intervention, CICO, is effective in reducing classroom problem behavior across four elementary aged students. In addition, this study also demonstrated that along with reductions in problem behavior, academic engagement increased across all four participants. This is of particular 70 importance because the ultimate goal of school is for students to learn academic content, not simply behavior in a socially appropriate manner. Schools are in growing need of effective and efficient behavior interventions for students who are at-risk for developing significant problem behavior. This study demonstrated that a relatively simple and efficient intervention can reduce classroom problem behavior and increase academic engagement. Further, results maintained over time. In addition to the efficacy of CICO, it was an intervention that was implemented with high fidelity. Fidelity of implementation is a major concern when providing school- based behavior support because of the increasing demands being placed on classroom teachers (e.g., increased instruction time, larger class sizes, and differentiation for multiple skill levels). Teachers often have a variety of interventions and strategies that are implemented across multiple students, which results in little time to spend on behavior interventions. It is encouraging and important that CICO was implemented with near perfect fidelity, indicating that it is feasible for teachers. Anecdotally, teachers reported that the intervention was relatively simple to implement and that they felt they could utilize the same strategies with other students in their classes. Teachers also rated CICO of having high contextual fit during initial implementation and at the completion of the study. This indicates that ClCO procedures are not only acceptable at the beginning of the intervention, but also throughout time. This may suggest that CICO is a sustainable practice, with ratings high ratings of acceptability over time. 71 The present study also indicated that it is possible to fade certain components of CICO with reductions in problem behavior maintaining. For all participants, fading was successful through the second fading phase, where teacher feedback was provided one time per day. It is impressive that reductions maintained with a limited amount of scheduled feedback. Observed adult attention was low throughout the entire study and for each participant. Despite this, there were marked decreases in problem behavior. This may indicate that reductions in problem behavior may be related less to the amount of attention, and more to the manner in which it is delivered. For example, teachers continued to provide attention for problem behavior throughout the study, but decreased the dependency of attention on problem behavior during CICO and some fading phases. This is an important finding because it may be more beneficial to provide teachers with assistance on how to deliver attention, rather than simply encouraging increased attention delivery. Based on the results and limitations of the present study, several areas for future research will be discussed. First, research needs in the area of demonstrating the efficacy of CICO will be discussed. Second, future studies on fading methods will be discussed. Finally, future research on the impact of adult attention will be addressed. The present study added to the emerging literature base supporting CICO as an evidence-based practice, however many more direct and systematic replications are needed. To date, all experimental studies examining 72 CICO have utilized single subject research methodology. As defmed by Homer, Carr, Halle, McGee, Odom, & Wolery (2007), in order for a practice to be determined as evidence based with single subject research there must be a minimum of5 methodologically sound studies with a total of 20 participants across the studies. In addition, studies must be conducted in a minimum of three locations by three different researchers. Although this study contributes to the evidence based, CICO is yet to be determined an evidence based practice according to the aforementioned standard. A total of three experimental studies have been conducted; however all studies have been conducted in rural and suburban schools in the Pacific Northwest. Future research should be conducted in different locations to increase the evidence of CICO. Another limitation of the present study was that problem behavior and academic engagement were only observed for I5-min each session. Given that CICO is an intervention that is implemented across the entire school day, future studies should conduct observations during multiple times of the day. For example, it is important to know whether CICO impacts behavior in different settings (different classrooms, playground, lunchroom, etc.) and also at different times of the day (early morning, late afternoon, etc.). This is particularly important because the majority of students who are recommended for CICO display low level problem behavior throughout the day (Crone et aI, 2003). One area of research that is lacking is a component analysis ofCICO. Check- in/Check-out has several different components to it, and it is unclear which components 73 are necessary for obtaining similar effects. For example, a study that examines the individual and combined effects of the checking in and out, the daily behavior card, the home report, and the incentive system would help to determine the necessary components ofCICO. Further, the way in which schools implement specific components is varied. For example, the number of times each day the student receives feedback varies, and it is unknown what the optimal number of times per day is. Future studies should begin to identify and elaborate on characteristics of intervention features. In addition to the intervention features specific to CICO, there are several systems level variables that may impact whether students are successful on the intervention. No research to date has examined systems level features and effects on student behavior. For example, the present study, as well as all previous studies, was conducted in a school implementing school-wide positive behavior support with fidelity. It is unclear whether results would differ in schools that do not currently have SWPBS in place. In addition, other system features such as the CICO coordinators role, team-based planning, and the use of data based decision making have not been examined in the context of CICO. Future research should examine how the presence or absence of certain features predicts student outcomes. The present study documented that some components of CICO can be faded successfully, but did not fully demonstrate a successful fading procedure. There are several features of the fading procedures that may have impacted the success, and future research should address these limitations. First, the order in which specific 74 features were removed was held constant across all participants. It is unknown if results would have been different if components were removed in a different order. Future studies should examine if removing the checking in and out component first, followed by reductions in the classroom feedback would yield favorable results. Another limitation was the relatively short amount of time that participants were on CICO. On average, participants were on CICO for approximately six weeks prior to fading. It may have been too short of a time of full implementation to begin fading. Future studies should vary the amount of time that students remain on CICO prior to fading to determine if a longer time may increase the success of the fading procedures. In addition to increasing the amount of time with full implementation ofCICO, future studies should examine increasing the amount of time spent in each fading phase. Fading was rapid in the present study due to time constraints (the school year ending), and may have weakened the results. The low amount of observed adult attention significantly limited the ability to determine if adult attention predicted success on CICO and fading. Because of the low levels of attention, several sessions could not used in calculating conditional probabilities, and therefore the results are questionable. To address this limitation, future studies should strive to observe in settings where adult attention delivery may be richer. All observations in the present study took place in large group settings, where it can be expected that adult attention would be relatively low. Small group instruction or individual instructional settings may be a better setting to observe environmental behavior relations. 75 Another limitation in terms of examining environmental behavior relations was that adult attention was the only consequence recorded. Although it appeared that adult attention was the primary maintaining consequence for all participants, other environmental contingencies may have influenced the efficacy of CICO and fading. Future studies should include all possible environmental consequences (e.g., peer attention, avoidance of task, obtaining a preferred activity) and determine if the contingencies change across phases of the study. Future studies should also examine different methods of fading interventions. One possible problem with the fading strategy is that the schedule of reinforcement (i.e., adult attention) became too low during fading to elicit continued responding. Adult attention did not significantly change during the different phases of the intervention, and by removing scheduled adult feedback times attention delivery became lower. In addition, event-based conditional probabilities were slightly increased during fading. This indicates that during fading, adult attention became more dependent on problem behavior. One strategy that may be effective as a fading method is self-monitoring and self-recruited praise. The reasoning for this is twofold; first, this strategy may teach students increase the amount of attention by self-recruiting, and second, it may reduce the dependency of adult attention on problem behavior. One potential concern with this strategy as a fading method is that it may in fact temporarily increase the amount of time the teacher spends implementing the intervention. Teaching students to self-monitor and self-recruit praise requires the teacher to (1) teach the student how to evaluate his or her self, (2) how to record the evaluation, and (3) how to elicit feedback. This teaching process typically 76 involves the teacher providing ratings not only for behavior, but also for accuracy of scoring. Therefore, implementation of this strategy may be more difficult and time- consuming than the standard CleO program. However, the long term benefits may outweigh the additional time if the strategy allows for long term maintenance. Future studies should examine self-monitoring and self-recruited praise as a fading method. In addition, future studies should also examine if there are more efficient ways of teaching these skills to students. APPENDIX A CHECK-IN/CHECK-OUT SELF-ASSESSMENT 77 78 Check-In / Check-Out Self-Assessment School: Date: --------- Instructions: As a team, review and record each of the CICO elements. For all elements that are rated as "in progress" or "not in place" build action planning steps. In In Not In cleo Element Place Prog,ress Place 1. Faculty and Staff Commitment for CICO defined 2. Team Defined and Available to Coordinate program 3. School-wide PBS in place 4. Student Identification Process for CICO exists 5. Daily CICO progress report card developed 6. Home report process defined 7. Point Trading System established 8. Process for collecting, summarizing and using data developed 9. Morning check-in routine established 10.Teacher check-inI check-out routine established 1l.Afternoon check-out routine established 12.Home review routine established 13. Team meeting schedule, routine, process 14. Planning for Success in place 15. Planning for Individualized Support Enhancement in place 16. Substitute Teacher routine developed 17. Playground, cafeteria, bus routine developed APPENDIXB CONSENT FORMS 79 80 Informed Consent to Participate in a Research Study Parent/GuardianlFamily/Student Consent The impact of Check-in/Check-out on students' social and academic success in school Your son/daughter is invited to participate in a study designed to increase students' social and academic success in school. The study will be conducted by Amy Kauffman, Cynthia Anderson, and Rob Homer from the University of Oregon's College of Education. The purpose ofthe study is to examine how the Check-in/Check-out program can increase students' success at school through structured and frequent teacher and adult feedback. In addition, this study will examine which components of the intervention can be removed while maintaining positive outcomes. Your son/daughter was selected as a possible participant in this study because he/she was identified as possibly benefiting from the Check-in/Check-out program. The study will begin in October 2006 and end in June 2007. To provide students with structured and frequent teacher and adult feedback, the Check- in/Check-out program consists of the following steps: • Each morning the student checks in at the front office with a designated member of the School Behavior Support Team to start the school day prepared to learn and to receive a check-in form to use throughout the day. • At each class period, the student will bring the card to the teacher. At the end of each class period the student will retrieve the card from the teacher and receive feedback on his/her behavior during that class period. • At the end of the day the student takes the card back to the front office for a check-out with the behavior support team member. Ifthe student has met 80% of possible points for that day, he or she will receive a small reward. • The student will take home the daily Check-in/Check-out card to be signed by a parent. The next morning at check-in the student will return the signed card. To tailor the Check-in/Check-out program to your child's needs, the teacher and school personnel would complete the following activities: • Review your child's academic and behavioral school records, including scores from standardized reading tests and oral reading passages, social strengths and weaknesses, attendance and discipline referral patterns, if applicable. • Complete a daily rating of your child's classroom behavior. • Complete an interview with your child's teacher assessing social strengths and weaknesses. 81 To conduct the study, researchers from the University of Oregon will complete the following activities: • collaborate with school personnel to collect and review data • conduct direct observations of your child in his or her classroom to collect data on social behavior and academic engagement Your child will not be identified in written or professional presentations of the results of this study. Every effort will be made to organize information using altered names, and professional presentations will never refer to your child by name. In addition, all information will be kept in a lockable location and destroyed after the study and holding period are complete. There remains, however, a small risk that your student may be identified as a participant in this study. There is a distinct likelihood that your student may benefit from participation in the study. The Check-in/Check-out program has shown promising results in previous studies where social and academic gains were documented for participating students. Your consent to your child's participation in the study is voluntary. Your decision whether or not to allow your child to participate will not affect your relationship with the school district or the instruction your child receives in his or her schooL If you allow your child to participate, you are free to withdraw your consent and terminate your child's participation in the study at any time without penalty.. If your child participates a gift certificate to at local shopping center will be provided to your family at the completion of the study in appreciation ofyour participation. Prior to your child's participation in the study, we will also ask your son/daughter if he or she give their assent to participate. Their assent will be necessary for participation in the study. If you have any questions, please feel free to contact Amy Kauffman at the University of Oregon (543-3230). If you have questions regarding your rights as a research subject, contact the Office of Human Subjects Compliance, University of Oregon, Eugene, OR 97403, (541) 346-2510. You have been given a copy of this form to keep. Your signature indicates that you have read and understand the information provided above, that you willingly agree to participate, that you may withdraw your consent at any time and discontinue participation without penalty, that you have received a copy of this form, and that you are not waiving any legal claims, rights or remedies. Parent/Legal Guardian _ Signature _ Name of Child ------------- Date -------- 82 Informed Consent to Participate in a Research Study Teacher Consent The impact of Check-in/Check-out on students' social and academic success in school You are invited to participate in a research study conducted by Amy Kauffman, Rob Homer, and Cynthia Anderson from the University ofOregon's College of Education. The purpose of the current study is to examine how the Check-inlCheck-out program can increase students' social and academic success through structured and frequent teacher and adult feedback and maximize efficiency in providing needed behavioral support to students. In addition, the study wiD examine which components of the intervention can be removed after a sustained reduction in student problem behavior has been achieved. The goal is to remove components to make the intervention easier to implement without reducing its effectiveness. You were selected as a possible participant, because you teach students who might benefit from additional behavioral support. The Check-inlCheck-out program consists of the following steps: • Each morning the student checks in at the front office with a designated member of the School Behavior Support Team to start the school day prepared to learn and to receive a check-in form to use throughout the day. • At each class period, the student will bring the card to the teacher. At the end of each class period the student will retrieve the card from the teacher and receive feedback on hislher behavior during that class period. • At the end of the day the student takes the card back to the front office for a check-out with the behavior support team member. Ifthe student has met 80% of possible points for that day, he or she will receive a small reward. • The student will take home the daily Check-inlCheck-out card to be signed by a parent. The next morning at check-in the student will return the signed card. If you choose to participate in the study, researchers will ask you to complete the following activities to maximize the interventions benefit for the student: • Provide a daily I-item rating of the student's problem behavior • Provide feedback on student's behavior via the check-in form after each class period the student attends. • Provide baseline information on participating students to research staff via the Functional Assessment Checklist for Teachers and Students (FACTS). This information will be used to identify the function of students' problem behavior. 83 Researchers from the University of Oregon will conduct direct observations of participating students in their classrooms to collect data on social behavior and academic engagement. In addition, environmental variables (context, adult attention) will be collected. Neither you nor the student will be identified in written or professional presentations concerning this study. Every effort will be made to organize information using altered names, and professional presentations will never refer to you or your student by name. In addition, all information will be kept in a lockable location, and destroyed after the study and holding period are complete. There remains, however, a small risk that you may be identified as a participant in this study. There is a distinct likelihood that your student may benefit from participation in the study. Preliminary evidence exists that participation in the Check-in/Check-out Program leads to students' social and academic gains. You will receive a gift certificate to a local shopping center upon the completion of the study in recognition and appreciation for your participation in the study. Your participation is voluntary. Your decision whether or not to participate will not affect your relationship with the school district. If you decide to participate, you are free to withdraw your consent and discontinue participation at any time without penalty. If you have any questions, please feel free to contact Amy Kauffinan at the University of Oregon (543-3230). If you have questions regarding your rights as a research subject, contact the Office ofHuman Subjects Compliance, University of Oregon, Eugene, OR 97403, (541) 346-2510. You have been given a copy of this form to keep. Your signature indicates that you have read and understand the information provided above, that you willingly agree to participate, that you may withdraw your consent at any time and discontinue participation without penalty, that you have received a copy of this form, and that you are not waiving any legal claims, rights or remedies. Print Name -------------------------- Signature _ Date ------------ 84 Student Assent The impact of Check-in/Check-out on students' social and academic success in school We want to ask you if you want to be part of a research study. Your (parents, guardian, etc) has told us it is ok for you to be part of this project, but we want to ask you if you are willing to help us by being part of the study. If you are part of the study you will do the Check-in/Check-out program which means you will work with your teachers, parents, and (CICO coordinator) to learn some ways to be more successful in school. We will have people come to watch you and the other students to see if the program is helpful, but you will work primarily with your teacher and (CICO coordinator). If you choose to be part of the study, we will not use your name when we share with other people how you do in school. Nobody except the people you work with will know who you are. We expect to work with you for several weeks, and in the end you should have skills that will help you in school. If you choose to be part of the study, you can always change your mind, and ifyou choose not to be part of the study, it will not affect anything else about what you do at school. If you choose to be part of the study, your family will receive a $25.00 gift certificate to a shopping center at the end of the study to thank you for your participation. Do you have any questions? If you are willing to be part of the study we would ask you to sign this form. Student Signature: _ APPENDIXC FUNCTIONAL ASSESSMENT CHECKLIST FOR TEACHERS AND STAFF (FACTS) 85 86 Functional Assessment Checklist for Teachers and Staff (FACTS-Part A) Student! Grade: _ Interviewer: -------------- Date: _ Flespondent(s): __ Student Profile: Please identify at least three strengths or contributions the student brings to school. Problem Behavior(s): Identify problem behaviors _Tardy _ Unresponsive Withdrawn _ Fight/physical Aggression _Inappropriate Language Verbal Harassment _ Verbally Inappropriate Describe problem behavior: _Disruptive Insubordination Work not done _ Self-injury Theft Vandalism Other _ Identifying Routines: Where, When and With Whom Problem Behaviors are Most Likely. Schedule Activity Likelihood ofProblem Behavior Specific Problem Behavior (Times) Low High I 2 3 4 5 6 I 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 I 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 Select 1-3 Routines for further assessment: Select routines based on (a) similarity of activities (conditions) with ratings of 4, 5 or 6 and (b) similarity of problem behavior(s). Complete the FACTS-Part B for each routine identified. March, Homer, Lewis-Palmer, Brown, Crone, Todd & Carr (2000) 87 Functional Assessment Checklist for Teachers & Staff (FACTS-Part B) Student/ Grade: Date: _ Interviewer: Respondent(s): _ Routine/Activities/Context: Which routine(only one) from the FACTS-Part A is assessed? Routine/Activities/Context Problem Behavior(s) Provide more detail about the problem behavior(s): What does the problem behavior(s) look like? How often does the problem behavior(s) occur? How long does the problem behavior(s) last when it does occur? What is the intensity/level of danger of the problem behavior(s)? What are the events that predict when the problem behavior(s) will occur? (predictors) Related Issues (setting events) Environmental Features - illness Other: _ reprimand/correction _ structured activity _drug use _ physical demands - unstructured time _ negative social _ socially isolated _ tasks too boring - conflict at home _withpeers _ activity too long - academic failure - Other - tasks too difficult What consequences appear most likely to maintain the problem behavior(s)? Things that are Obtained Things Avoided or Escaped From - adult attention Other: hard tasks Other: - peer attention _ reprimands _ preferred activity _ peer negatives _ money/things _ physical effort adult attention fb h ""n bh SUMMARY OF BEHAVIOR Id f hentHy t esummary t atwi e used to build a plan 0 e aVlor support. Settin2 Events & Predictors Problem Behavior(s) Maintainin2COnSeQuence(s) How confident are you that the Summary of Behavior is accurate? Not very confident 123 4 5 Very Confident 6 88 bl b h . ?I hdbhWhat current e oris ave een use to contro t epro em e aVlOr. Strategies for preventing problem behavior Strategies for responding to problem behavior _ schedule change Other: _reprimand Other: _ seating change - office referral curriculum change detention March, Horner, Lewis-Palmer, Brown, Crone, Todd, & Carr (2000) ---- ----_. __ . APPENDIXD CHECK-IN/CHECK-OUT POINT RECORDS 89 Week of: ------ CICO Student Account Record 90 Day Check in? Signed Today's Check out Total pts Student of home goal range goal met? earned week report? today M y N Y N Y N Tu Y N Y N Y N W y N Y N Y N Th y N Y N Y N F Y N Y N Y N M y N Y N Y N Tu Y N Y N Y N W y N Y N Y N Th y N Y N Y N F Y N Y N Y N M Y N Y N Y N Tu Y N Y N Y N W Y N Y N Y N Th y N Y N Y N F Y N Y N Y N M Y N Y N Y N Tu Y N Y N Y N W y N Y N Y N Th y N Y N Y N F Y N Y N Y N M Y N Y N Y N Tu Y N Y N Y N W y N Y N Y N Th y N Y N Y N F Y N Y N Y N M Y N Y N Y N Tu Y N Y N Y N W y N Y N Y N Th y N Y N Y N F Y N Y N Y N M y N Y N Y N Tu Y N Y N Y N W y N Y N Y N Th y N Y N Y N F Y N Y N Y N ____________ CICO Account Page 91 Date Points earned Total Points spent Total balance APPENDIXE CHECK-IN/CHECK-OUT FIDELITY MEASURE 92 Fidelity Data: Check in Check out Phase 93 Time/Location Area Student checked in with adult Yes No NA Staff member provided daily Yes No NA Check in point card Staff member provided a prompt for the student to be Yes No NA successful that day Student turned in home report Yes No NA Student approached teacher to receive feedback Yes No NA Time: Classroom Teacher assigned points to Yes No NA student Teacher provided verbal feedback regarding the Yes No NA student's behavior Student checked out with adult Yes No NA Student presented complete Yes No NA Check out card to adult Staff member added up total Yes No NA points, and recorded Staff member provided verbal back regarding student's Yes No NA behavior Staff member completed the parent report and handed to Yes No NA student Student: School: Date: Number of Yes I Number of Yes + No = ----- APPENDIXF TEACHER PERCEPTION QUESTIONNAIRE 94 Student: Teacher: Date: 1. Compared to other students in my classroom, 's behavior is 95 1 much better 2 better 3 similar 4 worse 5 much worse 2. Compared to other students in my classroom, the amount of effort I put into managing's behavior is 1 much less 2 less 3 similar 4 more 5 much more APPENDIXG SELF-ASSESSMENT OF CONTEXTUAL FIT 96 97 Self-Assessment of Contextual Fit in Schools Homer, Salentine, & Albin, 2003 The purpose of this interview is to assess the extent to which the elements of a behavior support plan fit the contextual features of your school environment. The interview asks you to rate (a) your knowledge of the elements of the plan, (b) your perception of the extent to which the elements of the behavior support plan are consistent with your personal values, and skills, and (c) the school's ability to support implementation of the plan. This information will be used to design practical procedures that will help school personnel support children with problem behaviors. The information you provide will be maintained and reported in a confidential manner consistent with the standards of the American Psychological Association. You will never be identified. Please read the attached behavior support plan, and provide your perceptions of the specific elements in this plan. Thank you for your contribution and assistance. Name of Interviewee: Role: -------------- -------- Support plan reviewed: _ Knowledge of elements in the Behavior Support Plan. 1. I am aware of the elements of this behavior support plan. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 2. I know what I am expected to do to implement this behavior support plan. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree Skills needed to implement the Behavior Support Plan 3. I have the skills needed to implement this behavior support plan. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 98 4. I have received any training that I need to be able to implement this behavior support plan. No training needed _ 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree Values are consistent with elements ofthe behavior support plan 5. I am comfortable implementing the elements of this behavior support plan 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 6. The elements of this behavior support plan are consistent with the way I believe students should be treated. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree Resources available to implement the plan 7. My school provides the faculty/stafftime needed to implement this behavior support plan. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 8. My school provides the funding, materials, and spaced needed to implement this behavior support plan. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 99 Administrative Support 9. My school provides the supervision support needed for effective implementation of this behavior support plan. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 10. My school administration is committed to investing in effective design and implementation of behavior support plans. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree Efftctiveness ofBehavior Support Plan 11. I believe the behavior support plan will be (or is being) effective in achieving targeted outcomes. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 12. I believe the behavior support plan will help prevent future occurrence of problem behaviors for this child. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree Behavior Support Plan is in the best interest ofthe student 13. I believe this behavior support plan is in the best interest of the student. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 14. This behavior support plan is likely to assist the child to be more successful in school. 100 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree The Behavior Support Plan is efficient to implement 15. Implementing this behavior support plan will not be stressful. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree 16. The amount of time, money and energy needed to implement this behavior support plan is reasonable. 1 Strongly Disagree 2 Moderately Disagree 3 Barely Disagree 4 Barely Agree 5 Moderately Agree 6 Strongly Agree APPENDIXH CHECK-IN/CHECK-OUT AGREEMENT 101 102 Check-in/Check-out Agreement Student Responsibilities: 1. Remenlber to go to check in and check out in room 11 2. Keep track of ClCO card 3. Be safe while walking down to room 11 4. Bring signed home report everyday Teacher Responsibilities: 1. Provide a rating at designated times 2. Provide reminders to attend check in and check out 3. Provide support and encouragement to the student 4. Provide updates to ClCO staff ClCO StaffResponsibilities: 1. Check students in and out 2. Provide ClCO cards and home report 3. Provide updates to classroom teachers ClCO Staff Teacher Student APPENDIX I DAILY POINT CARD 103 104 CICO Record Name: _ Date: _ 1 h d .2 OK3 =great = = ar time Safe Responsible Respectful Check In 3 2 1 3 2 1 3 2 1 Morning: 3 2 1 3 2 1 3 2 1 Lunch 3 2 1 3 2 1 3 2 1 Afternoon: 3 2 1 3 2 1 3 2 1 Check Out 3 2 1 3 2 1 3 2 1 Today's goal Today's total points Comments: APPENDIX] CHECK-IN/CHECK-OUT HOME REPORT 105 106 crco Home Report Name: _ Date: _ My goal today is: _ r met my goal today One thing r did really well today was: Something r will work on tomorrow is: Comments: Parent/Guardian Signature: Comments: ___ r had a hard day 107 REFERENCES Burkwist, B. J., Mabee, W. S., & McLaughlin, T. F. (1987). The effect of a daily report card system on inappropriate classroom verbalizations with a junior high school learning-disabled student. Techniques, 3(4), 265-272. Chafouleas, S. M., McDougal, J. L., Riley-Tillman C.T., Panahon, C. J., & Hilt, A. M. (2005). What do daily behavior report cards (DBRC's) measure? An initial comparison ofDBRC's with direct observation for off-Task behavior. Psychology in the Schools, 42(6), 669-676. Chafouleas, S. M., Riley-Tillman, C. T., McDougal, J. L. (2002). Good, bad, or in-between: How does the daily behavior report card rate? Psychology in the Schools, 39(2), 157-169. Crone, D. A., Homer, R. H., & Hawken, L. (2003). Responding to problem behavior in the school: The behavior education program. New York: The Guildford Press. Carr, E. G. (1977). The motivation of self-injurious behavior: A review of some hypotheses. Psychological Bulletin, 84, 800-816. Davies, D. E., & McLaughlin, T. F. (1989). Effects of a daily report card on disruptive behavior in primary students. B. C. Journal ofSpecial Education, 13, 173-181. Dougherty, E. H., & Dougherty, A. (1977). The daily report card: A simplified and flexible package for classroom behavior management. Psychology in the Schools, 14, 191-195. Dryfoos, J. (1990). Adolescents at Risk. New York: Oxford University Press. Fairbanks, S., Sugai, S., Guardino, D., & Lathrop M., (2007). Response to intervention: An evaluation of a classroom system of behavior support for second grade students. Exceptional Children, 73, 288-310. Fairchild, Thomas N. (1983). Effects ofa daily report card system on an eighth grader exhibiting behavioral and motivational problems, School Counselor, 31(1),83-86. 108 Filter, K., Benedict, E., McKenna, M., Homer, R.H., Todd, A. & Watson, J. (2007). Check in! Check out: A post-hoc evaluation of an efficient, secondary-level targeted intervention for reducing problem behaviors in schools. Education and Treatment ofChildren, 30(1), 69-84 .. Galloway, J., & Sheridan, S. M. (1994). Implementing scientific practices through case studies: Examples using home school interventions and consultation. Journal of School Psychology, 32(4), 385-413. Gordan, R S. (1983). An operational classification of disease prevention. Public Health Reports, 98, 107-109. Hawken, L. S. & Homer, R H. (2003). Evaluation of a targeted intervention within a schoolwide system of behavior support. Journal ofBehavioral Education, 12, 225-240. Hawken, L. S. & Hess, R. S. (2006). Special section: Changing practice, changing schools. School Psychology Quarterly, 21, 91-211. Homer, R, Carr, E., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single subject research to identify evidence-based practices in special education, Exceptional Children, 71, 165-179. Homer, R., Todd, A., & Dickey, C. (2005) Check-in!Check-out Self-Assessment. Homer, R, Salantine, S., & Albin, R (2003). Self Assessment of Contextual Fit in Schools. Homer, R. H., Sugai, G., Todd, A.W., & Lewis-Palmer, T. (2005). School-wide Positive Behavior Support: An alternative approach to discipline in schools. In L. Bambara & L. Kern (Eds.) Positive Behavior Support. New York: Guilford Press. Homer, R H. (2000). Positive behavior support. Focus on Autism and Developmental And Other Developmental Disabilities, 15,97-105. Homer, R H., Todd, A. W., Lewis-Palmer, T., Irvin, L. K., Sugai, G., & Boland, J. B. (2004). The School-Wide Evaluation Tool (SET): A Research Instrument for Assessing School-Wide Positive Behavior Support. Journal ofPositive Behaviorallnterventions, 6 (1), 3-12. Kauffman, J. M., & Brigham, F. J. (2000). Zero tolerance and bad judgment in working with students with emotional or behavioral disorders. Behavioral Disorders, 25, 277-279. 109 Lewis, 1. J., & Sugai, G. (1999). Effective behavior support: A systems approach to proactive school-wide management. Effective School Practices, 17, 47-53. March, R E. & Homer, R H. (2002). Feasibility of contributions of functional behavioral assessment in schools. Journal ofEmotional and Behavioral Disorders, 10, 158- 171. March, R, Homer, R, Lewis-Palmer, T., Brown, Crone, D., Todd, A., & Carr, E. (2000) Functional Assessment Checklist for Teachers and Staff (FACTS). Mayer, G. R, & Butterworth, T. (1979). A preventative approach to school violence and vandalism: An experimental study. Personnel and Guidance Journal, 57, 436-441. Mayer, G. R (1995). Preventing antisocial behavior in the schools. Journal ofApplied Behavior Analysis, 28, 467-478. McIntosh, K., Chard, D., Boland, J., and Homer, R (2006). Demonstration of combined efforts in school-wide academic and behavioral systems and incidence of reading and behavior challenges in early elementary grades. Journal ofPositive Behavior Interventions, 8(3), 146-153. McCord, J. (Ed.). (1995). Coercion andpunishment in long-term perspective. New York: Cambridge University Press. Metzler, C., Biglan, A., & Rusby, J., (2001). Evaluation of a comprehensive behavior management program to improve school wide positive behavior support. Education and Treatment ofChildren, 24(4), 448-479. Nelson, J. R, & Carr, B.A., (1996). The Think Time strategy for schools: Bringing order to the classroom. Longmont, CO: Sopris West. Nelson, J.R, Martella, R. M. & Marchand-Martella, N. (2002). Maximizing student learning: The effects of a comprehensive school-based program for preventing problem behaviors. Journal ofEmotional and Behavioral Disorders 10(3), 136- 148. No Child Left Behind Act of2001, Pub. L. No. 107-110, 115 Stat. 1425 (2002). Rusch, F. R. & Kazdin, A. E. (1981). Toward a methodology ofwithdrawal designs for the assessment of response maintenance. Journal ofApplied Behavior Analysis, 14, 131-140. 110 Scott, T. M., & Eber, L. (2003). Functional assessment and wraparound as systematic school processes: Primary, secondary, and tertiary systems examples. Journal of Positive Behavior Interventions, 5, 131-143. Schumaker, J. B., Hovell, M. F. & Sherman, J. A. (1977). An analysis of daily report cards and parent-managed privileges in the improvement of adolescents' classroom performance. Journal ofApplied Behavior Analysis, 10(3), 449-464. Skiba, R. J. (2002). Special education and school discipline: A precarious balance. Behavioral Disorders, 27(2), 81-97. Skiba, R. J., & Peterson, R. L. (2000). School discipline at a crossroads: From zero -tolerance to early response. Exceptional Children, 66(3), 335-356. Sugai, G., Horner, R., & Sprague, J. (1999). Functional assessment-based behavior support planning: Research-to-practice-research. Behavior Disorders, 24, 223- 227. Sugai, G. Lewis-Palmer, T., Todd, A. & Homer, R. (2002). School-wide Evaluation Tool: Version 2.0. Sugai, G., & Horner, R.H. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy, 24, 23-50. Struckoff, P. M.; McLaughlin, T.F. & Bialozor, R.C. (1987). The effects of a daily report card system in increasing homework completion and accuracy in a special education setting. Techniques, 3(1), 19-26. Sugai, G., Lewis-Palmer, T., & Hagan, S. (1998). Using functional assessments to develop behavior support plans. Preventing School Failure, 43, 6-13. Todd, A. W., Homer, R. H., Sugai, G., & Sprague, J. R. (1999). Effective Behavior Support: Strengthening School-Wide Systems Through a Team-based Approach. Effective School Practices, 17(4),23-37. Todd, A.; Homer, R., Sugai, G., & Colvin, G. (1999). Individualizing School-wide Discipline for Students with Chronic Problem Behaviors: A Two Level Team Approach. Effective School Practices, 17(4), 72-82. Todd, A., Campbell, A., Meyer, G. & Horner, R. (2008). Evaluation of a targeted group intervention in elementary students: The Check-infCheck-out Program. Journal ofPositive Behavior Interventions, 10 (1),46-55. 111 Walker, H. M., Homer, R. H., Sugai, G., Bullis, M., Sprague, J. R., Bricker, D., & Kaufman, M. J. (1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal ofEmotional and Behavioral Disorders, 4, 194-209.