A PILOT FEASIBILITY AND EFFECTIVENESS TRIAL OF THE FAMILY CHECK-UP PARENTING INTERVENTION WITH SPANISH PREADOLESCENTS AND THEIR FAMILIES: A CULTURAL ADAPTATION AND FEASIBILITY STUDY TO ENHANCE EVIDENCE-BASED INTERVENTION RESEARCH IN SPAIN by KATHRYN L. MARGOLIS A DISSERTATION Presented to the Department of Counseling Psychology and Human Services and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Philosophy June 2013 ii DISSERTATION APPROVAL PAGE Student: Kathryn L. Margolis Title: A Pilot Feasibility and Effectiveness Trial of the Family Check-Up Parenting Intervention with Spanish Preadolescents and Their Families: A Cultural Adaptation and Feasibility Study to Enhance Evidence-Based Intervention Research in Spain This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Philosophy degree in the Department of Counseling Psychology and Human Services by: Elizabeth Stormshak, PhD Chairperson Benedict McWhirter, PhD Member Laura Lee McIntyre, PhD Member Philip Fisher, PhD Outside Member and Kimberly Andrews Espy Vice President for Research & Innovation; Dean of the Graduate School Original approval signatures are on file with the University of Oregon Graduate School. Degree awarded June 2013 iii  2013 Kathryn Leah Margolis iv DISSERTATION ABSTRACT Kathryn L. Margolis Doctor of Philosophy Department of Counseling Psychology and Human Services June 2013 Title: A Pilot Feasibility and Effectiveness Trial of the Family Check-Up Parenting Intervention with Spanish Preadolescents and Their Families: A Cultural Adaptation and Feasibility Study to Enhance Evidence-Based Intervention Research in Spain There is strong research support for the effectiveness and feasibility of family- centered, evidence-based programs (EBPs) to prevent the developmental and negative effects of youth problem behaviors. Despite this support, there is a relatively low rate of disseminating existing EBPs to diverse nations and cultures, and there is even less research being conducted to evaluate the existing efforts towards dissemination and uptake. Youth problem behavior prevention is a burgeoning area of psychological study in Spain, yet Spanish psychologists do not currently utilize evidence-based prevention approaches. This study aimed to fill gaps in the research related to understanding best practices in disseminating and evaluating the dissemination of EBPs to international settings and to evaluating the uptake of a family-centered EBPs for use in Spain to prevent problem behavior in adolescence and adulthood. The current study utilized a pretest/posttest with a follow-up, randomized control design to conduct a pilot feasibility and effectiveness trial of a family-centered EBP proven to effectively prevent problem behavior during adolescence. Seventeen pre-adolescents (ages 9-12) and their parents from the Seville metropolitan area in v Spain were randomly assigned to receive the Family Check-Up intervention (FCU) or waitlist-control condition. This study used a multimodal, multi-agent approach to (1) examine intervention feasibility and uptake, (2) measure trends in youth adjustment and family management practices in the study sample, (3) examine differences in youth behavior and internalizing problems, and positive parenting, limit setting, and monitoring based on intervention group assignment, and (4) measure motivation to change based on random assignment to the intervention condition. Results from mixed effects repeated analysis of variance analyses indicated that the intervention group made significant improvements in conduct and internalizing problems and in parental limit setting, positive parenting, and family problem solving. These quantitative findings coupled with confirmatory qualitative themes suggest that the intervention was both effective in reducing youth adjustment problems and enhancing parenting skills and feasible when applied within the Spanish cultural context. Implications of culturally sensitive, community-based methods of intervention dissemination are discussed. vi CURRICULUM VITAE NAME OF AUTHOR: Kathryn L. Margolis GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene Indiana University, Bloomington University of Seville, Seville, Spain DEGREES AWARDED: Doctor of Philosophy, Counseling Psychology, 2013, University of Oregon Master of Science, Counseling, Family, & Human Services, 2010, University of Oregon Bachelor of Arts, Psychology, 2004, Indiana University Bachelor of Arts, Communication and Culture, 2004, Indiana University AREAS OF SPECIAL INTEREST: Counseling Psychology Prevention psychology, International Psychology, & Cultural Psychology Clinical Psychology Child Development/Disabilities Pediatric Psychology PROFESSIONAL EXPERIENCE: Pre-Doctoral Psychology Intern, University of Southern California, University Center of Excellence in Developmental Disabilities, Children’s Hospital Los Angeles, 7/2012-6/2013 Fulbright U.S. Student Fellow, Fulbright Commission in Spain, University of Seville, 9/2011-6/2012 Bilingual Parent Consultant, Early Steps Project, Child & Family Center, University of Oregon, 2010-2011 Assessment Extern, Child Development & Rehabilitation Center, Oregon Health & Science University, Eugene, Oregon, 2010-2011 vii Instructor/Graduate Teaching Fellow, Service Learning Program, University of Oregon, 2009-2011 Bilingual Practicum Intern Therapist, Child & Family Center Clinic, University of Oregon, 9/2009-8/2011 Bilingual Group Interventionist, ALAS Latino Drop-Out Prevention Trial, University of Oregon, 2009-2010 Practicum Intern Therapist, University Counseling & Testing Center, University of Oregon, 2008-2009 University Supervisor/Graduate Teaching Fellow, Family & Human Services Program, College of Education, University of Oregon, 2007-2009 Teaching Assistant/Graduate Teaching Fellow, Family & Human Services Program, College of Education, University of Oregon, 2007 GRANTS, AWARDS, AND HONORS: Fulbright U.S. Student Grant, Internationalizing Children’s Psychological Services to Strengthen our Global community, University of Oregon and University of Seville, 2011-12 Ethnic and Racial Diversity Community Research Award, Doctoral Dissertation Research, Department of Counseling Psychology and Human Services, University of Oregon, 2012 Graduate Service Award, Graduate School of the University of Oregon, 2011 Clare Wilkins Chamberlain Memorial Research/Doctoral Research Award, Doctoral Dissertation Research, College of Education, University of Oregon, 2011 Graduate School Doctoral Award, Doctoral Dissertation Research, Graduate School of the University of Oregon, 2011 Jean Twohig Scholarship, College of Education, University of Oregon, 2011 General University Scholarship, University of Oregon, 2011 viii Research Travel Award for Biennial Conference of the National Latino Psychological Association, ALAS: An After-School Program for Spanish Speaking Latina/o Students. Counseling Psychology & Human Services Department, University of Oregon, 2010 Ethnic Diversity Research Travel Award for the Annual Convention of the American Psychological Association, The Circle of Care: Examining Collaborative Caretaking in Adolescent Development, Counseling Psychology Program, University of Oregon, 2010 Stout Scholarship/General University Scholarship, University of Oregon, 2010 Mary Russell Scholarship/General University Scholarship, University of Oregon, 2009 Student Immersion Travel Award to Guatemala for the Society of Interamerican Psychology Immersion Program and Conference, Counseling Psychology & Human Services Program, 2009 Excellence in Campus Leadership Award, Advocacy Coordinating Teach for the American Psychological Association of Graduate Students (APAGS), 2008 Culbertson Scholarship, University of Oregon, 2008 Florence Wolfard Scholarship, College of Education of the University of Oregon, 2008 Travel Award for the International Conference of Counseling Psychology, Office of International Affairs, University of Oregon, 2008 PUBLICATIONS: Margolis, K., Fosco, G., & Stormshak, E. A. (under review – second review). Circle of Care: Extending Beyond Primary Caregivers to Examine Collaborative Caretaking in Adolescent Development. Journal of Family Issues. Margolis, K., & Stormshak, E. A. (under review – first review). The Family Check-Up: A Family Centered Strategy to Strengthen Youth Outcomes and Prevent Substance Use. In J. A. García del Castillo, & P. C. Días (Eds.), Studies About Alcohol, Tobacco and Other Drugs: Research Contributes to Practice. Portugal: Edições Pedago. ix Stormshak, E. A., Margolis, K., Huang, C.Y., & Dishion, T.J. (2012). Implementation of a family-centered, school-based intervention to prevent student academic and behavioral problems. In Kelly, B., & Perkins, D. F., (Eds.), Handbook of Implementation Science for Psychology in Education (pp. 264-276). New York: Cambridge University Press. Margolis, K. (2012). International research: The challenges and benefits for graduate students. International Section Newsletter for the International Section of Counseling Psychology Division 17, American Psychological Association, Winter Issue, 6-7. Margolis, K. (2011). International Practicum Supervision Manual. Unpublished manuscript available from Family and Human Services Program, Counseling Psychology and Human Services Department, University of Oregon. Santana Pellicier, A., Margolis, K., Harley, E., Medina, C., & Camacho, C. (2010). The Guatemalan Immersion Program: Five Perspectives. The Multicultural Counselor Newsletter 1(1), 15-17. Margolis, K., Ramzy, L., Wang, J., & Chronister, K. M. (2008). Violence Against Immigrant and Refugee Women. Fact Sheet: American Psychological Association Public Interest Government Relations Office. x ACKNOLWEDGEMENTS I would like to express my gratitude to my advisor and dissertation chairperson, Dr. Beth Stormshak, for implementing the perfect mentoring balance between teaching and fostering of independence. Thank you for your positive attitude, innovativeness, and supporting me to develop my own unique path. I also thank the members of my dissertation committee, Dr. Benedict McWhirter, Dr. Laura Lee McIntyre, and Dr. Philip Fisher, for dedicating your time, wisdom, and expertise to strengthen my study. I acknowledge the contribution of the University of Oregon Child & Family Center, the Department of Developmental and Educational Psychology at the University of Seville, and the local schools that helped to develop the project. I wish to express my sincere appreciation for Dr. Lucia Antolín Suárez, Diana Ma Pascual García, and Rosa Ma Estévez Campos for their thoughtfulness, dedication, and hard work in executing this project. Without the collaboration and contribution of each of these individuals and organizations, this study would not be possible. This investigation was supported in part by a Fulbright U.S. Student Grant, Number 34110762, from the Spanish Fulbright Commission and U.S. Department of State, and by a Doctoral Research Award from the Clare Wilkins Chamberlain Memorial fund from the College of Education and Graduate School at the University of Oregon. xi I would like to acknowledge my family and friends, whose unwavering belief in my ability to realize any task that I put my mind and heart into, helped me to complete this three-year project. I am grateful to my partner and my friends for motivating me every step of the way and for helping me to maintain balance. Thank you to my grandparents for opening my mind to the world by teaching me about the importance of cross-cultural understanding and reciprocity. Finally and most importantly, this dissertation is dedicated to my parents, for their positive nature, belief in my perseverance and success, and for instilling in me the value that education is the greatest investment one can make in herself. xii TABLE OF CONTENTS Chapter Page I. INTRODUCTION .......................................................................................................... 1 Youth Problem Behavior ........................................................................................... 2 Youth Problem Behavior in Spain ............................................................................. 4 Prevention of Youth Problem Behavior ..................................................................... 8 Prevention of Problem Behaviors in Spain ................................................................ 9 Dissemination of Evidence-Based Prevention Practices ......................................... 12 The Family Check-Up: Model Overview ................................................................ 14 Spain and the Family-Check-Up .............................................................................. 18 The Present Study .................................................................................................... 19 II. METHOD .................................................................................................................... 22 Participants and Sampling........................................................................................ 22 Intervention Fidelity................................................................................................. 25 Intervention Procedures ........................................................................................... 28 Addressing Attrition................................................................................................. 32 Data Collection ........................................................................................................ 33 Measures .................................................................................................................. 34 Data Analysis ........................................................................................................... 43 III. RESULTS ................................................................................................................... 45 Cultural Embedding of the Family Check-Up Model.............................................. 45 Trends in Youth Adjustment, Family Background and Support, and Parenting ..... 57 Intervention Effectiveness on Youth Adjustment .................................................... 58 xiii Chapter Page Intervention Effectiveness on Family Management Practices ................................. 61 Intervention Effectiveness on Parent Motivation to Change ................................... 65 Feasibility of the Family Check-Up in Spain .......................................................... 67 IV. DISCUSSION ............................................................................................................ 78 Embedding the Family Check-Up in the Spanish Culture ....................................... 78 The Family Check-Up and Spanish Youth Adjustment .......................................... 82 The Family Check-Up and Family Management in Spain ...................................... 84 Motivation to Change .............................................................................................. 88 Feasibility & Uptake of the FCU in Spain ............................................................... 90 Limitations and Strengths ........................................................................................ 92 Recommendations for Future Research, Education, and Policy .............................. 94 APPENDICES ................................................................................................................... 97 A. TABLES............................................................................................................. 97 B. FIGURES ......................................................................................................... 117 C. STUDY MATERIALS ..................................................................................... 121 REFERENCES CITED ................................................................................................... 122 xiv LIST OF TABLES Table Page 1. Descriptive Statistics for Youth Adjustment and Demographic Variables ................. 98 2. Measures of Central Tendency for Parent Report on Youth Adjustment Variables .. 99 3. Descriptive Statistics for Parent Report of Family History and Background Support .................................................................................................. 101 4. Descriptive Statistics for Parent and Therapist Report of Family Management Variables .................................................................................. 103 5. Mixed Effect Repeated Measures ANOVA to Examine Intervention Effects .......... 108 6. Mean Differences Across Time Points for Intervention and Control Groups ........... 112 7. Descriptive Statistics for Treatment Fidelity and Program Evaluation ..................... 115 xv LIST OF FIGURES Figure Page 1. Trend-level interaction for parent report of Negative Limit Setting across three time-points: pretest, posttest, and follow-up. ........................................ 118 2. Significant interaction for therapist observation of Parental Limit Setting from pretest to posttest. ....................................................................... 119 3. Trend-level interaction for therapist observation of Family Problem Solving skills from pretest to posttest. ........................................................ 120 1 CHAPTER I INTRODUCTION There is strong research support for the effectiveness and feasibility of family- centered, evidence-based prevention programs (EBPs) in reducing the negative effects of youth problem behaviors (Mrazek & Haggerty, 1994). However, there is a relatively low international dissemination rate of existing EBPs, and there is even less empirical evaluation of the EBPs that have been cross-culturally disseminated (Sanders, 2010). As a result, our understanding of the cultural factors that make interventions effective in preventing youth problem behavior is incomplete and we are limited in the benefit of exchanging health-related information across cultures. This limitation substantially decreases the likelihood that at-risk children will receive necessary and effective services that may prevent the long-term, negative outcomes of youth problem behavior. Families, schools, and communities all over the world are affected by youth problem behavior, such as aggression, stealing, substance abuse, and bullying. The risk trajectory of problem behavior is developmental, so that risk factors accumulate over time causing the expression of these behaviors to become exacerbated as children grow (Dishion & Patterson, 1997). Consequently, when problem behaviors are left untreated in childhood, individuals become substantially more at risk for developing chronic mental health disorders in adulthood (Hinshaw & Lee, 2003). In Spain, youth problem behavior and its effective prevention is a burgeoning area of psychological study (Vera, Ezpeleta, Granero, & de la Osa, 2010). Findings indicate a high prevalence of problem behavior among Spanish youth and a developmental etiology that resembles youth all over the world (Vera, et al., 2010). 2 Despite the high prevalence of problem behavior in this context, Spain’s prevention efforts against problem behavior is relatively low. Like many countries, prevention psychology in Spain is affected by the overall low rate of disseminating and evaluating EBPs. As a result, Spain has a scarcity of available evidence-based, family-centered programs. Recently, Spanish psychologists and policy-makers have responded to the lack of effective EBPs for preventing problem behavior by prioritizing research that will support the uptake of such programs (Garrido, et al., 2006). However, without sufficient literature in this area, the process of locating effective and culturally-sensitive EBPs can be costly and inefficient. In an effort to bridge the gap between the science of intervention development and the practice of effective intervention implementation into Spanish communities-in-need, the current project aims to encourage international dissemination of EBPs and increase further evaluation of the programs that have already been disseminated. In collaboration with Spanish psychologists, the proposed research uses a longitudinal, pretest/posttest with follow-up, randomized control group design to examine the feasibility and effectiveness of the Family Check-Up (Dishion & Stormshak, 2007) for use with families of pre-adolescents from Seville, Spain. Youth Problem Behavior Behavior problems in early childhood are associated with more serious conduct problems in preadolescence and adolescence, which if not treated can develop into delinquent behavior in adulthood. These behavior problems can result in chronic arrests or other mental health problems that impair typical functioning in adulthood (Reid & Eddy, 1997). While only about one to ten percent of youth in the general population are 3 estimated to meet diagnostic criteria for conduct disorder, many children exhibit problem behaviors that impact their own development and other related family functioning (Breslau, Saito, Tancredi, Nock, & Gilman, 2012; Hinshaw & Anderson, 2003). The etiology of problem behavior is developmental and consists of interacting biological, psychological, and social factors that become exacerbated as risk increases over time (Dodge & Petit, 2003). This research suggests that if problem behaviors and risk factors are targeted at a young age, the trajectory of the symptoms developing into significant chronic mental health disorders can be reduced substantially or even reversed (Reid & Eddy, 1997). Theories of child development help explain how biopsychosocial processes exacerbate early childhood behavioral problems into chronic mental health concerns later in life. For example, Patterson’s Coercion Theory (1982) explains the social processes that are involved in the risk trajectory of developing problem behavior. In short, when coercive social processes manifest within family systems, youth learn to employ similar processes, such as escape conditioning, as a way of interacting with their world. If no intervention occurs that helps children learn positive social skills and strategies for problem solving, it is possible for children to generalize their behaviors into an interpersonal style that is employed across multiple environmental settings, like home, school, and peer circles (Patterson, 1993). When these coercive processes develops into an interpersonal style, youth become at higher risk for failing to develop prosocial behaviors and skills, which can lead to a risky developmental trajectory of peer rejection, affiliation with deviant peers, and eventually, the expression of problematic and unhealthy behaviors, like substance abuse, aggression, and stealing 4 (Reid & Eddy, 1997). Consequently, when problem behaviors are left untreated in childhood and adolescence, individuals become substantially more at risk for developing chronic mental health disorders in adulthood (Hinshaw & Lee, 2003). Youth Problem Behavior in Spain Youth problem behavior and its effective prevention is a burgeoning area of study within the area of developmental psychology in Spain. Conduct problems are among the most commonly diagnosed problems among youth in Spain, and researchers estimate that about 20% of youth display symptoms consistent with DSM-IV criteria of conduct disorder (OMS, 1993; Navarro-Pardo, Meléndez Moral, Sales Galán, & Sancerni Beitia, 2012) As a result, Spanish psychologists along with the broader public health sector have recently prioritized etiological research about conduct-related disorders due to high prevalence and negative impact on society (Vera, Ezpeleta, Granero, & de la Osa, 2010). These etiological studies help intervention researchers evaluate the appropriateness of adapting and applying existing theory-based interventions to support the developing prevention efforts in Spain. Etiological research produced in Spain that compares antisocial behavior etiology across cultures reveals that the developmental risk trajectory and the expression of antisocial behaviors is similar to research that has been conducted elsewhere in the world. For example, in a large, clinical sample of Spanish adolescents, similarities in the expression of antisocial behaviors between Spanish and U.S. youth were identified (Vera, et al., 2010). Findings revealed that expression of antisocial behaviors did not significantly differ by age or sex, but that similar to existing U.S. research on antisocial behavior, the presentation of these antisocial behaviors did 5 contain differences (Silverthorn & Frick, 1999). For example, the highest prevalence of symptoms of conduct disorder existed among boys and adolescents ages 13-17, suggesting that boys tended to exhibit more overt, externalizing behaviors. On the other hand, females tended to endorse more covert behaviors than boys, such as lying, but that the antisocial behaviors were still present. Additionally, findings revealed sex variations in the links between certain symptoms and symptoms of psychopathology. Aggression constitutes an aspect of antisocial behavior that is at the forefront of Spanish literature on antisocial behavior. Aggression across contexts is both a problem behavior and a critical risk factor in the developmental trajectory of antisocial behavior (Tolan, Guerra, & Kendall, 1995; Fraser, 1996). A large study of Spanish adolescents examined the impact of the home and school environment on aggression (López, Pérez, Ochoa, & Ruiz, 2008). Findings revealed that the home environment was a stronger predictor of aggression for females than males. Additionally, López and colleagues’ findings suggested that a more positive home environment was linked with lower rates of expression of aggressive behaviors for females, indicating that a positive home environment may serve more strongly as a protective factor for females. On the other hand, for adolescent males in the sample, youth perception of the school environment was more strongly linked to aggression, with a more positive environment being a protective factor against aggression. Within the existing literature in Spain, researchers asserted the salience of the relational aspects of the home environment for preventing female aggression. For example, findings about the environmental influence on aggression were linked to theories stating that the relational aspects of parenting and the family environment may 6 be more salient for Spanish females than for males - explaining the stronger link between youth self-reports of a negative home environment and incidences of aggression (López, et al., 2008). Additionally, the strong link between male aggression and classroom environment may be explained by evidence demonstrating the stronger impact of problems of social acceptance and peer integration on males’ development of aggressive behavior (López, et al., 2008; McDougall, Hymal, Vaillancourt, & Mercer, 2001). Findings from this study are consistent with social developmental research that links both the quality of the home and school contexts with the expression of problem behaviors (Dishion & Stormshak, 2007). Specifically, factors discussed in López and colleagues’ study (2008), such as parent-child relationships, family cohesion, student- teacher relationships, and peer rejection are critical points of intervention for preventing youth problem behaviors (Dishion & Patterson, 1997; Dishion & Owen, 2002; Hughes, Cavell, & Wilson, 2001; Fraser, 1996). The foundational influence of the family environment on youth problem behavior is well established within the Spanish research base, but peer groups have also been determined to serve as catalysts in the expression of antisocial behaviors outside of the home (Dishion & Andrews, 1995, Dishion & Owen, 2002; Fraser, 1996; Stormshak, Comeau, & Shepard, 2004). In fact, victimization and rejection by peers often lead youth to affiliate with deviant peer groups – a critical factor contributing to antisocial behavior (Dishion, French & Patterson, 1995; Dishion & Andrews, 1995). In a study of 1319 Spanish adolescents, ages 16-19, researchers found important links between reports of peer victimization and the family and school environments (Cava, Musitu, Buelga, & Murgui, 2010). Findings indicated that more positive perceptions of both the 7 school and family environments were linked to fewer reports of relational victimization among participants. Additionally, both male and female participants reported significant associations between peer victimization and reports of loneliness and sociometric status (peer nominations of being liked or disliked), and males endorsed a link between lower self-esteem and higher victimization. These findings support the connection between youth well-being and becoming the victim of bullying or peer rejection, which is consistent with U.S. and Spanish research linking problem behaviors with other aspects of youth adjustment (Hinshaw & Lee, 2003; Vera, et al., 2010). These findings also further establish the ability of the home and school environments to protect against or exacerbate incidences of problem behavior among Spanish youth (Cava, et al., 2010). This examination of the existing research about youth problem behavior in Spain indicates that the etiology and risk factors for problems behaviors are consistent with U.S. findings from available research in this area. Findings indicated that for clinical populations, the incidence of problem behaviors is similar across genders though females tend to express antisocial tendencies later than males (Silverthorn & Frick, 1999). The highest frequency of expression of antisocial behaviors in clinical and normative samples occurs between ages 13-17(Navarro-Pardo, et al., 2012; Vera, et al., 2010). However, rates of conduct problems among the general sample appear to be higher in Spanish youth (~20%) than in U.S. youth (1-10%), with the peak of problems occurring in early adolescence (Navarro-Pardo, et al., 2012; OMS, 1993). In summary, evidence suggests that environmental factors, such as peers, family, and the school setting contribute strongly to the development of problem behaviors. Consistent with findings in the U.S. related to peer rejection and deviant peer affiliation, 8 studies indicated that problems between peers are correlated with factors of youth adjustment, such as self-esteem and feelings of isolation (Fraser, 1996; Cava, et al., 2010). Finally, evidence suggests that the quality of family and school settings have an important impact on problem behaviors, such as peer victimization and aggressive behaviors for both males and females (Vera, et al., 2010; López, et al., 2008). Overall, the high prevalence of early adolescent youth who meet criteria for conduct problems suggests that prevention studies for problem behaviors in Spanish youth should target pre-adolescents across multiple domains in the youth ecology. Prevention of Youth Problem Behavior Prevention interventions are a widely accepted form of ameliorating the negative effects of problem behaviors in youth due to the cost/benefit effect of prevention programs (Mrazek & Haggerty, 1994). Family-centered, parenting interventions that target problem behaviors across the family and other social contexts are accepted as one of the most effective approaches for preventing these behaviors in youth (Dishion & Stormshak, 2007; Henggeler, Schoenwald, Bordouin, Rowland, & Cunningham, 1998; Johnson, Kent, & Leather, 2005; Marcus, Swanson, & Vollmer, 2001; Nowak & Heinrichs, 2008; Stallard, 2006; Stormshak, Connell, Véronneau, Myers, Dishion, Kavanagh, & Caruthers, in press; Webster-Stratton, 1992; Welsh & Farrington, 2006). Family-centered interventions are successful by helping parents to manipulate social influences on problem behavior by creating environmental stability and reducing coercive dynamics in the family system (Patterson, 1982; Dishion & Patterson, 1997). Working with parents to reduce coercive cycles and enhance family management practices (i.e. parental monitoring, proactive parenting, limit setting, problem-solving) 9 has been shown to promote healthy youth adjustment, to enhance parent and family well-being, and to reduce youth problem behavior and family stress (Dishion & Stormshak, 2007). By strengthening parenting and family management practices, the influence of biological (e.g. temperament) and psychological factors (e.g. parent psychopathology or youth Attention-Deficit Hyperactivity Disorder) on problem behavior is reduced, and along with it the risk of chronic behavioral problems (Dodge & Petit, 2003; Ackerman, Brown, & Izard, 2004; Raver, Gershoff, & Aber, 2007). Prevention of Problem Behaviors in Spain Psychologists in Spain agree that empirically validated prevention programs are the most promising mechanism for reducing the harmful effects of problem behaviors (Garrido, Farrington, & Welsh, 2006). Yet, with the exception of progressive prevention services, such as the Triple P – Positive Parenting Program (a parenting program developed in Australia that has been adapted, tested, and disseminated in several nations and cultures), few randomized studies have been conducted that test interventions outside of their country of origin (Sanders, Markie-Dadds, & Turner, 2003; Matsumoto, Sofronoff, & Sanders, 2010; Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009; Bodenmann, Cina, Ledermann, & Sanders, 2008; Beelmann & Lösel, 2006). As a result, there is a scarcity of evidence-based family prevention programs in Spain and most other nations (Suelves & Sánchez-Turet, 2008; Sanders & Murphy-Brennan, 2010). Similar to many nations in the global community that do not yet have access to EBPs, the challenge in Spain lies in translating evidence-based programs to the public sector in order to support their uptake (Sanders, 2010; Garrido, et al., 2006). However, 10 psychologists in Spain have an advantage over many other nations who may encounter political, cultural, and other barriers to introducing external services. The current economic and social situation in Spain is such that international collaborative efforts among psychologists are welcomed by both the university system and the government. This climate is due in part to Spain’s relatively recent shift to a modern democracy, which has caused Spanish policy makers to reevaluate their goals to prioritize empirical efforts that can improve the public health situation (Garrido, et al., 2006). As a result, Spanish psychologists have been charged to more actively seek out culturally-congruent EBPs, conduct empirical prevention trials, and evaluate the feasibility of program uptake within its nationalized healthcare system (Garrido, et al., 2006). This makes Spain an excellent setting for acquiring and testing EBPs, which would have a positive impact on the Spanish the public healthcare system, and creates opportunity for prevention researchers all over the world to learn from cross-cultural collaborations – especially in the area of intervention dissemination. There research about the use of evidence-based programs within Spanish psychology is scarce. The minimal literature suggests that Spain has a positive position on evidence-based programs for use in the prevention of youth problem behavior, but most of this literature draws largely from meta-analyses of EBPs developed elsewhere in the global community (Beelmann & Lösel, 2006). The gap in the literature may be due in part the structure of the educational system in psychologist training in Spain. In order for a psychologist to obtain a degree in clinical psychology, s/he must elect to pursue exclusively clinical training. On the other hand, a psychologist interested in research must elect to abandon clinical training and pursue only research education. 11 This leads to highly trained clinicians or researchers, and while evidence-based programs are respected within Spanish psychology, there are virtually no trained scientist-practitioners – a necessary educational background for conducting intervention research. As such, there is a dearth of longitudinal, empirical studies examining treatment effectiveness with samples of clinical families in Spain. However, findings from one longitudinal study of treatment effectiveness with Spanish families revealed that parent- training programs yielded better outcomes than traditional therapy (de Paúl & Arruabarrena, 2003). The support for family-centered programs put forth by this study is consistent with trends in Spanish journal publications that draw from international research to encourage the use of family-centered programs and those promoting contingencies for prosocial skills (Rodríguez & Paíno, 1994; Beelmann & Lösel, 2006). The literature indicating that Spain favors family-centered interventions supports that this type of EBP would be welcomed in Spain, and would likely result in positive uptake and feasibility. A trend towards strength-based and contextually-sensitive services has emerged within the Spanish psychology literature. Consistent with prevention research that supports an ecologically based, family-centered framework, there is a movement within Spanish literature encouraging researchers to consider the unique components that will motivate youth to engage in behavioral changes. Specifically, the extant literature supports interventions that utilize a strengths-based approach for youth, or one considering the needs, goals, and priorities of the target youth (Rodríguez & Paíno, 1994). Additionally, interventionists are encouraged to consider the broader social 12 context in the etiology of problem behavior expression, with a particular focus on the impact of societal marginalization, through factors like poverty or other minority status (Rodríguez & Paíno, 1994). Finally, one study incorporated the concept of motivation into the study of problem behavior by examining adolescents’ motivations to engage in problem behaviors. Researchers determined that the motivations to engage in problem behaviors must be considered when constructing prevention efforts, as these behaviors have been linked with social recognition and antisocial goals (e.g. drug use) among Spanish teens (López-Romero & Romero, 2010). As such, EBPs with a strengths-based approach to targeting ecological factors and motivation to change are likely to have the most positive impact in Spain. In summary, Spanish policy makers and psychologists favor the adoption and uptake of EBPs as a method for preventing and treating problem behaviors. The current research in Spain is mostly focused on the etiology of problem behaviors, yet there is increasing support for the use of family-centered interventions with a strengths-based, motivational tone as an effective prevention method. Given the lack of EBPs and intervention research and the current sociopolitical climate in Spain that supports international collaborations towards effective prevention of youth problem behavior, Spain is a well-positioned setting for acquiring and testing family-centered preventative EBPs. Dissemination of Evidence-Based Prevention Practices Among the global community, Spanish psychologists are not alone in their recognition of the need for a broader dissemination of evidence-based practices. Despite rich evidence supporting family-centered prevention efforts in nations, such as the 13 United States and Australia, knowledge about empirically validated and effective services rarely extends beyond national borders (Sanders, 2010; Dishion & Stormshak, 2007; Henggeler & Borduin, 1990). Access to evidence-based parenting programs by nations across the world is low, and there is a great need for increased research in this area (Sanders & Murphy-Brennan, 2010). Consequently, psychology researchers worldwide spend valuable resources attempting to replicate services that already exist elsewhere in the global community, and families-in-need are rarely able to access services with proven research effectiveness (Sanders & Murphy-Brennan, 2010). In addition to the relatively low rate of disseminating existing EBPs to diverse nations and cultures, there is even less research being conducted to evaluate the dissemination and uptake that is taking place (Turner & Sanders, 2006). Without empirical evaluation of these dissemination programs, there is no way to know the impact that EBPs are having, not only on the clientele, but also on stakeholders like service providers and government officials that have the power to support the uptake of important programs into policy (Turner & Sanders, 2006). Replicating interventions without using an existing framework and empirical evaluation methods is not only a costly process for nations without available funds, but also fails to utilize valuable knowledge gained through dissemination trials that could be used more efficiently to enhance services for children all over the world. On the other hand, intervention research conducted with populations across cultures and nations can serve as an important source of information that would contribute to our current understanding of the theory and prevention of youth problem behavior (Venkatraman, Dishion, Kiesner, & Poulin, in press; Bernal, Jiménez-Chafey, 14 & Domenech Rodríguez, 2009). While the importance of this type of research is widely acknowledged in the field, relatively little research evidence has been documented to date (Venkatraman, et al., in press). Knowledge produced from embedding existing psychological interventions (e.g. assessments and treatments) to specific cultural and linguistic systems would facilitate ease of dissemination of effective interventions to psychologists from diverse cultural contexts, which would improve service quality for families from diverse ethnocultural groups (Bernal, et al., 2009). The research that has been conducted in the area of cultural adaptation of EBPs suggests that positive results in the form of recruitment, retention, and outcomes can be obtained by disseminating existing EBPs to various ethnocultural groups (Bernal, et al., 2009). To assist in culturally competent intervention dissemination, researchers have created models that provide frameworks through which to view the intervention context. Bernal and colleagues’ Ecological Validity Model (EVM) is one such model that has yielded positive results as a framework in the adaptation of interventions to various cultural groups (Bernal, Bonilla, & Bellido, 1995). The EVM provides eight dimensions (language, persons, metaphors, content, concepts, goals, methods, and context) for researchers to consider in their culturally sensitive delivery of evidence- based prevention practices. The Family Check-Up: Model Overview The Family Check-Up is a family-centered EBP that lends well to broad international dissemination because a flexible and tailored approach to family and cultural value systems are at the core of the model (Dishion & Stormshak, 2007). Furthermore, 20 years of longitudinal, federally-funded research with at-risk and 15 culturally diverse youth across early childhood and late adolescence has consistently found the FCU to be effective in reducing and preventing child psychological and behavioral problems, improving parenting skills and family wellbeing, and increasing parental motivation to participate in treatment (Dishion & Kavanagh, 2003; Stormshak, Dishion, Light & Yasui, 2005; Dishion & Stormshak, 2007). FCU Process. The FCU is a prevention approach that targets youth who are identified as at-risk by parents, teachers, or service professionals. The FCU comprises three steps: (1) an intake interview, (2) a multimodal and multiagent assessment of child and family wellbeing through self-report questionnaires and researcher observation that occurs in various contexts (e.g. school and home); and (3) a feedback session comprising provision of assessment results to the family and collaboration in setting goals with parents to improve youth and family wellbeing. By tailoring the feedback session (and particularly the goal-setting process) to fit family values and resources, the FCU is culturally adaptable and removes obstacles that traditionally cause at-risk families to drop out of treatment. Finally, by providing families with a “menu of options,” the FCU process strategically meets caregivers at their stage of change and uses motivational interviewing to support the change process (Prochaska & Norcross; Miller & Rollnick, 2002). As a result, caregivers who participate in the FCU tend to evidence higher motivation to change than those who do not participate in the intervention (Dishion & Stormshak, 2007). Empirically derived from Patterson’s Coercion Theory, the Family Check-Up intervention effectively targets the multiple ecological influences on problem behavior (Patterson, 1982; Loeber & Dishion, 1983; Dishion & Stormshak, 2007). Furthermore, 16 by grounding the intervention in the family context by supporting positive parenting and consistent family management practices (e.g. positive behavior support, limit setting, problem solving, parent-child relationship quality), the FCU process empowers the social agents who have the most formative influence on the lives of children and adolescents. Effectiveness of the Family Check-Up. Multiple randomized prevention trials have provided evidence that the FCU intervention is successful in targeting multiple dimensions that influence youth problem behavior. Intervention effectiveness has been found across varying youth risk levels, contexts, genders, and ages. The FCU has been found to effectively engage families with multiple risk factors that would typically serve as barriers to treatment access. Despite common belief that higher risk families would be less likely to engage in treatment, trials with the FCU model have consistently yielded positive results related to family engagement. Findings yielded model success in engaging the most at-risk families, such as those with single parent status or rated as highest risk by teachers (Connell, Dishion, & Deater- Deckard, 2006). The FCU has been successfully applied as a tool for preventing problem behaviors in samples of both pre-adolescent and adolescent youth. For example, the FCU has effectively reduced adolescent substance, which was mediated by increased parental monitoring (Dishion, Kavanagh, Schneigher, Nelson & Kaufman, 2002), absence or truancy from school (Stormshak, Connell, & Dishion, 2009), deviant peer affiliation (Dishion, Bullock, & Granic, 2002), and percentage of arrests (Connell, Dishion, Yasui, & Kavanagh, 2007). In addition to strong effects in adolescence, strong 17 intervention effects have been found with pre-adolescent youth who first received the model in sixth grade – providing additional support for the FCU as a preventive tool if used during pre- or early adolescence (Stormshak & Dishion, 2009). The FCU has also yielded strong support as a prevention tool of problem behavior in early childhood. In a randomized, longitudinal study of ethnically diverse participants in early childhood, the FCU’s brief, flexible, and tailored approach increased parents’ positive behavior support practices in a sample of 2-year-old children that were followed through age 4 (Dishion, Shaw, Connell, Gardner, Weaver, & Wilson, 2008). In the same study, the increase in positive behavior support practices was linked to reduction in growth of problem behaviors in early childhood. The FCU has also been successful in supporting school engagement and academic outcomes for middle- and high school-age youth. Specifically, engagement in the FCU helped to decrease depression and increase self-regulation in middle school- age youth (Stormshak, Fosco, & Dishion, in press). The increase in self-regulation was associated with better school engagement in this study – a crucial component supporting the transition from middle to high school (Stormshak, Fosco, & Dishion, in press). A second study demonstrated the positive impact of engagement in the FCU intervention on academic success (Stormshak, Connell, & Dishion, 2009). Specifically, high-risk youth who received the FCU intervention maintained their GPA from middle to high school, whereas high-risk youth in the control group evidenced a decline in GPA for the same transition. In the same study, the effect of engaging in the FCU also prevented a growth in absence rate for the intervention group as opposed to the control group who demonstrated a steady increase in absence rate (Stormshak, Connell & Dishion, 2009). 18 Cultural adaptability of the FCU. The cultural-adaptability of the FCU has been demonstrated through successful delivery to U.S. Spanish-speaking Latino families (Dishion, et al., 2008) and to various ethnocultural groups in the U.S. and abroad - most recently to multiple Swedish elementary schools through a federal grant aimed to reduce problem behavior. However, since psychological services for youth rely on local cultural practices of healing, family values, and child rearing, more research is needed to determine the effectiveness of the FCU in yet other cultures. Spain and the Family-Check-Up The movement within Spanish psychology to not only better understand the etiology of problem behavior, but also to seek out family-centered, evidence-based strategies to prevent these behaviors in youth makes Spain an excellent candidate for a dissemination trial of existing EBPs (de Paúl & Arruabarrena, 2003; Rodríguez & Paíno, 1994). The evidence supporting the effective uptake of the Family Check-Up (FCU) with culturally diverse, Spanish speaking families in the United States, coupled with the model’s successful uptake in international contexts, like Sweden, suggests that the FCU model may work well within the Spanish context. Furthermore, the goals and procedures of the FCU model fit well with recommendations of Spanish psychologists that interventions for preventing problem behaviors assume a family-centered approach that target a variety of risk factors across multiple contexts (Vazsonyi, Chen, Jenkins, Burcu, Torrente, & Sheu, 2010; Vera, et al., 2010; Cava, et al., 2010; López, et al., 2008). Finally, the cost-effectiveness and brief duration of the FCU along with the experience of over fifteen years of efficacy trials makes embedding the model and testing its uptake in the Spanish context a feasible task. 19 The Present Study The minimal research examining the cross-cultural effectiveness of evidence- based programs to prevent youth problem behavior limits our understanding of the factors that contribute to intervention effectiveness and that facilitate successful dissemination of these interventions to healthcare systems and at-risk children in need. While there is a large research base examining youth problem behavior prevention in the U.S., there is a dearth of literature related to (1) promoting successful dissemination and evaluation of EBPs in diverse cultural and linguistic contexts, (2) understanding the environmental influences on youth problem behavior in Spain, and (3) the effectiveness of evidence-based intervention programs to prevent youth problem behaviors in Spain. In an effort to fill these gaps, the current study examined the feasibility of the FCU in a pilot effectiveness trial for use with families from Seville, Spain. The goal of this project was to draw from Spanish culture and psychology to enhance the international feasibility and multicultural effectiveness of this existing psychological intervention for children and families. Specific aims of this study included: (1) adapt and embed the Family Check-Up for use in the Andalusian culture of Spain; (2) measure trends in youth adjustment, family background and support, and family management practices in Spanish families; (3) randomly assign 17 youth and families from Seville, Spain to the Family Check-Up and examine differences in youth adjustment and family management; (4) measure family engagement and motivation to change based on random assignment to the Family Check-Up model; and (5) measure the feasibility and uptake of the Family Check-Up model in Spain. 20 To examine the first aim of adapting and embedding the FCU for use in the Andalusian culture of Spain, the FCU model was conducted in the city of Seville, Spain with a sample of local resident families and with the assistance of Spanish psychologists. The implementation procedure was carefully documented with special consideration given to challenges that arose related to feasibility of the model in the Spanish context. To examine the second aim of measuring trends in youth adjustment, family background and support, and family management practices in Spanish families, descriptive data was analyzed to identify patterns across all three domains in the research sample. To examine the third aim of measuring group differences in youth adjustment and family management based on assignment to the Family Check-Up or control condition, this study investigated whether (1) there were longitudinal differences between pretest, posttest, and follow-up scores for youth adjustment based on assignment to the Family Check-Up condition, and (2) if there were differences between pretest, posttest, and follow-up scores for family management practices based on assignment to the Family Check-Up conditions. The fourth aim of this study was to measure family engagement and motivation to change based on random assignment to the FCU. This aim was examined by measuring if assignment to the FCU condition accounted for variance in family engagement and motivation to change. It was hypothesized that families who engaged in the FCU would show higher motivation to change than families assigned to the FCU but who do not engage in the intervention, and families in the control group condition. 21 Finally, the last aim (aim 5) was to measure the fidelity and uptake of the FCU model in Spain. By collecting data from therapists and interventionists on model feasibility in this context, this aim was examined by analyzing statistical patterns from data collected on model feasibility in the Spanish culture and healthcare system. Implications of the proposed study include promoting cross-cultural psychology collaborations in order to improve the efficiency of dissemination of critical, time- sensitive evidence-based practices for preventing problem behaviors in at-risk youth and families. More specifically, enhancing the international feasibility and multicultural applicability of Family Check-Up will help to expand our knowledge about cross- cultural adaptation and dissemination of existing EBP services. An increase in cross- cultural collaborations to promote broader dissemination of EBPs will contribute to promoting the awareness about the value of prevention services and cross-cultural collaboration for our global community (Leung, 2003). 22 CHAPTER II METHOD The present study included 17 families (6 waitlist-control and 11 intervention) with pre-adolescents ranging in age from 9 to 12 from the greater Seville, Spain area. The project was completed in collaboration with the University of Seville, three local public schools, and the Instituto Municipal del Bienestar Social (IMBS) of the Ayuntamiento de Mairena del Aljarafe. The project utilized a pretest/posttest with follow-up, randomized control group, experimental design to determine the feasibility and effectiveness of the Family Check-Up with the sample of Spanish caregivers and their children. With the support of consulting psychologists at the University of Seville and the IMBS, the Family Check-Up model was adapted for appropriate implementation within the cultural context of Seville, Spain. Dimensions specified by the Ecological Validity Model (Bernal, et al., 1995) were utilized as a framework for considering the intervention context while embedding the FCU into Andalusian culture. Participants and Sampling Participants. The study consisted of a sample of 17 volunteer families seeking assistance with parenting and/or youth problem behavior. Families who were eligible to participate were comprised of one or two primary caregivers and at least one child between the ages of 9 and 12 years who met screening criteria. This age range was chosen due to existing research evidence from prevention trials in the U.S. that demonstrated strong intervention effects in response to receiving the FCU, coupled with research findings indicating that Spanish youth ages 13-17 evidenced the highest prevalence of antisocial symptoms, which made them an appropriate prevention sample 23 (Stormshak, et al., 2009; Vera, et al., 2010). Families with multiple children meeting inclusion criteria were asked to identify one target child (TC) as the focus of the intervention. Exclusionary criteria included families who were unable to initially commit to attending each of the research sessions, and families undergoing certain legal proceedings (e.g. divorce, custody trials, domestic violence altercations, etc.) that could have posed a threat to validity due to a legal motivation to participate. Families were randomly assigned to either the intervention or control group. Recruitment. Recruitment occurred over a period of five months on an ongoing basis. First, the school principal disseminated an information flyer about the study to teachers with students, ages 9-12 years old who then requested that children take the information sheet home to their parents. The information flyer contained basic information about the project, including the purpose, benefits, compensation, and time commitment (Form A; see Appendix C for a list of all forms). The information flyer also contained an invitation for interested parents to attend a meeting where they could obtain more information about the study. Next, principals sent e-mails home to qualifying families to remind them of the recruitment meeting dates. Second, a recruitment meeting was held at each of the three schools during a time when parents were most likely to attend (i.e. during parent-teacher conference night). At the recruitment meetings interventionists presented a PowerPoint presentation to interested parents that explained the study and allowed parents to ask questions about the study. Next, all parents were given an interest form where they could indicate if they were interested in participating or if they were not (See Form B). The form contained both options so that other parents would have no way of knowing who was indicating interest 24 in participating and who was not. The interest sheets contained space for families to provide their contact information so that interventionists could call them to begin their participation. Each principal was given a stack of information forms and interest forms to pass out to parents who were unable to attend the meetings. During the second phase of recruitment, the study interventionists responded to questions and established family participation. Interventionists conducted 30-minute phone calls to families who had expressed interest through the forms at the recruitment meetings with the goals of responding to any questions, administering a screening questionnaire, and setting a date for the pretest meeting. The screening procedure consisted of administration of a brief survey that included a measure of parenting problems and parents’ motivation to change parenting behaviors (see Form C for screening measure). Caregivers who endorsed the presence of problems related to their parenting behaviors met inclusion criteria for the study. A script was utilized during recruitment and screening phone calls in order to ensure standardization of recruitment (see Form D). Informed consent that was determined to comply with Spanish research requirements about confidentiality, video-taping, and rights of minors was briefly explained over the phone to reduce the likelihood that families would drop out of the study after receiving informed consent during the pretest meeting. Treatment group assignment. Assignment to the intervention or waitlist- control group occurred prior to conducting the screening phone calls. The principal investigator generated a list of numbers that randomly coincided with assignment to treatment or waitlist-control groups. Next, having no knowledge of the families, the principal investigator randomly assigned a confidential participant number to the 25 interested families so that family identification numbers could be matched with assignment to the treatment or control groups. The principal investigator then provided interventionists with the family contact information and their group assignment prior to conducting the recruitment/screening phone calls so that interventionists could plan the pretest meeting times accordingly. A total of 11 families were assigned to the treatment group condition and 6 families were assigned to the waitlist-control group condition. Families were assigned to a condition immediately upon expressing interest in the study in order to allow sufficient time for scattered intervention administration given the longitudinal methodology and to avoid ethical concerns of having parents wait for services who had expressed a need for help with parenting. Families assigned to the intervention group received elementary school as usual plus the intervention portion of the Family Check-Up. Families assigned to the waitlist- control group received elementary school as usual and were not given any restrictions. The families in the waitlist-control group were given the option of participating in the intervention portion after they had completed the final data collection point. All families in the waitlist-control condition selected the option to participate in the intervention program, which the research staff delivered to them once their data collection was complete. Intervention Fidelity Interventionist and staff training. Interventionists were selected with the support of the University of Seville’s Developmental and Educational Psychology program. Requirements for interventionists included that they had received a Spanish licensure in psychology, knowledge of research methodology and procedures, and 26 theoretical knowledge of child psychopathology and ecological principles of child development. Two interventionists were selected and an estimate of the hours they would contribute to the study determined a stipend of €1,000 and a certificate of participation and training for their contribution to the study. Both interventionists were females from the greater Seville area, licensed psychologists, and currently employed as child-psychology research assistants by the Department of Developmental and Educational Psychology at the University of Seville. One interventionist was working on completing a research doctorate in developmental psychology and the second interventionist was completing a clinical masters in developmental and family psychology. Both interventionists had extensive training in core principles of the Family Check-Up, including Motivational Interviewing (Miller & Rollnick, 2002), Coercion Theory (Patterson, 1982), and the Ecological Model of child development (Bronfenbrenner, 1989). Standardized training of the interventionists occurred through several steps, using didactic trainings, video, role-plays, readings, peer support, and supervision (Turner & Sanders, 2006). The principal investigator facilitated a four-day didactic training using training materials from the Child and Family Center that were developed for this purpose. The first day comprised a review of all assessment and intervention materials to provide interventionists with an overview of the entire project (e.g. informed consent, delivering surveys, conducting observational task assessments, delivering feedback, using the family profile, and setting family goals). The second training day included a didactical seminar about coercion theory and family management practices from which the Family Check-Up was derived and a presentation 27 about utilizing motivational interviewing to deliver therapeutic feedback. The third training day was focused on coding family interactions using a Spanish-translated version of the FAST Observational Coding Manual (Fosco, Doyle, Dishion, Kavanagh, & Stormshak, 2010). The focus of this training day was to practice coding observation tasks and use the entire group to understand the objective codes and become reliable. The fourth day of training was focused on preparing a feedback session by transferring pretest data onto the FCU feedback profile, developing a conceptualization for feedback, and using role-plays to practice the feedback delivery. The second phase of training was ongoing and was focused on clinical supervision and reliability/treatment fidelity. Interventionists were required to submit their coded pretest observations of the FCU FAST tasks for each family to the principal investigator to determine reliability. Additionally, a supervision meeting was conducted prior to each feedback session in order to review feedback conceptualizations, how to use video segments in feedback, potential family goals, and appropriate resources from the menu of options. Finally, and after delivering feedback sessions, interventionists were required to evaluate their delivery of the model using the COACH framework that was developed for treatment fidelity and therapist self-evaluation of the Family Check- Up model (Dishion, Knutson, Brauer, Gill, Risso, & Kavanagh, 2010). Each fidelity evaluation was reviewed by the principal investigator as a supervision tool and to ensure treatment model fidelity. Treatment Fidelity. Treatment fidelity was ensured and measured in a variety of ways. The fact that all of the FCU materials had previously been standardized and translated into Spanish provided a structure that facilitated ease of measurement of, and 28 adherence to treatment fidelity. First, interventionists utilized the same presentation for each recruitment meeting (see Form C). Second, interventionists used scripted interviews for the screening/recruitment phone calls and templates were followed for the motivational interview that was conducted at the pretest meeting for the treatment group families (see Form E). Third, interventionists were required to follow a fidelity checklist that documented the procedures of each step and the time spent during each step of the research and intervention process (see Form F). Fourth, as an additional fidelity measure, the pretest and posttest observational family interaction tasks along with the feedback sessions were video recorded. These recordings allowed the principal investigator to randomly select and view meetings from the study to ensure that model adherence and fidelity. Fifth, the observational family interaction tasks were coded by interventionists and half of the tasks were randomly selected and checked for reliability of coding by the principal investigator. Sixth, interventionists were required to utilize the COACH framework, which is a treatment adherence model for each feedback session, to ensure that they adhered to the feedback model correctly (see Form G). The PI also randomly selected feedback session video recordings to review with interventionists and utilized the COACH worksheet to discuss and provide support for their adherence to treatment fidelity. Finally, intervention staff participated in advisory committee meetings in order to ensure standardization and address any concerns that arose in regards to cultural sensitivity or model fidelity. Intervention Procedures Informed consent. In accordance with human subjects protocol at the University of Oregon and with ethical research guidelines at the University of Seville, 29 caregivers were provided with informed consent and youth were provided with a document of informed assent, which they were required to sign prior to commencing participation in the study (See Form H for informed consent and assent documents). These documents were reviewed by the advisory committee and adapted to fit with Spanish cultural norms for consent and assent. Consistent with ethical guidelines of the American Psychological Association, both the informed consent and assent documents were written at a sixth grade reading level, and participants were able to elect to have the documents read to them (APA, 2002). Assessment protocol. The FCU is a three-step process, including an intake interview, assessment, and feedback session – though for purposes of internal validity, the assessment protocol were administered prior to the treatment group’s initial interview (Dishion, et al., 2008). All 17 participant families received the multi-agent and multi-modal assessment, which occurred in two parts (behavioral observations of family interactions and survey measures completed by the child and at least one caregiver), and were administered at pre- and post-test. Additionally, the primary caregiver completed a screening measure prior to the pretest and a follow-up survey measure one-week after completing the posttest evaluation. The family observation task consisted of five standardized discussion tasks requesting the family to discuss (1) school goals, (2) supervision, (3) limit setting, (4) problem solving, and (5) planning a family activity. The discussion tasks were designed to provide multimodal data about the following aspects of family management: positive parenting-school goals, supervision and monitoring, limit setting, family problem solving, and family relationship quality. The standardized tasks were adopted from the 30 Family Assessment Task (FAST; Fosco, Doyle, Dishion, Kavanagh, & Stormshak, 2010) family discussion activity that has been utilized by federally funded projects, such as Early Steps and Project Alliance 2, and with both normative and clinical samples at the Child & Family Center (CFC) clinic. Observational coded data from the FAST task were collected in accordance with the FAST Observational Coding Manual that was developed for the above projects, and through which standardization and reliability were ensured (Fosco, et al., 2010). The purpose of observation was to gather information about family strengths and weaknesses along multiple dimensions. (Refer to Form I for the FAST coding manual that includes the family discussion tasks.) The second form of assessment was a survey that was administered to all families at pretest, posttest, and follow-up assessments. The family assessment (EFAB) contained all relevant items that, for treatment group families, informed the results provided during the feedback session (Doyle, Fosco, & Dishion, in preparation; see Form J for the caregiver assessment and Form K for the youth assessment of the EFAB). The EFAB was reduced for posttest (see Form L and M) and follow-up (see Form N) to contain only relevant items that measured the outcome constructs as defined in the research aims. Additionally, certain measures that families responded to negatively at pretest were removed in subsequent assessments as a result of consultation with the advisory committee (see Chapter III for a complete discussion of removed items). Intervention protocol. The intervention group received the Family Check-Up, which for purposes of this study consisted of two components that the control group did not receive: the initial motivational interview and the feedback session. The initial 31 interview, lasting 20 to 40 minutes, was administered during the initial assessment, and provided an opportunity for interventionists to get to know the caregiver(s) and their perspective on the family’s strengths and areas of concern, and to use motivational interviewing to gauge parents’ stage of change and help prepare them for the feedback session. The feedback session was administered in a separate session that only the parents in the treatment groups attended. The first part of the feedback session consisted of a presentation of pretest assessment results using a simple tool, called the “child and family profile,” which included empirically-based results from the FAST observation and the EFAB surveys. This tool allowed for results to be presented in a flexible, conversational style that helped interventionists to understand family value systems and assess motivation to change. The second part of the feedback session consisted of collaborating with caregivers to utilize assessment results to set family goals for the upcoming year and to select treatment options from a scaled “menu of options” that was tailored to family culture and engagement level (see Form O for the feedback materials, including the family profile and the menu of options). In order to reduce the threat to internal validity represented by resentful demoralization of subjects receiving less desirable treatments, participants were not explicitly told to which group they had been assigned (Heppner, Kivlighan, & Wampold, 2008). Instead, participants were informed that due to interventionist availability, all families would have the option to participate in the feedback session meeting but that the order of when the meeting occurred would change depending on staff availability. This allowed the families in the waitlist-control group to elect to 32 participate in the feedback session if they desired after their data collection was complete. Procedural summary. In summary, the procedural structure comprised a five- part process for intervention group families and a four-part process for control group families. First, all families from the control and intervention groups participated in a screening phone interview. Second, all families participated in a 1-2-hour intake assessment (pretest questionnaires and participation in structured observational tasks to gain information about family dynamics). Third, only the intervention families participated in the initial interview and feedback session. Fourth, all families from the control and intervention groups completed a follow-up assessment four to eight weeks after starting the study, where they completed the same surveys and observational tasks that they completed during the pretest assessment. Fifth, all primary caregivers from the family participated in a 30-minute follow-up phone interview assessment that measured follow-up outcomes and a brief evaluation of the program. Finally, in an effort to reduce the threat to internal validity, participants in the control condition were offered the option of participating in the feedback session after all of their data had been collected (Heppner, Kivlighan, & Wampold, 2008). Addressing Attrition Attrition was prevented in three ways. First, at each data collection meeting the interventionists explained the next step of the research process in order to build motivation to continue engagement in the study. Second, interventionists made reminder phone calls prior to data collection meetings in order to ensure attendance or address ambivalence. Finally, cash stipends were provided incrementally over the study 33 duration in order to increase participation rate and reduce the chance of attrition (pretest = €15 for one caregiver and €25 for two caregivers, and a small prize valued at €3 to €5 for each target child; posttest = €15 for one caregiver and €25 for two caregivers, and a small prize valued at €3 to €5 for each target child; total stipend per family = €30 to €50 and 2 small prizes). See Form P for receipts that were given to participants to document monetary compensation. Data Collection Collection of data occurred in two overlapping phases for the participants and for the therapists. The first set of data was collected from participants at four time points for the control and intervention groups. The first data time point was the screening measure that was administered to all potential participants. The second data point comprised multimodal and multiagent data that included the pretest survey packet (see measures below) and the family observation task, and was collected from both the control and intervention groups. The third data point was a multimodal, multiagent assessment collected from the intervention and control groups. This data measure consisted of the posttest survey packet (which was adapted from baseline survey packets and only included items that coincided with outcome measures), and a second family observation assessment. The second observation was the same as the initial assessment observation, and utilized the FAST procedures described above (Fosco, et al., 2010). In order to avoid relationship bias in coding, the second FAST observational recording for each family was coded by the interventionist who was not assigned to work with that family during the research study. 34 The second stage of data collection occurred in accordance with the fifth aim of this study (measure the uptake and feasibility of the FCU model in Spain). Data was collected from the two interventionists who implemented the study as well as the advisory committee members and school principals who helped to oversee the implementation of the study (see Form Q for interventionist survey and Form R for advisory committee survey). The data was collected in order to gain an objective measure of model feasibility in the Spanish cultural context. Measures It is recommended by U.S. and Spanish researchers that child and family research studies contain multiagent sources of information, but that parent reports are preferred due to the stronger correlations between parent reports of youth problem behaviors and youth psychopathology (Kazdin, 1995; Vera, et al., 2010). As such, multiagent reports from multiple caregivers, target children, teachers, and interventionists were obtained in this study in order to inform the feedback session for the intervention. Data that was analyzed for the results of this study was collected from the primary caregiver only. The two surveys that were administered over the phone could only be administered to one person for ease of data collection, and the primary caregiver was chosen to be the sole reporter due to recommendations from the extant literature (Kazdin, 1995). Surveys were administered in Spanish by the interventionists. The survey measures that were administered in person were administered with the option of participants completing them on their own or with the support of the interventionist reading it out loud to each participant. 35 Intervention status. Random assignment was coded as 1 (intervention) and 2 (control). Demographics. Youth were asked to provide demographic information about their age/date of birth, and gender. Caregivers were asked to provide information about their contact information, date of birth, gender, ethnicity, nationality, marital status, custody agreement, family members’ ages, and socioeconomic status (see Form J). Screening measure. The screening procedure consisted of the administration of a brief survey that included demographic information about the family participants and a measure of perceived parenting problems and their motivation to change those problems. The screening tool was adopted from the CFC clinic to screen clinical families for service eligibility. (See Form D for the screening measure.) Parenting Problems. Scales from the Parenting Children and Adolescents measure (PARCA) was used to screen for problems that parents experience related to their child’s behavior (McEachern, Dishion, Weaver, Shaw, Wilson, & Gardner, in preparation). This portion of the PARCA was comprised of four scales related to positive parenting, limit setting, proactive parenting, and parental monitoring. The positive parenting scale contained seven items and was measured on a 7-point scale (1= never, 4 = sometimes, and 7 = always). Sample items included, “In the past month, you played with your child in a way that was fun for both of you.” The limit setting scale contained seven items and was measured on a 7-point scale (1= never, 4 = sometimes, and 7 = always). Sample items included, “In the past month, you set rules and stuck to them.” The proactive parenting scale contained seven items and was measured on a 7- point scale (1= never, 4 = sometimes, and 7 = always). Sample items included, “In the 36 past month, I avoided arguments with my child by giving her/him clear options (for example, offer an option for activities that s/he could do.” The monitoring scale contained eight items and was measured on a 7-point scale (1= never, 4 = sometimes, and 7 = always). Sample items included, “In the past month were you able to keep tabs on your child’s daily activities with friends/peers?” The PARCA is a reliable measure of this construct (Positive Parenting Scale Chronbach’s α= .76; Proactive Parenting Scale Chronbach’s α= .83; Limit Setting Scale Chronbach’s α= .84; Monitoring Scale Chronbach’s α= .73). Motivation to Change. The scale measuring parents’ motivation to change was measured in conjunction with the screening measure of parenting problems on the PARCA. At the bottom of each of the four scales on the PARCA, parents are asked to respond to a question about how much they want to change that area of parenting skills. The item stated “Thinking about the items above, would you like to do things differently in this area of parenting?” Items were measured on a 7-point scale (1 = I want to change, 4 = some changes are needed, and 7 = okay the way it is). This measure was used to assess caregivers’ motivation to change parenting behaviors, and was administered at screening, posttest, and follow-up data points. Model Feasibility and Uptake. This construct was based on family participant, interventionist, and advisory committee reports. This information was used to assess the extent to which Spanish psychologists, school staff, and parents believed that the FCU model was a useful and culturally sensitive tool that could be feasibly implanted into the Spanish cultural context. Given the context-specific nature of the assessment, a measure of feasibility and uptake was constructed for this research, and was derived from a 37 similar measure that was used in previous dissemination research on the Family Check- Up model. See Form Q and R for the implementation and uptake surveys that were given to interventionists and advisory committee members to inform the measurement of this construct. Measures included quantitative items, such as “The Family Check-Up program is useful for targeting youth behavior problems,” and items were rated on a 4- point scale (1 = not true, 2 = somewhat useful, and 3 = certainly useful). Advisory committee members and principals also responded to open-ended questions, such as “Which materials from the study did you find to be the most useful for the study,” or “Which methods of interaction with participants in the study did you find to be the most and least culturally appropriate?” Open-ended questions were coded for emerging ideas, and themes that were identified if two or more people discussed a similar idea. Only themes were reported in the qualitative results. The parent reports of the feasibility and uptake of the model were included in the follow-up questionnaires (See Form N) and the advisory committee and interventionist surveys were distributed after conclusion of data collection (See Form Q and R). Family observation data. The Family Check-Up Coding Sheet was used in conjunction with the FAST Coding Manual as a measure of therapist observed ratings of participant families (Form I; Fosco, et al., 2010). The FAST coding sheet is a 31-item measure of family management skills that is rated on a 5-point scale (1 = low skill set and 5 = high skill set). Family management skills are divided into five domains, including positive parenting-school goals, supervision and monitoring, limit setting, family problem solving, and family relationship quality. The coding sheet also contains a measure of participant affect and engagement that was used to inform feedback data in 38 this study. Sample items include “Clear, measurable goals” (measure of positive parenting-school goals) and “caregiver active listener” (measure of supervision and monitoring). Family discussion tasks were self-evaluated by caregivers and the target child, and observationally coded by the interventionists. These scores were combined with primary caregiver data to measure the outcome variables that were defined in the research aims. Ecological Assessment Battery (EFAB; Doyle, et al., in preparation). The Ecological Family Assessment Battery is a set of survey packets comprised of youth and caregiver measures that have been translated to Spanish, and that were developed and validated for the Family Check-Up by CFC researchers. This survey packet was administered to all participants at pretest, posttest, and follow-up data points. Data was used (1) to inform results presented to intervention families during the feedback session and (2) to examine trends in the entire sample for youth adjustment, family background, and family management practices. The youth portion of the EFAB is called the FCU- YA and the caregiver portion is called the FCU-CA (see below). EFAB FCU Youth Assessment (FCU-YA): This packet comprises various measures that are divided into five sections: youth self-report information (31%), family interactions (28%), coping and self-regulation (26%), friendships/peer relations (10%), and school success (5%). (See Form K for the complete FCU-YA measure). Child report variables from the EFAB Youth Assessment packet were used to inform the feedback session and can be found below: Demographics. See above. 39 Youth substance use and problem behavior. Youth completed a self-report survey about their substance use and antisocial behaviors (Form K). They were asked to report on the frequency with which they use tobacco, alcohol, and drug use during the previous month, ranging from 1 (not at all) to 5 (very often). Youth engagement in antisocial behavior was measured with 6-items on a 3-point scale, ranging from no, never true to sometimes true to yes, definitely true. Sample items included “purposely damaged or tried to damage property.” These items have been used in previous research on antisocial behavior and are considered to be a reliable estimate of this construct (Cronbach’s α = .84; Stormshak, et al., 2011). This measure was adopted from the Conduct Problems Scale on the Strengths and Difficulties Questionnaire (Goodman, Metzler, &Bailey, 1998). A measure of antisocial behavior that was created in Spain and validated on Spanish adolescents (ages 11-17) was also administered to youth in this sample as an exploratory measure, as both a comparative measure of antisocial behavior and as a construct validation tool (Antolín Suarez, in preparation). See Form K, page 3 for Spanish antisocial behavior measure. Additionally, the Child Peer Social Skills scale (CPRSK; Dishion & Kavanagh, 2003), which assesses deviant peer affiliation, was measured and incorporated into this construct. Items on the CPRSK are measured on a 5-points scale (0 = less than 25%, 2 = 50%, and 4 = more than 75%), and included items such as “How many of your friends misbehaved or broke rules?” the CPRSK is an 8-item scale. CPRSK: Cronbach’s α = .66). Positive parenting. The Positive Reinforcement Scale from the Community Action for Successful Youth (CASEY; Metzler, Biglan, Ary, & Li, 1998) was used to 40 measure positive parenting in this study (Form K). This is a four-item scale, with items rated on a 5-point scale (0 = never, 2 = sometimes, and 4 = very often). Sample items include, “In the last month, when you followed a household rule or did a good job, your parents or guardians have given you a hug, kiss, or kind word.” The CASEY has been deemed a reliable measure of this construct (Cronbach’s α = .86). Parental monitoring. This is version of the parent monitoring scale in the CASEY (Metzler, et al., 1998) that was revised for the Project Alliance 2 Student Survey to measure youth report of parent monitoring behaviors (Stormshak, et al., 2009). This is a 19-item scale that was measured on a 5-point scale (0 = never, 2 = sometimes, and 4 = always). Sample items included, “How often does one of your parent’s know what you are doing during your free time?” This scale has been deemed a reliable measure of this construct (Cronbach’s α = .98). See Form K for an example of this measure. EFAB Family Check-Up Caregiver Assessment (FCU-CA). This packet includes various measures that provide comparison scores for the youth constructs described above, and is divided into the following three sections: family history and support (33%), youth adjustment (33%), and family management and relationships (34%). (See Form J for the FCU-CA measure). Variables from the EFAB packet that were used to examine research questions are below: Demographics. See above. Parenting problems scale. See above. Motivation to change. See above. 41 Parent perception of youth substance use and problem behavior. Parents completed a survey about their perception of their child’s substance use and antisocial behaviors (see Form J). They were asked to report on the frequency with which their child used tobacco, alcohol, and drugs during the previous month, ranging from 1 (not at all) to 5 (very often). Two measures of antisocial behavior were utilized that included the Conduct Problems Scale portion of the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) and the Parent Report Peer Social Skills scale (PPRSK; Dishion & Kavanagh, 2003). The SDQ survey consists of 5-items that were measured on a 3-point scale (0 = Not true, 1 = Somewhat true, and 2 = True), and included items such as, “In the past month I am certain that my child stole things from home, school, or other places” or “my child destroyed things on purpose.” Items on the PPRSK were measured on a 5-point scale (0 = less than 25%, 2 = 50%, and 4 = more than 75%), and included items, such as “How many of your child’s friends misbehaved or broke rules?” These scales were deemed reliable measures of this construct (SDQ: Cronbach’s α = .70-.85; PPRSK: Cronbach’s α = .72). Additionally, parents were asked to report on an adapted, exploratory measure of antisocial behavior that developed in Spain for children ages 11-17 (Antolín Suarez, in preparation). This measure was used only to inform feedback for the treatment group. Parent perception of youth internalizing behaviors. Parents completed a survey about their perception of their child’s emotional adjustment and internalizing behaviors (see Form J). Youth emotional adjustment was measured using the Emotional Symptoms Scale portion of the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997). The survey consists of 5-items that were measured on a 3-point scale 42 (0 = Not true, 1 = Somewhat true, and 2 = True), and included items such as, “In the past month my child got a lot of headaches, stomach-aches, or sickness” or “my child was often unhappy, depressed, or tearful.” This scale has been deemed a reliable measure of this construct (SDQ: Cronbach’s α = .70-.85). Positive Parenting. Two measures were utilized to measure the construct of positive parenting in this study (see Form J). First, the Good Behavior Support measure, taken from the Parenting Children and Adolescents scale (PARCA; McEachern, Dishion, Weaver, Shaw, Wilson, & Gardner, in preparation), was used. This is a 19- item measure that is rated on a 5-point scale (0 = never, 2 = sometimes, and 4 = very often). Sample items included, “During the last month, you noticed and praised your child’s behavior.” This survey has been deemed a reliable measure of this construct (Cronbach’s α = .76). Second, the Proactive Parenting scale from the Parenting Children and Adolescents survey (PARCA; McEachern, et al., in preparation) was used to measure positive parenting. This is a 7-item scale, with items rated on a 5-point scale (0 = not at all, 2 = sometimes, and 4 = very often). Sample items included, “In the last month, how often did plan ways to prevent problem behavior?” This survey has been deemed a reliable measure of this construct (Cronbach’s α = .83). Parental monitoring. The Parent Monitoring scale of the PARCA was used to measure this construct (Form K; McEachern, Dishion, Weaver, Shaw, Wilson, & Gardner, in preparation). This is a 17-item scale that is measured on a 5-point scale (0 = never, 2 = sometimes, and 4 = very often) with the last item as a write-in item that asks about the number of unsupervised hours that their child spent in the past week. Sample 43 items included, “How often do you know who your child is with during her/his free time?” This survey has been deemed a reliable measure of this construct (Cronbach’s α = .73). Data Analysis Qualitative design. The first aim of the study, implementation of the FCU in Spain, required careful documentation of procedures in accordance with the Ecological Validity Model (Bernal, Bonilla, & Bellido, 1995). A narrative account of the meetings with the advisory committee was provided to describe the procedures that were taken to adapt the FCU model to the Spanish context in a culturally sensitive way. The fifth aim of the study examined the uptake and feasibility of the FCU model in the cultural context of Spain. Participants, school staff, interventionists, and advisory committee members were provided with questionnaires with items requiring write-in responses. These items were coded into themes as determined by the identification of two similar ideas by at least two different reporters (Goodkind, LaNoue, Lee, Freeland, & Freund, 2012). The themes were presented to supplement the quantitative data that was obtained to satisfy this research aim. Statistical design and analytical procedure. All statistical analyses were conducted using SPSS v.20.0 for Mac. Descriptive statistics were utilized to examine the second aim (measuring trends in youth adjustment, family background and support, and family management in the intervention group) and the fifth aim (measure the fidelity and uptake of the FCU model in Spain) of this study. The third and fourth aims of this study require the measurement of group differences. First, data from the pretest time point was analyzed using a One-Way Analysis of Variance in order to determine if 44 the intervention and control groups had statistically different group means for the outcome variables. Second, a mixed effects between, within repeated measures Analysis of Variance was conducted to measure longitudinal group differences due to intervention assignment. Finally, post-hoc paired samples t-tests were conducted to examine within group change separately for each group across the three time points. Variables. There was one categorical independent variable with two levels: receipt of the Family Check Up intervention (1 = intervention; 2 = no intervention – control). The third study aim included five continuous dependent variable constructs, including positive parenting, parental limit setting, parental monitoring, antisocial behaviors, and internalizing behaviors. No covariate was included due to a low power estimation from having a small sample size. Motivation to change was measured for the fourth study aim at screening, post-test, and follow-up collection points. Variables that were measured in the descriptive analyses for the second aim of the study are as follows: (1) youth adjustment that consists of measures of behavior, emotional adjustment, peer relationships, self regulation/coping, substance use, and school success; (2) family background and support that includes the following observed variables: parent wellbeing, family stress, parent substance use, partner support, caring adults/support network, and parent coping; (3) family management and relationships, that includes the observed variables: parent-child relationship quality, positive behavior support, monitoring, limit setting, and communication/problem solving. Additionally, descriptive variables representing fidelity and uptake of the FCU model (aim 5) were measured using a survey constructed for this purpose. 45 CHAPTER III RESULTS Cultural Embedding of the Family Check-Up Model Descriptive data was collected to document the process of adapting the Family Check-Up to be implemented within the culture of southern Spain in an appropriate and culturally sensitive manner. Information was gathered along the following domains: forming and utilizing an advisory committee, culturally and linguistically adapting intervention and research materials, and identifying and liaising with school participants. The eight domains of the Ecological Validity Model (EVM; Bernal, Bonilla, & Bellido, 1995) were utilized as a framework to consider the Spanish cultural context when implementing Family Check-Up. Forming and utilizing an advisory committee. An advisory committee was formed in consultation with doctoral-level psychologist researchers at the University of Seville. The advisory board comprised seven individuals (5 females and 2 males) ranging from age 27 to 64 from a variety of educational and professional backgrounds, including four university personnel (1 = A full professor and researcher of developmental psychology; 2= A first-year faculty member and researcher of developmental psychology; 3 = A doctoral student of developmental psychology and university research staff; and 4 = A master’s student of clinical family psychology and university research staff), two personnel from elementary schools in two different communities (5 = A public school administrator/general education teacher; and 6 = A public school bilingual education teacher), and a community member (7 = retired medical physician). Advisory committee members agreed to meet several times over the 46 course of the study in order to support the cultural sensitivity and effective implementation of the research project. The goal of the first advisory board meeting was to provide an overview of the intervention model and research design, to discuss the task of cultural adaptation, identify remaining advisory board members, and discuss options for participating schools. The meeting was held at the university and was attended by the university personnel and community member. The principle investigator presented goals of the project and provided information about the intervention and its research history. Committee members addressed concerns related to how to deliver feedback to intervention group families that whose data was categorized based on norms from the United States. To address this issue, members decided to include a measure developed in Spain to provide secondary measure of youth adjustment and to discuss the issue of norms with family participants during each feedback session while also allowing for clinical judgment to help inform the risk-level of outcomes on the feedback profiles. Board members were concerned that adjusting risk-level of outcomes on the feedback profile would influence the data subjectively; however, they withdrew the concern when the principle investigator explained that the outcomes placed on the feedback profile were mutually exclusive from the data points used to analyze intervention effectiveness. Next the committee members developed a plan to support the cultural adaptation of the research materials and measures. Two members on the advisory committee were identified to review the measures for cultural congruency and accuracy of English to Spanish translation. An additional psychologist in the community was identified as another consultation source to address the clinical aspects of the model adaptation. 47 Citations for each measure were provided in order for the reviewers of the measures to search for existing measures that had already been successfully adapted within their university library system. Remaining advisory members were assigned to the development of recruitment materials in collaboration with school staff to determine the most appropriate way to attract participants. Committee members identified schools with which they had existing relationships through prior research collaborations, and from those schools the committee identified the school advisory board members. The final task of the meeting was to identify interventionists and to discuss the appropriate form of compensation for them. The two student advisory board members from the university were recommended to be the interventionists. This decision was made based on their status as licensed psychologists (“licensiadas”), their experience within graduate psychology research and clinical training, and their ability to dedicate a substantial amount of time to the project. The appropriate compensation was determined to be a €1,000 stipend for each interventionist, which was approximated due to the hours they would dedicate to the project. Advisory committee members were told they would have the opportunity to collaborate in publications related to the research study. Cultural and linguistic adaptation of model and materials. The first task in the cultural adaptation of the intervention was to examine the title of the Family Check- Up and the assumptions of the model. The entire advisory committee participated in translating the name of the Family Check-Up to an appropriate name in Spanish. The literal translation of the Family Check-Up is “Chequeo Familiar.” In discussion with the committee, it was determined that despite the fact that Spaniards use the word “Chequeo” or “Check-Up” to refer to going to a check-up at the doctor’s office, that this 48 name may be interpreted negatively by family participants. Given the strength-based meaning of the Family Check-Up, the committee was concerned that families would interpret the name “chequeo” to be invasive and that they may think the name was suggestive of allowing themselves to be the subject of judgment and scrutiny. After extensive discussion about the purpose of the Family Check-Up as a strength-based program to support family growth in the context of children’s behavior and emotional functioning, the committee identified two options for names: “Valoración Familiar” (translation: “Family Evaluation” with an inherent strength-based tone indicating the identification of positive values and characteristics of the family) or “Programa de Apoyo Familiar” (translation: “Family Support Program”). The decision was made to name the Family Check-Up project the “Program de Apoyo Familiar” because the committee felt that this best captured the programmatic aspects of the intervention that included a longitudinal and collaborative process that was designed to support the positive development of families across several domains. The committee also decided that including the programmatic aspects of the intervention in the title would make it more appealing to families since the name indicates that a service in the form of a program is being provided to families rather than simply an evaluation. The assumptions of the model were presented to the advisory committee by using the domains of the child and family feedback profile and discussing the theories from which the model was founded. The advisory committee was presented with each of the three domains of the feedback (youth adjustment, family background and support, and family management) and they were asked to discuss the different ways that psychologists, community members, school teachers, and physicians in Spain think 49 about child development and parenting. The psychology researchers at the University of Seville conduct research from a combination of theoretical perspectives, including Cognitive Behavioral Theory, Bronfenbrenner’s Ecological Model of Development, and a variety of attachment theories. School personnel and researchers on the committee discussed a recent surge in perspectives using “desarrollo positivo” (positive development) that are receiving government funding to support additional research that can be translated for use in the community. A review of the University of Seville’s research in this area revealed that the concept of “desarrollo positivo” can be captured by the concept of “positive parenting” and “positive behavior support” among the research in the United States (Antolín, Oliva, Pertegal, & López, 2011; Patterson, 1997; Webster-Stratton, 1992). It was concluded among the advisory committee that the theories that drive the Family Check-Up were consistent with the most current theories that inform perspectives on child development and parenting in Spain. As such, the committee members agreed that the domains captured by the Family Check-Up feedback profile were consistent with their cultural values and views about child development and parenting, and therefore appropriate to use with this cultural sample. The initial interventionist training and the cultural adaptation of materials were conducted during the same time period of about three months. The first training meeting consisted of presenting an overview of the project design and preliminary study materials to the interventionists. Per the recommendation of the advisory committee, the interventionists divided the pre-existing Family Check-Up materials and reviewed them for accurate translation. They were instructed to highlight any items that were questionable for translation and/or cultural appropriateness. The decision to have them 50 review the data collection materials prior to undergoing an extensive interventionist training was made with the thought that reading through the materials would provide them with a deeper understanding of the constructs and clinical domains that inform the ecological approach to intervention with the Family Check-Up model. Once their review of the measures was complete, the measures were sent back to the principle investigator to be back-translated to ensure the meaning of each item was the same as originally intended. Items that were questionable for translation and cultural understanding were then reviewed in collaboration with the advisory committee. Most of the items were deemed to be appropriate within the Spanish culture, and many of the scales had been previously used before by the researchers. There were four concepts of the intervention model and surveys that the advisory committee had questions about related to cultural congruence. These concepts were the “cultural connectedness” scale that informed the “family background and support” domain of the feedback profile, the concept of “time out” that appeared on the survey packets, the substance use scale for youth, and the antisocial behavior measures. The cultural connectedness scale was discussed and the committee decided to leave the scale in the questionnaires and feedback profile, but to discuss the scale with parent participants. The committee concerns was that the scale targeted minority individuals and cultural concerns that exist within the United States but that the issues do not translate well culturally within Spain. The issue discussed surrounding the idea of having a strong cultural identification and how this could be perceived by family participants as questioning whether they have a sense of patriotism. The committee discussed the implications within Spanish culture of coming out of the Franco 51 dictatorship and that families have mixed views of patriotism, which are often negative due to how Spanish nationalism was forced on individuals during the Franco regime. It was determined that families who did not have a strong sense of Spanish nationalism would come out on the feedback risk categories as being “red: needs attention” or “yellow: at-risk,” which was inconsistent with a reports of many parents that modern Spanish cultural values tend to reject nationalism. Consistent with the committee beliefs, participants in the study interpreted this scale as confusing. It was the scale that most often contained written or oral comments by participants that described their confusion or belief that the scale did not accurately capture their understanding of culture. Results indicated that the majority of participants scored within the yellow or red categories of the cultural connectedness scale. Discussions with families about their red or yellow outcome included an explanation of the construct of cultural connectedness as having a sense of tradition and cultural identity, and most often resulted in families moving their categorical outcome from “yellow” or “red” to the “green” (positive qualification) domain. The advisory committee spent time discussing the implications of including the substance abuse and antisocial behavior scales within the packet because they felt these items may not be necessary within the sample of children ages 9 to 12 years old that was being recruited in this sample. Given the pilot nature of the project, the committee decided to keep these scales in the pre-test packet and reevaluate their utility when it was time to finalize the post-test and follow-up measures. The last concept that was in question with the advisory committee was the concept of “time out”. The conclusion was that psychological researchers and some followers of pop culture are aware of this 52 concept due to researchers being connected to U.S. parenting strategies and due to the fact that the television show, “Super Nanny” was recently included into Spanish television programming. However, the committee determined that the concept of “time out” was not a widely understood or practiced strategy within Spanish culture. As a result, the items describing “time out” in the survey measures were changed to “mandarle a su cuarto” or “send child to their room.” The committee also discussed the implications of including an item about “ethnicity” within the demographic measures. Some committee members asserted that the general public in Spain would be confused about the meaning of ethnicity since it is not a concept that is as widely discussed in Spain as it is in the United States. The research committee members were familiar with the concept and its use in research in the United States; however, they questioned whether the sample would understand the difference between nationality and ethnicity. A discussion occurred about the utility of identifying marginalized ethnocultural groups within the sample, such as a variety of immigrants from African countries or the self-identified “Gitano” (gypsy) ethnicity. Committee members agreed about the positive benefit of identifying children from minority groups for purposes of delivering therapeutic feedback with this information in mind. However, committee members reported that ethnic group identification questions are not typically included on demographic forms so they questioned whether an item about ethnicity would make individuals from minority groups worry that they would be unfairly targeted by discrimination within the study. Ultimately, the decision was made to add an item about nationality to the demographic information and to keep ethnicity as an item, but to leave it as a fill-in-the-blank item rather than to provide categories. This 53 would allow individuals to self-identify if they felt comfortable but not remove some of the pressure of identifying across specific categories. Ultimately, all participants either left the ethnicity item blank or wrote in “Spanish.” As such, it did not appear to be a useful item in distinguishing the different cultural heritages of the families. The adaptation and development of the recruitment and pretest data collection materials was the most time intensive and detailed phases of the cultural adaptation period. The time spent refining the initial materials was extensive. The advisory committee decided that the remainder of the adaptation of the materials would be part of an ongoing process of training and cultural adaptation that would be informed by the way that participants responded to the materials. The advisory committee met prior to beginning each new phase of the data collection in order to address any changes that needed to be made based on participant responses. One decision that was made when finalizing the post-test measures was to remove the substance use questions from the survey packets. It was determined that at the time that posttest packets would be distributed, none of the children in the study who had completed the pretest packet had endorsed any tobacco, alcohol, or drug use. Additionally many parents had responded to these questions by telling interventionists that their children were too young for those questions. As a result, the substance abuse scales were removed from the post-test and follow-up packets. The same issue occurred with the Spanish version of the antisocial behavior scale that had been included in the pretest packet based on a recommendation of the advisory committee. Parents responded to the scale with concerns that the items were extreme or may be more appropriate for clinical samples. Additionally, the children in the sample endorsed the items on the antisocial behavior scale with minimal 54 to no incidence and frequency. As a result, these scales were removed from the post-test and follow-up packets and only the original Family Check-Up measure for conduct problems were included during subsequent phases. The final aspect of the intervention that was adapted to the cultural context of Seville, Spain was the menu of options that is provided at the end of the feedback session. The advisory committee recommended that the menu of options should be created in consultation with local psychologists who provide parenting support services to families. A meeting was set up with the managing psychologist of a government-run community mental health agency to consult about the available services for parenting support within the communities of the participating schools. Upon presentation of the feedback profile and the types of goals that parents would identify through the intervention model, the psychologist consultant shared a variety of tiered services that families could receive to address those goals. The tiers included individual, couples, family, and group-based therapy services in both the public and private sectors of the community. This allowed for families to participate in a range of services that were either free or sliding-scale. Additionally, parenting resources through the schools were identified, such as parent-education classes. Finally, a list of culturally-sensitive books and online resources were compiled based on the recommendations of the consulting psychologists. Identifying and liaising with schools. Public schools were identified by the advisory committee as the optimal setting in which to recruit participants and carry out the intervention and data collection. School personnel representing two public schools in the greater Seville, Spain area were invited to participate on the advisory committee 55 based on existing relationships with other advisory committee members and previous research collaborations. The advisory committee conducted one meeting to discuss the interest of local public schools to participate in the study and another meeting to discuss recruitment strategies. The principle investigator presented the overview of the research project and intervention to school personnel on the advisory committee who then agreed to liaise with principals at local schools to determine their interest in being involved with the project. Three local schools were identified by the school personnel based on their current or previous employment in those schools. Each of the identified elementary schools were public schools that served students from kindergarten through 5th grade. The grades eligible for participation in the study were 3rd through 5th grade. The process of schools agreeing to participate occurred through two different strategies: seeking approval from a government-elected entity that was in charge of the public education in the province of Seville or seeking approval from a local school board consisting of parents and administrators. Each school went through a minimum of three hierarchical levels of administrative officials to obtain approval for the study. The approval process for each school took between one to two months. Other than needing to obtain the approval through a bureaucratic process, the only concern that was raised by one of the schools was regarding the financial compensation for family participants. For this school the administrators were concerned about the implications related to their status as a public school that was advertising participation with financial compensation. They requested proof of the funding grant and the source of the funds be presented to officials. Once the grant was determined to be from a collaboration with the Spanish government, administrators agreed to allow for the financial compensation. 56 Recruitment materials were designed in collaboration with school and research personnel on the advisory committee, and in consultation with participating school administrators. Originally a flyer had been designed to capture the attention of parents, but school administrators reported that a less enticing and more detailed informational flyer would be more effective in recruiting family participants. As such, an informational flyer was designed that was distributed to all eligible students and parents across the three local schools. School administrators requested that a recruitment meeting that was directed by both the interventionists and the principle investigator would be their preferred method of recruiting families. They suggested that the nature of the intervention research study and the aspect of financial compensation was a novel concept in Spain, and as a result, families would be more likely to trust the program if they could attend a recruitment meeting in a large group. This meeting would allow for families to ask questions and discuss the aspects of the study that were less typical in Spain. During the recruitment meetings, the most common question asked was related to the financial compensation. Parents wanted to know why they would be receiving financial compensation for participating in a service that was designed to benefit their families. The advisory committee had anticipated this question because financial compensation is not typically used in research within Spanish psychology. The response prepared for participants was that the finances would help provide an incentive for the continued participation in the follow-up stages of the study, and that it would provide families with financial support for missing work hours and to provide for transportation in the time of an economic crisis so that they could attend the meetings without hardship. Families appeared to be satisfied with this response and on their pretest 57 questionnaires, some parents responded that their participation was positively influenced by the knowledge that they would obtain some financial support given the economic crisis. Trends in Youth Adjustment, Family Background and Support, and Parenting To examine the second aim of measuring trends in youth adjustment, family background and support, and family management practices in Spanish families, measure of central tendency, correlations, and distribution were analyzed to identify patterns over time and to check that no assumptions were violated for later analyses. Analyses were conducted using SPSS 20.0 for Mac. Data were screened for missing values and the few examples of missing data that were found were replaced with mean values of the sample for particular scales. Data was screened for extreme outliers, and no extreme outlier cases from parent report data were identified. Demographics of the sample were examined according to age, sex, sex of primary caregiver participant, monthly income, the number of individuals supported by income, and parental perceived financial stress. Demographics were compared between the intervention and control groups, which can be found in Table 1 (see Appendix A for all tables). An examination of distribution revealed that all variables for youth adjustment, family background and support, and family management contained a negative skew. This sample tended to endorse items positively, suggesting that this was a normative sample without trends toward clinically elevated behaviors. However, taking into account the negatively skewed data, the scores for each variable were roughly normally distributed. 58 Measures of central tendency were examined for each of the variables of interest: youth adjustment (see Table 2), parent report of family support and background (see Table 3), and family management (see Table 4). A one-way analysis of variance was conducted to examine potential group differences at each time point for all variables. The one-way ANOVA yielded no significant differences between group means for any variables at any of the three time points. Despite the fact that no group differences were found, trends in variable changes over time were noted. Youth adjustment variables revealed trending improvements for conduct problems, hyperactivity, emotional symptoms, and peer problems with different patterns observed based on reporter and group assignment. Similarly, means appeared to increase over time for family management variables, including parent-child relationship quality (parent-child conflict and FAST relationship quality), positive parenting (proactive parenting, good behavior support, and FAST positive parenting), limit setting (negative parenting, positive limit setting, and FAST limit setting), and supervision & monitoring (parental monitoring, communication and problem solving, FAST monitoring task, and FAST problem solving task). A change in the means of parent reports of motivation to change was also observed over time (see Table 4). Intervention Effectiveness on Youth Adjustment The third aim of this study was to measure if there were longitudinal group differences in youth adjustment and family management practices. A mixed effect repeated measures analysis of variance was conducted to assess the impact on youth adjustment (youth problem behaviors and internalizing problems) of group assignment (Family Check-Up intervention group or control group) across three time periods 59 (pretest, posttest, and one-week follow up). Univariate and multivariate assumptions of normality, sphericity, equality of error variances, equality of covariance matrices were tested, and no violations of these assumptions were evident for the Conduct Problems or the Emotional Symptoms scales. Youth problem behaviors. A mixed effect repeated measures ANOVA was conducted to measure the longitudinal differences in group assignment on youth problem behavior. The parent-reports on the Conduct Problems Scale were used to assess problem behavior, and were measured across three time periods (pretest, posttest, and follow-up). All results can be found in Table 5. There was no significant interaction between group assignment and time for parent-report of youth problem behavior, Wilks Lambda = .91, F(2, 14) = 3.86, p = .50, partial eta squared = .10. There was a substantial main effect for time, Wilks Lambda = .65, F(2, 14) = 3.86, p < .05, partial eta squared = .36, with both groups showing significant changes in problem behaviors over time. The main effect comparing the intervention and waitlist-control groups was not significant, F(1, 15) = 1.80, p = .20, partial eta squared = .11, suggesting no difference in the effectiveness of the intervention on problem behavior. Given the main effect of time for both groups and the fact that descriptive statistics indicated a positive improvement in problem behaviors for the intervention group and a decline in means over time for the control group, a closer examination of the changes over time was indicated. A post-hoc paired samples t-test was conducted to evaluate the longitudinal impact of the intervention group on the problem behavior (see Table 6 for all t-test results). There was a trend level improvement in problem behavior 60 scores from pretest (M = 2.32, SD = .53) to posttest (M = 2.45, SD = .45), t (11) = -2.04, p = .07. The mean change in problem behavior scores was 0.14 with a 95% confidence interval ranging from -.29 to .01. The eta squared statistic (.29) indicated a large effect size. The decrease in problem behavior was maintained at the follow-up data collection point (M = 2.45, SD = .30) with no significant changes from posttest to follow up. On the contrary, the control group showed no significant improvements from pretest (M = 2.64, SD = .27) to posttest (M = 2.75, SD = .20). Furthermore, the control group trended toward a decline in problem behavior from posttest to follow-up (M = 2.47, SD = .34), though changes were not significant at the p < .05 - level. Youth internalizing behaviors. A mixed effect repeated measures ANOVA was conducted to measure the longitudinal differences in group assignment on internalizing problems. The parent-report on the Emotional Symptoms Scale was used to assess internalizing problems, and was measured across three time periods (pretest, posttest, and follow-up). Parent-reports indicated no significant interaction between group assignment and time for the Emotional Symptoms scale, and also no significant main effect for between group differences at the p < .05-level. There was a substantial main-effect for time on the parent-report of this scale, Wilks Lambda = .65, F(2, 14) = 3.80, p < .05, partial eta squared = .35, with both groups showing significant changes for internalizing problems over time. Paired samples t-tests were conducted to more closely examine the change over time for each group along the internalizing problems variable. The intervention group yielded a significant improvement for parent report of internalizing behaviors from pretest (M = 2.33, SD = .48) to follow-up (M = 2.73, SD = .21), ), t (10) 61 = -2.91, p < .05. The mean change in pretest to follow-up scores was -.40 with a 95% confidence interval ranging from -.70 to -.09. The eta squared statistic (.46) indicated a large effect size. A significant change was also observed for the intervention group from posttest to follow-up at the p < .05-level (see Table 6). The control group yielded no significant mean differences over time for parent report of internalizing behaviors (see Table 6). Intervention Effectiveness on Family Management Practices Positive parenting. A mixed effects repeated measures analysis of variance was conducted to measure the longitudinal differences by group assignment on a variety of positive parenting measures across parent and therapist observational report. Significant effects were found for therapist observational report on the Family Assessment Task (FAST) Positive Parenting measure. Univariate and multivariate assumptions of normality, sphericity, equality of error variances, equality of covariance matrices were tested to ensure no violations to assumptions would impact the interpretation of findings. There was a substantial main effect for time on the therapist-observational report of this scale, Wilks Lamda = .58, F(1, 15) = 10.83, p < .01, partial eta squared = .42, with both groups showing significant changes in observations of their positive parenting over time. No significant effects were found for the interaction of time and group or for a main effect of group at the p < .05-level, which suggests that there were no differences between groups for observations of positive parenting. However, results from paired samples t-tests indicated a significant improvement for the intervention group in observed positive parenting from pretest (M = 3.57, SD = .63) to posttest (M = 4.20, SD 62 = .42), t (10) = -4.30, p < .01. The mean change in pretest to posttest scores was -.63 with a 95% confidence interval ranging from -.95 to -.30. The eta squared statistic (.67) indicated a large effect size. Neither the parent reports on the same FAST positive parenting scale or the parent reports on the parent-child conflict scale yielded significant effects on the mixed effect repeated measures ANOVA at the p < .05-level. Results for the mixed effects repeated measures ANOVA on all scales of positive parenting can be found in Table 5. Parental limit setting. A mixed effects repeated measures analysis of variance was conducted to measure the longitudinal differences by group assignment on a variety of limit setting measures by parent and therapist observation report. A negative parenting measure was administered to parents at all three time points, a positive limit setting measure was administered to parents at pretest and posttest, and the FAST limit setting measures were administered to parent and therapists at pretest and posttest. Univariate and multivariate assumptions of normality, sphericity, equality of error variances, equality of covariance matrices were tested, and no violations of these assumptions were evident for parental limit setting. Significant results were found for the parent-report negative parenting measure and the therapist observed limit setting measure. There was a low main effect for between group differences on the parent-report of the negative limit setting, Wilks Lamda = .13, F(1, 15) = .25, p < .05, partial eta squared = .02, with the means in the intervention group showing more substantial changes from pretest to posttest and follow-up. This suggests that there was a significant effect of receiving the Family Check-Up on parents’ use of parents’ negative limit setting behaviors. There was also a 63 trend-level interaction between groups across time, which was likely due to the change in mean differences from posttest to follow-up for both groups. Results from the paired- samples T-tests indicated that only the intervention group yielded a significant mean difference from pretest (M = 2.79, SD = .51) to follow-up (M = 2.29, SD = .37), t(10) = 4.74, p < .01. The mean change in pretest scores to follow-up scores was .50 with a 95% confidence interval ranging from .27 to .73. The eta squared statistic (.71) indicated a large effect size. Refer to Table 5 for results from the mixed effects repeated measures ANOVA and Table 6 for results from the paired-samples T-tests. Refer to Figure 1 (see Appendix B for all figures) for an illustration of the trend-level interaction for parent report of negative limit setting. Results from the parent-report of positive limit setting and from the parent- reports of the FAST Limit Setting measures did not yield significant effects at the p < .05-level for differences between groups or over time. However, results of the mixed effects repeated measures ANOVA of the therapist observation of the FAST limit setting activity yielded a substantial interaction effect between groups across time for limit setting, Wilks Lamda = .54, F(1, 15) = 12.63, p < .01, partial eta squared = .46. These results suggested that there was a significant impact of receiving the intervention on the change in observed limit setting skills from pretest to posttest (see Figure 2). An examination of the means in both the control and intervention group revealed that the intervention group showed significant improvements in observed limit setting skills after receiving the intervention, whereas the control group actually declined in limit setting skills from pretest to posttest (See Table 4). 64 Parental Monitoring. A mixed effects repeated measures ANOVA was conducted to measure the longitudinal differences by group assignment on a variety of different parental monitoring measures, including (1) the parent-report parental monitoring scale, (2) the parent and therapist-observed report of the FAST parental monitoring task, (3) the parent-report problem solving scale, and (4) the parent and therapist-observed report of the FAST problem solving task. Univariate and multivariate assumptions of normality, sphericity, equality of error variances, equality of covariance matrices were tested, and no violations of these assumptions were evident for parental monitoring. Significant effects at the p < .05-level were not found for the parent or therapist reports on the FAST parental monitoring scale, the parent-reports on the problem solving scale, or the parent-reports on the FAST problem solving scale (see Table 5). A small main effect of between group differences was found on the parent-report of the parental monitoring scale, Wilks Lamda = .12, F(1, 15) = .20, p < .05, partial eta squared = .01. An examination of the means differences indicated that both groups endorsed higher means over time in parental monitoring, but that the control group yielded significantly higher scores on parental monitoring over time than the intervention group. No significant main effects on time or interaction effects were found at the p < .05-level for this variable. On the other hand, a trend-level interaction between groups and time was found for the therapist observational report on the FAST problem solving activity, Wilks Lambda = .79, F(1, 15) = 3.95, p = .07, partial eta squared = .21. Results on this measure revealed improvement in problem solving abilities for the intervention group from pretest (M = 3.17, SD = .81) to posttest (M = 65 4.04, SD =.88), whereas the control group was observed to decline in problem solving abilities from pretest (M = 3.63, SD = .58 ) to posttest (M = 3.53, SD = .73) observations. See Figure 3 for an illustration of the trend-level interaction for observed problem solving skills. Intervention Effectiveness on Parent Motivation to Change The fourth aim of this study was to measure family engagement and motivation to change based on random assignment to the FCU. This aim was examined by measuring if assignment to the FCU condition accounted for variance in family engagement and motivation to change. It was hypothesized that families who engaged in the FCU will show higher motivation to change than families assigned to the FCU but who do not engage in the intervention, and families in the control group condition. Group differences in parent engagement in the intervention study was not measured using the originally planned analyses. No attrition occurred during the course of the study for participants who provided consent for participation. Only two families who had originally expressed interest decided not to participate. Both families dropped out prior to the pretest data collection. Both families reported that their schedules would not permit their complete participation and therefore preferred to not participate at all. All 17 families who consented and provided pretest data also completed posttest and follow-up measures. A mixed effect repeated measures ANOVA was conducted to measure the longitudinal impact on motivation to change by group assignment. There was no significant interaction between group assignment and time for motivation to change, Wilks Lambda = .24, F(2, 13) = 2.23, p = .15, partial eta squared = .26. There was a 66 substantial main effect for time, Wilks Lambda = .56, F(2, 13) = 5.03, p < .05, partial eta squared = .44, with both groups showing significant changes in motivation to change over time. The main effect comparing the intervention and waitlist-control groups was not significant, F(1, 14) = 1.83, p = .26, partial eta squared = .09, suggesting no difference in receipt of the intervention on motivation to change. Post-hoc paired sample t-tests were conducted to better understand the significant effect on motivation to change for both groups over time. Results from the t- tests indicated that the intervention group showed significant effects for motivation to change from the screening (M = 3.95) to posttest (M = 5.15), t (9) = -3.91, p < .01. The mean change in motivation to change scores was -1.20 with a 95% confidence interval ranging from -1.89 to -0.51. The eta squared statistic (.66) indicated a large effect size. T-test results from the intervention group from posttest to follow-up measures indicated no significant changes in motivation to change, which suggests that the change after receiving the intervention was maintained from posttest to follow-up measurement. These findings suggest that the intervention group’s motivation to change decreased after receiving the intervention, indicating that they had initially endorsed scores closer to “I want to change in ______ area of parenting” to endorsing scores closer to “I feel good about this area of parenting” after receiving the intervention. The waitlist control group showed no significant mean differences in motivation to change across time points. Overall results indicate that participants were highly engaged in the project and evidenced no attrition after they had consented to participation. Furthermore, participants on the waitlist control group all opted to participate in the intervention once 67 they had completed data collection. These findings suggest that the participants of this sample were highly engaged in their desire to participate in the intervention program that was offered in this study. Additionally, findings from the “motivation to change” measure indicated that participants in the intervention group were more likely to endorse changes in attitude related to “motivation to change” parenting behaviors. However, the decline in motivation to change that was evidenced by the intervention group was not the expected result, as the original aim was to determine whether receiving the intervention could increase motivation to change. Feasibility of the Family Check-Up in Spain The last aim of this study was to measure the fidelity and uptake of the Family Check-Up model in Spain. By collecting data from therapists and interventionists on model feasibility in this context, this aim was examined through a combination of quantitative and qualitative data from participants and the advisory committee. Fidelity of Model Implementation. The fidelity of the treatment model and intervention study was measured in a variety of ways, including fidelity check-lists, reliability of coding systems, supervision of therapists in their conceptualizations of feedback sessions, and through therapist self-evaluation using the COACH model in conjunction with review of videotapes. Fidelity check-lists were completed by interventionists after each of the pretest, posttest, and intervention sessions. Interventionists documented that they had completed each step of the research phase as well as core aspects of the intervention model that needed to be completed for intervention families (i.e. use of Motivational Interviewing during the initial interview at pretest). Interventionists also documented the time that each session took to complete. 68 The principal investigator tracked the dates that families completed each step and informed the interventionists when it was time for families to complete the next phase of participation. There were certain aspects of the training that were ongoing and were used to ensure intervention and research fidelity. After training interventionists in the Family Assessment Task Coding System, interventionists were required to submit their coding of the FAST assessments to the principal investigator for reliability comparisons. Meetings were conducted when discrepancies in coding were detected in order to discuss issues of cultural reliability and consistency. Additionally, interventionists completed a conceptualization sheet prior to each feedback session that would be discussed with the principal investigator. Conceptualizations incorporated checking for accuracy of placement of pretest data onto the feedback profile and discussing the data- driven narrative that would support therapists’ successful delivery of feedback to parents. Discussions often centered around how family data was consistent or not with existing research, and how to help parents create goals that utilize their strengths to address their problem areas. Conceptualization meetings comprised a fidelity check because interventions were being supervised by the same principal investigator to ensure that conceptualization within the core aspects of the Family Check-Up were consistent with existing research and within the sample. Finally, interventionists were required to review videotapes of each feedback session and evaluate themselves using the COACH model that was developed for the Family Check-Up intervention. Interventionists also met with the principal investigator to review feedback videotapes and discuss their COACH ratings in order to ensure 69 adherence to the model and accurate self-evaluation on the COACH. The COACH is rated on a 9-point scale (1 – 3 = Needs Improvement, 4 – 6 = Good Work, and 7 – 9 = Exceptional) across 5-domains (Knowledge of EcoFIT model, Observes and Actively Structures Session, Carefully Teaches and Provides Feedback, Health and Strength- based and Motivation, and Results). Interventionists completed a total of 12 COACH rating scale, which included one COACH that was scored for a waitlist-control family who received feedback after completing data collection. Overall, the mean COACH ratings fell in the “Good Work” category (M = 6.77, SD = .86) with a minimum of 5.6 and maximum of 8.0 (see Table 7 for all COACH ratings). The most highly evaluated domains of the COACH were the “Knowledge” domain (M = 7.17, SD = .72) and the “Results” (M = 7.17, SD = .84). The area that was evaluated the lowest was the “Teaching and Providing Feedback” domain (M = 6.42 SD = 1.17), which was still in the “Good Work” range. Overall, results indicated that the core aspects of the COACH model were adhered to in a satisfactory manner, suggesting the adherence of intervention fidelity. Evaluation of Model Uptake. The uptake of the Family Check-Up into the Spanish context was measured using a survey that was created for this purpose (see Form N, Q, and R). Family participants in the study completed the program evaluation as part of their follow-up data point. A second evaluation questionnaire was distributed to the principals of the schools that participated and to members of the advisory committee, which also included the two interventionists who were trained to provide the intervention. The survey was distributed 6 weeks after completing data collection and members were provided with a two-month time period to return it. Five surveys were 70 returned with data. One principal reported feeling that they did not know the study well enough to accurately report on the items and thus did not return a completed survey. Two other committee members did not return the survey within the time that was allotted for them to fill it out. All advisory committee members reported on their impressions of the materials utilized in the study, the types of interaction employed with the participants, the context of intervention application, and they were also asked to provide any additional comments and suggestions not represented in the aforementioned categories. Demographics of the advisory committee members, interventionists, and school personnel who completed the survey were analyzed. Individuals returning the survey included one doctoral-level psychologist, two licensed psychologists (one with an additional masters credential), and two educators with a diploma in that field. Individuals reported being in their current position for a minimum of three years to a maximum of seven years, but their years in their current profession ranged from five to more than ten. Individuals returning the evaluation survey endorsed identifying with a range of theories, including ecological/systemic (n = 3), cognitive-behavioral (n = 3), behavioral (n = 1), transtheoretical (n = 1), and constructivism (n = 2). Family participants completing the survey were consistent with the demographics provided previously (Intervention group: n = 11; Control group: n = 6). Respondents evaluated the program with scaled measures and open-ended questions according to four broad domains (training, materials used, interactions with participants, and research design). The first domain for evaluation was training. Interventionists were asked to respond to open-ended questions that addressed aspects 71 of the training that helped increase or decrease their comfort level in providing the intervention, increase or decrease their confidence in providing the intervention, and increase or decrease their self-efficacy in providing the model. They were also asked in which areas they needed more training and in which areas did they receive too much training. Responses were coded for themes and reported only if a minimum of two respondents endorsed the same theme. Interventionists indicated that the aspects of the training that increased their comfort with the model were the training workshops and the presence of the principal investigator at the initial meetings for each phase. Aspects of the training that increased their confidence included watching and evaluating the videotaped sessions. Interventionists responded that their self-efficacy was increased in training by receiving supervision when making the feedback conceptualizations and reviewing videotapes. They reported that there was no aspect of training that was too extensive, but they reported that they would have benefited from understanding the structure and flow of the feedback sessions prior to conducting the first feedback meetings. The second domain for program feasibility was the materials employed by the study, and was evaluated by both the family participants and the advisory committee. All participants and members on the advisory committee responded to scaled questions about the usefulness of the following aspects of the study: filling out questionnaires, conducting video taped family discussion tasks (FAST), the initial interview for intervention group families, and the feedback session for intervention group families (see Table 7 for evaluation ratings). Items were ranked on a 4-point scale (1 = not useful, 2 = a little useful, 3 = somewhat useful, and 4 = very useful) and responses can 72 be found in Table 9. Participants believed that the questionnaires were the least useful aspect of the study (M = 2.81, SD = .85), followed by the FAST (M = 2.95, SD = .79), the initial interview (M = 3.18, SD = .75), and the feedback session (M = 3.62, SD = .62). The interventionists and families shared the same order as the total group means, but the non-interventionist advisory committee members believed that the FAST was the least useful aspect of the study, followed by the questionnaires, initial interview, and finally the feedback session. Advisory committee members, including interventionists, were given open- ended survey questions that requested responses to the usefulness of the documents in the study, the cultural appropriateness of the documents in the study, and the usefulness of the materials for them in their role in the project. Themes were derived from advisory committee open-ended responses and were reported if two or more individuals indicated a similar theme. Qualitative themes were consistent with the quantitative trends that had placed a higher value on the feedback session and interpersonal interviews. Several respondents (n = 3) reported that all materials used in the study were culturally appropriate, while others discussed the extensiveness of the consent form as an aspect of the study that was less common in Spanish culture (n = 2). There were mentions of the “authorization to release school information” as being unnecessary to the study and that it should have been included as part of the broad consent form. Advisory committee members discussed the utility of collecting multimodal and multi-source survey data, reporting that this increased their confidence in the validity of the conclusions. See Table 7 for Model Usefulness evaluations. 73 The different types of interaction with participants was also addressed in the implementation questionnaire that advisory committee members (n = 5) completed. Themes that emerged included a positive response to in-person interviews that involved direct interaction with the interventionists (n = 4). Specific examples were provided by committee members about their appreciation of direct interviews and of interpersonal procedures, such as the recruitment meetings. A theme about the extensiveness of the questionnaires emerged through this question, as advisory members reported that nature of the questionnaire task may have detracted from the personal aspects of the program that had originally engaged parents. Advisory members (n = 2) discussed similar themes when asked about the usefulness and cultural appropriateness of the research design. They reported that the randomized, longitudinal design that included surveys and interviews was appropriate and typical for Spain; however, they found the posttest and feedback sessions to be the most useful for families involved (n = 2). A theme about families’ lack of knowledge of the research design may have contributed to some families questioning why they had to participate in multiple data collection points. Additionally, the theme of participant payment was addressed as an aspect of this project that raised questions for families since families do not typically expect payment for research participation. Advisory committee members (n = 5) responded both quantitatively and qualitative to the overall usefulness of the model for accomplishing proposed objectives. Quantitative items were scored on a 3-point scale (1 = not true; 2 = somewhat true; and 3 = certainly true). Respondents endorsed a high likelihood that they would continue participating in the intervention if given the opportunity (M = 2.80, 74 SD = .45). They believed that the Family Check-Up was a useful tool for helping parents to set parenting goals (M = 2.80, SD = .45), and they reported that the model is certainly useful when used as a guideline to help families achieve their parenting goals (M = 3.00, SD = .00). When asked to respond with additional comments about the project, themes among advisory members emerged that revolved around the usefulness and positive impact of the program. One school-based advisory committee member stated, It is a well-designed program, being based in the personal interviews, I think it greatly benefits the relationship between parents and child when the results are given to the families. This committee member discussed the effectiveness of the program’s feedback model in supporting that positive growth of the parent-child relationship. The desire for continued involvement and program implantation in Spain was further supported by other advisory committee members. Another school-based administrator and advisory member wrote, Research projects like this should be inserted into school plans for all elementary schools. If they are carried out in a correct form they can be the solution to the majority of conflicts that, today in our society, make the practice of education so difficult and sometimes impossible. 75 Family participants and advisory committee members were asked to rate their impressions of the model related to its effectiveness in improving behavior problems in children and as a parenting resource, and items were scored on a 3-point scale (1 = not true; 2 = somewhat true; and 3 = certainly true). Overall, respondents (n = 22) felt that the intervention was most useful for improving child behavior problems (M = 2.95, SD = 2.91), but they also felt it was helpful as a parenting resource (M = 2.82, SD = .40). Interventionists and other advisory committee members responded unanimously to both items, reporting that they certainly agree with the model’s utility in improving behavior problems and as a parenting resource (M = 3.00, SD = .00). Additionally, both family participants and advisory members reported that they would certainly recommend the program to others as a parenting resource (M = 3.00, SD = .00). Their quantitative indications were confirmed by write-in comments by advisory board members and participants that discussed themes about the desire to implant the program in Spain and their personal observations about improvements in their family dynamics and child’s behavior. One interventionist addressed the utility of model implementation in Spain, with special attention on the strength-based and preventative aspects of the program, I think that you should keep working to implant this program in Spain and to continue developing its existence. Well, I think it is a very useful tool for many families, regardless of whether they have problems or not. I think that this program strengthens parents, helps children understand their role in the family and all 76 across dynamics and actions that are very comfortable, equally for the families as for the professional that is working with them. Parent participants discussed their overall satisfaction with the model. Some parents and interventionists provided suggestions that developed into a theme of wanting more follow-up to support them with their goals. One parent wrote, I liked it a lot. I found it interesting and it has helped make me aware of some of the things that I consider myself to be improving in. I think it would be interesting if you could organize ‘Parenting Schools” that focus on the themes that we saw in these sessions. Other parent participants addressed a theme about the direct impact of the study on their child’s behavior. One parent from the control group wrote, “Participation for my son made him assume a commitment, something that I see as very favorable and that made him become more aware.” This parent’s comment supports the idea that participating in a structured intervention, even as control participants, can impact the child’s behavior – noting an observation about an increase in responsibility and self-awareness. This theme of behavioral improvement due to the intervention participation was further supported by an intervention group parent who noted, “I am very happy. The dynamic and form of intervening with her, she liked a lot. I have taken note of our closeness.” 77 In summary, a review of the quantitative and qualitative findings about the feasibility of implementation of the Family Check-Up in Spain suggests that the uptake of this model was successful. Parents and interventionists rated the overall model utility and effectiveness highly, indicating that the model was appropriately applied within the cultural context. Additionally, parents and advisory members continuously noted that the model would be helpful in addressing many of the problems that arise within the school setting due to issues at home. Respondents unanimously supported the school as the most appropriate context for implementing this intervention, since it is easily accessible for parents and further supports home-school exchange and involvement. Themes suggested that with modifications made to consent forms, the extensiveness of questionnaires, and the possibility for follow-up, they would recommend for this model to be further tested and implemented within their Spanish elementary school context. 78 CHAPTER IV DISCUSSION This study examined the feasibility and effectiveness of the Family Check-Up parenting intervention within a sample of 17 families with children ages 9 to 12 in the greater Seville metropolitan area in Spain. The process of adaptation of the evidence- based program was documented and modifications to the intervention model were examined. Additionally, descriptive information for youth adjustment, family background and support, and family management were analyzed to better understand trends within this cultural sample. Intervention effectiveness on youth adjustment, family management skills, and parental motivation to change was examined using an experimental randomized waitlist-control group design. Finally, the feasibility and uptake of the intervention model in this context was measured by examining intervention fidelity and family engagement, and through quantitative and qualitative data from program participants and advisory committee members. Embedding the Family Check-Up in the Spanish Culture The Family Check-Up model was successfully embedded into the Spanish cultural context with the support of an advisory committee, consisting of university faculty and research assistants, school personnel, and a community member. The name of the Family Check-Up and all research materials needed to be modified not only for direct English to Spanish translation, but also for accuracy of meaning within this specific cultural context. These modifications were effective given that the name, “The Family Support Program,” attracted participants who were interested in a strength- based, family intervention program and who were prepared to participate in a 79 collaborative program to support improvement in child behavior and family functioning. Measures were also modified to reflect the linguistic nuances of the southern Spanish culture. The fact that participants were able to complete questionnaires with full understanding suggests that the items in the measures are appropriate to be used within this context. However, the cultural connectedness scale was determined to be invalid within this culture. Participants were confused by the questions, which they perceived to be targeting Spanish nationalism (a perceived negative concept) in certain items and family unity in other items (a perceived positive concept). Many families’ results suggested that they were at high-risk for cultural connectedness, meaning that they were not connected to their culture – a construct that in the U.S. has represented a risk factor for minority families. However, in discussions that occurred during feedback session, families believed that they were connected to their culture and they did not agree with the high-risk categorization. Families in southern Spain commonly follow traditional cultural norms that, similar to the research on cultural connectedness, could likely serve as a protective factor against youth problem behaviors (Inman, Howard, Beaumont, & Walker, 2007). However, the topic of Spanish nationalism as linked to the history of dictatorship within Spain, and often was discussed within this scale. This suggests that cultural connectedness remains as an important construct for Spanish families, but that the scale items should be modified to reflect factors within the cultural norms that would more accurately represent protective factors against youth problem behavior. Given the history of national trauma experienced within the Franco dictatorship, future researchers 80 may consider including items that reflect a family’s ability to engage in dialogue that help prepare children to understand historical trauma within this culture and protect against the effects of this type of trauma (Kellermann, 2001; Brave Heart, 2003). Additional items may also target common cultural traditions that lead to family shared enjoyment and passing on of traditions, such as participation in regular meal times and the siesta break, and participation with their family in festivals, such as the “Feria de Abril” or passing on cultural dances, such as “Sevillanas” that are typical within the province of Andalucia. The majority of items within the youth adjustment and family management domains were deemed to be appropriate. Parents seemed to identify and relate with items that targeted their child’s behavioral and emotional adjustment, as well as items that targeted family management practices. Certain items, such as those that addressed discipline practices may benefit from further modification to more accurately reflect common parenting practices within this culture. For example, research staff could not find a culturally accurate translation for the concept of “time out.” They reported that this concept was not typically used in Spain, and that parents more commonly send children to their rooms for punishment. Research that has been conducted to validate parenting measures in Spain also lead to recommendations that measures need to be adapted to more accurately reflect the interactions between parent and child (Samper, Cortés, Mestre, Náchar, & Tur, 2006). As such, future research would benefit from further study of the discipline practices employed by Spanish parents - to both understand more completely the practices most commonly used, and to adjust the norms to be more appropriate for this context. 81 Similarly, the construct targeting ethnocultural group identification was an item that created much discussion within the advisory board and some confusion for participants. The advisory board discussed the implications of asking participants to identify their ethnic group, expressing concern that participants could feel negatively targeted if asked to disclose a minority ethnic group from which they may belong. Research on ethnic group differences is a relatively new area of study in Spain considering that high rates of immigration is only a recent phenomenon within the last two decades since the fall of communism in Europe and the Spanish dictatorship (Ribas-Mateos, 2004; Organisation for Economic Co-operation and Development [OECD], 2011). Existing research on ethnic group educational disparities among immigrant populations has found that immigrant children tend to perform lower than native born peers in the school setting (Azzolini, Schnell, & Palmer, 2012). Given the recent rise in immigration related-issues coupled with evidence of educational disparities between immigrant and native children in Spain, research that explores protective factors against youth problem behavior for minority ethnic groups may help to improve the resilience of youth and families from marginalized groups within this culture (OECD, 2011; Azzolini, et al., 2012; Phinney, Romero, Nava, & Huang, 2001). Yet, the fact that Spanish participants tended to respond with the same response for the nationality and ethnicity items suggests that researchers should begin by educating the general population about ethnocultural groups. It is recommended that education begin by discussing how ethnic groups differ from national or racial groups, and the implications of what it means to have varying ethnic identities within the Spanish society. Once families have a complete understanding of the construct of ethnicity, then 82 youth problem behavior and family management research may benefit from expanding the research on protective parenting practices for different ethnocultural groups. There were two aspects of the Family Check-Up model that required extending beyond the advisory committee to seek support from other entities within the community infrastructure. First, schools were recruited to inform the process of family recruitment and the logistical aspects of the study, such as how to best communicate with family participants. Second, school personnel and community psychologists were recruited to inform the Menu of Options that was provided to families in the intervention group as part of the feedback session. As expected, the participating schools and practicing psychologists were the groups that best understood the community sample, their desires for services, and the available resources. Their support and ideas provided credibility for the program and research study among participants, such that participants would often consult with participating school staff prior to making a final decision about their participation in the study. Similar to community participatory action research that recommends for researchers to collaborate with community leaders to build trust with community members, these collaborations in Spain lead to the ability to recruit enough participants and to deliver services that were in sync with the culture of the local communities (Prilleltensky, 1997, Nation, Crusto, Wandersman, Kumpfer, Seybolt, Morrissey-Kane, & Davino, 2003). The Family Check-Up and Spanish Youth Adjustment Effectiveness of the Family Check-Up intervention on youth problem behavior and internalizing behaviors was measured. Parents reported that youth problem behaviors declined for the entire sample from pretest to posttest and those changes were 83 maintained at follow-up data collection points. Post-hoc analyses showed that only children in the intervention group evidenced significant declines in problem behavior. The main effect that was found for the entire sample was likely due to the fact that the intervention group was almost double the size of the control group, suggesting that the intervention group contributed more substantially to the significant main effect. The significant reduction in problem behavior that was evidenced in the intervention group in the post-hoc tests, even within this small sample size suggests that this FCU intervention model positively impacts youth problem behavior. Findings suggest that post-intervention, children in the intervention group engaged in fewer problem behaviors, such as lying, stealing, or destroying property on purpose. Conversely, parents in the control group noticed no changes in their children’s behavior problems. Enhancing family management skills, such as limit setting, positive parenting, and parental monitoring, is an effective strategy to reducing youth problem behaviors (Dishion & Stormshak, 2007). Given that the feedback sessions were predominately focused on making links between family management practices and the impact on youth adjustment, it was expected that the youth in the intervention group would show reductions in problem behaviors. The most common focus in the feedback sessions was on the use of positive behavior reinforcement and using limits to help teach children how to remain on a positive behavior trajectory (Dishion, Stormshak, & Siler, 2010). These results indicate that making this link for parents about positive parenting and limit setting within the Spanish cultural context yields similar reductions in problem behaviors as in U.S. samples (Stormshak, Margolis, Huang, & Dishion, in press). 84 Parents’ reports of their children’s internalizing behaviors yielded findings similar to the outcomes for youth problem behavior. The main effect of time suggested that the entire sample improved in internalizing behaviors or emotional adjustment, meaning that their parents’ noted fewer incidents of sadness, depression, nervousness, or crying in their children at time of post-test. However, post-hoc analyses indicated that only the intervention group made significant reductions in internalizing behaviors, and the main effect of time was likely due to the larger size of the intervention group. It is widely accepted in developmental research that increasing parental warmth and affection or positive time spent with their children, even in small doses can positively impact self-esteem, which is associated with reductions in internalizing behaviors and increases in positive emotional adjustment in youth (Hiatt Racer & Dishion, 2012; Bulanda, Majumdar, 2009). These results are consistent with the existing research given that the model focused on enhancing the parent-child relationship, especially parental warmth and enjoyable family activities. In fact, several intervention group parents commented in posttest sessions that the feedback goal they had most enjoyed working on was increasing positive shared activities with their children. Qualitative themes from parents also endorsed the positive emotional adjustment they had seen in their children, simply from participating in the intervention study together. The Family Check-Up and Family Management in Spain Intervention effectiveness on positive parenting, parental limit setting, and parental monitoring were measured by therapist observation from pretest to posttest and by parent report from pretest to posttest and follow-up. The entire group showed improvements in observed positive parenting skills, which indicated that parents were 85 more supportive of their children’s goal setting and provided more verbal praise. Post- hoc analyses revealed that the intervention group made significant improvements in positive parenting but the control group did not. These results were expected given that existing research has shown the Family Check-Up to positively impact positive parenting behaviors (Dishion, Stormshak, & Siler, 2010). However, there were no significant improvements in positive parenting from the parent-report measures. Research has shown that observational reports are a valid form of measurement that may be more effective in detecting changes over a short period of time (Dishion, Nelson, & Kavanagh, 2003; Van Ryzin & Dishion, in press). As such, it is not surprising that therapists observed significant changes in positive limit setting given only a 4-week lapse from pretest to posttest, whereas parent reports did not indicate the same results. Parental limit setting yielded positive improvements for two different measures, the parent self-report negative limit setting measure and the therapist observation of parental limit setting. The intervention group parents reported that they used less negative parenting than control group parents after receiving the intervention, meaning that parents used less punitive means of setting limits (e.g. yelling at children or giving a punishment that they could not follow through with). This finding of group differences and the trend-level interaction of group differences over time for parent self- report was unique given the short time from pretest to posttest (4 weeks) and follow-up (1 week after posttest). Additionally, a significant interaction was observed in therapist observations of parental limit setting behaviors, suggesting that the intervention group parents demonstrated more positive limit setting skills from pretest to posttest than 86 control group parents. These skills consisted of the ability to define a clear limit and discuss both the limit and consequence in a neutral tone that is not punitive or critical of the child. Similar to results from the positive parenting outcome, it is not surprising that the observed limit setting behaviors improved from pre- to posttest; however, the fact that parent-report measures of limit setting also yielded significant effects suggests that the Family Check-Up intervention model was especially effective in targeting parental limit setting in a positive way. Parental monitoring was the final aspect of family management that was examined for intervention effectiveness. Two different measures of parental monitoring yielded significant results: the parent self-report parental monitoring measure and the therapist observation of family problem solving skills. Surprisingly, the control group showed improvements in the parental monitoring parent-self report measure more than the intervention group. This was not an expected finding, given the previous research showing that the Family Check-Up helps to increase parent report of their monitoring behaviors (Van Ryzin & Dishion, in press). Further research is needed to better understand why parents who did not receive the intervention reported improvements in their monitoring skills. One idea to investigate further would be that the overall sample evidenced high rates of parental monitoring, with children spending little time unsupervised by adults. Alternatively, it has been suggested that parental monitoring may serve as an outcome of higher problem behavior, such that children with more problem behavior need more monitoring (Van Ryzin & Dishion, in press). The present study’s findings are consistent with this theory since control families reported higher 87 monitoring scores without receiving the FCU and their children’s problem behavior increased more than the intervention group. The second measure of parental monitoring was family problem solving behaviors, which is an aspect of parental monitoring that involves parental involvement to help children solve both family and school-related issues. Therapists in the study observed that parents in the intervention group made significant improvements in demonstrated family problem solving behaviors, whereas families in the control group declined in these skills. Findings suggest that the Family Check-Up intervention helped parents to utilize neutral problem solving abilities to reach a functional solution rather than criticizing their children and becoming stuck in the problem identification stage. These results are consistent with existing research that supports intervention effects on observational outcomes and with previous FCU research that has demonstrated effectiveness in problem solving abilities (Dishion, Nelson, & Kavanagh, 2003). In summary, intervention group parents showed improvements in positive parenting, parental limit setting, and parental monitoring as a result of receiving the intervention. While not all measures of these constructs had significant results, there was at least one measure from each domain that did yield positive significant findings - suggesting that the improvements for the intervention group were likely due to receiving the FCU intervention. The fact that receiving the Family Check-Up impacted parents’ skills in positive parenting, limit setting, and monitoring after receiving one dose of the intervention, and that changes were observed over a short time period not only confirms existing findings from previous FCU research, but also reaffirms that the intervention is effective in targeting family management practices when applied in a 88 new cultural and linguistic setting. The model is designed to be flexible so that it can be tailored to diverse multicultural and family value systems; however, this is the first time that this model has been tested within a different country using a homogenous Spanish- speaking sample. As such, findings from this study provide additional support for the model’s multicultural sensitivity and effectiveness within diverse cultural samples both domestically and internationally. Motivation to Change Results from the parental motivation to change revealed that parents in both the intervention and control groups showed significant decreases in their motivation to change from pretest to post-test and follow-up. This was an unexpected result given that targeting parental motivation to change by using motivational interviewing is a core aspect of the Family Check-Up model (Stormshak, Margolis, et. al, in press). This is an important construct targeted by the model because research has shown consistently that parents are the most important agent of change on child adjustment and behavior problems (Dishion & Stormshak, 2007). There are several reasons, however, why parents in this study may have endorsed a reduction in motivation to change. First, parents may have felt more in need of help at the study onset than upon completion. Second, parents may have gained more self-efficacy and confidence in their parenting as a function of the project forcing them to examine their parenting from various different perspectives and through different activities. Specifically, the first time that parents were asked about their motivation to change was during the screening telephone interviews. It is possible that parents endorsed higher motivation to change at screening as a result of the family management 89 questions in the PARCA being novel to them, and this novel effect may have warn off by the time of follow-up. Another factor of parents endorsing higher motivation to change at screening may have been a function of parents wanting to be accepted into the study, and therefore making the assumption that endorsing a need for help or desire for change at time of the screening would lead to acceptance of their participation. The question was worded as: “Thinking about the items above, would you like to do things differently in this area of parenting?” with a high motivation to change as “I want to change” and a low motivation to change as “It is fine as it is.” It is possible that this wording did not accurately capture the concept of motivation to change within this sample. Given the quantitative results indicating parents’ recommendations from the model and their desire to continue participating coupled with qualitative themes asking for additional follow-up opportunities to continue the intervention, the latter possibility is the most likely candidate. There are many reasons why parents may have responded more closely to their parenting in a certain areas as being “fine as it is,” including the desire to appear that they have learned something in the project or to reduce potential shame they could experience in front of the interventionists whom parents came to know well through the course of the study. The contradiction in qualitative themes suggesting parents’ increased motivation to change and the quantitative item that designed to specifically measure this construct showing reduced motivation to change, suggests that further research is needed to determine the methods that could more accurately measure this construct. Researchers may consider using language that more directly targets parents desire to change, by using the word “motivation” or by asking 90 parents to compare their desire to change certain parenting skills from pretest to posttest. Feasibility & Uptake of the FCU in Spain The feasibility and uptake of applying the Family Check-Up model within the southern Spanish context was examined through a variety of measures, including: intervention fidelity, participant engagement, and quantitative and qualitative surveys that were given to participants and advisory committee members. Results from all measures revealed that the intervention model was implemented with fidelity and that the implementation of the model was viewed positively by all individuals involved. Treatment fidelity. The treatment fidelity measures that were completed included checklists to ensure that all procedural steps were carried about appropriately and therapist self-evaluations about their adherence to the intervention model when delivering the feedback sessions. Overall, findings were positive, suggesting that high adherence to treatment fidelity. Given that intervention effects were found for the intervention groups across all hypothesized domains, it is important to note that those findings occurred within the limits of adherence to treatment fidelity. Additionally, fidelity was maintained despite the cultural adaptations that were made to research materials. As such, it is reasonable to conclude that the intervention effects are due to a culturally sensitive delivery of this particular intervention model and not due to adaptations that changed core components of the model. FCU feasibility in Spain. Feasibility of the model in Spain was measured by questionnaires evaluating the materials, the usefulness of the model, and the interest of participants and committee members in evidence-based practices and continuing 91 participation with the intervention. Quantitative findings indicated that participants in the sample found the materials to be useful, and that they most preferred materials that involved direct interpersonal contact. Similarly, advisory committee members found the feedback session and materials to be the most useful aspect of the study. These findings were complimented by thematic statements that described the utility and appreciation for the feedback model in enhancing parent-school collaboration, helping parents to see their strengths, and improving family and youth difficulties. Participants and advisory committee members found the model to be very useful in improving behavior problems and parenting skills, and all reported that they would recommend others to participate in the intervention. Themes from open-ended questions ranged from endorsements of the intervention as an enjoyable experience, to noticing improvements in children’s behaviors and family relationships, to believing that interventions like the FCU could be a catalyst in improving education-related problems due to parenting and youth adjustment issues. All personnel involved in the study including school staff and interventionists reported that they would be highly likely to continue participating in the model because they believe it to be a useful tool for helping parents to set and achieve goals. School staff members were impressed by the ability to engage parents within the school context to target behaviors that impact their children’s adjustment in a variety of contexts, such as school. Overall, the goals proposed in this research study were met. The intervention was modified slightly to include additional opportunities for building trust among interventionists/research staff and participants and recommendations for adjusting certain scales within the research materials were recommended. Despite these minimal 92 modifications the Family Check-Up was delivered with fidelity, which supports existing research on the cultural adaptability and flexibility that is already inherent in the intervention model. Furthermore, when delivered in a new cultural setting by Spanish psychologists in the Spanish language, the Family Check-Up intervention yielded positive effects to the intervention group for youth behavior and internalizing problems, positive parenting, parental limit setting, and parental monitoring. These results have been confirmed in the United States for over 20 years of longitudinal evidence-based research studies (Dishion & Kavanagh, 2003; Stormshak, Dishion, Light & Yasui, 2005; Dishion & Stormshak, 2007). Most importantly, results from this study indicate that the FCU intervention successfully targets the same outcomes in a different cultural and linguistic setting. The intervention effects on youth adjustment and family management skills coupled with the positive reports from participant and advisory committee evaluations supports the feasibility and effectiveness of the Family Check- Up within this southern Spanish context. Limitations and Strengths The sample size of this study consisted of 17 participants, which was a limiting for statistical analyses and drawing conclusions about the study outcomes. Sample size was limited by recruitment availability and duration. This was a pilot study that was funded by a time limited grant, so resources were limited to hire interventionists and research assistants, which may have impacted recruitment ability and sample size in a negative way. Given the small sample size, repeated measures univariate analyses of variance were used to examine the intervention effects on youth adjustment and family management skills. Running several repeated measures ANOVAs lead to a reduction in 93 power and was not able to account for shared variance among outcomes. Additionally, the low statistical power due to a small sample size prevented covariates from being included in the analyses. The inclusion of pretest scores and gender as a covariate are is a recommended procedure for measuring intervention effects (Mertler & Vannatta, 2005; Silverthorn & Frick, 1999); however, these could not be accounted for within this study. As a result of low power and sample size, many analyses trended toward significant results but significance at a p < .05-level was difficult to attain. Additionally, the results that were significant in this study should be interpreted with caution given the lack of covariates and the inability to conduct a multivariate analysis of variance that included all outcomes in the same analysis (Mertler & Vannatta, 2005). Despite these limitations, there were several aspects of the project that strengthened the validity of the conclusions. The research design and methodology were adhered to with high fidelity, meaning that internal validity was upheld and conclusions should not be attributed to extraneous factors, such as procedural errors. Additionally, random assignment of participants helped to counter the inability to control for differences in gender and pretest scores. The follow-up data collection point increased the validity of conclusions, considering that this data point controlled for anomalies in outcomes that occurred at just one time point. Finally, given that this was a cultural adaptation study that was being piloted in a nation that does not typically conduct intervention research, the fact that a local and culturally representative advisory board and research staff were included to direct the procedural process of the intervention strengthened the cultural validity of the study (Bernal, et al., 1995). The inclusion of Spanish staff in the project made the research project more representative of the Spanish 94 culture, which strengthens the likelihood that future research of this nature may be replicated within the same or other cultural contexts. Recommendations for Future Research, Education, and Policy The findings of this pilot study supported the feasibility of the Family Check-Up model to be delivered in the Spanish language and within the Spanish culture. Additionally, findings supported existing research that the FCU is culturally adaptable and effectively improved youth behavior and internalizing problems, positive parenting, parental limit setting, and parental monitoring. Given the small sample size in the study, further research should be conducted in Spain with a larger sample in order to determine whether the findings can be replicated in a larger scale study. Additionally, researchers should develop and validate measures that more accurately represent the constructs of cultural connectedness, parental monitoring, and motivation to change. Finally, a thorough examination of the therapeutic processes that were applied within this culture would be an interesting contribution to literature about cultural competence in counseling. Intervention research in Spain is limited by the educational system for psychologists in Spain that typically educates either clinicians or researchers. However, this study determined that Spanish researchers and clinicians in this sample were highly interested in evidence-based practices and intervention studies (see Table 7). Spanish psychology faculty members may benefit from including research methodology courses into clinical curriculum or by including counseling skills and theory development into research curriculum. This modification would allow for more testing of the interventions that are occurring within the public and private sector in order to 95 streamline delivery and cost that are based on the most effective and efficient methods. Additionally, the cross-cultural collaboration in this project created opportunities for exchange of education practices and psychological practices between U.S. and Spanish psychology researchers. This type of educational exchange creates more thorough examination of standards of practice through alternative cultural lenses, which inevitably leads to enhanced growth and development as psychologists. Few exchange opportunities exist for psychology graduate students and there are even fewer funding sources to make those exchanges realistic or possible for students with that interest. As such, it is recommended that graduate psychology department heads consider opportunities for including funding for inviting international research students to participate in a residency at U.S. institutions as a mechanism of encouraging these types of collaborations in the future. For example, programs that have Spanish therapy certification components may consider funding an advanced doctoral student from Spain to assist with the course in exchange for that student’s participation in intervention research opportunities. Finally, evaluating the adaptability of existing preventative interventions in international settings has important implications for public policy. The more that opportunities that international researchers have the opportunity to participate in and experience the success of prevention interventions, the more likely they will be to adapt intervention research into common practice within that cultural setting. As preventive intervention programs gain more subscribers and exposure worldwide, it will become easier for intervention researchers to facilitate the uptake of translational prevention research into policy. In this study, Spanish teachers and administrators took note of the 96 potential for preventative, family-centered interventions to contribute to solving macro- systemic issues within the education system. One teacher commented: Research projects like this should be inserted into school plans for all elementary schools. If they are carried out in a correct form they can be the solution to the majority of conflicts that, today in our society, make the practice of education so difficult and sometimes impossible. It is generally accepted that prevention programs are the most cost-effective manner to target conduct and internalizing problems that can have a large financial impact on individuals and society (Mrazek & Haggerty, 1994; Hinshaw & Lee, 2003). Yet, the worldwide economic crisis makes it so that many countries, like Spain, are not prepared or financially equipped to implement prevention programs into policy because they are only financially able to react to the most urgent matters. However, prevention researchers are uniquely positioned to apply grant funding toward international collaborations to enhance the evaluation of preventive interventions. This type of decision-making could have a large impact on advancing our existing knowledge and power within the area of mental health prevention. The scope of the outcomes derived from international collaborations for prevention has the potential to impact multiple nations and cultures as a result of not only publications, but also the impact on the all individuals involved – participants and researchers. 97 APPENDIX A TABLES 98 Table 1 Descriptive Statistics for Youth Adjustment and Demographic Variables Intervention Group Control Group n M SD Min Max n M SD Min Max Child Age in Years 11 10.09 0.94 9 11 6 10.50 0.55 10 11 Males 7 - - - - 4 - - - - Females 4 - - - - 2 - - - - Primary Caregiver – Females 10 - - - - 4 - - - - Primary Caregiver – Males 1 - - - - 2 - - - - Monthly Household Income (€) 10 1850 1017 300 3500 6 2354 1283 426 3700 #Individuals Income Supports 11 3.91 1.14 2 5 6 4.17 0.41 4 5 Parent Perceived Financial Stress 11 2.27 0.65 2.00 4.00 6 2.17 0.75 1.00 3.00 99 Table 2 Measures of Central Tendency for Parent Report on Youth Adjustment Variables Time 2: Pretest Time 3: Posttest Time 4: Follow-Up Interventiona Controlb Intervention Control Intervention Control M SD M SD M SD M SD M SD M SD Problem Behaviors 2.32ab 0.53 2.64ab 0.27 2.45 0.45 2.75 0.20 2.45 0.30 2.47 0.34 Internalizing Behaviors 2.33ab 0.48 2.70ab 0.24 2.38 0.42 2.53 0.45 2.73 0.21 2.77 0.24 Hyperactivity 2.24ab 0.42 2.36ab 0.43 2.33 0.50 2.33 0.35 - - - - Prosocial Behaviors. 2.53ab 0.35 2.63ab 0.34 2.64 0.36 2.63 0.23 - - - - Prosocial Activity 3.55ab 0.65 3.39ab 0.85 3.36 0.88 3.50 0.51 2.81 0.78 2.61 0.71 Note. Subscripts at pretest time point indicate results of the One-Way ANOVA that was conducted to measure the existence of group mean differences at pretest for youth outcomes. Subscripts that are grouped together indicate no significant group differences for the corresponding scale. 100 Table 2 (Continued) Measures of Central Tendency for Parent Report on Youth Adjustment Variables Time 2: Pretest Time 3: Posttest Time 4: Follow-Up Interventiona Controlb Intervention Control Intervention Control M SD M SD M SD M SD M SD M SD Peer Problems 2.64ab 0.56 2.77ab 0.29 2.64 0.45 2.87 0.16 - - - - Peer Relationships 4.46ab 0.65 4.75ab 0.21 - - - - - - - - School Success 4.40ab 0.57 4.80ab 0.80 - - - - - - - - Effortful Control 3.63ab 0.61 3.71ab 0.34 - - - - - - - - Substance Use 0.00 0.00 0.00 0.00 - - - - - - - - Note. Subscripts at pretest time point indicate results of the One-Way ANOVA that was conducted to measure the existence of group mean differences at pretest for youth outcomes. Subscripts that are grouped together indicate no significant group differences for the corresponding scale. 101 Table 3 Descriptive Statistics for Parent Report of Family History and Background Support Time 2: Pretest Time 3: Posttest M SD Min Max M SD Perception of Financial Stress 2.24 0.66 1.00 4.00 - - Stressful Life Events 3.57 0.20 3.14 3.86 - - Parental Depression 2.00 0.66 1.00 3.50 - - Parental Anxiety 1.94 0.77 1.00 4.00 - - Parental Anger/Aggression 1.79 0.61 1.00 3.00 - - Self Perception of Well Being 2.00 0.71 1.00 4.00 - - Parental Coping Strategies 3.32 0.33 2.75 4.00 3.16 0.26 Intervention 3.36ab 0.38 2.75 4.00 3.20 0.25 Control 3.25ab 0.22 3.08 0.30 Note. Subscripts at pretest time point indicate results of the One-Way ANOVA that was conducted to measure the existence of group mean differences at pretest for parent data. Subscripts that are grouped together indicate no significant group differences for the corresponding scale. 102 Table 3 (Continued) Descriptive Statistics for Parent Report of Family History and Background Support Time 2: Pretest Time 3: Posttest  M SD M SD M SD Parental Social Support 14.24 1.82 10 17 14.41 3.71 Intervention 14.73 1.68 12 17 15.27 3.55 Control 13.33 1.86 10 15 12.83 3.76 Spouse/Partner Relationship Support 4.87 0.88 3.25 6.00 - - Parental Alcohol Use 5.44 2.19 1.00 9.00 - - Parental Marijuana Use 1.63 0.81 1.00 3.00 - - Parental Drug Use – Other 0.00 0.00 0.00 0.00 - - Family Cultural Connectedness 2.92 0.53 1.57 4.00 - - Note. Subscripts at pretest time point indicate results of the One-Way ANOVA that was conducted to measure the existence of group mean differences at pretest for parent data. Subscripts that are grouped together indicate no significant group differences for the corresponding scale. 103 Table 4 Descriptive Statistics for Parent and Therapist Report of Family Management Variables Time 2: Pretest Time 3: Posttest Time 4: Follow-Up Treatment Control Treatment Control Treatment Control M SD M SD M SD M SD M SD M SD Parent-Child Relationship Quality Parent Warmth 4.35 0.69 4.23 0.74 - - - - - - - - Parent-Child Conflict* 3.39 1.30 3.54 1.23 - - - - - - - - FAST - Relationship Quality 4.24 0.42 4.31 0.35 4.34 0.74 4.50 0.32 - - - - Observed 4.36 0.57 4.52 0.57 4.18 0.65 4.31 0.66 - - - - Note. Parent-Child Conflict scores indicate a reduction in conflict as scores decline. All scores are per parent report except for “Observed” headings indicate therapist observational coding. 104 Table 4 (Continued) Descriptive Statistics for Parent and Therapist Report of Family Management Variables Time 2: Pretest Time 3: Posttest Time 4: Follow-Up Treatment Control Treatment Control Treatment Control M SD M SD M SD M SD M SD M SD Positive Parenting Proactive Parenting 3.66 .44 3.45 1.06 3.81 0.57 3.52 1.11 - - - - Positive Parenting 3.92 0.42 3.68 0.59 3.94 0.43 3.98 0.56 4.20 0.39 3.77 0.99 FAST - Positive Parenting 3.84 0.59 4.20 0.40 3.93 0.61 4.33 0.70 - - - - Observed 3.57 0.63 3.85 0.73 4.20 0.42 4.17 0.67 - - - - Note. All scores are per parent report except for “Observed” headings indicate therapist observational coding. 105 Table 4 (Continued) Descriptive Statistics for Parent and Therapist Report of Family Management Variables Time 2: Pretest Time 3: Posttest Time 4: Follow-Up Treatment Control Treatment Control Treatment Control M SD M SD M SD M SD M SD M SD Limit Setting Negative Parenting* 2.79 0.51 2.44 0.65 2.55 0.59 2.31 0.41 2.29 0.37 2.56 0.48 Positive Limits 3.38 0.71 3.81 0.55 3.81 0.44 3.95 0.89 - - - - FAST –Limit Setting 3.75 0.49 3.97 0.59 3.83 0.83 3.77 0.59 - - - - Observed 3.19 0.61 4.10 0.69 3.67 0.74 3.23 0.23 - - - - Note. Negative Parenting: lower scores indicate a decline in negative parenting. All scores are per parent report except for “Observed” headings indicate therapist observational coding. 106 Table 4 (Continued) Descriptive Statistics for Parent and Therapist Report of Family Management Variables Time 2: Pretest Time 3: Posttest Time 4: Follow-Up Treatment Control Treatment Control Treatment Control M SD M SD M SD M SD M SD M SD Supervision & Monitoring Parental Monitoring 4.40 0.42 4.28 0.82 4.49 0.45 4.70 0.21 4.45 0.69 4.67 0.52 Problem Solving 4.61 0.33 4.52 0.51 4.56 0.44 4.48 0.42 4.45 0.51 4.29 0.53 FAST - Monitoring 3.97 0.71 4.25 0.36 4.14 0.62 4.39 0.20 - - - - Observed 4.41 0.44 4.47 0.66 4.42 0.44 4.00 0.95 - - - - FAST –Problem Solving 3.90 0.67 4.31 0.42 3.75 0.81 3.83 1.07 - - - - Observed 3.17 0.81 3.63 0.58 4.04 0.88 3.53 0.73 - - - - Note. All scores are per parent report except for “Observed” headings indicate therapist observational coding. 107 Table 4 (Continued) Descriptive Statistics for Parent and Therapist Report of Family Management Variables Time 1: Screening Time 3: Posttest Time 4: Follow-Up Treatment Control Treatment Control Treatment Control M SD M SD M SD M SD M SD M SD Motivation to Change* 3.95 .01 4.00 .74 5.20 .94 4.46 1.47 4.66 .60 4.08 .97 Note. “Motivation to Change” is the overall motivation to change parenting behaviors. 108 Table 5 Mixed Effect Repeated Measures ANOVA to Examine Intervention Effects Interaction Effects Time x Group Main Effects of Time Main Effect of Group Λ F ηp2 df p Λ F ηp2 df p R2 F ηp2 df p Youth Adjustment Problem Behaviors .91 3.86 0.10 2, 14 ns .65* 3.86 0.36 2, 14 < .05 .52 1.80 0.11 1, 15 ns Internaliz. Behaviors .84 1.36 0.16 2, 14 ns .65* 3.80 0.35 2, 14 < .05 .41 1.64 0.10 1, 15 ns Note. All variables are per parent report unless “observed” is indicated, meaning therapist report. * p < .05. 109 Table 5 (Continued) Mixed Effect Repeated Measures ANOVA to Examine Intervention Effects Interaction Effects Time x Group Main Effects of Time Main Effect of Group Λ F ηp2 df p Λ F ηp2 df p R2 F ηp2 df p Positive Parenting Positive Parenting .78 1.94 0.22 2, 14 ns .87 1.05 0.13 2, 14 ns .51 0.80 0.05 1, 15 ns Proactive Parenting 1.00 0.08 0.00 1, 15 ns .96 0.69 0.04 1, 15 ns 0.47 0.47 0.03 1, 15 ns FAST – Positive Parenting 1.00 0.02 0.00 1, 15 ns .97 0.49 0.03 1, 15 ns 1.15 2.28 0.13 1, 15 ns FAST Observed Positive Parenting .98 1.17 0.30 1, 15 ns .58* 10.8 0.42 1, 15 < .01 0.12 0.21 0.01 1, 15 ns Note. All variables are per parent report unless “observed” is indicated, meaning therapist report. * p < .05. 110 Table 5 (Continued) Mixed Effect Repeated Measures ANOVA to Examine Intervention Effects Interaction Effects Time x Group Main Effects of Time Main Effect of Group Λ F ηp2 df p Λ F ηp2 df p R2 F ηp2 df p Limit Setting Neg. Par. .60 3.13 0.31 2, 14 .08 .79 1.82 0.21 2, 14 ns .13* 0.25 0.02 1, 15 < .05 P. Limits .96 0.66 0.04 1, 15 ns .85 2.65 0.15 1, 15 ns 0.65 1.10 0.07 1, 15 ns FAST – Lim. .92 1.25 0.08 1, 15 ns .99 0.22 0.01 1, 15 ns 0.05 0.07 0.01 1, 15 ns Observe .54* 12.6 0.46 1, 15 < .01 .94 1.00 0.06 1, 15 ns 0.44 0.86 0.05 1, 15 ns Note. All variable are per parent report unless “observed” is indicated, meaning therapist report. Not significant = ns. * p < .05. 111 Table 5 (Continued) Mixed Effect Repeated Measures ANOVA to Examine Intervention Effects Interaction Effects Time x Group Main Effects of Time Main Effect of Group Λ F ηp2 df p Λ F ηp2 df p R2 F ηp2 df p Monitoring Monitor. .02 0.61 0.08 2, 14 ns .82 1.54 0.18 2, 14 ns .12* 0.20 0.01 1, 15 < .05 FAST Monit. 1.00 0.02 0.00 1, 15 ns 0.93 1.07 0.07 1, 15 ns 0.53 1.15 0.07 1, 15 ns Observe .86 2.36 0.14 1, 15 ns 0.87 2.18 0.13 1, 15 ns 0.25 0.49 0.03 1, 15 ns Prob. So. 1.00 0.01 0.00 1, 15 ns .98 0.34 0.02 1, 15 ns 0.06 0.19 0.01 1, 15 ns FAST - Pr. Solv. .96 0.59 0.04 1, 15 ns .88 2.09 0.12 1, 15 ns 0.48 0.58 0.04 1, 15 ns Observe .79 3.95 0.21 1, 15 .07 .86 2.48 0.14 1, 15 ns 0.01 0.01 0.00 1, 15 ns Motivate .74 2.23 .26 2, 13 ns .56* 5.03 .44 2, 13 <.05 1.83 1.40 .09 1, 14 ns Note. All variable are per parent report unless “observed” is indicated, meaning therapist report. Not significant = ns. * p < .05. 112 Table 6 Mean Differences Across Time Points for Intervention and Control Groups Intervention Group (n = 11) Waitlist Control Group (n = 6) Comparison M dif SD t ηp2 p M dif SD t ηp2 p Problem Behavior Pretest-Posttest -.24 .22 -2.04* .29 .07 -.11 .20 -1.35 .27 .34 Pretest-Follow Up -.14 .49 -.92 .08 .38 .17 .59 .70 .09 .52 Posttest-Follow Up .00 .40 .00 .00 1.00 -.28 .50 -1.36 .27 .23 Internalizing Behavior Pretest-Posttest -.05 .38 -.48 .02 .65 .17 .29 1.39 .27 .22 Pretest-Follow Up -.40 .46 -2.91* .46 .02 -.07 .30 -.54 .06 .61 Posttest-Follow Up -.35 .40 -2.86* .45 .02 -.23 .43 -1.34 .26 .24 * p < .05. 113 Table 6 (Continued) Mean Differences Across Time Points for Intervention and Control Groups Intervention Group (n = 11) Waitlist Control Group (n = 6) Comparison M dif SD t ηp2 p M dif SD t ηp2 p Positive Parenting -.63 .48 -4.30* .67 <.01 -.32 .70 -1.11 .32 Limit Setting Negative Parenting Pretest-Posttest .24 .53 1.51 .19 .16 .14 .36 .96 .16 .38 Pretest-Follow Up .50 .35 4.74* .69 <.01 -.11 .65 -.42 .03 .69 Posttest-Follow Up .26 .52 1.65 .21 .13 -.25 .64 -.96 .16 .38 Limits - Therapist .49 .79 2.04* .29 .07 -.87 .67 -3.19* .67 .02 * p < .05. 114 Table 6 (Continued) Mean Differences Across Time Points for Intervention and Control Groups Intervention Group (n = 11) Waitlist Control Group (n = 6) Comparison M dif SD t ηp2 p M dif SD t ηp2 p Monitoring Pretest-Posttest -.10 .42 -.80 .06 .44 -.43 .80 -1.31 .26 .25 Pretest-Follow Up -.06 .63 -.32 .01 .76 -.39 .60 -1.58 .33 .18 Posttest-Follow Up .04 .32 .42 .02 .69 .04 .44 .21 .01 .84 Motivation to Change Screen-Posttest -1.20 .97 -3.90* .60 <.01 -.46 1.44 -.78 .11 .47 Screen-Follow Up .70 .61 3.63* .57 <.01 .08 .44 .47 .04 .66 Posttest-Follow Up .55 .97 1.87 .26 .09 .38 1.18 .78 .11 .47 * p < .05. 115 Table 7 Descriptive Statistics for Treatment Fidelity and Program Evaluation Total Scores Intervention Group (n = 11) Control Group (n = 6) Interventionists (n = 2) Advisory Comm. (n = 3) n M SD M SD M SD M SD M SD COACH Total 12 - - - - - - 6.77 .86 - - Knowledge 12 - - - - - - 7.17 1.09 - - Structure 12 - - - - - - 6.50 1.17 - - Teaching 12 - - - - - - 6.42 1.08 - - Strengths 12 - - - - - - 6.58 .84 - - Results 12 - - - - - - 7.17 .86 - - Procedural Evaluation Questionnaires 22 2.82 .85 2.91 .83 2.50 .84 2.00 .00 3.67 .58 FAST 22 2.95 .79 3.09 .70 2.50 .84 3.00 .00 3.33 1.16 Initial Interview 16 3.25 .68 3.18 .75 - - 3.00 .00 3.67 .58 Feedback 16 3.62 .62 3.45 .69 - - 4.00 .00 4.00 .00 116 Table 7 (Continued) Descriptive Statistics for Treatment Fidelity and Program Evaluation Total Scores Intervention Group (n = 11) Control Group (n = 6) Interventionists (n = 2) Advisory Comm. (n = 3) n M SD M SD M SD M SD M SD Model Usefulness Improve Behavior 22 2.95 .21 2.91 .30 3.00 .00 3.00 .00 3.00 .00 Helps Parenting 22 2.82 .40 2.73 .47 2.83 .41 3.00 .00 3.00 .00 Recommend 22 3.00 .00 3.00 .00 3.00 .00 3.00 .00 3.00 .00 Advisory Committee Keep Participating 5 2.80 .45 - - - - 3.00 .00 2.67 .58 Use Goal Setting 5 2.80 .45 - - - - 3.00 .00 2.67 .58 Use Achieve Goals 5 3.00 .00 - - - - 3.00 .00 3.00 .00 Interest in EBPs 2 3.33 .00 - - - - 3.33 .00 - - Attitudes of EBPs 5 2.92 .67 - - - - 3.36 .10 2.62 .75 EBP in Practice 5 3.03 .22 - - - - 2.92 .12 3.11 .26 117 APPENDIX B FIGURES 118 Figure 1. Trend-level interaction for parent report of Negative Limit Setting across three time-points: pretest, posttest, and follow-up. 2 3 Pretest Posttest Follow-Up Time Point Intervention Control 119 Figure 2. Significant interaction for therapist observation of Parental Limit Setting from pretest to posttest. 3 4 5 Pretest Posttest Time Point Intervention Control 120 Figure 3. Trend-level interaction for therapist observation of Family Problem Solving skills from pretest to posttest. 3 4 5 Pretest Posttest Time Point Intervention Control 121 APPENDIX C STUDY MATERIALS (Original forms are available upon request from the author.) Form A. Recruitment Information Form B. Interest Form C. Recruitment Presentation D. Screening Questionnaire E. Screening script & Initial Interview Template F. Fidelity Check Lists G. COACH Fidelity Model for Intervention H. Informed Consent and Assent I. Family Assessment Task Coding Manual J. Family Assessment Packet – Pretest Caregiver Assessment K. Family Assessment Packet – Pretest Youth Assessment L. Family Assessment Packet - Posttest Caregiver Assessment M. Family Assessment - Posttest Youth Assessment N. Follow-Up Assessment O. Feedback Materials – Profile, Case Conceptualization From, Family Goals & Menu of Options P. Payment Receipts Q. Interventionist Program Evaluation Form R. Advisory Committee Member Program Evaluation Form 122 REFERENCES CITED Abad, J., Forns, M., Amador, J.A., & Martorell, B. (2000). Fiabilidad y validez del youth self report en una muestra de adolescentes. Psicothema, 12, 49-54. Ackerman, B. P., Brown, E. D., & Izard, C. E. (2004). The relations between contextual risk, earned income, and the school adjustment of children from economically disadvantaged families. Developmental Psychology, 40, 204–216. Antolín Suárez, L. Conducta antisocial en la adolescencia. Un análisis ecológico de los factores de riesgo y protección. Manuscript in Preparation. Antolín, L., Oliva, A., Pertegal, M. A. & López, A. (2011). Desarrollo y validación de una escala de valores para el desarrollo positivo adolescente. Psicothema, 23, 153-159. APA Ethical Principles of Psychologists and Code of Conduct, (2002). Azzolini, D., Schnell, P., & Palmer, J. R. B., (2012). Educational achievement gaps between immigrant and native students in two “new” immigration countries: Italy and Spain in comparison. The Annals of the American Society of Political Science, 643(1), 46-77. Beelmann, A. & Lösel, F. (2006). Child social skills training in developmental crime prevention: Effects on antisocial behavior and social competence. Psicothema, 18(3), 603-610. Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research: Issues for cultural adaptation and development of psychosocial treatments with Hispanics. Journal of Abnormal Child Psychology, 23, 67-82. Bernal, G., Jiménez-Chafey, M. I., & Domenech Rodríguez, M. M., (2009). Cultural adaptation of treatments: A resource for considering culture in evidence-based practice. Professional Psychology: Research and Practice, 40 (4), 361-368. Bodenmann, G., Cina, A., Ledermann, T., & Sanders, M. R., (2008). The efficacy of the Triple P-Positive Parenting Program in improving parenting and child behavior: A comparison with two other treatment conditions. Behavior Research and Therapy, 46, 411-427. Brave Heart, M.Y.H. (2003). The historical trauma response among natives and its relationship with substance abuse: A Lakota illustration. Journal of Psychoactive Drugs, 35(1), 7–13. Breslau, J., Saito, N., Tancredi, D. J., Nock, M., & Gilman, S. E. (2012). Classes of conduct disorder symptoms and their life course correlates in a US national sample. Psychological Medicine, 42(5), 1081-1089. 123 Bronfenbrenner, U. (1989). Ecological systems theory. Annals of Child Development, 6, 187–249. Bulanda, R. E., & Majumdar, D. (2009). Perceived parent-child relations and adolescent self-esteem. Journal of Child and Family Studies 18(2), 203-212. Cava, M. J., Musitu, G., Buelga, S., & Murgui, S. (2010). The relationships of family and classroom environments with peer relational victimization: An analysis of their gender differences. The Spanish Journal of Psychology, 13(1), 156-165. de Paúl, J., & Arruabarrena, I. (2003). Evaluation of a treatment program for abusive and high-risk families in Spain. Child Welfare, 82(4), 413-442. Connell, A., Dishion, T. J., & Deater-Deckard, K. (2006). Variable- and person- centered approaches to the analysis of early adolescent substance use: Linking peer, family, and intervention effects with developmental trajectories. [Special Issue]. Merrill-Palmer Quarterly, 52(3), 421–448. Connell, A., Dishion, T. J., Yasui, M., & Kavanagh, K. (2007). An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology, 75, 568–579. Dishion, T. J., & Andrews, D. (1995). Preventing escalations in problem behaviors with high-risk adolescents: Immediate and 1-Year Outcomes. Journal of Consulting and Clinical Psychology, 63, 538-548. Dishion, T. J., Bullock, B. M., & Granic, I. (2002). Pragmatism in modeling peer influence: Dynamics, outcomes, and change processes. In D. Cicchetti & S. Hinshaw (Eds.), How prevention intervention studies in the field of developmental psychopathology can inform developmental theories and models. [Special Issue]. Development and Psychopathology, 14(4), 995–1009. Dishion, T. J., French, D. C., & Patterson, G. R. (1995). The development and ecology of antisocial behavior. In D. Cicchetti, & D. J. Cohen (Eds.), Developmental Psychopathology, Vol. 2: Risk, Disorder, and Adaptation (pp. 421-471). Oxford, England: John Wiley & Sons. Dishion, T. J., & Kavanagh, K. (2003). Intervening in adolescent problem behavior: A family-centered approach. New York: Guilford Press. 124 Dishion, T. J., Kavanagh, K., Schneiger, A., Nelson, S. E., & Kaufman, N. (2002). Preventing early adolescent substance use: A family-centered strategy for the public middle school ecology. In R. L. Spoth, K. Kavanagh, & T. J. Dishion (Eds.), Universal family-centered prevention strategies: Current findings and critical issues for public health impact [Special Issue]. Prevention Science, 3, 191–201. Dishion, T. J., Knutson, N. M., Brauer, N., Gill, A., Risso, J., & Kavanagh, K. (2010). The COACH system for observing and supporting fidelity and effectiveness in the implementation of the EcoFIT model. Unpublished manuscript available from Child and Family Center, 195 W. 12th Ave., Eugene, OR 97401. Dishion, T. J., Nelson, S. E., & Kavanagh, K. (2003). The Family Check-Up for high- risk adolescents: Motivating parenting monitoring and reducing problem behavior. In J. E. Lochman & R. Salekin (Eds.), Behavior oriented interventions for children with aggressive behavior and/or conduct problems [Special issue]. Behavior Therapy, 34, 553-571. Dishion, T. J., & Owen, L. D. (2002). A longitudinal analysis of friendships and substance use: Bidirectional influence from adolescence to adulthood. Developmental Psychology, 38(4), 480-491. Dishion, T. J., & Patterson, G. R. (1997). The timing and severity of antisocial behavior: Three hypotheses within an ecological framework. In D. Stoff, J. Brieling, & J. Maser (Eds.), Handbook of Antisocial Behavior (pp. 205–217). New York: Wiley. Dishion, T. J., Shaw, D. S., Connell, A.M., Gardner, F., Weaver, C. M., & Wilson, M. N. (2008). The Family Check-Up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development, 79(5), 1395-1414. Dishion, T. J., & Stormshak, E. (2007). Intervening in children’s lives: An ecological, family-centered approach to mental health care. Washington, DC: American Psychological Association. Dishion, T. J., Stormshak, E. A., & Siler, C. (2010). An ecological approach to interventions with high-risk students in schools: Using the Family Check-Up to motivate parents' positive behavior support. In M. R. Shinn, H. M. Walker, & G. Stoner (Eds.), Interventions for achievement and behavior in a three-tier model including response to intervention. Bethesda, MD: National Association of School Psychologists. Dodge, K. A., & Petit, G. S., (2003). A biopsychosocial model of the development of chronic conduct problems in adolescence. Developmental Psychology, 39 (2), 349-371. 125 Doyle, S.L., Fosco, G.M, & Dishion, T. J. Evaluation of the Family Check-Up Ecological Family Assessment Battery. Manuscript in Preparation. Fosco, G., Doyle, S., Dishion, T. J., Kavanagh, K., & Stormshak, E., (2010). Family Assessment Task Coding Manual. Unpublished manuscript available from Child and Family Center, 195 W. 12th Ave., Eugene, OR 97401. Fraser, M. W. (1996). Aggressive behavior in childhood and early adolescence: An ecological–developmental perspective on youth violence. Social Work, 41, 347– 361. Garrido, V., Farrington, D. P., & Welsh, B. C. (2006). The importance of evidence- based approach in the current Spanish policy for crime prevention. Psicothema, 18(3), 591-595. Goodkind, J., LaNoue, M., Lee, C., Freeland, L., Freund, R. (2012). Feasibility, acceptability, and initial findings from a community-based cultural mental health intervention for American Indian youth and their families. Journal of Community Psychology, 40(4), 381-405. Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38, 581-586. Time frame modified. Goodman R, Meltzer H., Bailey V., (1998) The Strengths and Difficulties Questionnaire: A pilot study on the validity of the self-report version. European Child and Adolescent Psychiatry, 7, 125-130. Time frame modified. Henggeler, S. W. & Borduin, C. M. (1990). Family Therapy and Beyond: A Multisystemic Approach to Treating the Behavior Problems of Children and Adolescents. Pacific Grove, CA: Brooks/Cole. Henggeler, S., Schoenwald, S., Bordouin, C., Rowland, M., & Cunningham, P. (1998). Multisystemic treatment of antisocial behavior in children and adolescents. New York: Guilford Press. Heppner, P. P., Kivlighan, D. M., & Wampold, B. E. (2008). Research Design in Counseling. Belmont, CA: Brooks/Cole. Hiatt Racer, K. D., & Dishion, T. J. (2012). Disordered attention: Implications for understanding and treating internalizing and externalizing disorders in childhood. Cognitive and Behavioral Practice, 19, 31-40. Hinshaw, S.P., & Lee, S.S. (2003). Conduct and oppositional defiant disorders. In E.J. Mash, & R.A. Barkley (Eds.). (2003). Child psychopathology (2nd Ed.) (pp. 330- 371). New York: The Guilford Press. 126 Hughes, J. N., Cavell, T. A., & Wilson, V. (2001). Further support for the developmental significance of the quality of the teacher-student relationship. Journal of School Psychology, 39(4), 289-301. Inman, A. G., Howard, E. E., Beaumont, R. L., & Walker, J. A. (2007). Cultural transmission: Influence of contextual factors in Asian Indian immigrant parents’ experiences. Journal of Counseling Psychology, 54, 93-100. Johnson, G., Kent, G., & Leather, J. (2005). Strengthening the parent-child relationship: A review of family interventions and their use in medical settings. Child: Care, Health & Development, 31(1), 25-32. Kazdin, A. E. (1995). Conduct Disorders in Childhood and Adolescence. Thousand Oaks, CA: Sage Publications. Kellermann, N.P. (2001). Transmission of Holocaust trauma–An integrative view. Psychiatry, 64(3), 256–267. Leung, S. A. (2003). A journey worth traveling: Globalization of counseling psychology. The Counseling Psychologist, 31(4), 412-419. Loeber, R., & Dishion, T. J. (1983). Early predictors of male delinquency: A review. Psychological Bulletin, 94(1), 68–99. López, E. E., Pérez, S. M., Ochoa, G., M., & Ruiz, D. M. (2008). Adolescent aggression: Effects of gender and family and school environments. Journal of Adolescence, 31(4), 433-450. López-Romero, L., & Romero, E. (2010). Goals during adolescence and their relationships with antisocial behavior. The Spanish Journal of Psychology, 13(1), 166-177. Marcus, B. A., Swanson, V., & Vollmer, T. R. (2001). Effects of parent training on parent and child behaviors using procedures based on functional analyses. Behavioral Interventions, 16(2), 87-104. Matsumoto, Y., Sofronoff, K., & Sanders, M. R. (2010). Investigation of the effectiveness and social validity of the Triple P Positive Parenting Program in Japanese society. Journal of Family Psychology, 24(1), 87-91. McDougall, P., Hymel, S., Vaillancourt, T., & Mercer, L. (2001). The consequences of childhood peer rejection. In M. R. Leary (Ed.), Interpersonal rejection (pp. 213– 247). London: Oxford University Press. McEachern, A. D., Dishion, T. J., Weaver, C. M., Shaw, D., Wilson, M., & Gardner, F. (in preparation). Parenting young children (PARYC): Validation of self-report parenting measure. 127 Mertler, C.A., & Vannatta, R.A. (2002). Advanced and multivariate statistical methods (2nd ed.). Los Angeles: Pyrczak Publishing. Metzler, C. W., Biglan, A., Ary. D. V., & Li, F. (1998). The stability and validity of early adolescents' reports of parenting constructs. Journal of Family Psychology, 12, 600-619. Miller, W.R., & Rollnick, S. (2002). Motivational interviewing: Preparing people for change (2nd ed.). New York: Guilford. Mrazek, P. J., & Haggerty, R. J. (Eds.). (1994). Reducing risks for mental disorders: Frontiers for preventive intervention research. Washington, DC: Institute of Medicine. Nation, M., Crusto, C., Wandersman, A., Kumpfer, K. L., Seybolt, D., Morrissey-Kane, E., & Davino, K. (2003). What works in prevention. American Psychologist, 58, 449-456. Navarro-Pardo, E., Meléndez, Moral, J. C., Sales Galán, A., & Sancerni Beitia, M. D. (2012). Desarrollo infantile y adolescente: Trastornos mentales más frecuentes en función de la edad y el género. Psícothema, 24(3), 322-383. Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of Triple P- Positive Parenting Program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical Child Family Psychology Review. 11(3), 114- 144. OECD, 2011. International and migration outlook 2011. Paris: OECD. Organización Mundial de la Salud (1993). Clasificación estadística interanacional enfermedades y otros problemas de salud (CIE-10). Madrid: Meditor. Patterson, G. R. (1982). Coercive Family Process. Eugene, OR: Castalia Publishing. Patterson, G. R. (1993). Orderly change in a stable world: The antisocial trait as a chimera. Journal of Consulting and Clinical Psychology, 61(6), 911-919. Patterson, G. R. (1997). Performance models for parenting: A social interactional perspective. In J. E. Grusec & L. Kuczynski (Eds.), Parenting and children’s internalization of values: A handbook of contemporary theory (pp. 193 – 226). New York: Wiley. Prilleltensky, I. (1997). Values, assumptions, and practices: Assessing the moral implications of psychological discourse and actions. American Psychologist, 52, 517-535. 128 Prinz, R. J., Sanders, M. R., Shapiro, C. J., Whitaker, D. J., & Lutzker, J. R. (2009). Population-based prevention of child maltreatment: The U.S. Triple P system population trial. Prevention Science, 10, 1-12. Prochaska, J. O., & Norcross, J. G. (1999). Systems of psychotherapy. Pacific Grove, CA: Brooks/Cole. Raver, C. C., Gershoff, E. T., & Aber, J. L. (2007). Testing equivalence of mediating models of income, parenting, and school readiness for White, Black, and Hispanic children in a national sample. Child Development, 78(1), 96–115. Reid, J. B., & Eddy, J. M. (1997). The prevention of antisocial behavior: Some considerations in the search for effective interventions. In D. M. Stoff, J. Breiling, & J. D. Maser (Eds.) Handbook of Antisocial Behavior, 343-356. Ribas-Mateos, N. (2004). How can we understand immigrants in Southern Europe? Journal of Ethnic and Migration Studies, 30(6), 1045-63. Rodríguez, F. J., & Paíno, S. G. (1994). Violencia y desviación social: Bases y análisis para la intervención. Psicothema, 6(2), 229-244. Samper, P., Cortés, M. T., Mestre, V., Náchar, M. J., & Tur, A. M. (2006). Adaptación del Child’s Report of Parent Behavior Inventory a población española. Psicothema, 18, 263-271. Sanders, M. R. (2010). Adopting a public health approach to the delivery of evidence- based parenting interventions. Canadian Psychology, 51 (1), 17-23. Sanders, M. R.,, Markie-Dadds, C., & Turner, K. M. T., (2003). Theoretical, scientific and clinical foundations of the Triple P-Positive Parenting Program: A population approach to the promotion of parenting competence. Parenting Research and Practice Monograph, 1, 1-21. Sanders, M. R., & Murphy-Brennan, M. (2010). Creating conditions for success beyond the professional training environment. Clinical Psychology: Science and Practice, 17(1), 31-35. Silverthorn, P., & Frick, P.J. (1999). Developmental pathways to antisocial behavior: the delayed-onset pathway in girls. In D. Cicchetti (Ed.) Development and Psychopathology. New York: Cambridge University Press, pp 101–126 Stallard, P. (2006). Psychological interventions for post-traumatic reactions in children and young people: a review of randomized controlled trials. Clinical Psychology Review. 26(7), 895-911. Stormshak, E. A., Comeau, C. A., & Shepard, S. A. (2004). The relative contribution of sibling deviance and peer deviance in the prediction of substance use across middle childhood. Journal of Abnormal Child Psychology, 32, 635–649. 129 Stormshak, E. A., Connell, A., & Dishion, T. J. (2009). An adaptive approach to family- centered intervention in schools: Linking intervention engagement to academic outcomes in middle and high school. Prevention Science, 10, 221-235. Stormshak, E. A., Connell, A. M., Véronneau, M.-H., Myers, M. W., Dishion, T. J., Kavanagh, K., & Caruthers, A. S. (2011). An ecological approach to promoting early adolescent mental health and social adaptation: Family-centered intervention in public middle schools. Child Development, 82(1), 209-225. Stormshak, E. A., & Dishion, T. J. (2009). A school-based family centered intervention to prevent substance abuse: The Family Check-Up. The American Journal of Drug and Alcohol Abuse, 35(4), 227-232. Stormshak, E. A., Dishion, T. J., Light, J., & Yasui, M. (2005). Implementing family- centered interventions within the public middle school: Linking service delivery change to change in problem behavior. Journal of Abnormal Child Psychology, 33(6), 723–733. Stormshak, E. A., Fosco, G. M., & Dishion, T. J. (in press). Implementing interventions with families in schools to increase youth school engagement. The Family Check-Up model. School Mental Health. Stormshak, E. A., Margolis, K., Huang-Vissering, C. Y., & Dishion, T. (in press). Implementation of a family-centered, school-based intervention to prevent student academic and behavioral problems. In Kelly, B., & Perkins, D. F., (Eds.), Handbook of implementation science for psychology in education. New York: Cambridge University Press. Suelves, J. M., & Sánchez-Turet, M. (2008). Comportamiento antisocial y uso de drogas: Implicaciones para la prevención. Anales de Psiquiatría, 24(6), 258-263. Tolan, P. H., Guerra, N. G., & Kendall, P. C. (1995). A developmental ecological perspective on antisocial behavior in children and adolescents: Toward a unified risk and intervention framework. Journal of Consulting and Clinical Psychology, 63(4), 579-584. Turner, K. M. T., & Sanders, M. R. (2006). Dissemination of evidence-based parenting and family support strategies: Learning from the Triple P-Positive Parenting Program system approach. Aggression and Violent Behavior, 11, 176-193. Van Ryzin, M. J., & Dishion, T. J. (in press). The impact of a family-centered intervention on the ecology of adolescent antisocial behavior: Modeling developmental sequelae and trajectories during adolescence. Development and Psychopathology. 130 Vazsonyi, A. T., Chen, P., Jenkins, D. D., Burcu, E., Torrente, G., & Sheu, C. (2010). Jessor’s problem behavior theory: Cross-national evidence from Hungary, the Netherlands, Slovenia, Spain, Switzerland, Taiwan, Turkey, and the United Nations. Developmental Psychology, 46(6), 1779-1791. Venkatraman, S., Dishion, T. J., Kiesner, J., & Poulin, F. (in press). Cross-cultural analysis of parental monitoring and adolescent problem behavior: Theoretical challenges of model replication when East meets West. In V. Guilamo-Ramos, P. Dittus, & J. Jaccard (Eds.), Parental monitoring of adolescents. New York: Columbia University Press. Vera, J., Ezpeleta, L., Granero, R., & de la Osa, N. (2010). Antisocial behavior, psychopathology and functional impairment: Association with sex and age in clinical children and adolescents. Child Psychiatry and Human Development, 41, 465-478. Webster-Stratton, C. (1992). Individually administered videotape parent training: “Who benefits?”. Cognitive Therapy and Research, 16(1), 31-52. Welsh, B. C., & Farrington, D. P. (2006). Effectiveness of family-based programs to prevent delinquency and later offending. Psicothema, 18(3), 596-602.