RE-ENGAGING OUT OF SCHOOL STUDENTS by MATTHEW ALAN EIDE A DISSERTATION Presented to the Department of Educational Methodology, Policy, and Leadership and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Education December 2019 DISSERTATION APPROVAL PAGE Student: Matthew Alan Eide Title: Re-engaging Out of School Students This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Education degree in the Department of Educational Methodology, Policy, and Leadership by: Gerald Tindal, Ph.D. Chairperson and Advisor Michael Bullis, Ph.D. Core Member Joanna Smith, Ph.D. Core Member Christopher Murray, Ph.D. Institutional Representative and Kate Mondloch Interim Vice Provost and Dean of the Graduate School Original approval signatures are on file with the University of Oregon Graduate School. Degree awarded December 2019 ii © 2019 Matthew Alan Eide This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs (United States) License. iii DISSERTATION ABSTRACT Matthew Alan Eide Doctor of Education Department of Educational Methodology, Policy, and Leadership December 2019 Title: Re-engaging Out of School Students Each year, thousands of students leave school without completing their high school education. Considerable research has been conducted on students’ dropout and disengagement behavior, but much less is understood about why students decide to return to school. Additionally, few instruments have been developed to measure the re- engagement behavior of out-of-school youth. To address these gaps, this two-phase study used qualitative and quantitative methods to develop, field test, and validate a survey instrument designed to identify factors associated with students’ decision to return to school after having dropped out or disengaged. Guided by input from expert and participant focus groups and a push/pull factor theoretical framework described in the limited literature on re-engaging out-of-school students, a re-engagement behavior instrument was developed and pilot tested in the first phase of the study. In the second phase, the instrument was field tested with a sample of students enrolled in a dropout recovery program in order to examine (a) the factors students indicate contributed to their decision to return, (b) how those factors varied across demographic groups, and (c) the relationship between dropout and re-engagement behaviors. Results from the study will inform the creation of similar instruments and the development of programs designed to re-engage students who have dropped out of school. iv CURRICULUM VITAE NAME OF AUTHOR: Matthew Alan Eide GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene University of California, Los Angeles DEGREES AWARDED: Doctor of Education, 2019, University of Oregon Preliminary Administrator License, 2018, University of Oregon Master of Education, 2001, University of California, Los Angeles Bachelor of Arts, History and Anthropology, 1997, University of California, Los Angeles AREAS OF SPECIAL INTEREST: Dropout Recovery Alternative Education PROFESSIONAL EXPERIENCE: Director of High School Success, Portland Public Schools, Portland, OR: 2019- Present Administrator, Reconnection Services, Portland Public Schools, Portland, OR: 2017-2019 Project Director, EIR Grant, Portland Public Schools, Portland, OR: 2017-2019 Senior Advisor, Education Northwest, Portland, OR: 2013-2017 Administrator and Lead Teacher, Outside In, Portland, OR: 2006-2017 Social Studies Teacher and Department Chair, Manual Arts High School, Los Angeles Unified School District, Los Angeles, CA, OR: 2000-2005 v ACKNOWLEDGMENTS I would like to thank Dr. Jo Smith and Dr. Michael Bullis for their guidance throughout the D.Ed. program and Dr. Gerald Tindal for helping me cross the finish line. Additionally, this study would not have been possible without the support of Portland Public Schools (PPS) Multiple Pathways to Graduation and the Reconnection Services staff and students. vi I dedicate this dissertation to my wife, Kate, and my daughters, Liliana and Araceli. They all made sacrifices to make this possible. vii TABLE OF CONTENTS Chapter Page I. INTRODUCTION .................................................................................................... 1 II. LITERATURE REVIEW ........................................................................................ 4 Literature Search Procedures ................................................................................. 4 Selection Criteria and Final Literature Pool .......................................................... 7 Review and Analysis Guidelines ........................................................................... 9 Results of the Literature Review ............................................................................ 12 Theoretical Frameworks .................................................................................. 12 Subjects ............................................................................................................ 14 Type of Research ............................................................................................. 17 Demographics .................................................................................................. 20 Settings ............................................................................................................. 21 Measures .......................................................................................................... 23 Analysis and Results ........................................................................................ 28 Demographic Variation .............................................................................. 28 Push and Pull Factors ................................................................................. 29 Institutional Factors ................................................................................... 31 Individual Factors ...................................................................................... 32 Summary ................................................................................................................ 33 Theoretical Model and Measurement Approach.................................................... 34 III. METHODS ............................................................................................................ 37 Setting and Context ................................................................................................ 38 viii Chapter Page Population and Sample .......................................................................................... 40 Phase I: Instrument Development .......................................................................... 41 Literature Review............................................................................................. 42 Focus Groups ................................................................................................... 43 Protocol Development ..................................................................................... 46 Expert Validation ............................................................................................. 48 Pilot Testing ..................................................................................................... 48 Phase II: Field Testing ........................................................................................... 50 Sampling Plan .................................................................................................. 50 Time Aspect ..................................................................................................... 51 Unit of Analysis ............................................................................................... 52 Survey Administration ..................................................................................... 52 Analysis............................................................................................................ 53 RQ 2 .................................................................................................. 53 RQ 3 .................................................................................................. 55 RQ 4 .................................................................................................. 56 Validity and Reliability .......................................................................................... 56 IV. RESULTS ............................................................................................................. 58 Research Question One .......................................................................................... 58 Dropout Factors ............................................................................................... 58 Re-engagement Factors .................................................................................... 60 Characteristics of a New School ...................................................................... 61 ix Chapter Page Variables .......................................................................................................... 62 Research Question Two ......................................................................................... 62 Reliability ......................................................................................................... 62 Construct Validity ............................................................................................ 64 Research Question Three ....................................................................................... 69 Dropout Factors ............................................................................................... 69 Re-engagement Factors .................................................................................... 70 School Characteristics ...................................................................................... 72 Open-ended Responses .................................................................................... 73 Research Question Four ......................................................................................... 75 Variation by Sex .............................................................................................. 75 Variation by Socioeconomic Status/Parent Educational Level ....................... 78 Variation by Race/Ethnicity ............................................................................. 79 Variation by Special Education Status ............................................................. 84 V. DISCUSSION ......................................................................................................... 88 Research Question 1: Re-engagement Instrument Variables ................................. 85 Dropout Variables ............................................................................................ 85 Re-engagement Variables ................................................................................ 87 School Characteristic Variables ....................................................................... 88 Research Question 2: Validity and Reliability....................................................... 89 Research Question 3: Variables Associated with Return ...................................... 92 x Chapter Page Dropout Responses .......................................................................................... 92 Re-engagement Factors .................................................................................... 93 Research Question 4: Demographic Variation ...................................................... 94 Variation by Sex .............................................................................................. 94 Variation by Parent Educational Level ............................................................ 95 Variation by Race and Ethnicity ...................................................................... 96 Limitations ............................................................................................................. 97 Sample .................................................................................................. 97 Instrumentation ................................................................................................ 98 Analysis .................................................................................................. 99 Qualitative ........................................................................................................ 100 Implications and Recommendations for Future Research ..................................... 101 Future Research ............................................................................................... 102 Implications for Practice ........................................................................................ 103 Reconnection Services ..................................................................................... 103 Districts .................................................................................................. 104 Personal Practice .............................................................................................. 106 APPENDICES ............................................................................................................. 108 A. RESEARCH ARTICLES IN LITERATURE REVIEW .................................. 108 B. OUTREACH COORDINATOR SURVEY ADMINSTRATION LETTER .... 110 C. COVER LETTER TO PARENTS .................................................................... 111 xi Chapter Page D. PARENT INFORMED CONSENT FORM ..................................................... 112 E. STUDENT ADVISORY GROUP NGT PROTOCOL ..................................... 114 F. RE-ENGAGEMENT SURVEY ........................................................................ 116 REFERENCES CITED ................................................................................................ 121 xii LIST OF FIGURES Figure Page 1. Summary of Literature Selection Process .............................................................. 4 2. Theoretical Framework .......................................................................................... 37 3. Research Phases ..................................................................................................... 39 xiii LIST OF TABLES Table Page 1. Literature Search Results by Keyword Combination ............................................ 6 2. Summary of Theoretical Frameworks ................................................................... 14 3. Summary of Subjects ............................................................................................. 16 4. Types of Research Methods ................................................................................... 18 5. Summary of Subject Demographics ...................................................................... 21 6. Summary of Settings .............................................................................................. 23 7. Summary of Dependent Variable Measures .......................................................... 25 8. Summary of Predictor Variables ............................................................................ 27 9. Summary of Key Findings ..................................................................................... 30 10. PPS, MPG, and Reconnection Services Demographics......................................... 39 11. Demographic Characteristics of Sample Compared to Population........................ 41 12. Re-engagement Behavior Survey Variables .......................................................... 47 13. Summary of Content Expert and Student NGT Focus Groups .............................. 59 14. Summary of Re-engagement Survey Variables ..................................................... 63 15. Independent and Dependent Variables Use in Multiple Regression ..................... 65 16. Pearson Correlations Between Leave Prompts ...................................................... 66 17. Pearson Correlations Between Return Prompts ..................................................... 67 18. Summary of Multiple Regression Results for Push IVs ........................................ 68 19. Summary of Multiple Regression Results for Pull IVs ......................................... 69 20. Summary of Responses to Dropout Factor Items .................................................. 71 21. Summary of Responses to Re-engagement Factor Items ...................................... 72 xiv Table Page 22. Summary of Responses to School Characteristic Items ........................................ 73 23. Summary of Open-Ended Responses..................................................................... 74 24. One-Way ANOVA of Responses to Survey Items by Sex .................................... 76 25. Summary of Descriptive Statistics for One Way ANOVA ................................... 77 26. One-Way ANOVA of Survey Responses by Parent Educational Level................ 79 27. One-Way ANOVA of Responses to Survey Items by Race/Ethnicity .................. 81 28. Tukey Post-Hoc Results for Race/Ethnicity ANOVA ........................................... 82 29. Descriptive Statistics for Race/Ethnicity ANOVA ................................................ 83 30. Variable Alignment ................................................................................................ 90 xv CHAPTER I INTRODUCTION According to the Oregon Department of Education, 7,649 students dropped out of Oregon high schools during the 2014-2015 school year (Hansell, 2016). Those young people will add to the already staggering total of 6.7 million youth between the ages of 16 and 24 who are disconnected from education and the workforce (White House Council for Community Solutions, 2012). Dropping out of school has significant consequences for these youth and for society as well (Belfield, Levin, & Rosen, 2012; Catterall, 2011). In addition to facing poorer life outcomes, dropouts are more likely to be incarcerated and require expensive health care and social services (Belfield, et al., 2012; Catterall, 2011). Belfield et al. (2012) estimated that allowing these youth to remain disconnected would impose a cumulative taxpayer burden of up to $1.56 trillion. Dropping out also has serious implications for the achievement gap. Students of color and those experiencing poverty are more likely to drop out of high school, and are disproportionately represented in the disconnected student population (Belfield et al., 2012; Entwisle, Alexander, & Olson, 2004; Tyler & Lofstrom, 2009; White House Council for Community Solutions, 2012). Belfield et al. (2012) report that the 16-24- year-old population is 18% Latino/a and 15% African American, but Latino/as comprise 32% and African Americans 46% of the disconnected youth population. This suggests that improving school systems’ ability to re-engage high school dropouts could narrow achievement gaps by increasing completion and graduation rates for students of color. Faced with high dropout rates, considerable resources have been invested in dropout prevention, resulting in a promising body of research that addresses factors 1 associated with dropping out (Balfanz et al., 2014; Rumberger, 2011, 2004; Suh, Suh, & Houston, 2007). According to Rumberger (2011), one of the most prominent researchers in this field, dropping out as an event is usually the culmination of a long process of disengagement that is often influenced by a combination of individual and institutional factors. Individual factors like student motivation, family life, mental health, or the need to work can exert pressures that pull students out of school. For example, parenting students may feel they need to leave school to care for a child. Similarly, institutional factors such as school structure, retention policies, and discipline practices may push students out. Students may choose to disengage and dropout if they have been retained for a grade or have not earned enough credits to graduate on time. Despite this emerging understanding of why students leave school, much less is known about factors that contribute to successfully re-engaging students after they drop out. The focus on dropout prevention is warranted and reasonable, but it also highlights the relative lack of attention and research devoted to reengaging the 6.7 million young people who have disconnected from education and the workforce. To improve graduation rates, narrow achievement gaps, and reduce the social and economic burden of dropping out, practitioners and policymakers need to improve their understanding of why students decide to return to school and which individual and institutional factors influence their decision to re-engage. With that need in mind, the purpose of this study to (a) examine the re- engagement behavior of a group of students in a large urban school district in the Pacific Northwest; (b) examine the relationship between dropout and re-engagement behavior; and (c) analyze the impact of race, ethnicity, gender, and other demographic variables on 2 students’ decision to return to school. More broadly, I hope to develop an instrument to collect information on the factors that impact students’ decision to return to school that can be used by districts to improve services for students who have dropped out, or are in the process of dis-engaging. 3 CHAPTER II LITERATURE REVIEW The following section describes the review of literature on student re-engagement behavior. It begins with a description of the process used to identify relevant literature and the criteria for inclusion. Next, it reports results and findings from the review, including a discussion of theoretical themes that emerged from the literature. Finally, the review culminates in an explanation of the research questions that guide this study. Literature Search Procedures The following section documents the process I used to select peer-reviewed articles for a final literature pool. It describes the (a) key words used in the search, (b) databases utilized, (c) search parameter definitions, (d) initial number of citations and the process to exclude irrelevant articles, and (e) criteria used to select the final pool. It also outlines the process I used to review and analyze the selected literature. A summary of this procedure is illustrated in Figure 1. • 46 manuscripts selected for abstract review • 42 articles from University of Oregon library database search 46 • 4 additional articles from Google Scholar search (same keyword combinations) • 11 manuscripts selected for final pool 11 • 4 articles added through ancestral search 4 • 13 published in peer-reviewed journals • 2 REL studies 15 • 14 published since 2000 (one published in 1997) • 3 eliminated after deep read 14 • 2 articles added through second ancestral search Figure 1. Summary of literature selection process. 4 Inconsistency is present in the field, and in the research literature, regarding how to refer to students who have dropped out, and for the process of re-engaging them in their education. These students have been called dropouts, disconnected youth, disengaged youth, out-of-school youth, and more recently, Opportunity Youth (White House Council for Community Solutions, 2012). Similarly, the process of re-enrolling students who have dropped out has been referred to as dropout recovery, re-engagement, and reconnection. As such, it is plausible that research conducted on the topic of interest may have used different terms to describe the same population and re-engagement processes. Therefore, any attempt to exhaustively search the literature on re-engaging out-of-school students should include the variety of terms that have been used over the last 20 years to describe the practice. With those inconsistencies in mind, my search for relevant literature included a combination of the following key words: (a) dropout recovery, (b) reconnecting, (c) re- engagement, (d) out-of-school youth, and (e) opportunity youth. As Table 1 illustrates, I used various combinations of these key words in each database to maximize the number of articles returned. For example, as the second column in Table 1 summarizes, the combination of re-engage and dropout yielded 123 relevant results, but reconnect and out of school youth identified 152 results. 5 Table 1 Literature Search Results by Keyword Combination Keywords Number of initial results Unduplicated articles Re-engage, dropout 123 19 Reconnect, dropout 186 7 Dropout recovery 58 4 Opportunity youth 176 5 Re-engage, out of school youth 46 7 Reconnect, out of school youth 152 0 Total 741 42 I used the University of Oregon library online portal for an initial search that included the ERIC, SAGE, JSTOR, and ProQuest Education Journal databases. Those databases were selected because they contain comprehensive and complementary collections of peer-reviewed articles on educational research. The same search process was replicated in Google Scholar because it explores a broader range of academic databases that could provide additional results and confirm the sources identified though the University of Oregon library. In addition to the key word combinations discussed earlier, the literature search used other criteria and parameters to narrow the focus of the search and reduce the number of irrelevant articles. For example, limiting the search to original empirical research in peer-reviewed journal articles published since 2000 reduced the number of policy papers and non-academic results. One exception to the peer-reviewed criteria was a report published by the Regional Educational Laboratory (REL) system. Although not peer-reviewed, REL reports undergo a rigorous review process by the Institute for 6 Educational Sciences. I focused on articles published after 2000 based on the assumption that dropout recovery research would be informed by the policy landscape created by the No Child Left Behind Act. Finally, I further refined each step in the search by limiting the results by subject (i.e. education, youth, adolescent). Those search criteria narrowed and focused the results. For example, a search without the date and peer-reviewed criteria yielded over 400 results with the reconnect and dropout search terms, but was reduced to 186 when restricted to peer-reviewed articles published since 2000. I reviewed the titles of each of the articles using the search process described above, and eliminated articles whose titles suggested that they did not focus on re- engaging high school dropouts. For example, several articles on dropout predictors and re-engaging community college students were eliminated. Nonetheless, because of the relative scarcity of peer-reviewed articles on dropout recovery, I retained any article that appeared to be even marginally relevant, trusting that false positives would be eliminated in subsequent rounds of review. Through this iterative process, the initial list of 186 possible articles was reduced to 42. I then subjected these 42 articles to another round of review and exclusion described in the next section. Selection Criteria and Final Literature Pool The University of Oregon Library and Google Scholar searches yielded 42 articles that were subjected to another round of evaluation and exclusion. Based on a review of each article’s abstract, I eliminated any that did not meet each of the following criteria: (a) a focus on student re-engagement behavior, (b) a focus on individual and/or institutional factors influencing re-engagement in school, (c) original peer-reviewed research, and (d) research conducted in the United States. These exclusion criteria 7 eliminated 14 articles that did not represent original research, 20 articles that did not specifically focus on the re-engagement behavior of high school dropouts, and one article that was conducted in Australia. For example, although I limited the initial search to peer- reviewed articles, the abstract review eliminated several research syntheses that made it into the pool of 42 articles. Similarly, after reading abstracts, I disqualified 11 articles that focused on high school dropouts, but did not examine the behaviors and processes associated with their re-engagement and re-enrollment in school. This literature search and selection process yielded only 11 peer-reviewed research articles published since 2000 that met the inclusion criteria. To identify additional sources, I read each of the 11 articles and examined their literature reviews and reference lists for additional articles that may have been overlooked in the initial search. This ancestral search process yielded four additional articles that met each of the selection criteria. For example, Berliner, Barrat, Fong, & Shirk (2008) list of citations yielded two additional articles by Wayman (2001, 2002). At this point, the literature search process had identified 15 peer-reviewed articles, but after carefully re-reading each article, I eliminated three articles because they focused primarily on students’ post re-engagement experience, or on the factors that contributed to their decision to drop out. For example, this close reading of the articles eliminated a promising qualitative study of an alternative school that had successfully re- engaged out-of-school students. I retained the article initially because its unit of analysis was students who had successfully re-engaged with their education, but a closer examination of the article revealed that it focused on the experience of the students within 8 the school, not the factors that contributed to their re-engagement (Franklin, Streeter, Kim, & Tripodi, 2007). Concerned that the pool of literature was too small, I conducted a second ancestral search of recent re-engagement policy papers and webinar proceedings published by the National League of Cities and the Center on College and Career Readiness and Success. These sources, although not original research, provided citations for articles not identified through the initial search processes. For example, resources from the National League of Cities yielded two articles (Zaff, Kawashima, Boyd, & Kakli, 2014; Zaff et al., 2016) that met the search criteria. In total, I identified 14 articles for this literature review, listed in Appendix A. Those articles included two REL reports. As discussed earlier, REL reports were not published in a peer-reviewed journal, but they represent original research and undergo a rigorous review and evaluation process by the US Education Department’s Institute for Educational Sciences. The REL reports were not discovered through the database searches described earlier, but through referral from professional colleagues at the Northwest Regional Educational Laboratory. Additionally, I elected to include a qualitative study from 1992, and a seminal study on re-engagement behavior from 1997 because it was cited by several of other articles in the final pool, and because the publication dates were relatively close to the 2000 cut-off date. With that in mind, the following section describes review and analysis procedures for the final literature pool. Review and Analysis Guidelines I utilized several strategies to maximize the accuracy of the review during the search, selection, and analysis process. To improve the accuracy of the final literature 9 pool, I selected articles that not only met the inclusion criteria described earlier, but also cross-cited each other. The presence of multiple cross-citations increased confidence that the selected sources represented the universe of possible articles. For example, the article by Chuang (1997) was selected because it was cited by eight of other articles, and because it was described as a seminal study by several authors (e.g. Barrat, 2016; Berliner et al., 2008). Similarly, Boylan and Renzulli (2016), Iachini, Buettner, Anderson-Butcher, & Reno (2013), and Lagana-Riordan et al. (2011) all cited Entwisle et al.’s (2004) study on the distinction between permanent and temporary dropouts. I also searched for references to each of the selected articles in policy proposals, practitioner guides, reports, and other non-academic sources. The consistent presence of cross- citations in my selected pool of articles increased my confidence that the final pool accurately represented the existing literature base on student re-engagement behavior. After selecting the final literature pool, I analyzed and summarized the 14 articles according to five broad categories: (a) theoretical frameworks, (b) research designs, (c) subjects and settings, (d) measures and instruments, and (e) findings. For example, the section on research design discusses patterns and themes in the types of methodologies and analyses utilized by the studies. These methodological patterns, in turn, suggest gaps that could be addressed by this study. Additionally, as described in greater detail later in the review, there is a lack of mixed method and instrument development studies that this study intends to address. The review also analyzes the theoretical frameworks used by the studies in the literature pool, and provides a rationale for the selection of a push/pull framework for this study. The majority of the studies examined in this review explicitly identified a 10 framework that guided the selection of methods and informed the interpretation and analysis of the results. As such, this review not only identifies theoretical themes across the studies, but also evaluates the extent to which the frameworks’ expected findings matched empirical results. These findings, in turn, are used to justify the use of a push/pull framework to guide the conceptualization and development of this study. To evaluate the generalizability of the literature’s findings to the population this study examined, I also review the context of each study. The studies’ populations, demographics, and the size and location of the studies’ settings establish a context for interpreting each study’s findings. For example, a finding that re-engagement behavior varies according to race and ethnicity should be understood within the demographic context of the setting. A finding that African American students were less likely to re- engage and re-enroll has different implications in a predominantly White setting than in a more diverse context. With that in mind, the following sections describe the subjects and settings of each of the studies, and identifies patterns and trends across the literature. The measures section analyzes the quantitative and qualitative instruments and protocols utilized by each of the studies. Because the type of instruments and variables that a researcher chooses can constrain or limit a study’s analyses and findings, the measures section provides additional context by identifying patterns and themes regarding the use of instruments and outcome variables. More importantly, this review of the measures, variables, and instruments used by studies relevant to this instrument development dissertation was used to inform my methods and analytical approach. Finally, the last section summarizes key findings from each article and identifies themes, inconsistencies, and overlaps in results and findings. As the final section in this 11 review, it addresses the inquiry question and synthesizes the factors that the literature in this pool associates with student re-engagement behavior. Results of the Literature Review The following sections describe and provide examples of the articles’ (a) theoretical frameworks, (b) research methods, (c) subjects, (d) settings and locations, (e) measures and variables, and (f) key findings. Somewhat surprisingly, none of the 14 articles examined the effect of interventions. It is plausible that interventions were not studied because disengaged students are less available or are less amenable to an intervention study. As such, interventions are not be addressed in this literature synthesis. Theoretical frameworks. Table 2 summarizes the theoretical frameworks used by the articles in the final literature pool. There were eight articles that explicitly addressed frameworks that informed the researchers’ methods, instruments, and analyses. These frameworks included (a) push/pull factors, (b) life course theory (Entwisle et al., 2004), (c) hope/persistence theory (McDermott, Donlan, Zaff, & Prescott, 2016), (d) educational resilience (Wayman, 2002), (e) positive youth development (Zaff et al., 2016), and (f) productive engagement (Zaff et al., 2014). Four of the articles (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Epstein, 1992) explicitly noted that they used a push/pull factor framework to conceptualize and measure student dropout and re-engagement behavior. This push/pull framework distinguishes between factors that push students out of school, such as academic performance or discipline issues, and factors that pull students out of school, such as the need to work or parent. In this framework, student re-engagement behavior is expected to vary according to whether students were pushed or pulled out of school. 12 Students who were pushed out of school are more likely to become permanent dropouts, and are more likely to complete with a GED or other alternative credential if they choose to return (Boylan & Renzulli, 2014; Entwisle et al., 2004). This behavior reflects the idea that push-out students decide to leave school because of lack of engagement, or a sense that the structure of schooling is a poor fit for them. On the other hand, students who are pulled out of school do so because they need to work, parent, or care for a family member. In other words, they drop out not because they have problems with the nature of their schooling, but because of other life priorities. As such, they are more likely to be temporary dropouts, and to return to more traditional programs (Boylan & Renzulli, 2014; Entwisle et al., 2004). Boylan and Renzulli (2014), in particular, explicitly framed their study in terms of the push/pull factor framework and attempted to explore the relationship between individual push/pull factors and subsequent re-engagement behavior. Although six of the studies did not report the use of a theoretical framework (Bickerstaff, 2010; Epstein, 1992; Iachini et al., 2013; Lagana-Riorden et al., 2011; McDermott et al., 2016; Zaff et al., 2016), the presence of a framework in eight studies suggests that one may be useful when conceptualizing and designing research on re- engagement behavior. In particular, the push/pull framework that was utilized by four articles may be of utility. With that in mind, the next section describes the context of the 14 studies selected for this review. 13 Table 2 Summary of Theoretical Frameworks Life Positive No Push/Pull Hope and Educational Productive Citation Course Youth Framework Factors Persistence Resilience Engage Theory Development 1 X 2 X 3 X 4 X 5 X 6 X X 7 X 8 X 9 X 10 X 11 X 12 X 13 X 14 X Total 6 4 1 1 1 1 1 Subjects. Table 3 summarizes key characteristics of the subjects in the literature pool, including sample size and the subjects’ enrollment status. Reflecting the diversity of methods used in the literature pool, the number of subjects varied from a sample size of two (Epstein, 1992) to 41,496 (Barrat, 2016). The seven quantitative studies utilized relatively large samples ranging from 680 to 41,496 subjects, with a mean of 7,340 and a median of 1,071. The four qualitative studies used much smaller samples that ranged 14 from two to 27 subjects. Finally, the three mixed method studies had samples of 27, 38, and 3,856 subjects. It is plausible that the varying aims and methods in the selected articles influenced the size of the samples. For example, Epstein’s (1992) qualitative case study examined the experience of two students the researcher had followed for eight years. The decision to limit the study to two students allowed Epstein (1992) to gain in-depth perspective on their experiences and allowed her to examine the impact of various push and pull factors on students’ decision to leave and return to school. Conversely, Barrat (2016) described the graduation and re-engagement patterns of all students in Utah who were expected to graduate in 2011. This broader examination of more than 41,000 students provided a sample large enough to identify demographic variation in re-enrollment patterns. Studying the re-engagement behavior of students often requires examining students who have re-enrolled and are currently in school, as well as students who have not yet re-engaged and are out of school. All 14 of the selected articles described research that involved some students who were in school at the time of the study. With that in mind, only seven of the studies also included students who were out of school. Interestingly, all of the studies that examined out-of-school youth were either quantitative or mixed methods in design. It is plausible that the four qualitative studies that did not include out-of-school youth did so because in-school youth may be easier to contact, follow, and study. 15 Table 3 Summary of Subjects Subjects Citation Sample In School Out of School 1 41,496 X X 2 3,856 X X 3 9 X 4 680 X X 5 1,795 X X 6 239 X X 7 2 X 8 13 X 9 47 X 10 1,173 X 11 1,071 X X 12 1,071 X X 13 27 X 14 38 X Total 14 7 Therefore, quantitative studies like Chuang’s (1997) analysis of panel data from the National Longitudinal Survey of Labor Market Experiences included outcomes for youth who successfully re-enrolled and completed and students who remained dropouts. In contrast, several of the quantitative and mixed methods studies used large longitudinal data sets that tracked school completion outcomes. 16 Type of research. As described in Table 4, the 14 articles in the peer-reviewed empirical research literature pool include seven quantitative studies, five qualitative studies, and three mixed methods studies. It is noteworthy that none of the selected studies employed experimental or quasi-experimental designs that might explore the potential causal effects of program interventions on the outcomes of interest investigated in this study. Of the seven studies that used quantitative methods, six employed logistic regression to analyze extant data or survey responses. Those six studies used logistic regression analyses because the dependent variables of interest, coded as in-school or out-of-school, are categorical (in these cases they are dichotomously coded) in nature. As Babbie (2013) notes, logistic regression is appropriate when researchers wish to identify the predictive relationship between continuous or categorical independent variables, such as ratings on a motivation scale or ethnicity, and a categorical dependent variable of interest, such as school enrollment status. For example, Entwisle et al. (2004) used logistic regression to examine the extent to which working while in school and teacher ratings of student motivation predicted re-engagement after dropping out. One of the quantitative studies also used exploratory and confirmatory factor analyses in an attempt to validate two survey instruments with populations of students who had returned to school after having dropped out. 17 Table 4 Types of Research Methods Quantitative Logistic Factor Citation Regression Analysis Qualitative Mixed Methods 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 X 9 X 10 X 11 X 12 X 13 X 14 X Total 6 1 5 2 The five qualitative studies include three case studies (Epstein, 1992; Iachini et al., 2013; Lagana-Riordan, et al., 2011), one narrative approach (Bickerstaff, 2010), and one phenomenological study (Zaff et al., 2016). The three case studies varied in size and scope. For example, Iachini et al. (2013) conducted a single focus group with one group 18 of six students and one group of seven students. In contrast, Epstein (1992) followed two students over the course of eight years. There was one narrative study, which according to Creswell (2014) are characterized by the collection, analysis, and sometimes synthesis of participant stories. With that in mind, Bickerstaff’s (2010) narrative study of nine students who had dropped out and re-enrolled in an alternative school included 100 hours of observation and nine conversation groups with participants. As she analyzed the qualitative data from interviews and observations, Bickerstaff (2010) utilized several validation strategies, such as member checks and triangulation, to improve the accuracy and representativeness of her findings. For example, she solicited feedback from students on preliminary themes that emerged from conversation groups. The two mixed methods studies combined interviews with administrators, teachers, and students with hierarchical multiple regression (Zaff et al., 2014) and descriptive statistical analyses of extant data (Berliner et al., 2008). Berliner and colleagues utilized an explanatory sequential mixed method design to describe the characteristics of students who dropped out and then re-enrolled in a traditional or alternative high school. Creswell (2014) notes that the intent of explanatory mixed method designs is to use qualitative data to help explain the results of the quantitative analysis. After analyzing re-enrollees quantitatively, Berliner et al. (2008) interviewed district administrators, principals, and re-enrolled students. Consistent with Creswell’s (2014) description of explanatory designs, the researchers asked participants to describe student re-engagement behavior and identify challenges to re-enrollment in the district. 19 This scan of the methods, analyses, and research designs of the 14 articles finds that a majority of the research conducted on student re-engagement behavior is quantitative in design. Additionally, at least in this literature pool, logistic regression appears to be a promising analytical tool for identifying factors associated with re- enrollment behavior. Demographics. As described in Table 5, 12 of the 14 studies provided detailed descriptions of the racial and ethnic demographics of the sample. In 8 of the 14 studies, African American or Latino/a students constituted the majority of participants (i.e. 50% or greater), whereas White students represented a majority (i.e. 78%, 53%, and 50%,) in three of the studies. This discrepancy is consistent with findings from research on dropping out that youth of color are disproportionately represented in the disengaged youth population (Belfield et al., 2012; Rumberger, 2004; Tyler & Lofstrom, 2009; White House Council for Community Solutions, 2012). If a student’s enrollment and completion status is correlated with race and ethnicity, it may be important to analyze outcomes by race, and it is therefore important to provide detailed demographic data. Nonetheless, Chuang (1997) and Boylan and Renzulli (2014) did not provide detail regarding the demographic composition of their sample. This omission of demographic details is particularly problematic in the context of the current study because the researchers in both articles found that there were ethnic/racial variations in re- engagement behaviors and outcomes. 20 Table 5 Summary of Subject Demographics Demographics African Citation Sample AI/AN Amer Asian Latino/a White Other 1 41,496 2% 1% 4% 14% 78% 1% 2 3,856 1% 22% 3% 53% 20% 3 9 56% 44% 4 680 5 1,795 6 239 50% 50% 7 2 100% 8 13 85% 15% 9 47 6% 1% 39% 55% 10 1,173 1% 80% 1% 18% 11 1,071 65% 35% 12 1,071 65% 35% 13 27 45% 24% 12% 19% 14 38 8% 26% 40% 8% 18% Total 3 10 5 8 10 3 Settings Table 6 summarizes key characteristics of the studies’ settings, including the region and urbanity/rurality of each of the settings selected. As described earlier, any research conducted outside of the United States was excluded during the search process, therefore all 14 of the articles selected were set in the United States. As demonstrated in the Region columns of Table 6, there is at least one article set in each of the four major 21 US geographic regions, with three studies conducted in the Northeast, one in the Midwest, one in the South, and five in the West. Additionally, four of the 14 articles described research in more than one geographic region, including two studies that used national data sets. For example, Boylan and Renzulli, (2014) utilized data from the Educational Longitudinal Study of 2002 to analyze the dropout and re-engagement behavior of 680 students. Additionally, to select a nationally representative sample for their qualitative study, Zaff et al. (2016) intentionally selected subjects from re- engagement programs located in 15 urban communities in each of the four national geographic regions. The 14 research articles also reflect a diversity of community size and urbanity, although it should be noted that none of the authors described the criteria they used to classify the studies’ locations as urban, suburban, or rural. All the articles included subjects drawn from urban settings, and six also included participants from both suburban and rural locations. Of the six articles that included subjects from urban, suburban, and rural communities, four articles utilized large national or state data sets that were indiscriminately representative of a broad range of communities. As an exception, both of Wayman’s (2001, 2002) quantitative studies intentionally surveyed youth from three Southwest communities of varying size: (a) a city with 400,000 people, (b) a midsized city with 90,000 people, and (c) a smaller town with 30,000 people. 22 Table 6 Summary of Settings Location Region Citation Urban Suburban Rural NE MW South West USA 1 X X X X 2 X X 3 X X 4 X X X X 5 X X X X 6 X X 7 X X 8 X X 9 X X 10 X X X X 11 X X X X 12 X X X X 13 X X 14 X X Total 14 6 6 3 1 1 5 4 Measures. As summarized in Table 7, the 14 articles in the literature pool used a variety of instruments and measures. Of the 10 quantitative and mixed method studies selected, eight of these studies utilized some form of enrollment and completion status to measure youth re-engagement behavior. The use of enrollment and completion status as a measure is to be expected because dropouts are defined by a combination of these two student-level factors (Rumberger, 2011). In other words, a dropout is defined as someone 23 who is not currently enrolled in school, and who has not completed school with a diploma. Therefore, especially as a measure for dependent variables, it is not surprising that a majority of the quantitative and mixed method studies used enrollment and completion status as a measure of youth re-engagement behavior. For example, Entwisle et al. (2004) used student enrollment and completion status to test their hypothesis that certain student characteristics, such as employment history, socioeconomic status, and motivation, predicted whether or not individual students were temporary or permanent school dropouts. Similar to the other seven other studies, Entwisle et al. (2004) used logistic regression to determine the relationship between independent variables, such as SES, with enrollment and completion status that they specified in their models as categorical (coded as dichotomous) dependent variables. The logic of using a categorical variable like enrollment and completion to measure re-engagement youth behavior helps to explain why six of the seven quantitative studies utilized logistic regressions. In addition to enrollment and completion, three of the 14 articles (McDermott et al., 2016; Wayman, 2001, 2002) reported using survey instruments, and one study (Zaff et al., 2014) utilized a workforce development rubric in their research. However, it should be noted that the survey instruments used by McDermott et al. (2016) and in Wayman’s two studies (2001, 2002) had not been specifically developed to measure the re-engagement behavior of high school dropouts. 24 Table 7 Summary of Dependent Variable Measures Quantitative Measures Enrollment Test Qualitative Completion Surveys Rubrics Citation Status Scores Interviews 1 X X 2 X X X 3 X 4 X X X 5 X X 6 X X 7 X 8 X 9 X 10 X 11 X X X 12 X X X 13 X 14 X X X X Total 8 8 1 3 1 7 The seven qualitative and mixed methods studies all used interviews or focus groups. Zaff et al. (2016) analyzed 27 transcripts from group interviews conducted with a pool of 203 students. Interviewers asked students to reflect on their reasons for leaving school and then deciding to return. The selected transcripts were analyzed for themes that aligned with five social-emotional learning (SEL) components derived from the Collaborative for Academic, Social, and Emotional Learnings SEL framework. Similarly, 25 Berliner et al. (2008) and Epstein (1992) developed semi-structured interview protocols based on a framework of push and pull factors. As such, they asked participants to reflect on the factors that pushed (e.g. academic performance, discipline, relationships with teachers and peer) and pulled them out of school (e.g. need to work, need to parent). Of the 14 studies selected for the final literature pool, eight reported predictor variables. Although several of the articles identified these as independent variables, this terminology usually connotes causation. Because the studies used logistic regression, it may be more appropriate to refer to them as predictor variables. With that distinction in terminology in mind, the five qualitative studies and the instrument validation article (McDermott et al., 2016) did not explicitly identify independent or predictor variables. Table 8 summarizes the predictor variables from the eight articles that reported them, organized by demographics, push factors, and pull factors. Each of the eight studies reported demographic predictor variables, such as race/ethnicity, sex, SES, and age. For example, Barrat (2016) examined the extent to which enrollment and completion rates varied by race/ethnicity, age at dropout, sex, and SES. Several studies also examined the relationship between push and/or pull factors on student dropout and re-engagement behavior (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Wayman, 2001, 2002). Push factors, such as poor academic performance, discipline problems, and poor relationships with teachers and students were associated with students’ sense that they do not belong, or cannot be successful in school. Pull factors, such as the need to work or parent, are variables that may influence students’ decisions that the costs of remaining in school outweigh the benefits. 26 Table 8 Summary of Predictor Variables Demographic Variables Push Factors Pull Factors Test Work Pregnant/ Adverse Race Sex SES Age GPA Motivation Relationship Discipline Citation Score Status parenting event 1 X X X X 2 X X X X X X X X X X 3 4 X X X X X X X X X X X 5 X X X X X 6 X X X X X X 7 8 9 10 11 X X X X X X X X X 12 X X X X X X X X 13 14 X X X X Total 8 8 8 8 4 5 1 3 2 5 4 1 27 Analysis and results. As summarized in Table 9, four themes emerged from the literature pool’s findings: (a) re-engagement behavior varied by demographics, (b) re- engagement behavior varied according to whether students were pushed or pulled out of school, (c) institutional factors influenced students’ decision to re-engage, and (d) individual factors influenced students’ decision to return to school. With that in mind, the following section describes each of these thematic categories and summarizes findings that support the themes. Demographic variation. As shown in Table 9, eight articles reported variation in re-engagement behavior according to demographics, such as race/ethnicity, age, parenting status, and SES. Each of the eight qualitative and mixed methods studies utilized demographics as a predictor variable in their analyses. Of these eight studies, four of the quantitative studies were designed to explicitly examine how re-engagement behavior varied by demographics (Barrat, 2016; Berliner et al., 2008; Chuang, 1997; Wayman, 2001). Additionally, articles like Entwisle et al. (2004) utilized demographics as a covariate, and explored demographic variation in their construct of interest (i.e. temporary vs permanent dropouts). Although all eight of the studies reported demographic variation, there was inconsistency related to some of their specific findings. For example, Barrat (2016) found that African American and English Learners were more likely than White and native English speakers to drop out, and less likely to re-enroll. In contrast, Berliner et al. (2008) found that African American students re-enrolled at a higher rate than other racial/ethnic groups, and Wayman (2001) reported that race/ethnicity was not a statistically significant predictor of re-enrollment. It is plausible that differences in methodology, sample size, 28 participants, and settings may have influenced researchers’ findings regarding demographic variation. For example, Barrat (2016) and Berliner et al. (2008), methodologically similar descriptive studies, were conducted in very different settings (i.e. Utah vs. San Bernadino, CA) with demographically dissimilar populations. Finally, it is notable that Boylan and Renzulli (2014), Entwisle et al. (2004), and Wayman (2001) found that SES predicted whether students returned to complete with a high school diploma or a GED. For example, Entwisle et al. (2004) reported that students with a lower SES were less likely to return to school than students with a higher SES, but when they did return, they were more likely to complete via a GED. Entwisle et al. (2004) noted that this is problematic given recent research that GED recipients fare worse on important life outcomes than equivalent students with high school diplomas (Heckman & Rubinstein, 2001). Push and pull factors. As noted earlier, four studies in Table 9 (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Epstein, 1992) explicitly utilized a push/pull factor framework to conceptualize and measure student dropout and re- engagement behavior. They found that classification as a push-out or pull-out student predicted (Boylan & Renzulli, 2014; Entwisle et al., 2004), or was associated with (Berliner et al., 2008), subsequent re-engagement behavior. For example, Berliner et al. (2008) found that students who were pulled out of school because they needed to work reported that they could be pulled back into school if they were unable to secure employment, or if they lost a job. 29 Table 9 Summary of Key Findings Push/Pull Demographic Factor Institutional Individual Citation Variation Variation Factors Factors 1 X 2 X X 3 X X 4 X X 5 X 6 X X X 7 X X 8 X X 9 X 10 X 11 X 12 X X 13 X 14 X Total 8 5 4 6 The bi-directionality of these pull factors is consistent with findings that pull-out students tend to make rational cost/benefit decisions regarding school (Boylan & Renzulli, 2014; Entwisle et al., 2004). The benefits of working may outweigh the costs of dropping out, but when that calculus changes, after losing a job for example, students may re-assess the benefits of returning to school. Similarly, the articles also found that students who were pushed out of school (e.g., because of poor fit, academic failure, 30 negative relationships with peers and teachers) were more likely to remain disconnected. If they return, they tend to do so in non-traditional programs or through the GED (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004). In addition to these four articles, Bikerstaff’s (2010) qualitative study reported findings that were consistent with the push/pull framework. For example, Bickerstaff (2010) found that the students she interviewed felt pushed out of school because of (a) feelings of fear and discomfort with the physical environment of their school, (b) poor relationships with students and teachers, and (c) poor fit and a lack of belonging that contributed to poor academic performance. These findings are congruent with the push factors that Boylan and Renzulli (2014) described, and are consistent with findings from Epstein’s (1992) qualitative study. Like the two students in Epstein’s (1992) case study, the nine students Bickerstaff (2010) interviewed reported that they returned to school after finding a program that mitigated the impact of the factors that pushed them out of their previous school. As such, Bickerstaff (2010) and Epstein (1992) argue that programs seeking to re-engage out-of-school youth should consider ways to address the factors that push students out of school. Institutional factors. As shown in Table 9, four articles in the literature pool identified characteristics of dropout recovery programs that positively influenced student re-engagement behavior (Bickerstaff, 2010; Epstein, 1992; Iachini et al., 2013; Lagana- Riodan et al., 2011). The four articles, all qualitative in design, found that students identified that the following school-level factors influenced their decision to return to school: (a) positive peer and adult relationships, (b) individualized instruction and curriculum, and (c) flexible schedules. In other words, programs that prioritized positive 31 adult relationships, a healthy school climate, and flexible individualized instruction were more likely to be successful at re-engaging students. As noted in the previous section, these characteristics that students identified as important re-engagement factors align with a push/pull factor framework. As such, they can be interpreted as institutional responses to the factors that push and pull students out of school. Few researchers concurred on the importance of program structure and high expectations. Epstein (1992) reported that the students she interviewed specifically identified the highly-structured environment of their re-engagement program as an important factor in their decision to return. One student noted that teachers had high expectations for student achievement and behavior. Although teachers were willing to provide extra support and flexibility, “They showed their authority. If you wanted to act silly, they weren’t babysitters” (p. 59). In contrast, Lagana-Riordan et al. (2011) found that students preferred a less authoritarian environment that allowed for greater student autonomy. The students in Lagana-Riordan et al.’s (2011) study felt pushed out of schools that exercise “overly rigid authority” (p. 109), and were re-engaged back into schools where “students are expected to act in a mature manner and be responsible for their actions, rather than relying on a system of rules and consequences” (p. 110). Individual factors. In addition to school-level elements, five articles in Table 9 also identified non-demographic student-level factors that influenced re-engagement behavior. The individual factors included (a) student motivation and desire to graduate (Entwisle et al., 2004; Iachini et al., 2013), (b) student persistence and resiliency (McDermott, et al., 2016; Wayman, 2002), and (c) social emotional learning (Zaff, 2016). The three quantitative studies (Entwisle et al., 2004; McDermott, et al., 2016; Wayman, 32 2002) utilized surveys and logistic regression to estimate the impact of individual factors, such as motivation or resilience, on re-engagement outcomes. For example, Entwisle et al. (2004) and Wayman (2002) identified a predictive relationship between measures of student motivation and resilience and subsequent re-engagement behavior. In addition, Wayman (2002) found that student resiliency, as measured by a survey that included a resiliency scale, predicted re-engagement and degree attainment. Summary In this review of the literature on student re-engagement behavior, several themes emerged related to both the nature and results of the research conducted on this topic. The studies described in the 14 articles indicate that re-engagement behavior (a) varies according to race/ethnicity, age, parenting status, and SES; (b) based on whether students were pushed or pulled out of school; and (c) may be influenced by individual and institutional factors. Overall, this suggests students who drop out may decide to return based on a variety of factors, some of which may be malleable, such as institutional characteristics, and some that may be less amenable to manipulation. Nonetheless, it is also clear that our understanding of why students decide to return is incomplete. I also found that relatively little research has been conducted on the re- engagement behavior of students who have dropped out of school. My search process revealed only 14 peer-reviewed articles related to the topic. Of the nine quantitative and mixed method studies, seven described characteristics associated with students who return to school, but only four (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Wayman, 2002) attempted to explain how these characteristics influenced re- engagement behavior. The five qualitative studies explored student perceptions of their 33 dropout and re-enrollment decisions, but they were primarily exploratory and descriptive in design. In summary, there is little literature on re-engagement behavior in general, and it is particularly limited regarding the factors that students identify contributed to their decision to return. Finally, only two studies utilized a mixed method design, and only one study attempted to develop an instrument to measure re-engagement behavior. In particular, the scarcity of reliable and valid measures specifically designed to explore re- engagement behavior may represent an important gap in the research on re-engaging out- of-school students. With that in mind, additional mixed methods research and/or instrument development may be warranted. This study seeks to address the gap in the literature by utilizing mixed methods to develop and field test a re-engagement behavior instrument designed to explore why students decide to return to school. More specifically, the study described in the following pages addresses the following four research questions.  RQ1: What variables comprise an instrument on school re-engagement?  RQ2: What are the validity and reliability characteristics of that instrument?  RQ3: What do students who have dropped out, then re-engaged with school, identify as key variables associated with their decision to return to school?  RQ4: How do those re-engagement variables vary by sex, socioeconomic status, race/ethnicity, and special education status? Theoretical Model and Measurement Approach This study used a push/pull factor theoretical model, primarily described by Boylin and Renzulli (2014), to conceptualize re-engagement and inform the development of this study’s methods. As described in the literature synthesis, several researchers used 34 the push/pull factor model as a framework for their studies on re-engagement behavior (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Epstein, 1992). Utilized by researchers studying dropout behavior for decades (Rumberger, 2004, 2011; Stearns & Glennie, 2006; Suh, et al., 2007), the push/pull framework posits that some students disengage from schooling because they feel pushed out of school. This may be because of (a) a sense of poor fit; (b) a lack of belonging; (c) academic issues, such as being behind in credits; or (d) discipline issues, such as suspensions or expulsions. Alternatively, students may be pulled out of school because of some adverse life event, or because of the need to work, parent, or care for family members. The framework has been adapted by re-engagement researchers to explain patterns of disengagement, and to understand and predict subsequent re-engagement behavior. For example, students who are pushed out of school may be less likely to return to traditional programs and more likely to complete a GED (Boylin & Renzulli, 2014; Entwisle et al., 2004). Conversely, pull-out students are conceptualized as rational actors who make cost/benefit analyses regarding the opportunity costs of staying in school. As such, they may return when they decide the benefits of completing high school outweigh the costs, and often do so through relatively traditional routes (Boylin & Renzulli, 2014; Entwisle et al., 2004). Therefore, under a push/pull theoretical framework, the factors that contribute to students’ decisions to return to school should be understood within the context of the factors that either pushed or pulled them out of school in the first place. With that in mind, the study described in the following section utilized the push and pull factor theoretical framework illustrated in Figure 2 to develop a survey instrument to evaluate and understand students’ re-engagement behavior through the lens 35 of their disengagement behavior. The push/pull theoretical model predicts that students who were pushed out of school would identify re-engagement factors that are different than students who felt pulled out of school. With that in mind, I developed an instrument that allowed me to analyze relationships between the disengagement and re-engagement factors that students identify. If the push/pull construct is valid for the population of students that I examined, I would expect that students’ responses to the dropout factor items to predict responses to the re-engagement factor prompts. The research reviewed in the above literature synthesis analyzed extant data, used qualitative interviews and focus groups, or adapted instruments designed with dropout behavior in mind. For example, even Boylan and Renzulli’s (2014) research on the impact of push and pull factors on student re-engagement behavior did not utilize an instrument specifically designed with re-engagement in mind. Instead they analyzed extant data from the Educational Longitudinal Study of 2002. With this in mind, an instrument developed with a push/pull factor lens, specifically examining re-engagement behavior, and validated with a disengaged student population, may address important gaps in thTe hliteeroaturree otni couat-lo Ff-srcahomol yeowutho. rk Pull Out Push Out Rational cost/benefit decision Nature of school is a poor fit Poor Sense of Belonging Need to Work Need to Parent Academic Issues Adverse Life event Lack of Engagement Discipline Cultural Incongruence More likely to return Less likely to return More likely to complete with More likely to complete with diploma GED Figure 2. Theoretical framework. 36 CHAPTER II METHODS The following section describes the methods this study used to address the four research questions described in the previous chapter. It includes an explanation of the methods for developing and field testing an instrument that was used to gather data on variables associated with high school students’ decisions to return to school after dropping out. In it, I explain the qualitative and quantitative procedures used to develop the instrument, and describe how the instrument was administered to a sample of disconnected students in a large urban school district. Additionally, this methods section will explain the qualitative and quantitative techniques I used to analyze data from a pilot and field test of the instrument. The section concludes with an overview of the methodology’s limitations and threats to internal and external validity. It may be helpful to conceptualize the study as occurring in two phases, as illustrated in Figure 1. In phase one, I reviewed the literature and solicited input from content experts and the population of interest to develop and validate a survey instrument. In the second phase, I field tested the instrument with a sample of students who have decided to re-engage in their education after dropping out. I then analyzed the data collected from the survey instrument, explored how the results varied across key demographic variable, and evaluated the extent to which the results were congruent with the literature and push/pull factor theoretical framework. 37 • Instrument development and Phase One validation •RQ 1 • Instrument field test Phase Two •Data collection & analysis •RQ 2-4 Figure 3. Research phases. Setting and Context The population targeted for this study were students who received services through Portland Public Schools’ (PPS) Reconnection Services. PPS is the largest school district in the state of Oregon, serving over 48,000 students and employing around 3,000 teachers. The district serves Portland, the economic and cultural center of the state, which with almost 600,000 residents is the largest city in Oregon. Portland has experienced significant growth and demographic change during the last two decades, which has contributed to gentrification and a lack of affordable housing. With that in mind, PPS serves a student population that is considerably more diverse than the city it serves. For example, students of color represent around 45% of the student population in a city that is almost 75% white. Reconnection Services is PPS’s dropout recovery program and is part of the district’s Multiple Pathways to Graduation (MPG) alternative education system, which provides a diverse range of options for young people wanting to re-enter the district. Multiple Pathways to Graduation includes two in-district alternative schools, a Reconnection Center, and nine private community-based alternative schools that contract 38 with the district. The schools and programs that constitute MPG currently enroll around 2,300 students, representing about 16% of all high school students in the district (PPS, 2019a). It is important to note, as described in Table 10, that the MPG schools and programs serve a student population that differs from the district’ high school population on several key demographic variables. For example, students in MPG are more likely to be (a) from an historically underserved community, (b) pregnant or parenting, (c) experiencing poverty, and (d) homeless. Table 10 PPS, MPG, and Reconnection Services Demographics Total Students Mean Age Pregnant/ Home- District Male SpEd FRL Students of Color at Entry Parenting less PPS 14,589 46.3% 52.6% 15.9 0.5% 15.2% 26.1% 2.5% MPG 2,138 57.6% 57.3% 17.4 2.5% 23.7% 37.3% 5.7% RS 883 61.5% 56.0% 17.8 6.3% 28.5% n/a 7.8% Reconnection Services helps students who have dropped out, or are in danger of dropping out, explore and access educational options. These options include (a) returning to or remaining in their current school, (b) enrolling in a comprehensive or focus option high school, (c) enrolling in a district administered or contracted alternative school, or (d) joining the Reconnection Center. The Reconnection Center is a transitional program that provides temporary skill remediation and credit recovery for students awaiting placement in another school/program. As described in Table 10, the Reconnection Services dropout recovery program serves a student demographic that is even more likely than the larger MPG alternative high school student population to be (a) older, (b) from a historically underserved community, (c) experiencing homelessness or housing instability, (d) pregnant or parenting, (e) male, and (f) and eligible for Special Education services. 39 Population and Sample The population targeted in this study were the 217 students that engaged with Reconnection Services during the period between November 15, 2018 and February 28, 2019. As described earlier, this population represents students who have been referred, or have self-referred, to Reconnection Services because they have dropped out or disengaged from school. Of these 217 students, 103 responded to the survey, representing approximately 47% of the overall population. As described in Table 11, this sample is demographically similar to the larger population, but a few key differences should be noted. Although the difference in the overall percentage of students of color between the sample and population is only about one percent, African American and American Indian/Alaska Native students appear to overrepresented and Latino/a students underrepresented in the sample. Similarly, the sample is a little less male than the population, and slightly more likely to have an IEP. Finally, the mean age of the sample was 17.01 years compared to 16.87 years for the population. Finally, the relatively small size of some of the demographic sub-groups should be noted. For example, the Asian and Pacific Islander group was comprised of only two students. As will be discussed later, the small sub-group sizes should be considered when making inferences from any analysis intended to identify significant variation across demographic groups. 40 Table 11 Demographic Characteristics of Survey Sample Compared to Population Sample Percent Population Percent Characteristic (n = 103) (n = 217) Race/Ethnicity AI/AN 7.8 3.7 Latino/a 13.6 17.5 African American 30.1 22.6 Asian 1.0 3.6 PI/Native Hawaiian 1.0 0.9 White 39.8 41.0 Multiracial 6.8 8.8 Declined 0.0 1.8 Gender Female 49.5 46.0 Male 50.5 51.6 Non-binary 0.0 2.3 SpEd Yes 30.1 27.6 No 69.9 72.4 Pregnant/Parenting Yes 8.7 6.4 No 91.3 93.6 Homeless/couch-surfing Yes 13.6 12.4 No 86.4 87.6 Phase I: Instrument Development The study used a five-step iterative process, adapted from Gehlbach and Brinksworth (2011), to develop the instrument and address the first research question. The development process included the following steps: (a) a review of the literature, (b) 41 focus groups with content experts and the population of interest, (c) protocol development, (d) expert validation of the protocol, and (e) a pilot of the protocol. Literature review. The first step in Gehlbach and Brinksworth’s (2011) instrument development model involves a review of the instruments used in relevant literature. Although none of the 14 articles described in the above literature synthesis described instruments that addressed factors related to students’ decision to re-engage, several of the articles provided a theoretical foundation that guided the conceptualization of the re-engagement construct and development of protocol items (Boylin & Renzulli, 2014; Entwisle et al., 2004; Iachini et al., 2013). As described earlier, the push/pull factor theoretical model posits that students withdraw from school in response to factors that either push or pull them out of school. Moreover, the model argues that the re- engagement behavior of push and pull dropouts may vary in ways that can have implications for program design. For example, push-out students may be more likely to remain permanent dropouts, and if they return, to do so in non-traditional or GED programs (Entwisle et al., 2004). A set of dropout and re-engagement behavior variables, categorized as either push or pull factors, were developed based on this push/pull model. Although there appears to be a promising theoretical basis for these factors (Boylin & Renzulli, 2014; Entwisle et al., 2004; Iachini et al., 2013), input and feedback from focus groups with content experts helped to establish the content validity of instrument items that were derived from the push/pull framework. The focus groups, framed as advisory groups, provided consultation and feedback throughout the instrument development process. In addition to 42 contributing to the identification of construct variables, the groups reviewed the draft protocol and debriefed the findings at the end of the study. Focus groups. Two advisory groups, a content expert and participant group, provided feedback and consultation at three points throughout the instrument development process. At the beginning of the protocol development process, the advisory groups clarified the re-engagement construct of interest and informed the development of instrument items. Next, I reconvened the student advisory group to review and critique a draft protocol during a member validation stage. Instead of meeting again, I asked the content expert group to review and provide feedback on the draft via email. Finally, as recommended by Creswell (2014), both advisory groups member-checked the results after the field test. I recruited six dropout and re-engagement content experts and practitioners to participate in a content expert advisory group. Participants included dropout and re- engagement researchers and practitioners, such as re-engagement outreach workers and the current and former directors of PPS’s alternative education system. I recruited these members from my existing network of re-engagement experts and asked them to nominate additional members. The final composition of the expert group included (a) leaders from PPS alternative high schools, (b) members of the National League of Cities’ Re-engagement Network, (c) the former and current Executive Directors of PPS’s Multiple Pathways to Graduation program, and (d) PPS re-engagement outreach workers. The second advisory group was comprised of seven members from the population of interest, students who have successful re-engaged with school after withdrawal. This student advisory group offered feedback on the factors that contributed to their decision 43 to leave and then return to school, providing student-level perspectives on re-engagement behavior that complemented and validated the system-level perspectives of the content experts. Recruited from students enrolled at the Alliance at Benson Alternative High School and the Reconnection Center during the spring of 2018, these students were selected because they (a) represented the gender and demographics of the Reconnection Services population, (b) had been disengaged from school for at least two weeks, and (c) had been enrolled at the Alliance at Benson Alternative High School or the Reconnection Center for at least four weeks. To aid recruitment, I provided members of the focus group with pizza during the sessions and a $5 gift card to Starbucks. Despite almost a full school year elapsing between the first and third student advisory groups, only two of the seven members were unable to participate in all three focus groups. I facilitated the content expert and student focus groups using a Nominal Group Technique protocol. Nominal Group Technique (NGT) is a structured group interaction process in which groups identify, prioritize, and reach consensus on responses to a prompt or question (Delbecq, Van de Ven, & Gustafson, 1975). For example, the student advisory group responded to questions about (a) why they decided to leave their last school, (b) why they decided to re-engage in their education, and (c) what characteristics they desired in a new school. For each of these prompts, the NGT involved (a) individual reflection and generation of ideas, (b) reporting and synthesis of the ideas, (c) discussion and clarification of emergent response categories, and (d) reaching consensus on themes and priorities. The NGT protocol script used with the student advisory group is included in Appendix E. 44 During the student and content expert NGT focus groups, participants recorded their individual responses to the prompts on a note-taking form. After approximately ten minutes, I asked members to share their top response for each prompt using a round-robin technique while I recorded the responses on poster paper, noting when members identified a common factor. I conducted several rounds of reporting, until each member had an opportunity to share all the factors they had recorded on their form. Participants then discussed and clarified the dropout, re-engagement, and school characteristic variables listed on the posters. Finally, I provided the student advisory group with nine sticker dots that they used to vote for the top three variables from each category. Instead of providing the expert advisory group with stickers, I facilitated a discussion to identify the factors they wished to prioritize. The NGT focus groups provided two data sources for qualitative analysis, individual member’s notes and the list of prioritized factors on the posters. Following Creswell’s (2014) qualitative analysis recommendations, I organized and prepared the data for analysis, coded the qualitative data, and identified emergent themes. I compared members’ individual notes with the posters, and added factors that were not shared during the round-robin report out phase. Additionally, I identified cases where a participant skipped a factor that had already been recorded. I transcribed the synthesized poster and individual reflections into Excel, noting the number of votes each received, and how many participants identified a given factor in the round-robin session. Factors that received at least two votes during the NGT session, or were identified by at least three members, were then coded as a push or pull factor, as suggested by the theoretical 45 framework and supporting literature (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Epstein, 1992; Rumberger, 2011). This synthesized and prioritized list of responses, coded as push or pull factors, helped establish the content and construct validity of the variables I elected to include in the re-engagement behavior instrument. As will be described in the results section, a high degree of agreement between the advisory groups, and with the factors predicted by the theoretical framework, increased my confidence in the content validity of the instrument. Factors identified by the advisory groups, such as incarceration and legal issues, but not predicted by the theoretical model, were used to refine the framework. Protocol development. The first two steps in the instrument development process addressed research question one: What variables comprise an instrument on school re- engagement? I synthesized findings from the literature review and the two focus groups to develop instrument items aligned with the push/pull theoretical framework. As noted earlier, this was an iterative process in which feedback from content experts and students was used to refine the theoretical model and inform the selection of variables to measure. With that in mind, I developed items for three broad re-engagement behavior domains to address research questions three and four: (a) reasons for leaving the last school, (b) reasons for reconnecting, and (c) characteristics desired in a new school. On the survey, each re-engagement behavior domain included a series of 8-12 variables or factors that students rated as (a) not important, (b) somewhat important, (c) important, or (d) very important to their decisions to leave and return to school. Most of these closed-option items were coded as a pull or push factor. Additionally, after each of the rating scale items, the instrument included open response items for respondents to 46 provide additional detail related to their decision to leave or return to school. However, as will be discussed in the results section, very few students elected to respond to the open items, which limited their value during analysis. Because I believed it to be unlikely that the target population would be able to provide reliable information regarding their family’s income and socio-economic status (SES), the instrument included a compositional SES item asking respondents to indicate their parents’ highest educational level (Shavers, 2007). This item was used as an imperfect proxy for SES. Similarly, I developed a race and ethnicity item using categories from PPS’s enrollment paperwork to ensure consistency with how student demographics are reported by the district. Table 12 provides an overview of the number of survey items associated with each re-engagement behavior domain and a final draft of the instrument is included in Appendix F. Table 12 Re-engagement Behavior Survey Variables Re-engagement Behavior Domain Close-Ended Questions Open-Ended Questions Reasons for leaving last school 12 1 Reasons for reconnecting 9 1 Characteristics desired in new 8 1 school Total 29 3 47 Expert validation. As suggested by Gelbach and Brinksworth (2011), after developing a draft of the survey instrument, I asked the content expert and student advisory groups to review the survey protocol, assess its face and content validity, and evaluate it for clarity and accessibility. I emailed the content expert group, comprised of researchers and practitioners with experience and expertise in dropout prevention and recovery, a draft of the protocol with a request for feedback on (a) the clarity and readability of the survey, (b) the extent to which items addressed the variables and constructs of interest, and (c) recommendations for improvement. Additionally, I reconvened all seven members of the student advisory group to review the survey and provide feedback on the readability and accessibility of the instrument. I provided the student group with pizza and a $5 Starbucks gift card, and asked them to provide feedback on the following criteria: (a) how easy it would be to read, (b) the extent to which questions could be misinterpreted, and (c) whether it reflected their input from the first session. Based on input from the two groups, I re- worded several of the survey items to improve readability and added non-binary as an option to the gender demographic question. Pilot testing. In the final step of the instrument development process, a sample of students recruited from the Reconnection Center and the Alliance at Benson Alternative High School pilot-tested the protocol. Dillman, Smyth, and Christian (2014) recommend piloting surveys with approximately 10% of the population that will be sampled during the implementation phase. Based on Reconnection Services data on spring referral rates from the last three years, I estimated that approximately 150-200 students would be offered the survey during the winter 2018-19 data collection window. With that in mind, 48 the pilot included a purposive sample of 15 students enrolled in the Reconnection Center and Alliance at Benson Alternative High School during fall of 2018. Babbie (2013) describes purposive sampling as a method for identifying units of observation that are representative in some way that is useful for the study. With that in mind, I recruited individuals for the pilot test that were demographically representative of the larger Reconnection Services population. The pilot test sample included participants that mirrored the race, ethnicity, sex, and socioeconomic status of the students who were likely to complete the survey during the implementation phase. The purpose of the pilot test was to determine preliminary reliability and collect feedback to further refine the instrument. I used a pre-test/post-test method for determining reliability, and calculated Pearson correlations to identify the consistency of responses. As noted by Thorndike and Thorndike-Christ (2010), a re-test with the same form after a time interval may limit the impact of participants remembering their initial response, but can introduce the possibility that changes in mood might influence results. With that in mind, I asked all 15 students to re-take the pilot version of the survey one week after initial administration, but only 11 were able or willing to re-take it. I solicited feedback during the pilot test in three ways: (a) qualitative survey questions (b) post-survey interviews, and (c) cognitive pre-testing. At the end of the pilot survey, three open-ended questions asked for feedback on the survey’s strengths and weaknesses. I also conducted brief interviews with the 11 students after they completed the survey, and asked about their experience taking the survey, especially regarding its readability and accessibility. Finally, I conducted a cognitive pre-test with four students. Cognitive pre-testing is a structured process for understanding how participants 49 understand and interpret the questions (Gehlbach & Brinksworth, 2011). Towards that end, I asked students to re-phrase questions in their own words and describe their thought processes when they responded to questions (Gehlbach & Brinksworth, 2011). Phase II: Field testing I conducted a field test of the instrument from November 2018 through the end of February 2019 with a goal of determining (a) the validity and reliability characteristics of the instrument (RQ 2), (b) the variables students identified as associated with their decision to return to school (RQ 3), and (c) how those responses varied according to key demographic variables (RQ 4). With that in mind, the following section describes how I administered the instrument and analyzed the results. Sampling plan. I utilized a single stage convenience sample during the field test phase. Originally, as the administrator for PPS’s Reconnection Services, I planned on offering the survey to the entire population of students that engaged with the program during the administration window. In this model, the survey would have been part of the intake and enrollment packet, and Reconnection Services Outreach Coordinators would offer the survey to all students during intake meetings. Families and students would have been given the opportunity to opt out, but I anticipated that including the survey as part of the intake process would have increased response rates. However, the district’s research compliance department prohibited me from requiring Outreach Coordinators from administering the survey as part of their regular duties. Instead, Outreach Coordinators were instructed to inform students and families that the survey was available, then provide informed consent documents and a paper copy of the survey if the student and guardian indicated a desire to respond. In other words, instead of framing the survey as 50 part of the standard Reconnection Services intake process, the survey was characterized as an optional item. As will be discussed in the limitations and threats to validity section, this convenience sampling technique limits the generalizability of the study’s findings. Outreach Coordinators reported that they made each student and family aware that the survey was an option, but it is unclear if they elected to withhold the survey from students they believed might be triggered by survey questions, or if any individual bias influenced how they framed the opportunity to complete the survey. Nonetheless, as described in the population and sample section, the sample that responded to the survey was generally representative of the population that engaged with Reconnection Services during the data collection period. Time aspect. The survey field test constituted a cross-sectional analysis of student re-engagement behavior. As noted by Babbie (2013), a cross-sectional analysis provides a snapshot of the re-engagement attitudes and behaviors of the population being studied. Unlike a longitudinal study, it does not allow for any analysis of change over time, but it is an efficient way to evaluate the attitudes of the population of interest at a specific point in time. More specifically, the survey instrument collected data during a three-month period from November 2018 through February 2019. 51 Unit of analysis. The unit of analysis was individual students who engaged with PPS’ Reconnection Services during the winter of 2018-19. This unit of analysis allowed the study to examine individual beliefs about the factors that contributed to students’ decisions to return to school. Offered to students who participated in intake meetings with Reconnection Services Outreach Coordinators, the survey provided data to analyze and quantify the “trends, attitudes, or opinions of a population” (Creswell, 2014, p. 13). Survey administration. Reconnection Services Outreach Coordinators offered a paper version of the re-engagement survey to students during a three-and-a-half-month period from November 15th, 2018 to February 28th, 2019. As discussed earlier, although the district approved this study, they prohibited Reconnection Services staff from administering the survey as part of their regular duties. With this in mind, Reconnection Services staff informed students and families that (a) a survey is included as part of the intake paperwork, (b) a cover letter provides details about the intent of the survey, and (c) their participation is completely voluntary and will have no impact on their ability to access services. I provided written guidance to staff and conducted a training on survey administration during a team meeting. This guidance included a script that could be used when offering the survey to students that (a) described the purpose of the survey, (b) explained that all results will be confidential, and (c) emphasized that participation is voluntary and that a decision to not participate will not adversely impact their ability access Reconnection Services. In addition to the script, the survey packet included a parent cover letter, a parent informed consent letter, a student assent letter, and an informed consent letter for students 18 years or older. A sample survey administration 52 letter, parent cover letter, parent informed consent letter, and student assent form are included in Appendices B, C, and D. Outreach Coordinators returned the completed surveys and informed consent letters either directly to me or placed them in folder kept in a cabinet in a secure office. At the end of each day, I (a) assigned a separate code to the survey and consent/assent letters, (b) recorded the codes on a password secured electronic code key document, and (c) separated the surveys from the signed consent/assent letters. This increased the security and confidentiality of the surveys, but allowed me to document that each survey was accompanied by the appropriate consent and assent forms. Analysis. The following section describes data analysis procedures for research questions two through four (RQ2 – RQ4). RQ 2. The first stage of analysis for research question two addressed the reliability of the measure. As with the pilot test, I used a test-retest technique for determining the reliability of the instrument. Of the 103 students who responded to the survey, I recruited 23 students to re-take the survey one to two weeks after the initial administration. These students represented a convenience sample that was either enrolled in the Alliance at Benson Alternative High School or the Reconnection Center after having taken the initial survey. Each student was offered a $5 gift card for the re-test. I used SPSS statistics software to identify the correlation of individual survey items from pre-test to post-test, with a goal of reaching the r = .85 reliability threshold established by Thorndike and Thorndike-Christ (2010). The second stage of analysis focused on the more difficult task of establishing the construct validity of the instrument. Challenges to establishing construct validity included 53 (a) a lack of criterion that could be used to calculate predictive validity, (b) the absence of other measures purporting to measure the same construct that could be used to determine construct-related validity, and (c) the time frame of the proposed study (Thorndike & Thorndike-Christ, 2010). With these challenges in mind, I used SPSS to conduct two sets of multiple regression analyses on the relationship between students’ responses to the leave and return items. In the first set of analyses, I entered the seven leave prompts that I categorized as push factors (e.g. feelings that you did not fit in or belong at your last school) as predictor variables, and individual return prompts as a dependent variable. In other words, I conducted a multiple regression with the seven combined push predictor variables for each of the nine return items. I then repeated the process for a second set of analyses using the five dropout prompts that I categorized as pull factors as predictors. This study’s theoretical framework suggests that students who are pushed or pulled out of school experience different re-engagement trajectories. If the instrument possessed relatively high levels of construct validity, then I would expect students’ responses to the dropout factor items to predict their responses to the re-engagement factor items. In other words, students who identified push factors as important or very important in their decision to leave, should be more likely to select return factors that are hypothesized to be related to the variables that push students out of school. At the end of the field test period, I reconvened the content expert and student advisory groups for the third time to discuss preliminary results from the survey and identify further revisions. This member-check also helped to establish the face and content validity of the survey and its results. Content experts were asked to assess the extent to which the results were congruent with their experience as researchers and 54 practitioners. The student group, which included five of the original seven members, was asked to reflect on how well the results aligned with their experiences. I also asked both groups to discuss any discrepant results and provide recommendations regarding how to improve the design and administration of the survey. RQ 3. I used descriptive statistics to address the third research question: What do students who have dropped out, then re-engaged with school, identify as key variables associated with their decision to return to school? More specifically, I used SPSS to determine the percentage of respondents who identified individual push or pull factors as (a) not important, (b) somewhat important, (c) important, and (d) very important. As discussed earlier, fewer students responded to the open-ended items than anticipated, limiting their utility. Nonetheless, I followed Creswell’s (2014) recommendations for analyzing qualitative data. Because of the limited amount of data available from the open-ended items, I was able to use Excel to (a) organize and prepare the data for analysis, (b) code the qualitative data, (c) identify emergent themes, and (d) interpret and triangulate the themes with my quantitative findings. Data coding included three stages. In the first stage, I identified open-ended responses that were elaborations or extensions of the closed-ended items. For example, some students described how their anxiety or mental health contributed to their decision to leave school in the open-response section. Responses like these were assigned a code based on the associated closed-ended item (e.g. mental health and anxiety). Second, I assigned each response a code of push-factor, pull-factor, or uncategorized. Responses that could be assigned to a closed-ended item, and could be categorized as push or pull, were used to validate the closed-ended survey responses. Finally, I planned to use an 55 open coding process to identify emergent themes from responses that did not fall into the pre-identified categories (Babbie, 2013), but none of the responses remained uncategorized. RQ 4. The fourth research question asked how survey responses varied by sex, socioeconomic status, race/ethnicity, and Special Education status. I assigned a numeric value to each point on the Likert scale, then used SPSS to conduct one-way analysis of variance (ANOVA) to evaluate whether the means of individual items varied across demographic groups in ways that were statistically significant. For example, I used an ANOVA to identify the presence of statistically significant differences in responses to the re-engagement behavior survey items among White, Latina/o, African American, American Indian/Alaska Native, and Asian students. I entered race/ethnicity as the independent variable and responses to the 29 survey items as dependent variables. I then used Tukey post hoc tests to verify which pairs contributed to the significant variation. Validity and reliability. Several threats to internal validity are addressed in the following section. The most significant of the quantitative validity threats is selection. Selection threats occur when the participants being studied vary in important ways from the larger population that they are supposed to represent (Parker, 1990). The convenience sample in this study included students who agreed to participate in the program’s re- engagement services. As such, because they chose to re-engage, they may be predisposed to respond to the survey in ways that may be different than students who remained out of school. More importantly, only 47% of the students who were referred to Reconnection Services during the data collection period completed the survey. Although the 56 demographics of the sample were largely representative of the larger population, it is plausible that they differed in ways that could have impacted the results. Additionally, it is unclear if Outreach Coordinator bias affected which students were offered the survey in ways that could have skewed the sample. The characteristics of the convenience sample also pose an interaction of selection and treatment quantitative external validity threat. Creswell (2014) describes interaction of selection and treatment as a threat that occurs when the specific characteristics of the participants limit the generalizability of the findings to a larger population. Similar to the selection threat described above, the fact that the participants chose to participate in the process may make them different in important ways from the larger population of dropouts. This has important implications for the claims that can be made about the results and may limit the ability to generalize findings from this study. 57 CHAPTER IV RESULTS The results described in the following section are organized by research question and describe findings from the development and administration of the instrument, and analyses of its results. Research Question One The results of student and content expert focus groups, facilitated with a Nominal Group Technique (NGT) protocol, are described in Table 13 and the narrative that follows. The wording of the discussion prompts used with the student advisory group differed slightly from those used with the content experts, but both groups provided input on the following: (a) factors associated with dropping out or disengaging, (b) factors associated with re-engagement, and (c) characteristics students desire in a new school. One advantage of using an NGT protocol is that participants are asked to reach consensus on a prioritization of factors and characteristics during the focus group, which aids in the analysis of the qualitative results. Table 13 provides an overview of the priorities identified by each group in the NGT sessions. Dropout factors. The student and content expert advisory groups identified several common dropout factors. Both groups highlighted the impact of pull factors on student disengagement, such as (a) mental health and anxiety, (b) housing instability, and (c) adverse life events or home life stress. The student group, in particular, emphasized the importance of general, and school-related, anxiety. While the students attributed this anxiety to feelings of being overwhelmed or lost in large public schools, the content 58 experts linked student anxiety to discriminatory discipline policies, a lack of cultural relevance, and rigid school structures. Table 13 Summary of Content Expert and Student NGT Focus Group Priorities Student Advisory Group Content Expert Advisory Discussion Prompt Priorities Group Priorities Why do students drop out? Mental health and anxiety Mental health and anxiety Home life stressors Housing instability Housing instability Racism and cultural Adverse life events incongruence Racism/cultural irrelevance Lack of monitoring Not engaging/boring Exclusionary discipline curriculum & instruction policies Lack of relevance Un-engaging curriculum Classes move “too quick” and teaching “One size fits all” approach to schooling Structural rigidity and institutional arrogance Why do students decide to Opportunity to return to Opportunity to return to re-engage/return to school? different /non-traditional different /non-traditional learning environment learning environment Belief that diploma Belief that diploma increases options increases options Peer/family pressure to Peer/family pressure to return return Desire for a “fresh start” Flexibility to work/parent Flexibility to work Change in mental health Change in mental health What characteristic do Individualized instruction Individualized instruction students desire in a new Ability to work at “own Social-emotional/mental school? pace” health support Social-emotional/mental Dual credit/CTE health support opportunities Culturally relevant Engaging learning curriculum and teaching opportunities Flexible High levels of structure schedules Small schools/small class sizes 59 Additionally, both groups also agreed on several factors that push students out of school, such as boring or irrelevant curriculum, but the content expert group prioritized more of these institutional push factors than the student group. Of the eight factors prioritized by the content expert group, six were institutional push factors, whereas the student group identified equal numbers of push and pull factors. Interestingly, the content experts were more likely than the student group to blame rigid school structures, institutional racism, and uninspiring learning experiences as key factors in students’ decisions to drop out. Re-engagement factors. There was also a high degree of consensus regarding the reasons students decide to re-engage. In the NGT focus groups, both the content expert and student advisory groups prioritized the opportunity to return to an educational program that offered a learning experience that was different than the one from which they dropped out. Not surprisingly, this feedback was also consistent with both groups’ descriptions of the characteristics students desire in a new school when they re-engage. As one student noted, “Why would I want to return to the type of school that I hated, and that hated me?” The two advisory groups also identified several internal and external factors that apply pressure on students to re-engage. Both groups pointed to differences in life outcomes for students with and without a high school diploma, and noted that students often decide to return because they believe a diploma or GED will increase their postsecondary options and earning potential. The advisory groups described this as both an internal and external pressure to return. Students may experience an intrinsic desire to 60 return and finish their high school education, while simultaneously feeling pressured by families and friends to complete their high school education. Characteristics of a new school. The advisory groups agreed that re-engaged students desire to return to schools with high degrees of personalization and social- emotional support, but had a different perspective on the amount of structure students need. Both groups prioritized personalization and individualized instruction. The student group emphasized the importance of schools allowing students to work at their “own pace,” while the content experts prioritized the type of personalized learning experiences where students are able to focus on interests relevant to their lives and postsecondary goals. The groups also addressed returning students’ need for individualized academic support. The content expert group, comprised of several alternative school administrators, noted that is why students often choose to return to small alternative schools with small class sizes. Finally, there was clear consensus that students who re-engage desire schools and programs with robust social-emotional supports and highly relevant and engaging learning experiences. Despite these areas of consensus, the advisory groups appeared to disagree on the type of structure students desire in a new school or program. More specifically, the students stated that disengaged students want to return to tightly structured programs with high levels of accountability regarding attendance and behavior. The content experts, on the other hand, claimed that students want to return to programs with high degrees of scheduling and attendance flexibility, so that they can fit school into busy work and childcare schedules. 61 Variables. Results from the NGT-facilitated student and content expert focus groups were combined with findings from the literature review to create a list of variables that were included in the final draft of the re-engagement instrument. Input from the advisory groups was largely consistent with the factors included on the first draft of the survey instrument, but the instrument was amended to reflect the priorities identified by the advisory group members. For example, the instrument was revised to include (a) an item that more clearly identified mental health and anxiety as a dropout factor, (b) revised wording for the cultural incongruence dropout item, and (c) dual credit opportunities as a school characteristic. As described in Table 14, the majority of these variables can be categorized as push or pull factors. Research Question Two Reliability. During the pilot testing phase, 11 out of the 15 students recruited for the pilot completed the survey a second time, approximately 10-12 days after initial administration. Although the retest sample was too small to make any definitive conclusions about reliability, the test-retest reliability analysis indicated a promisingly high degree of consistency in responses to individual survey items. Pearson correlation coefficients for individual items ranged from a low of r = .74, p = .010, n = 11 to a high of r = 1.00, p = .000, n = 11. Only three of the items fell below the r =. 85 reliability threshold recommended by Thorndike and Thorndike-Christ (2010). 62 Table 14 Summary of Re-engagement Survey Variables Variable Categorization Push Factor Pull Factor Uncategorized Dropout factors Sense of belonging Adverse life events Conflict with adults Mental health or or peers anxiety Discipline and Need to work behavior Pregnant or Safety parenting Academic Legal issues performance Engagement and relevance Cultural relevance Re-engagement Safe environment Changes in childcare Intrinsic motivation factors Engaging/non- Changes in Postsecondary goals traditional learning family/parenting Family or peer environment responsibilities pressure Change in work schedule Flexible schedules School Safe and welcoming Flexible schedule Dual credit characteristics environment Credit recovery opportunities Positive Opportunity to relationships with complete quickly teachers Social emotional Individualized support instruction Test-retest reliability was also analyzed during the field test phase. A convenience sample of 23 of the 103 survey respondents completed the survey again, approximately 9- 15 days after the first administration. All of these students either enrolled in the Reconnection Center or Alliance at Benson Alternative High School after the first survey administration or were actively engaged with their Reconnection Services case manager during the retest window. Consistent with test-retest reliability results from the pilot test 63 phase, Pearson correlations from the field test retests indicated a relatively high level of consistency between individual items. Correlations between the 12 individual dropout factor items ranged from a low of r = .92, p = .00, n = 23 and a high of r =. 98, p = .00, n = 23. Similarly, Pearson correlations for the nine re-engagement factors ranged from r = .89, p = .00, n = 23 to r = 1.00, p = .00, n = 23, and from r = .92, p =. 00, n = 23 to r = .97, p = .00, n = 23 for the eight school characteristic items. Construct validity. As described in the methods section, the instrument is based on a theoretical model that argues that there should be a relationship between dropout and re-engagement behavior, and that the re-engagement trajectory of students who were pushed out may be different from those who felt pulled out (Boylan & Renzulli, 2014). Therefore, confidence in the construct validity of the instrument would be increased if students who identified dropout factors categorized as push or pull were more likely to identify corresponding push or pull re-engagement factors. More specifically, a stronger predictive relationship between push or pull dropout and re-engagement factors, as measured by a multiple regression analysis, should indicate stronger construct validity. Table 15 summarizes the independent and dependent variables used in the multiple regression analysis. Independent variables one through seven were entered into SPSS to identify the combined relationship of push categorized dropout items with each individual re-engagement item. The process was then repeated with the five pull categorized independent variables. 64 Table 15 Independent and Dependent Variables Use in Multiple Regression Category Independent Variables Dependent Variables Push 1. Feelings that you did not fit in 1. Belief that a new school will or belong at your last school have an environment that will help you feel safe 2. Problems with some of your teachers and/or other students 2. Belief that a new school will 3. Disciplinary issues, such as help you be academically suspensions or expulsions successful 4. Not feeling safe at school 5. Academic issues, such as poor grades or not being on track to graduate 6. Engagement issues, such as feeling like school-work is boring or not relevant 7. Cultural issues, such as feeling that school does not fit in with how you like to learn Pull 8. Legal issues, such as an arrest 3. Changes in childcare that allow or incarceration you to return to school 9. Mental health or anxiety 4. Changes in responsibilities to 10. The need to work other family members that 11. Pregnancy or the need to take allow you to return care of your child 5. Changes in employment status 12. Something happened to you or that allows you to return to your family that made it school difficult to stay in school 6. Belief that a new school’s flexible schedule will allow you to work or care for family while attending school Not 7. Your motivation to return to categorized school and complete your high school education 8. Your desire to go to college or a trade school 9. Someone in your life is encouraging or pressuring you to return 65 Tables 16 and 17 summarize correlations between independent and dependent variables. The push categorized independent variables have relatively strong positive correlations with other push variables, and negative correlations with pull variables. A similar pattern emerged from an analysis of the correlations between dependent variables. In general, the push categorized dependent variables were positively correlated with each other and negatively correlated with pull factors. Table 16 Summary of Pearson Correlations Between Leave Prompts 1 2 3 4 5 6 7 8 9 10 11 2 .58* 3 .32* .72* 4 .62* .63* .55* 5 .25* .50* .46* .14 6 .42* .46* .35* .31* .60* 7 .38* .49* .53* .39* .55* .64* 8 .09 .37* .56* .23* .05 .11 .21* 9 .19 -.09 -.18 .09 -.04 -.04 -.22 -.23* 10 -.28* -.35* -.30* -.20* -.06 -.22* -.14 -.25* -.22* 11 -.27 -.20* -.20* -.23* -.22* -.32* -.21* -.14 -.34 .22* 12 -.05 -.17 -.31* -.11 -.33* -.22* -.19 -.07 .16 .23* .24* *Significant at p < .05. 66 Table 17 Summary of Pearson Correlations Between Return Prompts 1 2 3 4 5 6 7 8 2 .37* 3 -.30* -.32* 4 -.27* -.30* .82* 5 -.43* -.29* .26* .38* 6 -.44* -.18 .42* .45* .73* 7 -.08 .00 -.04 .02 .26* .25* 8 -.06 .15 .11 .09 .14 .13 .51* 9 .24* .27* -.19* -.12 -.34* -.36* -.09 .04 Note. *p < .05. Tables 18 and 19 describe results of a multiple regression analysis that measured the combined impact of dropout prompts categorized as push or pull, summarized in Table 15, on individual re-engagement factors. As Table 18 indicates, the seven dropout 2 push factor independent variables were associated with 44% of the variance (R = .44, F(7, 95) = 10.61, p = .00) on the Belief a new school will make you feel safe re- 2 engagement factor and 25% of the variance (R = .25, F(7, 95) = 4.50, p = .00) on the Belief a new school will help you be academically successful re-engagement item. As described in Table 19, the dropout independent variables categorized as pull had an even stronger relationship with the subsequent selection of a pull-categorized re- engagement factor. In the multiple regression analysis, the pull independent variables 2 were associated with 77% of the variance (R = .77, F(5, 95) = 64.60, p = .00) on the 2 Changes in childcare, 55% of the variance (R = .55, F(5, 95) = 23.70, p = .00) on the 67 2 Changes in your responsibilities to other family members, and 74% of the variance (R = .44, F(5, 95) = 14.12, p = .00) on the Changes in employment status items. The multiple regression also indicates that two of the re-engagement items have a significant relationship with both the push and pull independent variables. In the model, 2 25% of the variance (R = .25, F(7, 95) = 4.50, p = .00) in the Belief that a new school’s flexible schedule will allow you to work or care for family while attending school re- 2 engagement item is explained by push independent variables and 58% (R = .44, F(5, 95) = 27.09, p = .00) is explained by the pull independent variables. Similarly, both the push 2 2 (R = .44, F(7, 95) = 10.61, p = .00) and pull (R = .30, F(5, 95) = 8.44, p = .00) independent variables were associated with a significant portion of the variance on the Belief a new school will make you feel safe re-engagement item. Table 18 Summary of Multiple Regression Results for Push IVs Dependent Variable R 2 2R Adj R SE Changes in childcare that allow you to return .41 .17 .11 .63 to school Changes in your responsibilities to other .41 .17 .11 .72 family members that allow you to return Changes in employment status or work .44 .19 .13 .90 schedule that allow you to return to school Belief new school’s flexible schedule will allow you to work/care for family while also .50 .25 .20 1.19 attending school Belief that a new school will have an .66 .44 .40 .77 environment that will help you feel safe Belief that a new school will help you be .50 .25 .19 .67 academically successful Your motivation to return to school and .34 .12 .05 .72 complete your high school education Your desire to go to college or a trade school .43 .18 .12 .96 Someone else in your life who is encouraging or pressuring you to return to school .39 .16 .10 1.00 68 Table 19 Summary of Multiple Regression Results for Pull IVs Dependent Variable 2 2R R Adj R SE Changes in childcare that allow you to return .88 .77 .76 .33 to school Changes in your responsibilities to other .74 .55 .53 .53 family members that allow you to return Changes in employment status or work .86 .74 .74 .51 schedule that allow you to return to school Belief new school’s flexible schedule will allow you to work/care for family while also .76 .58 .56 .88 attending school Belief that a new school will have an .55 .30 .27 .85 environment that will help you feel safe Belief that a new school will help you be .36 .13 .08 .72 academically successful Your motivation to return to school and .41 .17 .12 .69 complete your high school education Your desire to go to college or a trade school .37 .14 .09 .98 Someone else in your life who is encouraging or pressuring you to return to school .46 .21 .17 .96 Research Question Three During the four-month data collection window, 103 students responded to the survey instrument, representing approximately 47% of the 217 students referred to Reconnection Services during that time period. The following section summarizes results from each section of the survey instrument. Dropout factors. Participants were asked to rate the importance of 12 factors in their decision to leave their last school as (a) not important, (b) somewhat important, (c) important, or (d) very important. Seven of 12 factors were categorized as push factors and five as pull factors. The responses were relatively evenly distributed, with only the Academic issues and Engagement issues items rated as important or very important by more than 50% of respondents (Table 20). Nonetheless, it should be noted that on 69 average, 43% of students identified a push factor as important or very important, compared to 30% of students who identified a pull factor as important or very important. Despite the relatively even distribution of responses, a couple of observations should be noted. First, each of the dropout factors was rated as very important by at least some students. Even the lowest rated item, legal issues or incarceration, was identified as very important in 5% of students’ decisions to leave school. Second, two of the pull categorized dropout items were rated as very important by more than a third of respondents. The adverse life event item was rated as very important by 38% of respondents and the mental health item by 36% of students. While on average more students identified a push categorized item and important or very important, the two highest rated items were both categorized as a pull factor. Re-engagement factors. In the second section of the survey instrument, students were asked to rate the importance of nine factors in their decision to return or re-engage with school. Of these nine factors, two were related to push factors and four to pull factors. Three factors were not categorized as either push or pull. Unlike the dropout factor section, clear patterns emerged from student responses. The uncategorized and push factor items were each rated as important or very important by more than 50% of participants (Table 21). Interestingly, almost 90% of students rated Belief that a new school will help you be academically successful (87%) and Your motivation to return to school and complete your high school education (89%) as important/very important. As will be discussed later, these two items were not an explicit part of the push/pull theoretical framework, but they were identified as individual factors in some of the literature that I reviewed. 70 Table 20 Summary of Responses to Dropout Factor Items Not Somewhat Very Category Item Important Important Important Important Push Feelings that you did not fit in 20% 32% 34% 14% or belong at your last school Problems with some of your 25% 31% 20% 23% teachers and/or other students Disciplinary issues, such as 50% 19% 16% 16% suspensions or expulsions Not feeling safe at school 38% 24% 25% 13% Academic issues, such as poor grades or not being on 19% 27% 37% 17% track to graduate Engagement issues, such as feeling like school work is 14% 36% 24% 26% boring or not relevant Cultural issues, such as feeling that school does not fit 35% 28% 16% 21% in with how you like to learn Pull Mental health or anxiety 38% 15% 12% 36% The need to work 66% 4% 15% 16% Pregnancy or the need to take 91% 0% 1% 8% care of your child Something happened to you or your family that made it 43% 9% 11% 38% difficult to stay in school Legal issues, such as an arrest 79% 9% 8% 5% or incarceration Overall, on average only about 19% of students rated a pull factor as important or very important, compared to an average of 75% of students who rated a push factor as important or very important. This is unsurprising, because only 9% of students identified a pregnancy or the need to parent as an important factor in their decision to leave school. Nonetheless, as described in the earlier construct validity section, students who identified a pull factor as important in their decision to leave school were likely to identify a corresponding pull factor in their decision to return to school. 71 Table 21 Summary of Responses to Re-engagement Factor Items Not Somewhat Very Category Item Important Important Important Important Push Belief that a new school will have an environment that 18% 19% 43% 20% will help you feel safe Belief that a new school will help you be academically 4% 9% 53% 34% successful Pull Changes in childcare that 92% 1% 3% 4% allow you to return to school Changes in your responsibilities to other 84% 6% 7% 4% family members Changes in employment status or work schedule that 74% 6% 14% 7% allow return to school Belief new school’s schedule will allow you to 62% 3% 9% 26% work or care for family N/A Your motivation to return to school and complete your 3% 9% 49% 40% high school education Your desire to go to college 17% 27% 32% 24% or a trade school Someone in your life is encouraging or pressuring 20% 22% 35% 22% you to return to school School characteristics. In the last section of the survey, students rated the importance of selected school characteristics that they desired in a new school. Although not phrased this way on the instrument, the intent of this section was to identify the characteristics that might attract students back into school. With that in mind, as summarized in Table 22, only Opportunities to earn dual credit was rated as important or very important by fewer than 66% of students. Interestingly, with only 23% of students rating it as important or very important, this finding contradicts a prediction of the expert advisory group that students could be drawn back with the opportunity to earn college 72 credit. Instead, 83% of respondents identified personalized and individualized instruction as important or very important, and more than 77% rated (a) flexible schedules, (b) safe environments, and (c) opportunities to make up credits as important or very important. Unlike the dropout and re-engagement sections, the school characteristics section did not reveal large differences in average student responses to items categorized as push or pull. Table 22 Summary of Responses to School Characteristic Items Not Somewhat Very Category Item Important Important Important Important Push Safe and welcoming 7% 17% 31% 46% environment Positive relationships with 8% 25% 38% 29% teachers Personalized and individual 4% 14% 49% 34% instruction Pull Flexible schedule 4% 18% 35% 43% Opportunities to make up 13% 11% 34% 43% credits Opportunities to finish 1% 29% 20% 50% quickly Extra supports, such as 7% 17% 55% 21% counseling or tutoring N/A Opportunities to earn dual 48% 30% 12% 11% credit/college credit Open-ended responses. Very few students responded to the open-ended questions. Out of 103 surveys, only 18 included legible responses to an open-ended item. All 18 of the students who answered an open-ended question responded to the Please use the space below if you would like to tell us anything else about your decision to leave your last school prompt, and two of the 18 also responded to the Please use the space below if you would like to tell us anything else about your decision to return to school open-ended prompt. None of the students answered the prompt requesting additional information about the characteristics they desired in a new school. 73 All qualitative data points were categorized as a push or pull factor and could also be connected to a closed-ended dropout or re-engagement variable. Table 23 summarizes the qualitative data from open-ended items. It appears that pull factors (e.g., mental health and adverse life events) played a significant role in the decision to leave school for the students who responded. This pattern is consistent with results from the closed-ended dropout factor items. As noted earlier, the two items rated as very important by the greatest percentage of students were mental health and adverse life events. Table 23 Summary of Open-Ended Responses Domain Closed-ended Code Comment Push/Pull Leave Discipline and behavior Was expelled Push I got kicked out Engagement and School was boring Push relevance Academic performance Was not passing classes Push Stopped going because I was failing Failed algebra twice Mental health or anxiety Had to be hospitalized for depression Pull Too anxious to go to school Depression Pregnant or parenting No childcare; Baby still breastfeeding Pull Need to work Had to work at family restaurant Pull Working full time Working 30 hours each week Adverse life event My grandma was sick and I had to take Pull care of her My mom lost her job and we had to move apartments Health issues I got sick I was kicked out of the house. Couch surfing Return Safe environment Lots of support so I feel safe Push Changes in childcare My grandma is able to watch my baby Pull so I can go to school 74 Research Question Four One-way ANOVA and Tukey post-hoc analyses indicated significant variation (p < .05) in how students responded to the survey based on sex, socioeconomic status, and race/ethnicity. Significant variation was not found with SPED as an independent variable. The following section describes results from the ANOVA for each demographic variable. Variation by sex. As described in Table 24, an ANOVA indicated that male and female responses to nine survey items varied in ways that were statistically significant. Because the analysis included fewer than three groups, post-hoc tests could not be conducted to identify the source of variation. Nonetheless, as described in Table 25, the direction of the statistically significant variation could be inferred from the descriptive statistics provided by the ANOVA. In SPSS, a value of one was assigned to Not Important and four to Very Important. I then made inferences about whether males or females were more likely to rate an item as important to their decision to leave or return by comparing the means of items for which the ANOVA had identified significant variation. With that in mind, sex had a significant impact on how students responded to five dropout factor items. The ANOVA and descriptive statistics suggested that males were more likely than females to rate poor sense of belonging, F(1, 101) = 6.10, p = .015, problems with teachers and students, F(1, 101) = 9.07, p = .003, academic issues, F(1, 101) = 9.34, p = .003, and engagement issues, F(1, 101) = 6.87, p = .01, as important factors in their decision leave school. On the other hand, females were more likely to identify pregnancy or the need to care for a child as an important reason in their decision to leave, F(1, 101) = 10.77, p =. 001. 75 Significant variation was also detected in three re-engagement factor and school characteristic items. ANOVA indicated that sex significantly impacted responses to changes in childcare, F(1, 101) = 8.61, p = .004, responsibilities to family, F(1, 101) = 7.22, p = .008, and desire to go to college or trade school, F(1, 101) = 4.90, p = .029 re- engagement factor items. Responses to the extra supports school characteristics item varied significantly, F(1, 101) = 5.60, p = .02. In each of these cases, descriptive statistics suggest that females were more likely than males to rate these as important. Table 24 One-Way ANOVA of Responses to Survey Items by Sex Domain Item Source df SS MS F p Leave Feelings that you did B/t groups 1 5.40 5.40 6.10 .015 not fit in or belong W/in groups 101 89.47 .89 Total 102 94.87 Problems with some B/t groups 1 10.31 10.31 9.07 .003 of your teachers W/in groups 101 114.74 1.14 and/or students Total 102 125.05 Academic issues, B/t groups 1 8.45 8.45 9.34 .003 poor grades or not W/in groups 101 91.30 .91 being on track Total 102 99.75 Engagement issues, B/t groups 1 6.75 6.75 6.87 .010 such as feeling like W/in groups 101 99.23 .98 school is boring Total 102 105.98 Pregnancy or need to B/t groups 1 6.69 6.69 10.77 .001 take care of a child W/in groups 101 62.75 .62 Total 102 69.44 Return Changes in childcare B/t groups 1 3.57 3.57 8.61 .004 that allow you to W/in groups 101 41.92 .42 return to school Total 102 45.50 Changes in B/t groups 1 4.01 4.01 7.22 .008 responsibilities to W/in groups 101 56.05 .56 family members Total 102 60.06 School Your desire to go to B/t groups 1 4.98 4.98 4.89 .029 college or trade W/in groups 101 102.73 1.02 school Total 102 107.71 Extra supports, such B/t groups 1 3.47 3.47 5.59 .020 as counseling or W/in groups 101 62.74 .62 tutoring Total 102 66.21 76 Table 25 Summary of Descriptive Statistics for One Way ANOVA 95% CI Item Sex n M SD SE Min Max Lower Upper Feelings that you M 52 2.63 .97 .14 2.36 2.90 1 4 did not fit in or F 51 2.18 .91 .13 1.92 2.43 1 4 belong Total 103 2.41 .96 .10 2.22 2.60 1 4 Problems with some M 52 2.73 1.09 .15 2.43 3.03 1 4 of your teachers F 51 2.10 1.04 .15 1.80 2.39 1 4 and/or students Total 103 2.42 1.11 .11 2.20 2.63 1 4 Academic issues, M 52 2.79 .94 .13 2.53 3.05 1 4 such as poor grades F 51 2.22 .97 .14 1.94 2.49 1 4 Total 103 2.50 .99 .10 2.31 2.70 1 4 Engagement issues M 52 2.88 .96 .13 2.62 3.15 1 4 F 51 2.37 1.02 .14 2.09 2.66 1 4 Total 103 2.63 1.02 .10 2.43 2.83 1 4 Pregnancy or need M 52 1.00 .00 .00 1.00 1.00 1 4 to take care of a F 51 1.51 1.12 .16 1.19 1.82 1 4 child Total 103 1.25 .83 .08 1.09 1.41 1 4 Changes in M 52 1.00 .00 .00 1.00 1.00 1 4 childcare that allow F 51 1.37 .92 .13 1.12 1.63 1 4 you to return Total 103 1.18 .67 .07 1.05 1.31 1 4 Changes in M 52 1.12 .38 .05 1.01 1.22 1 4 responsibilities to F 51 1.51 .99 .14 1.23 1.79 1 4 family members Total 103 1.31 .77 .08 1.16 1.46 1 4 Your desire to go to M 52 2.42 .98 .14 2.15 2.70 1 4 college or trade F 51 2.86 1.04 .15 2.57 3.16 1 4 school Total 103 2.64 1.03 .10 2.44 2.84 1 4 Extra supports M 52 2.73 .74 .10 2.52 2.94 1 4 F 51 3.10 .83 .12 2.86 3.33 1 4 Total 103 2.91 .81 .08 2.76 3.07 1 4 77 Variation by socioeconomic status/parent educational level. As recommend by Shavers (2007), the highest educational level attained by students’ parents was used as a compositional variable to serve as a proxy for socioeconomic status (SES). The choice to use parent educational level as a proxy for SES was in response to three factors: (a) concerns regarding the validity and reliability of free and reduced lunch data, (b) lack of access to student-level free and reduced lunch data, and (c) reservations about the ability of students to provide reliable information about their family’s income and SES (Shavers, 2007). Although there is some research supporting the use of educational level as a proxy for SES (Galobardes, Shaw, Lawlor, Lynch & Smith, 2006; Shavers, 2007), it may be more appropriate for this context to discuss how students’ responses to survey items varied according to their reports of the highest educational level attained by either of their parents. As described in Table 26, one-way ANOVA indicated that variation based on parent education level was significant (p < .05) on responses to four survey items. The variation was significant on responses to three dropout factor items: (a) Academic issues, such as poor grades or not being on track to graduate, F(6, 96) = 2.54, p = .025; (b) Engagement issues, such as feeling like school is boring or irrelevant, F(6, 96) = 2.60, p = .022; and (c) Cultural issues, such as feeling like school does not fit with how you like to learn, F(6, 96) = 3.00, p = .011. Additionally, parent educational level had a significant impact on responses to one re-engagement factor item, Someone else in your life is encouraging or pressuring you to return to school, F(6, 96) = 2.29, p = .041. Post- hoc tests were not performed because at least one group had fewer than two cases. As such, while this analysis is able to detect the presence of significant variation, the sample 78 was too small to identity which educational attainment variables contributed to the significance. Table 26 One-Way ANOVA of Survey Responses by Parent Educational Level Domain Item Source df SS MS F p Leave Academic issues, such B/t groups 6 13.66 2.28 2.54 .025 as poor grades or not W/in groups 96 86.09 .90 being on track Total 102 99.75 Engagement issues, B/t groups 6 14.81 2.47 2.60 .022 such as feeling like W/in groups 96 91.17 .95 school is boring Total 102 105.98 Cultural issues, such as B/t groups 6 20.96 3.49 2.96 .011 feeling the school does W/in groups 96 113.45 1.18 not fit how you learn Total 102 134.41 Return Someone else in your B/t groups 6 14.15 2.36 2.29 .041 life is encouraging or W/in groups 96 98.73 1.03 pressuring… Total 102 112.87 Variation by race/ethnicity. A one-way ANOVA, summarized in Table 27, indicated that student’s self-reported race/ethnicity contributed to significant variation on six dropout factor items (p < .05), but was not associated with any significant variation on responses to the re-engagement or school characteristic items. As described in Table 28, Tukey post-hoc tests revealed that the variation was predominantly between African American and Latino/a and White students, and between American Indian/Alaska Native and Latino/a students. Note that only two students out of the 103-student sample identified as Asian or Pacific Islander, which initially prevented SPSS from conducting a Tukey post-hoc analysis. With that in mind, the results described below are based on a one-way ANOVA and Tukey post-hoc test with the two Asian and Pacific Islander students removed from the sample. 79 All six items in which race/ethnicity significantly impacted responses to the survey items were from the dropout factor section. Items with significant variation included prompts addressing (a) problems with students and teachers, F(4, 96) = 4.00, p = .005; (b) disciplinary issues, F(4, 96) = 8.18, p = .000; (c) feeling unsafe, F(4, 96) = 4.82, p = .001; (d) academic issues, F(4, 96) = 3.30, p = .014; (e) engagement issues, F(4, 96) = 4.44, p = .002; and (f) cultural issues, F(4, 96) = 15.43, p = .000. Interestingly, all of these items were coded as push factors, suggesting that a student’s race/ethnicity is associated with variation in how they experience being pushed out of school. Additionally, as described in Tables 28 and 29, the Tukey post-hoc test indicated that the difference in means between African American students and Latino/a, White, and multiracial students contributed to this significant variation on all six of the items. Additionally, differences in means between American Indian/Alaska Native and Latino/a students contributed to significant variation on the discipline and feeling unsafe at school items. It should be noted that in all of these cases, the means of African American and American Indian/Alaska Native student responses were greater than their counterparts. In other words, African American and American Indian/Alaska Native students were more likely to rate these push-categorized dropout factors as important or very important factors in their decisions to leave school. 80 Table 27 One-Way ANOVA of Responses to Survey Items by Race/Ethnicity Item Source df SS MS F p Problems with some of B/t groups 4 17.52 4.38 4.00 .005 your teachers and/or other students W/in groups 96 105.17 1.10 Total 100 122.69 Disciplinary issues, such B/t groups 4 33.04 8.26 8.18 .000 as suspensions and expulsions W/in groups 96 96.92 1.01 Total 100 129.96 Not feeling safe at B/t groups 4 18.85 4.71 4.82 .001 school W/in groups 96 93.92 .98 Total 100 112.77 Academic issues, such B/t groups 4 12.01 3.00 3.30 .014 as poor grades or not being on track to W/in groups 96 87.22 .91 graduate Total 100 99.23 Engagement issues, such B/t groups 4 16.42 4.11 4.44 .002 as feeling like school is boring or irrelevant W/in groups 96 88.75 .92 Total 100 105.17 Cultural issues, such as B/t groups 4 51.98 12.99 15.43 .000 feeling the school does not fit how you learn W/in groups 96 80.84 .84 Total 100 132.81 81 Table 28 Tukey Post-Hoc Results for Race/Ethnicity ANOVA 95% CI DV Race (I) Race (J) M (I-J) SE Lower Upper Problems with some African AI/AN .44 .42 -.72 1.59 of your teachers American Latino 1.15* .34 .21 2.09 and/or other White .57 .25 -.12 1.26 students Multi 1.22* .44 .00 2.44 Disciplinary issues, African AI/AN .05 .40 -1.06 1.16 such as suspensions American Latino 1.46* .32 .56 2.36 and expulsions White .97* .24 .31 1.64 Multi 1.39* .42 .22 2.56 AI/AN Afr. Amer. -.05 .40 -1.16 1.06 Latino 1.41* .45 .17 2.65 White .92 .39 -.16 2.00 Multi 1.34 .52 -.11 2.79 Not feeling safe at African AI/AN -.23 .39 -1.32 .86 school American Latino 1.23* .32 .34 2.12 White .39 .24 -.25 1.05 Multi .80 .41 -.35 1.95 AI/AN Afr. Amer. .23 .39 -.86 1.32 Latino 1.46* .44 .25 2.68 White .63 .38 -.43 1.69 Multi 1.04 .51 -.39 2.46 Academic issues, African AI/AN .12 .38 -.93 1.17. such as poor grades American Latino .09 .31 -.77 .94 or not being on track White .65* .23 .02 1.28 to graduate Multi 1.01 .40 -.10 2.12 Engagement issues, African AI/AN .41 .38 -.65 1.47 such as feeling like American Latino .73 .31 .33 2.57 school is boring or White .70* .23 .06 1.33 irrelevant Multi 1.45* .40 .33 2.57 Cultural issues, such African AI/AN .88 .36 -.13 1.89 as feeling the school American Latino 1.19* .30 .37 2.01 does not fit how you White 1.70* .22 1.09 2.30 learn Multi 1.26* .38 .19 2.33 Note. *p < .05 82 Table 29 Descriptive Statistics for Race/Ethnicity ANOVA AI/AN Latina/o Afr. American White Multi Item M SD M SD M SD M SD M SD Problems with some of your teachers 2.50 1.31 1.79 .89 2.94 .96 2.37 1.14 1.71 .76 and/or other students Disciplinary issues, such as suspensions 2.63 1.30 1.21 .80 2.68 1.11 1.71 .98 1.29 .49 and expulsions Not feeling safe at 2.75 1.49 1.29 .47 2.52 .89 2.12 1.10 1.71 .76 school Academic issues, poor grades or not 2.75 1.17 2.79 .89 2.87 .92 2.22 .91 1.86 1.22 being on track to graduate Engagement issues, feeling like school is 2.75 1.28 2.43 .85 3.16 .93 2.46 .98 1.71 .76 boring or irrelevant Cultural issues, feeling school does 2.38 1.30 2.07 1.14 3.26 .89 1.56 .71 2.00 1.16 not fit how you learn 83 Variation by special education status. A one-way ANOVA did not detect any significant variation based on enrollment in Special Education. Special Education status, or the possession of an Individualized Education Plan (IEP) was not addressed in the demographics section of the survey. Instead, student’s status as a Special Education student was determined by looking up each student who completed a survey in the district’s student information system. Students determined to be enrolled in Special Education were assigned a code in the master key. Thirty one of the 103 students who responded to the survey were identified as SpEd, which is consistent with Reconnection Services referral patterns, but the ANOVA did not identify any significant variation (p < .05). 84 CHAPTER V DISCUSSION In the section that follows, I provide a summary of findings for each of my research questions. With those findings in mind, I then discuss the limitations of this study, and identify key considerations for interpreting the results. Finally, I discuss the implications of this study for re-engaging students and recommendations for practice and future research. Research Question 1: Re-engagement Instrument Variables This study’s first research question asks what variables should be included in an instrument designed to identify why students decide to return to school, and how those decisions are related to the factors that caused them to leave. In other words, which dis- engagement and re-engagement factors should students be asked to rate in terms of importance in their decisions to leave and return to school? At its heart, this is a question about the content and construct validity of the instrument I developed. With that in mind, the goal of the first phase of the study was to identify variables for which there was a high degree of alignment across the literature, theoretical framework, and expert and student advisory groups. As will be discussed in the following section, there was substantial, but incomplete agreement on key variables between the advisory groups and with the literature. Dropout variables. In general, there was a high degree of congruence for the dropout factors that were (a) cited in the literature, (b) predicted by the theoretical framework, (c) identified by content experts, and (d) listed by students representing the population of interest. The literature I reviewed, and theoretical framework I derived 85 from it, suggests that students leave school in response to institutional and individual push and pull factors. As summarized in Table 29, these include push factors like (a) poor academic performance (Berliner et al., 2008; Boylan & Renzulli, 2014; Chuang, 1997; Entwisle et al., 2004; Rumberger, 2004, 2011; Wayman, 2002, 2004); (b) conflicts with peers and adults (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Rumberger, 2004, 2011; Wayman, 2002); (c) discipline (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Rumberger, 2004, 2011); (d) low levels of engagement (Boylan & Renzulli, 2014; Entwisle et al., 2004; Rumberger, 2004, 2011); and (e) poor sense of belonging (Bickerstaff, 2010). Additionally, students may feel pulled out of school in response to (a) adverse life events, (b) pregnancy or the need to care for family members, and (c) the need to work (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Epstein, 1992; Rumberger, 2004, 2011). As summarized in Table 29, the expert and student advisory groups largely agreed with each other, and identified dropout factors that were highly consistent with those variables predicted by the literature and theoretical framework. Despite this congruence, a few nuanced differences should be noted. Both advisory groups highlighted the importance of mental health, especially anxiety, in pulling students out of school. While the literature I reviewed discussed a poor sense of belonging and adverse life events as factors associated with dropping out, it did not explicitly address mental health or anxiety as key factors. The student group, in particular, identified anxiety as one of the most important factors in their personal decision to leave school. Additionally, although the students and content experts agreed about the importance of a lack of cultural relevance and engaging learning experiences in their decision to leave, the students were more 86 likely to focus on the symptoms of disengagement than the content experts, who focused on the structural roots. For example, students shared that they “got bored” or “lost motivation” in their classes, while the content experts pointed to curricular and instructional deficiencies. Re-engagement variables. As indicated in Table 29, there were similarly high levels of agreement across the literature, theoretical framework, and advisory groups regarding re-engagement variables. A key tenet of the theoretical framework derived from my review of the literature is that students’ re-engagement behavior is related to their dropout behavior. As Bickerstaff (2010) noted, students who have been pushed out are likely to return if they believe that can return to a program that will mitigate the factors that pushed them out of their previous school. Students who are pulled out, on the other hand, often choose to return if their employment status changes or if they are able to secure childcare (Boylan & Renzulli, 2014; Entwisle et al., 2004). In other words, push-out students are likely to return if they are confident that the nature of their school experience will be different, while pull-out students are more likely to return if the circumstances that made remaining in school untenable change. As predicted by the theoretical framework, both the student and content expert advisory groups identified re-engagement factors that could be categorized as responses to being pushed or pulled out of school. For example, both groups highlighted the importance of a “fresh start” in a school or program that might be a better fit, and where students believed they could find a sense of belonging and academic success. In my theoretical model, these would be factors that students who were pushed out of school might identify as important to their decision to return. Similarly, the advisory groups 87 noted that changes in mental health, employment status, or childcare may contribute to students’ decisions to re-engage. As predicted by the theoretical model, these could be categorized as re-engagement factors that pull-out students might identify as associated with their decision to return to school. School characteristic variables. Input from the advisory groups on the characteristics students desire in a new school was also aligned with the literature, with the exception of dual credit opportunities. In particular, both groups identified school characteristics for which there was some consensus in the limited literature that I reviewed, such as: (a) positive peer and adult relationships, (b) individualized instruction and curriculum, and (c) flexible schedules (Bickerstaff, 2010; Epstein, 1992; Iachini et al., 2013; Lagana-Riodan et al., 2011). The first two of these characteristics can be conceptualized as efforts to change the nature of the student-school relationship, and as responses to the needs of students who have been pushed out of school. Flexible schedules, in turn, can be viewed as an attempt to make it easier for students who were pulled out to return. It should be noted that all of the students in the advisory group were enrolled in an alternative program organized along these principles, and several of the content experts were current or former administrators of alternative schools. As such, it is not surprising that there was consensus regarding these characteristics between the groups and with the literature. Interestingly, disagreement between the two group regarding the amount of structure students desire in new programs mirrored a lack of consensus in the literature. As I noted in the review, some researchers (Epstein, 1992) reported that students desired highly structured environments, while others (Lagana-Riordan et al., 2011) found that 88 students preferred environments that allowed for greater student autonomy. The content expert group believed that students would prefer flexible programs that treated them like adult learners. In contrast, several members of the student advisory group said that they would be more successful in programs that limited their choice and flexibility. Future research could examine how structural preferences varied according to student age and other demographic variables. In general, as described in Table 29, there was a high degree of consensus between the advisory groups, and with this study’s theoretical framework and the literature from which it was derived. As discussed earlier, this consensus and alignment increased my confidence in the content and construct validity of the variables I included in the survey instrument. Research Question 2: Validity and Reliability A key goal of this study was to pilot and field test a re-engagement behavior survey instrument to determine whether it possessed enough reliability and validity to warrant inclusion in the PPS Reconnection Services intake process. Thorndike and Thorndike-Christ (2010) recommend a reliability threshold of at least r = .85 for high stakes assessments and survey instruments. With that in mind, all of the items in the survey instrument demonstrated Pearson correlations above the r = .85 threshold in a sample that re-took the survey one to two weeks after initial administration. Having said that, the sample only represented a convenience sample of 23 of the 103 students who completed the survey during the enrollment window. While promising, if use of the survey is expanded, test-retest reliability should continue to be assessed with a larger sample of students. 89 Table 30 Variable Alignment Lit./ Expert Student Domain Instrument variable Theory Group Group Dropout Lack of belonging X X X Problems with teachers or peers X X X Disciplinary issues X X X Not feeling safe at school X X X Legal issues (arrest or incarceration) X X Academic issues (poor grades or off-track) X X X Engagement issues (school boring/irrelevant) X X X Cultural incongruence X X Mental health or anxiety X X Need to work X X X Pregnancy or need to care for child X X X Adverse life event X X X Return Change in childcare X X X Change in caregiving responsibilities X X X Change in employment status/work schedule X X X Belief flexibility would allow work/parenting X X X Belief new school would increase safety X X X Belief new structure would increase success X X X Internal motivation to complete high school X X X Desire to attend college or trade school X X X External pressure (parent, friends, parole) X X X School Flexible schedule X X X Safe and welcoming environment X X X Positive relationships with teachers X X X Personalized/individual instruction X X X Opportunity to make up credit X X X Opportunity to finish quickly X X X Opportunity for dual credit/college credit X Extra supports (counseling, tutoring) X X X 90 The theoretical framework on which the survey instrument was based assumes a relationship between students’ dropout and re-engagement behavior. More specifically, it predicts that students who indicate that they were pushed out of school will choose to re- engage if they believe that they can return to a school or program that will mitigate the factors that contributed to their decisions to leave. As such, they would be more likely than pull-out students to identify factors such as a belief that a new school would allow them to be academically successful and provide an increased sense of belonging and safety. Additionally, the framework views students who say they were pulled out of school as rational actors who make informed decisions regarding the opportunity costs of staying in school. Therefore, they would choose to return if those costs change (e.g. lose a job, obtain childcare) or if they believe that a new school can provide the flexibility and support they need to balance all of their responsibilities. Therefore, if the survey instrument is a valid measure of the student dropout and re-engagement behavior construct, I would expect to see a relationship between responses to push/pull dropout and re-engagement items. With this expectation in mind, results from a multiple regression analyses provided promising data in support of construct validity. A combination of independent variables comprised of dropout items categorized as push factors predicted a significant percentage of the variables on two re-engagement items categorized as push factors. Similarly, the pull factor dropout independent variables explained a significant portion of the variance on three pull-categorized re-engagement variables. These results suggest that the type of factors that students associate with their decisions to leave or disengage with school predict the factors they say influence their choice to return. In other words, as 91 predicted by the theoretical model, the multiple regression analysis results indicate that student’s disengagement and re-engagement behavior appear to be linked. If a student selects dropout factors that are categorized as push or pull, they are more likely to select re-engagement factors that are correspondingly categorized as push or pull. Furthermore, the multiple regression results support the categorization of re- engagement variables as push or pull factors. Each re-engagement variable predicted by the theoretical framework to be a response to a push or pull factor had a high degree of its variation explained by the corresponding set of push and pull predictor variables. Research Question 3: Variables Associated with Return The third research question addresses the overall purpose of the survey instrument and this study, to gather information on why students decide to return to school after dropping out or disengaging. Although the question specifically focuses on why students return to school, I will also discuss findings regarding why students dropped out. Dropout responses. No clear patterns emerged from students’ responses to the Why did you decide to leave your last school prompt. Roughly equal numbers of students rated four of the seven items categorized as a push factor as not important/somewhat important and important/very important. Slightly more students rated disciplinary issues, not feeling safe at school, and cultural issues as not important or somewhat important than important or very important. Compared to the responses to the push factors, fewer students rated the five pull factors as important/very important, but both mental health and adverse life events were rated as such by more than 40% of respondents. It should be noted that only 9% of respondents rated pregnancy or the need to care for a child as 92 important/very important in their decision to leave, which is roughly equal to the number of students (8.7%) who identified as pregnant or parenting in the demographics section. Overall, the results from the dropout factor items support a couple of cautious conclusions. First, students appear to decide to dropout or disengage in response to a diverse set of factors. Interestingly, every dropout factor was rated as very important by at least a few students, but only two of the factors were rated as very important by more than a third of respondents. This suggests that dropout behavior is not monolithic and should be understood in the context of individual experiences. Second, slightly more students identified a push factor as important or very important than a pull factor. This does not suggest that factors like the need to work or parent are less important than lack of engagement of poor academic performance, but it does help frame interpretation of the re-engagement factor responses. Re-engagement factors. A clear majority of students rated the push-coded re- engagement factors as important or very important. About 87% of students rated Belief that a new school will help you be academically successful as important/very important to their decision to return, and 63% rated Belief that a new school will have an environment that will help you feel safe as important/very important. This suggests that students, especially those who felt pushed out, will choose to return if they believe that a new learning environment can offer them a significantly different experience. Fewer students rated the pull-coded re-engagement factors as important/very important. This does not imply that changes in childcare, family responsibilities, or work schedules are not important in students’ decisions to return to school. Instead, it likely reflects the fact that fewer students rated those as factors in their decision to leave school. 93 If 66% of students indicated that the need to work was not important to their decision to leave, it should not be surprising that 74% responded that a change in their work status was not an important factor in their decision to return. Moreover, like the pattern of responses to the dropout prompt, each of the pull-coded re-engagement factors was rated as very important by at least a portion of respondents. Finally, intrinsic motivation, and extrinsic factors, such as the desire to attend college or external pressure to return, emerged as important variables in students’ decision to return. Even though these factors were not explicitly predicted by the push/pull factor theoretical model, they were discussed as individual factors in some of the literature that I reviewed (Entwisle et al., 2004; Iachini et al., 2013; McDermott et al., 2016; Wayman, 2002; Zaff et al., 2014). Additionally, both the student and expert advisory groups identified students’ desire to complete their secondary education and external pressure from family and peers as important factors in their decision to return. Research Question 4: Demographic Variation Demographic variation in student re-engagement behavior was reported in eight of the 14 articles I reviewed for this study (Barrat, 2016; Berliner et al., 2008; Boylan & Renzulli, 2014; Chuang, 1997; Entwisle et al., 2004; Wayman, 2001, 2002; Zaff et al., 2014). Consistent with findings from the literature review, students’ responses on this study’s survey instrument exhibited statistically significant variation on three demographic variables: sex, parent educational level (SES proxy), and race/ethnicity. Variation by sex. Results from an ANOVA suggest that males were more likely to leave school in response to push factors than females, and females were more likely to identify pregnancy or the need to care for a child as important factors in their decision to 94 leave. More specifically, the males in this study were more likely than females to feel pushed out of school because of (a) a poor sense of belonging, (b) problems with teachers and students, (c) academic problems, and (d) a lack of engagement. However, sex was not associated with statistically significant variation in how males and females responded to the push-coded re-engagement factors. In other words, even though males were more likely to say that they were pushed out of school, they were not more likely than females to indicate that they chose to return because they believed a new school would help them feel safe and academically successful. Note that this does not imply that males did not rate those factors as important, only that their responses did not vary from females’ responses in ways that were statistically significant. Females, however, were more likely to rate changes in childcare, changes in caregiving responsibilities, and the desire to go to college as important in their decision to return. Overall, these findings suggest that the boys in this study experienced school in ways that were different than girls in several key respects. More specifically, they were more likely to feel pushed out of school than their female counterparts. It should be noted that this is consistent with referral patterns to the Reconnection Services dropout recovery program and PPS’ graduation rates. During the 2018-19 school year in which this study was conducted, 56% of students referred to the program were males (PPS, 2019c). Additionally, the 2017-18 four-year cohort graduation rate for boys was five percentage points lower than girls (PPS, 2019b). Variation by parent educational level. As described in the methods section, this study used parent educational level as a proxy indicator for socio-economic status (Shavers, 2007). An ANOVA detected the presence of statistically significant variation 95 on three dropout factors, all of which were categorized as push factors. Although a post- hoc test could not be performed because of the size of the sample, which contributed to at least one group having fewer than two cases, it is possible to infer that differences in a parents’ educational level contributed to differences in students’ responses to some of the dropout push factors. Moreover, if one accepts parent educational level as a proxy for SES, then student SES may be associated with the degree to which students feel pushed out of school. Variation by race and ethnicity. In Portland Public Schools, African American and American Indian/Alaska Native students are less likely to graduate as part of their four-year cohort than their White counterparts, and are more likely to be referred to Reconnection Services (PPS, 2019c). During the 2017-18 school year, 71% of African American students and 41% of American Indian/Alaska Native students graduated on time, as compared to 83% of White students (PPS, 2019b). Additionally, during the 2018-19 school year, African American students represented 22.3% of referrals to Reconnection Services and American Indian/Alaska Native students 2.2% of referrals, even though the two student groups were 9.3% and 0.6% of the high school population, respectively (PPS, 2019c). With this in mind, it is unsurprising that ANOVA analyses revealed statistically significant variation in how these two groups responded to the dropout factor prompts. Nonetheless, the ANOVA did not identify any significant variation in how racial and ethnic groups responded to the re-engagement and school characteristic prompts. All of the significant variation was related to push-coded dropout items, including (a) problems with peers and teachers, (b) discipline, (c) safety, (d) academic struggle, (e) 96 lack of engagement, and (f) cultural incongruence. In all cases, post-hoc tests indicated that African American and/or American Indian/Alaska Native students were more likely than other racial/ethnic groups to rate these push-out factors as important in their decisions to leave school. While it is possible that these findings would not be surprising to practitioners and researchers, it is worth noting that the two groups with the lowest graduation rates in the district were the most likely to report that they did not feel safe, engaged, or successful in schools. Limitations Several limitations in this study’s methodology should be considered when interpreting its results and discussing opportunities for future research. With that in mind, the following section discusses key quantitative and qualitative data collection and analysis limitations. Sample. The size of the survey sample introduced potential selection bias that limits the generalizability of this study’s findings (Babbie, 2013). As discussed earlier, only 47% of the students who engaged with Reconnection Services during the data collection period responded to the survey. In my original sampling plan, I intended to offer the survey instrument to the entire population, but the school district prohibited me from requiring Reconnection Services staff to directly administer the survey during intake meetings. I believe that this impacted the number of students who were offered the survey, and adversely affected the response rate. Although this convenience sample was demographically similar to the population, it is plausible that selection bias introduced variation that could have skewed the results and impacted the generalizability of the results for the Reconnection Services population 97 and the larger disengaged student community. For example, because Reconnection Services staff were asked to offer the survey to students, instead of administering it as part of the intake process, it is possible that individuals exercised conscious or unconscious bias when determining which students should be offered the instrument. Even though staff were asked to offer the survey to all students, they may have elected to withhold the survey from students they perceived to be vulnerable. On the other hand, they may have decided to be more assertive when offering it to students who they believed would respond in a way that was consistent with their own values or beliefs about the educational system. Similarly, there may have been meaningful differences between students and families who agreed to take the survey, and those who refused. Consequently, it is plausible that the results from the survey may not be representative of the population that engaged with Reconnection Services during the data collection period. Instrumentation. The structure of the survey instrument may also have introduced bias that should be considered when interpreting the results. The survey instrument was primarily comprised of closed-ended items suggested by the literature, advisory groups, and theoretical framework. Despite efforts to establish the content validity of the items, constructing an instrument out of closed-ended items inevitably constrains participants’ responses. In other words, closed-ended items delineate the universe of possible responses and do not provide opportunities for choices that fall outside of the theoretical framework (Babbie, 2013). Students may have decided to return to school for reasons that were not identified by the advisory groups or suggested by the theoretical framework. I added open-ended items in an effort to mitigate the potentially negative impact of the closed-ended items, but as discussed earlier, few students chose to 98 respond to the open-ended questions. In summary, it is possible that the results were influenced or biased by the instrument and/or that students would have responded differently had other response options been available. It is also important to note limitations regarding the reliability of the instrument. Although the results from pre-test/post-test reliability checks during the pilot and field test were promising, the samples were relatively small. Additionally, the convenience sample of students who were available to re-take the survey one to two weeks after initial administration may not have been representative of the larger survey sample or Reconnection Services population. If the district decides to adopt the survey as part of its intake process, additional reliability testing with a larger sample may be warranted. Analysis. At its heart, this study was an attempt to establish the construct validity of the instrument and the push/pull theoretical framework upon which it is based. The multiple regression analysis was an attempt to determine the extent to which results from the survey were congruent with predictions from the theoretical framework. Results from the field-test were promising, suggesting that students’ responses to the dropout behavior items predicted responses to the re-engagement behavior items. However, this was based on students’ ability to accurately reflect on their motivations and behavior. In other words, the argument I made for construct validity was based on an analysis of students’ responses to two related engagement behavior domains, not on any external criterion (Babbie, 2013). As will be discussed in the implications section, additional research following the re-engagement trajectory of students who responded to the survey could strengthen claims of construct validity. 99 Because of the relatively small sample, caution should also be exercised when making inferences from the ANOVA. For example, although the ANOVA indicated that American Indian/Alaska Native students responded to the survey in ways that were significantly different than other racial/ethnic groups, the American Indian/Alaska Native portion of the sample was comprised of only eight students. Qualitative. Qualitative techniques were primarily used with the expert and student advisory groups during the instrument development and content validation phase. As discussed earlier, little qualitative data was collected during the field test, significantly limiting its utility. Having said that, it is important to note the biases of the researcher and subjects when interpreting the qualitative results. As Creswell (2014) notes, it is important to recognize the positionality of the researcher and the impact of their potential biases during qualitative analysis and interpretation. During data collection and analysis, I was the administrator of the Reconnection Services dropout recovery program and the Reconnection Center transitional school. More broadly, I have been involved in alternative education for 13 years and have engaged in advocacy at the district and state level. I am not neutral when it comes to alternative education and the role of traditional conceptualizations of schooling in pushing students out. Both the expert and student advisory groups were aware of my role and beliefs regarding alternative education and disengaged students, which may have influenced the nature of their participation. The expert advisory group was primarily comprised of current and former leaders of alternative schools and re-engagement programs. As such, it is plausible that their feedback regarding the reasons why students leave and then return to school could have been influenced by design decisions they have made in their own schools or programs. 100 For example, a leader of a program with high degrees of scheduling flexibility may have a vested interest in arguing that students decide to come back to school when they believe flexibles schedules will allow them to balance their work or childcare responsibilities. Similarly, all of the participants recruited for the student advisory group were enrolled in an alternative program, which may have biased their input. Instead of identifying why students decide to return, they could have simply been describing the school in which they were currently enrolled. In summary, the students and experts were recruited for their experience and expertise, but that expertise may have biased their perspectives. Implications and Recommendations for Future Research As described in the literature review, although there is a growing body of research that examines student dropout behavior, research into the reasons why students decide to re-engage is limited. I designed this study to help address that gap in the literature by developing and field testing an instrument that re-engagement programs could use to identify the factors that contribute to students’ decisions to return. Based on a push/pull factor theoretical framework described in the literature (Berliner et al., 2008; Boylan & Renzulli, 2014; Entwisle et al., 2004; Epstein, 1992), the instrument assumes that students’ dropout and re-engagement behavior is linked. Students who are pushed out leave school because the nature of the learning environment feels incompatible, and they decide to return when they believe that a new school will be substantially different than the one they left. Students who are pulled out often make rational economic decisions about the costs of remaining in school. They return when the opportunity cost of remaining out of school exceeds the cost of not working or childcare, or if they believe a new school will provide the flexibility they need to balance all of their responsibilities. 101 Findings from this exploratory study provide preliminary support for these assumptions and have implications for future research and practice. Future research. As discussed earlier, the construct validity of the re- engagement survey instrument is based on the relationships between responses to items from the dropout and re-engagement domains. Future research using this instrument, or something similar, could establish criterion-related validity (Creswell, 2014) by following the re-engagement trajectories of students who take the survey. Logistic regression, or similar techniques, could be used to evaluate the extent to which responses to the dropout and re-engagement items predict future events, such as (a) enrollment status; (b) persistence and future disengagement behavior; (c) graduation or completion; and (d) enrollment in a traditional, alternative, or GED program. For example, the push/pull theoretical framework predicts that students who identify push-related dropout and re-engagement factors on the survey would be less likely to return than pull students. If they do return, they are more likely to do so in non-traditional alternative schools and GED programs. Testing that prediction is beyond the scope of this study, but future research could help establish the criterion-related construct validity of instruments like the one in this study, increasing their utility as tools to guide support and intervention. It is clear to me that mental health, especially anxiety, is a dropout and re- engagement factor that is underrepresented in the re-engagement literature and push/pull theoretical framework. In this study, I categorized mental health as a pull factor, but the epidemiology of anxiety is complicated. As such, the extent to which institutional factors and school characteristics cause, exacerbate, or mitigate anxiety is unclear. With that in 102 mind, future research exploring how anxiety, and other forms of mental health, push and/or pull students out of school may be warranted. Implications for Practice Although limited in scope, results from this study suggest that students who indicate they were pushed out of school identify different reasons for returning compared to students who were pulled out of school. On the surface, this is not a particularly profound finding, especially for those who have worked with disengaged students, but it does have implications for how school systems can re-engage students who have dropped out. Importantly, it may inform efforts to prevent students from dropping out initially. With that in mind, in this final section, I discuss implications and recommendations for re-engagement programs like Reconnection Services, districts like Portland Public Schools, and practitioners like myself. Reconnection services. Portland Public Schools’ Reconnection Services, and dropout recovery programs like it, should consider adopting an instrument like the one described in this study, or developing their own intake document to systematically collect data on student dropout and re-engagement behavior. Case managers discuss students’ experience in schools and their educational plans during intake, and Reconnection Services collects and monitors data on a variety of demographic and outcome variables. However, prior to the field test at the heart of this study, the program had not attempted to collect data on the factors that led students to leave school, their reasons for returning, and the characteristics they desire in new programs. As the lower than expected response rate to my survey illustrates, a survey should not be framed as optional, but become a required part of the intake process. Aggregate data collected from an intake instrument 103 could be used to (a) inform whole system reform efforts, (b) guide Reconnection Services student support, and (c) identify high leverage outreach and engagement strategies. Reconnection Services should also consider adopting a theoretical framework like the push/pull framework that guided this study. A theoretical framework could provide a protocol for making sense of student dropout and re-engagement behavior, and shape conversations about how to serve students and their families. Behavior that may be confusing in the absence of a framework may be easier to interpret when guided by a theory that views re-engagement as a reflection of dropout behavior. A framework can also help staff make more accurate educational referrals and placements. For example, understanding that push-out students will be less likely to return to traditional programs could prevent staff from placing students in settings that will yield the same results as previous environments. Districts. Findings from this study suggest that school systems should develop a continuum of re-engagement options that consider the dropout and re-engagement trajectories of push and pull students. Students who felt pushed out of school indicate that they will return when they believe a new school or program can provide a learning environment where they feel safe, seen, and successful. Districts should consider ways to increase students’ sense of belonging and improve the relevance and cultural congruence of the curriculum when structuring schools and programs to attract and retain these students. Traditional approaches will not work for them. They have already told us what they think of our educational system when they walked out the door. As one member of the student advisory group so eloquently noted, “Why would I want to return to the type of school that I hated, and that hated me?” 104 Students who have been pulled out of school in response to an adverse life event, or the need to work or parent, tell us they will return when something related to those circumstances changes. They return when they lose a job, change their work schedule, secure childcare, or find a program that allows them to balance their responsibilities. Unlike students who have been pushed out, these students did not necessarily leave school because the nature of the institution was incompatible. This suggests that school systems can bring these students back by providing additional flexibility and support in alternative or traditional comprehensive schools and programs. Data from surveys or intake instruments like the one described in this study should be considered when developing a continuum of educational options for students. If students’ dropout and re-engagement behavior is linked, then the portfolio of schools and programs that districts use to entice students back into school should be designed with the needs of push and pull students in mind. For example, a system based on the factors that influenced the return of students in this study would include (a) diploma and GED options, (b) scheduling flexibility, (c) childcare and support for parenting students, (d) intensive and individualized academic and social-emotional support, and (e) engaging culturally relevant curriculum. To increase students’ sense of safety and belonging, schools serving former dropouts should also be small and nimble, which would allow for greater personalization and strong student and adult relationships. It is clear that schools need to address students’ social-emotional needs and mental health. Almost half of the students who responded to the survey indicated that mental health or anxiety was important or very important in their decision to leave. Although not explicitly addressed in most of the literature that I reviewed, the student and 105 content expert advisory groups emphasized the role that mental health, especially anxiety, played in students’ decisions to leave school. As discussed earlier, it is unclear whether students’ anxiety was caused by their experience in school, or if their struggle in school was a consequence of deteriorating mental health. The former has implications for the structure of both traditional and alternative schools, and the latter for the set of support services schools designed to re-engage students need to offer. Either way, as any principal will attest, our schools are currently not equipped to meet the social-emotional and mental health needs of students. Personal practice. Since starting this study, I have left my position as the administrator of the Reconnection Services dropout recovery program to take a role in PPS’ central office. As the Director of High School Success, I manage implementation of a strategic plan for the district’s high schools and a large grant from the Oregon Department of Education. The goals of the $10-11 million annual grant are (a) dropout prevention, (b) Career and Technical Education (CTE) enhancement and expansion, and (c) reductions in chronic absenteeism. With this new role in mind, I have an opportunity to direct investment of grant resources to address many of the dropout and re-engagement factors identified by the students who participated in my study. Specifically, since becoming Director, I have allocated grant resources to (a) provide release time for ninth grade teacher to identify and intervene with students who are off-track, (b) increase access to CTE in order to improve the relevance of students’ school experience, (c) provide instructional coaches to each high school, and (d) build teacher capacity to implement culturally responsive practices. Additionally, I have ensured that the district’s alternative schools and re- 106 engagement programs receive grant resources and are provided flexibility to implement strategic plan activities in ways that respect their unique contexts. Finally, I will help fund a segmentation analysis of disengaged students in the district that will inform a solicitation for contracted alternative education services next year. This segmentation analysis will improve the district’s understanding of the characteristics of students who are out of school, where they live, and the types of schools and programs that might attract them back to complete their secondary education. As such, this analysis potentially represents a scaling up of the study described in this manuscript, especially if it is based on the push/pull construct that guided this study. In conclusion, we need to continue to ask students why they left school, why they want to return, and what they need in order to be successful. We will probably be surprised by the diversity of answers, but as this study found, clear themes will likely emerge that can inform efforts to re-engage out-of-school students. Towards that end, instruments like the one described in this study should continue to be developed, tested, and refined. 107 APPENDIX A RESEARCH ARTICLES IN LITERATURE REVIEW 1. Barrat, V. X. (2016). Characteristics and education outcomes of Utah high school dropouts who re-enrolled (REL 2017–206). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. 2. Berliner, B., Barrat, V., Fong, A., & Shirk, P. (2008). Reenrollment of high school dropouts in a large, urban school district (Issues & Answers Report, REL 2008– No. 056). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. 3. Bickerstaff, S. (2010). "I felt untraditional": High school leavers negotiating dominant discourses on "dropout". Journal of Education, 190, 37-45. 4. Boylan, R., & Renzulli, L. (2014). Routes and reasons out, paths back: The influence of push and pull reasons for leaving school on students' school re-engagement. Youth & Society, 1, 1-26. 5. Chuang, H. (1997). High school youth´s dropout and re-enrolment behavior. Economics of Education Review, 16, 171–186. 6. Entwisle D., Alexander K., & Olson L. (2004) Temporary as compared to permanent high school dropout. Social Forces, 82, 1181-1205. 7. Epstein, K. (1992). Case studies in dropping out and dropping back in. The Journal of Education, 174, 55-65. 108 8. Iachini, A., Buettner, C., Anderson-Butcher, D., & Reno, R. (2013). Exploring students' perceptions of academic disengagement and re-engagement in a dropout recovery charter school setting. Children & Schools, 35, 113-120. 9. Lagana-Riordan, C., Aguilar, J., Franklin, C., Streeter, C., Kim, J., Tripodi, S., & Hopson, L. (2011). At-risk students' perceptions of traditional schools and a solution-focused public alternative school. Preventing School Failure, 55, 105- 114. 10. McDermott, E., Donlan, A., Zaff, J., & Prescott, J. (2016). A psychometric analysis of hope, persistence, and engagement among re-engaged youth. Journal of Psychoeducational Assessment, 34, 136-152. 11. Wayman, J. C. (2001). Factors influencing GED and diploma attainment of high school dropouts. Education Policy Analysis Archives, 9, 1-19. 12. Wayman, J. C. (2002). The utility of educational resilience for studying degree attainment in school dropouts. Journal of Educational Research, 95, 167-178. 13. Zaff, J., Aasland, K., McDermott, E., Carvalho, A., Joseph, P., & Jones, E. (2016). Exploring positive youth development among people who leave school without graduating high school: A focus on social and emotional competencies. Qualitative Psychology, 3, 26-45. 14. Zaff, J., Kawashima, K., Boyd, M., & Kakli, Z. (2014). Reconnecting disconnected youth: Examining the development of productive engagement. Journal of Research on Adolescents, 24, 526-540. 109 APPENDIX B OUTREACH COORDINATOR SURVEY ADMINISTRATION LETTER March 1, 2018 Reconnection Center Staff 46 NE 12th Ave, Portland, OR 97232 RE: Reconnection Survey Dear Reconnection Center Staff, A survey on the factors that contribute to students’ decision to disengage, and then return to school, is included in the intake packet for students referred to Reconnection Services. The survey is part of a study being conducted by a graduate student at the University of Oregon. Data collected from the survey will be used for a dissertation and will be shared with PPS and the Multiple Pathways to Graduation Department in an effort to inform and improve services for disconnected youth. Participation in the survey is completely voluntary for students. In order to respond to the survey, a parent/guardian or a student who is over 18 years old must sign an active consent form that clearly articulates the purpose of the survey, that participation is voluntary, and that the decision to opt out will not negatively impact access to Reconnection Services. Your participation in the study is also completely voluntary. As such, even though the survey will be included in the Reconnection Services intake packet, you will not be asked to administer the survey or in any way try to persuade students to participate. If you have any questions, please contact Matt Eide at meide@uoregon.edu. Thank you! Matt Eide Graduate Student University of Oregon 110 APPENDIX C COVER LETTER TO PARENTS April 15, 2018 Dear Reconnection Services parent(s): My name is Matt Eide. I am a doctoral student at the University of Oregon. As part of both my professional and academic work, I am interested in learning more about why students decide to return to school after dropping out or disengaging. With that in mind, I am writing to invite your student to participate in a study on student re-engagement behavior. All students who are working with Reconnection Services are being offered the opportunity to take a short five-minute survey on why they have decided to return to school. Your Outreach Coordinator will provide your student with the survey at some point during the intake process. If you provide permission and your student agrees to take the survey, we will use the results to improve our understanding of the factors that contribute to students’ decisions to return to school. This will help us improve our services and improve graduation rates. Participation in this study is completely VOLUNTARY and CONFIDENTIAL. Your student’s name will NOT be on the questionnaire. Any information that is obtained in connection with this study that can be can be linked to your son or daughter will be kept confidential. Additionally, your ability to work with Reconnection Services will not be affected if you or your student decide not to participate. If you have any concerns or problems about your son or daughter’s participation in this study or his or her rights as a research subject, please contact the Institutional Review Board at the University of Oregon, 5237 University of Oregon, Eugene, OR 97403. If you have questions about the study, contact Matthew Eide at 971.593.1906. Sincerely, Matthew Eide Graduate Student University of Oregon 111 APPENDIX D PARENT INFORMED CONSENT FORM Your child is invited to participate in a research study conducted by Matthew Eide, a doctoral student at the University of Oregon. The research seeks to understand the factors that contribute to students’ decision to return to school after disengaging or dropping out and is part of a dissertation at the University of Oregon. If your child decides to participate, they will be asked to take a short five-minute survey during the Reconnection Services intake process. Your child’s participation in this study will be kept confidential. Your child’s name will NOT be on the survey. Any information that is obtained in connection with this study that can be linked to your son or daughter will be kept confidential. Participation is entirely voluntary. Your child’s decision to participate or not will not affect their ability to receive support from Reconnection Services. If your child decides to take part in the study, he or she may choose to withdraw at any time without penalty. Please keep a copy of this letter for your records. If you have any concerns or problems about your son or daughter’s participation in this study or his or her rights as a research subject, please contact the Institutional Review Board at the University of Oregon, 5237 University of Oregon, Eugene, OR 97403. If you have questions about the study, contact Matthew Eide at 971.593.1906. Your signature means that you have read and understand the above information and agree that your child has permission to take part in this study. Please understand that you may withdraw your consent at any time without penalty, and that, by signing, you are not waving any claims, rights or remedies. The researcher will provide you with a copy of this form for your own records. _ _______ Signature of parent Date Print the name of the child 112 Student Assent Form Student’s name: You are being offered the opportunity to participate in a study that will be used to improve our schools. If you choose to do it, you will be asked to take a five-minute survey about why you want to return to school. This survey is entirely voluntary and your decision to participate will have no impact on your ability to work with Reconnection Services. Additionally, you can change your mind at any time regarding your willingness to take the survey. Signed: Date: ____________ 113 APPENDIX E STUDENT ADVISORY PANEL NGT FOCUS GROUP PROTOCOL Introduction Script Thank you for your willingness to serve on this advisory panel. Our goal today to get your perspective on why some students decide to drop out of school, and they why they decide to come back. I believe that if we have a better understanding of why students leave and return to school, then we can design schools and programs that make it easier for students to complete their high school education. I will be using your insight to design a survey that we will give to all students who work with Reconnection Services. This will allow us to gather a lot of information that we can use to improve our school system. During our meeting today, we will be using something called Nominal Group Technique. This technique will give you some time to think about the questions individually. We will then share our ideas, discuss them as a group, and prioritize them. Hopefully, this will make sure that everybody has an opportunity to contribute. At this point, I want to reiterate that your participation on this advisory panel is completely voluntary and confidential. You can decide to opt out at any point, and everything that you say in the group will be kept confidential. Any question? Silent generation of ideas During this stage of the NGT process, advisory panel members will be asked to individually respond in writing to the following prompts. Members will be provided with a piece of paper that includes the prompts. Prompt 1: Why do you believe that some students struggle in school or decide to drop out? As you respond to this question, please think about factors in a student’s life or school experience that may contribute to struggling in school. Prompt 2: Why do you believe that students decide to return to school after having disengaged or dropped out? As you respond to this question, please think about factors that might lead a student to consider returning to school. Script I am going to give you about ten minutes to do some thinking about a couple of important questions. I will pass out a piece of paper that has the questions on them. Please write your thoughts on the sheet of paper, because after this stage, I am going to ask you to share your ideas. 114 Share out During this stage in the NGT process, I will ask participants to share the ideas that they generated during the silent reflection phase. To ensure that all members have an equal opportunity to contribute, I will ask members to share one idea at a time. I will take notes on a whiteboard or flip chart Script Now you get to share the ideas that you wrote down in the previous stage. We are going to go around in a circle, and give each person an opportunity to share one idea at a time. We will keep going around the circle until everyone has had a chance to share all their ideas. At the end of this stage, you will have a chance to talk about the ideas that other members of the group shared, so feel free to take some notes or write down questions that you have. Group discussion During this stage of NGT, I will facilitate a whole group discussion of the ideas that were recorded in the previous phase. Members will be asked to share observations about themes and trends in the responses, and be given the opportunity to ask clarifying questions. At the end of the whole group discussion round, we will have clarified and synthesized the groups input. Script Now we are going to be able to talk about all of your ideas. More specifically, this is your chance to point out some themes or patterns. We may decide to combine several responses into one big idea. Also, this is your chance to ask questions. Prioritization During the last stage of the NGT, members will be asked to prioritize the ideas/themes that emerged from the individual reflection and whole group discussion. During a brief break after the group discussion, I will re-organize the ideas and themes that emerged from the discussion. Members will then be given eight sticker “dots” and be asked to indicate which four dropout/disengagement factors they believe are most important and which four re-engagement factors are most important. At the end of the dot polling phase, we will identify which factors the group ranked as most important. Script We have come up with some really interesting ideas about why some students disengage from school, then decide to come back. Now we are going to do an activity where you get to say which of the factors you think are the most important. Each of you is going to get eight dots. I want you to put four of the dots next to the factors that you think are most important in students’ decision to leave school, and then put four dots next to factors that you believe are most important in their decision to return to school. 115 APPENDIX F RE-ENGAGEMENT SURVEY 116 117 118 119 120 REFERENCES CITED Babbie, E. (2013). The practice of social research, 13th edition. Belmont, CA: Wadsworth, Cengage Learning.
 Balfanz, R., Bridgeland, J., Fox, J., DePaoli, J., Ingram, E., & Maushard, M. (2014). Building a grad nation: Progress and challenge in ending the high school dropout epidemic. Washington, DC: Civic Enterprises. Barrat, V. X. (2016). Characteristics and education outcomes of Utah high school dropouts who re-enrolled (REL 2017–206). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. Belfield, C., Levin, H., & Rosen, R. (2012). The economic value of opportunity youth. Washington, DC: Corporation for National and Community Service. Berliner, B., Barrat, V., Fong, A., & Shirk, P. (2008). Reenrollment of high school dropouts in a large, urban school district (Issues & Answers Report, REL 2008–No. 056). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory West. Bickerstaff, S. (2010). “I felt untraditional”: High school leavers negotiating dominant discourses on “dropout”. Journal of Education, 190, 37-45. Boylan, R., & Renzulli, L. (2014). Routes and reasons out, paths back: The influence of push and pull reasons for leaving school on students' school re-engagement. Youth & Society, 1, 1-26. Catterall, J. (2011). The societal benefits and costs of school dropout recovery. Education Research International, 1, 1-8. Chuang, H. (1997). High school youth´s dropout and re-enrolment behavior. Economics of Education Review, 16, 171-186. Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches, 4th Edition. Los Angeles, CA: Sage. Delbecq, A., Van de Ven, A., & Gustafson, D. (1975). Group techniques for program planning: A guide to nominal group and delphi processes. Glenview, IL: Scott, Foresman, and Co. Dillman, D., Smyth., J, & Christian, L. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design method. Hoboken, NJ. Wiley. Entwisle, D., Alexander K., & Olson, L. (2004) Temporary as compared to permanent high school dropout. Social Forces, 82, 1181-1205. 121 Franklin, C., Streeter, C., Kim, J., & Tripodi, S. (2007). The effectiveness of a solution- focused, public alterative school for dropout prevention and retrieval. Children & Schools, 29, 133-144. Gehlbach, H., & Brinksworth, M. (2011). Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15, 380-387. Galobardes, B., Shaw, M., Lawlor, D. A., Lynch, J. W., & Smith, G. D. (2006). Indicators of socioeconomic position (part 1). Journal of Epidemiology and Community Health, 60(1), 7-12. Hansell, W. (2016). Re-engaging Oregon’s high school dropouts. Salem, OR: Oregon Youth Development Council. Heckman, J. & Rubinstein, Y. (2001). The importance of noncognitive skills: Lessons from the GED testing program. The American Economic Review, 91, 145-149. Iachini, A., Buettner, C., Anderson-Butcher, D., & Reno, R. (2013). Exploring students' perceptions of academic disengagement and re-engagement in a dropout recovery charter school setting. Children & Schools, 35, 113-120. Lagana-Riordan, C., Aguilar, J., Franklin, C., Streeter, C., Kim, J., Tripodi, S., & Hopson, L. (2011). At-risk students' perceptions of traditional schools and a solution- focused public alternative school. Preventing School Failure, 55, 105-114. McDermott, E., Donlan, A., Zaff, J., & Prescott, J. (2016). A psychometric analysis of hope, persistence, and engagement among re-engaged youth. Journal of Psychoeducational Assessment, 34, 136-152. Parker, R. (1990). Power, control, and validity in research. Journal of Learning Disabilities, 23, 613-617. PPS. (2019a). Alternative accountability report: PPS alternative high schools (Unpublished report). Portland, OR: Portland Public Schools. PPS. (2019b). Four and five-year cohort graduation and completion rates: All students by subgroup. Portland, OR: Portland Public Schools. PPS. (2019c). Reconnection Services Referrals, 2018-19 [Data file]. Unpublished raw data. Portland, OR: Portland Public Schools. Rumberger, R. (2011). Dropping out: Why students drop out of high school and what can be done about it. Cambridge, MA: Harvard University Press. Rumberger, R. (2004). Why students drop out of school. In G. Orfield (Ed.), Drop outs in America: Confronting the graduation rate crisis (pp. 131-155). Cambridge, MA: Harvard Education Press. 122 Shavers, V. (2007). Measurement of socioeconomic status in health disparities research. Journal of the National Medical Association, 99, 1013-1023. Suh, S., Suh, J., & Houston, I. (2007). Predictors of categorical at-risk high school dropouts. Journal of Counseling & Development, 85, 196-203.
 Thorndike, R, & Thorndike-Christ, T. (2010). Measurement and evaluation in psychology and education. Boston, MA: Pearson. Tyler J., & Lofstrom, M. (2009). Finishing high school: Alternative pathways and dropout recovery. The Future of Children, 19, 77-103. Wayman, J. C. (2001). Factors influencing GED and diploma attainment of high school dropouts. Education Policy Analysis Archives, 9, 1-19. Wayman, J. C. (2002). The utility of educational resilience for studying degree attainment in school dropouts. Journal of Educational Research, 95, 167-178. Zaff, J., Aasland, K., McDermott, E., Carvalho, A., Joseph, P., & Jones, E. (2016). Exploring positive youth development among people who leave school without graduating high school: A focus on social and emotional competencies. Qualitative Psychology, 3, 26-45. Zaff, J., Kawashima, K., Boyd, M., & Kakli, Z. (2014). Reconnecting disconnected youth: Examining the development of productive engagement. Journal of Research on Adolescents, 24, 526-540. 123