DO STUDENT PERCEPTIONS OF CURRICULUM MATTER? EVIDENCE FROM SECONDARY SOCIAL STUDIES STUDENTS by JENOGE SORA KHATTER A DISSERTATION Presented to the Department of Educational Methodology, Policy, and Leadership and the Division of Graduate Studies of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Education June 2021 DISSERTATION APPROVAL PAGE Student: Jenoge Sora Khatter Title: Do Student Perceptions of Curriculum Matter? Evidence from Secondary Social Studies Students This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Education degree in the Department of Educational Methodology, Policy, and Leadership by: David Liebowitz, Ed. D. Chairperson & Advisor Charles Martinez, Ph.D. Core Member Jo Smith, Ph.D. Core Member Surendra Subramani, Ph.D. Core Member Gerald Rosiek Institutional Representative and Andy Karduna Interim Vice Provost for Graduate Studies Original approval signatures are on file with the University of Oregon Division of Graduate Studies. Degree awarded June 2021 ii © 2021 Jenoge Sora Khatter This work is licensed under a Creative Commons Attribution-NoDerivs (United States) License. iii DISSERTATION ABSTRACT Jenoge Sora Khatter Doctor of Education Department of Educational Methodology, Policy, and Leadership May 2021 Title: Do Student Perceptions of Curriculum Matter? Evidence From Secondary Social Studies Students A large body of literature documents the negative effects of student disengagement from K-12 schooling and chronic absenteeism. Practitioners and policy makers often theorize that shifts in the curriculum and instruction might improve student engagement. In this study, I examine whether students’ perception of the curriculum they experience in social studies classes is related to their achievement and engagement. To test this hypothesis, I develop a novel instrument, the Critical Cultural Perception Measure (CCPM), to measure the extent to which a curriculum aligns with the goals of critical bicultural pedagogy (Darder, 1991, 2012). I find that students’ perception of curricular alignment with these goals differ by students’ social studies achievement levels and curriculum engagement levels, and by their socioeconomic level. Comparisons among racial and gender pronoun groups result in statistical significance at the item level for several CCPM items of special interest. I further contextualize these pronounced demographic differences in students’ perception of the curriculum through textual analysis of their open-responses. Evidence generated within this study indicates that iv curriculum perception measures may be valuable instruments to assess and respond to student disengagement. v CURRICULUM VITAE NAME OF AUTHOR: Jenoge Sora Khatter GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene, Oregon Oregon State University, Corvallis, Oregon DEGREES AWARDED: Doctor of Education, 2021, University of Oregon Master of Education, 2008, University of Oregon Bachelor of Science, 2007, Oregon State University AREAS OF SPECIAL INTEREST: Curriculum Evaluation Ethnic Studies Professional Development Pre-Service Teacher Education Equity-based MTSS PROFESSIONAL EXPERIENCE: Teacher on Special Assignment, Eugene School District, 2018-Present Teacher (Social Studies, ELD, Math), Eugene School District, 2012-2018 Teacher (Math, Science, Social Studies), Alameda County Schools, 2010-2011 Special Education Social Studies Teacher, Liberty Point (Staunton, VA), 2009 Graduate Teaching Fellow, University of Oregon, 2008 Outdoor School Head Staff & Counselor, Corvallis School District, 2001-2003 GRANTS, AWARDS, AND HONORS: Eugene 4J School District Secondary Humanities Program Grant, Eugene Education Foundation, 2021 Civic Scholar, Oregon House District 13, 2020 Civic Scholar, Oregon House District 11, 2019 vi Civic Scholar, Oregon Senate District 7, 2018 Travel Grant, University of Oregon, 2018 Civic Scholar, Oregon Senate District 4, 2017 Math in Real Life 2.0 Grant, Oregon Department of Education, 2017 Civic Scholar, Oregon House District 8, 2016 Multicultural Literature for the Commons, Eugene Education Foundation, 2015 Graduate Teaching Fellowship, University of Oregon, 2008 PUBLICATIONS: 2018. Monograph. Teacher Facilitated Critical Civic Engagement: An Implementation Guide for Student Voice in Local Decision-Making (6-12 social studies), (3rd ed.). (n.p.): Author. vii ACKNOWLEDGEDMENTS My sincerest appreciation to Dr. Liebowitz for his mentorship in the preparation of this manuscript; to Dr. Martinez for the formulation of the measures and scope of study; to Dr. Smith for the approach to writing and inquiry; to Dr. Subramani for the international and colonial/neocolonial paradigms; and to Dr. Rosiek for over a decade of modeling critical approaches to education. I am also thankful to Sue Wilson and Oscar Loureiro for helping me target the utility of this study, and to my Oregon State University professors Dr. Sharyn Clough, Dr. Jon Katz, and Dr. Matt O’Laughlin, and my Corvallis High School A.P. U.S. History teacher, Alan Taylor, for helping to inspire my passion for inclusive, pluralistic social science curricula. viii Dedicated to my mother, Kathy Ann, for her love for learning; my eldest sister, Shantee Sunshine, for prioritizing connection with others; my children, Rowan Avery, Cresa Muin, and Elowyn Vine, for their inspiration to transform what is possible; to my other half, Kelsey Sora, for her steadfast partnership and support, especially in the midst of familial crises; to my sister, Haseena Harmony, for her boundless commitment to her family; to my brother, Mukesh Michael, for his modeling of humility and malleability; and to my father, Kashmiri Lal, for demonstrating the capacity for one to write their own story. ix TABLE OF CONTENTS Chapter Page I. INTRODUCTION ......................................................................... 1 II. LITERATURE REVIEW ............................................................... 6 Search Procedures ................................................................ 6 Summary of the Literature Pool ........................................... 8 Types of Research ...................................................... 9 Participants and Settings ............................................ 10 Measures and Instruments .......................................... 11 Findings ................................................................................ 13 Factors Related to Enhanced Academic Achievement ............................................................... 14 Factors Related to Enhanced Academic Participation 17 Factors Related to Diminished Academic Participation ............................................................... 18 Factors Related to Enhanced Sense of Connection to School or Society........................................................ 19 Utilitarian Value ............................................... 19 Understanding the World Complexly and Accurately ........................................ 19 Relating to People of Diverse Backgrounds ........................................... 20 Sensing a Greater Number of Possibilities for the Future ...................... 20 Feature/Represent People of Similar Racial or Experiential Backgrounds as Students ............. 21 Link Historical Events to Current Events ........ 22 Engage Students with Topics Directly Tied to Their Interests and Values ................................ 22 Factors Related to Diminished Sense of Connection to School or Society ................................................... 22 Factors Affecting Students’ Perceptions of Curricula 23 Summary .................................................................... 24 Research Gaps ............................................................ 25 Theoretical Framework ........................................................ 25 Research Question ...................................................... 28 III. METHODS ..................................................................................... 29 Sample .................................................................................. 29 Setting ............................................................... 33 Procedure .............................................................................. 33 Measures ............................................................................... 35 x Chapter Page Curriculum Alignment with Community Service Self-Efficacy Measure ................................................ 37 Community Service Self-Efficacy Scale .......... 37 Validity and Reliability .......................... 38 Social Studies Curriculum Engagement Measure ...... 39 School Engagement Instrument ....................... 39 Validity and Reliability .......................... 41 My Class Activities .......................................... 42 Validity and Reliability .......................... 42 Open Response Items ................................................. 43 Achievement Levels ................................................... 44 Engagement Levels .................................................... 44 Sociodemographic Constructs .................................... 45 Racial Categories .............................................. 45 Gender Pronouns .............................................. 46 Socioeconomic Levels ...................................... 46 Critical Curriculum Perception Measure .................... 46 Validity ............................................................. 48 Analytic Plan ........................................................................ 49 Excluded Items and Imputation Methods ................... 52 IV. RESULTS ....................................................................................... 54 Differences by Levels of Achievement ................................ 54 Differences by Levels of Engagement ................................. 55 Differences by Sociodemographic Constructs ..................... 56 Race ............................................................................ 56 Gender Pronouns ........................................................ 57 Socioeconomic Level ................................................. 57 CCPM Survey Items of Special Interest .............................. 58 Cultural Relevance Items ........................................... 59 Item 2: The Content and Activities of My Social Studies Classes Have Often Mentioned Perspectives, Individuals, or Events I Can Relate To .......................................................... 59 Achievement Level ................................ 59 Engagement Level .................................. 59 Race ........................................................ 59 Gender Pronouns .................................... 60 Socioeconomic Level ............................. 60 Item 11: The Content and Activities of My Social Studies Classes Often Help Me Feel Like I Understand Myself Better ...................... 61 Achievement Level ................................ 61 xi Chapter Page Engagement Level .................................. 61 Race ........................................................ 61 Gender Pronouns .................................... 61 Socioeconomic Level ............................. 61 Item 15: The Content and Activities of My Social Studies Classes Often Help Me Feel That I Belong in This Country ......................... 61 Achievement Level ................................ 61 Engagement Level .................................. 62 Race ........................................................ 63 Gender Pronouns .................................... 63 Socioeconomic Level ............................. 63 Civic Engagement Items ............................................ 63 Item 3: The Content and Activities of My Social Studies Classes Have Often Helped Me Think About How I Can Try to Make the World a Better Place ........................................ 63 Achievement Level ................................ 63 Engagement Level .................................. 63 Race ........................................................ 64 Gender Pronouns .................................... 65 Socioeconomic Level ............................. 65 Item 6: The Content and Activities of My Social Studies Classes Often Make Me More Hopeful About the Future ................................ 65 Achievement Level ................................ 65 Engagement Level .................................. 65 Race ........................................................ 65 Gender Pronouns .................................... 65 Socioeconomic Level ............................. 66 Item 9: The Content and Activities of My Social Studies Classes Often Help Me See How, By Working Together With Others, I Can Make the World a Better Place ................. 66 Achievement Level ................................ 67 Engagement Level .................................. 67 Race ........................................................ 67 Gender Pronouns .................................... 67 Socioeconomic Level ............................. 67 Differences by Coding of Student-Reported Purpose of School ................................................................................... 68 To Gain Knowledge ................................................... 68 Achievement Level .......................................... 68 xii Chapter Page Race .................................................................. 70 Gender Pronouns .............................................. 70 Socioeconomic Level ....................................... 70 To Gain Skills for Financial Independence and/or General Life Success .................................................. 70 Achievement Level .......................................... 70 Race .................................................................. 70 Gender Pronouns .............................................. 70 Socioeconomic Level ....................................... 70 To Grow in Awareness or Become a Better Person in General ....................................................................... 72 Achievement Level .......................................... 72 Race .................................................................. 72 Gender Pronouns .............................................. 72 Socioeconomic Level ....................................... 73 Qualitative Analysis ............................................................. 73 To Gain Knowledge ................................................... 74 School Purpose ................................................. 75 Social Studies Purpose ..................................... 76 Recommended Shifts ........................................ 76 To Gain Skills for Financial Independence and/or General Life Success .................................................. 77 School Purpose ................................................. 78 Social Studies Purpose ..................................... 78 Recommended Shifts ........................................ 80 To Grow in Awareness or Become a Better Person in General ....................................................................... 81 School Purpose ................................................. 82 Social Studies Purpose ..................................... 83 Recommended Shifts ........................................ 83 V. DISCUSSION ................................................................................ 85 Threats to Validity ................................................................ 85 Threats to Internal Validity ........................................ 85 Instrumentation …………................................. 86 Content Validity of the Measures ........... 86 Bias in the Instruments ........................... 86 Bias in Interpreting Directionality .......... 87 Selection ........................................................... 87 Unobserved Confounding Variables ................ 87 Type I Errors .................................................... 88 Type II Errors ................................................... 89 Voluptuous Legitimation .................................. 89 Confirmation Bias ............................................ 89 xiii Threats to External Validity ....................................... 90 A Lack of Representativeness of Available and Target Populations ..................................... 90 Hawthorn Effect ............................................... 91 Sensitization/Reactivity to Research Conditions ........................................................ 91 Invalidity or Unreliability of Instruments ........ 91 Ecological Validity ........................................... 92 Evaluative Validity ........................................... 92 Order Bias ........................................................ 92 Summary of Findings ........................................................... 92 Implications for Curriculum Inquiry .................................... 93 Implications for Policy and Practice .................................... 94 Cultural Relevance Items ........................................... 95 Item 2 ................................................................ 95 Item 11 .............................................................. 96 Item 15 .............................................................. 96 Civic Engagement Items ............................................ 97 Items 3 .............................................................. 97 Items 6 and 9 .................................................... 98 Areas for Future Research .................................................... 98 Future Research for Student Curriculum Perception 99 Future Research for Student Evaluation of Instructional Materials ................................................ 99 Future Research for Inclusivity and Responsiveness of Curriculum Implementation Methods .................... 100 APPENDICES ................................................................................ 101 A. CCPM EXPERT REVIEW BY ANTONIA DARDER .. 101 B. ASSESSMENT BATTERY OF SURVEY ITEMS ADMINISTERED TO STUDENTS ................................ 102 REFERENCES CITED .................................................................. 107 xiv LIST OF TABLES Table Page 1. Student curriculum perception data in the literature pool. ............................ 10 2. Participants and settings in the literature pool. .............................................. 12 3. Measures and methodologies utilized in the literature pool. ......................... 15 4. Topics of curricula perception research findings in the literature pool. ........ 16 5. Self-reported demographics of research participants. ................................... 32 6. Demographics of the research settings. ......................................................... 34 7. Summary of items, responses, and methods used for socioeconomic level disaggregation. .............................................................................................. 47 8.1. CCPM means using t-test comparisons by achievement levels. ................... 55 8.2. CCPM means using t-test comparisons by Higher/Lower achievement levels. ............................................................................................................. 55 9. CCPM means using t-test comparisons by engagement levels. .................... 56 10. CCPM means using t-test comparisons by race. ........................................... 57 11. CCPM means using t-test comparisons by gender pronouns. ....................... 57 12. CCPM means using t-test comparisons by socioeconomic level. ................. 58 13. CCPM Item 2 (relevance of content) means by achievement, engagement, and sociodemographic constructs. ................................................................. 60 14. CCPM Item 11 (understand self better) means by achievement, engagement, and sociodemographic constructs. ........................................... 62 15. CCPM Item 15 (belong in United States) means by achievement, engagement, and sociodemographic constructs. ........................................... 64 16. CCPM Item 3 (make the world better) means by achievement, engagement, and sociodemographic constructs. ........................................... 66 17. CCPM Item 6 (hopeful about the future) means by achievement, engagement, and sociodemographic constructs. ........................................... 67 xv Table Page 18. CCPM Item 9 (collectively make the world better) means by achievement, engagement, and sociodemographic constructs. ........................................... 69 19. “To Gain Knowledge” CCPM scores by achievement and sociodemographics. ....................................................................................... 71 20. “To Gain Skills for Financial Independence and/or General Life Success” CCPM scores by achievement and sociodemographics. ............................... 72 21. “To Grow in Awareness or Become a Better Person in General” CCPM scores by achievement and sociodemographics. ........................................... 73 xvi CHAPTER I INTRODUCTION Previous research has documented irrelevance of curriculum and feelings of boredom as top-cited reasons for school disengagement.1 Levels of disengagement during the middle school years are highly predictive of levels of disengagement in high school and the likelihood of leaving school before graduation.2 For many students, school disengagement is a causal factor for their departure from school before graduation (Hynes, 2014). Student retention and engagement continue to be broadly studied in education research, but rarely so through curriculum inquiry and evaluation (Aoki, 1990a, 1990b). To the extent it has been studied, the quality and content of curriculum—the materials and activities constituting a unit or course of study—have been closely linked with levels of engagement and the likelihood of on-time graduation. A growing body of evidence suggests delivering curricula perceived by students as interesting, challenging, and personally relevant can ameliorate many of the factors associated with school disengagement and non-attendance.3 1 See, for example, Bellamy (2005), Behrend (2012), Hynes (2014), Manning (2005), Preckel et al. (2010), and Saldaña (2009). 2E.g., Alexander et al. (1997), Balfanz et al. (2007, 2009), Orthner et al. (2010), and Woolley and Bowen (2007). 3Consider: Abubakar et al., (2017), Aidinopoulou and Sampson (2017), Alazzi and Chiodo (2004), Allensworth and Easton (2007), Bando et al. (2019), Caraballo (2017), Cahill et al. (2018), Cammarota (2007), Carter (2008), Carter and Welner (2013), Castellano et al. (2003), Celio et al. (2011), Chen and Yang (2019), Chiodo and Byford (2004), Cornelius-White and Harbaugh (2010), Creghan and Adair- Creghan (2015), Dee and Penner (2017), Dunbar (2020), Eccles et al. (1993), Eck et al. (2017), El-Amin et al. (2017), Ford and Harris (1996), Fredricks et al. (2004), Governale (1997), Halagao (2010), Hansberry (2000), Horak and Galluzzo (2017), Huang & Cornell (2017), Kelly and Abruzzo (2021), Kenny et al. (2006), Kpolovie et al. (2014), Lawrence et al. (2017), Lei et al. (2018), Martin and Bolliger (2018), Martell (2013), Mata et al. (2012), Mayhew (2005), Mikami et al. (2017), Milo (2017), Naegele (2017), Orthner (2007), Orthner et al. (2013), Plank (2001), Perry (2008); Perry et al. (2010); Priniski et al. (2018), Ringwalt et al. (2009), Rogers and Terriquez (2013), Schug et al. (1988), St. Mary et al. (2018), Sue et al. (2009), van Rooij et al. (2017), Villa et al., (2005), Voight and Velez (2018), Wang and Eccles (2011), and Yildirim (2017). Furthermore, research on education reform has stressed that instructional materials are significantly understudied (Chingos & Whitehurst, 2012). This is in spite of evidence that curriculum qualities are highly predictive of student performance, on scale with other major factors, such as teacher effectiveness (Agodini et al., 2010; Chingos & Whitehurst, 2012; Dee & Penner, 2016). A promising step forward in curriculum evaluation appears to be addressing the qualities and content that lead curriculum to be perceived by students as boring, irrelevant, and overly-facile. Concurrently, curriculum that students connect with and find interesting can be studied to inform future design. Approaching curriculum evaluation from the perspective of student perception could possibly facilitate enhanced engagement, achievement, and on-time graduation. While the literature indicates several qualities and content foci for high engagement curricula, no validated or common measures for evaluating curricula have been in use. As a result, there are significant gaps in education research about linkages between curriculum and engagement. The present study addresses some of these gaps by providing a novel theory-based measure of alignment with factors identified for their capacity to promote student connectedness to the curriculum. Several prominent, often overlapping and complementing, theories inform current research of student connection to curriculum, including critical bicultural pedagogy (Darder, 1991, 2012), culturally sustaining pedagogy (Paris, 2012), and culturally responsive pedagogy (Ladson-Billings, 1995, 2014; Gay, 2010). Each of these approaches emphasizes qualities of curriculum intended to be as equitable for diverse student populations as possible. Each approach also stresses student perceptions of curriculum as key for bridging cultural differences, access to academic skills, and 2 understanding about intersectional identities and perspectives. Informed by these theories, and framed by critical bicultural pedagogy in particular, the novel Critical Curriculum Perception Measure is administered in this exploratory study to build evidence for: (a) how students experience curriculum differently across engagement levels and sociodemographics, and (b) evaluating the hypothesis that increased levels of curricular alignment with critical bicultural pedagogy (CBP) are associated with increased achievement and engagement across diverse groups. CBP identifies several student perceptions of interest, such as: • Does the curriculum help students connect problems faced by people in the past to problems people face today? • Does the curriculum mention perspectives, individuals, or events students can relate to? • Does the curriculum prompt students to consider how working together with others can allow them to positively transform their community? In her work, Darder (1991, 2012) defines bicultural as having experienced enculturation within the United States that is distinct from that of affluent monocultural Whites. However, this study uses the term bicultural more broadly in order to encapsulate those with experiences of enculturation that are also distinct from cisgender males and heterosexuals in order to more fully address the palette of dominant social narratives that students navigate. Darder’s (1991, 2012) theory of CBP appears to be capable of being adapted to this broader definition of biculturalism that captures intersections of individuals’ ethnoracial, class, gender, and sexual-orientation identities. 3 In brief, several significant findings emerge from the present study. Student participants experience curriculum differently in relation to their engagement, achievement, and socioeconomic levels. Students with higher engagement levels perceive curriculum much more favorably. To a narrower extent, higher achieving students also have more positive curriculum perceptions. Compared to their lower and higher socioeconomic peers, middle level students have significantly more negative curriculum perceptions. At the item level, students also experience curriculum differently by race and pronouns. Validity for the measures appears tenable, in part as a result of the strong association between critical curriculum perceptions and curriculum engagement. Several implications for policy and practice are surfaced. In the four chapters that follow, I describe a systematic literature review on the topic of student perceptions of curriculum and associated outcomes, review the measures and methods applied to the current research, present quantitative and qualitative results, and discuss findings and implications. More specifically, Chapter II: Literature Review provides details about a pool of 15 studies related to student curriculum perception in social studies and language arts classes at the primary, secondary, and tertiary levels. The studies are organized by type of evidence generated in regards to perceptions related to enhanced or diminished class participation, perceptions related to enhanced or diminished connectedness to school and society, and variables shaping curriculum perceptions. Chapter III: Methods describes the research design, sample, setting, procedures, measures, and analytic plan for this study. A mixed methods survey design is used to collect responses from 192 students in grades six through ten from four schools in one of Oregon’s largest districts. The survey consists of three measures, three open-response 4 items, and nine sociodemographic prompts. The measures gather information about student curriculum perceptions in regards to critical bicultural pedagogy, civic engagement efficacy, and engagement. In Chapter IV: Results, quantitative and qualitative findings are presented. Evidence for significant differences between student engagement levels, achievement levels, socioeconomic levels, race, and pronouns is generated using t-tests. Students’ open-responses about the purpose of school are coded and used to guide a heuristic process of identifying students with pronounced differences in their curriculum perceptions. For each of the three most frequent purposes of school, open-responses from a pair of students are compared and juxtaposed with quantitative data. Finally, Chapter V: Discussion addresses threats to validity, summarizes findings of the study, and proposes implications for policy and practice, as well as for future research. Internal and external validity threats are considered for both the quantitative and qualitative methods employed. Strategies for improving students’ experience of curriculum are identified, as are opportunities for future research. Curriculum aimed at promoting student civic engagement appears to hold promise as both an improvement strategy and opportunity for further study. 5 CHAPTER II LITERATURE REVIEW I conducted a systematic literature review of humanities (social studies and language arts) curricula research to find evidence of student outcomes associated with curricular perceptions. The systematic approach allows for replication. In addition to surfacing research gaps, the review facilitates a basal synthesis of documented student humanities curriculum perception associations to frame the outcomes of the study. After documenting search procedures, the research literature is summarized in terms of its types, participants and settings, measures and instruments, findings, and gaps. A theoretical framework of critical bicultural pedagogy addresses some research gaps and provides a basis for the present study’s research question. Succinctly put, the literature pool provides evidence that humanities curriculum perceptions associate with factors enhancing achievement, factors enhancing and diminishing class participation, factors enhancing and diminishing connectedness to school and society, and other factors that may shape students’ perceptual lenses (e.g., race, gender, familial historical narrative). No rigorously validated measures are apparent in the literature pool, and curriculum perception data is not directly tested for its associations with engagement and achievement. The present study builds evidence for addressing these gaps. Search Procedures I used 12 keyword combinations while article database searching on ERIC and Google Scholar, which included (a) secondary social studies curriculum students; (b) secondary social studies curriculum self concept; (c) curriculum self concept secondary; (d) student self concept secondary social studies; (e) curriculum identity secondary 6 students; (f) secondary student perceptions curriculum; (g) curriculum achievement secondary students; (h) curriculum engagement secondary students; (i) social studies perceptions secondary; (j) secondary students curriculum impacts; (k) secondary students curriculum effects; and (l) student curriculum validity reliability. Articles that did not focus on humanities curricula were excluded. To capture one seminal article (Grice & Vaughn, 1992) emphasized in relevant theoretical literature (Gay, 2010), research published up to 30 years ago was included. The average year of publication was 2008. My searches initially yielded over 18 million articles. I engaged in four steps of literature exclusion based on an analysis of abstracts and construct definitions. First, in order to manage the high volume of yields from database searches, I only analyzed the first five webpages of search results, which were algorithmically designed to contain highly relevant results (Beel & Gipp, 2009; ERIC, 2020; Goossen et al., 2020; Harari et al., 2020). This limited the maximum number of results encountered for each search query to no more than 200. After eliminating articles that did not relate to humanities curricula, 174 studies remained. 42 more studies were eliminated after I omitted studies about special education and tertiary education. One exception was made (Sue et al., 2009) in order to keep a tertiary study about student perceptions of microaggressions in the curriculum that I thought could directly relate to secondary settings. Next, I omitted articles that were not specifically related to student perceptions of curricula, as opposed to featuring other student perceptions. 116 additional studies were eliminated, draining the pool to 12. I then title-scanned reference sections of books and articles (Darder, 2012; Gay, 2010; Paris, 2017; Tintiangco-Cubales et al., 2015) related to culturally relevant 7 curriculum and searched for articles with titles relating to (a) curriculum, (b) student perceptions, (c) text, (d) literature, (e) books, or (f) learning materials. Darder (2012), Gay (2010), Paris (2017), and Tintiangco-Cubales et al. (2015) were chosen because of their affiliation with the largely overlapping fields of critical bicultural pedagogy (Darder 1991, 2012), culturally responsive pedagogy (Ladson-Billings, 1995; Gay, 2010), culturally sustaining pedagogy (Paris, 2012), and ethnic studies pedagogy (Tintiangco- Cubales et al., 2015). All four fields highlight the importance of curricular connections to student lifestyles, interests, and backgrounds. Using these criteria, three studies with student perception data found from title-scanning were included in the literature pool (Epstein, 1998; Epstein, 2000; Grice & Vaughn, 1992), raising the total number to 15. Grice and Vaughn’s (1992) “Third Graders Respond to Literature for and About Afro- Americans” is an example of a study’s title matching the search terms. The search term “student perceptions” matched with the word “respond,” and “literature” identically matched “literature.” Summary of the Literature Pool In this section, I summarize key features of the literature pool. The types of research reviewed include studies employing quantitative, qualitative, and mixed methods approaches. The participants and settings involved in 14 of the 15 studies are public school students in grades 3-11 in the United States. Australian students in this grade span participated in the regional exception. Tertiary US students participated in one of the U.S. studies. The measures and techniques used by these studies include descriptive and statistical quantitative analyses, and qualitative analysis of interviews, conversations, discussions, and written reflections. The results section presents constructs 8 and variables that were found to shape student perceptions of curricula, student perceptions related to enhanced or diminished participation, and perceptions related to enhanced or diminished connectedness to school and society. Types of Research Table 1 summarizes the types of research in the student perceptions of humanities curricula research pool. The 15 studies were categorized by research method and whether student perceptions of humanities were directly or indirectly studied. For example, Billig et al.’s (2005) research contained direct student perception data about how much high school students reported liking school in relation to whether they had participated in a service-learning curriculum. In contrast, Creghan and Adair-Creghan (2015) studied how the administration of a project-based curriculum impacted attendance for economically disadvantaged high school students. Creghan and Adair-Creghan (2015) did not collect any direct data about student perception, but the difference in attendance rates among students administered the curriculum with those not may reflect underlying dimensions of one or more curriculum perception constructs. I placed the research into one of four categories: (a) direct quantitative data about student perceptions of curricula (3 studies); (b) direct qualitative data about student perceptions of curricula (8 studies); (c) indirect quantitative data about student perceptions of curricula (5 studies); and (d) indirect qualitative data about student perceptions of curricula (4 studies). Six of the studies yielded data placed in more than one category, as shown in Table 1. 9 Table 1. Student curriculum perception data in the literature pool. Direct quantitative Direct qualitative Indirect quantitative Indirect qualitative data (3) data (8) data (5) data (4) Billig et al. (2005) Chiodo and Byford Billig et al. (2005) Berman and White (2004) (2013) Wanzek et al. Creghan and Adair- (2015) Ebe (2010) Creghan (2015) Epstein (1998) Ebe (2010) Epstein (1998) Dee and Penner Epstein (2000) (2017) Epstein (2000) Webster (2001) Orthner et al., Grice and Vaughn (2013) (1992) Wanzek et al. Johnson (2016) (2015) Martell (2013) Webster (2001) Sue et al. (2009) Participants and Settings Table 2 summarizes the participants and settings in the curriculum perceptions literature pool. 14 of the 15 studies took place in the United States, and one in urban Melbourne, Australia (Berman & White, 2013). Indigenous and international research was invited in the literature search process, but no empirical works were found except for the Australian study. Elementary school students participated in two of the studies, lower secondary (grades 6-8) students in six, upper secondary (grades 9-12) students in 11, and tertiary students in one. Grice and Vaughn (1992) provide an elementary example from the research pool, featuring interviews with third-grade students in the urban South about their perceptions of age-level texts with African American protagonists. At the lower secondary level, Orthner et al. (2013) studied how attitudes about school and school engagement behaviors changed among lower secondary students in the urban South as a 10 result of a shift toward more career-preparatory curricula. Martell (2013) studied how upper secondary students in the Urban Northeast varied by race in their perceptions about U.S. history curriculum. Sample size, gender, and race differed in the sampling characteristics of the literature pool. Johnson (2016) had the smallest sample size (N = 8), while Orthner et al. (2013) had the largest (N = 3,493), the average being 478 participants. Six studies did not mention gender demographics. Females were 60% or more of the participant population in two studies (Billig et. al, 2005; Sue et. al 2009), and in a distinct minority in two studies (Dee & Penner, 2017; Grice & Vaughn, 1992). In the remaining studies, there was relative gender parity in the sampling. None of the studies addressed non-binary genders. In two studies, racial demographics were not mentioned. For the other 13 studies, the proportion of participants of color ranged from 19% (Chiodo & Byford, 2004) to 100% (Martell, 2013; Sue et. al, 2009) (M = 63%). Measures and Instruments Table 3 summarizes the methods of each study, noting when assessment methods were rooted in particular traditions. Of the 15 studies, 10 used interviews based on researcher-developed questionnaires to collect qualitative data. For instance, Epstein (2000) used interviews to reveal student perceptions about their U.S. history curriculum regarding how it addressed racial diversity. Two of these 10 also used researcher field notes about student behavior and self-disclosed perceptions (Johnson, 2016; Martell, 2013). Each of the 10 collected data about student perceptions of curriculum. In five studies, researcher-developed surveys were used to collect quantitative data. Of these five, one (Wanzek et al., 2015) statistically evaluated internal consistency 11 Table 2. Participants and settings in the literature pool. Study Authors Setting(s) Sample size Grade level(s) Gender Race Berman and White (2013) Urban Australia 47 8 >50% female Not stated Billig et al. (2005) Midwest, Northwest, 1,052 9-12 60% female 69% of color South, Southeast Chiodo and Byford (2004) Suburban Southwest 48 8, 11 50% female 19% of color Creghan and Adair-Creghan (2015) Suburban South 390 9-12 Not stated Not stated Dee and Penner (2017) Urban California 1,405 9 42% female >90% of color Ebe (2010) Urban Northeast 9 3 Not stated >50% of color Epstein (1998) Urban Midwest 49 11 Not stated 45% of color Epstein (2000) Urban Midwest 10 11 Not stated 50% of color Grice and Vaughn (1992) Urban South 13 3 38% female 69% of color Johnson (2016) Urban California 8 10-12 ~50% female >50% of color Martell (2013) Urban Northeast 49 10-11 Not stated 100% of color Orthner et al. (2013) Urban South 3,493 8 48% female >53% of color Sue et al. (2009) Urban Northeast 14 >12 79% female 100% of color Wanzek et al. (2015) Varied Southwest, 512 8, 11 53% female 53% of color Southeast Webster (2001) Suburban South 76 9 Not stated 74% of color 12 reliability via Cronbach’s (1951) alpha coefficients. Reliability was addressed in one other quantitative study (Webster, 2001) to evaluate the ability for multiple raters to reliably categorize using Banks’s (1981, 1997) Typology of Ethnic Identity. Statistical analysis of survey validity was not present in any of the studies. Of the five studies that used surveys, two collected longitudinal data (Billig et al., 2005; Orthner et al., 2013), and three collected data at a single interval (Ebe, 2010; Martell, 2013; Wanzek et al., 2015). Among the latter was Ebe’s (2010) study, which used a survey to collect students’ responses about the extent of cultural relevance extant in two different texts. Ebe’s (2010) study also collected quantitative data with the Miscue Analysis Procedure (Goodman et al., 2005) tool in order to gauge how comprehensible students found the two different texts. Two studies made use of statistical analysis to indirectly address student curriculum perceptions (Creghan & Adair-Creghan, 2015; Dee & Penner 2017). To evaluate the impact of a project-based curriculum on attendance, Creghan and Adair- Creghan (2015) used independent t-testing. Dee and Penner (2017) used regression discontinuity to measure the impact of an ethnic studies curriculum on attendance rate, GPA, and credits earned. Besides the miscue tool in Ebe’s (2010) study and the typology in Webster’s (2001) study, the surveys mentioned in this section and the analyses addressed in this paragraph constitute all of the quantitative data reported in the literature pool. Findings My literature search process revealed that secondary student perception of humanities curriculum is not a commonly studied topic in education or related fields. 13 While student perceptions and attitudes were commonly studied, seldom so in relation to curriculum. In the course of further interrogating the studies in the literature pool by exploring publications that cited them, I found no evidence suggesting that an extant body of previously explored or synthesized research directly tied to how students perceive and respond to humanities curricula. The limited studies comprising my literature pool initially appeared to me to be highly discordant from one another. However, despite the paucity of research, it became evident that data in the literature pool could be categorized according to certain factors related to curricular perceptions: (a) enhanced academic achievement, (b) enhanced academic participation, (c) diminished academic participation, (d) enhanced connectedness to school and society; (e) diminished connectedness to school and society; and (f), factors affecting students’ perceptions of curricula. These findings are summarized in Table 4. Factors Related to Enhanced Academic Achievement Perceptions about teacher expectations, the utility of curricular content, and the cultural relevance of curricula were associated with enhanced academic achievement (Dee & Penner, 2017; Ebe, 2010; Grice & Vaughn, 1992; Wanzek et al., 2015). In Wanzek et al.’s (2015) study of secondary students’ perceptions of social studies instruction, students’ perceptions of teacher expectations played a statistically significant role in reading achievement scores. Students who perceived high expectations from teachers to complete their work and students who perceived that the teacher expected them to frequently share their ideas or opinions during class had higher reading achievement scores (p < 0.05) (Wanzek et al., 2015). Conversely, students who perceived 14 Table 3. Measures and methodologies utilized in the literature pool. Study Author(s) Data Type Variables of Interest Methodology Most Significant Change Berman and Student perceptions of Technique (Dart & Davies, White (2013) Qualitative curriculum’s personal 2003) interviews using impact researcher-designed questionnaire Student perceptions of Billig et al., Quantitative/ school, student Researcher-designed survey (2005) Descriptive attachment to school in without published relation to curriculum psychometrics (pre/post) Chiodo & Byford Qualitative Student perceptions Interviews using researcher- (2004) of social studies classes designed questionnaire Creghan & Adair- Quantitative/ Attendance rate in Creghan Descriptive relation to project-based Independent sample t-tests (2015) curriculum Dee & Quantitative/ Attendance rate, GPA, Penner Descriptive credits in relation to Regression discontinuity (2017) ethnic studies curriculum Student perceptions of Researcher-designed Quantitative/ curriculum’s cultural Cultural Relevance Rubric Ebe (2010) Descriptive relevance; interview questionnaire and & Qualitative comprehension of survey, Miscue Analysis curriculum Procedure (Goodman et al., 2005) Epstein Student perceptions of (1998) Qualitative history and history Interviews using researcher- curriculum designed questionnaire Epstein Student perceptions of Case study interviews using (2000) Qualitative history and history researcher-designed curriculum questionnaire Grice & Vaughn Qualitative Student perceptions of Interviews using researcher- (1992) curriculum designed questionnaire Participatory Action Johnson Student perceptions of Research (Lewin, 1946); (2016) Qualitative curriculum; student- researcher notes; interviews authored curriculum using researcher-designed questionnaire 15 Table 3. (continued). Interviews using researcher- Martell Quantitative / Descriptive Student perceptions of designed questionnaire; (2013) curriculum researcher notes; researcher-& Qualitative designed survey without published psychometrics Orthner et al., Quantitative / School valuing, school Researcher-designed survey (2013) Descriptive engagement without published psychometrics (longitudinal) Sue et al., Qualitative Student perceptions of Focus group interviews (2009) curriculum using researcher-designed questionnaire Wanzek et Quantitative / Student perceptions of Researcher-designed survey al., (2015) Descriptive curriculum and instruction with limited reliability data Banks’ (1981, 1997) Quantitative / Student perceptions of Typology of Ethnic Identity Webster Descriptive curriculum in relation to (interrater reliability); case (2001) & Qualitative student cultural schemata study interviews using researcher-designed questionnaire Table 4. Topics of curricula perception research findings in the literature pool. Factors Factors Factors Factors Factors Factors Enhancing Enhancing Diminishing Enhancing Diminishing Affecting Achievement Participation Participation Connectedness Connectedness Perception Dee and Chiodo and Epstein Berman and Johnson Berman and Penner (2017) Byford (1998) White (2013) (2016) White (2004) Ebe (2010) Johnson Billig et al. (2013) Creghan and (2016) (2005) Sue et al. (2009) Epstein Grice and Adair- Grice and (1998) Vaughn Creghan Sue et al. Vaughn (1992) Epstein (1992) (2015) (2009) Johnson (2000) Wanzek et al. Grice and (2016) Grice and (2015) Vaughn Martell (2013) Vaughn (1992) (1992) Orthner et al. (2013) Martell (2013) Webster (2001) Sue et al. (2009) 16 that course texts or concepts were too difficult to understand had lower reading achievement scores (p < 0.01) (Wanzek et al., 2015). Positive perceptions of the cultural relevance and utility of curricula also were positively associated with achievement (Dee & Penner, 2017; Ebe, 2010; Grice and Vaughn, 1992). For example, in their analysis of an ethnic studies curriculum’s implementation in San Francisco, Dee and Penner (2017) found that participation in settings with the ethnic studies curriculum had significant causal effects on the academic achievement of students with low GPAs. For the 9th grade participants receiving the ethnic studies curriculum, attendance improved by 21%, GPA by 1.4 grade points, and credits earned by a noteworthy (yet unspecified) extent. Factors Related to Enhanced Academic Participation Increased student attendance and engagement were related to positive curricular perceptions in three of the reviewed studies (Chiodo & Byford, 2004; Creghan and Adair- Creghan, 2015; Grice & Vaughn, 1992). Also, the increased achievement of students with curriculum perceived as more relevant in Dee and Penner (2017) and Ebe’s (2010) studies was linked to increased academic participation. Creghan and Adair-Creghan’s (2015) study of the effects of systematically implementing a project-based curriculum found that economically disadvantaged students attended 18 more school days than economically disadvantaged students at a comparison school without a project-based curriculum. Student engagement was highly related to students’ positive perceptions about the utilitarian value of curriculum in Chiodo and Byford’s (2004) study of students’ attitudes towards the social studies content area. In that study, students’ perceptions of high teacher enthusiasm and joy toward the curricula predicted willingness to participate, 17 even when an activity or topic was initially perceived as boring. Social studies topics that high school students had especially favorable perceptions of included understanding the political process and civic duties and responsibilities. Interviews with the students revealed that classes with those types of content were perceived to have a direct relation to their lives in the short- and long-term future. Grice and Vaughn (1992) found that “happy events” such as parties, trips, and family bonding in the curriculum were perceived to be of utilitarian value to third grade students. The presence of such events in the curricula enhanced the students’ levels of participation and interest in the related texts and discussion. Factors Related to Diminished Academic Participation The presence of microaggressions and/or racial bias in the curriculum negatively impacted students’ perceptions about its legitimacy in ways that related to their diminished levels of engagement (Epstein, 1998; Johnson, 2016; Sue et al., 2009). Furthermore, Sue et al. (2009) determined that the inability of a teacher to identify and deconstruct microaggressions when they arose in the curriculum exacerbated the negative perceptions students of color had of it and further diminished their participation. In Epstein’s (1998) interviews of high school African American students about their perceptions of social studies curricula, students repeatedly affirmed that they perceived textbooks and “mainstream” sources of information to be of limited legitimacy because of the texts’ chronic exclusion or misrepresentation of African Americans. Instead, family members were perceived as better sources of accurate information. Many of these students also revealed that their engagement depended on their perception of a teacher’s 18 credibility, which was determined by the students’ ability to relate to the teacher’s life experiences or to the teacher’s perspectives on race and power in U.S. history. Factors Related to Enhanced Sense of Connection to School or Society Connectedness to school and society was enhanced when the curriculum was perceived (a) to be of utilitarian value (Berman & White, 2013; Billig et al., 2005; Chiodo & Byford, 2004; Johnson, 2016; Martell, 2013; Webster, 2001), (b) to feature/represent people of similar racial or experiential backgrounds as students (Epstein 1998, 2000; Martell, 2013), (c) to corroborate/affirm historical narratives received from family (Epstein 1998, 2000; Martell, 2013), (d) to link historical events to current events (Epstein 1998, 2000; Martell, 2013), and (e) to engage students with topics directly tied to their interests and values (Billig et al., 2005; Orthner et al., 2013). Utilitarian Value. Within the quality of utilitarian value, a three-classification typology emerged: understanding the world complexly and accurately (Berman & White, 2013; Johnson, 2016; Martell, 2013); relating to people of diverse backgrounds (Johnson, 2016); and sensing a greater number of possibilities for the future (Billig et al., 2005; Chiodo & Byford, 2004; Johnson, 2016; Martell, 2013; Webster, 2001). Understanding the World Complexly and Accurately. An emblematic example of a curriculum’s perceived utility to facilitate a fuller understanding of the world came from Martell (2013). In studying ethnoracial minority students’ perceptions of a curriculum, Martell found a common student perception: textbooks have an agenda, and the extent to which one adheres to that agenda will result in how narrow a sense a person will have of “why things [are] like this.” The complex power dynamics that have shaped the asymmetries between sociodemographic groups becomes clearer, students perceived, 19 when the textbook was considered to contain a narrative that ultimately best served individuals in high positions with backgrounds almost alien to anything students would themselves aspire toward (Martell, 2013). Relating to People of Diverse Backgrounds. Evidence of curriculum’s perceived utility to facilitate the ability to better relate to others was mentioned only by Johnson (2016). Several participants in Johnson’s (2016) study perceived that participating in a curriculum aimed at changing a school’s hidden curriculum allowed them to better relate to others in ways they found personally meaningful. Sensing a Greater Number of Possibilities for the Future. A heightened sense of possibilities for the future was identified as a utilitarian perception of curriculum in five studies (Billig et al., 2005; Chiodo & Byford, 2004; Johnson, 2016; Martell, 2013; Webster, 2001). For example, students in Billig et al.’s (2005) study who engaged in service-learning were much more likely to state they planned to vote than those who did not. Chiodo and Byford (2004) studied student perceptions of the social studies subject area and found that students favorably perceived curriculum that prepared them with “good citizenship skills” and allowed them to gain an understanding of “civic duties and responsibilities.” Webster (2001) studied how culture-specific schemata impacted students’ perceptions of curricula. The researcher found that student perception of curriculum requiring them to self-reflect on a) their personal level of multicultural awareness, b) the extent to which they saw multicultural representation in school texts, and, c) racism, was associated with the number of future opportunities students saw available to themselves and the capacity students perceived themselves to have, individually, to make good decisions (Webster, 2001). Martell (2013) repeatedly 20 observed that representation of relatable non-fictional and fictional people in the curriculum affected the possibilities students saw available to themselves. Engaging in a curriculum aimed at changing a school’s hidden curriculum required imagining new possibilities and yielded favorable student perceptions in another study (Johnson, 2016). In that study (Johnson, 2016), students addressed the hidden curriculum of the school’s culture by reimagining the school culture and the steps it would take to get there. This finding corroborates Martell’s (2013), who found that students had favorable perceptions of curriculum prompting them to reimagine previously received curricula. Feature/Represent People of Similar Racial or Experiential Backgrounds as Students. Three studies examined perceptions related to how well people of diverse backgrounds were represented in the curriculum (Epstein 1998, 2000; Martell, 2013). In Martell’s (2013) study of ethnoracial differences in response to a history curriculum, students who had immigrated from Brazil had more favorable perceptions of curricula that featured Brazilian culture and history. In Epstein’s studies (1998, 2000) of how students’ race associated with their views of U.S. history, Black students had more favorable perceptions of curricula that expanded on textbook versions of Black history and culture, often contending that textbooks were reductive and assimilative toward inaccurate renderings of history. The extent to which curriculum corroborated narratives of racial injustice and struggles for justice that students had learned about at home was isolated as a particularly important perception of the quality of curriculum by students of color in Epstein’s (1998, 2000) and Martell’s (2013) studies. 21 Link Historical Events to Current Events. Students also perceived curriculum as being of higher quality when it helped them draw connections between the past and current events (Epstein 1998, 2000; Martell, 2013). For example, study of the Zoot Suit Riots and the United Farm Workers under Cesar Chavez’s leadership added positively to Mexican American students’ perceptions of how the curriculum added to their understanding of recent events in personally meaningful ways (Martell, 2013). Black students in Epstein’s (1998) study assigned positive attributes to curricula that connected historic civil rights struggles to contemporary ones. Engage Students with Topics Directly Tied to Their Interests and Values. Finally, a curriculum’s alignment with students’ interests and values also led to it being perceived more favorably (Billig et al., 2005; Chiodo & Byford, 2004; Grice & Vaughn, 1992; Orthner et al., 2013). Billig et al. (2005) found that students who participated in service-learning maintained higher levels of enjoyment of school from pre-test/post-test comparison with a control group (N = 1,042, p < 0.05). Orthner et al. (2013) found that students valued school more when it provided them career-relevant curricula with direct connections to careers that interested them (N = 3,493, p < 0.01), especially for female (n = 926, p < 0.001) and racial minority students (n = 1,122, p < 0.001). Curriculum with material that students perceived as helpful for achieving future goals was also perceived more favorably (Chiodo & Byford, 2004; Grice & Vaughn, 1992). Factors Related to Diminished Sense of Connection to School or Society Three studies found evidence for curricular perceptions that diminished students’ sense of connection to school or society (Epstein, 1998; Johnson, 2016; Sue et al., 2009). Each study mentioned ways in which students perceived the curriculum to exclude one or 22 more sociodemographic groups with which they identified. Sue et al. (2009) studied the impacts of perceived racial microaggressions in the curriculum on tertiary students of color. Perceptions of microaggressions, especially when the teacher did not deconstruct incidents that students of color considered obvious, diminished how connected students of color felt to their White classmates, the teacher, the curriculum, the school, and society. Likewise, Johnson (2016) found that perceived class-based microaggressions in the hidden curriculum of an affluent school were associated with students from lower- income homes feeling disconnected from the school and their affluent peers. Black students in Epstein’s (1998) study perceived most history in their social studies classes as “white people’s history,” and this was associated with their sense of disconnectedness from the curriculum and the school. Factors Affecting Students’ Perceptions of Curricula Several studies examined factors affecting how students perceived curricula (Berman & White, 2013; Epstein 1998, 2000; Grice & Vaughn, 1992; Martell 2013; Sue et al., 2009). Race and gender were identified as aspects of identity that accounted for variation in perception. Five qualitative studies revealed race as the primary factor differentiating how students perceived history curricula (Epstein 1998, 2000; Grice & Vaughn, 1992; Martell, 2013; Sue et al., 2009). In Martell’s (2013) study of the interplay of racial identity and experiences in learning U.S. history, race played no statistically significant role in students’ perceptions of a history curriculum except for in one instance. When it came to agreeing that the curriculum had caused them to “view history as made up of many different perspectives,” students of color (n = 49) were much more likely to say it had (p < 0.05). Gender was revealed as a primary factor affecting curriculum 23 perception in Berman and White’s (2013) study of students’ level of critical media awareness in relation to a curriculum targeting gender scripts in advertising. Values and preferences gained from home culture, and life experiences, were also identified as shaping curriculum perceptions (Berman & White, 2013; Epstein 1998, 2000; Grice & Vaughn, 1992). Content that aligned with students’ values was better liked and considered more interesting by participants in Grice and Vaughn’s (1992) study. The medium of curricula’s conveyance (e.g., film media, magazine or news articles, textbooks) was also identified as shaping students’ perceptions (Berman & White, 2013; Epstein 1998, 2000; Grice & Vaughn, 1992). Epstein’s (1998, 2000) studies of Black students’ responses to US history curricula identified the extent to which narratives and perspectives about academic topics were affirmed or invalidated by trusted family members, as the best explanation for perception difference. Summary Perceptions about humanities curricula have a major impact on student experience in schools, and are associated with significant outcomes such as attendance, GPA, and beliefs about what is possible. Several salient perceptions have been surfaced, including teacher expectations, utility of content, representation of students’ backgrounds, corroboration of familial historical narratives, relatedness to historical events that link to current events, and inclusion of topics directly tied to student interests and values. The presence of microaggressions or racial bias in the curriculum has pernicious impacts on levels of engagement. Race, gender, home/family culture, and life experiences are identified as factors shaping how students perceive curricula. In addition to their 24 perceptions of extant curricula, students have perceptions of how to improve and redesign curricula to better serve them. Much research on student curriculum perceptions remains to be done. Testing of the aforementioned relationships is understudied, limiting generalizability and inference making. Other constructs outside the scope of the literature pool’s findings may also be as or more prognostic of the relationships between curriculum perceptions, engagement, and achievement. Research Gaps Research gaps from the literature pool inform the research question and methods that follow. None of the studies in the literature pool features a student perception measure connected to theory with validity evidence. None directly and systematically collects and statistically analyzes student perceptions of a social studies curriculum. No study directly compares how students’ curriculum perceptions relate to achievement and engagement, and few robustly address how they relate to sociodemographic constructs. Furthermore, critical bicultural pedagogy (Darder 1991, 2012) is not a focus of any reviewed studies. Research addressing these gaps could inform policy and practice aimed at retaining and engaging diverse youth across educational settings. Theoretical Framework The present study relies on Antonia Darder’s (1991, 2012) theoretical framing of critical bicultural pedagogy (CBP) to conceptualize areas of disconnect between schools and bicultural students. Within that framing, this study focuses on areas of disconnect between humanities curricula and bicultural students, as will be further addressed in the Discussion. Critical bicultural pedagogy is a theory of teaching that aims to prepare 25 bicultural students “to become transformative agents in their world, on behalf of themselves as individuals and collectively for their communities” (Darder, 2012, p. 101). Underpinning CBP is the notion that schools rarely prepare bicultural students to be such transformative agents, and instead tend to create obstacles keeping bicultural students from accessing their political potential. The experience of exclusion from the hidden and explicit curricula of schools and school systems disempowers bicultural students often to such an extent that they believe they should not attend school or should not try hard if they do attend (Darder, 2012). Implementing CBP requires teachers to actively use tools of critical democracy (Darder 1991, 2012) in order to connect school experiences to the identities of bicultural students. Critical democracy builds upon ideas originating in the field of critical pedagogy (Freire, 1970; Greene, 1967, 1973; Illich, 1971; Kohl, 1967, 1969; Kozol, 1967, 1972), and the concept of cultural democracy (Ramirez & Castañeda, 1974) in particular. Briefly, critical democracy is an interpretation of democracy in which only acts of struggle, conflict, and dissent aimed toward subverting inequities are considered democratic (Carr, 2010). CBP posits that schools should be sites in which bicultural students learn the skills and knowledge necessary to actively participate in political processes designed to positively transform their circumstances and environments. Darder (2012) refers to such schooling as being emancipating and empowering. Emancipation is discussed as engagement with liberatory democratic and participatory action within one’s institutions, community, region, or state/nation. Emancipation is experiencing liberation from oppression or injustice. Empowerment is discussed as the establishment of power 26 allowing for the transformation of social inequities and injustices. Empowerment is consonant with manifest social agency. Curricula aligned with CBP foster the skills and knowledge requisite for meaningful civic engagement. In particular, a type of civic engagement aligned with emancipation and empowerment. Building evidence for what students perceive to be most useful for their emancipation and empowerment could inform curriculum development aimed at engaging and retaining students. As a result of CBP having no associated measure, this study utilizes a newly developed survey instrument, the Critical Curriculum Perception Measure (CCPM; Khatter, 2018). The instrument measures perceptions of how well a social studies curriculum aligns with the emancipation and empowerment goals of CBP. Through the lens of CBP, it would be predicted that students would have the most favorable perceptions of curricula with the greatest focus on civic engagement and that they would find such curricula more engaging. The latter assumption is directly tested in the present study, allowing for important gaps in the literature on student perceptions of curriculum to be addressed. Developmental and validity considerations for the CCPM and other subscales of the assessment battery will be provided in the Measures section. Darder (2012) draws explicit connections between CBP’s goals of emancipation and empowerment with civic engagement and curricular engagement. The assessment battery in this study allows students’ levels of social studies curriculum engagement and achievement to be measured in relation to their perceptions of how aligned a social studies curriculum is with both CBP and civic engagement (vis-à-vis community service self-efficacy). 27 Research Question With these gaps and the potential utility of addressing them in mind, the research question for the present study is: Do student perceptions of alignment between social studies curriculum and the empowerment and emancipation goals of critical bicultural pedagogy (Darder, 1991, 2012) differ in relation to students’ levels of achievement, levels of engagement, and/or sociodemographic constructs, such as race and gender pronouns? 28 CHAPTER III METHODS To answer the research question presented in the previous section, I employ a mixed methods survey research design (Creswell & Clark, 2017). My sample of 192 secondary students is drawn from four schools in a single school district. Average scores on a scale of curricular alignment to the goals of critical bicultural pedagogy (Darder, 1991, 2012) are compared by achievement level, engagement level, race, gender pronouns, and socioeconomic status. Open-responses to an item about the purpose of school are coded to drive a second iteration of comparisons and provide a basis for qualitative analysis between groups with pronounced differences. A more detailed description of the analytical methods employed is provided in the sections that follow. A mixed methods approach was chosen because it allows for added layers of meaning to be applied to data analysis (Cohen et al., 2018). The relationships between the constructs may be complex, and a fuller rendering of their interactions can emerge from a mixed methods approach (Creswell & Creswell, 2018; Babbie, 2021). In the present study, mixed methods allow for coded qualitative data to be a sieve for further quantitative and qualitative comparisons. The research design allows for pronounced quantitative differences between disaggregated groupings to be qualitatively examined. Sample I draw my sample of students from four different secondary schools in one of Oregon’s largest school districts (N ≈ 15,000). Student participants were volunteers in a non-random convenience sample from 11 sections taught by four social studies teachers from four schools. The teachers were selected out of an expression of interest in 29 generating baseline data to inform a curriculum adoption process. Students were in grades six through ten. After the elimination of blank and ostensibly mischievous responses, 192 of 266 student participants remain in the study. Mischievous responses are characterized by repeated or non-existent values for all Likert-style items and incomprehensible (e.g., random keystrokes) or non-existent responses to open-response items. Table 5 contains the students’ self-reported sociodemographic information. Eighty-three students identify as multiracial or non-White, constituting 43% of the sample, in a district that is 69% White (ODE, 2018). However, comparisons between students’ self-reported demographics in this study and Oregon Department of Education’s (ODE) district reporting are limited because multiracial students are accounted for differently in each instance. Social studies classes are selected because, in addition to complementing the literature review and research question, they offer unique opportunities for research. This is a result of how unconstrained their curriculum and evaluation is in some states. In Oregon, for instance, no mandated standardized social studies test or text selection exists. There is wide variability in how social studies is taught, with individual districts, schools, and teachers making many of the decisions about topics, methods, and materials (Fitchett & Vanfossen, 2013; Ross, 2001). Social studies coursework is uniquely responsible, per state and national standards, for addressing controversial matters and civic skills (National Council for the Social Studies, 2010; ODE, 2021b). The use of a convenience sample exacerbates a sample’s bias and limits generalizability. Even so, education research often relies on convenience sampling because of the naturally formed groupings of students cohorted into sections (Creswell & 30 Creswell, 2018). In an effort to reduce sampling bias, I solicited participation from at least one social studies teacher from each middle and high school in the district. However, only four of twelve schools ultimately participated. Despite the limitations of the sampling technique, differences in settings and social studies curricula allow for a meaningful investigation to occur. Four other factors affecting bias in the sample can be explored to add context. First is the absence of data from the 74 participants who gave blank or mischievous responses and the uncounted number of students who chose not to take the survey. Without data from these populations, the sample only reflects students with the ability and motivation to want to complete the survey. Taking the average class size at the four participating schools (29.25), multiplying by the eleven sections to which the survey was administered and subtracting out the total number of respondents, I can approximate that the number of students who chose not to take the survey is between 50 and 60. Combined with non-responses present in my data, this suggests my overall response rate is around 60 percent. This is roughly in range of large portions of survey research in schools (Nulty, 2008). Third, students of color are overrepresented in the sample at 43%, compared to the average of 31% at the four schools. This suggests that a wider variety of perspectives from students of color are represented than from White students. Fourth, higher achieving students, who identified as achieving at an A or highly proficient level, are likely also overrepresented (53%). As a result, the findings of this study may be least generalizable to lower achieving students. A second source of bias, the accessibility and interference of alternative tasks, concerns the 74 excluded participants, who appear to have initiated the survey and then 31 Table 5. Self-reported demographics of research participants. Demographic Categories N Race Asian, Asian American, or Asian Multiracial 15 Black, African American, or Black Multiracial 18 Indigenous American, Native American, or Indigenous American Multiracial 18 Latina/o/x, Hispanic, or Latina/o/x Multiracial 32 Pacific Islander, Native Hawaiian or Samoan, or Pacific Islander Multiracial 2 White, European American, not Multiracial 109 Gender Pronouns Male 83 Female 90 Non-binary 13 Grade Sixth 18 Seventh 31 Eighth 88 Ninth 29 Tenth 26 Social Studies Achievement Level A/Highly Proficient 101 B/Proficient 61 C/Nearly Proficient 20 D/Emerging Proficiency 3 F/Not Enough Evidence 7 decided not to complete it. Several plausible factors may account for their disengagement with the survey. One is that the novelty of computer access in the classroom setting without an assessed task. At the time data was collected, use of computers in these students’ classrooms was relatively infrequent. Without the accountability of an 32 assessment related to the choice of tasks, students may have been tempted to engage in different tasks. Finally, the third and fourth factors may relate to the survey administration protocol and the design of the survey. Without an option to have the prompts read aloud, survey administration required a level of sustained reading that some students may have found too onerous. The English-only design of the survey may have also prohibited engagement. The assessment contains vocabulary some students may be unfamiliar with. Emergent multilingual learners and students with reading comprehension gaps may have found decoding the survey prompts to be overly complex. Setting The assessment was administered in 11 social studies sections at four different secondary schools in one of Oregon’s largest school districts (N ≈ 15,000). None of the schools implemented the same social studies curricula or shared a curricular map or scope and sequence. School and student demographics from the four schools are summarized in Table 6. Procedure In the midst of their regular social studies programming, students were administered the assessment via Google Forms. On a day of their choice, teachers improvised from a script to inform students of the opportunity to take the survey or engage in a separate computer-based academic activity that was also provided. Students were made aware that the survey data would be used to help make purchasing decisions about social studies curriculum materials as part of a district-wide process. Students were given approximately 45 minutes to complete the survey. 33 Table 6. Demographics of the research settings. Angelou Middle Fields Middle Parish Middle School School School Green High School (5 sections) (3 sections) (1 section) (2 sections) Student Population 568 422 504 1,363 Median Class Size in Core Content 33 28 29 27 Areas Free/Reduced Lunch Served 29% >95% 34% 28% Students with Disabilities 7% 18% 10% 11% Ever Classified as ELL <5% 7% 5% 28% Students Attending 90% of School 86% 80% 85% 76% Days Students of Color 29% 41% 27% 28% Comparison to State Average for Standardized ELA +14% +8% +7% Not available Assessment Licensed Teachers with More Than 3 Years of Teaching 87% 86% 92% 96% Experience Average Teacher Turnover Rate 15% 18% 23% 13% Note: All table data collected from the 2019 Oregon Report Card. Data collection was approved by the district’s Curriculum Director as part of an internal curriculum adoption process. There was no plan to use the data for the present study4 at the time of its collection and it was collected anonymously. As a result, there was not an opportunity to gain consent from students or their guardians for the data to be 4In March 2020, I was preparing a dissertation that utilized the measures mentioned in this study as well as two more. The primary purpose of the originally planned study was to measure the extent to which the civic engagement orientation of curricular units related to students’ CCPM scores. The COVID-19 pandemic has, as of the May 2021 submission of this manuscript, made data collection for the previously planned study untenable. The present study’s dataset was selected because it largely reflects the originally planned assessment battery. 34 used in a research project. Communication with administrators in the district revealed that student surveys were routinely used for a wide-variety of purposes, thus no special permission was considered necessary to collect the data used in this study. The district’s Research and Planning Director granted use of the data for this study. The assessment was administered between December 2019 and February 2020. Measures Three open-response items and 29 Likert-style scale-response items constitute the assessment battery. In addition, there is a prompt about students’ self-assessment of their social studies achievement level and nine sociodemographic prompts (e.g., gender pronouns, race, and socioeconomic indicators). Scale-response items are derived from theoretical text and adapted from extant measures. Each item’s data was converted from categorical (e.g., “Somewhat Agree”) to continuous (e.g., 0.5) before analysis because perceptions are considered to exist on valence continuums (Sharot & Garrett, 2016; Lebrecht et al., 2012). Students’ assessment of the purpose of school, the purpose of social studies coursework, and how their social studies courses could be improved constitute the three open-response items. There are two units of analysis: (1) individuals, and (2) disaggregated sociodemographic groups. Sixteen of the 29 scale-response items are derived from Darder’s (1991, 2012) theory of critical bicultural pedagogy (CBP) to measure students’ perceptions of the extent to which their social studies curricula align with the CBP goals of student emancipation and empowerment. These items constitute the Critical Curriculum Perception Measure (CCPM; Khatter, 2018). The remaining thirteen assessment items are adapted from student community service self-efficacy and school engagement measures. 35 No extant measures are used. A summary of this decision’s rationale follows: 1. The gaps in the literature are so vast on the potentially crucial topic of student perception of curriculum that they need to be immediately and directly addressed. There was insufficient opportunity to psychometrically evaluate a new measure before administering it, and no extant measures sufficiently captured the nuances of the constructs. For example, while some psychometrically sound school engagement measures do exist, none focus on interrogation of student engagement specifically in social studies or humanities courses. 2. Relevant extant measures used vocabulary that many students may have struggled to understand, so adaptations are used wherever possible. Participant burden (Denscombe, 2009), for students, especially those who may struggle to read, write, or use digital media, was considered too risky for a longer or more complex survey administration. There are also related validity concerns about the accuracy of student responses on a longer or more complex measure. Darder (personal communication, August 10, 2018) confirmed the importance of simplicity and directness in survey item terminology designed for diverse secondary students. As indicated above, items with the strongest factor loadings available were targeted for adaptation, but many of them fell under the .70 threshold considered authoritative (Hair et al., 2010; MacCallum et al., 1999, 2001). Below that threshold, or even below .80, loadings indicate that items are associated with multiple or multidimensional constructs. Because no extant measures were used, a decision was 36 made to adapt from the best items possible (e.g., the items with the highest factor loadings) for the scale constructs of interest in order to evaluate validity evidence for the CCPM. Items with the strongest factor loadings for their scales can be used, or adapted, to at least proximally represent their scales. Psychometricians’ evaluations of appropriate factor loadings cut-offs differ somewhat broadly. Loadings at or above .40 are often considered appropriate for identifying items of practical significance (Ertz et al., 2016; Tabachnick & Fidell, 2007; MacCallum et al., 1999, 2001), which is a threshold all items adapted for the present study consistently performed above in validity research mentioned below. All measures and their psychometric properties are described in greater detail below. Appendix B contains comparisons of the original versions and adaptations (where applicable) of each item. Curricular Alignment with Community Service Self-Efficacy Measure Three items with high face validity and strong factor loadings are adapted from the Community Service Self-Efficacy Scale (CSSES; Reeb, et al., 1998) in order to generate convergent construct validity for the CCPM by constituting a comparison scale. The underlying hypothesis is that curricular alignment with CBP goals is associated with curricular alignment to nurturing community service self-efficacy to a statistically significant extent. In other words, it seems reasonable to hypothesize that a curriculum perceived to be empowering and emancipating would also be perceived as preparatory for making contributions to the community. Community Service Self-Efficacy Scale. The CSSES was designed in 1998 by Roger Reeb, Ronald Katsuyama, Julie Sammon, and David Yoder of the University of 37 Dayton to advance research about the role of self-efficacy in secondary and tertiary service-learning. More specifically, the CSSES was designed to “assess the student’s confidence in his or her own ability to make clinically significant contributions to the community through service” (Reeb et al., 1998, p. 49). It was generated from the theory of self-efficacy proposed by Bandura (1977). Each original CSSES item was given a Likert-style score, adapted for the present study to the aforementioned -1 to 1 scale. A scale score is calculated by adding scores on each of the three items and dividing by the maximum possible score of three. The quotient yields an overall score on a -1 to 1 scale for the extent to which students perceive a curriculum to be aligned with community service self-efficacy. Three items are adapted from the CSSES (Reeb et al., 1998) for the present study. The items were chosen because they (a) possess high factor loadings (.70 to .88 in the studies mentioned in the next paragraph), (b) appear to provide a proximal measurement of civic service self-efficacy, (c) appear to be capable of being adapted to relate to student perception of curriculum, and (d) appear to be capable of being used in adapted form to create convergent validity evidence for the CCPM. Validity and Reliability. Reeb et al. published original validity and reliability research for the CSSES in 1998. Construct validity evidence was generated through factor analysis, revealing loadings ranging .67 to .81. An evaluation of scale inter-item consistency yielded a Cronbach’s alpha coefficient of .92. Participants (N = 676, 53% female, Mage = 20.6) were undergraduates at an undisclosed location. 83% identified as Christian, 88% White, 3% Black, 1.5% Hispanic, 1% Asian, and 6.5% identified as multiracial, “other,” or refused to self-identify. Reeb et al.’s (1998) findings were 38 corroborated by additional validity and reliability research (Bringle et al., 2004; Reeb 2006; Stewart & Bai, 2010). The three selected CSSES items appear to be capable of being validly adapted to gather evidence about the alignment of curriculum with community service self-efficacy and of being split from their scale to give proximal scale data, despite a lack of research directly supporting these claims. For instance, “I am confident that, through community service, I can make a difference in my community,” seems to largely overlap with the adapted item, “The content and activities of my social studies classes often help me feel more confident that, by working with others, I can make a positive difference in my community.” The items’ tenability is supported by the strong factor loadings they maintained and stability of the measure across contexts. Social Studies Curriculum Engagement Measure Ten items are drawn from two school engagement measures to differentiate student levels of engagement with their social studies curriculum and create further convergent construct validity evidence for the CCPM. The more empowering and emancipating students perceive their social studies curricula to be, the more they are hypothesized to find those curricula engaging. Two items with high face validity and relatively strong factor loadings are from the Student Engagement Instrument (SEI; Appleton & Christenson, 2004). Eight items with high face validity and relatively strong factor loadings are from an instrument titled My Class Activities (MCA; Gentry & Gable, 2001). School Engagement Instrument. The SEI was developed in 2004 by James Appleton of Gwinnett County Schools in Georgia and Sandy Christenson of the 39 University of Minnesota to assess secondary students’ cognitive and affective school engagement through self-reported survey responses. While academic and behavioral engagement are measurable from grade point average (GPA) and attendance/participation data, cognitive and affective engagement require self-reporting. Appleton and Christenson (2004) theorized that these four domains (academic, behavioral, cognitive, and affective) comprise a comprehensive reflection of student engagement. The SEI contains two meta-scales, each comprised of three subscales. The affective engagement meta-scale features 19 items, and the cognitive engagement meta- scale features 16 items. Two of the nine cognitive engagement items on the control and relevance of schoolwork subscale are adapted for the curriculum engagement scale of the present study. The two items derived from the control and relevance of schoolwork subscale were chosen because they (a) possess the highest factor loadings for their subscale (.60 to .68 in the studies mentioned in the next paragraph), (b) provide a proximal measurement of student engagement, (c) appear to relate or be capable of being adapted to relate to student perception of curriculum, and (d) appear to be capable of being used in adapted form to create convergent validity evidence for the CCPM. Each original SEI item was given a Likert-style score, adapted for the present study to the aforementioned -1 to 1 scale. A scale score is calculated by adding scores on each of the two SEI items with the scores from eight other items adapted from the My Class Activities (MCA; Gentry & Gable, 2001) and dividing by the maximum possible score of ten. The quotient yields an overall score on a -1 to 1 scale for the extent to which students perceive a curriculum to be engaging. 40 Validity and Reliability. Appleton, Christenson, Kim, and Reschly published original validity and reliability research for the SEI in 2006. Construct validity evidence was generated through factor analysis, revealing loadings ranging .45 to .89. Exploratory evaluations of subscale inter-item consistency yielded Cronbach’s alpha coefficients between of .72 and .88. The research was gathered from 75% of ninth grade participants who were solicited (N = 1940) at random from an urban upper Midwest school district. Forty percent of participants identified as African American, 35% as White, 11% as Asian, 10% as Hispanic, and 4% as American Indian. 51 percent identified as female, and 23% of students said their primary home language was not English. Appleton et al.’s (2006) findings were corroborated by additional validity and reliability research (Betts, Appleton, Reschly, Christenson, & Huebner, 2010; Lovelace, Reschly, Appleton, & Lutz, 2014). Items from the SEI appear to be capable of being validly adapted to gather evidence about social studies curriculum engagement and of being split from their subscale to give proximal student engagement data, despite a lack of research directly supporting these claims. “The tests in my classes do a good job of measuring what I’m able to do” seems to largely overlap with the adapted item, “The tests or big assignments in my social studies classes often do a good job of measuring what I’m able to do.” The other item (“Most of what is important to know you learn in school.”) will be used in its original form to anchor items about curriculum engagement to school engagement. The items’ tenability is supported by the relatively strong factor loadings they maintained and the stability of the measure across contexts. 41 My Class Activities. Marcia Gentry of Minnesota State University and Robert Gable of the University of Connecticut-Storrs developed the MCA in 2001 to measure student motivation. It was designed for lower secondary students and asks about their perceptions of classroom activities. The MCA features four subscales (Interest, Challenge, Choice, and Enjoyment), and two items from each subscale are adapted for the present study. These items were chosen because they (a) possess relatively high factor loadings (M = .63 in the studies mentioned in the next paragraph), (b) provide a proximal measurement of student engagement, (c) appear to relate or appear to be capable of being adapted to relate to student perception of curriculum, and (d) appear to be capable of being used in adapted form to create convergent validity evidence for the CCPM. Each original MCA item was given a Likert-style score, adapted for the present study to the aforementioned -1 to 1 scale. As mentioned for the Student Engagement Instrument, a curriculum engagement scale score is calculated by adding scores from two adapted SEI items with the scores from eight adapted MCA items and dividing by ten. The quotient yields an overall score for the extent to which students perceive a curriculum to be engaging. Validity and Reliability. Gentry and Gable published original validity and reliability research for the MCA in 2001. Construct validity evidence was generated through factor analysis, revealing loadings ranging .32 to .86. Evaluations of scale inter- item consistency for each subscale yielded Cronbach’s alpha coefficients between of .75 and .92. The research was gathered from 1,523 grade 6-8 students in 61 core content area classrooms, each with a different teacher, from eight randomly selected schools in varying regions of the United States (one urban, five suburban, two rural). All classrooms 42 were solicited through the network of schools associated with the University of Connecticut’s National Research Center on the Gifted and Talented. 83% of respondents identified as White, 5% as African American, 9% as Asian, 2% as Hispanic, and 51% as male. Gentry and Gable’s (2001) findings were corroborated by additional validity and reliability research (Gentry, Rizza, & Gable, 2001; Pereira, Peters, & Gentry, 2010; Yang, Gentry, & Choi, 2012). Items from the MCA appear to be capable of being validly adapted to gather evidence about student perceptions of social studies curriculum engagement and of being split from their subscales to gather proximal curricular perception data. The relative consistency and strength of the items’ factor loadings across varied contexts support these claims about the items’ flexibility. My adaptations of items largely overlap with their original forms. For instance, “I like what I do in my class,” largely overlaps with, “I often like what I do during my social studies classes.” Open Response Items Using Creswell and Creswell’s (2018) suggestion to utilize prompts or questions that directly relate to my research, I designed three open-response items. Similar to interview items, open-response items that prompt participants to type can be delivered through a protocol designed to present an item and solicit a response (Creswell & Creswell, 2018). These three items capture qualitative data that can (a) inform decisions about instruction and content, and (b) be interpreted within the framework of Darder’s (2012) theory of critical bicultural pedagogy: • What do you think the purpose of school should be? Why? 43 • What do you think the purpose of social studies classes should be? How well does this class line up with what you think the purpose should be? Explain • How would you recommend the last unit in your social studies class be changed? Why? Do any of the items you just rated about your last social studies curriculum unit help you identify areas that should be changed? [This follows all of the scale-response items.] Explain. Achievement Levels Students self-reported their social studies course achievement in response to the multiple choice prompt “What grade or mark do you usually get in your social studies classes? (Choose the best answer.)” For statistical comparison across averages, the largest achievement group (“A / Highly Proficient”; “A”; N = 101) is compared with three others: “B / Proficient” (“B”; N = 61), “C / Almost Proficient” (“C”; N = 20), and “F / Not Passing / Not Enough Evidence” (“F”; N = 7). Some comparisons of “Higher” and “Lower” achievement levels are also conducted to compare students in the two highest achievement groups (“A” and “B”) with students in the two lowest (“C” and “F”). Students who self-reported “D / Emerging Proficiency” are omitted from achievement analysis because of that category’s very low response frequency (N = 3). Engagement Levels Social studies curriculum engagement scores are disaggregated into Lower, Middle, and Higher levels. The scores are generated from the assessment scale combining items adapted from the School Engagement Instrument (Appleton & Christenson, 2004) and My Class Activities (Gentry & Gable, 2001) measures. Students in the Lower 44 engagement category (N = 59) have overall engagement scores at least 0.5 SDs below the sample mean. Students in the Middle category (N = 64) score within ± 0.5 standard deviations of the sample mean. The Higher category reflects scores at least 0.5 SDs above the sample mean (N = 69). Sociodemographic Constructs Racial Categories. Race was self-selected from one or more of the following categories: • Indigenous American | Native American | Alaska Native | American Indian • Native Hawaiian | Indigenous Pacific Islander | Polynesian | Samoan • Latina/o/x | Mestiza/o/x | Hispanic • African American | African | Black • Asian American | Asian • European American | European | White • Other (explain) For race-based analysis, student selections are entered into the following categories: Indigenous American or Indigenous American Multiracial (N = 18), Latinx or Latinx Multiracial (N = 32), Black or Black Multiracial (N = 18), Asian or Asian Multiracial (N = 15), and White-only (N = 109). Students who responded “Native Hawaiian | Indigenous Pacific Islander | Polynesian | Samoan,” or “Other” are excluded from race-based analysis because each category had only two to three respondents. Students who selected all of the racial options are excluded from race-based analysis as mischievous respondents. Race-based statistical analysis compares the largest racial category, White-only, with the four others. 45 Gender Pronouns. Students self-reported their preferred gender pronouns in response to the multiple choice prompt, “What are your preferred pronouns?” Gender pronoun analysis compares the largest self-selected pronoun category, she/her/hers (N = 90) with students who selected he/him/his (N = 83) and students who selected they/them/theirs or either of the two other categories and they/them/theirs (N = 13). Six students were omitted from pronoun-based analysis because they submitted alternative responses that could not be interpreted for preferred pronoun data, such as the entry “racecar”. Socioeconomic Levels. Like engagement, approximated Socioeconomic Level scores are disaggregated into Lower, Middle, and Higher levels, then statistical analysis is used to compare the largest group (Middle) with the other two. SES levels are generated through a scale score created from student responses to three literature- indicated questions as summarized in Table 7. The quotient of each student’s score on the nine-point scale was used to differentiate scores by standard deviations. Students in the Lower SES (N = 51) category have SES-scale scores at least 0.5 SDs below the sample mean. Students in the Middle category (N = 93) score within ± 0.5 standard deviations of the sample mean. The Higher category reflects scores at least 0.5 SDs above the sample mean (N = 48). Critical Cultural Perception Measure The CCPM (Khatter, 2018) relies on the theoretical framing of critical bicultural pedagogy (Darder, 1991, 2012) to measure the extent to which students perceive curricula to be emancipating and empowering. I developed the CCPM by translating content from Darder’s (2012, p. 101-102) synthesis of critical bicultural pedagogy’s key 46 Table 7. Summary of items, responses, and methods used for socioeconomic level disaggregation. Questions Response Choices Scaling Method Approximately how many • Less than 20 Reponses are converted to a books are in your home? • ~100 score of 1 (“Less than 20”), • Hundreds 2 (“~100”), 3 (“Hundreds”), (OECD, 2017, 2020) • Thousands or 4 (“Thousands”). Try to think about any Students entered a numeric Responses at or below the allowance money your value. mean are converted to a parents/guardians give you score of 1. Responses above or that you earn from them the mean are converted to a to spend on stuff you want score of 2. (food, clothes, going to movies, whatever else). Approximately how much PER WEEK is that? (It's ok to round.) (Gwon & Lee, 2015; Soteriades & DiFranza, 2002; Heo et al., 2014) Math class you're in this • 6th grade math Responses that match the school year? • 7th grade math student’s grade level are • 8th grade math/pre- given a score of 1; that are (Bailie & Wiseman, 2018; Algebra one year above the student’s Oakes, 1987; Useem, 1992; • Algebra grade level a 2; more than Oaks & Guiton, 1995) • Geometry one year above a 3. • Algebra 2 • Pre-Calculus • Calculus • Other: (explain) tenets into Likert-style items. In one example, Darder’s (2012, p. 101) mention of critical pedagogy holding “the possibility for a discourse of hope in light of the tensions, conflicts, and contradictions that students must face in the process of their… development” was translated into the item, “The curriculum for the last unit in my social studies class made me more hopeful about the future.” The final version of the CCPM contains a total of 16 items and features a single scale. Items are responded to on a -1 to 1 scale with -1 corresponding to “strongly 47 disagree,” -.5 to “somewhat disagree,” .5 to “somewhat agree,” and 1 to “strongly agree.” A scale score is calculated by adding scores on each of the 16 items and dividing by the maximum possible score of 16. The quotient yields an overall score on a -1 to 1 scale for the extent to which students perceive a curriculum to be emancipating and empowering through a critical bicultural pedagogy lens (Darder, 2012). Validity. Expert review, literature comparison, and correlations with adapted measures of community service self-efficacy and school engagement appear to provide tenable validity evidence for the CCPM. Expert review and literature comparison (Bannigan and Watson, 2009) were utilized to generate content validity evidence. Darder reviewed (personal communication, August 10, 2018) my survey translation of her (2012) synthesis of CBP’s goals. No item-level edits were recommended, but four additional items were suggested, all of which are included in the CCPM. Correlations between the CCPM and the three CSSES items adapted for the community contribution self-efficacy scale (r = .63, p < .001), and between the CCPM and ten SEI/MCA items adapted for the engagement scale (r = .66, p < .001) generate convergent construct validity. The correlations, and the logic for the measures’ inclusion and adaptations, evince tenable construct validity evidence and support the a priori notions that (a) curricular alignment with the goals of CBP significantly relate to curricular alignment with community service self-efficacy (CSSES, Reeb et al., 1998); and, (b) curricular alignment with the goals of CCPM significantly relate to curricular engagement (SEI, Appleton & Christenson, 2004; MCA, Doolittle & Faul, 2013). Thus, a meaningful analysis of the data can occur in order to address the research question. 48 Concurrent criterion validity evidence for the CCPM is generated from differences in Higher and Lower achieving students’ CCPM scores (see Chapter IV: Results). It is reasonable to predict that students who achieve higher marks in social studies would find it more emancipating and empowering. It would be more surprising to find that lower achieving students consider social studies more emancipating and empowering. Findings that corroborate linkage between achievement levels and CCPM scores generate concurrent criterion validity evidence, though it is also possible that an unobserved factor is driving that relationship. The capacity for qualitative results complementing quantitative results to generate further concurrent criterion validity evidence is addressed in Chapter V: Discussion. Analytic Plan Four phases of analysis occur in the following quantitative-dominant sequential mixed methods survey analysis (Creswell, 2017; Johnson, Onwuegbuzie, & Turner, 2007; Teddlie & Tashakkori, 2009) involving conversion (Onwuegbuzie & Johnson, 2006a). Several t-tests generate evidence for evaluating the null hypothesis: student perceptions of alignment between social studies curriculum and the empowerment and emancipation goals of critical bicultural pedagogy, as captured by the Critical Curriculum Perception Measure (CCPM), do not differ significantly across (a) students’ levels of achievement, (b) levels of curriculum engagement, nor, (c) sociodemographic constructs, on average in the population. Tests are run for the CCPM in its entirety as well as for items of interest that may have special capacity for informing instructional practice and curricular decisions. Comparisons occur using pairwise t-tests to generate empirical 49 evidence of the statistical significance of differences between groups, with one group (the group with the largest n) serving as a reference. I set an alpha threshold of 0.10, at which level there is less than a 10 percent chance that I will commit a Type I error by rejecting the null hypothesis when, in fact, there is no relationship between student characteristics and their perception of the curriculum in the population. For smaller samples, an alpha threshold of 0.10 is often considered appropriate (see, for example: Betensky, 2019). The higher threshold addresses the risk of committing Type II errors, in which a null hypothesis is accepted when, in fact, there really is a significant relationship between constructs. Because it seems reasonable to expect that student characteristics would be associated with perceptions of curriculum, a slightly less stringent significance threshold allows for a wider discussion of the constructs and their possible interactions. The first phase of analysis involves a single iteration of etically coding open- responses to the question, “In a few words or sentences, what do you think the purpose of school should be? Why?” according to nine categories: To gain knowledge (N = 118), To gain skills for financial independence and/or general life success (N = 101), To grow in awareness or become a better person in general (N = 26), To gain skills for working well with others (N = 25), To gain employment/trade skills (N = 20), To have fun (N = 17), To build a healthy community or improve the world (N = 13), To gain skills for success in college (N = 13), and No Response (N = 7). For example, “I think the purpose of school should be learning about what we are currently learning about but also about how to save money and how to do things with your money because I feel like that would be helpful later in life” was coded as “To gain skills for financial independence and/or general life 50 success” because of the student’s emphasis on saving money and using it effectively to help ensure a better future for the individual. In some instances student responses were coded into more than one category, such as “i think that the purpose of school should be to learn important things but to also have to have fun doing it.” This response was coded as “To gain knowledge” because of the emphasis on learning, and also “To have fun” because of how the student paired that idea with learning. The second phase of analysis involves using t-tests to compare group averages on the CCPM by achievement level, curriculum engagement level, race, gender pronouns, and socioeconomic status. All statistical tests are performed through Microsoft Excel. Detailed reporting of these tests appears in the Results section. The third phase involves an application of conversion methodology to use coding categories from the first phase as the basis for additional t-test comparisons among most of the same groups. The three etic categories with highest frequencies are utilized for this analysis. Engagement levels are excluded because their strong correlation (r = .66, p < .001) with the CCPM suggests that their inclusion could be redundant and less explanatory for qualitative analysis of the intersections of CCPM scores and purpose of school categories. In the fourth phase, pronounced quantitative differences between groups within each of the three highest frequency etic categories become the basis for qualitative analysis of open response data. Intersections of race, gender pronouns, socioeconomic status, and self-reported achievement level are used to select a student with a high CCPM score and a student with a low CCPM score within each category to compare. In other words, certain students’ open responses are used to explore pronounced differences in 51 quantitative response patterns on the Critical Cultural Perception Measure with high frequency coding categories as a sieve. Within the scope of this quantitative-dominant mixed methods plan, the use of etic coding and conversion constitute a limited utilization of qualitative methods for exploring qualitative complements to the quantitative analysis. In the Discussion, comparisons of these submissions are interpreted for evidence explaining differences in CCPM scores through the lens of Darder’s (1991, 2012) rendering of emancipation and empowerment. Details on selection heuristics appear in the Results section. Excluded Items and Imputation Methods Student alternatives to the Likert-style response options led to the exclusion of approximately 60 item responses and inclusion of approximately 100 item responses through the application of imputation methods (Dale, 2006; Durrant, 2009). Close to 3% of the 5,568 Likert-style data were impacted by alternative responses. In the event of one or more excluded item responses for a student, as many fewer items constituted a scale score for that individual. In the case of excluded item responses, students’ alternative responses offered insufficient information to estimate how the student would respond on the Strongly Disagree to Strongly Agree continuum. For example, in response to, “The content and activities of my social studies classes have often helped me understand why some people have a lot of money and influence and why most people do not” a student chose to type, “it kinda confuses me why we have to learn this.” When imputation methods were applied, students’ alternative responses appeared to offer sufficient information to estimate their placement on the continuum. For 52 example, in response to, “The content and activities of my social studies classes often help me become more thoughtful and wise,” a student wrote, “disagree.” In this instance, the student’s response was estimated as Somewhat Disagree. In another example, a student responded to the same prompt with, “Sometimes i guess,” which was estimated as Somewhat Agree. 53 CHAPTER IV RESULTS This chapter has six primary sections, five of which focus on quantitative relationships, and the sixth on qualitative relationships. In the first three sections, disaggregated group means of CCPM scale scores are compared by achievement levels, engagement levels, and sociodemographic constructs. In the fourth section, achievement, engagement, and sociodemographics are again compared at the item level for six CCPM items of interest, bifurcated in the analysis for either their focus on cultural relevance curriculum qualities or civic engagement curriculum qualities. Then, achievement and sociodemographic constructs are again compared for students etically coded into the three highest frequency purpose of school categories. In the qualitative analysis, pairs of students are heuristically drawn from each of the three purpose of school categories through reliance on the quantitative analysis documented in the fifth section in order to compare open response data from students with high and low CCPM scores. Differences by Levels of Achievement Students with higher self-identified levels of social studies achievement perceive their social studies curriculum to align with the critical bicultural pedagogy (CBP) goals of student emancipation and empowerment, compared to students with lower self- identified levels of social studies achievement. Students self-reporting B/Proficient as their achievement mark had the highest CCPM scores, followed by A/Highly Proficient, C/Nearly Proficient, and F/No Evidence, as summarized in Table 8.1 below. Likewise, when aggregated into Higher (“A” and “B”) and Lower (“C” and “F”) achievement groups, students in the higher achievement group had higher CCPM scores (mean = 0.25) 54 than students in the lower achievement group (mean = 0.12), as summarized in Table 8.2. I reject the null hypothesis that the mean difference between aggregated lower-achieving and higher-achieving students (.13) is indistinguishable from zero, on average in the population, allowing for a slightly less stringent threshold limit than traditionally held to (p < 0.10). Table 8.1. CCPM means using t-test comparisons by achievement levels. A/HP B/P C/NP F/NE M 0.22 0.29 0.13 0.11 SD .32 .33 .31 .43 N 101 61 20 7 t-statistic - 1.22 1.24 0.69 p 0.23 0.22 0.49 Note. A/HP = A/Highly Proficient. B/P = B/Proficient. C/NP = C/Nearly Proficient. F/NE = F/No Evidence. Each achievement level is compared to A/HP as the reference. Table 8.2. CCPM means using t-test comparisons by Higher/Lower achievement levels. Lower Higher M 0.12 0.25 SD 0.34 0.33 N 27 162 t-statistic 1.78 - p 0.08 - Note. Higher = students in the A/Highly Proficient and B/Proficient categories. Lower = students in the C/Nearly Proficient and F/No Evidence categories. Differences by Levels of Engagement Students with higher scaled levels of social studies curriculum engagement perceive their social studies curriculum to better align with the CBP goals of student emancipation and empowerment, compared to students with lower scaled levels of social studies curriculum engagement. Students in the Higher engagement level had the highest 55 CCPM scores (mean = 0.49), followed by students in the Middle level (mean = 0.19), and students in the Lower level (mean = -0.04), as summarized in Table 9. I reject the null hypothesis that the mean differences between less-engaged and higher-engaged students (.53 and .30) are indistinguishable from zero, on average in the population (p < 0.0001). Table 9. CCPM means using t-test comparisons by engagement levels. Lower Middle Higher M -0.04 0.19 0.49 SD .26 .30 .21 N 59 64 69 t-statistic 12.54 6.64 - p 0.00*** 0.00*** - Note. ***Significance level of < 0.0001. Each engagement level is compared to Higher as the reference. Differences by Sociodemographic Constructs Race Student perceptions of the extent to which their social studies curriculum aligns with the CBP goals of student emancipation and empowerment differ by self-identified racial categories. Students self-identifying as Asian or Asian Multiracial had the highest CCPM scores, followed by Black or Black Multiracial, Indigenous or Indigenous Multiracial, White-only, and Latinx or Latinx Multiracial, as summarized in Table 10 below. I accept the null hypothesis that the mean differences between students of color and White-only students on the CCPM scale are statistically indistinguishable from zero, on average in the population. 56 Table 10. CCPM means using t-test comparisons by race. Asian/ Black/ Indigenous/ Latinx/ White-Only Multiracial Multiracial Multiracial Multiracial M 0.34 0.27 0.25 0.20 0.22 SD .35 .36 .29 .37 .34 N 15 18 18 32 109 t-statistic 1.19 0.51 0.40 0.29 - p 0.24 0.61 0.69 0.78 - Note. Each racial group is compared to the White-Only group as the reference. Gender Pronouns Student perceptions of the extent to which their social studies curriculum aligns with the CBP goals of student emancipation and empowerment differ by self-identified gender pronoun categories. Students self-identifying with masculine pronouns had the highest CCPM scores, followed by feminine, and non-binary, as summarized in Table 11 below. I accept the null hypothesis that the mean differences between gender pronoun categories are statistically indistinguishable from zero, on average in the population. Table 11. CCPM means using t-test comparisons by gender pronouns. Feminine Masculine Non-Binary M 0.22 0.25 0.18 SD .32 .37 .24 N 90 83 13 t-statistic - 0.52 0.55 p - 0.60 0.59 Note. Each pronoun group is compared to Feminine as the reference. Socioeconomic Level Students with the lowest scaled socioeconomic level perceive their social studies 57 curriculum to align with the CBP goals of student emancipation and empowerment, compared to students with higher scaled socioeconomic levels. Students in the Lower socioeconomic level had the highest CCPM scores (mean = 0.29), followed by students in the Higher level (0.24), and the Middle level (0.14), as summarized in Table 12. I reject the null hypothesis that the mean difference between low socioeconomic level students and medium socioeconomic level students (0.15) is indistinguishable from zero, on average in the population (p < 0.01). I accept the null hypothesis that the mean difference between low socioeconomic level students and high socioeconomic level students is statistically indistinguishable from zero, on average in the population. Table 12. CCPM means using t-test comparisons by socioeconomic level. Lower Middle Higher M 0.29 0.14 0.24 SD .33 .34 .34 N 78 63 51 t-statistic - 2.79 0.93 p - 0.00* 0.35 Note. *Significance level of < 0.01. Each socioeconomic level is compared to Lower as the reference. CCPM Survey Items of Special Interest Items appearing to have special capacity to inform instruction and curricular decision-making constitute the reporting of this section. The average CCPM item score was 0.23 (N = 16), with a standard deviation of .60. The standard deviation of all item scores is .20. Two items fall below one standard deviation of the mean CCPM score: Item 6 (M = -0.03, SD = 0.62) and Item 11 (M = -0.22, SD = 0.65). Each is included below. Two items are also above one standard deviation of the mean CCPM score: Item 7 (M = 58 0.60) and Item 14 (M = 0.55). However, neither is included in this analysis because (a) their small standard deviations (< .50) relative to the other items indicate that they are less likely to surface differences relevant to the research question, and (b) they do not appear to carry special capacity to inform decision-making. Items included in the analysis either associate with either cultural relevance qualities of the curriculum (Items 2, 11, and 15) or civic engagement qualities of the curriculum (Items 3, 6, and 9). Following the results provided below, these items are addressed again in Chapter V: Discussion. Cultural Relevance Items Item 2: The content and activities of my social studies classes have often mentioned perspectives, individuals, or events I can relate to. Results are summarized in Table 13 and addressed briefly by comparison type below. Achievement Level. Students self-reporting B/Proficient as their achievement mark have the highest scores on this CCPM item (mean = 0.30), to a statistically significant extent (p < 0.05) when compared to all other achievement groups. No other statistically significant relationships are evident in the analysis. This finding partially supports the rejection of the null hypothesis. Engagement Level. Students with Higher scaled curriculum engagement levels (mean = 0.48) have higher scores on this CCPM item, to a statistically significant extent (p < 0.0001) when compared to each other level, supporting the rejection of the null hypothesis. Race. Asian and Asian Multiracial identifying students have higher scores on this CCPM item (mean = 0.30) than all other racial groups, but no group of BIPOC students differs from White-only students to a statistically significant extent. 59 Gender Pronouns. No meaningful differences between students identifying with different gender pronouns are evident in the analysis. Socioeconomic Level. No meaningful differences between scaled student socioeconomic levels are evident in the analysis. Table 13. CCPM Item 2 (relevance of content) means by achievement, engagement, and sociodemographic constructs. M SD N t-statistic p Achievement Level (I) A/HP 0.14 .59 101 - - B/P 0.30 .57 61 1.61 0.11 C/NP -0.05 .67 20 1.21 0.23 F/NE 0.43 .67 7 1.09 0.43 Achievement Level (II) B/P 0.30 .57 61 - - All Other Levels 0.10 .62 128 2.11 0.04 Engagement Level Lower -0.08 .67 59 5.32 0.00*** Middle 0.10 .62 64 3.97 0.00*** Higher 0.48 .46 69 - - Race Asian/Multiracial 0.30 .62 15 0.74 0.46 Black/Multiracial 0.28 .60 18 0.68 0.50 Indigenous/Multiracial 0.25 .65 18 0.47 0.64 Latinx/Multiracial 0.11 .66 32 0.50 0.62 White-Only 0.17 .58 109 - - Gender Pronouns Feminine 0.19 .60 90 - - Masculine 0.19 .59 83 0.08 0.93 Non-binary 0.15 .75 13 0.19 0.85 Socioeconomic Level Lower 0.17 .60 78 - - Middle 0.18 .64 63 0.09 0.93 Higher 0.19 .58 51 0.12 0.90 Note. ***Significance level of < 0.0001. For each comparison, the group with the largest population size is the reference. 60 Item 11: The content and activities of my social studies classes often help me feel like I understand myself better. Results are summarized in Table 14 and addressed briefly by comparison type below. Achievement Level. No statistically significant relationships are surfaced by the analysis. Engagement Level. Students with Higher scaled curriculum engagement levels have higher scores on this CCPM item (mean = 0.14), to a statistically significant extent (p < 0.0001) when compared to each other level, supporting the rejection of the null hypothesis. Race. No statistically meaningful differences are surfaced in the analysis between racial groups. Gender Pronouns. The analysis surfaces no statistically significant relationships. Socioeconomic Level. No statistically meaningful differences between scaled student socioeconomic levels are surfaced by the analysis. Item 15: The content and activities of my social studies classes often help me feel that I belong in this country. Results are summarized in Table 15 and addressed briefly by comparison type below. Achievement Level. Students self-reporting a social studies achievement level of F/No Evidence have the lowest scores on this CCPM item (mean = -0.21), to a statistically significant extent when compared to the A/Highly Proficient group (allowing for a slightly lax significance threshold; p < 0.10). No other statistically significant relationships are surfaced by the analysis. This finding partially supports the rejection of the null hypothesis. 61 Table 14. CCPM Item 11 (understand self better) means by achievement, engagement, and sociodemographic constructs. M SD N t-statistic p Achievement Level (I) A/HP -0.28 .61 101 - - B/P -0.11 .66 61 1.60 0.11 C/NP -0.35 .61 20 0.45 0.65 F/NE -0.07 .89 7 0.62 0.54 Achievement Level (II) B/P -0.11 .66 61 - - All Other Levels -0.28 .63 128 1.65 0.10 Engagement Level Lower -0.54 .53 59 7.00 0.00*** Middle -0.32 .63 64 4.33 0.00*** Higher 0.14 .58 69 - - Race Asian/Multiracial -0.03 .69 15 1.28 0.20 Black/Multiracial -0.06 .66 18 1.31 0.19 Indigenous/Multiracial -0.17 .64 18 0.66 0.51 Latinx/Multiracial -0.31 .61 32 0.30 0.76 White-Only -0.28 .64 109 - - Gender Pronouns (I) Feminine -0.29 .64 90 - - Masculine -0.13 .66 83 1.58 0.12 Non-binary -0.31 .60 13 0.11 0.92 Gender Pronouns (II) Masculine -0.13 .66 83 - - All Other Pronouns -0.29 .63 109 1.65 0.10 Socioeconomic Level Lower -0.24 .62 78 - - Middle -0.19 .71 63 0.41 0.68 Higher -0.24 .61 51 0.02 0.99 Note. ***Significance level of < 0.0001. For each comparison, the group with the largest population size is the reference. Engagement Level. Students with Higher scaled curriculum engagement levels have higher scores on this CCPM item (mean = 0.38), to a statistically significant extent (p = 0.01) when compared to each other level, supporting the rejection of the null hypothesis. 62 Race. Indigenous and Indigenous Multiracial identifying students have the highest score on this CCPM item (mean = 0.47), to a statistically significant extent when compared to all other racial groups and to specifically White-only students, allowing for a somewhat lax significance threshold (p < 0.10). No other statistically significant relationships are surfaced by the analysis (p > 0.50). These findings partially support the rejection of the null hypothesis. Gender Pronouns. No statistically meaningful differences between students identifying with different gender pronouns are surfaced by the analysis. Socioeconomic Level. No meaningful statistical differences between scaled student socioeconomic levels are surfaced by the analysis. Civic Engagement Items Item 3: The content and activities of my social studies classes have often helped me think about how I can try to make the world a better place. Results are summarized in Table 16 and addressed briefly by comparison type below. Achievement Level. Students self-reporting A/Highly Proficient as their social studies achievement level have the highest scores on this CCPM item, but not to a statistically significant extent when compared to other achievement groups. An exploratory comparison of Higher and Lower achievement groups also did not provide evidence suggesting statistically significant differences. Engagement Level. Students with Higher scaled curriculum engagement levels have higher scores on this CCPM item (mean = 0.63), to a statistically significant extent (p < 0.0001) when compared to each other level, supporting the rejection of the null hypothesis. 63 Table 15. CCPM Item 15 (belong in United States) means by achievement, engagement, and sociodemographic constructs. M SD N t-statistic p Achievement Level A/HP 0.25 .59 101 - - B/P 0.26 .61 61 0.10 0.92 C/NP 0.28 .66 20 0.14 0.89 F/NE -0.21 .70 7 1.73 0.09 Engagement Level Lower 0.10 .68 59 2.56 0.01 Middle 0.20 .58 64 1.83 0.01 Higher 0.38 .56 69 - - Race (I) Asian/Multiracial 0.27 .62 15 0.11 0.91 Black/Multiracial 0.14 .68 18 0.64 0.53 Indigenous/Multiracial 0.47 .53 18 1.64 0.10 Latinx/Multiracial 0.19 .66 32 0.46 0.64 White-Only 0.25 .60 109 - - Race (II) Indigenous/Multiracial 0.47 .53 18 - - All Other Races 0.23 .62 172 1.84 0.07 Gender Pronouns Feminine 0.25 .61 90 - - Masculine 0.25 .60 83 0.03 0.97 Non-binary 0.15 .69 13 0.48 0.63 Socioeconomic Level Lower 0.28 .58 78 - - Middle 0.18 .64 63 0.89 0.28 Higher 0.25 .63 51 0.37 0.78 Note. For each comparison, the group with the largest population size is the reference. Race. Asian and Asian Multiracial identifying students have the highest score on this CCPM item (mean = 0.53), to a statistically significant extent when compared to an aggregation of all other racial groups and to specifically White-only students, allowing for a somewhat lax significance threshold (p < 0.10). This finding partially supports a rejection of the null hypothesis. No other statistically significant relationships are evident in the analysis (p > 0.50). 64 Gender Pronouns. No meaningful differences between students identifying with different gender pronouns are evident in the analysis. Socioeconomic Level. No meaningful differences between scaled student socioeconomic levels are evident in the analysis. Item 6: The content and activities of my social studies classes often make me more hopeful about the future. Results are summarized in Table 17 and addressed briefly by comparison type below. Achievement Level. No meaningful differences between self-reported social studies achievement levels are evident in the analysis. Engagement Level. Students with Higher scaled curriculum engagement levels have higher scores on this CCPM item (mean = 0.28), to a statistically significant extent (p < 0.0001) when compared to each other level, supporting the rejection of the null hypothesis. Race. Black and Black Multiracial identifying students have the highest score on this CCPM item (mean = .25), to a statistically significant extent when compared to all other racial groups and to specifically White-only students (p ≤ 0.05). The analysis surfaced no other statistically significant relationships. This finding partially supports a rejection of the null hypothesis. Gender Pronouns. Non-binary pronoun identifying students have the lowest score on this CCPM item (mean = -0.38), to a statistically significant extent when compared to Feminine pronoun identifying students (mean = -0.02; p < 0.05). A comparison of Non-binary pronoun identifying students with Masculine pronoun students (mean = 0.01) also yielded a statistically significant difference (p < 0.05). This finding 65 Table 16. CCPM Item 3 (make the world better) means by achievement, engagement, and sociodemographic constructs. M SD N t-statistic P Achievement Level A/HP 0.34 .57 101 - - B/P 0.33 .58 61 0.09 0.93 C/NP 0.23 .57 20 0.80 0.43 F/NE 0.00 .82 7 1.07 0.29 Engagement Level Lower -0.07 .63 59 7.45 0.00*** Middle 0.30 .53 64 4.20 0.00*** Higher 0.63 .37 69 - - Race (I) Asian/Multiracial 0.53 .48 15 1.84 0.07 Black/Multiracial 0.33 .51 18 0.39 0.69 Indigenous/Multiracial 0.28 .60 18 0.01 0.99 Latinx/Multiracial 0.34 .53 32 0.57 0.57 White-Only 0.28 .63 109 - - Race (II) Asian/Multiracial 0.53 .48 15 - - All Other Races 0.30 .60 175 1.79 0.07 Gender Pronouns Feminine 0.29 .60 90 - - Masculine 0.28 .60 83 0.12 0.90 Non-binary 0.32 .51 13 0.58 0.56 Socioeconomic Level Lower 0.32 .59 78 - - Middle 0.32 .60 63 0.03 0.98 Higher 0.26 .56 51 0.54 0.60 Note: ***Significance level of < 0.0001. For each comparison, the group with the largest population size is the reference. partially supports a rejection of the null hypothesis. Socioeconomic Level. No meaningful differences between scaled student socioeconomic levels are evident from the analysis. Item 9: The content and activities of my social studies classes often help me see how, by working together with others, I can make the world a better place. Results are summarized in Table 18 and addressed briefly by comparison type below. 66 Table 17. CCPM Item 6 (hopeful about the future) means by achievement, engagement, and sociodemographic constructs. M SD N t-statistic p Achievement Level A/HP 0.00 .59 101 - - B/P -0.04 .65 61 0.35 0.72 C/NP -0.05 .65 20 0.29 0.77 F/NE 0.00 .82 7 0.02 0.99 Engagement Level Lower -0.42 .54 59 7.52 0.00*** Middle 0.00 .60 64 2.89 0.00*** Higher 0.28 .52 69 - - Race (I) Asian/Multiracial 0.17 .67 15 1.31 0.19 Black/Multiracial 0.25 .65 18 1.98 0.05 Indigenous/Multiracial -0.03 .70 18 0.26 0.79 Latinx/Multiracial -0.03 .61 32 0.35 0.73 White-Only -0.07 .60 109 - - Race (II) Black/Multiracial 0.25 .65 18 - - All Other Races -0.04 .62 172 2.40 0.02 Gender Pronouns (I) Feminine -.02 .59 90 - - Masculine .01 .65 83 0.36 .72 Non-binary -.38 .55 13 2.21 .03 Gender Pronouns (II) Non-binary -0.38 .55 13 - - Masculine 0.01 .65 83 2.37 0.20 Socioeconomic Level Lower -.05 .64 78 - - Middle .03 .63 63 0.77 .44 Higher -.07 .59 51 0.16 .88 Note: ***Significance level of < 0.0001. For each comparison, the group with the largest population size is the reference. Achievement Level. Students self-reporting a social studies achievement level of F/No Evidence have the lowest scores on this CCPM item (mean = -0.29), to a statistically significant extent when compared to the A/Highly Proficient group, allowing for a somewhat lax significance threshold (p < 0.10). No other statistically significant 67 relationships are surfaced by the analysis. This finding partially supports the rejection of the null hypothesis. Engagement Level. Students with Higher scaled curriculum engagement levels have higher scores on this CCPM item (mean = 0.57), to a statistically significant extent (p < 0.0001) when compared to each other level, supporting the rejection of the null hypothesis. Race. Asian and Asian Multiracial identifying students have the highest score on this CCPM item (mean = 0.63), to a statistically significant extent when compared to all other racial groups and to specifically White-only students (p < 0.001). Also, Latin American heritage students score high (mean = 0.38) to a statistically significant extent when compared to White-only students, allowing for a somewhat lax significance threshold (p < 0.10). No other statistically significant relationships are surfaced by the analysis. These findings partially support a rejection of the null hypothesis. Gender Pronouns. No meaningful differences exist between students identifying with different gender pronouns. Socioeconomic Level. No meaningful differences between scaled student socioeconomic levels are evident from the analysis. Differences by Coding of Student-Reported Purpose of School To Gain Knowledge Results are summarized in Table 19 and addressed briefly by comparison type below. Achievement Level. Students reporting B/P (mean = 0.34) and A/HP (mean = 0.19) achievements had the highest scores on the CCPM within this category, followed 68 Table 18. CCPM Item 9 (collectively make the world better) means by achievement, engagement, and sociodemographic constructs. M SD N t-statistic P Achievement Level A/HP 0.22 .59 101 - - B/P 0.36 .58 61 1.46 0.15 C/NP 0.23 .57 20 0.02 0.99 F/NE -0.29 .76 7 1.74 0.08 Engagement Level Lower -0.08 .60 59 6.85 0.00*** Middle 0.21 .59 64 3.94 0.00*** Higher 0.57 .43 69 - - Race (I) Asian/Multiracial 0.63 .40 15 3.93 0.00** Black/Multiracial 0.28 .55 18 0.79 0.43 Indigenous/Multiracial 0.25 .65 18 0.52 0.61 Latinx/Multiracial 0.38 .55 32 1.83 0.07 White-Only 0.17 .62 109 - - Race (II) Asian/Multiracial 0.63 .40 15 - - All Other Races 0.22 .61 175 3.63 0.00** Gender Pronouns Feminine 0.26 .58 90 - - Masculine 0.25 .63 83 0.15 0.88 Non-binary 0.23 .56 13 0.18 0.86 Socioeconomic Level Lower 0.24 .61 78 - - Middle 0.32 .60 63 0.78 0.43 Higher 0.19 .58 51 0.48 0.64 Note: ***Significance level of < 0.0001. **Significance level of <0.001. For each comparison, the group with the largest population size is the reference. by F/NE (mean = 0.11) and C/NP (mean = 0.02) students. B/P students outperformed their A/HP peers to a statistically significant extent (p < 0.05). When aggregations of Higher and Lower achieving groups is made, Higher achieving students (mean = 0.24) outperform Lower achieving students (mean = 0.05) to a statistically significant extent (p = 0.05). This finding partially supports the rejection of the null hypothesis. 69 Race. Asian heritage students (mean = 0.41) have the highest score on the CCPM within this category when compared to all other races, allowing for a broad significance threshold (M = 0.21, p = 0.10). No other statistically significant relationships are surfaced by the analysis. These findings partially support the rejection of the null hypothesis. Gender Pronouns. No statistically significant relationships are surfaced by the analysis. Socioeconomic Level. No meaningful statistical differences between scaled student socioeconomic levels are surfaced by the analysis within this category. To Gain Skills for Financial Independence and/or General Life Success Results are summarized in Table 20 and addressed briefly by comparison type below. Achievement Level. No statistically significant relationships are surfaced by the analysis. Race. No statistically significant relationships are surfaced. Gender Pronouns. Students with non-binary pronouns have the lowest CCPM score (M = 0.07) within this category, to a statistically significant extent when compared to Feminine pronoun identifying students (M = 0.21, p = 0.04), and to Masculine pronoun identifying students (M = 0.24, p = 0.04). These findings partially support the rejection of the null hypothesis. Socioeconomic Level. No meaningful statistical differences between scaled student socioeconomic levels are surfaced by the analysis within this category. 70 Table 19. “To Gain Knowledge” CCPM scores by achievement and sociodemographics. M SD N t-statistic P Achievement Level (I) A/HP 0.19 .35 65 - - B/P 0.34 .33 33 2.02 0.05 C/NP 0.02 .32 10 1.59 0.12 F/NE 0.11 .43 7 0.50 0.62 Achievement Level (II) Lower 0.05 0.36 17 1.98 0.05 Higher 0.24 0.34 98 - - Race Asian/Multiracial 0.41 .36 10 1.62 0.11 Black/Multiracial 0.27 .39 14 0.50 0.62 Indigenous/Multiracial 0.23 .30 12 0.20 0.85 Latinx/Multiracial 0.16 .40 21 0.56 0.58 White-Only 0.21 .37 60 - - Race (II) Asian/Multiracial 0.41 .36 10 - - All Other Races 0.21 .36 107 1.68 0.10 Gender Pronouns (I) Feminine 0.21 .34 61 - - Masculine 0.25 .42 44 0.59 0.56 Non-binary 0.09 .19 9 1.51 0.14 Gender Pronouns (II) Masculine 0.25 .42 44 - - Non-binary 0.09 .19 9 1.79 0.08 Socioeconomic Level Lower 0.21 .37 48 - - Middle 0.23 .39 36 0.27 0.79 Higher 0.20 .32 34 0.22 0.83 Note. For each comparison, the group with the largest population size is the reference. 71 Table 20. “To Gain Skills for Financial Independence and/or General Life Success” CCPM scores by achievement and sociodemographics. M SD N t-statistic p Achievement Level (I) A/HP 0.22 .33 52 - - B/P 0.27 .33 31 0.70 0.49 C/NP 0.09 .31 8 1.04 0.30 F/NE 0.03 - 1 - - Achievement Level (I) Lower 0.09 .29 83 - - Higher 0.24 .33 9 1.44 0.15 Race Asian/Multiracial 0.32 .39 7 0.69 0.49 Black/Multiracial 0.20 .32 7 0.11 0.92 Indigenous/Multiracial 0.29 .27 7 0.67 0.52 Latinx/Multiracial 0.24 .36 18 0.29 0.77 White-Only 0.21 .35 54 - - Gender Pronouns (I) Feminine 0.21 .31 43 - - Masculine 0.24 .39 38 0.31 0.76 Non-binary 0.07 .13 8 2.12 0.04 Gender Pronouns (II) Masculine 0.24 .39 38 - - Non-binary 0.07 .13 8 2.07 0.04 Socioeconomic Level Lower 0.23 .32 39 - - Middle 0.17 .40 28 0.64 0.52 Higher 0.24 .29 26 0.10 0.92 Note. For each comparison, the group with the largest population size is the reference. To Grow in Awareness or Become a Better Person in General Results are summarized in Table 21 and addressed briefly by comparison type below. Achievement Level. No statistically significant relationships are surfaced by the analysis Race. No statistically significant relationships are surfaced. Gender Pronouns. No statistically significant relationships are surfaced. 72 Socioeconomic Level. No meaningful statistical differences between scaled student socioeconomic levels are surfaced by the analysis within this category. Table 21. “To Grow in Awareness or Become a Better Person in General” CCPM scores by achievement and sociodemographics. M SD N t-statistic p Achievement Level A/HP 0.26 0.33 20 - - B/P 0.47 0.30 3 1.11 0.28 C/NP - - - - - F/NE - - - - - Race Asian/Multiracial 0.34 .28 4 0.12 0.91 Black/Multiracial 0.66 - 1 - - Indigenous/Multiracial 0.09 - 1 - - Latinx/Multiracial -0.44 - 1 - - White-Only 0.36 .33 15 - - Gender Pronouns Feminine 0.34 .26 14 - - Masculine 0.05 .44 6 1.48 0.16 Non-binary 0.48 .24 2 0.77 0.45 Socioeconomic Level Lower 0.28 .24 5 - - Middle 0.40 .31 11 0.85 0.41 Higher 0.19 .40 8 0.46 0.65 Note. For each comparison, the group with the largest population size is the reference. Qualitative Analysis The three most frequent coding categories focus the results in this section. Student reports of the purpose of school can inform decisions about instruction and content and can be interpreted within a CBP lens. In addition, categories of school purpose can help organize qualitative and quantitative comparisons. This organizational utility is employed in the analysis that follows in order to heuristically identify a pair of students within each category to compare. To gain knowledge (N = 118), To gain skills for financial independence and/or general life success (N = 101), and To grow in awareness or 73 become a better person in general (N = 27) collectively represent 74% of the codes assigned. Each of the coding category comparisons that follow is preceded by a description of their intersectional context. Students’ open responses are quoted without any revisions or markings in regards to writing conventions such as standard capitalization in order to preserve the authentic rendering of their ideas. To Gain Knowledge Of all disaggregated groups, self-reported achievement level groups surface the only statistically significant (p < 0.10) CCPM relationship among responses coded as To gain knowledge (N = 118). Students in this category reporting a “B/Proficient” achievement level in social studies (N = 33) have the highest CCPM scores (M = .34, p < 0.05), while those reporting a “C/Nearly Proficient” (N = 10) achievement level have the lowest (M = .02). A comparison of two student responses, one from the “B” achievement level and one from the “C,” are used to explore differences. In the qualitative analysis that follows, race and gender intersections with achievement within this school purpose category are used to select participant responses to compare. Asian heritage and Black heritage students have the highest CCPM scores of all racial groups. Masculine pronoun students have the highest CCPM scores among pronoun groups. Non-binary pronoun students and students of Latin American heritage have the lowest CCPM scores. Searches for students belonging to the highest and lowest scoring identity intersections are the basis for further analysis. A search for male pronoun students of Asian or Black heritage reporting at the “B” achievement level and scoring above the overall CCPM average score (M = .23) was conducted. There was one positive result, a 74 student of Black heritage scoring .66 (Student 99). Another search was made for non- binary students of Latin American heritage reporting at the “C” achievement level and scoring below the overall CCPM average. There were no positive results, so the lowest scoring (-.38) student of Latin American heritage reporting at the “C” achievement level was selected instead (Student 29). School Purpose. Differences in Student 99 and Student 29’s responses are noticeable in their length and content focus. Student 99, with a CCPM score 1.1 standard deviations higher than his Black heritage peers, submitted nine words, while Student 29, scoring 1.6 standard deviations lower than his Latin American heritage peers, submitted 101. The difference in content focus related to how deeply the students provided extensions of their own thoughts on the matter. Student 29 questions if the school (i.e., its leaders) knows how to keep up with its supposed purpose, while Student 99 indicates that the purpose of school could not be more obvious: “Learning, because what else would we do in school.” At the beginning of his response, Student 29 largely concurs with Student 99, “i think the purpose is to have kids learn but make it fun so we want to learn,” but then continues in another direction, i have been seeing a lot of questions in school telling us what to do put some questions in so we can figure out what to do because when your grown you wont be told to do thing you have to figure out what to do and try to keep up with what is happening in the world because it is changing a lot but school are not changing when they do it is very slow While his school promotes forward thinking for a rapidly changing world, the school itself faces perpetual obsolescence, maintaining a broad gap between communicating the need to prepare for the future and actually equipping students with that preparation. 75 Social Studies Purpose. A shared vision for the core of social studies crystallizes from the students’ responses, but their experience of alignment with that vision differs. Both students stress the centrality of studying culture and multiculturalism in social studies. Student 29 states the purpose of social studies as being “about all culture,” and Student 99 writes that it is to “learn things from other cultures and ethnicities. We also should learn about different countries and the people in them.” Their submissions, roughly the same length at 22 and 27 words respectively, then differ in regards to whether they noted if their experience aligned with that purpose. Differences in their marks on two CCPM items also correspond to their differences of experience. While Student 99 marked Somewhat Agree in response to each of the following, Student 29 Strongly Disagreed: • The content and activities of my social studies classes have often mentioned perspectives, individuals, or events I can relate to. • The content and activities of my social studies classes often help me feel like I understand myself better. In his open response to the social studies purpose prompt, Student 99 offers no information about his experience of his social studies classes aligning with his evaluated purpose for them, but Student 29 writes, “i haven't heard a single thing about my culture or anybody else's.” As with school purpose, Student 29 indicates there is a broad gap between the purpose of social studies and the actual content of social studies classes. Recommended Shifts. Student 29 chooses to write “skip” for this prompt, while Student 99 provides 26 words focused on late work policy. He writes about how his assessed achievement level is an underrepresentation of his capacity to achieve because 76 either he cannot get credit for his late work or he cannot find time to finish it. He says, “I have some things that I haven’t turned in because i have been so busy and if i had more time i would have better scores.” To Gain Skills for Financial Independence and/or General Life Success Of all disaggregated groups, self-reported gender pronoun groups surface the only statistically significant (p < 0.10) CCPM relationship among responses coded as To gain skills for financial independence and/or general life success (N = 101). Students selecting Non-binary pronouns (N = 8) have the lowest CCPM scores (M = 0.07, p < .05), while those selecting Masculine pronouns have the highest (M = 0.24). In the qualitative analysis that follows, race and achievement intersections with gender pronoun within this school purpose category are used to select two sets of participant responses to compare. Asian heritage and Indigenous heritage students have the highest CCPM scores of all racial groups. “B” level and “A” level students have the highest CCPM scores among achievement groups. A sole “F” achievement level student with a score of 0.03, “C” achievement level students, and students of Black heritage and White-only heritage have the lowest respective CCPM scores. Searches for students belonging to the highest and lowest scoring identity intersections are the basis for further analysis. A search for male pronoun students of Asian or Indigenous heritage reporting at the “B” or “A” achievement level and scoring above the overall CCPM average score (M = .23) was conducted. There were two positive results. The student with the highest score, a student of Asian heritage scoring 0.59 (Student 127), was selected. Another search was made for non-binary students of Black or White-only heritage reporting at the “C” or “F” achievement level and scoring 77 below the overall CCPM average. There were no positive results, so the lowest scoring (- 0.19) student of Black or White-only heritage reporting at the “C” or “F” achievement level was selected instead (Student 61). Student 61 is White-only, selects Masculine pronouns, and self-reports achieving at the “C” level. School Purpose. Differences in Student 127 and Student 61’s responses are noticeable in their length and degree of focus on the preparatory nature of school for life after school. Student 127, with a CCPM score 0.7 standard deviations higher than his Asian heritage peers, submitted 46 words, while Student 61, scoring 1.2 standard deviations lower than his White-only heritage peers, submitted 13. The difference in degree of focus has to do with the level of specificity each student raises. Student 61 writes that school should “teach [students] things that could help [them] in the ‘real world,’” which is broader than Student 127, who says school should connect students with “something [they] are really good at and want to pursue.” Continuing in that vein, Student 127 continues that school can do that by giving students the opportunity to “look at aspects of all kinds of professions.” Both students have a sense of how school can prepare them for life after school, but Student 127 offers more details of how that preparation would be experienced by students. Social Studies Purpose. Students 61 and 127 differ in terms of the specific proficiencies they believe social studies education should provide, and are similar in that they both offer an evaluation of how well their current class aligns with those beliefs. For Student 61, social studies should “teach people who have no idea what happened in history.” In other words, social studies should nurture proficiency in students about knowledge of the past. He follows that with an evaluation of his current social studies 78 class, saying that it aligns with that purpose only “a little” because the class mostly addressing “things that [students] already know.” Student 127 names four purposes for social studies: a) learn about history, b) learn about current events, c) discuss connections between history and current events, and d) discuss “what an ideal society should look like.” Rearticulated, social studies should nurture proficiency in students about knowledge of the past, about what is going on in the world today, connections between those, and an ability to imagine changes to society informed by that learning. In his evaluation of his current social studies class, Student 127 indicates it falls short on three of the four purposes: “this class… focuses more on history and facts rather than future and current events.” Both students consider their current social studies classes to significantly under-actualize their potential. Their evaluations of how well their courses nurture proficiency about knowledge of the past corresponds to their marks on two related items from the CCPM: • The content and activities of my social studies classes have often helped me connect the problems humans faced in the past to problems humans face today. • The content and activities of my social studies classes often help me to understand more about my own culture and history. Student 61 marked Somewhat Agree to the first of these, and Somewhat Disagree to the second, while Student 127 marked “Strongly Agree” to both. In spite of reporting that his class does not stimulate connecting past to present, and not commenting on whether the class helps him connect content to his identity, Student 127 reports experiencing those connections. 79 Recommended Shifts. The students differ in their level of engagement with the prompt and the degree to which they address substantive recommendations. Student 61 does not engage fully with this prompt, only writing “blah blah.” Student 127 offers two recommendations and a rationale. First, he says social studies classes should be “faster paced,” and then he says they should be “more challenging.” He explains that these shifts would make social studies “more useful and [serve] the purpose of social studies more.” The students’ responses to this item may relate to their engagement scale scores. Like the CCPM, the adapted MCA-SEI scale for curriculum engagement has a score range of -1.00 to 1.00. The average score for all students is 0.14 (SD = 0.41). Student 127 scores 0.35 on the curriculum engagement scale, .5 standard deviations above the mean. His lowest item responses on that scale are Somewhat Disagree in response to, “The tests or big assignments in my social studies classes often do a good job of measuring what I’m able to do,” and Strongly Disagree in response to, “The activities I do in my social studies classes have often been challenging.” The latter item overlaps with his recommendation for “more challenging” social studies coursework. On the other items, Student 127 marks Somewhat Agree or Strongly Agree. By comparison, Student 61’s curriculum engagement scale score is -0.10, .6 standard deviations below the overall mean. On items extracted for analysis of Student 127’s responses, Student 61 makes different evaluations: and he marks Somewhat Agree to both. Most of the other items he marks Somewhat Disagree in response to, with no Strongly Agree or Strongly Disagree marks whatsoever. All of Student 61’s Strongly Disagree marks, which could hold clues for what recommendations he would offer upon further prompts, come from the CCPM in response to these four items: 80 • The content and activities of my social studies classes have often mentioned perspectives, individuals, or events I can relate to. • The content and activities of my social studies classes often make me more hopeful about the future. • The content and activities of my social studies classes often help me feel more connected to the peers in my social studies class. • The content and activities of my social studies classes often help me feel like I understand myself better. To Grow in Awareness or Become a Better Person in General Of all disaggregated groups, self-reported gender pronoun groups surface the only relationship approaching statistical significance among responses coded as To grow in awareness or become a better person in general (N = 26). Students selecting Masculine pronouns have the lowest CCPM scores, while students selecting Non-binary pronouns have the highest. In the qualitative analysis that follows, race and achievement intersections with gender pronoun within this school purpose category are used to select two sets of participant responses to compare. A Black heritage student with a score of 0.66 and White-only students have the highest CCPM scores of all racial groups. “B” level students have the highest CCPM scores among achievement groups, while “A” level students have the lowest. The one Indigenous heritage student (CCPM score of 0.09) and the one Latin American heritage student (CCPM score of -0.44) who are in this coding category have the lowest CCPM scores among racial groups. 81 Searches for students belonging to the highest and lowest scoring identity intersections are the basis for further analysis. A search for Non-binary pronoun students of Black or White-only heritage reporting at the “B” or “A” achievement level and scoring above the overall CCPM average score (M = .23) was conducted. There was one positive result: a White-only student achieving at the “B” level (Student 170) with a score of 0.66. Another search was made for male pronoun students of Indigenous or Latin American heritage reporting at the “A” achievement level and scoring below the overall CCPM average. There was one positive result: a Latin American heritage student (Student 73) with a score of -0.44. School Purpose. The students’ responses differ in length and focus, but share one theme. In their 18 words, Student 170 posits three purposes for school: a) “to learn [a] good work ethic,” b) to “be surrounded by good people,” and c) “to learn other types of cultures.” In his 35 words, Student 73 differentiated both what school’s purpose should be and what it should not. It should be to “focus on what happened to other nationalities.” It should not be to “just focus on US history” and “old rich white guys.” Student 73’s evaluation of the purpose of school overlaps significantly with the third purpose Student 170 gave. The students’ written responses are explored for their relation to the marks they gave on a curriculum engagement item: “Most of what is important to know you learn in school.” Student 170 marks Somewhat Disagree, while Student 73 marks Somewhat Agree. The purpose of school and the importance of the learning that occurs at school appear to possibly have less overlap for Student 170 than they do for Student 73. 82 Social Studies Purpose. Again, the students differ in length and focus, with Student 170 emphasizing connections with peers and Student 73 emphasizing how individuals can impact groups. With their five words, Student 170 reports that the purpose of social studies is “to connect with each other.” Given the context of the assessment’s administration, it is reasonable to assume “each other” means classmates. Student 73 reports a different focus: social studies should be “about how someones actions affected different nationalities.” He goes on to critique his current social studies class, stating, “i have 0 connection with the stuff we learn.” Each of the students’ written responses relate to different scale-response items. Student 170’s purpose of social studies relates to, “The content and activities of my social studies classes often help me feel more connected to the peers in my social studies class,” while Student 73’s critique relates to, “The content and activities of my social studies classes have often mentioned perspectives, individuals, or events I can relate to.” On the former, the students respond on opposite ends of the continuum with Student 170 marking Strongly Agree (1.4 standard deviations above the mean of 0.13) and Student 73 marking Strongly Disagree (1.8 standard deviations below the mean). On the latter, Student 170 marks Strongly Agree (1.3 standard deviations above the mean of 0.18) and Student 73 marks Somewhat Disagree (1.1 standard deviations below the mean). Their marks on these items appear to tightly overlap with their written responses about the purpose of social studies. Recommended Shifts. Length and focus again differ on this prompt, with Student 170 focusing on peer interactions and Student 73 calling out a specific curricular topic to end use of. In their 32 words, Student 170 states, “don split the group of kids up 83 [because] we have all grown to know each other very well and enjoy being with each other.” This response is reminiscent of Student 170’s aforementioned strong agreement with the statement that “The content and activities of my social studies classes often help me feel more connected to the peers in my social studies class.” Also, recalling that Student 170 identifies with non-binary pronouns, another research gap is indicated: critical perceptions of social safety in school for non-binary pronoun students and other institutionally marginalized sociodemographic groups. Student 73 calls for an end to emphatic teaching about John Adams, the second president of the United States. This is reminiscent of his earlier statements about school purpose that school “should not… just focus on US history” and “old rich white guys.” It also echoes his somewhat disagreement with the statement, “The content and activities of my social studies classes have often mentioned perspectives, individuals, or events I can relate to.” This indicates, again, the gap of critical perception of social safety in school research, and points to curriculum’s relation to school social safety (e.g., perceiving one’s heritage or interests/values to be reflected in the curriculum) as an area to investigate. It also raises questions about how much his experience of social studies curriculum reflected his Latinx identity, and what resources/training teachers need to implement more Latinx focused curricula. 84 CHAPTER V DISCUSSION This study provides evidence that CBP alignment perceptions strongly associate with curriculum engagement and achievement levels. Also, that many CBP curriculum qualities are experienced differently across categories of identity and levels of engagement and achievement. Directionality and magnitude of relationships among constructs remains unknown, however. The utility of students’ curriculum perception in school system decision making can possibly be inferred, but this study does not provide robust evidence for what student outcomes would gain, if anything, from focusing on raising curriculum perceptions compared to other interventions. Beyond suggesting the likelihood of more positive achievement and engagement levels, in general, being associated with more positive CBP-centered perceptions, this study does not indicate what a CBP-centered curriculum is or how it would relate to other student outcomes. Such topics are appropriate for future research. Findings from the present study offer compelling evidence to further investigate student curriculum perceptions and CBP as an inquiry framework. To explore the significance and utility of these findings, the discussion that follows has five sections: (a) validity threats, (b) findings summary, (c) curriculum inquiry implications, (d) policy and practice implications, and (e) future research implications. Threats to Validity Threats to Internal Validity 85 Five quantitative and two qualitative internal validity threats are addressed. Quantitative threats addressed are (a) instrumentation; (b) selection; (c) unobserved confounding variables; (d) Type I errors; and, (e) Type II errors (Campbell & Stanley, 1963; Bracht & Glass, 1968; Lewis-Beck, 1993). Qualitative threats addressed are (f) voluptuous legitimation, and, (g) confirmation bias (Onwuegbuzie & Leech, 2006). Fitting with the scope of this exploratory mixed methods study, a discussion of the preceding threats follows. Instrumentation. Three threats can be addressed: the content validity of the measures, bias in the instruments, and, bias in interpreting directionality. Content Validity of the Measures. Evidence was generated to suggest that scale measures (CCPM, CSSES, MCA-SEI) are tenably valid, but levels of achievement and socioeconomic status were not measured rigorously, and therefore may not be represented accurately in the analysis. Also, racial and pronoun identities were not compared to any records allowing them to be confirmed, which is a potential threat to their validity. Bias in the Instruments. The English-only assessment with no built-in option for audio to be played aloud or dictation of responses may have presented threats to the accuracy of information gathered. In particular, the assessment may have presented challenges for emerging multilingual students and students with certain disabilities. It is possible that students responded to the survey differently than in another context, such as a casual conversation, an interview, a class discussion, a “clicker quiz,” a “kahoot game,” or another event that was not as formal as silent computerized survey administration. To minimize this risk, the CCPM could be administered through a wider variety of media 86 and activities, which could potentially reduce participant burden and enhance validity. Comparing assessment outcomes across administration methodology could be an understudied field in education research capable of yielding more culturally and ability responsive data collection methods. Bias in Interpreting Directionality. In the Methods section, I revealed bias about the directionality between constructs of interest when I stated, “it seems reasonable to hypothesize that a curriculum perceived to be empowering and emancipating would also be perceived as preparatory for making contributions to the community.” However, achievement and/or engagement levels may drive curriculum perceptions. More construct validity evidence about the measures is necessary in order to properly assess their utility for informing directionality. Selection. Participating students are not representative of the district, much less the state or nation. The students chose to take a survey over an alternative activity, and were cohorted in classes with teachers who had volunteered access. Lower-achieving students are clearly under-represented in the sample, thus limiting the sample’s representation of variability in each of the class cohorts. A larger sample that included a higher proportion of each class cohort’s members would limit the risk of this threat. Another consideration is to compare a random selection of higher achieving students with the lower achieving students. Unobserved Confounding Variables. Parent/guardian attitudes toward school, students’ perceptions of their social studies teacher’s teaching efficacy, or nutrition levels are among the many variables that could have affected how students responded (or whether they chose to participate). Furthermore, evidence from previous research 87 suggests that the civic engagement qualities, such as the presence and nature of an intended civic engagement outcome, of a curriculum may better predict critical curriculum perception, achievement, and engagement levels.5 Type I Errors. Each instance of the null hypothesis’s rejection introduces the possibility of a Type I error. It was rejected: • when evaluating the significance of the difference between Higher and Lower achievement levels (p < 0.10); • when evaluating the significance of the difference between engagement levels (p < 0.0001); • and, when evaluating the significance of the difference between Lower and Medium socioeconomic levels (p < 0.01). Sampling idiosyncrasies could have led to erroneous rejections of the null hypothesis. However, the small sample size, simplicity of statistical methods, and results that largely match what previous research would have predicted (e.g., that students with more favorable perceptions of the curriculum would also have higher levels of achievement and engagement), seem to indicate that erroneous rejection is unlikely. The most under- supported rejection relates to differences in socioeconomic levels. The SES measure is research informed, but not validated. If valid, the differences in CCPM scores across socioeconomic levels appear to indicate that Middle level students in the sample experience a unique level of disenfranchisement from the curriculum. More research on this topic is recommended. Perhaps socioeconomic level is related to students’ perceptions about their social mobility. 5 See, for instance, Cammarota and Fine (2008), Celio et al. (2011), Delgado (2015), Levinson (2012), Lockeman and Pelco (2013), Reynolds (2014), Tuck and Yang (2013), Van Goethem et al. (2014), and Westheimer (2015). 88 Type II Errors. Each instance of failing to reject the null hypothesis introduces the possibility of a Type II error. It was rejected: • when evaluating the significance of the difference between racial categories (p > 0.20); • and, when evaluating the significance of the difference between gender pronoun categories (p > 0.50). Insufficient statistical power resulting from a small sample and small group sizes for comparisons may have lowered the signal-to-noise ratio such that not all significant relationships extant in the broader population were identified. A larger sample, a sample inclusive of a higher proportion of class cohort members, and using factor analysis to evaluate the utility of sub-scales within the CCPM could yield further evidence for whether a Type II error may have occurred in regards to race and gender pronouns. Voluptuous Legitimation. In the Results, renderings of quantitative data complemented each set of qualitative comparisons were that, to me, seemed appropriate. A common heuristic could have framed all complementations, but they were rendered ad hoc. It is possible that these interpretations of what to choose for complementation were overly-selective or overly-reductive, or otherwise misinformed. However, the concurrence of qualitative and quantitative data that not only appear to hold the possibility of complementation, but actually deliver on that possibility in virtually every instance, generates concurrent criterion-related validity. Confirmation Bias. Likewise to voluptuous legitimation, it is possible that the seeming confirmation of complementation produced by comparing qualitative and quantitative data in this study is a result of my bias. By focusing on extreme scores (e.g., 89 by heuristically selecting students with low and high CCPM scores), my intent was to surface qualitative data with explanatory power for quantitative patterns. This also introduced bias, as I was essentially fishing for qualitative data I had predicted to be likely to surface differences that could tie to the theoretical framework. Qualitative comparisons of larger numbers of students, at the individual or group level, could result in fewer instances of qualitative data appearing to confirm quantitative data. Continued qualitative analysis of this study’s dataset would narrow the risk of this threat and potentially generate additional concurrent criterion-related validity. Threats to External Validity Five quantitative research external validity threats are discussed: a) lack of representativeness of available and target populations; b) Hawthorne effect; c) sensitization/reactivity to research conditions; d) invalidity or unreliability of instruments; and, e) ecological validity (Campbell & Stanley, 1963; Bracht & Glass, 1968; Hammersley & Atkinson, 1983; Vulliamy, 1990; Lewis-Beck, 1993; Onwuegbuzie & Johnson, 2006b). Two qualitative research external validity threats are also discussed: f) evaluative validity, and, g) order bias (Onwuegbuzie & Johnson, 2006b). A Lack of Representativeness of Available and Target Populations. To be generalizable enough to inform implications for broader policy and practice in the district, the sample would need to be representative of a wider variety of students, as well as additional classes and schools. For informing policy and practice in the state, the sample would need to represent multiple districts in different settings. Random selection could help identify multiple nations (including U.S. Tribal lands), states and districts to constitute a representative sample for informing public education more generally. 90 Hawthorne Effect. It is possible that students responded to the survey differently than they would have if the prompts had come up in a way that masked the collection of data for analysis. For instance, inconspicuous audio recording of class or small group discussions related to the prompts could have yielded different, possibly more valid, information. Also, observations of how much time students spent with class media without ostensibly directing their attention elsewhere and the quality and content of work submitted in response could also inform researchers’ understanding of student experience and engagement with curriculum without producing the Hawthorne effect. Sensitization/Reactivity to Research Conditions. Related to Order Bias (discussed below), there may be priming effects within the environments where data collection occurred or how data collection occurred that limit the validity of the data. For instance, being indoors or away from natural light, or, being in a classroom where a student was accustomed to experience academic success or struggle, could have created confounding variables. Administering the CCPM in more general public, charter, alternative, private, Bureau of Indian Affairs, youth detention, home, and online learning environments could yield evidence for further examining this threat. Invalidity or Unreliability of Instruments. More research needs to be conducted on the CCPM, adapted CSSES, and adapted MCA-SEI scales to evaluate their validity and reliability. Factor loading and further administration of the measures could generate validity and reliability evidence. Likewise, more data sources (e.g., school forms or family interviews) could validate the means of placing students in sociodemographic categories for the sake of analysis. 91 Ecological Validity. Dee and Penner’s (2017) study provides a possible frame for evaluating ecological validity. Student perceptions about the ethnic studies curriculum they experienced in that study were not directly collected, but can be somewhat inferred by the achievement, credit-earning, and attendance outcomes associated with having experienced it. Embedding the CCPM in ethnic studies curriculum research similar to Dee and Penner’s (2017) could inform how curricular shifts translate to perception shifts, such that directionality and causality of variables—and the roles of other variables—can be better understood. Evaluative Validity. The findings of this study could be used to inform decision- making about curriculum content and instructional methods. It remains to be tested whether doing so would lead to increased levels of critical curriculum perception, achievement, and/or engagement. Such testing would generate evidence for evaluating the risk of this validity threat. Order Bias. Cognitive priming can occur through the succession in which schemata are stimulated (Schacter & Buckner, 1998; Tulving & Schachter, 1990). It is possible that by presenting the CCPM first, the adapted CSSES second, and the adapted MCA/SEI third, priming occurred that affected outcomes. The punctuated open-response prompts, their order, and the order of items on the scale measures may have also affected outcomes. Administering the assessment items in different orders could generate evidence for evaluating the risk of this validity threat. Summary of Findings Given the evidence suggesting the tenable validity of the CCPM, adapted CSSES, and adapted MCA/SEI measures, a summary of key findings follows: 92 Among participating students, perceptions of alignment between social studies curriculum and the empowerment and emancipation goals of critical bicultural pedagogy (Darder, 1991, 2012) differ in relation to their levels of achievement, levels of engagement, and sociodemographic constructs, both at the item and scale level. Race and gender pronoun differences are pronounced on some items; and, achievement, engagement, and socioeconomic status differences are evident across the scale. Overall, surveyed students report essentially no empowerment, emancipation, or engagement from their social studies curriculum. Critical bicultural pedagogy (Darder, 1991, 2012), as operationalized through the Critical Curriculum Perception Measure (Khatter, 2018), offers a prism for bridging gaps between students’ identities and the curriculum. The CCPM appears to be capable of capturing salient and measurable qualities of humanities curriculum that relate to students’ diverse curriculum experiences. Where compared, quantitative trends concur with qualitative data, strengthening the validity of the findings, which support the notion that efforts to align curriculum with CBP would be linked to enhanced engagement and achievement. CCPM data could inform policy and instructional practice aimed at enhancing curriculum. Additional opportunities for inquiry surfaced by this study are mentioned in the Areas for Future Research section. Implications for Curriculum Inquiry One implication is that a significant relationship exists between curriculum perception and curriculum engagement. In other words, students at different levels of curriculum engagement appear to have different curriculum experiences. Seemingly, the CCPM may be capable of capturing evidence of student curriculum experience. 93 Second, in regards to some curriculum perceptions, race, gender pronouns, achievement level, and socioeconomic level may play significant roles in how curriculum is experienced. What does equity look like in regards to student curriculum experience? Researching curricular qualities that promote inclusion, safety, and anti-discrimination for diverse groups of students could lead to better strategies for addressing school responses to bullying, White nationalism, and other threats to maintaining a positive school climate. critical bicultural pedagogy and other culturally centered pedagogies offer potential tools for identifying and leveraging those qualities. Third, students may be generally experiencing social studies curriculum unaligned with the goals of critical bicultural pedagogy. The CCPM and MCA-SEI means of 0.23 and 0.13, respectively, do not evince the notion that students generally experience connection or engagement with their social studies curricula, much less empowered and emancipated by them. There is more to unpack from the MCA-SEI results in future research, but on one item relevant to this implication, the mean score of 0.15 (SD = 0.68) is sobering: “The content and activities of my social studies classes have often been interesting to me.” Implications for Policy and Practice The links between curriculum perception and (a) curriculum engagement and (b) curriculum achievement raise the possibility of enhanced engagement and enhanced achievement resulting from student curriculum perception data. Operationalized, student curriculum perception data could be used to drive decisions about curriculum and instruction. Decision-making rubrics could be informed by or reliant upon student perceptions. Without direct data from students, the extent of empirical knowledge about 94 their experience of curriculum is quite narrow. Such data could inform (or drive) shifts in curriculum maps, text selection, and thematic or objectives-based alignment at the school, district, regional consortium (e.g., education service districts in Oregon), or a state-level department. Reviewing aggregated quantitative data, triangulated with salient qualitative features, such as those surfaced through interview and follow-up interview data from diverse student groups and teachers at different points in their curriculum experience, could potentially be useful at virtually any organizational level in a school system. Each of the CCPM items of special interest noted in Chapter IV are thought to have capacity for informing policy and practice. As previously mentioned, items 2, 11, and 15 are closely associated with cultural relevance qualities of the curriculum, and items 3, 6, and 9 with civic engagement qualities of the curriculum. These items are addressed as follows: Cultural Relevance Items Item 2: The content and activities of my social studies classes have often mentioned perspectives, individuals, or events I can relate to. Despite the importance of providing relatable windows and mirrors (Style, 1988) that culturally centered pedagogies have emphasized for decades, the mean student response on this item is lower than the CCPM average. Among racial groups, Latinx heritage and White-only students have particularly low scores. This particular constellation of data prompts the questions, “Where and how are Latinx and White identities being represented in the curriculum?” and, “What training and resources do teachers need in order to represent empowering and emancipating Latinx and White identities?” Teachers, of whom 86% are classified White in this district (ODE, 2019), may not have access to or training in social studies 95 curriculum that significantly features people of Latinx and White heritage who have engaged or are engaging in liberative transformation. For instance, Whites could largely be represented as aggressors or Latinos as victims. In such instances, it is reasonable to predict students could find those groups and individuals less relatable than Whites who participated in antebellum justice movements or Latino responses to property theft following the Mexican-American War. Focusing social studies curriculum on groups and individuals across diverse races, ethnicities, and other sociodemographics involved in justice, liberation, and social transformation would align with CBP and could have the capacity to enhance student perceptions of this item. Item 11: The content and activities of my social studies classes often help me feel like I understand myself better. My discussion points for Item 11 are largely consistent with my discussion for Item 2. It bears repeating that Item 11 is the lowest- scoring item of the CCPM. The low scores seem to indicate that social studies teachers need access to materials and training for making curriculum more student-centered. Item 15: The content and activities of my social studies classes often help me feel that I belong in this country. In her review of the CCPM, Antonia Darder (personal communication, August 10, 2018) offered this as a supplementary item in order for the CCPM to better target bicultural aspects of critical pedagogy. Indigenous heritage students have the highest CCPM mean for this item, likely reflecting their connection to pre-colonial American geography. Lower scores among other groups may indicate something about the extent to which students feel a sense of belonging in their school and community. School climate issues of student belonging are central to the recently implemented Every Student Belongs Oregon Department of Education administrative rule 96 (ODE, 2021a). Previously cited research on social studies curriculum, civic engagement curriculum in particular, and culturally centered theory suggests that social studies classrooms should promote students’ experience of personally relevant connections with academic content and skills. In other words, social studies curriculum should nurture students’ sense of belonging. Through the lens of CBP, social studies curriculum should facilitate students’ understanding of their political agency and of their capacity to channel that agency in ways that would promote their sense of belonging to the school and community. Justice-oriented civic engagement curriculum, detailed below, appears relevant for addressing low perception levels for this quality. Civic Engagement Items Item 3: The content and activities of my social studies classes have often helped me think about how I can try to make the world a better place. Civic engagement orientation quality of the curriculum, which this item attempts to approximate, is a core interest of culturally centered pedagogies. Perceived relevance of the curriculum has been linked to its civic engagement qualities (see Unobserved Confounding Variables earlier in this chapter). Policy could support a literature-informed practice of developing vertically (and horizontally) articulated district curriculum guides with civic engagement projects specified for social studies classes. Broadening the civic engagement opportunities that public schools provide may be more important now than it has ever been. In the present era of unfolding climate change (McKibben, 2011), resource scarcity (Meadows, Randers, & Meadows, 2004), exacerbation of social disparities (Hite & Seitz, 2021; Milanovic, 2016; Sernau, 2013), and epochal extinctions (Kolbert, 2014), scholars note that widespread and sustained 97 civic engagement is necessary to build community resilience, or perhaps even to ensure human survival (Diamond, 2005). Furthermore, aligning curriculum with critical civic issues could enhance its relevance and rigor such that student retention and engagement are higher. Westheimer and Kahne (2004)’s typology for civic engagement orientation offers three categories: personally responsibility, participatory, and justice-oriented. Curricula capable of being categorized as justice-orientated require students to “critically assess social, political, and economic structures and consider collective strategies for change that challenge injustice and, when possible, address root causes of problems” (Westheimer & Kahne, 2004, p. 242). These skills overlap with many qualities of critical bicultural pedagogy. Piloting and implementing curriculum matching the justice orientation would be a reasonable strategy for addressing student retention and engagement. Items 6 and 9. My discussion points for Item 6, “The content and activities of my social studies classes often make me more hopeful about the future,” and Item 9, “The content and activities of my social studies classes often help me see how, by working together with others, I can make the world a better place,” are mostly consistent with my discussion of Item 3. Perhaps meaningful civic engagement curricula could help address student perceptions of this curriculum quality as well. Areas for Future Research Most of the areas for future research touched on by this study fall into one of three categories: (a) student curriculum perception; (b) student evaluation of curriculum; and, (c) inclusivity and responsiveness of curriculum implementation methods. 98 Future Research for Student Curriculum Perception Questions are raised by the findings about how students connect or disconnect from curriculum in regards to elements of their identity. While disengagement and disconnect from the curriculum was widespread, more inquiry is merited about how and why students experience curriculum differently as it relates to their self-identity. Further psychometric research for theory-aligned measures of student curriculum perception and curriculum engagement would likely yield information helpful for retaining and engaging students. Additional administration of the CCPM and MCA-SEI scales could be informative. Psychometrically sound measures of student perception could promote testing the utility of decisions made about the scope, sequence, and qualities of curriculum in social studies and other areas. One example could be conducting research similar to Dee and Penner’s (2017) that also utilizes the CCPM or a measure like it. Such a study could inform the directionality of construct associations, e.g., whether engagement drives perceptions or perceptions drive engagement. Bigger picture, longitudinal associations of retention and engagement with curricula perceived to be aligned with CBP would serve to inform wider policy on a larger scale. Student interviews, teacher observations, and material samples are among the data that could be sampled to address aspects of how and why students experience curriculum differently and the qualities of positively perceived curricula. In turn, such research could inform policy priorities and future research into best practices, especially in regards to cultural relevance and identity consonance of curriculum. Future Research for Student Evaluation of Instructional Materials Related to the content of the previous heading, additional research could also 99 address the efficacy of allowing student perceptions or student-generated rubrics to make decisions about instructional materials or curricular scope and sequences. For instance, site and district curriculum decisions could use disaggregated CCPM data to target shifts for better providing culturally resonant content to all students. Or, the CCPM could be used as an evaluation tool. Scores from the CCPM or another measure applied to material sets could be compared to student engagement and achievement, or attendance and on- time graduation rate. Such a project could be framed by Youth Participatory Action Research (Cammarota & Fine, 2008) and the justice-orientation of civic engagement curriculum described earlier (Westheimer & Kahne, 2004). Future Research for Inclusivity and Responsiveness of Curriculum Implementation Methods Many of the findings of this study directly support Darder’s (1991, 2012) claims about students’ experience of disconnect from school and curriculum, as mentioned in the Theoretical Framework of Chapter III. More research is needed to understand how to address bias in curriculum and its implementation. Additional research could test the capacity of curriculum and curriculum implementation methodologies to be inclusive of and responsive to students’ contexts and identities. As Sabzalian (2019a) stated, “For Indigenous youth, civic engagement and action are often connected to their cultural identities, as well as their political identities as citizens of Indigenous nations.” Citing Castro and Knowles (2017), Sabzalian went on to identify Indigenous students’ civic engagement as political action expressive of their Indigeneity as a critical under-studied area for further research. For example, the sacredness of heritage culture and sovereignty, and the concepts of Indigenous citizenship, rights, and nationhood, are completely absent 100 in most civic engagement, citizenship, and character-building curricula (CNAY, 2018; Haynes Writer, 2002; Rains, 2002, 2003; Rains & Swisher, 1999; Sabzalian & Shear, 2018; Sabzalian, 2019b; Westheimer, 2014; Westheimer & Kahne, 2004). Research addressing these gaps could inform how youth identity is studied and understood for curriculum inquiry and evaluation. Testing materials and methods for implementing civic engagement curriculum could be especially important for the sake of providing access to empowerment and emancipation for all students. 101 APPENDIX A CCPM EXPERT REVIEW BY ANTONIA DARDER Original e-mail with timestamp: 102 APPENDIX B ASSESSMENT BATTERY OF SURVEY ITEMS ADMINISTERED TO STUDENTS Items presented in the order students encounter them: Original Measure Original Version Adapted Version Open What do you think the purpose of response, school should be? Why? - researcher generated CCPM The content and activities of my social studies classes have often helped me - connect the problems humans faced in the past to problems humans face today. CCPM The content and activities of my social studies classes have often mentioned - perspectives, individuals, or events I could relate to. CCPM The content and activities of my social studies classes have often helped me - think about how I can try to make the world a better place. CCPM The content and activities of my social studies classes have often helped me - understand why some people have a lot of money and influence and why most people do not. CCPM The content and activities of my social studies classes have often been - presented in a way similar to how language is used in my home. CCPM The content and activities of my social studies classes have often made me - more hopeful about the future. CCPM The content and activities of my social studies classes have often brought up - how those with power often misuse it, and how those with less power can respond to misuses of power. 103 CCPM The curriculum content and activities of my social studies classes have often - helped me reflect on the world and my place in it. CCPM The curriculum for the last unit in my social studies class helped me see how, - by working together with others, I can make the world a better place. CCPM The content and activities of my social studies classes have often helped me - feel more connected to the peers in my social studies class. CCPM The content and activities of my social studies classes have often helped me - feel like I understand myself better. CCPM The content and activities of my social studies classes have often helped me - become more thoughtful and wise. CCPM The content and activities of my social studies classes have often helped me to - understand more about my own culture and history. CCPM The content and activities of my social studies classes have often helped me to - understand the effects of inequalities in our society. CCPM The content and activities of my social studies classes have often helped me - feel that I belong in this country. CCPM The content and activities of my social studies classes have often helped me to - feel good about my culture and language. CSSES I am confident that, through community The content and activities of my social service, I can help in promoting social studies classes often help me feel more justice. confident that, by working with others, I can help make the world more fair and just. 104 CSSES I am confident that, through community The content and activities of my social service, I can make a difference in my studies classes often help me feel more community. confident that, by working with others, I can make a positive difference in my community. CSSES Through community service, I can The content and activities of my social apply knowledge in ways that solve studies classes often help me feel more “real-life” problems confident that, by working with others, I can use knowledge to solve real-life problems. Open What do you think the purpose of social response, studies classes should be? How well - researcher does this class line up with what you generated think the purpose should be? Explain. SEI Most of what is important to know you - learn in school. SEI The tests in my classes do a good job of The tests or big assignments in my measuring what I’m able to do. social studies classes often do a good job of measuring what I’m able to do. MCA What I learn in my class is interesting The content and activities of my social to me. studies classes have often been interesting to me. MCA What I do in my class is interesting. What I do in my social studies classes has often been interesting to me. MCA The activities I do in my class are The activities I do in my social studies challenging. classes have often been challenging. MCA I am challenged to do my best in class. I was challenged to do my best during the last unit of my social studies class. MCA I can choose my own projects. On the major assignment(s) or project(s) of my social studies classes, I have often been able to choose what to focus on. MCA I can choose materials to work with in In my social studies classes, I have the class. often been able to choose materials (readings, photos, etc.) to work with. MCA I like what I do in my class. I often like what I do during my social studies classes. MCA The activities I do in my class are The activities I have done in my social 105 enjoyable. studies classes have often been enjoyable. Open How would you recommend the last response, unit in your social studies class be researcher changed? Why? Do any of the items - generated you just rated about your last social studies curriculum unit help you identify areas that should be changed? Explain. 106 REFERENCES CITED Abubakar, A., Abubakar, Y., & Itse, J. B. (2017). Students’ engagement in relationship to academic performance. Journal of Education and Social Sciences, 8(1), 5-9. Agodini, R., Harris, B., Thomas, M., Murphy, R., & Gallagher, L. (2010). Achievement effects of four early elementary school math curricula: Findings for first and second graders. NCEE 2011-4001. National Center for Education Evaluation and Regional Assistance. Aidinopoulou, V., & Sampson, D. G. (2017). An action research study from implementing the flipped classroom model in primary school history teaching and learning. Journal of Educational Technology & Society, 20(1), 237-247. Alazzi, K., & Chiodo, J. J. (2004). Students’ perceptions of social studies: A study of middle school and high school students in Jordan. International Journal of Scholarly Academic Intellectual Diversity, 8(1), 3-13. Allensworth, E. M., & Easton, J. Q. (2007). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Consortium on Chicago School Research. Alexander, K. L., Entwisle, D. R., & Horsey, C. S. (1997). From first grade forward: Early foundations of high school dropout. Sociology of Education, 70, 87-107. Aoki, T. (1990a). Themes of teaching curriculum. In J. T. Sears, & J. D. Marshall (Eds.) Teaching and thinking about curriculum: Critical inquiries, (pp. 111-114). Teachers College Press. Aoki, T. (1990b). Sonare and videre: Questioning the primacy of the eye in curriculum talk. In George Willis & William H. Schubert (Eds.), Reflections From the heart of educational inquiry: Understanding curriculum and teaching through the arts, (pp. 182-189). SUNY Press. Appleton, J. J., & Christenson, S. L. (2004). Scale description and references for the Student Engagement Instrument. Unpublished manuscript. Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument. Journal of School Psychology, 44(5), 427-445. Babbie, E. R. (2021). The practice of social research. Cengage. Bailie, C., & Wiseman, A. (2018). Improving equity and driving degree completion through acceleration in mathematics. E3 Alliance. 107 Balfanz, R., Herzog, L., & Mac Iver, D. J. (2007). Preventing student disengagement and keeping students on the graduation path in urban middle-grades schools: Early identification and effective interventions. Educational Psychologist, 42(4), 223-235. Balfanz, R., Fox, J. H., Bridgeland, J. M., & McNaught, M. (2009). Grad nation: A guidebook to help communities tackle the dropout crisis. America's Promise Alliance. Bando, R., Näslund-Hadley, E., & Gertler, P. (2019). Effect of inquiry and problem based pedagogy on learning: Evidence from 10 field experiments in four countries (No. w26280). National Bureau of Economic Research. Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191. Banks, J. A. (1981). Multiethnic education: Theory and practice. Allyn & Bacon. Banks, J. A. (1997). Teaching strategies for ethnic studies. Allyn & Bacon. Beel, J., Gipp, B., & Wilde, E. (2009). Academic search engine optimization: Optimizing scholarly literature for Google Scholar & co. Journal of Scholarly Publishing, 41(2), 176-190. Bellamy, L. (2005). A Critical Analysis of How Differentiation Can Promote the Full Inclusion of Three Gifted and Talented Students in a Mixed-Ability, Year 9 Class Studying" Macbeth". English Teaching: Practice and Critique, 4(2), 72-83. Bennett, M. J. (1986). A developmental approach to training for intercultural sensitivity. International Journal of Intercultural Relations, 10(2), 179-196. Behrend, A. (2013). Self-perceptions of gifted achievers and underachievers: A phenomenological study [Unpublished doctoral dissertation]. Liberty University. Berman, N., & White, A. (2013). Refusing the stereotype: Decoding negative gender imagery through a school-based digital media literacy program. Youth Studies Australia, 32(4), 38-47. Betensky, R. A. (2019). The p-value requires context, not a threshold. The American Statistician, 73(1), 115-117. Betts, J. E., Appleton, J. J., Reschly, A. L., Christenson, S. L., & Huebner, E. S. (2010). A study of the factorial invariance of the Student Engagement Instrument (SEI): Results from middle and high school students. School Psychology Quarterly, 25(2), 84. 108 Billig, S., Root, S., & Jesse, D. (2005). The impact of participation in service-learning on high school students' civic engagement. College Park, MD: The Centre for Information & Research on Civic Learning and Engagement, University of Maryland. Bracht, G. H., & Glass, G. V. (1968). The external validity of experiments. American educational research journal, 5(4), 437-474. Bringle, R. G., Phillips, M. A., & Hudson, M. (2004). The measure of service learning. American Psychological Association. Bureau of Labor Statistics. (2019, October 21). Median weekly earnings $606 for high school dropouts, $1,559 for advanced degree holders. U.S. Bureau of Labor Statistics. Retrieved from https://www.bls.gov/opub/ted/2019/median-weekly-earnings-606-for-high- school-dropouts-1559-for-advanced-degree-holders.htm Burris, C. C., & Welner, K. G. (2005). Closing the achievement gap by detracking. Phi Delta Kappan, 86(8), 594-598. Cahill, M. J., McDaniel, M. A., Frey, R. F., Hynes, K. M., Repice, M., Zhao, J., & Trousil, R. (2018). Understanding the relationship between student attitudes and student learning. Physical Review Physics Education Research, 14(1), 010107. Cammarota, J. (2007). A social justice approach to achievement: Guiding Latina/o students toward educational attainment with a challenging, socially relevant curriculum. Equity & Excellence in Education, 40 (1), 87-96. Cammarota, J., & Fine, M. (2008). Youth participatory action research. Revolutionizing education: Youth participatory action research in motion, 1-12. Routledge. Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Handbook of research on teaching, 171-246. Rand McNally. Carr, P. R. (2010). Democracy and critical pedagogy: Does your vote count. Peter Lang. Carter, D.J. (2008). Cultivating a critical race consciousness for African-American school success. Educational Foundations, 22 (1-2), 11-28. Carter, P.L. & Welner, K. (2013). Closing the opportunity gap: What America must do to give every child an even chance. Oxford University Press. Castellano, M. (2004). The Effect of CTE-Enhanced Whole-School Reform on Student Coursetaking and Performance in English and Science. National Research Center for Career and Technical Education. Catsambis, S. (1994). The path to math: Gender and racial-ethnic differences in mathematics participation from middle school to high school. Sociology of Education, 67, 199-215. 109 Castro, A. J., & Knowles, R. T. (2017). Democratic citizenship education: Research across multiple landscapes and contexts. In M. Manfra & C. M. Bolick (Eds.), The Wiley handbook of social studies research (pp. 287–318). Wiley-Blackwell. Celio, C. I., Durlak, J., & Dymnicki, A. (2011). A meta-analysis of the impact of service-learning on students. Journal of Experiential Education, 34(2), 164-181. Chan, W. Y., Ou, S. R., & Reynolds, A. J. (2014). Adolescent civic engagement and adult outcomes: An examination among urban racial minorities. Journal of youth and adolescence, 43(11), 1829-1843. Chen, C. H., & Yang, Y. C. (2019). Revisiting the effects of project-based learning on students’ academic achievement: A meta-analysis investigating moderators. Educational Research Review, 26, 71-81. Chingos, M. M., & Whitehurst, G. (2011). Class size: What research says and what it means for state policy. Brookings Institute. Chiodo, J. J., & Byford, J. (2004). Do they really dislike social studies? A study of middle school and high school students. Journal of Social Studies Research, 28(1), 16-26. CNAY. (2018). Generation Indigenous: The state of Native youth 2018. Center for Native American Youth. Cohen, L., Manion, L., & Morrison, K. (2011). Surveys, longitudinal, cross-sectional and trend studies. Research methods in education, 261-4. Routledge. Cohen, L., Manion, L., & Morrison, K. (2017). Research methods in education. Routledge. Cornelius-White, J. H. D. & Harbaugh, A.P. (2010). Learner-centered instruction: Building relationships for student success. Sage. Creghan, C., & Adair-Creghan, K. (2015). The positive impact of project-based learning on attendance of an economically disadvantaged student population: A multiyear study. Interdisciplinary Journal of Problem-Based Learning, 9(2), Article 7. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Prentice Hall. Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage. Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches. Sage. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. 110 Cross, T. L. (1989). Towards a culturally competent system of care: A monograph on effective services for minority children who are severely emotionally disturbed. Georgetown University Child Development Center. Dale, A. (2006). Quality issues with survey research. International Journal of Social Research Methodology, 9(2), 143-158. Darder, A. (1991). Culture and power in the classroom: A critical foundation for bicultural education. Greenwood. Darder, A. (2012). Culture and power in the classroom: Educational foundations for the schooling of bicultural students. Routledge. Dart, J., & Davies, R. (2003). A dialogical, story-based evaluation tool: The most significant change technique. American Journal of Evaluation, 24(2), 137-155. Dee, T. S., & Penner, E. K. (2017). The causal effects of cultural relevance: Evidence from an ethnic studies curriculum. American Educational Research Journal, 54(1), 127-166 Delgado, M. (2015). Community practice and urban youth: Social justice service- learning and civic engagement. Routledge. Denscombe, M. (2009). Item non‐response rates: A comparison of online and paper questionnaires. International Journal of Social Research Methodology, 12(4), 281-291. Diamond, J. (2005). Collapse: How societies choose to fail or succeed. Penguin. Doolittle, A., & Faul, A. C. (2013). Civic engagement scale: A validation study. Sage Open, 3(3), 2158244013495542. Dunbar, L. (2020). Remedying educational racism? Studying ethnic studies (Order No. 28000612). Available from Dissertations & Theses @ University of Oregon. (2436390572). Retrieved from http://libproxy.uoregon.edu/login?url=https://www-proquest- com.libproxy.uoregon.edu/dissertations-theses/remedying-educational-racism- studying-ethnic/docview/2436390572/se-2?accountid=14698 Durm, M. 1993. An A is not an A is not an A: A history of grading. The Education Forum 57, 294-97. Durrant, G. B. (2009). Imputation methods for handling item‐nonresponse in practice: methodological issues and recent debates. International Journal of Social Research Methodology, 12(4), 293-304. Ebe, A. E. (2010). Culturally relevant texts and reading assessment for English language learners. Reading Horizons, 50(3), 193-210. 111 Eccles, J., Wigfield, A., Harold, R. D., & Blumenfeld, P. (1993). Age and gender differences in children's self‐and task perceptions during elementary school. Child development, 64(3), 830-847. Ehrlich, T. (Ed.). (2000). Civic responsibility and higher education. Greenwood. El-Amin, A., Seider, S., Graves, D., Tamerat, J., Clark, S., Soutter, M., ... & Malhotra, S. (2017). Critical consciousness: A key to student achievement. Phi Delta Kappan, 98(5), 18-23. Epstein, T. (1998). Deconstructing differences in African‐American and European- American adolescents' perspectives on US history. Curriculum Inquiry, 28(4), 397-423. Epstein, T. (2000). Adolescents' perspectives on racial diversity in US history: Case studies from an urban classroom. American Educational Research Journal, 37(1), 185-214. ERIC. (2021). How does the ERIC search work? ERIC Institute of Education Sciences. Retrieved from https://eric.ed.gov/?advanced Ertz, M., Karakas, F., & Sarigöllü, E. (2016). Exploring pro-environmental behaviors of consumers: An analysis of contextual factors, attitude, and behaviors. Journal of Business Research, 69(10), 3971-3980. Faul, A. C., & Hudson, W. W. (1997). The index of drug involvement: A partial validation. Social Work, 42, 565-572. Fitchett, P. G., & Vanfossen, P. J. (2013). Survey on the status of social studies: development and analysis. Social Studies Research & Practice , 8(1). Ford, D. Y., & Harris III, J. J. (1996). Perceptions and attitudes of Black students toward school, achievement, and other educational variables. Child Development, 67(3), 1141-1152. Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59-109. Freire, P. (1970). Pedagogy of the oppressed. Seabury Press. Gay, G. (2010). Culturally responsive teaching: Theory, research, and practice. Teachers College Press. Gentry, M., & Gable, R. K. (2001). From the student's perspective—My Class Activities: An instrument for use in research and evaluation. Journal for the Education of the Gifted, 24(4), 322-343. 112 Gentry, M., Rizza, M. G., & Gable, R. K. (2001). Gifted students' perceptions of their class activities: Differences among rural, urban, and suburban student attitudes. Gifted Child Quarterly, 45(2), 115-129. Glenn, D. I., & Taylor, C. L. (1990). Can response order bias evaluations? Evaluation and Program Planning, 13(4), 365–371. Goossen, K., Hess, S., Lunny, C., & Pieper, D. (2020). Database combinations to retrieve systematic reviews in overviews of reviews: a methodological study. BMC medical research methodology, 20(1), 138. Grice, M. O., & Vaughn, C. (1992). Third graders respond to literature for and about Afro-Americans. The Urban Review, 24(2), 149-164. Goodman, Y., Watson, D., & Burke, C. (2005). Reading Miscue Inventory. Katonah, NY: Richard C. Owen Publishers, Inc. Governale, J. (1997). Improving attitudes of students toward social studies. Available from Saint Xavier University. Retrieved from https://eric.ed.gov/?id=ED424173 Green, M. (1967). Existential encounters for teachers. Random House. Green, M. (1973). Teacher as stranger: Educational philosophy for the modern age. Wadsworth. Gwon, S. H., and Lee, C. Y. (2015). Factors that influence sexual intercourse among middle school students: using data from the 8th (2012) Korea Youth Risk Behavior Web-based Survey. Journal of Korean Academy of Nursing, 45(1). Hair, J.F. Jr, Black, W.C., Babin, B.J., & Anderson, R.E. (2010). Multivariate data analysis: A global perspective. Prentice-Hall. Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural sensitivity: The intercultural development inventory. International Journal of Intercultural Relations, 27(4), 421-443. Hammersley, M., & Atkinson, P. (1983). Ethnography: Principles in practice. Tavistock. Hanley, J. (1999). Beyond the tip of the iceberg: Reaching today’s youth. The Community Circle of Caring Journal, 3(2), 9-12. Hansberry, L. E. A. (2000). Year 9 student attitudes toward social studies in a western Australian government high school: A Case Study [Unpublished master’s thesis]. Edith Cowan University. Harari, M. B., Parola, H. R., Hartwell, C. J., & Riegelman, A. (2020). Literature searches in systematic reviews and meta-analyses: A review, evaluation, and recommendations. Journal of Vocational Behavior, 118, 103377. 113 Haynes Writer, J. (2002). Terrorism in Native America: Interrogating the past, examining the present, and constructing a liberatory future. Anthropology & Education Quarterly, 33(3), 317-330. Heo, J., Oh, J., Subramanian, S. V., & Kawachi, I. (2014). Household and school-level influences on smoking behavior among Korean adolescents: a multilevel analysis. PLoS One, 9(6). Hite, K. A., & Seitz, J. L. (2021). Global issues: an introduction. John Wiley & Sons. Hoffer, T. B. (1992). Middle school ability grouping and student achievement in science and mathematics. Educational evaluation and policy analysis, 14(3), 205-227. Horak, A. K., & Galluzzo, G. R. (2017). Gifted middle school students’ achievement and perceptions of science classroom quality during problem-based learning. Journal of Advanced Academics, 28(1), 28-50. Huang, F. L., & Cornell, D. G. (2017). Student attitudes and behaviors as explanations for the Black-White suspension gap. Children and Youth Services Review, 73, 298-308. Hynes, M. (2014). Don't call them dropouts: Understanding the experiences of young people who leave high school before graduation. America's Promise Alliance. Illich, I. (1971). Deschooling society. Harper & Row. Johnson, N. K. (2016). Hidden among privilege: Increasing belonging for low SES students at affluent schools through participatory action research. ProQuest Dissertations & Theses A&I; ProQuest Dissertations & Theses Global. (1812534999). Retrieved from https://search.proquest.com/docview/1812534999?accountid=14512 Kelly, A. V. (2009). The curriculum: Theory and practice. Sage. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. Kelly, S., & Abruzzo, E. (2021). Using Lesson-Specific Teacher Reports of Student Engagement to Investigate Innovations in Curriculum and Instruction. Educational Researcher, 0013189X20982255. Kenny, M. E., Blustein, D. L., Haase, R. F., Jackson, J., & Perry, J. C. (2006). Setting the stage: Career development and the student engagement process. Journal of Counseling Psychology, 53(2), 272. Khatter, J. S. (2018). Critical Curriculum Perception Measure. Department of Educational Methodology, Policy, and Leadership, University of Oregon. 114 Knackmuhs, E., Farmer, J., & Reynolds, H. L. (2017). Student outcomes of eco-restoration service-learning experiences in urban woodlands. Journal of Experiential Education, 40(1), 24-38. Knight, S. L., & Wiseman, D. L. (2005). Professional development for teachers of diverse students: A summary of the research. Journal of Education for Students Placed at Risk, 10(4), 387-405. Kohl, H. (1967). Teaching the “Unteachable.” The story of an experiment in children’s writing. New York: New York Review of Books. Kohl, H. (1969). The open classroom. New York: A New York Review Book. Kolbert, E. (2014). The sixth extinction: An unnatural history. A&C Black. Kozol, J. (1967). Death at an early age. New York: Houghton Mifflin. Kozol, J. (1972). Free schools. Boston: Houghton Mifflin. Kpolovie, P. J., Joe, A. I., & Okoto, T. (2014). Academic achievement prediction: Role of interest in learning and attitude towards school. International Journal of Humanities Social Sciences and Education (IJHSSE), 1(11), 73-100. Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American Educational Research Journal, 32(3), 465-491. Ladson-Billings, G. (2001). Crossing over to Canaan: The journey of new teachers in diverse classrooms. John Wiley & Sons. Ladson-Billings, G. (2014). Culturally relevant pedagogy 2.0: aka the remix. Harvard Educational Review, 84(1), 74-84. Landa, C. (2011). Cultural proficiency in education: A review of the literature focused on teachers, school leaders, and schools. University of Massachusetts: Gastón Institute for Latino Community Development and Public Policy Publications. Lawrence, S. A., Johnson, T., Baptiste, M., Caleb, A., Sieunarine, C., & Similien, C. (2017). Pre-service teachers’ use of multicultural literature. Journal of Inquiry and Action in Education, 9(1), 3. Lebrecht, S., Bar, M., Barrett, L. F., & Tarr, M. J. (2012). Micro-valences: perceiving affective valence in everyday objects. Frontiers in psychology, 3, 107. Lei, H., Cui, Y., & Zhou, W. (2018). Relationships between student engagement and academic achievement: A meta-analysis. Social Behavior and Personality: an international journal, 46(3), 517-528. Levinson, M. (2012). No citizen left behind. Cambridge, MA: Harvard University Press. 115 Lindsey, R. B., Roberts, K. N., & Campbell-Jones, F. (2005). The culturally proficient school. Corwin. Lindsey, R. B., Robins, K. N., & Terrell, R. D. (2003). Cultural proficiency: A manual for school leaders. Cowin. Lovelace, M. D., Reschly, A. L., Appleton, J. J., & Lutz, M. E. (2014). Concurrent and predictive validity of the student engagement instrument. Journal of Psychoeducational Assessment, 32(6), 509-520. Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2(4), 34-46. Lewis-Beck, M. S. (1993). Experimental design and methods. Sage. Lockeman, K., & Pelco, L. (2013, November 8). The impact of service-learning on degree completion. International Association for Research on Service-Learning and Community Engagement Annual Conference, Omaha, NE. Retrieved from https://scholarscompass.vcu.edu/community_resources/23/ MacCallum R.C., Widaman K.F., Zhang S. and Hong S. (1999) Sample size in factor analysis. Psychological Methods 4(1), 84-99. MacCallum R.C., Widaman K.F., Preacher K.J. and Hong S. (2001) Sample size in factor analysis: The role of model error. Multivariate Behavioral Research 36, 611-637. Manning, S. J. L. (2005). An examination of curricula modifications employed by classroom teachers for intellectually gifted students in the general education classroom. Available from the University of Southern Mississippi. (3217209). Retrieved from https://search.proquest.com/openview/91a1f20d157c8fe6bed84727a06b609b/1?p q-origsite=gscholar&cbl=18750&diss=y Martell, C. C. (2013). Race and histories: Examining culturally relevant teaching in the US history classroom. Theory & Research in Social Education, 41(1), 65-88. Martin, A. J., & Martin, A. (2003). How to motivate your child for school and beyond. Bantam. Martin, F., & Bolliger, D. U. (2018). Engagement matters: Student perceptions on the importance of engagement strategies in the online learning environment. Online Learning, 22(1), 205-222. Mason J. L. (1993). Culturally competence self-assessment questionnaire. Portland State University Multicultural Initiative Project. 116 Mata, M. D. L., Monteiro, V., & Peixoto, F. (2012). Attitudes towards mathematics: Effects of individual, motivational, and social support factors. Child Development Research, 2012. Mathison, S., Ross, E. W., & Vinson, K. D. (2001). Defining the social studies curriculum: The influence of and resistance to curriculum standards and testing in social studies. The social studies curriculum: Purposes, problems, and possibilities, 87-102. McKibben, B. (2011). Eaarth: Making a life on a tough new planet. Vintage Books. Meadows, D., Randers, J., & Meadows, D. (2004). Limits to growth: The 30-year update. Chelsea Green. Mikami, A. Y., Ruzek, E. A., Hafen, C. A., Gregory, A., & Allen, J. P. (2017). Perceptions of relatedness with classroom peers promote adolescents’ behavioral engagement and achievement in secondary school. Journal of Youth and Adolescence, 46(11), 2341-2354. Milanovic, B. (2016). Global inequality: A new approach for the age of globalization. Harvard University Press. Milo, G. (2017). Rebooting social studies: Strategies for reimagining history classes. Rowman & Littlefield. NCSS. (2010). National curriculum standards for social studies: A framework for teaching, learning, and assessment. National Council of the Social Studies. Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment & evaluation in higher education, 33(3), 301-314. Oakes, J. (1987). Tracking in secondary schools: A contextual perspective. Educational Psychologist, 22(2), 129-153. Oakes, J., & Guiton, G. (1995). Matchmaking: The dynamics of high school tracking decisions. American Educational Research Journal, 32(1), 3-33. ODE. (2019). Oregon at-a-glance school profiles. Oregon Department of Education. Retrieved from https://www.oregon.gov/ode/schools-and- districts/reportcards/reportcards/Pages/default.aspx ODE. (2021a). Every Student Belongs: Initial guidance. Oregon Department of Education. Retrieved from https://www.oregon.gov/ode/students-and- family/equity/SchoolSafety/Documents/Every%20Student%20Belongs%20Initial %20Guidance.pdf 117 ODE. (2021b). 2021 Oregon Social Science Standards integrated with Ethnic Studies Standards. Oregon Department of Education. Retrieved from https://drive.google.com/drive/folders/1AbsRuQEd4Y4ax7ZMDNYBtEigx41sue W5 Onwuegbuzie, A. J., & Leech, N. L. (2006a). Linking research questions to mixed methods data analysis procedures. Qualitative Report, 11(3), 474-498. Onwuegbuzie, A. J., & Johnson, R. B. (2006b). The validity issue in mixed research. Research in the Schools, 13(1), 48-63. Orthner, D. K. (2007). Public schools: Building capacity for hope and opportunity. In J. Edwards, M. Crain, & A. L. Kalleberg (Eds.). Ending poverty: How to restore the American dream (pp. 218–229). New Press. Orthner, D. K., Akos, P., Rose, R., Jones-Sanpei, H., Mercado, M., & Woolley, M. E. (2010). CareerStart: A middle school student engagement and academic achievement program. Children & Schools, 32(4), 223-234. Orthner, D. K., Jones-Sanpei, H., Akos, P., & Rose, R. A. (2013). Improving middle school student engagement through career-relevant instruction in the core curriculum. The Journal of Educational Research, 106(1), 27-38. Nielsen, J. & Kjaer, T. (2011). Does question order influence sensitivity to scope? Empirical findings from a web-based contingent valuation survey. Journal of Environmental Planning & Management 54(3), 369–381 (2011) OECD. (2017). PISA 2015 Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving, revised edition, PISA, OECD Publishing, Paris. http://dx.doi.org/10.1787/9789264281820-en OECD. (2020). Early Learning and Child Well-being in the United States. Paris: OECD Publishing. https://doi.org/10.1787/198d8c99-en Onwuegbuzie, A. J., & Leech, N. L. (2006). Validity and qualitative research: An oxymoron?. Quality & Quantity, 41(2), 233-249. Paris, D. (2012). Culturally sustaining pedagogy: A needed change in stance, terminology, and practice. Educational researcher, 41(3), 93-97. Paris, D., & Alim, H. S. (Eds.). (2017). Culturally sustaining pedagogies: Teaching and learning for justice in a changing world. Teachers College Press. Pereira, N., Peters, S. J., & Gentry, M. (2010). My class activities instrument as used in Saturday enrichment program evaluation. Journal of Advanced Academics, 21(4), 568-593. 118 Plank, S. (2001). A question of balance: CTE, academic courses, high school persistence, and student achievement. Journal of Vocational Education Research, 26(3), 279-327. Preckel, F., Götz, T., & Frenzel, A. (2010). Ability grouping of gifted students: Effects on academic self‐concept and boredom. British Journal of Educational Psychology, 80(3), 451-472. Priniski, S. J., Hecht, C. A., & Harackiewicz, J. M. (2018). Making learning personally meaningful: A new framework for relevance research. The Journal of Experimental Education, 86(1), 11-29. Ramirez, M., & Castañeda, A. (1974). Cultural democracy: Bicognitive development and education. New York: Academic Press. Rains, F. (2002). From the eyes of the colonized: Rethinking the legacy of colonization and its impact on American Indians. Journal of Philosophy and History of Education, 52, 125-130. Rains, F. V. (2003). To greet the dawn with open eyes: American Indians, White privilege and the power of residual guilt in social studies. In G. Ladson-Billings (Ed.), Critical race theory perspectives on social studies: The profession, policies, and curriculum (pp. 199–227). Information Age. Rains, F. V., & Swisher, K. G. (1999). Authentic voices: Advice for incorporating American Indians and Alaska Natives in the elementary school curriculum. Social Education, 63(1), 46–50. Reeb, R. N. (2006). The community service self-efficacy scale: Further evidence of reliability and validity. Journal of prevention & intervention in the community, 32(1-2), 97-113. Reeb, R. N., Katsuyama, R. M., Sammon, J. A., & Yoder, D. S. (1998). The Community Service Self-Efficacy Scale: Evidence of Reliability, Construct Validity, and Pragmatic Utility. Michigan Journal of Community Service Learning, 5, 48-57. Reynolds, N. P. (2014). What Counts as Outcomes? Community Perspectives of an Engineering Partnership. Michigan Journal of Community Service Learning, 21(1), 79-90. Ringwalt, C., Pankratz, M., Gottfredson, N., Jackson-Newsom, J., Dusenbury, L., Giles, S., Curry, D., & Hansen, B. (2009). The effects of students' curriculum engagement, attitudes toward their teachers, and perception of their teachers' skills on school-based prevention curriculum outcomes. Journal of Drug Education, 39(3), 223-237. 119 Rogers, J. & Terriquez, V. (2013). Learning to lead: The impact of youth organizing on the educational and civic trajectories of low-income youth. Los Angeles, CA: Institute for Democracy, Education, and Access. Ross, E. W. (2001). Social studies teachers and curriculum. The social studies curriculum: Purposes, problems, and possibilities, 3-15. Sabzalian, L. (2019a). The tensions between Indigenous sovereignty and multicultural citizenship education: Toward an anticolonial approach to civic education. Theory & Research in Social Education, 47(3), 311-346. Sabzalian, L. (2019b). Indigenous children’s survivance in public schools. Routledge. Sabzalian, L., & Shear, S. (2018). Confronting colonial blindness in civics education: Recognizing colonization, self-determination, and sovereignty as core knowledge for elementary social studies teacher education. In S. B. Shear, C. M. Tschida, E. Bellows, L. B. Buchanan, & E. E. Saylor (Eds.), (Re)Imagining elementary social studies: A controversial issues reader (pp. 153–176). Information Age. Schacter, D. L., & Buckner, R. L. (1998). Priming and the brain. Neuron, 20(2), 185-195. Schug, M. C. (1982, November). Why kids don't like social studies [Paper presentation]. National Council of the Social Studies Annual Conference, Boston, MA. https://files.eric.ed.gov/fulltext/ED224765.pdf Sernau, S.R. (2013). Global problems: The search for equity, peace, and sustainability. Pearson. Sharot, T., & Garrett, N. (2016). Forming beliefs: Why valence matters. Trends in cognitive sciences, 20(1), 25-33. Soteriades, E. S., & DiFranza, J. R. (2003). Parent’s socioeconomic status, adolescents’ disposable income, and adolescents’ smoking status in Massachusetts. American Journal of Public Health, 93(7), 1155-1160. St. Mary, J., Calhoun, M., Tejada, J., & Jenson, J. M. (2018). Perceptions of academic achievement and educational opportunities among Black and African American youth. Child and Adolescent Social Work Journal, 35(5), 499-509. Stevenson, B., & Wolfers, J. (2008). Happiness inequality in the United States. The Journal of Legal Studies, 37(S2), S33-S79. Stewart, T., & Bai, H. (2010). Community service self-efficacy and summer service-learning: Comparative analyses among academically talented youth. Gifted Education International, 27(2), 149-160. Style, E. (1988). Curriculum as window and mirror. In Listening for all voices. Oak Knoll School. Retrieved from https://nationalseedproject.org/Key-SEED- Texts/curriculum-as-window-and-mirror 120 Suarez-Orozco, C. (2008). Transnational identities in our globalized societies. In Stories of identity: religion, migration, and belonging in a changing world. Facing History and Ourselves. Suarez-Orozco, M. M., & Sattin, C. (2007). Learning in the global era. In M. M. Suarez- Orozco (Ed.), Learning in the global era: International perspectives on globalization and education. University of California Press. Sue, D. W., Lin, A. I., Torino, G. C., Capodilupo, C. M., & Rivera, D. P. (2009). Racial microaggressions and difficult dialogues on race in the classroom. Cultural Diversity and Ethnic Minority Psychology, 15(2), 183-190. Sussmuth, R. (2007). On the need for teaching intercultural skills. In M. M. Suarez- Orozco (Ed.), Learning in the global era: International perspectives on globalization and education. University of California Press. Tabachnick, B.G., & Fidell, L.S. (2007). Using multivariate statistics. Pearson. Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Sage. Tintiangco-Cubales, A., Kohli, R., Sacramento, J., Henning, N., Agarwal-Rangnath, R., & Sleeter, C. (2015). Toward an ethnic studies pedagogy: Implications for K-12 schools from the research. The Urban Review, 47(1), 104-125. Tuck, E., & Yang, K. W. (Eds.). (2013). Youth resistance research and theories of change. Routledge. Tulving, E., & Schacter, D. L. (1990). Priming and human memory systems. Science, 247(4940), 301-306. Useem, E. L. (1992). Middle schools and math groups: Parents' involvement in children's placement. Sociology of education, 263-279. Van Eck, K., Johnson, S. R., Bettencourt, A., & Johnson, S. L. (2017). How school climate relates to chronic absence: A multi–level latent profile analysis. Journal of School Psychology, 61, 89-102. Van Goethem, A., Van Hoof, A., Orobio de Castro, B., Van Aken, M., & Hart, D. (2014). The role of reflection in the effects of community service on adolescent development: A meta‐analysis. Child development, 85(6), 2114-2130. van Rooij, E. C., Jansen, E. P., & van de Grift, W. J. (2017). Secondary school students' engagement profiles and their relationship with academic adjustment and achievement in university. Learning and Individual Differences, 54, 9-19. 121 Villa, R. A., Thousand, J. S., Nevin, A., & Liston, A. (2005). Successful inclusive practices in middle and secondary schools. American Secondary Education, 33(3), 33-50. Voight, A., & Velez, V. (2018). Youth participatory action research in the high school curriculum: Education outcomes for student participants in a district-wide initiative. Journal of Research on Educational Effectiveness, 11(3), 433-451. Vulliamy, G. (1990). The potential of qualitative educational research strategies in developing countries. Doing educational research in developing countries: Qualitative strategies, 7-25. Wang, M. T., & Eccles, J. S. (2012). Adolescent behavioral, emotional, and cognitive engagement trajectories in school and their differential relations to educational success. Journal of Research on Adolescence, 22(1), 31-39. Wanzek, J., Kent, S. C., & Stillman-Spisak, S. J. (2015). Student Perceptions of Instruction in Middle and Secondary US History Classes. Theory & Research in Social Education, 43(4), 469-498. Webster, J. W. (2001). Effects of ninth graders' culture-specific schemata on responses to multicultural literature. The Journal of Educational Research, 95(1), 12-25. Westheimer, J., & Kahne, J. (2004). What kind of citizen? The politics of educating for democracy. American Educational Research Journal, 41(2), 237-269. Westheimer, J. (2015). What kind of citizen? Educating our children for the common good. Teachers College Press. Woolley, M. E., & Bowen, G. L. (2007). In the context of risk: Supportive adults and the school engagement of middle school students. Family Relations, 56(1), 92-104. Yang, Y., Gentry, M., & Choi, Y. O. (2012). Gifted students’ perceptions of the regular classes and pull-out programs in South Korea. Journal of Advanced Academics, 23(3), 270-287. Yildirim, I. (2017). The effects of gamification-based teaching practices on student achievement and students' attitudes toward lessons. The Internet and Higher Education, 33, 86-92. 122