IS CS FOR ALL LEARNERS? INVESTIGATING THE INTERSECTION OF ENGLISH LEARNER AND COMPUTER SCIENCE INSTRUCTION by ANDREA PARTSAFAS A DISSERTATION Presented to the Department of Educational Methodology, Policy and Leadership and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Education June 2019 DISSERTATION APPROVAL PAGE Student: Andrea Partsafas Title: Is CS for All Learners? Investigating the Intersection of English Learner and Computer Science Instruction This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Education degree in the Department of Educational Methodology, Policy and Leadership by: Kathleen Scalise Chairperson Joanna Smith Advisor Gerald Tindal Core Member Joanna Goode Institutional Representative and Janet Woodruff-Borden Vice Provost and Dean of the Graduate School Original approval signatures are on file with the University of Oregon Graduate School. Degree awarded June 2019 ii © 2019 Andrea Partsafas This work is licensed under a Creative Commons Attribution-Noncommercial-NoDerivs (United States) License. iii DISSERTATION ABSTRACT Andrea Partsafas Doctor of Education Department of Educational Methodology, Policy and Leadership May 2019 Title: Is CS for All Learners? Investigating the Intersection of English Learner and Computer Science Instruction There is a consensus that today’s students in the U.S. are not prepared to fill the almost 1 million unfilled computing jobs in 2024. In the wake of the Computer Science For All initiative in 2016, advocacy groups, such as the Association for Computing Machinery, Code.org and the Computer Science Teachers Association, actively encourage diversity in computer science classes. However, there are large equity and opportunity gaps across the nation. Less than 20% of students taking the AP Computer Science exam are females, and even fewer are from traditionally underrepresented groups such as African American and/or Latinx. Fewer than half of K-12 schools in the United States offer computer science courses that would meet the K-12 Computer Science Framework standards definition. English Learners (EL), the second largest subgroup in K-12 education, is noticeably absent from the CS diversity conversation. The College Board does not collect EL data on AP test takers and discrepancies in defining CS make it difficult to collect universally comparable data around enrollment and achievement. However, more data have been collected on the efficacy of EL instructional strategies in some other technical subjects, notably science. Using data from both EL and iv CS research, instructional strategies can be employed in CS classrooms for maximum leverage. In this project, I employ a sequential explanatory mixed methods research design to address this gap in the research by surveying and interviewing experts in both computer science and English learner instruction. Quantitative survey data were collected using the Computer Science: Best Practices in Instruction for ELLs instrument along with open-ended questions. Qualitative data were collected through interviews of Computer Science and English Language teachers. The results of this study can inform state officials about the importance of implementing and supporting instructional strategies in CS courses and curriculum to ensure equitable instructional practices for all students, especially English Learners. v CURRICULUM VITAE NAME OF AUTHOR: Andrea Partsafas GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene St. Mary’s College of California Southern Oregon University DEGREES AWARDED: Doctor of Education, 2019, University of Oregon Master’s Degree in Education, 2005, St. Mary’s College of California Bachelor of Arts, 1996, St. Mary’s College of California AREAS OF SPECIAL INTEREST: English Learners Equity STEM PROFESSIONAL EXPERIENCE: Consultant, WriteWise Consulting, 2008-Present Technology Integration Coach, The Valley School Charter, 2015-2018 Adjunct Faculty, University of Phoenix Online, 2009-2016 Adjunct Faculty (Education) , Southern Oregon University, 2006-2013 Teacher (ELD, K-5), Jackson County District #9, 2010-2012 Teacher (ELD, 6-8), Jackson County District #9, 2005-2010 Teacher (ELD, 6-8) San Lorenzo Unified, 1998-2004 PUBLICATIONS: Scalise, K., Irvin, P. S., Alresheed, F., Zvoch, K., Yim-Dockery, H., Park, S., ... & Partsafas, A. (2018). Accommodations in Digital Interactive STEM Assessment Tasks: Current Accommodations and Promising Practices for vi Enhancing Accessibility for Students with Disabilities. Journal of Special Education Technology, 0162643418759340. PRESENTATIONS Science for ELs- Strategies and Activities, OSTA Fall Conference, 2014 Inexpensive Technology to Increase Oral Fluency & Student Engagement, TESOL International Conference, 2011 On-Site Trainer for Write Traits, Writing Across the Curriculum and ACCESS, Houghton Mifflin Harcourt, 1999-2010 vii ACKNOWLEDGMENTS I wish to thank my committee members for their support and guidance on this project: Dr. Kathleen Scalise, Dr. Ilana Umansky, Dr. Joanna Goode, Dr. Jo Smith and Dr. Gerald Tindal. I honestly don’t know that I could have done this without the mentorship, support and kindness from all of you throughout the whole process. To my cohort- I truly love you all. How lucky am I to have had you all in my corner? I will miss our cohort dinners, text that made me laugh out loud, and cheers of success at every turn. I also wish to thank my husband and children. Aaron- your constant belief that I could do this, from encouraging me to apply to discussing ad nauseum my ten thousand drafts. I have nothing but gratitude for your calmness and kindness during this experience. My boys- thanks for washing extra dishes, feeding the dogs without being asked, and staying off the internet while I was in class. viii TABLE OF CONTENTS Chapter Page I.INTRODUCTION AND LITERATURE REVIEW ................................................. 1 Literature Review .................................................................................................. 3 A Brief Discussion of the State of Computer Science in the U.S. .................. 3 Computer Science in Oregon ........................................................................... 5 Existing EL Equity Gaps in Computer Science ............................................... 6 Instructional Strategy Literature Search and Review ...................................... 9 Search Procedures ...................................................................................... 11 Instructional Strategy Literature Search Results ............................................. 15 Theme 1: EL Instructional Strategies in Science ...................................... 15 Pedagogical EL Strategies in Science ................................................ 16 Domain-Specific EL Strategies ......................................................... 19 Theme 2: CS/CT Instructional Strategies .................................................. 23 Pedagogical CS/CT Strategies ........................................................... 24 Domain-Specific CS/CT Strategies ................................................... 27 Summary .................................................................................................... 31 Gaps in the Literature ................................................................................ 31 Theoretical Framework .......................................................................................... 32 Summary of Findings of the Literature Review .................................................... 36 II. METHODOLOGY ................................................................................................. 38 Methodological Overview ..................................................................................... 39 Research Design .................................................................................................... 40 ix Chapter Page Phase I: Survey ................................................................................................ 41 Sampling Logic and Participants ............................................................... 41 EL-T ..................................................................................................... 41 CS-T ..................................................................................................... 41 Phase II: Interviews ......................................................................................... 42 Sampling Logic and Participants ......................................................... 42 Data Collection and Analysis ................................................................................ 43 Phase I: Quantitative Survey Data ................................................................... 43 Description of the CS: BPIE ...................................................................... 43 Administration of the CS: BPIE ................................................................ 45 EL-T ..................................................................................................... 45 CS-T ..................................................................................................... 45 CS: BPIE Data Analysis ............................................................................ 46 Phase II: Qualitative Data ................................................................................ 47 Description of the Interview Protocol ....................................................... 47 Interview Administration ........................................................................... 48 Interview Analysis ..................................................................................... 48 Limitations to the Methodology ............................................................................ 49 Phase I Limitations .......................................................................................... 49 Internal Validity Threats ............................................................................ 49 Construct Validity Threats ......................................................................... 50 Phase II Limitations ....................................................................................... 52 x Chapter Page Internal Validity Threats ............................................................................ 52 External Validity Threats ........................................................................... 54 Qualitative Validity Threats ...................................................................... 55 III. RESULTS .............................................................................................................. 57 Phase 1: Survey Results ......................................................................................... 57 Descriptive Statistics of the Sample ................................................................ 57 Research Question 1a: Most Commonly Chosen Instructional Strategies ...... 60 Research Question 1b: Perceptions of Usefulness as Measured by Frequency of Use ....................................................................... 63 Research Question 2: How, When and Why Are Strategies Used/Recommended ....................................................................... 67 How Do Teachers Utilize Instructional Strategies? .................................. 68 When Do Teachers Utilize Instructional Strategies? ................................. 68 Why Do Teachers Utilize Instructional Strategies? .................................. 68 Phase 2: Interview Results ..................................................................................... 72 Descriptive Statistics and Protocol .................................................................. 72 Analysis of Interview Transcriptions .............................................................. 73 Recommendation Versus Frequency of Strategies .................................. 73 Role of Curriculum and Instructional Frameworks ................................... 77 IV. DISCUSSION AND IMPLICATIONS ................................................................. 79 Discussion of Results by Research Question ........................................................ 79 Research Question 1a (RQ 1a) ........................................................................ 79 Research Question 1b (RQ 1b) ....................................................................... 80 xi Chapter Page Research Question 2 ........................................................................................ 84 Summary of Discussion ................................................................................... 87 Limitations ............................................................................................................. 90 Contributions to the Literature ............................................................................. 90 Research Dissemination ........................................................................................ 92 Implications for Future Research ......................................................................... 92 Implications for Policy and Practice ...................................................................... 94 State ............................................................................................................. 95 District ............................................................................................................. 96 School ............................................................................................................. 97 Conclusion ............................................................................................................. 98 APPENDICES ............................................................................................................. 101 A. CS:BPIE SURVEY ........................................................................................... 101 B. EMAIL TO EL-T PARTICIPANTS ................................................................. 106 C. FOLLOW-UP EMAIL TO EL-T PARTICIPANTS ......................................... 107 D. EMAIL TO CS-T PARTICIPANTS ................................................................ 108 E. INTERVIEW PROTOCOL .............................................................................. 109 F. RESPONSES TO SURVEY QUESTIONS 5 AND 6 ....................................... 111 REFERENCES CITED ............................................................................................... 115 xii LIST OF FIGURES Figure Page 1. Mixed method sequential research design ............................................................. 39 2. Summary of qualitative data from Q5 and Q6 ...................................................... 67 xiii LIST OF TABLES Table Page 1. Digital Search Results and Articles Selected ........................................................ 14 2. EL Instructional Strategies Literature Synthesis Summary ................................... 21 3. Report of EL Instructional Strategies in Science from Literature Synthesis ......... 22 4. CS/CT Instructional Strategies Literature Synthesis Summary ............................ 29 5. Report of CS/CT Instructional Strategies from Literature Synthesis .................... 30 6. Constructivist Factors Mapped to EL and CS/CT Strategies ................................ 36 7. Phase I Descriptive Data ........................................................................................ 59 8. Q1 Summary of Most Recommended Strategy by Teacher Group ....................... 61 9. Q2 Summary of Most Recommended Strategy by Teacher Group ....................... 62 10. Q3 Summary of Most Recommended Strategy by Teacher Group ....................... 62 11. Chi-Square Distribution Significance by Question ............................................... 63 12. Strategy Use Frequency by Teacher Group with Chi-Square Significance ......... 65 13. EL Teacher Recommended Strategy v. Every Day Frequency ............................. 70 14. Computer Science Teacher Recommended Strategies v. Every Day Frequency 71 xiv CHAPTER I INTRODUCTION AND LITERATURE REVIEW Computers and an increase in the reliance on technology have changed the economy, education and even the very fabric of society (Barr & Stephenson, 2011; Guzdial, 2017; Guzdial & Morrison, 2016). More than 7.7 million Americans use computers in complex ways in their jobs—almost half of these jobs are in fields not directly related to science, technology, or math fields (Change the Equation, 2015). Over the last five years, money and time have been spent investigating how to integrate computers, including computer science, into K-12 education. A number of researchers and practitioners have focused K-12 computer science education on students who are traditionally underrepresented in both the computer science industry and high-school computer courses, such as girls, students with disabilities, and African-American and Hispanic/Latinx students. A poll taken by Google Inc. and Gallup (2016) illustrate these barriers, particularly for Hispanic students. In the poll, Hispanic students were less likely (31%) than White students (42%) to use a computer most days of the week. Hispanic students were more likely than any other subgroup of students to rate themselves low in computer skills, with more than half of the respondents indicating they were “not confident” in their computer science skills. However, Hispanic students had high interest in computer science at 35% compared to 21% of White students, and 92% of Hispanic parents wanted their children to learn CS, the highest of all the subgroups in the poll data. 1 One often noticeable omission in the “underrepresented” category is English Learners (ELs), or students who are identified as non-proficient in listening, speaking, reading and/or writing in English (U.S. Department of Education Office of English Language Acqusition, 2016). There is a dearth of data about ELs in computer science courses reported. Large-scale surveys, such as that by Google and Gallup (2016), reported data on several subgroups, i.e. female, Hispanic and Black students; however, they did not report data for ELs. Nor for instance did The College Board indicate if any of the 2016 AP Computer Science exams were taken by ELs. The College Board reported that students of color took 33, or 8%, of the exams in 2016, but reported no data on ELs (Code.org, 2014) and additionally no data on language subgroups within this population. While we might be able to extrapolate that low numbers of enrolled and successful Hispanic students in CS classes may have a relationship with low numbers of EL students from a Spanish-speaking background, we do not know how many EL students are enrolled in CS classes. Simply focusing on higher enrollment does not translate to high success rates. As time and resources continue to be put into K-12 computer science education, it behooves the educational community to put EL students’ needs at the forefront. Thoughtful consideration of instructional practices at this early stage of implementation are more likely to embed equity into the educational landscape than attempting a restructuring at a later date. This study seeks to explore which, if any of these research-based, effective instructional strategies for ELs that are utilized in content areas are relevant to computer science/computational thinking (CS/CT) courses. The research questions guiding the study are, for the study sample: 2 1a. Which instructional strategies do EL teachers (EL-T) and CS teachers (CS-T) believe are most likely to increase success for ELs in computer science classes? 1b. Are there differences between EL-T and CS-T perceptions of usefulness and frequency of use of instructional strategies? 2. How, when and why would EL-T and CS-T utilize or recommend particular strategies in computer science classes? Literature Review Next I will review the literature pertinent to the emerging needs of ELs in CS. First, I begin with an overview of the state of computer science education in the United States and then, more specifically, in the state of Oregon. Then, I focus on EL equity gaps in computer science. To help fill these gaps, I next take the approach of examining overlaps in research-based strategies for support of ELs in science education to identify some potentially impactful instructional approaches for ELs in CS/CT courses. Finally, I summarize my results. A Brief Discussion of the State of Computer Science Education in the U.S. In 2014, Gallup conducted a national survey of students, parents, teachers and principals about their definition of computer science. The majority of teachers and principals felt that “searching the internet” (60%/54%) and creating documents or presentations on the computer (75%/63%) were core computer science activities (Google, 2015). However, in the current K-12 Computer Science Framework widely deployed currently in the U.S., computer science is not defined as any of these things. Computer science is defined as: (a) knowing how and why computers work, (b) the study of computers, algorithmic processes, hardware and software designs, applications, and (c) 3 the societal impact of computer technology (The K-12 Computer Science Framework, 2016). Thus, computer science is not just using computers. Rather, it requires the application of computational thinking (CT), broadly defined as the conceptualizing of abstract thought through decomposition, logic-based problem solving skills and algorithmic thinking (ISTE & CSTA, 2011; Wing, 2006). In 2015, fewer than half of our nation’s schools were teaching meaningful computer science courses that applied these CT and problem-solving skills, such as coding, computer engineering, or computer science principles (Gallup, 2015). Focused on a growing trend in the economy and the need to introduce computer science before college, the United States Department of Education earmarked $4 billion for a national computer science initiative, Computer Science For All. This enterprise, first described in President Barack Obama’s final state of the Union address in 2016, was to empower “all American students from kindergarten through high school to learn computer science and be equipped with the computational thinking skills they need” (Smith, 2016, p.1). Later the same year, the U.S. K–12 Computer Science Framework was developed, led by the Association for Computing Machinery, Code.org, Computer Science Teachers Association, Cyber Innovation Center, and the National Math and Science Initiative in partnership with states and districts (K-12 Computer Science Framework, 2016). The K- 12 Computer Science Framework is not a curriculum guide, but rather a conceptual guideline to “inform the development for computer science standards and curriculum, build capacity for teaching computer science, and implement computer science 4 pathways” (p. 1). This high-level guide is intended to describe “a baseline [computer science] literacy for all students” and provide guidance in the area of implementation of practices and concepts (p. 15). The K-12 Computer Science Framework is not a set of standards. Rather, it is a foundation from which states or organizations can build their own standards, or choose already published standards, curriculum, computing tools and instructional materials. Two national organizations, the Computer Science Teachers Association (CSTA) and the International Society for Technology in Education (ISTE), have updated their computer science standards in alignment with the K-12 Computer Science Framework. The K-12 Computer Science Framework provides guidance for implementation of CS programs at the K-12 level. There are six principles guiding the document: (1) broaden participation in CS, (2) focus on the essentials, (3) do not reinvent the wheel, (4) inform with current research and guide future research, (5) align to nationally recognized frameworks, and (6) inspire broad implementation. Computer Science in Oregon In Oregon, there are currently more than 5,000 unfilled computer science-related jobs, while nationwide there are more than 500,000 current open computing jobs (Code.org, 2014; Kessler, 2017). Currently, Oregon is one of 18 states that has not adopted K-12 computer science standards and one of 21 states that does not have a clear pathway for teacher certification in computer science or defined computer science course definitions (“State tracking 9 policies (Public),” n.d.). The Oregon Teacher Standards and Practices Commission, the credentialing division of the Oregon Department of Education, does not offer a specific Computer 5 Science authorization (OAR 584-220-0010) nor other technology-related certification. Currently, any teacher with any secondary license can be approved to teach a computer science course. For example, to teach AP Computer Science A in Oregon, a teacher needs only two things: (a) a secondary teaching license in any subject area and (b) a recommendation from the school or district. In 2016, only 11% of Oregon schools with AP programs, or 25 schools total, offered an AP Computer Science course (Code.org, 2014). In the state of Oregon, it is difficult to assess how many CS courses that meet the K-12 Computer Science Framework guidelines are offered beyond Advanced Placement (AP) courses. It is not possible to draw objective conclusions about the quality and/or experience of the teachers in terms of computer science competency. Existing EL Equity Gaps in Computer Science Education Stereotypes, lack of opportunities, structural barriers and implicit bias contribute to the growing equity gap in science and technical coursework for ELs and language minority students (Callahan, Wilkinson, Muller, & Frisco, 2009; Chang & Kim, 2009; Kanno & Kangas, 2014; Margolis, Estrella, Goode, Jellison Holme, & Nao, 2001; Umansky, 2016). Opportunity gaps for ELs in general in K-12 schools are well-documented (Callahan et al., 2009; Kanno & Kangas, 2014; J. Margolis et al., 2008; Umansky, 2016). Callahan et al. (2009) found that opportunities to learn may “stem from a paucity of ESL classes and the limited availability of trained teachers and through scheduling constraints” (p. 375). For instance, in schools with low concentrations of EL students, ESL and sheltered classes had a statistically significant negative predictive effect on both 6 enrollment and achievement in math and science, finding that EL students were 70% less likely to enroll in Algebra II or Chemistry. Umansky’s (2016) analysis of statewide assessment data for enrollment and achievement in the areas of English proficiency, math and English Language arts (ELA) shows how proficiency levels corresponded with access to core classes. Based on EL proficiency levels, the results illustrated substantial tracking in middle school, including over-representation in lower track classes and under- representation in upper track classes. Consequently, fewer ELs take honors classes and grade-level ELA courses. Nationally, 78% of school districts have English Learner (EL) populations. Oregon is one of 12 states with more than 10 percent of its student population speaking more than one language (U.S. Department of Education, 2017) . In 2016-17, more than 60,000 students in Oregon were designated as English Learners (ELs). The only larger subgroup in Oregon is Latinx, at 23% percent, at least 68 percent of who are identified in both subgroups: EL and Latinx (Oregon Department of Education, 2017). Researchers find that exposure to computer science and algebra are statistically significant predictors of success in computer science courses (Grover, Pea, & Cooper, 2015; J. Margolis et al., 2008). English language proficiency is also a critical component of success in computational thinking, algebraic thinking and computing project outcomes (Pudyastuti, Palandi, & Kom, 2014). As the K-12 Computer Science Framework begins to influence policies in K-12 schools, there is an opportunity at this initial implementation point to incorporate research-based instructional practices into CS classrooms that promote equitable opportunities to learn for traditionally underserved students, including ELs. 7 Margolis et al.’s (2001) study in Los Angeles discovered that student enrollment in math classes often predicted student schedules, overriding both student interest and desire for their enrollment choices. Kanno and Kangas (2014) found their subjects routinely transitioned from sheltered courses to the remedial-level courses of the same subjects regardless of their performance in their class. This is attributed to the course sequencing set up by the faculty, not due to student preference. These studies illustrate how ELs are systematically excluded from core academic subject areas when the schedules were tightly focused around English-speaking students. There is also a lack of data published regarding successful strategies or programs for ELs, even overall and especially by language subgroup. This lack of data around enrollment or success rates for ELs in computer science leaves stakeholders to make assumptions (i.e., Latinx/Hispanic student data includes ELs) or to ignore the subgroup altogether. In a data-driven educational culture, the “way to increase student achievement levels is that school staff [base] their decisions on data” (Schildkamp & Kuiper, 2010, p. 483). Absence of data or lack of clarity contribute to exclusivity in CS/CT courses and curriculum, as the first step in a data-driven decision-making process is to collect data. According to Issacs (2003, p. 290), “Measurement in education is meant to communicate information about problems or their solutions, about making consistent decisions about student performance and learning, about what discrepant performance or problems might be.” When data are not collected, decision making does not always reflect the true issue. For example, Margolis et al. (2011) found that students of color and language minorities were infrequently asked their opinions around course offerings. They reported 8 that when AP courses were cut at one of the sample schools, a technology magnet school, the lack of access to the course was blamed largely on structural constraints (the absence of the right teacher, budget shortages, or testing priorities) and was justified by a perceived lack of interest or ability on the part of students. However, when asked, the students indicated that the teacher did not have the appropriate skills (i.e. the students knew more than the teacher) but that interest in computer science was still high. Students were disappointed that the course had been eliminated entirely, effectively removing computer science from the schedule entirely. This is one example of how equity gaps persist in schools where inadequate data collection exists. Even though the K-12 Computer Science framework does not address ELs directly, it does dedicate an entire chapter of the framework to Equity in Computer Science Education. It also embeds recommendations for equity throughout. For example, the first of seven core practices for computer science is “Fostering an inclusive computing culture” (p. 68). Also, Recommendation 4 of the framework’s Guidance for Standards Developers advocates, “diversity and equity be attended to by developing standards that allow for engagement by all students.” More specifically, it states, “Equitable standards are not biased for or against students from a particular background; this includes making standards accessible to students with special needs or English language learners” (p. 133). A focus on classroom instruction may have the potential to increase accessibility of CS classes for EL students. Instructional Strategy Literature Search and Review This study seeks to explore how to increase student achievement for ELs in computer science. To this end, I investigated the research-based, effective instructional 9 strategies for ELs that are utilized in specific content areas and looked for overlaps relevant to computer science/computational thinking (CS/CT) courses. There are well-documented effective EL strategies that have proven efficacy in all content areas. Several large-scale literature reviews and reports (August & Shanahan, 2017; Baker et al., 2014; Calderon, Slavin, & Sanchez, 2011; Goldenberg & Coleman, 2006) indicate that effective teaching for ELs is similar in many ways to effective teaching for English speakers; varied strategies, good relationships with students and high-quality instruction are required. Recommendations from Goldenberg and Coleman (2006) include taking cues from effective strategies for English speakers, ensuring instruction is reflective of teachers understanding and knowing their students. Research also indicates that while good practices are needed for all students, EL students need more specific instruction than generalized “good teaching”(Baker et al., 2014; Gersten & Baker, 2000; Goldenberg & Coleman, 2006; Moughamian, Rivera, & Francis, 2009; Short, Echevarría, & Richards-Tutor, 2011; Takanishi & Le Menestrel, 2017). Coleman and Goldenberg (2009) recommend teachers of EL students become familiar with research-based strategies, know when to use direct and interactive techniques, and implement instructional strategies that promote content-understanding and academic language development. For example, teachers should become more familiar with both Baker et al.’s (2014) practice guide for K-8 teachers and Takanishi and Le Menestrel’s (2017) report of seondary level promising practices. These reports indicate that good instruction for ELs includes: (a) teaching academic vocabulary words, (b) integrating oral and written English into content areas, (c) providing regular 10 structured opportunities to write in English, and (d) provide small group instructional interventions for struggling students. Student engagement is also a theme in the overall EL research (Genesee, Lindholm-Leary, Saunders, & Christian, 2005, 2006; Johnson, 2005) as is teacher fidelity to instructional practice (August et al., 2014; Calderon et al., 2011; Short et al., 2011). Genesee et al.’s (2006) synthesis of research evidence found classrooms need “active engagement” for ELs during the learning process (p. 477). Also, systematic employment of EL-specific strategies by teachers, such as those outlined in Short, Echevarria and Richards-Tutor’s (2011) report, indicate that as teachers are trained and implement strategies with fidelity, EL student achievement increases. When teachers do not implement with fidelity, student success is more limited as August et al.’s (2014) study suggests, indicating “if the [treatment] program had been more fully implemented, treated students would have done even better” (p.79). These promising EL practices and instructional strategies were used as a lens to view the more specific instructional strategies in science and technology that emerged from the literature. In this section I review the literature on instructional strategies for effective instructional strategies for (a) ELs in science and technology content areas and (b) computer science (CS) courses. I begin with an explanation of this literature search process and how I selected research articles for inclusion in my final literature pool. Next, I present the results from my research in themes from the literature pool. Finally, I present the gaps in the literature that outline the rationale for my exploratory study. Search procedures. To gather the most relevant references for my study, I followed several steps for each of my searches. I used the electronic databases available 11 to me through the University of Oregon library, using keywords and exclusionary subtopics to narrow down my results. Then, I used ancestral and descendent searches focused on references from research synthesis papers and journal reports in the areas of EL strategies and computational thinking strategies. Next, I applied my relevance criteria to the resulting studies to narrow it down to a manageable pool of peer-reviewed research. I began my search by looking for the intersection of EL instructional pedagogy and CS/CT instructional pedagogy and the impact of this on equity. A search that included all four of the following terms resulted in no articles: (a) EL instruction (b) computational thinking, (c) STEM, (d) scaffolding and (e) computer science. While many of these studies are seminal works in the area of EL instruction, I eliminated all articles and reviews relating solely to English language arts, because this specificity was out of scope for this study. To fully encompass STEM, I began my search looking for articles with all four STEM components: science, technology, engineering and math EL instructional strategies. I eliminated all articles regarding engineering due to the narrow focus on robotics, very specific programs such as Project Lead the Way, or lack of focus on K-12 education. Next, I looked at math studies focused on EL students. Eight studies (Aquino-Sterling, Rodríguez-Valls, & Zahner, 2016; Barwell, 2006; Borgioli, 2008; Moschkovich, 1999; Staples & Truxaw, 2012; Star et al., 2014; Tan, 2011; Zahner, Velazquez, Moschkovich, Vahey, & Lara-Meloy, 2012) focused on supporting ELs in math, but none directly applied to the research questions proposed, because they were focused too specifically on a math subject (i.e. Algebra skills), student motivation strategies or narrowly focused on 12 culturally responsive teaching. Therefore, I decided to drop both engineering and math from the search. This reduced my original STEM-related research to only science and technology. So, though at this point I identified the engineering and math literature as outside of the scope of this study, I do note here that they remain important topics for current and future research by others. For each subsequent search on the remaining body of literature, I applied the following three inclusionary criteria to judge the relevance of articles reviewed: (a) results were based on empirically-derived data and clear methodology, (b) content was applicable specifically to education methodology in K-12 settings or pre-service teachers, and (c) publication dates were in the specified scope of time, 2000-2018. Articles that did not fit into these categories were excluded, for this portion of my search. Then, I scanned the abstracts and titles and applied my inclusion criteria. My first digital search included the key words (a) science, (b) English Language Learners & ELL, and (c) computer science, which produced 3,237 articles, reduced to 158 after filtering for methodology, K-12 setting and within my timeline. My second digital search included the key words (a) science, (b) secondary education, (c) equity and (c) EL strategies which produced 214 articles. When the same inclusion criteria were applied, 39 articles remained. My third digital search included the key words (a) computational thinking, and (b) strategies, resulting in 432 articles. After applying the inclusionary criteria, 87 articles remained. After these three digital searches, I had a total of 282 articles. I first eliminated all duplicate articles and then skimmed the abstracts, which eliminated 90 articles from the first search as not pertaining to my topic, 31 from the second search, and 69 from the 13 third search. All articles removed were either (a) not empirical research, (b) not focused on K-12 schooling, or (c) did not directly relate to teaching strategies. This left me with a total of 94 articles. I then reviewed each of the articles to arrive at seven that fully met my criteria. Of the 94 fully reviewed articles, four met the criteria for EL teaching methods and three met the criteria of computational thinking or computer science strategies. Total results from my digital search are summarized in Table 1. Table 1 Digital Literature Search Results and Articles Selected Keywords Initial Exclusionary Abstract Final results criteria applied skimming Science, English Learners, computer science, math 3,237 158 68 3 Science, secondary education, SDAIE, EL instruction 214 39 8 2 Computational thinking OR computer science strategies 432 87 18 2 Total 3,883 282 94 7 Concerned that the pool of literature was insufficient, I used four literature syntheses found in my search to guide an ancestral research: Coleman and Goldenberg, 2009; Geneses, Lindholm-Leary, Saunders and Christian, 2005; Lye and Koh, 2014; Weintrop et al., 2016. First, I scanned the reference sections of each of my eight articles and compared them to those of the syntheses. I looked for authors that I may have missed in my digital search. From this search, I located another 22 articles that were (a) on at 14 least two different reference lists and (b) appeared to include empirical data. For example, both Weintrop et al. (2016) and Lye and Koh (2014) led me to the work of Sengupta, Kinnebrew, Basu, Biswas, and Clark (2013), an article that was not identified through the initial search process. After scanning the 22 articles and applying my previously mentioned inclusion criteria, I ended up with seven more articles. My selection process led me to include 14 peer-reviewed articles for this section of the literature review. Based on the criteria used for each phase, all articles address at least one area of interest: (a) EL instructional practices in science or (a) computer science/computational thinking strategies. The following section describes my review and analysis procedures for the literature pool. Instructional Strategy Literature Search Results The purpose of my literature review for this section was to summarize and synthesize the literature and look for the intersection of English Learner (EL) instructional strategies in science and computer science/computational thinking (CS/CT). Because my subjects are related, but not well connected in prior research, I organized the synthesis of research into two themes: (a) EL instructional strategies in science and (c) CS/CT instructional strategies. Then, I organized the categories of research-based instructional strategies that emerged from both subject areas, looked for areas of overlap, and identified gaps in the research. Theme 1: EL instructional strategies in science. Six of the 14 studies included in my research pool were selected based on my inclusion and exclusion criteria for EL instructional strategies and their outcomes on student achievement. Table 2 summarizes the methods, settings, and subjects for each of the six studies. 15 Four studies employed quantitative measures (August et al., 2014; Bravo & Cervetti, 2014; Carrier, 2003; Echevarría, Vogt, & Short, 2004), all using pre and post- testing to measure students after interventions. Two studies (Case, 2002; Garza, Kennedy, & Arreguín-Anderson, 2014) used a variety of qualitative measures for their respective studies. No studies included in this pool were mixed methods nor employed both quantitative and qualitative measures. Settings and subjects for these studies are conjoined variables: the setting is in direct relationship with the subjects in these studies. Four of the six studies (August et al., 2009; Bravo & Cervetti, 2014; Case, 2002; Echevarria et al., 2004) investigated EL students in science classes. Two studies (Carrier, 2003; Case, 2002) evaluated secondary student outcomes with fewer than 20 participants. Carrier’s (2003) quantitative study focused on seven high-school ESL students in all of their classes. Because both secondary studies are small-scale, I also include research based on subjects in elementary schools and teacher preparation programs, including Garza et al.’s (2014) study of strategies used to instruct pre-service teachers in a science lesson conduced in Spanish. Studies were also evaluated overall for specific EL strategies that were found to empirically impact student achievement. Overall, the strategies can be categorized in two ways: (a) pedagogical strategies for ELs in science and (b) domain-specific EL instructional strategies. I present a summary of the strategies reported in each study in Table 3. Pedagogical EL strategies in science. All six of the 14 studies included in my research pool mention EL instructional strategies in science classes. Specifically, the studies noted EL achievement improved when teachers provide scaffolded instruction 16 (Bravo & Cervetti, 2014; Case, 2002; Echevarría et al., 2004), varied modalities (August et al., 2009; Bravo & Cervetti, 2014; Echevarría et al., 2004; Garza et al., 2014), and utilize a student’s prior knowledge (Bravo & Cervetti, 2014; Case, 2002; Echevarría et al., 2004). Bravo and Cervetti’s (2014) study of 10 teachers with a total of 115 EL and English-only 4th and 5th graders evaluated implementation of a variety of EL instructional strategies. ELs in treatment classrooms had significantly higher (F = 3.90; p < .05) scores on posttests than the control group ELs, making twice as much growth in their science conceptual understanding and vocabulary knowledge than those in the comparison group. They report the adaptations treatment group teachers made included “more teacher- student and student-student talk” and to use “models, employ hands on activities, activate student prior knowledge” (p.241). Using a variety of modalities, such as using multisensory approaches, varied interactions, allowing for choice between solo and group work, and kinesthetic activities also showed potentially interesting trends to investigate at larger sample size, but did not have statically significant relationships with EL achievement (August et al., 2009; Bravo & Cervetti, 2014; Echevarría et al., 2004; Garza et al., 2014). Echevarria et al. (2004), evaluated academic outcomes in science for students in a treatment or comparison group. Teachers in the treatment group were observed utilizing scaffolding strategies. The treatment group made greater gains, averaging 2.9 points between pre- and post-tests, in comparison to 0.7 points. The gains were not statistically significant but the trend toward improvement might suggest investigation at larger sample size (Echevarría et al., 2004) 17 Three of the six studies found that creating a classroom that links to students’ background knowledge had positive academic relationships, although none were statistically significant (Bravo & Cervetti, 2014; Case, 2002; Echevarría et al., 2004). Case’s (2002) small-scale case study of newcomers found that the intersection of students’ background knowledge and new knowledge could lead to both language and content knowledge. While not statistically significant, he found that “by asking students to justify their hypothesis, predictions or evaluations, the teacher encouraged students to build upon their experiences. This promoted more complex language” (pg. 73). August et al.’s (2009) study of science curriculum took place in a high-poverty school district in Texas. They examined 890 6th grade students’ science achievement in science classes that explicitly included the utilization of EL instructional strategies such as visuals, experiments and demonstrations, and teacher modeling for students. While the positive gains made by ELs in the treatment group were somewhat modest and not statistically significant, August et al. note that over a three-year middle school cycle, ELs using this program could make one full year of gains, which is “roughly double what native speakers of English would be expected to gain in the same time frame” (p. 367). The study noted curriculum is dependent upon teacher fidelity to the model, including use of a very specific set of EL instructional strategies. EL instructional strategies also offer opportunities for peer interaction and the use of collaborative and cooperative learning (Moughamian et al., 2009). Collaborative learning was also a common feature, mentioned specifically in four of the six articles in my research pool (August et al., 2009; Bravo & Cervetti, 2014; Echevarría et al., 2004; Garza et al., 2014). Bravo and Cervetti’s (2014) study of 10 teachers at five different 18 middle schools found that teachers in treatment classrooms that included professional development around EL instructional techniques were statistically significantly more likely to have higher scores on the post-test for both science understanding and science vocabulary, with more than double the growth from the comparison group. Garza et al.’s (2014) interview-based qualitative study asked a linguistically diverse group of 66 pre-service teachers about strategies employed during their own learning. The study centered on an experiential science activity taught entirely in Spanish. English-only speaking pre-teachers reported that having a partner was the most important linguistic bridge during their experience. During open-ended interviews, the majority of the pre-service teachers indicated that student-student interactions were useful pedagogical practices they would incorporate into their science classrooms. One participant shared that “having interactions and activities is key” (p. 3). Domain-specific EL strategies. All six articles included in this literature pool found that classrooms with ELs focus on language skills. These skills are generally organized into four domains: listening, speaking, reading and writing (Echevarria et al., 2004). Employing strategies to improve listening and speaking skills were specifically mentioned in five of the six articles (August et al., 2009; Bravo & Cervetti, 2014; Carrier, 2003; Case, 2002; Echevarría et al., 2004; Garza et al., 2014). Reading and writing strategies, categorized broadly as literacy strategies in my synthesis, were overall the second most frequently mentioned strategies necessary for student success overall, as indicated in four of six articles in my research pool (August et al., 2009; Bravo & Cervetti, 2014; Case, 2002; Echevarría et al., 2004; Staples & Truxaw, 2012). Case (2002) and Cervetti et al. (2014) specify writing as a discrete literacy skill essential for 19 success in math and science. Both Case (2002) and Cervetti et al. (2014) found linguistic structural components missing from ELs’ writing in science classes. None of these studies in my literature pool proposed a solution to the problem, but they did link the lack of writing skill to lower overall performance in science. 20 Table 2 EL Instructional Strategies Literature Synthesis Summary Study Type of Measures Subjects* Subject area research August et al. (2009) Quantitative Pre-Post tests 6th grade students, treatment n=562, Science comparison n=328 Bravo & Cervetti (2014) Quantitative Pre-Post test 4th & 5th grade students Science non-ELL n=57 ELL n=125 Carrier (2003) Quantitative Pre-Post test High school ESL students, N=7 ESL & all content area Case (2002) Qualitative Observation High School ESL students N=18 Science th Echevarria et al. (2004) Quantitative Pre-Post tests 6 grade students Science treatment n=241, comparison n=77 Garza et al (2014) Qualitative Interviews Pre-service teachers, N=66 Science *Note. ESL, ELL and EL are used interchangeably and reflect the nomenclature used in the cited study. 21 Table 3 Report of EL Instructional Strategies in Science from Literature Synthesis Study Strategies Prior Collaboration Varied Use of Vocabulary Speaking/ Scaffolded Literacy Knowledge modality visuals instruction Listening Instruction August (2009) - X X X X X - X Bravo & Cervetti X X X - - X X X (2014) Carrier (2003) - - - - - X - - Case (2002) X - - - X X X X Echevarria et al., X X X X X X X X (2004) Garza et al. - X X X - X - - (2014) Total 3 4 4 3 3 6 3 4 22 Theme 2: CS/CT instructional strategies. Eight of the studies included in my literature pool examined specific teaching strategies that empirically have a relationship with student achievement when examining either computational thinking (CT) or computer science (CS) instruction. Research suggests that computational thinking, defined as the conceptualizing of abstract thought through decomposition, logic-based problem solving skills and algorithmic thinking (Wing, 2006), is an essential component to computer science coursework, promoting algorithmic thinking, abstraction, problem- solving, resilience and self-efficacy (Sentance & Csizmadia, 2017; Yadav, Gretter, Hambrusch, & Sands, 2017). Table 4 summarizes the methodology, settings, and subjects for each of the eight studies. Five studies employed quantitative measures (Denner, Werner, Campe, & Ortiz, 2014; Fronza, Ioini, & Corral, 2017; Sengupta, Kinnebrew, Basu, Biswas, & Clark, 2013; Soh, Samal, & Nugent, 2007; Zendler & Klaudt, 2015) utilizing pre-post tests, surveys, and student assessments. Three studies (Fessakis, Gouli, & Mavroudi, 2013; Israel, Pearson, Tapia, Wherfel, & Reese, 2015; Sentance & Csizmadia, 2017) used a variety of qualitative measures for their respective studies ranging from a case study of 10 students (Fessakis et al., 2013) to an open-ended survey of more than 300 teachers (Sentance & Csizmadia, 2017). I had intended to use only studies conducted in the United States, but my literature search did not yield a great deal of empirical research. Five studies (Denner et al., 2014; Fessakis et al., 2013; Israel et al., 2015; Sengupta et al., 2013; Soh et al., 2007; Yadav, Mayfield, Zhou, Hambrusch, & Korb, 2014) took place in the United States. Because the literature pool was so small, I expanded the search to include three studies 23 with European settings: Fronza, Ioini, and Corral (2017) in Italy; Sentance and Csizmadia (2017) in the United Kingdom; and Zendler and Klaudt (2015) in Germany. Four studies include children as subjects: kindergarten students (Fessakis et al., 2013) and middle school students (Denner et al., 2014; Fronza et al., 2017; Sengupta et al., 2013). The remaining five studies analyzed adults; ranging from university undergraduates (Soh et al., 2007; Yadav et al., 2014) to teachers and administrators (Israel et al., 2015; Sentance & Csizmadia, 2017; Zendler & Klaudt, 2015). This wide variety of subjects lends itself to a wide variety of voices with very similar outcomes. For example, Fessakis et al.’s (2013) case study with kindergarten computer science students found “a variety of social interactions… helped their learning” (p. 96) and Soh et al.’s (2007) study of undergraduate computer science students found “a trend for the students working in groups to perform better than the students who worked individually” (p. 71). Similarly, Sentance and Csizmadia’s (2017) survey of over 300 computer science teachers found collaborative work to be the second most common theme that emerged. Studies were evaluated for specific teaching strategies that were found to empirically impact student achievement. Overall, the strategies can be categorized in the same two ways as Theme 1: (a) pedagogical strategies in CS/CT and (b) domain-specific CS/CT strategies. I present a summary of all the instructional strategies individually as reported in each study in Table 5. Pedagogical strategies in CS/CT. A variety of instructional strategies found positive, statistically significant relationships with student achievement in computational thinking, both qualitatively and quantitatively reported. These instructional strategies can be categorized as use of varied modalities (Fronza et al., 2017; Sentance & Csizmadia, 24 2017; Soh et al., 2007; Zendler & Klaudt, 2015), collaboration and interaction strategies (Denner et al., 2014; Fessakis et al., 2013; Fronza et al., 2017; Israel et al., 2015; Sentance & Csizmadia, 2017; Soh et al., 2007), use of sequenced learning activities (Israel et al., 2015; Sengupta et al., 2013; Sentance & Csizmadia, 2017; Zendler & Klaudt, 2015), and modeling tasks (Israel et al., 2015; Sengupta et al., 2013). Three studies (Fronza et al., 2017; Sentance & Csizmadia, 2017; Soh et al., 2007; Zendler & Klaudt, 2015) illustrated that varied modalities, or the use of different types of visual, aural or kinesthetic learning activities, may be useful for teaching computer science. For example, Soh et al. (2007) found a statistically significant increase in post- test scores (p < .001) after employing a variety of modalities for undergraduates in an introductory CS course. Fronza et al. (2017) reported the use of a framework utilizing both paper-pencil and computer-based activities to teach computer science resulted in 78% of 42 sixth graders receiving a positive evaluation after the first paper-pencil activity and 100% of the students receiving sufficient or good evaluations after integration of paper-pencil and computer-based. No statistical inference tests were provided for the percentages, which were treated as descriptive statistics only. Sentance and Csizmadia’s (2017) sizable qualitative study (n=339) in the United Kingdom of members of Computer At School organization found that varied instructional strategies defined as unplugged and/or hands-on activities were the most commonly coded key strategy, coded 115 times, more than double the next category. Zendler and Klaudt (2015) surveyed 120 computer science teachers in Germany regarding their application of instructional methods. Based on an ANOVA analysis of the survey results, 25 instructional methods and knowledge processes showed a statically significant relationship with learning computer science in this study. Collaboration and interaction in computational thinking involves both teacher- directed collaborative interactions between students (Denner et al., 2014; Fessakis et al., 2013; Fronza et al., 2017; Sentance & Csizmadia, 2017; Soh et al., 2007) and authentic communication initiated by students (Fessakis et al., 2013). In the eight of articles reviewed, paired programming was the most commonly mentioned specific sub-strategy, showing up in six studies (Denner et al., 2014; Fessakis et al., 2013; Fronza et al., 2017; Israel et al., 2015; Sentance & Csizmadia, 2017; Soh et al., 2007). Sentance & Csizmadia’s (2017) survey of 339 members of Computers At School in the United Kingdom found collaboration was the second most commonly employed strategy among CS teachers with 24 mentions of pair work, 32 mentions of peer- mentoring and 22 mentions of overall collaboration. While this does not indicate efficacy, it does indicate that CS teachers utilize collaborative strategies in the CS classroom, in the context of this study. Two studies (Soh et al., 2007, Denner et al., 2014) did show some evidence, however small, that cooperative groupings are associated with improved achievement. Soh et al. (2007) developed and tested an integrated framework for undergraduates studying computer science at the University of Nebraska-Lincoln, finding that students who participated in the cooperative learning treatment group performed “significantly better than the direct instruction group, as measured by the final laboratory assignment grade” (p.70). Denner et al.’s (2014) study of 320 students in a United States middle school found that among students with low prior computer use, those who worked in 26 pairs increased their programming knowledge and computational thinking performance significantly, as measured by pre- and post-testing. Also, in a case study with 10 kindergartners, Fessakis et al. (2013) found that paired programming enhanced not only mathematical skills, but also communication and collaboration skills through authentic interactions. Domain-specific CS/CT strategies. Four articles included in this literature pool explicitly list strategies related to teaching computational thinking (Fronza et al., 2017; Zendler & Klaudt, 2015) and/or teaching of computer science skills (Fronza et al., 2017; Sengupta et al., 2013; Sentance & Csizmadia, 2017; Zendler & Klaudt, 2015). Zendler and Klaudt (2015) found the main effect of direct instruction to be statistically significant, as per the survey given. Three studies found direct instruction in computer programming specifically appears to be useful when teaching specific and discrete skills such as decomposition strategies and debugging (Fronza et al., 2017; Sentance & Csizmadia, 2017), or construction, execution, reflection/analysis, and/or refinement of work (Sengupta et al., 2013). Zendler and Klaudt (2015) surveyed German teachers, asking them to select which instructional methods and knowledge processes they both utilize and would recommend. Direct instruction, analyzed by cluster analysis and ANOVA, was reported to be the most important factor in the acquisition of knowledge in computer science classes, as reported by teachers. Specifically, Zendler and Klaudt found that “direct instruction should be used in combination with problem-based learning and augmented by learning tasks” (p. 922). Even though frequency of use in this study was not correlated to academic success, 27 it does indicate this sample of CS teachers, with an average of 7.5 years of CS teaching experience have an instructional preference. Fronza et al.’s (2017) study was a combination of both direct instruction and computational thinking and focused on the direct teaching of computational thinking. In an analysis of student outcomes, 100% of the participants “delivered simple software systems that represented effectively the assigned part of history and technology” and of eight direct-instruction concepts in computational thinking taught, five had a 100% achievement rate. Contextualization of concepts in real-world applications and project- based/problem-based learning (PBL) was also commonly mentioned as a teaching strategy that was particularly helpful in CS courses. Sentance & Csizmadia (2017) indicate that contextualization is the second most commonly identified strategy used by CS teachers, after collaboration. Zendler and Klaudt (2015) found that, according to teacher recommendations, problem-based learning has a statistically significant relationship to the act of learning in computer science, particularly in application, transfer of knowledge and assessment. Conversely, Israel et al. (2015) found that the most common challenge for teachers integrating computer science lessons was “meaningful access to computing activities” (p. 272); but that when the students were guided towards an activity that was related to prior knowledge or an interest (i.e. making a steering wheel turn) students were highly engaged. 28 Table 4 CT/CS Strategies Literature Synthesis Summary Study Type of research Measures Subjects Setting Denner et al. (2014) Quantitative Survey & Assessment Students N= 320; United States, middle treatment n= 180, school comparison n=138 Fessakis et al. Qualitative Case study Students N= 10 United States, (2013) Kindergarten class Fronza et al. (2017) Quantitative Survey & assessment N= 42 Italy, middle school Israel et al. (2015) Qualitative Interviews & Teachers n=7; Midwestern United States, observation administrators n=2 elementary school Sentance & Qualitative Open-ended survey Members of CAS United Kingdom, online Csizmadia (2017) n= 339 Sengupta (2013) Quantitative Pre-post test Treatment n= 15, United States, Tennessee comparison n=9 6th grade Soh et al. (2007) Quantitative Student assessment Treatment 1 n=55, United States, University treatment 2 n= 65, of Nebraska-Lincoln, comparison, n= 64 undergraduates Zendler & Klaudt Quantitative Survey N=120 secondary CS Germany, Baden- (2015) teachers Wurttemberg 29 Table 5 Report of CS/CT Instructional Strategies from Literature Synthesis Study Strategies Collaboration Varied Sequenced Modeled Explicit Direct Contextual modality learning tasks Computational Learning/ activities/ thinking PBL scaffolding instruction Denner et al. (2014) X - - - - - - Fessakis et al. (2013) X - - - - - - Fronza et al. (2017) X X - - X X X Israel et al. (2015) X - X X - - - Sentance & Csizmadia (2017) X X X - - X X Sengupta et al. (2013) - - X X - X X Soh et al. (2007) X X X - - - - Zendler & Klaudt (2015) - X X X X X* X Total 6 4 5 3 2 4 4 *Note: Discussed as part of project-based learning (PBL) 30 Summary As described above, two themes were explored in this literature review: (a) EL instructional strategies in science, and (b) CS/CT instructional strategies. Within each theme, research was examined to identify the impact of specific instructional strategies on student achievement. The results from the studies in each theme were synthesized into two types of instructional strategies: (a) pedagogical and (b) domain-specific. While the domain-specific categories were clearly focused on course context, the pedagogical strategies showed intersection. Broadly, three similar, overarching categories of pedagogical instructional strategies overlapped: (a) collaboration, (b) scaffolded instruction, and (c) use of varied modalities. Gaps in the Literature Several gaps in the literature emerged. First, none of the studies located used mixed methods to triangulate data. Knowledge development is not a linear process; rather, it creates feedback loops that require a variety of investigations (Earle & Maynard, 2013). Creswell and Creswell (2017) argue the mixed methods model provides for deeper exploration of results. Nine of the 14 studies were quantitative studies and six were qualitative. Second, although there were several EL investigations that took place in science and classrooms utilizing technology, I did not locate any published studies that looked at EL student performance in CS/CT classes. Nor did I locate any studies of teacher use of specifically mentioned EL strategies in CS/CT. Even though Denner et al.’s (2014) sample included EL students, no disaggregated data analysis for this subgroup was 31 reported. Therefore, there is almost no representation of EL student data in CS/CT classrooms identified in my literature review. Additionally, my search uncovered an absence of published empirical data on beliefs about and use of research-based EL instructional strategies collected from CS/CT classrooms in the United States. Sentance and Csizmadia (2017) and Zendler and Klaudt (2015) investigated teacher use and perceptions of CS/CT strategies in the UK and Germany, respectively. These studies found commonly employed instructional strategies in CS/CT classrooms, but did not correlate that data with any other field of study such as language learning. Nor did either researcher ask specific questions about the make-up of the teachers’ classrooms, such as in regard to language learners. Thus, there appears to be a lack of empirical research on the efficacy of specific, research-based instructional strategies for ELs in CS/CT classrooms. Focus on this gap is important because access to the academically rigorous CS coursework at the secondary level may be dependent upon implementation of EL instructional pedagogy. Theoretical Framework A theoretical framework is defined as the “foundation from which all knowledge is constructed” for a research study (Grant & Osanloo, 2014, p. 12). I relied upon the constructivist learning theory as my foundation to build both my study design and measurement tools. A constructivist lens helped define more clearly where strategies for English language learning and computer science/computational thinking and shaped survey instrument design. Constructivism is not a unitary theoretical position, but is more commonly seen as a continuum influenced by seminal research from John Dewey, Jean Piaget, and Lev 32 Vygotsky (Doolittle & Camp, 1999). While there are nuanced differences between several sects of constructivist theory, as pedagogy all have a common thread: knowledge is actively constructed by the student, not passively absorbed (Ben-Ari, 1998). Student learning, in this context, depends primarily on what students do rather than what the teacher does (Mustafa & Fatma, 2013). The constructivist pedagogy posits that learning is most effective in contexts where learners construct knowledge and develop competences in personally relevant contexts while being consciously engaged in their learning (Papert & Harel, 1991; Przybylla & Romeike, 2014; Resnik, 2007). Ben-Ari (1998) integrates a constructivist approach in teaching computer science, utilizing social interaction, motivation and exposing the prior knowledge of students before they begin applying concrete experiences to the process. Below is a summary of Doolittle and Camp’s (1999) synthesized definition of eight essential factors of constructivist pedagogy as applied to learning. 1. Learning should take place in authentic and real-world environments 2. Learning should involve social negotiation and mediation. 3. Content and skills should be relevant to the learner. 4. Content and skills should be understood within the framework of learner’s prior knowledge. 5. Students should be assessed formatively, serving to inform future learning experiences. 6. Students should be encouraged to become self-regulatory, self-mediated, and self-aware. 7. Teachers serve primarily as guides and facilitators of learning, not instructors. 33 8. Teachers should provide for, and encourage, multiple perspectives and representations of content. Similarly, constructivist classrooms for second language learners are predicated on active learning, students’ prior knowledge, authentic communication experiences and integrate and embed cooperative group work (August & Shanahan, 2006; Cline & Necochea, 2003; Coleman & Goldenberg, 2009; Goldenberg & Coleman, 2010; Moughamian, Rivera, & Francis, 2009; Murillo, 2013Goldenberg & Coleman, 2006, Fradd & Lee, 1999; Gibbons, 2003). Also, prominent second language acquisition theorist Stephen Krashen advocates for constructivism in second language acquisition. Krashen (2011) recognized that students become experts in areas not by learning facts, but rather “by trying to solve problems of great interest to them” (p. 385). This type of EL instruction takes authenticity, relevancy, social negotiation, formative assessment and self-mediation into account when planning and delivering instruction. Researchers (Baker et al., 2014; Goldenberg & Coleman, 2006; Coleman & Goldenberg, 2009; Genesee et al., 2006) indicate that non-constructivism methods such as direct instruction are not only suitable, but often required for ELs. According to research synthesized by August and Shanahan (2006), this is primarily focused around the teaching of reading and early literacy skills. Also, Tanishi and Le Minstrel (2017) indicate that research supports direct and explicit reading comprehension strategy instruction in grades 9-12. Tanishi and Le Minstrel go on to conclude, however, that direct and explicit comprehension strategies are not enough; students must be motivated and engagement in learning “may be an important factor for ELs in their learning to read, in their academic language learning from school texts, and in their literacy and academic 34 achievement” (p. 327). Therefore, I posit that constructivist learning can coexist with direct instruction if contextualized and offered as a component of authentic motivation. Vocabulary instruction may also benefit from direct instruction, but the research has demonstrated mixed results and may favor a more constructivist approach. August and Shanahan (2006) reported “ELLs learn more words when the words are embedded in meaningful contexts and students are provided with ample opportunities for their repetition and use” (p.45). Additionally, Wright and Cervetti’s (2016) most recent review of nearly 50 years of vocabulary instruction research supports the idea that active processing of word meanings has greater impact on comprehension than passive approaches, stating the “long-term direct teaching of a large number of word meanings did not impact students’ generalized comprehension compared with no-treatment control groups” (Wright & Cervetti, 2016, p. 221). The constructivist framework is applicable to both computer science and English language learning and provides a context for creating my survey and interview questions in order to answer my research questions. The literature review uncovered several types of effective instructional strategies in both effective EL and computer science instruction. There are several areas of overlapping strategies between the two instruction that also map to Doolittle and Camp’s (1999) constructivist factors, as seen in Table 6. For example, authentic or real-world learning environments are modeled with the teacher uses visual aids, mimicking presentations at a business meeting or varies the type of instruction, perhaps by asking students to learning from a video or from a classmate, not just relying on lectures. Also, when a teacher understands a student’s prior knowledge, 35 they can sequence the learning, adding or subtracting information based on the previous learning or student understanding. Table 6 Constructivist Factors Mapped to EL and CS/CT Strategies CS/CT instruction Constructivist Factors EL instruction Varied modalities, 1. Authentic/real world learning Use of visuals, varied contextualized learning environment modalities Collaboration 2. Social negotiation Collaboration Sequenced learning Utilizing prior activities, modeling 4. Prior knowledge knowledge, scaffolded tasks, contextualization instruction Utilizing prior Contextualized learning 6. Relevance to learner knowledge Scaffolded instruction, 8. Multiple acceptable outcomes Scaffolded instruction, PBL Summary of Findings of the Literature Review EL students could be better equipped for the emerging job market in they had CS training starting in K-12 schools. While we might be able to extrapolate that low numbers of enrolled and successful Hispanic students in CS classes in Oregon, for instance, may have a relationship with low numbers of EL students, since this is the largest EL group in Oregon, still we simply do not know how many EL students are enrolled in CS classes. However, the opportunity gap will not be fixed simply by enrolling higher numbers of 36 underrepresented students. Instruction in CS classes must be research-based. For the EL population, there is scant evidence-based research currently available for CS teachers. The literature does indicate, however, there may be an efficient route to EL success in CS classes: a focus on the theoretical constructivist instructional strategies, such as collaborative groupings, sheltered strategies and utilizing a variety of modalities. Based on the findings of this review, I conduced a mixed-methods research study. The next chapter provides a review of the research questions, a methodological overview of my study by phase, data collection and analysis by phase, and a summary of the limitations to the methodology. 37 CHAPTER II METHODOLOGY Based on the gaps found in the introduction and literature review in Chapter I, I conducted a multiple-methods study that collected survey data and conducted in-depth interviews. Prior research suggests that the three categories of instruction, a) collaboration, (b) scaffolded instruction, and (c) the use of varied modalities, may provide the most efficient route to EL success in CS/CT classes. My study investigated if these three categories of research-based EL instructional strategies emerge from the data and appear to be (a) perceived as useful to teachers and/or (b) commonly employed in CS classes. To answer my research questions (listed again below) and to better understand the perceptions of EL and CS teachers, I completed a two-phase, sequential exploratory mixed methods research design. The context of quasi-experimental studies evaluating instructional strategies is complex and thus, a mixed methods design involving multiple expert groups and data points is useful (August & Shanahan, 2017). I sequentially collected, analyzed, and ultimately combined quantitative and qualitative data (Creswell & Creswell, 2017). I conducted my study in two phases. A visual display of this sequential two-phase design is included in Figure 1. Phase I addressed both RQ 1a and RQ 1b research questions: • RQ 1a. Which specific instructional strategies do EL teachers (EL-T) and CS teachers (CS-T) believe are most likely to increase success for ELs in computer science classes? 38 • RQ 1b. Are there differences between EL-T and CS-T perception of usefulness and frequency of use of instructional strategies? In Phase I, I analyzed data from the Computer Science: Best Instructional Practices for ELLs survey (CS: BIPE), completed by Oregon EL and computer science teachers (N=58). I designed this survey to measure educator perceptions of EL instructional strategy efficacy. I employed a purposive sample (see description in Sample section) of both EL and computer science (CS) teachers across the state of Oregon. Phase II addressed the second research question: • RQ 2, How, when and why would EL-T and CS-T utilize or recommend particular EL strategies in computer science classes? In Phase II, I conducted 4 qualitative interviews: 2 ESOL-endorsed teachers who work as district level instructional coaches and 2 CS teachers in schools with high proportions of EL students. These discussions explored how teacher perceptions impact the integration and employment of specific instructional strategies. I built upon the work by Sentance and Csizmadia (2017) and Zendler and Klaudt (2015) by focusing my study on teacher experts in both EL and CS/CT, not just CS/CT. Phase I Phase II QUANT QUAL Data Analysis Survey Interviews Figure 1. Mixed method sequential research design. 39 Methodological Overview I integrated quantitative and qualitative data designed to help provide analysis of the problem by both triangulation, or collecting information using a variety of methods, and cross-validating data to look for convergence and divergence (Maxwell, 2012, Morgan, 2019). As Creswell and Creswell (2012) describe, in an explanatory mixed methods design “the quantitative results are then used to plan the qualitative follow-up” (p. 222). By mixing modes, I hoped to improve interpretation and minimize error within my resource and time constrains (Dillman, Smyth, & Christian, 2014). Data for Phase I was taken as a snapshot (Babbie, 2013) of teachers and their recommended strategies from a given set of research-based instructional strategies. As a sequential study, Phase II was dependent upon Phase I in content only and only three weeks separated data collection between phases. Data from both phases were analyzed to synthesize the study findings. In the following sections, I will describe my research design, settings, participants, instruments and data collection procedures, data analysis and interpretation, and validity threats and limitations of the study that investigates the what, how and why of EL instructional practices in CS classrooms. Research Design In this section, I present a description of the sampling logic and participants and setting for both phases of this project. Since these phases draw on different populations and employ different sampling logics, I describe each phase separately. The unit of analysis is teacher, grouped into the EL and CS teacher groups, as defined by Babbie 40 (2013) as “the what or whom being studied” (p. 97). The perspective of both expert groups examined in both phases of my study allowed me to compare suggested teaching strategies, which are important malleable factors associated with educational outcomes (Earle & Maynard, 2013). Overall, I included a total of 244 teachers for this study. Phase I: Survey Sampling logic and participants. The survey study sample for Phase I included two groups of teachers: (a) EL teachers (EL-T) and (b) computer science teachers (CS-T). The two samples of teachers were a non-random, purposive convenience sample. Email distribution lists were obtained through state-wide organizations with self-selected memberships. EL-T. In the state of Oregon, there are 355 teachers at the K-12 level who hold an active ESOL endorsement. This information does not, however, indicate whether or not the teachers are actively teaching or using the ESOL endorsement (http://tspc.oregon.gov/licensure/licensure.asp). I began my search for survey study participants by reaching out to 11 Title III coordinators at the Education Districts around the state of Oregon. After three weeks of no reply, I used the Oregon Department of Education’s website to locate the Title III Contact List for the 2018-19 school year, which listed 411 teachers, administrators and other school employees. CS-T. Currently, the Oregon Department of Education has no licensure requirement for computer science, it is a district-by-district designation. Thus, there is no implicit commonality available through TSCP databases. Thus, instead of recruiting through school districts, I contacted the Oregon Computer Science Teachers Association 41 (OCSTA), the premiere computer science organization in Oregon, and obtained permission for their president to share my electronic survey with their 150 members. While there is no requirement of level of expertise or classroom experience to be part of OCSTA, the members are a part of “a statewide network of teachers” who work together to share curriculum, resources and knowledge for the sake of student achievement and exposure to opportunities in the high-tech workplace (“Join OCSTA”, 2019). Phase II: Interviews Sampling logic and participants. According to Babbie (2013), “sometimes it’s appropriate to select a sample on the basis of knowledge of a population, its elements, and the purpose of the study” (p.190). In this case, a purposive, small sample for in-depth interviews was chosen primarily for instructional expertise of the EL population and/or an understanding of the content of a CS class. According to Remler and Van Ryzin (2015), “people or cases are chosen for a specific purposes… and the number of people or cases is necessarily limited” (p. 62). The study sample for Phase II included the same two groups of teachers as Phase I: (a) EL teachers (EL-T) and (b) computer science teachers (CS-T). However, this sample is much smaller, with only four participants. The four participants, two EL-T and two CS-T, were chosen purposefully to provide a level of expertise and knowledge necessary to better understand the use of EL strategies in a CS class. CS-Ts were chosen for demonstrated expertise in the area of Computer Science instruction, as defined by (a) recommendation by OCSTA personnel and (b) teach recognized computer science course, such as AP Computer Science Principles, at the secondary level. 42 The EL-T were chosen from volunteers from the CS: BIPE survey conducted in Phase I who indicated that they were interested in participating in the survey. Overall, six EL-T volunteered to be interviewed. Two were eliminated because they did not have active roles in schools that utilized their ESOL endorsement. Three volunteers were teachers on special assignment (TOSA) for their school districts and acted as ESOL coaches. I invited these three volunteers to be interviewed, as their district level perspective offered insight to the frequency of instructional strategies on a district-wide level. Data Collection and Analysis In keeping with the sequential, mixed-methods design, I collected two sets of data, quantitative and qualitative. First, I collected data from the CS: BIPE survey and used quantitative approaches through frequency counts to identify instructional strategies that EL and CS teachers recommended. Second, I collected qualitative data from four teachers in interviews, two EL and two CS to further clarify what, how and when these strategies seemed to be most useful. Finally, I analyzed each data set separately, and then subsequently examined results together to explore patterns and relationships. In this section, I discuss each phase of the data collection sequentially, beginning with the Computer Science: Best Instructional Practices for ELLs (CS: BIPE) survey and how I analyzed the data. Next, I describe the in-depth interview qualitative phase of the research project and explain how I conducted the four interviews, how I collected data from those discussions, combined them with the qualitative date from the CS:BPIE, and analyzed the data. Phase I: Quantitative Survey Data 43 In this section, I present description of the CS: BIPE survey, describe how the survey was administered and delivered to the two teacher groups, and explain how the data from the survey were analyzed. Description of the CS: BIPE. The Computer Science: Best Instructional Practices for ELLs (CS: BIPE) survey was a 17-question, anonymous electronic survey designed by me. The CS-BPIE is based on the literature review findings combined with overlapping components of the constructivist theory (see Table 6). I separated the CS: BIPE, found in Appendix A, into three distinct sections: Part A and B focused on instructional strategies beliefs and frequency of use and Part C collected demographic information. I did not collect names, email or IP addresses in order to maintain anonymity. The survey began with an informed consent screen which directed only participants who wished to consent to continue the survey. Part A consisted of three multiple-choice questions, asking teacher to identify which of four presented strategies they believe would be most useful in order to increase positive academic outcomes. Each answer set for Questions 1-3 contained (a) one of the three overlapping strategies and at least one (b) EL instructional strategy and (c) one CS instructional strategy. In all, 12 strategies identified in the literature review were presented as answer options. Eight strategies were overlapping, two were domain specific to EL instruction, and two were domain specific to CS instruction. In addition, Part A included the option for participants to see defined terms, by hovering over “Definitions here” listed at the end of each of question. To eliminate social desirability effects, Questions 1-3 did not use a Likert-style scale (Dillman et al., 2014). While a forced-choice survey limits the choices a participant 44 can have, a forced-choice survey has advantages in terms of construct and observed validity and eliminates the halo effect of Likert-scales (Bartram, 2007). Part B consisted of three questions, investigating the frequency of use of the same 12 strategies presented in Questions 1-3. Question 4 asked teachers to rank their use of each strategy on a 0-5 Likert-type scale, from Never to Everyday. Questions 5 and 6 were optional, open-ended questions that allowed teachers to explain their choices, if they wished to provide more information. These answers were combined with the qualitative data. Part C collected demographic information including number of years in the classroom, grade level(s) and subject (s) taught, and type of teaching credential. Part C employed skip-logic, so that if a teacher self-identified as a CS, EL, they only answered questions that pertained to that subject area. Administration of the CS: BIPE. As indicated by Dillman et al. (2014), electronic surveys are attractive to participants and researchers alike because of speed, economy, and scale. I used the online program Qualtrics to deliver the survey. I electronically administered the survey for 32 days, between November 2018 and December 2018. Delivery via email was different for each teacher group, as outlined below. EL-T. The Title III contact list obtained from the ODE website contained 411 email addresses. I eliminated all obvious non-teaching contacts, including secretaries and district office employees, reducing the number to 128 contacts. I emailed the remaining contacts, n=128 (Appendix B) requesting their participation in the study. Two emails came back undeliverable, reducing my population to 126. I sent a follow-up email (see 45 Appendix C) to remaining contacts three weeks later (n =126) referencing the first email and indicated the survey would remain open until mid-December. CS-T. The current OCSTA president forwarded my invitation to the study to 120 current members (Appendix D). Because of organization membership bylaws, a follow- up email was prohibited. While most members of OCSTA self-define as “computer science teachers,” some work in the computer science industry in other areas. Thus, I was unable to ascertain how many of the 120 members were actual CS teachers. Two members reached out and indicated that they were no longer CS teachers, so I subtracted them from my overall sample (n=118). CS: BIPE data analysis. I used SPSS software to analyze the CS: BPIE data from questions 1 through 4 through descriptive statistics and four chi-square distribution tests (contingency tables). Question responses were analyzed using a two-way contingency table to evaluate whether a statistical relationship existed between two variables (Green & Salkind, 2016). “A two-way contingency table consists of two or more rows and two or more columns. The rows represent the different levels of one variable, and the columns represent different levels of a second variable,” (p. 263) In this study, I used 2 x 4 contingency tables to evaluate Questions 1-3; two teacher groups by the four possible answers for results on each of the first three questions. Question 4 was evaluated similarly, analyzing strategy frequency by teacher group. Data from Questions 5 and 6, the optional, opened ended questions, were analyzed with the Phase II Qualitative interview transcriptions. For this analysis, the expected frequency in some cells was less than five and required correction because in small studies, “when the expected frequency in one or 46 more cells is less than 5, the analysis is appropriately performed with the Fisher exact test” (Daya, 2002). SPSS allows the Crosstabs procedure to print the Fisher-Freeman- Halton exact test of independence, also known as Fisher’s Exact test, even when the contingency table tables were larger than 2x2 (SPSS manual). Thus, the reported p value reflects the Fisher-Freeman-Halton critical values of significance. Phase II: Qualitative Data In this section, I present description of the interview protocol, describe how the interviews were administered and explain how the data from the interviews were coded and analyzed. Note that the CS:BPIE contained two optional qualitative questions (Questions 5 and 6) regarding frequency of use of strategies. Of the 52 survey takers, 27 (52%) answered at least one of the questions. I sorted answers by teacher type, CS-T, EL-T, or Both, and combined responses with the interview responses for qualitative coding. Description of the interview protocol. Participants were contacted via email or phone call to set up the interviews. The interview protocol (Appendix E) began with informed consent, describing the study and the manner in which the results will be kept anonymous. The questions needed to be both broad enough to cover both teacher groups and specific enough to engage the perspective of the particular teacher (Remler & Van Ryzin, 2015). Teachers were asked to provide their insight from their position and experience. The protocols were slightly altered for the CS-T and EL-T. For example, Question 1 was slightly altered between groups to keep the focus of the study on computer science classrooms. 47 The three questions and probes were developed from the literature review results and Phase I data (Creswell & Creswell, 2017). Following the structure of a semi- structured interview, it was comprised of both open-ended questions and probes to guide the interview (Remler & Van Ryzin, 2015). All questions were designed to answer RQ2: How, when and why particular instructional strategies are chosen by EL and CS teachers. Interview administration. I conducted semi-structured interviews with all four teachers. Interviews took place in January 2019 over a web-based meeting tool. Interviews were recorded, with interviewee consent, for transcription accuracy. The interviews followed the interview guide as well as soliciting participants’ feedback during the conversation. In some cases, unscripted follow-up questions were asked for clarification and to deepen the conversation. To ensure anonymity recorded interviews were given numbers, not names. Interview analysis. I based my analysis on Creswell and Creswell’s (2017) first six steps for analyzing qualitative data: (a) transcribe the interviews, (b) read all the transcripts as a whole document to get an overall sense of the information, (c) code the data, (d) identify themes that emerge from the coding process, (e) analyze the themes both vertically and horizontally, (f) interpret the findings, and (g) write up a narrative to report on the data. Because my interview sample was small, I compressed steps (a) and (b) steps, skipped step (c) and proceeded with the next three recommended steps. I finished with step (g), and wrote up the overall narrative by themes. I transcribed the recorded interviews verbatim into word processing documents and read all the transcripts as a whole document to get an overall sense of the information. According to Marks and Yardley (2004) “on a small sample size only the 48 descriptive use of thematic coding is advisable” (p. 66), thus, for this small sample I identified themes that emerged from evaluating the interview transcripts and applied them to both my research questions and the data that had emerged from Phase I data collection. (Remler & Van Ryzin, 2015). Finally, I reviewed the themes both between and within my two teacher groups, interpreted the findings and summarized the data in narrative form. Limitations to the Methodology Mixed methods research design is an integration of qualitative and quantitative data, which provides additional insight beyond collection of one single type of data (Creswell & Creswell, 2017; Dillman et al., 2014). Employing triangulation, or collecting two forms of data, increases internal validity (Maxwell, 2012; Creswell & Creswell, 2017). In this case, data were collected both quantitatively and qualitatively, thus mitigating the threat of mono-method bias (Parker, 1990). An exploratory study such as this intends to inform policy, practice, and/or future development of instructional models. Thus, it is important to establish the validity of both the quantitative and qualitative data collected separately, as well as the validity of the overall interpretation of findings. In the following sections, I will outline how I identified and mitigated threats to internal validity and content validity in Phase I and reduced researcher bias in Phase II (Creswell & Creswell, 2017). Phase I Limitations Phase I includes several limitations that should be considered when interpreting results. Quantitative and qualitative data were collected in Phase I, therefore limitations 49 in validity and reliability are reported. Limitations include, but are not necessarily limited to, internal validity, construct validity, and reliability threats. Internal Validity Threats First, a possible self-selection sampling bias needs to be considered. While the sample frame intended to include a diverse set of teachers throughout the state, and thereby increasing the generalizability, both sample sets were convenience samples from listservs created by outside organizations. Not only does the self-selection bias affect the response rate to my survey, but also to the very nature of the population that was sampled. Computer science teachers were recruited from a self-selected organization that requires an online sign up. EL teachers were recruited from a list of EL personnel compiled by the Oregon Department of Education. Due to the blind nature of these email lists, there was no way to tell if the participate were (a) active teachers and/or (b) credentialed in CS or ESOL. The sample size and low participation rates also should be taken into consideration. Overall response rate was quite low, with only 24% of the total sample completing the survey. However, it is worth noting that online response rates often have lower response rates, sometimes as low as 33%, particularly if the email is answered on a mobile phone (Dillman et al., 2014; Fincham, 2008; Nulty, 2008) Unequal response rates is also worth noting. Of the 52 responses, 63% self-identified as CS teachers. Finally, the current political climate of anti-immigration rhetoric may have influenced the response rate. Respondents may have been reluctant to participate in the survey, the title of which indicated a focus on English learners. Teachers may have been 50 disinclined to give any opinion about students who do not speak English as their first language, positively or negatively. Construct Validity Threats Content validity and low reliability need to be considered due to the survey design and implementation. The design of the survey needs to “accurately measure the thing you want to measure” (Fitzner, 2007). Content validity must attempt to ensure that all test items are as free of bias as possible and the questions accurately measure the construct (Messick, 1996; Tindal & Marston, 1990). I designed the CS: BIPE instrument and there have been no other studies of its use. However, several experts in the area of computer science, EL, and research were consulted before employing the survey to increase content validity (Dillman et al., 2014; Maxwell, 2012). The survey was revised five times under direct supervision and field tested with both EL teachers and CS teachers who did not participate in the study. Questions and answers were refined between each field test. Lack of assessment retests, low response rates, and no alternative form of the survey may have influenced both reliability and content validity measures. Additionally, I purposely did not overlap all twelve strategies presented in Q1-3 and Q4. Based on the literature review, replaced computational thinking strategies in questions 13 with prior knowledge in Question 4. Results in my analysis indicated that all teachers used this most frequently. Because prior knowledge was not an option in Q1, Q2 or Q3, content validity may have been compromised in the survey. However, steps to ameliorate low validity and reliability were taken. Several experts in the area of computer science, EL, and research were consulted before employing the survey to increase content validity (Dillman et al., 2014; Fitzner, 2007, 51 Maxwell, 2012). The survey was revised five times under direct supervision and field tested with both EL teachers and CS teachers who did not participate in the study. Questions and answers were refined between each field test. Since this survey did not allow for a “None of the Above” response, it is reasonable to consider that Questions 1 through 3 of the survey may force participants to provide an answer that may not be true for them. Even though this potential existed, the clarity of the survey, having an upper limit of four items in each block, and having option choices that were not attributes at opposite ends of a scale decreased this likelihood. Because all answers were considered equally “correct, omitting “None of the Above” increased validity and reduced the effects of overly high ratings (Bartram, 2007; Brown & Maydeu-Olivares, 2011). Item nonresponse bias is also to be considered. I did not require respondents to answer all questions before moving forward, however. This allowed respondents to skip questions and had an unintended consequence. Two questions had extremely low response rates, with fewer than half of the CS-T respondents answering, potentially altering the accuracy and power of the statistical analysis of these oft-skipped questions. Social desirability bias influence must also be examined. Questions 1-3 did no not lend themselves towards this. However, Question 4 required teachers to self-report their use of research-based instructional strategies. Self-reports of desirable behaviors, i.e. more frequent use of instructional strategies, can inflate scores. However, the teacher groups are compared to one another within the same survey. Social desirability bias can be considered equal between the groups and essentially cancel each other out. Phase II Limitations 52 Internal Validity Threats First, sampling for my four in-depth interviews was not random. Instead, I chose a purposive sample (Creswell & Creswell, 2017; Dworkin, 2012). Of the four interviewees, three were selected from a pool of volunteers from the Phase I survey. The fourth was chosen out of convenience due to time constraints on the study because two unknown volunteers failed to attend the scheduled interviews. I chose a geographically local expert who met the same qualifications as the other three interviewees. All four were specifically chosen for their broad understanding and implementation of teaching strategies as expert teachers, regardless of their subject-area expertise. Second, the setting of the fourth interview differed from that of the first three. This interview was conducted in person, as the other three were conducted via video conferencing. Researcher interaction bias should be considered. Third, the overall sample size was on the small end of the acceptable range (Dworkin, 2012). The question of “enough” is often debated in qualitative research, (Creswell & Creswell, 2017; Dworkin, 2012). According to Dworkin (2012) “An extremely large number of articles, book chapters and books recommend guidance and suggest anywhere from 5 to 50 participants as adequate” (p. 1319). Because my survey had two open-ended, qualitative questions, I chose to do only four interviews. It may be fewer than recommended span but not largely out of the scope of “adequate.” This may increase the margin of error but since I was not compelled to use the interview data as a separate set of data. I combined the data from the four interviews with the 27 anonymous responses from the survey to contribute to the overall findings. This gave me a larger set of qualitative responses overall. 53 Also, by triangulating the data from both quantitative and qualitative sources, this small sample size is not likely to have decrease internal validity (Maxwell, 2012; Creswell & Creswell, 2017). Themes overlapped both (a) among the interviews and (b) between Phase I and Phase II data sets. Several qualitative data points were collected in both phases and evaluated as a complete data set; thus, the information gleaned from four interviews suggests satisfactory thematic saturation. Finally, some alterations to transcript data , including written responses to open ended questions on the survey, were made. Slight corrections for grammar, punctuation and/or spelling for readability and identifying features of language and/or geographic landmarks were deleted. External Validity Threats Overall, the outward generalizability of this study may be limited. The overall low response to Phase I and the sampling science in Phase II may not reflect the experiences of all teachers such as first-year teachers or those who pursued non-traditional credentialing pathways. However, the issues and concerns brought forward by the experienced, veteran teachers likely encapsulate the overall picture of computer science teachers in the State of Oregon, even though with the current surge in new computer science education standards and curriculum experienced teachers may not speak for all computer science teachers’ experiences.. Therefore, the direction of generalization for this study should be inward, evaluating similarity from the study’s population towards a localized, comparable population. This study does not necessarily make any claims outward from the sample about the full teacher population. 54 Generalizability may also be limited by researcher bias. My values and expectations may have influenced the conclusions of the study (Maxwell, 2012). Creswell, (2017) asserts that all research is biased because the researcher’s interpretation of finding are always shaped by background experiences. However, I endeavored to counter this by editing my survey extensively for neutral language and employing respondent validation techniques throughout my interviews. Sampling for my four in-depth interviews was not random. Instead, teachers were chosen for their broad understanding and implementation of teaching strategies as expert teachers in both CS and EL. These views may not reflect, for instance, views of beginning teachers or teachers who teach CS or EL part-time. Qualitative Validity Threats Two important threats to qualitative research validity are researcher bias and reactivity (Maxwell, 2012). I attempted to ameliorate both my subjectivity and any influence I may have had on the interviewees by addressing the threats directly. The interview protocol begins with an informed consent that clearly outlines the subject’s role, ability to opt out of any questions and my role as a researcher. Researcher bias, also known as subjectivity, addresses the concern that a “researcher’s values and expectations may have influenced the conduct and conclusions of the study” (Maxwell, 2012, p. 124). For example, a researcher may agree with a participant’s answer in an interview and not ask follow-up questions to ensure clarity of the response. The interview process included respondent validation, or getting feedback from participants about their answers, as I conducted the interview. Respondent 55 validation decreases the chances that I would have misinterpreted the data during analysis (Maxwell, 2012). Reactivity is the influence the researcher has the on the setting or sample. I realized that my background as an ESL teacher may have influenced my sample subgroups: EL teachers may have felt an affinity and CS teachers may have felt scrutinized. Therefore, I did not share my teaching background with either group. I shared only that I was a teacher interested in promoting computer science equity in the state of Oregon. Additionally, I was careful in choosing settings that put my subjects at ease, in a setting of their choice at a time that was convenient for them. Finally, Creswell and Creswell (2017) discuss the importance of not making generalizations based on data from a specific time frame. This is critical, because the field of CS is changing rapidly, as new research and funding is available. Thus, the recommendations from this study should be considered an isolated snapshot that describes the factors of CS in Oregon in 2018 and thus the direction of generalization for this study should be inward. Evaluation of similarity should be done via comparables and documenting my process, rather than outward from sample to claims about the full population. 56 CHAPTER III RESULTS As noted in Chapter 1, I designed this dissertation study to answer two research questions, which are repeated here for clarity. Research Question 1a (RQ 1a): Which instructional strategies do EL teachers (EL-T) and CS teachers (CS-T) believe are most likely to increase success for ELs in computer science classes? Research Question 1b (RQ 1b): Are there differences between EL-T and CS-T perception of usefulness and frequency of use of instructional strategies? Research Question 2: How, when, and why would EL-T and CS-T utilize or recommend particular strategies in computer science classes? I organized this Results chapter around each phase of my study. I present Phase I data in four sections: (a) descriptive statistics of the survey sample, (b) RQ 1a, (Questions 1-3), (c) RQ 1b (Question 4 ), and (d) RQ 2 (Questions 5 and 6). Phase II consists of data analysis of the transcripts of four interviews. I analyzed the interviews in the context of the summarized data from the CS: BPIE data, seeking to clarify areas of confusion and/or support consistent information from the survey data. The chapter concludes with a discussion of the results for each research question and a preview of the next chapter. Phase 1: Survey Results Descriptive Statistics of the Sample Study participants were (a) self-defined as computer science teachers (CS-T) and members of the Oregon Computer Science Teacher Association (OCSTA) or (b) ESOL endorsed teachers (EL-T) in Oregon. The purpose of grouping particular teachers 57 together was to investigate if there would be commonalities and/or different of specific issues for each teacher group. As per the research questions, results were analyzed by these two teacher groups. Overall rate of completion was low, with only 24% (n = 58) of the total sample (N = 244) responding to the survey. This means my survey suffers from a 73% nonresponse bias, which lowers its representativeness (Fincham, 2008). The response rate means that more extreme responders can be expected or lack of representativeness of the sample may be present due to lower numbers. Internal reliability of the survey is low when measured by Cronbach’s alpha. Overall, Cronbach’s alpha for the survey is 0.62, indicating a low reliability. This is likely due to the fact that Questions 1-3 do not measure the same things, so comparing them does not provide useful data. Question 4, which is made up of 12 different components, has a slightly higher Cronbach’s alpha, at .73, indicating a fair level of internal consistency. However, the small sample size and other internal validity threats are more likely to blame for low reliability. Further implications of response rate a and internal reliability are discussed in the Limitations section in Chapter 4. Computer science teachers made up the majority of the respondents that identified themselves by subject area, 32 total responses out of 52 (63%). Of the 118 members of the OCSTA, 32 responded (28%) to the survey. Only 19 of 126 (15%) ESOL teachers/coordinators invited to participate completed the entire survey. One respondent reported they were a member of both OCSTA and an ESOL endorsed teacher. This participant is listed in the descriptive statistics as Both. Descriptive information collected on this survey is presented in Table 7. 58 Table 7 Phase I Descriptive Data Variable ESOL CS Both Years Teaching 1-5 3 2 - 6-10 3 10 - More than 10 12 20 1 Grade Span K-5 5 5 - 6-8 1 5 - 9-12 3 22 - Other 10 0 - Currently teaching CS - 24 1 course EL school population Low 9 - - Moderate 6 - - High 4 - - Five respondents did not identify their subject area and one did not give consent to finish the survey, leaving 52 participants. Of the 52 respondents, 100% completed Q1- 3, but only 22 of answered every question on the survey. I chose to not correct for partial completions for two main reasons. First, the only imputation that makes sense in this survey is to substitute no response for Never. Not all teachers surveyed may have had the same opportunities to utilize every instructional strategy listed, therefore, I cannot assume 59 that no response is equal to Never. Second, I acknowledge in the limitations the lack of responses in the two questions that had fewer than half of the respondents and concede that the validity of those results are questionable. Of the 32 CS teachers, 63% had more than 10 years of teaching experience and 69% taught at the secondary level. The majority of respondents (75%) were teaching computer science courses such as AP Computer Science, Intro to Computer Science, Python and Web Design at the time of the survey. Of the 19 ESOL teachers, 63% taught for more than 10 years. The majority (53%) of ESOL teachers taught a grade span not presented, primarily broken down by primary grades (K-6), secondary only (7-12) or full span (K-12). Even though EL school populations varied, the majority of respondents, 9/19 (47%) indicated a “low” population of ELs in their school. Only 4 out of 19 teachers reported an EL population larger than 25%. For statistical analysis, I eliminated the Both category by counting the responses in both EL-T and CS-T data sets from the one teacher identified as Both. These treatments were intended to provide a more cohesive view of the set of data collected here. Research Question 1a: Most Commonly Chosen Instructional Strategies RQ1 was designed to identify which strategies EL teachers and CS teacher believe are most likely to increase success for ELs in computer science courses. For Questions 1-3, I conducted three separate three by four contingency tables with crosstabs to evaluate whether a statistical relationship exists between my two teacher 60 groups and the respective recommended instructional strategies, as illustrated in Tables 8 through 10. Both EL and CS teachers selected collaboration as the most recommended strategy in Question 1, with teacher group Both selecting project-based learning (PBL). PBL was the second most commonly selected strategy (38%) by the CS group. Scaffolded instruction was chosen most frequently by all three teacher groups in Question 2. Notably, CS-T chose scaffolded instruction 34% of the time and visual aids 33% of the time, a nearly equal percentage. For Question 3, varied modalities was chosen by all three groups as the most recommended EL strategy. Table 8 Q1 Summary of Most Recommended Strategy by Teacher Group Teacher Group Total Strategy EL CS Collaboration 14 15 29 PBL 5 13 18 Direct instruction 0 1 1 Literacy instruction 4 1 5 Total 20 33 53 61 Table 9 Q2 Summary of Most Recommended Strategy by Teacher Group Teacher Group Total Strategy EL CS Vocabulary instruction 3 5 8 Visual aids 3 10 13 Scaffolded instruction 12 12 24 Contextualizing content 3 6 8 Total 20 33 53 Table 10 Q3 Summary of Most Recommended Strategy by Teacher Group Teacher Group Total Strategy EL CS Varied modality 10 20 30 Computational 6 12 18 thinking tasks (CT) Speaking/Listening 4 0 4 Lecture with 0 1 1 PowerPoint Total 20 33 53 Chi-square test of independence was calculated using SPSS 24.0, presented in Table 11. I compared the frequency of each strategy as selected by (a) EL and (b) CS teachers. Because more than 20% of the cells had expected frequencies less than 5, Fisher’s exact test was applied to calculate independence (Shan & Gerstenberger, 2017). 62 No statistically significant interaction was found between teacher group and preferred instructional strategy (c2 = 3.15, p =.36; c2 = 3.13, p = .39; c2 = 6.91, p = .052), although different modalities proportions (67% versus 50% respectively) approached statistical significance (Corder & Foreman, 2014, p. 60). Table 11 Chi-square Distribution Significance by Question Strategies by Question c2 p* 1 Collaboration, PBL, DI or Literacy 3.15 .36 2 Vocabulary, Visuals, Scaffolding, or Contextualing 3.13 .39 3 Different modalities, CT, Listening/Speaking, Lecture 6.91 .05** Note: * Significance calculated using Fisher exact test., **p = .052. Research Question 1b: Perceptions of Usefulness as Measured by Frequency of Use To determine if there are differences in perception of usefulness between EL-T and CS-T, I conducted a crosstab analysis of Question 4, using the Fisher exact test. Because more than 20% of the cells had expected frequencies less than 5, the Fisher exact test was applied to calculate p values and determine significance. Even though the sample is small, the Fisher exact text allows for analysis of discrete data from small, independent samples (Corder & Foreman, 2014). I chose to examine the data by subsections for this question, utilizing the crosstabs method to look for relationships within the data that may not be readily apparent when viewing the data holistically (Kent State University Libraries, 2019). For additional clarification of results from Question 4, I collapsed the Never and Rarely answers into one category to represent Rarely/Never, since there were only two Never responses total. 63 Participants were not required to provide answers to each question, in order to increase overall response rates (Dillman et al., 2014). Instances of item nonresponse vary between questions, with response rates ranging from 52.7% to 100%. Statistical analysis was based on actual responses, with no imputation methods used to correct for nonresponse variables. Item nonresponse bias was discussed further in the Limitations section of Chapter 2. Five strategies were found to be statically significantly used more frequently by EL-T than CS-T based on this sample: vocabulary instruction (c2 = 19.98, p < .01), literacy skill instruction (c2 = 14.39, p =.01), teaching listening and speaking skills (c2 = 20.83, p < .01), connecting new learning to prior knowledge (c2 = 12.08, p = .04), and contextualizing content to the real world (c2 = 8.82, p = .03). There were no significant interactions between teacher type and frequency for the other seven strategies. Results are presented in Table 12. Collaboration frequency, which was most recommended in Q1, was used Frequently to Every Day by 93% (12/13) EL-T respondents, and 63% (10/16) of CS-T respondents. Of the CS-T, 35% integrate collaborate Never to Rarely. Of note, this was the least answered question on the survey, with 52.7% of total respondents answering. There was no significant difference by teacher group for frequency of Scaffolding instruction, reported as the most recommended in Q2. EL-T reported it used Every Day. However, CS-T reported the same rate of implementation- Every Day (38%) and Frequently (38%). Three CS-T (9%) either Rarely or Never scaffold instruction, compared to zero EL-T. 64 Table 12 Strategy Use Frequency by Teacher Group with Chi-Square Significance Frequency x Frequency c2 p* Teacher group Rarely Often Frequently Every Day Collaboration activities 4.23 .24 EL 1 0 5 7 CS 2 4 5 5 Direct instruction .44 .96 EL 1 5 8 5 CS 2 9 15 7 Vocabulary instruction 17.46** <.01 EL 0 1 7 11 CS 5 12 11 3 Literacy skill 10.90** .01 instruction EL 0 4 5 10 CS 8 9 3 6 Lectures with 5.99 .08 PowerPoint notes EL 15 2 0 0 CS 13 6 4 2 Using visual aids 2.92 .41 EL 0 2 5 12 CS 1 9 7 15 Varied Modality 6.12 .08 EL 0 2 8 9 CS 2 7 18 5 Scaffolding instruction 3.00 .39 EL 0 2 6 11 CS 3 6 12 12 Speaking & listening 18.07** <.01 skills EL 0 4 4 11 CS 12 8 5 3 Project based learning 5.51 .14 EL 4 5 6 4 CS 1 6 13 13 65 (continued) Table 12 (continued) Prior knowledge 9.05** .02 EL 0 1 1 17 CS 2 4 11 15 Contextualize content 7.30** .02 EL 0 0 5 14 CS 0 6 14 12 Note: * Significance calculated using Fisher’s Exact test. ** p <.05 No statistically significant differences appeared between teacher groups using different modalities, the most frequently recommended strategy in Q3. However, analysis indicated that EL-T are more likely to change instructional modality more frequently. For EL-T, 47% reported Every Day use of this strategy and 43% Frequent use. Only 16% of CS-T implemented it Every Day and 56% implemented it Frequently. Based on the literature driving this study, vocabulary instruction, literacy skill instruction, listening and speaking skills, and connections to prior knowledge are primarily EL best practices. For all four, EL-T were significantly more likely to use these strategies on a Every Day or Frequently basis. However, contextualizing content and real-world application was an overlapping strategy between EL and CS literature. Data analysis indicates a significant difference of self-reported frequency between EL-T and CS-T in connecting curriculum to the real world. Nearly three-fourths of the EL-T in the survey reported contextualizing learning Every Day. Only 38% of CS-T did the same and nearly 20% report using this strategy only once a week. Research Question 2: How, When, and Why Are Strategies Used/Recommended For Questions 5 and 6 in the survey, optional open-ended questions, I analyzed the answers through the (a) how, (b) when, and (c) why lenses, consistent with RQ 2 66 (Appendix F). Themes from the qualitative data were identified and compared by teacher group, see Figure 2. I then compared themes from the quantitative and qualitative data from the survey to find the main messages that emerged. Summarized data by theme and question stem are presented in Figure 2. Figure 2. Summary of qualitative data from Q5 and Q6. Overall, 30 teachers responded to the optional questions. I sorted comments by (a) how, (b) when, and (c) why teachers utilized instructional strategies, summarized in Figure 2. Some single responses address more than one category and were counted in each area, so total comments recorded add up to more than 30. Four responses were identified as relevant but did not address the research question directly. These responses are discussed in more detail in the Discussion chapter. Six responses addressed issues out of the scope of this study or were irrelevant to the study. For an example of such a response, one participant’s response was “Go Beavers!” referencing the University of Oregon’s rival school, Oregon State University. How do teachers utilize instructional strategies? A total of 20 responses were categorized as “how” questions. The use of explicit curriculum or instructional models was mentioned 50% of the time, the most commonly mentioned way instructional 67 strategies are employed. CS-T mentioned a specific instructional model almost twice as often (41%) as EL-T, including frameworks such as the gradual release of responsibility and a flipped classroom and curriculum such as Exploring Computer Science. Combining strategies was the second most common response in this category, mentioned in 30% of the “how” responses. Teachers reasoned that ”students need a variety of strategies” and “multiple strategies work for most learning styles.” Vocabulary instruction and time and effort investment by teachers were each mentioned twice. When do teachers utilize instructional strategies? A total of 11 responses addressed the time question. When analyzing the responses, 45% of them stated the word daily or every lesson/day. EL-T were more likely to use a strategy daily, but marginally so. Overall, four of 11 response, equally chosen by both teacher groups, noted that strategies were chosen specifically for students. For example, one teacher said they “choose the strategies that will reach the majority of the students.” Two teachers suggested they chose their strategies based on the project or assignment. Why do teachers utilize instructional strategies? Overall, 21 total responses from both teacher groups indicated a reason for choosing a specific strategy. Six different themes emerged from this category: (a) reduce challenge or increase success for students, (b) provide real world/authentic instruction, (c) increase student collaboration skills, (d) increase student engagement and (f) ensure students’ overall future success. The majority (38%) of all the responses implied a desire to reduce the challenge or stress for students and increase the chances for success. One CS instructor said they break large projects into smaller pieces so “students do not become frustrated and feel the 68 task is not possible” and an EL teacher said students “need to know what they are supposed to be able to do.” Connecting instruction to real-world applications was the second most popular theme, with three of the four responses from CS-T. All three CS-T responses indicate that authentic learning experiences mimic real-life applications, such as wanting “students to see real-world relevance in what they are doing.” Three other reasons both groups of teachers chose particular instructional strategies emerged to a lesser degree: (a) increasing collaboration (3/21), (b) to meet the needs of diverse student populations (3/21), and (c) increasing student engagement (2/21). One EL teacher mentioned he/she choses strategies to ensure future success so students are “able to fulfill their hopes and dreams.” Phase I data results indicate that while both teacher groups show knowledge of best practices, as reflected in Q1-Q3, implementation is less consistent. Because my statistical power was low and I had fewer than 20 respondents in my EL-T group, I also evaluated the results by distribution across frequencies in Q4. Ten strategies from Q1-3 and Q4 overlapped. I utilized computational thinking tasks as an option in Question 3, but this was replaced by prior knowledge in Question 4, as EL teachers would not likely use computational thinking tasks in their practice. I compared the non-overlapping strategies first, and then looked more closely at the ten overlapping strategies. A comparison of results based on total count of responses per question in Q1-Q3 were compared to Q4 Every Day total responses, as illustrated in Tables 13 and 14. 69 Of the overlapping strategies, recommendation and frequencies appeared dissimilar. Collaboration was most recommended by EL-T, but frequency results indicate it was just as likely to be used daily as direct instruction, which was not recommended by any EL-T. Only six of the 29 teachers who recommended collaboration in Q1 indicated they use collaboration Every Day. Also, even though varied modality was highly recommended by both EL-T and most recommended by CS-T, for daily use the strategy is ranked 8th by both teacher groups. Overall, neither teacher group described that they used the top three recommended strategies daily. Table 13 EL Teachers Recommended Strategies versus Every Day Frequency Recommended Strategies Most Frequently Used Daily 1. Collaboration 1. Prior knowledge* 2. Scaffolded instruction 2. Contextualizing content 3. Varied Modality 3. Visual aids 4. Computational thinking tasks* 4. Vocabulary instruction Speaking/Listening skills Scaffolded instruction Project-based learning Listening/Speaking skills Literacy Instruction 7. Literacy Skill instruction 8. Vocabulary instruction 8. Varied modalities Visual aids Collaboration Contextualizing instruction Direct instruction 11. Direct instruction 11. Project based learning Lecture with PowerPoint 12. Lecture with PowerPoint Note:*Non-overlapping strategies from CS:BPIE 70 Table 14 Computer Science Teachers Recommended Strategies versus Every Day Frequency Recommended Strategies (total count) Most Frequently Used Daily 1. Varied Modality 1. Prior knowledge* 2. Collaboration Visual aids 3. Computational thinking tasks* 3. Project based learning Project-based learning 4. Contextualizing content 5. Scaffolded instruction Scaffolded instruction 6. Visual aids 6. Direct instruction 7. Contextualizing content 7. Literacy skill instruction 8. Vocabulary instruction 8. Varied modalities 9. Direct instruction Collaboration Literacy instruction 10. Listening/Speaking skills Direct instruction Vocabulary Instruction 12. Lecture with PowerPoint 12. Lecture with PowerPoint Note:*Non-overlapping strategies from CS:BPIE An examination of computational thinking tasks and prior knowledge strategies showed some consistency between the teacher groups. Computational thinking tasks (CT) were the third most recommended instructional strategy for ELs, tied with project-based learning, and CT tasks were the fourth most recommended strategy by EL-T. Prior knowledge was the most frequently used instructional strategy of those presented and, even though it was statistically more likely to be used every day by EL-T, 15 of 32 CS-T indicated they use student’s prior knowledge Every Day, and 11 used it Frequently. Overall, assessing and capitalizing students’ prior knowledge appears to be used most frequently by both teacher groups. 71 In the next section, I present data analysis from Phase II, in which I conducted four interviews. I analyzed the data set from these interviews to further explore the inconsistencies between the recommendations and use of strategies and examine the role curriculum and instructional frameworks may play in strategy implementation. Phase II: Interview Results In this section, I present findings from four transcribed recordings of teacher interviews, conducted in January 2019. First, I describe the interviewees and review the protocol. Because there were only four interviews, I elected not to code data, but instead to apply the data from the transcriptions to the themes that emerged from my Phase I data collection. Through this lens, I provide analysis of the interview transcriptions as related to explanation of (a) the inconsistencies between recommendation and use of strategies and (b) the role curriculum and instructional frameworks may play in strategy implementation. Finally, I summarize key findings from the interviews. Descriptive Statistics and Protocol All four of my interviewees were teachers originally trained in a subject area other than computer science and teaching experience ranging from six to 15 years. Both computer science teachers were recommended by a member of the Oregon Computer Science Teachers Association, for experience and current teaching assignment. Four EL district-level Teachers on Special Assignment (TOSAs) volunteered via the CS:BPIE. Three of the four declined interview invitations and an alternate was selected based on availability. One EL TOSA coached teachers K-12 and the other 72 coached 7-12. Both EL TOSAs had technology experience, but neither taught any sections of computer science. Analysis of Interview Transcriptions Overall there was general thematic consensus between data generated from the four interviews. Overall, interviews also provided some insight and possible explanations for the difference between recommended strategies and frequency of use. In addition, the interviews reflected the qualitative data collected in Phase I. Recommendation versus frequency of strategies. In regard to (a) collaboration, (b), scaffolded instruction and (c) varied modalities, interview data supported the recommendation data from the survey. However, interview data did not support the results around recommendation and frequency of CS teachers connecting to student’s prior knowledge; there was no difference between teacher groups. The EL-T group also suggested that the teacher credentialing path may play a role in the survey results as a whole. All four interviewees indicated that collaboration is a part of their classroom and they would recommend it. Both EL-T said they recommend collaboration daily. It was less cohesive between the CS-T interviewees; one CS-T described using it “every day, every class” and the other CS-T described using collaboration “50% of the time.” Both EL-T expressed the survey results support their professional observations: they see little collaboration happening in non-EL classrooms, particularly in classrooms with computers. One said, “I don’t see collaboration happening a lot anywhere. When you’re walking into a class where there are computers, everyone is plugged in.” This is not to say that what EL-Ts report seeing indicates that there is not collaboration 73 happening. Students may have been collaborating online, i.e. providing synchronous or asynchronous collaboration or reviewing peers’ work or even chatting digitally. Thus, defining collaboration may be tricky and may be tied to area of expertise. All four teachers defined collaboration somewhat differently, from paired programming to sharing journal entries to students talking together. One of the EL-T shared, “it’s tricky because I think lots of people have different definitions of what collaboration is or what it looks like.” Even though all four teachers loosely described collaboration as intentional student-to-student interaction, further clarification about nuanced collaboration, such as asking if I meant pairs or groups, was asked for by three of the four interviewees. One CS teacher expressed her confusion about this ambiguity: I don’t know if what you're calling [collaboration is] the sense of creating something together-- which I don’t know if that always needs to happen. Meaning the product that they’re creating doesn't necessarily need to have both of them but you can have individual projects where you share it and give feedback to each other. Assessing and grading collaborative work also came up as a barrier for both CS-T and one EL-T. All three expressed concern about assessment of collaborative projects; two asked me for advice. One CS teacher offered, “I haven’t found a good way to gauge each person’s understanding with collaborative model, so that's when I switch to individual so everyone can show how much they understand.” Varied modalities was the third most recommended strategy in the survey and one of the least frequently utilized strategies from survey data, as per Tables 13 and 14. All four interviewees agreed that varying modalities is important for student learning and engagement: “if you learn visually but [teachers] only use visual clues once a week, what’s your student experience on the other three days?” One CS teacher varied 74 instruction regularly, and the other described following the curriculum closely and “it is generally just one modality per lesson.” EL teachers both felt a “variety of modalities” should be regularly utilized in every lesson but recognized that curriculum may inhibit that ability. Interviewees included direct instruction in this category, each mentioning it at least once during the interview in relation to using varied modalities. Interviewees saw direct instruction as a type of modality, as a view of teaching as “the telling of what I know” and the expectation that the teacher is an expert who is “talking and giving directions.” One EL-T and one CS-T offered that teachers being “creatures of habit” that focus on their own “learning paradigm” may be partially to blame for the lack of frequency. Other strategies mentioned were hands-on/kinesthetic (4), using visuals (3), and SIOP strategies (1), generally referencing a variety of interaction, questioning and lesson delivery techniques as outlined by Echevarria et al.(2004). Scaffolding was universally the most agreed upon strategy used and recommended by all four teachers, even though it was ranked fourth in self-reported frequency in the survey. Both EL-T mentioned a specific scaffolding model: the gradual release of responsibility. However, neither CS-Ts mentioned any specific frameworks. The two CS-T interviewees both felt that their curriculum was well-designed in this aspect. Both felt scaffolding was important and a sign of “good teaching.” The most experienced CS-T explained why she scaffolds her instruction regularly: I think that what works for me is that I do it all the time basically. It's not only for the ELL students it’s for all the students because sometimes I do have SPED students in the class, I have students with 504 and so on, so I found this strategy almost universally useful at all times. So I would say this is one of the most strategies that I use consistently. Almost all the time. 75 One interesting tension arising in Phase II was scaffolding on the micro and macro level. Both CS teachers said introductory courses require more scaffolding than higher-level courses. One CS-T summarized this: “By the time they are in AP Computer Science, my job is to just bring up the subject to them. Sometimes I do a quick refresher of what we did before … but it's not by any means the same as what I do with the intro class.” This CS-T indicated that there were a few ELs in the higher-level CS classes but did not indicate whether or not those students received any scaffolded instruction. Both CS-T use the same published CS curriculum in their introductory level courses, in which individual units are scaffolded on each other, building complexity through the course. Both CS-T have ELs in these introductory level courses. The EL teachers interviewed agreed with the micro and macro scaffolding idea, mentioning the arc of scaffolding across a pathway from introductory to advanced course work. However, one EL-T did clarify that scaffolding is necessary for EL students and “needs to be part of every lesson for high quality instruction. And to be able to make the content meaningful and comprehensive to our ELL students.” Other useful strategies were mentioned by the interviewees. While survey data indicates that EL-T are more likely to capitalize on students’ prior knowledge, both CS-T explicitly mentioned integrating student’s prior knowledge. The regular use of journal prompts to begin class for one CS-T is an effective way to “connect with something in their own life” and promote student connection to the curriculum, particularly at the introductory level. Both EL-T also felt it was important for curriculum to connect to students because “there is so much in computer science that you can relate to.” 76 Because the EL-T interviewees are instructional coaches who work with a variety of schools, they seemed to have a particular district-level view that shed light on some variation that was seen in the survey data. Both EL-Ts described the need for professional development for all teachers in student pairing strategies, literacy instruction and making instruction meaningful for students. Both noted that teachers with less training were less likely to provide high-quality instruction, and both asked whether or not the CS survey respondents were credentialed teachers. One EL-T who works with secondary teachers mentioned CTE teachers specifically: The big problem I see in CTE in general- here that includes computer science- are people who come from industry who … haven’t been through a teacher education program…they tend to not have a thorough understanding of [what] the job of teaching is – they don’t even know what good instruction looks like period. And so, sometimes the instruction is such a train wreck the whole idea about talking to them about instructional strategies for ELLs—the language is transparent to them. They are fish in water who don’t know they are wet. Role of curriculum and instructional frameworks. No survey questions explicitly asked teachers about curriculum or instructional frameworks. However, answers to Questions 5 and 6 from the survey indicated that curriculum and instructional frameworks may determine for some teachers a great deal about how and when strategies are implemented and utilized. To investigate this gap further, interviews were analyzed though this lens. Interviews supported the survey findings and presented teacher- experience as an additional factor. Both CS-Ts utilize commercial curriculum, one CS-T uses AP CS curriculum published by The College Board and both use Exploring Computer Science (ECS). Implementation may be experience-dependent. The shared introductory computer curriculum, ECS, integrates several instructional strategies that were presented in the 77 survey. The newer CS-T, who is “following the… curriculum closely” was easily able to identify which strategies she uses less frequently because they “aren’t in the curriculum.” She attributes her adherence to the curriculum on newness to the subject area, quick adoption of the curriculum and only having one section of the course, saying “I think as I teach this class and have more time I'll be more skilled at bringing in those different modalities.” The more experienced CS teacher is a full-time CS teacher and has taught the same curriculum a few times. She has modified her curriculum over the years: The first couple of times I taught the curriculum, I gained experience of what works better when and that guided my decisions for the next time I was teaching the class. Having said that, sometimes the makeup of the class- I might have more freshman than seniors or more upperclassmen than freshman or sophomores- so I may change a little bit. But it's mostly decided on subject by subject basis. An EL teacher, the most experienced teacher in the interview sample with nearly 20 years of teaching experience, suggested that not only is it critical to embed EL instructional strategy into curriculum, it may be virtually impossible for newer teachers to find ways to integrate these strategies into pacing, assessment and or instructional plans. If what [the curriculum is] telling you to do is keeping you on the right track, then you’re more likely to do it and more likely to try out those strategies and make them be part of your practice. If what you’re being given doesn’t have that, it’s not that it’s impossible for you to figure out those strategies and embed them, but it requires a lot of motivation and that teacher to be passionate about that and be willing to try things. It’s much harder, because you basically have to rewrite your curriculum. Overall, there was a general thematic consensus between data generated in Phase I and Phase II: theoretically, both teacher groups have an understanding of which EL instructional strategies would likely be most useful in CS classroom. The next chapter will present the conclusions of the study by research question, limitations of the study, and discuss future research needed in this field. 78 CHAPTER IV DISCUSSION AND IMPLICATIONS In this chapter, I first present a summary of my study findings and discuss how my findings contribute to the literature on English learners (ELs) in computer science (CS). I will then discuss limitations of the study and how the limitations can inform future research. I then discuss some specific practice and policy implications at the state, district and school levels. I conclude with my plan for dissemination of the findings from this research study aimed at supporting equity in computer science programs. Discussion of Results by Research Question In this section, I present an integrated summary about findings by each research question. The summary for each research question integrates results from each phase of my study. Finally, I conclude with an overall summary including how results from each research question overlap and provide a rationale for the study’s conclusions. Research Question 1a (RQ 1a) The data from Phase I and Phase II provide answers for RQ 1a: Which instructional strategies do EL teachers (EL-T) and CS teachers (CS-T) believe are most likely to increase success for ELs in computer science classes? Quantitative and qualitative data from this study suggest both teacher groups recognize collaboration, varied modalities and scaffolded instruction as strategies for ELs in computer science classes. My findings are consistent with the literature review. Collaboration was the most frequently mentioned overlapping strategy in the literature review, mentioned ten times in total (August et al., 2009; Bravo & Cervetti, 2014; Denner et al., 2014; Echevarría et al., 79 2004; Fessakis et al., 2013; Fronza et al., 2017; Garza et al., 2014; Israel et al., 2015; Sentance & Csizmadia, 2017; Soh et al., 2007). Scaffolded instruction (Case, 2002; Cervetti, Kulikowich, & Bravo, 2015; Echevarría et al., 2004; Israel et al., 2015; Sengupta et al., 2013; Sentance & Csizmadia, 2017; Soh et al., 2007; Zendler & Klaudt, 2015) and varied modalities (August et al., 2009; Bravo & Cervetti, 2014; Echevarría et al., 2004; Flanigan, Peteranetz, Shell, & Soh, 2017; Fronza et al., 2017; Garza et al., 2014; Sentance & Csizmadia, 2017; Zendler & Klaudt, 2015) were mentioned eight times each. These three strategies emerged from the literature as the three most frequently recommended from my literature review. All three strategies are features of the Sheltered Instruction Observation Protocol, SIOP, strategies for EL student success (Echevarria, Richards-Tutor, Canges, & Francis, 2011; Echevarría et al., 2004; Moughamian et al., 2009; Short et al., 2011). Also, the findings are supported by Takanishi and Le Menestrel’s (2017) consensus study report of EL strategies and programs. Of nine promising and effective practices for English Learners in grades 6-12 presented in their report, two are directly supported by this study: (a) provide ELs access to core curriculum through scaffolded instruction and (b) use collaborative, peer learning communities. Research Question 1b (RQ 1b) Results from RQ 1b are less consistent between the two teacher groups, Are there differences between EL-T and CS-T perception of usefulness and frequency of use of instructional strategies? Quantitative data analysis for my sample, admittedly small, suggests that the EL teachers more frequently used EL domain-specific strategies such as teaching speaking and listening, vocabulary, and literacy skill instruction as compared to 80 the CS teachers in the sample. This aligns with findings from my literature review; no CS studies reported any findings for speaking and listening, vocabulary or literacy skill instruction for any students, including ELs. However, EL teachers in my sample also were statistically more likely to contextualize the curriculum. I expected to find the opposite, with CS instructors more likely to contextualize. In my literature review, four CS studies (Fronza et al., 2017; Sengupta et al., 2013; Sentance & Csizmadia, 2017; Zendler & Klaudt, 2015) reported contextualized learning as a component of student success in CS classes. Survey results showed 38% of CS teachers in my sample claimed to contextualize content Every Day, compared to 74% of EL teachers. Nearly 20% of CS teachers reported using this strategy Never/Rarely, compared to no EL teachers making this claim in my sample. Qualitative data from the open-ended questions and Phase II interviews did not offer a clear explanation but suggested perhaps a theory-practice gap with CS teachers as a result from a lack of clear training in implementation of CS instructional strategies. Quantitative and qualitative data suggest that frequency of collaboration ranged considerably between the groups sampled. Based on the literature review of EL and CS instructional strategies, I expected to see collaboration as both a highly recommended and frequently used strategy for both groups of teachers, as it was the most recommended overlapping strategy (August et al., 2009; Bravo & Cervetti, 2014; Denner et al., 2014; Echevarría et al., 2004; Fessakis et al., 2013; Fronza et al., 2017; Garza et al., 2014; Israel et al., 2015; Sentance & Csizmadia, 2017; Soh et al., 2007). Collaboration was most recommended in Q1 on the survey, was used most frequently Every Day by EL-T and was recommended for daily use by three of four interviewees. Overall, Phase I results 81 showed collaboration was used three times per week or more by 76% of all respondents. However, CS-T frequencies were nearly evenly distributed between Every Day, Frequently and Often, and 13% used collaboration Rarely/Never. Of note, also, this question was the least answered in the survey, fewer than half of the respondents in both teacher groups answering. However, note that definitions of collaboration seemed to vary, based on interview data, and this should be a topic of future research for the field. For instance, whether online collaboration is included in the definition and in what form is an outstanding question. Forms could include for instance either synchronously through chat or texting, or likely more often asynchronously through postings, reviews, annotation, tagging and other methods for sharing of intellectual and social capital through types of collaboration across students. Hence, the qualitative results helped clarify a reason for the incongruity and perhaps explain the lack of responses in the quantitative data: knowing what teachers consider to be collaboration can be problematic. Even within the literature, there is a wide variation in the strategies defined as collaboration. For example, Sentence and Csizmadia (2017) grouped several strategies under the collaboration theme in their open-ended survey, namely (a) pair work and (b) peer-mentoring, while also keeping the more general term, collaboration. Paired programming was the most mentioned specific collaboration strategy from the CS literature (Denner et al., 2014; Fessakis et al., 2013; Fronza et al., 2017; Israel et al., 2015; Sentance & Csizmadia, 2017; Soh et al., 2007). EL literature was less specific. Overall, collaboration is defined broadly as “student interaction” (August et al., 2009, 2014; Echevarría et al., 2004; Garza et al., 2014; Takanishi & Le Menestrel, 2017). An example of a more specific instructional 82 frameworks such as SIOP, recommends that “grouping configurations support language and content objectives” but does not necessarily prioritize or recommend specific structures (Echevarria et al, 2004, p. 225). In the most recent update of curriculum, the College Board (2017) mentions “collaboration” 67 times in the Course and Exam Description: AP Computer Science Principles, defining it both broadly and specifically as brainstorming, working together, providing feedback, providing technical support and suggests specific strategies such as think-pair-shares and pair programming as examples of instructional strategies that support computer science learners. However, collaboration as an intervention itself may present itself in the classroom as a more complicated concept. Teacher knowledge as well as other influences such as classroom culture, school culture, and the overall makeup of the class may influence how, when and why a teacher employs collaboration strategies. Therefore it is not surprising that teachers, experienced and not, in both teacher groups expressed confusion about what types of collaboration seemed to “count” toward what my survey and interview protocol was attempting to assess. This could be clarified in the survey questions with more examples or anchoring cases, such as those provided by the AP CS Principles framework, but still would need to fundamentally answer the question of what structures of collaboration might be able to support EL learners, or would best support especially in a CS context, which has implications for future work. In summary, CS teachers implement all EL-specific strategies less frequently than EL teachers. Of particular note is the rarity of CS teacher utilization of contextualizing content, which is inconsistent with the literature. While quantitative data suggests both teacher groups would recommend collaboration, varied modality, and scaffolding as 83 strategies most likely to be helpful for ELs, the frequency of reported use of these strategies was highly variable between the two teacher groups. Data suggests that even though these three strategies were recognized as most helpful, the majority of CS teachers did not use them every day. This may be an indication of a theory-practice gap with CS teachers. Alternatively, it may also be an indication of either greater awareness or of more socially desirable reporting by EL teachers, or lack of experience by EL teachers about what is possible in CS classrooms. Direct observation or other modes to help further reduce self-reporting bias if it is present are outside the scope of this study but can suggest implications for future work. Research Question 2 (RQ2) A summary of results for RQ 2 indicate there was little difference, beyond the frequency results in RQ 1, between the teacher groups for when and why strategies were chosen, How, when and why would EL-T and CS-T utilize or recommend particular EL strategies in computer science classes? However, descriptions of how CS teachers utilized strategies appeared in the interview results to be captured only by the CS teacher group in their comments. Neither EL teacher interviewed had directly observed a CS course being taught at the secondary level. In the survey, frequency of instructional strategies was highly variable across each category for the CS teacher group. One possible reason for inconsistency here is that CS teachers may be reliant on explicit instructional models or curriculum to guide their instruction. For example, one CS teacher said, “My curriculum is still in development, so I focus more on content delivery than the effectiveness of that delivery.” Interviews in Phase II support this; both CS teachers either are currently or had relied heavily upon a 84 published curriculum. Because the survey did not ask what curriculum teachers were using, I am unable to determine the degree to which this is true for the survey results; this question might be added for future use of the survey, an implication for future work. Also, qualitative data from Phase I and Phase II suggest course level and curriculum used impact the use of instructional strategies for computer science teachers. For example, some teachers may have felt that students in AP Computer Science do not need explicitly scaffolded instruction at the same rate as an Introduction to Computer Science course, regardless of whether they were or were not EL students. This conclusion would not be supported by the research literature for English learners, so this raises questions of whether advanced subjects such as CS need to reconsider instructor approaches, and also the degree to which CS teachers are learning by experience through instructing English learners in advanced courses. For this, new data sources would be needed to better understand enrollment of subpopulations in advanced CS courses. This returns us to the topic of missing data discussed in Chapter 1 and underscores the need for more systematic data about enrollment and other instructional aspects if the goal is CS for all. Deeper examination of CS:BPIE data can help to illustrate, although the sample size here is small. Of the 24 CS-T in my sample who were currently teaching computer science, six taught an AP or advanced computer course(s) only. For those six teachers, results for scaffolding and collaboration are evenly split between (a) once a week or less, and (b) two to three times a week or more. Only one of these teachers uses all three instructional strategies daily. This could be dependent upon which AP computer science 85 course is being taught. For example, AP Computer Science Principles encourages collaboration as one of the big ideas in computer science and computational thinking. Comparatively, three of the four teachers who indicated they teach an introductory course use scaffolding every day. This may account for the discrepancy between recommendation and actual use. In the interviews, the CS-T who taught advanced CS courses indicated that the number of EL students in the advanced courses did not reflect the overall number of EL students in the school; it was far lower. In the research that I reviewed, there were no studies that examined the correlation between computer science course level and scaffolding needed. Further examination of how and why EL students enroll in and stay enrolled in computer science courses may be warranted. While curriculum use was not an explicit component of the Phase I survey, these findings aligned with results from two studies included in my literature review (Bravo & Cervetti, 2014; Echevarría et al., 2004). For example, Bravo and Cervetti’s (2014) study found that teachers in classrooms that included explicit professional development in EL instructional techniques were statistically more likely to have higher scores on science posttests. Similarly, Takanishi and Le Menestrel’s (2017, p. 318-19) consensus study reports that while no specific high school program or curriculum is more or less likely for success for ELs, “specific design elements with related instructional practices made a difference.” Overall, the triangulated results suggest utilization of instructional strategies is varied. Two factors, (a) curriculum and (b) general pedagogical teaching knowledge, 86 emerged as possible influences on how, when and why teachers reported that they choose strategies. Summary of Discussion The results from this study indicate that CS teachers may have the theoretical understanding but less of a practical implementation plan to execute EL instructional strategies. RQ 1a indicated that all teacher groups recommended the same instructional strategies for EL student success: collaboration, varied modality, and scaffolded instruction, although at different frequencies. Results from RQ 1b and RQ 2 highlighted areas of need with regards to fidelity of implementation and practice, by teacher group, with data from the CS teachers , in part perhaps because of the perceived level of student knowledge, the characteristics of the population, and/or use of curricular materials that do not suggest the additional scaffolding or support explicitly in the materials. This may reflect an overall difference in understanding of instructional strategies between the teacher groups in these regards or may reflect differences in self-reporting and knowledge base as well. Social desirability bias is common in Likert-type survey question (Dillman et al., 2014; Messick, 1996). For RQ 1b, I expected a social desirability bias, or “a need for social approval companied by a belief or expectancy that this need can be satisfied by engaging in culturally and situationally sanctioned behaviors” (Marlowe & Crowne, 1961, p. 113). However, teachers may or may not have overcorrected for frequency in collaboration or contextualizing instruction. One possible reason for this could be the Dunning-Kruger effect-where teachers are unaware and uninformed but confident in their practice (Kruger & Dunning, 1999). This may reflect a lack of knowledge in the field. 87 Both CS-T from Phase II indicated that computer science courses exist in their district because of their motivation to seek out professional development, curriculum, and course development. If this is the case state-wide, this teacher-dependent model of instruction is an ideal environment for unintentional Dunning-Kruger effects, as there is no method to check for gaps in knowledge. Overall results from RQ 2 highlight that EL and CS teacher groups desire to take student need into account when deciding when and why to implement instructional strategies, reflecting a sound understanding of theoretical knowledge. However, a reliance by CS teachers on curriculum and instructional pedagogy was an unanticipated finding, as well as that the formal materials might not include as many supports for English learners as might be desirable. This reliance on curriculum and instructional framework may be a sign of a more novice teacher regarding the EL population, and may help explain why inconsistencies between recommendations and use showed up in results for RQ 1b. CS teachers may theoretically know what strategies are most useful for ELs, but not have the opportunity for use due to curricular constraints, perceptions of CS course level, low rates of EL students in advanced courses, or lack of exposure to EL- specific instructional frameworks, suggesting perhaps a knowing/doing gap or a population that is not truly oriented toward “CS for all” in regard to the EL subgroup reviewed here. Also it should be noted that language subgroups within the EL population can be key to some strategies but this topic rarely arose in the interview data set or the literature review for the CS topic. Another reason for a gap between knowledge and practice, if it exists, may be lack of standardized teacher training for CS teachers. Because no credentialing pathway 88 currently exists in Oregon, there is currently no cohesive, state-wide way to ensure that research-based practices and instructional knowledge is being disseminating in a standardized format. It is not only solely based on teacher motivation, but also highly variable in quality of content. Teachers may be instructed by private online companies, such as Code.org, Khan Academy or College Board curriculum, or teachers may be attending workshops funded by an NSF-funded curriculum. There is simply no way to gauge what experience a teacher brings to their computer science classroom on the basis of formal standards in Oregon, since such standards are not in place in this area. A veteran CS teacher summarized the overall lack of resources and information in Oregon: Sometimes I have no clue what I am doing...from a pedagogical perspective. Up until recent years there has been a dearth of information on CS best practices from a teaching and learning standpoint. So most of the training I've participated in has been focused primarily (or solely) on the skills and languages that students should be learning...so much so that I stopped attending them for a time. I'm hopeful with the current surge in CS education that there will be a shift toward instructional best practices and effective pedagogy. As discussed in Chapter 1, there currently has been a national emphasis on computer science in K-12 schools, with attention seemingly placed on opportunity gaps and increasing enrollment of traditionally under-represented groups in computer science. However, I posit that these recruitment strategies will have limited impact on course completion and student success unless the instruction in classes is culturally and linguistically appropriate. This study explicitly investigated through a small sample the needs of one student subgroup, Oregon’s EL student population, in computer science by investigating which instructional strategies would be (a) recommended by both EL and 89 CS experts and (b) at what frequency these research-based strategies are currently reported to be used in these classrooms. Limitations This study acknowledges limitations that should be considered when interpreting results. Phase I limitations include internal validity, construct validity, and reliability threats. Phase II limitations include internal validity and generalizability threats. Refer to Chapter 2 for more detailed summary of limitations to this study. Contributions to Literature This study contributes to the research in computer science education in several ways. As of the date of this dissertation, there is no published study directly investigating how, when and why EL instructional strategies are employed in computer science classrooms in the United States. The results from my literature search indicated there was a need for research in the field of ELs in K-12 computer science, informing my decision to investigate this educational problem further. Primarily, this study contributes a first look at small scale into teacher practice factors and instructional conditions needed for EL student achievement in computer science. My literature review is a new contribution to the field. My literature review identified a scarcity of research in computer science education that included ELs as a population subgroup. I found no computer science studies that examined EL students as the primary population for the CS context. The literature review provides starting point for any future research in EL+CS instructional strategies by connecting the two fields. This contribution to the literature illustrates where EL instructional strategies in content areas and computer science instructional strategies overlap. As is the nature of early-stage 90 research, results from the study are limited, but provide both the literature review and results provide a foundation for further research. As is the nature of exploratory research, the purpose of my study was to establish the basis for the design or development of new interventions. By using a mixed-methods design, and multiple data collection activities—(a) quantitative survey questions, (b) qualitative survey questions, and (c) interviews—I captured both qualitative and quantitative data to establish new knowledge regarding the malleable factors association with learner outcomes. It is the intent of the study to inform further development of interventions that may impact the how, when and why teachers chose instructional strategies to support ELs in computer science classes. While it is beyond the scope of this dissertation to fully describe these strategies in new forms for CS, indications as to what such strategies may broadly consist of can be seen in the themes discussed in the literature and supported in the empirical data in this study. This research projects also increases general understanding about the complex interplay between knowledge and practice in education. Even though, in conception, the study is narrowly focused on EL instructional strategies in a specific content area, overall this study examined the ways in which teacher understanding of efficacy compared to actual use. Examined more generally as a theory-practice gap and unpacked from any bias that may be occurring through self-report, approaches to identification of theory- practice gaps in education can be applied broadly within the teaching field, given the multifaceted and highly nuanced nature of teaching. Finally, this research also supports the idea that there is a systemic problem for ELs in computer science. This speaks to school systems that perhaps limit EL access to 91 computer science, including but not limited to lack of EL training for instructional coaches, school schedules that do not allow EL students to enroll in computer science and endorsing systems of support for ELs in computer science pathways. Research Dissemination The findings from this research project will be shared with several different audiences. First, I will share it with the Oregon Computer Science Teacher’s Association’s (OCSTA) leadership team. This will include a short write up of the study’s main points and perhaps a presentation, depending upon their requests. The findings will also be shared with the Oregon Department of Education. Finally, results from dissertation will be edited based manuscript submission requirements and submitted for publication in a peer-reviewed journal, such as American Educational Research Journal, Journal of Educational Computing Research, Computers in Schools, or Journal of Research on Technology in Education. Implications for Future Research Understanding the needs of ELs in computer science classrooms is an important first step in bridging the equity gap. Results from the study indicate that CS teachers can identify EL strategies but are not reporting to be implementing them at a commensurate rate. Teachers appear to know what strategies to use, but implementation was inconsistent. Perhaps the most pressing next step would be further examination of successful EL students currently in computer science pathways. Case-studies of robust computer science programs at both the secondary and elementary levels, particularly those with diverse student populations could prove useful. These studies collecting additional survey 92 and observation results including teacher and student participants. This data may help further identify the ways students are thriving and inspire replication. This line of research is important because it would further test and explain the findings of this study as well as investigate systems of support that may be necessary for long-term EL success in computer science programs. Next, an intervention to examine of efficacy of the strategies identified in the literature review could prove useful to teachers, not just in CS but perhaps all marginalized students in career and technical education (CTE) fields. Comparing strategies and student achievement data collection could illustrate why some strategies are used more than others. Specific attention to collaboration and cooperative learning in computer science may be the most relevant place to start, as this is an essential component of computational thinking skills in both the AP Computer Science Principles course as well as the K-12 Computer Science Framework (College Board, 2017). Another area of study concerns an examination of computer science curriculum currently in use. This study discovered a surprising reliance on curriculum emerged through the qualitative data. Even the most experienced teachers in this study relied heavily upon the computer science curriculum. More robust qualitative or even quantitative studies evaluating computer science curriculum for EL strategy content and execution may help teachers engage EL students more uniformly and improve the body of materials available. A fourth area for future research would be to study the relationship between institutional supports and/or barriers and the computer science pathways in the state of Oregon, as a localized follow-up to the work by Margolis et al. (2014) and Umansky 93 (2016). Qualitative data from this study indicate that teachers believe institutional barriers, such as access to classes and graduation requirements may unfairly burden the EL population. This systemic look at ELs in CS may provide a clear path for implementing the K-12 Computer Science Framework. An additional implication for future research concerns the pathway to become a computer science teacher. One of the underlying assumptions of this project is that EL students require regular and competent implementation of research-based instructional strategies in all their classes. As computer science programs are funded and demanded by communities, filling these positions with qualified teachers, not just people who are computer science proficient, may become increasingly difficult. It may be well worth the research now to get ahead of this curve. Finally, this study did not examine the diversity of the EL student population nor did it take into account culturally responsive teaching practices. Future studies could examine varied nature of ELs in CS, including those who have exited EL programs and investigate culturally responsive teaching practices in computer science. Implications for Policy and Practice The field of CS education is changing rapidly, as new research and funding becomes available. At this early stage of implementation, the state of Oregon has the opportunity to improve equity for ELs in computer science in completion of computer science programs, not just in enrollment statistics. It is critical to keep the specific time frame of this study in mind to avoid making sweeping generalizations based on the data presented (Creswell & Creswell, 2017). The recommendations from this study should be 94 considered as an isolated snapshot that describes the current factors at play in computer science education in Oregon. Nationally, the College Board does not collect information about language learners who enroll in or take AP tests. This makes it difficult to track any AP computer science data on that level. Therefore, my implications are specific to state and local jurisdictions. State According to Fowler (2014), issue definition is a “political process that involves transforming a problem into an issue that the government can address” (p. 107). The study findings can help the state define the issues surrounding specific teacher credentialing and the overall quality of computer science education statewide. To address the issues of opportunity gap in computer science in the state, policy makers and the Oregon Department of Education should consider initiatives at the state level that address (a) clear pathways for offering Computer Science teaching endorsements that include ESOL instructional pedagogy and (b) clear definitions of computer science coursework and corresponding curriculum. Even though this study may suggest that both CS and EL teachers have the same knowledge base in this small sample, issues of practice are a concern. The Oregon Department of Education (ODE) does not currently offer a specific credential for computer science teachers. As computer science programs expand state-wide, filling positions for these jobs is critical for successful programs (Margolis, Estrella, Goode, Jellison Holme, & Nao, 2008; Margolis, Goode, Chapman, & Ryoo, 2014). ODE should 95 consider offering clear endorsement plans to ensure high-quality teachers who not only understand the content, but also diverse student learning needs. Second, the Oregon Department of Education (ODE) should also carefully consider which courses quality as computer science and which curriculums to formally adopt in the coming years. My findings illustrate that even veteran teachers are reliant on computer science curriculum. This places an importance on the curriculums adopted and used in classrooms. In order to support a statewide initiative driving design and implementation of computer science standards and courses, curriculum adoption should come after adopting the K-12 Computer Science Framework. Curriculum choices should not only support the computer science standards, but also ensure that the instructional pedagogy includes EL instructional practices from inception. Third, the ODE should mandate the collection of data of CS enrollment by various subgroups, including current and former ELs, in these recognized computer science classes. Only by collecting this data can districts and schools make appropriate determinations of training and funding for fully realized computer science pathways. District School districts are positioned to allocate resources towards curriculum, professional development, and programming that supports quality instruction and equity in computer science programs. Systemic support for ELs must include teacher support, student support and an overall view of computer science pathways as being valuable for all learners. First, districts must allocate time and resources for quality computer science programs to flourish. This may involve providing professional development during 96 school hours, active recruiting of computer science specialists, and reallocation of funds for additional full-time teachers. Second, EL teacher leaders at the district level should be encouraged take interest in supporting computer science programming on a district-wide level. The K-12 Computer Science Framework encourages collaboration and a focus on K-12 education. Third, district personnel in charge of curriculum adoption should ensure that computer science curriculum being used addresses the needs of diverse learners with a specific focus on integrated EL instructional strategies. Finally, professional development in implementation of instructional strategies, with a focus on EL students, should be mandatory for current and incoming computer science teachers. School Computer science programs at the state and district level have not been a priority; thereby, leaving schools to recruit, design and implement computer science programs within their current systems. Qualitative data from this study suggest that individual schools and teachers are at the forefront of implementing these programs; both CS teachers interviewed were the reason that their respective schools offered any computer science classes. Schools must make a commitment to offering equitable computer science pathways. This means allocating time and resources for developing both computer science classes and teachers. This may require purchasing equity-focused curriculum, providing a teacher mentor, or professional development to new teachers. Institutional barriers should be acknowledged and taken into consideration before and during course development and implementation. For example, if math sequencing (Geometry then 97 Algebra II) bars EL students from completing the computer science pathway, alternative math plans should be investigated. Principals and instructional coaches should encourage site-based collaboration among the staff. In our culture of instructional coaching, this study serves as a call to instructional coaches to observe and collaboration with computer science teachers to ensure quality instruction is happening for all students, including ELs. EL and CS teachers and instructional coaches should be encouraged to work together from the development stages of these courses. This is particularly important for teachers who are new to CS, even if experienced in other subject areas. Expert-area collaboration can ensure that EL instructional strategies are embedded into the course from the outset, not added in as a time-consuming afterthought. Conclusion This exploratory study explicitly investigated Oregon’s EL student population in computer science by investigating which instructional strategies would be (a) recommended by both EL and CS experts and (b) at what frequency these research-based strategies are currently reported to be used in these classrooms. Though small, the results of this study revealed discrepancies between recommendation and frequency of use. While all teacher groups recommended the same instructional strategies for EL student success, (a) collaboration, (b) scaffolded instruction, and (c) varied modality, the study highlighted areas of need with regards to fidelity of implementation and practice. Both EL and CS teachers recognize usefulness of three overlapping best practices: (a) collaboration, (b) scaffolded instruction, and (c) use of varied modalities; however, the triangulated results from this study suggest utilization of instructional strategies by 98 CS teachers was highly varied. Two factors, (a) curriculum and (b) general pedagogical teaching knowledge, emerged as possible influences on how, when and why teachers reported that they choose particular strategies. CS teachers may theoretically know what strategies are most useful for ELs, but in practice, these strategies were not reported to be used at a rate of frequency commensurate with research-based recommendations. From this, I inferred CS teachers may have the theoretical understanding but less of a practical implementation plan to execute EL instructional strategies in CS classes. This is especially important in light of the heavy emphasis on collaboration in the AP CSP coursework. Perhaps this is due in part to perceived level of student knowledge, the characteristics of the EL student population, and/or use of curricular materials that do not suggest the additional scaffolding or support explicitly in the materials. Invisible populations such as ELs will not thrive without data to support their inclusion in computer science pathways and the ever-changing workforce. Large scale studies indicate that women who are recruited and exposed to computer science in high school are ten times more likely to major in computer science in college; Black and Latinx students are seven times more likely (Morgan & Klaric, 2007). This study did not report data for ELs; likely because no data is being collected. Currently, Code.org does not collect data on EL student users or provide resources for these students. As discussed in Chapter 1, the national emphasis on computer science in K-12 schools appears to place priority on opportunity gaps and increasing enrollment of traditionally under-represented groups in computer science. This small study illustrates that the opportunity gap for ELs may not only be limited to enrollment barriers; overall a 99 lack of data collection, curriculum, and varied instructional strategies employed in CS classes may all present additional barriers for EL students. To this end, continued research is necessary to shine the light on invisible populations such as ELs to promote successful attainment of a diverse national computer science program truly deserving of the name CS For All. 100 APPENDIX A CS:BPIE SURVEY Computer Science: Best Instructional Practices for ELLs Welcome to the research study! By clicking the button below, you acknowledge that your participation in the study is voluntary, you are over 18 years of age, and that you are aware that you may choose to terminate your participation in the study at any time and for any reason. Please note that this survey will be best displayed on a laptop or desktop computer. Some features may be less compatible for use on a mobile device. Questions 5 and 6 are optional open-ended questions at the end of the survey. I would love to hear why you chose these strategies, but you may skip them if you wish. o I consent, begin the study o I do not consent, I do not wish to participate Skip To: End of Survey If = I do not consent, I do not wish to participate Part A: Think about useful instructional strategies that would have positive impacts on English Learners in a computer science course. Please choose the strategy within each group that you think would have the most impact. Q1: Which of these instructional strategies do you think would be most helpful for second language learners in a computer science class? Definitions here: Collaboration strategies: students working in small groups or pairs; Integrated computational thinking lessons: specific and targeted lessons to teach computational thinking; Direct instruction: teacher delivering information o Collaboration strategies o Project-based learning o Direct instruction o Literacy instruction (i.e. reading and writing) 101 Q2: Which of these instructional strategies do you think would be most helpful for second language learners in a computer science class? Definitions here: Scaffolded Instruction: a variety of instructional techniques used to move students progressively toward understanding and greater independence in the learning process; Contextualizing content: connecting the content of the lesson to "real world" applications o Vocabulary instruction o Visual aids o Scaffolded instruction/scaffolding tasks o Contextualizing content Q3: Which of these instructional strategies do you think would be most helpful for second language learners in a computer science class? / Definitions here: Varied Modalities: providing diverse presentations and experiences of the content during a single lesson; Prior knowledge: information students have already acquired or know o Employing different modalities o Teaching specific computational thinking tasks o Teaching speaking and listening skills o Lecturing with PowerPoint notes 102 Q4 Part B: How frequently do you use these strategies? Never Rarely Often Frequently Every (once a (About (2-3 times day month) once a a week) week) 0 1 2 3 4 Collaboration activities Direct instruction Vocabulary instruction Literacy skill instruction (teaching reading or writing) Lectures with PowerPoint notes Using visual aids Using different modalities in a single lesson Breaking projects/assignments into smaller pieces Teaching speaking and listening skills Project based learning Connecting new learning to prior knowledge Contextualizing content to real world applications Q5: Optional: Thinking about the strategies you use frequently or every day; how and/or why do you chose to implement these? Q6: Optional: Is there anything else you wish to share about your instructional practices? 103 Part C: Please respond to demographic questions. Q7: What grade span do you teach? o K-5 o 6-8 o 9-12 o Other ________________________________________________ Q8L How many years have you been teaching? o Fewer than 2 o 2-5 o 6-10 o More than 10 Q9: Which best describes you? o ESL/ESOL endorsed teacher o Computer science teacher o Both Skip To: Q10 If Q9 = Computer science teacher Skip To: Q12 If Q9 = ESL/ESOL endorsed teacher Q10: Are you currently teaching at least one computer science class? o Yes o No Skip To: Q13 If Q10 = No 104 Skip To: Q11 If Q10 = Yes Q11: What is/are the title(s) of the computer science course(s) you teach? Skip To: Q13 If Q11 Is Not Empty Skip To: Q13 If Q11 Is Empty Q12: How would you describe the EL population at your school? o High EL population ( >25%) o Moderate EL population (10% to 25%) o Low EL population ( <10%) Q13: Are you willing to be interviewed for the study? o Yes o No 105 APPENDIX B EMAIL TO EL-T PARTICIPANTS From: apartsaf@uoregon.edu Sent: To: English Language (ESOL) teachers Subject: Your participation is needed to impact EL policy and practice Dear Prospective Participant, My name is Andrea Partsafas I am a doctoral student from the University of Oregon. I am conducting an anonymous survey about the role English Language instructional strategies in Computer Science courses. You are receiving this invitation because you are identified as an expert in ESOL. You do not need to be an expert in computer science to participate. While traditionally underserved Computer Science students are being studied across the globe, there is very little empirical research about linguistic minority students. You have the opportunity to inform policy and practice with your responses to this survey. This study specifically examines the role of best practices in the context of Computer Science. This survey is short- only 18 questions- and should take approximately 10 minutes of your time. Please answer the questions to your comfort level. Your participation is voluntary with no risks or benefits to you for your participation. By answering “yes” to the first question, you agree to participate in this research. You may print or save a copy of this consent for your records. Feel free to forward the survey to colleagues. To begin the survey, simply click this link: https://oregon.qualtrics.com/jfe/form/SV_bmDZkP1e7jWwud7 Thank you for your consideration. If you have any questions or comments, please contact Andrea Partsafas, Principal Investigator at apartsaf@uoregon.edu, the faculty advisor for this research, Dr. Kathleen Scalise at kscalise@uoregon.edu, or Research Compliance Services at ResearchCompliance@uoregon.edu. Sincerely, Andrea Partsafas Principal Investigator & D. Ed candidate University of Oregon 106 APPENDIX C FOLLOW-UP EMAIL TO EL-T PARTICIPANTS From: apartsaf@uoregon.edu Sent: To: ESOL teachers Subject: EL+CS Survey- Response requested Dear EL Expert, Before Thanksgiving Break, you received an invitation from me to participate in my study. To recap, as a subgroup there is next to no data published or publicly collected about ELs in Computer Science. We know that $4 billion dollars is being spent on Computer Science across the nation- it is our hope that this study might provide guidance to policy makers (and educators) around linguistic equity in those programs. Because this survey is anonymous, we are unable to track who has completed the survey from our invitation list. Thank you if you have already completed the survey. So far, only 10 EL experts have participated; the majority of teachers who have participated are in the computer science field. If you have not yet taken the short survey, we are providing one last opportunity to have your voice heard. The survey closes in one week. If there are others in your building who would like to weigh in on this topic, please feel free to forward them the link. This survey is anonymous and voluntary. There are no risks or benefits for your participation. Your participation is voluntary with no risks or benefits to you for your participation. By answering “yes” to the first question, you agree to participate in this research. Link to Survey: https://oregon.qualtrics.com/jfe/form/SV_bmDZkP1e7jWwud7 Thank you for your consideration. If you have any questions or comments, please contact Andrea Partsafas, Principal Investigator at apartsaf@uoregon.edu, the faculty advisor for this research, Dr. Kathleen Scalise at kscalise@uoregon.edu, or Research Compliance Services at ResearchCompliance@uoregon.edu. Sincerely, Andrea Partsafas Principal Investigator & D. Ed candidate University of Oregon 107 APPENDIX D EMAIL TO CS-T PARTICIPANTS The message below is being shared on behalf of Andrea Partsafas, a doctoral student at the University of Oregon. Oregon CSTA agreed to forward the request to support research on a topic of interest to our community. My name is Andrea Partsafas and I am a doctoral student from the University of Oregon. I am conducting an anonymous survey about the role English Language Development strategies in Computer Science courses. You are receiving this invitation because you are identified as a Computer Science Educator. While traditionally underserved Computer Science students are being studied across the globe, there is very little empirical research about linguistic minority students. You have the opportunity to inform policy and practice with your responses to this survey. This study specifically examines the role of best practices in the context of Computer Science. This survey is short- only 18 questions- and should take approximately 10 minutes of your time. Please answer the questions to your comfort level. Your participation is voluntary with no risks or benefits to you for your participation. By answering "yes" to the first question, you agree to participate in this research. You may print or save a copy of this consent for your records. To begin the survey, simply click this link: https://oregon.qualtrics.com/jfe/form/SV_bmDZkP1e7jWwud7 Thank you for your consideration. If you have any questions or comments, please contact Andrea Partsafas, Principal Investigator at apartsaf@uoregon.edu, the faculty advisor for this research, Dr. Kathleen Scalise at kscalise@uoregon.edu, or Research Compliance Services at ResearchCompliance@uoregon.edu. Sincerely, Andrea Partsafas Principal Investigator & D. Ed candidate University of Oregon 108 APPENDIX E INTERVIEW PROTOCOL Informed consent Hi, my name is Andrea Partsafas. I am a graduate student at the University of Oregon. My research project as a whole focuses on the improvement of teaching and learning, with particular interest in understanding how teachers make decisions about instructional methodology in computer science-oriented classrooms. My study does not aim to evaluate your techniques or experiences. Rather, I am looking to understand how and why teachers decide how to teach. This interview is completely voluntary, and you may say no if you do not want this information used in the study. If you agree and we start talking and you decide you no longer want to do this, we can stop at any time. To facilitate our conversation, I am recording our conversation instead of taking notes today. For your information, only researchers on the project will be privy to the recording, which will be erased after they are transcribed. I will not identify you or use any information that would make it possible for anyone to identify you in any presentation or written reports about this study. If it is okay with you, I might want to use direct quotes from you, but these would only be cited as from a CS or EL teacher. There is no expected risk to you for helping me with this study. There are no expected benefits to you either. Do you still want to talk with me? (If yes, I will proceed - you may take notes if you want. If no, the interview stops immediately.) I have planned this interview to last no longer than one hour. During this time, I have several questions that we would like to cover, but you may choose not to answer any question at any time or to end the interview. Questions 1. Briefly describe your role in your school/district in terms of teaching ESOL or CS. (CS only) Tell me about your instructional strategies. (EL only) Tell me about what strategies you coach teachers to use for EL instruction. Probes: • Why do you use these strategies? • How do these strategies engage students? • When have you found these work best? • What are some of the challenges to using these strategies you’d want to tell others who might want to use them? 109 2. As you may recall from the survey, these are some strategies that I’m interested in for my study. Let’s talk about each of these and how you use them or not in your CS classes/have observed or would imagine are implemented in a CS class. I will start by sharing the findings about the first three strategies listed. {Show strategies} Collaboration Utilizing student’s prior Speaking/Listening skill Different modalities knowledge instruction Scaffolded instruction Use of visuals Literacy instruction Vocabulary instruction Direct instruction 3. One interesting finding from my survey was collaboration, scaffolded instruction and varied modalities were chosen most often by both CS and EL teachers as effective instructional practices in CS classrooms. As a {state their role}, I’d like your thoughts about these findings. • However, there was a range from rarely to daily use of collaboration strategies in CS classrooms- only 15 CS teachers responded to this question, mostly evenly split between sometimes, weekly and daily. That’s a big range. Do you think it’s important for this strategy to be used more frequently among teachers? As your role as {state their role, what are your thoughts about this finding?) How? When? Why? • 17 of 35 (49%) responded saying they utilize different modalities 2-3 times a week – does that surprise you? Do you think that’s sufficient as recommended or is it too much? Why? • 23 out of 35 (66%) responded saying that they used scaffolded instructional strategies to break assignments into smaller pieces several times a week to daily (3 said they do this monthly). What are your thoughts about this finding? {Bring attention back to the rest of the list.} Are there strategies listed here that you don’t use- why or why not? {go through each item and ask yes or no} 110 APPENDIX F RESPONSES TO SURVEY QUESTIONS 5 AND 6 HOW WHEN WHY EL-[vocabulary is] always connect the new EL- vocabulary instruction daily EL- to provide first-hand word with something familiar experience (project-based learning) EL- Whereas I also believe first-hand EL- the strategies I use depend upon the EL- successful in their future lives experience (project-based learning) is the student and to be able to fulfill their hopes best method for engaging students, I and dreams recognize the negative impact of such an experience if proper planning and organization is not first established. EL- using their home language, visual aids, EL- In order to successfully teach ELs it EL- Kids need to know what they technology, and other means is imperative that a variety of are supposed to be able to do. instructional strategies be utilized during every lesson, every day EL- time to practice as a whole group, in EL- I use informal assessment to EL- I also believe first hand smaller groups and on their own determine what skills students need and experience (project-based tailor instruction to meet those needs learning) is the best method for engaging students EL- These are effective strategies teachers at EL- I try to give students the EL- diverse range of students our school district have been trained to use opportunity to read, write, speak, and through Constructing Meaning. listen in English in every lesson. 111 HOW WHEN WHY EL- SIOP, CM, or AVID. Good instruction EL- Every instructional strategy EL- because I want to make sure is good instruction mentioned in the first few questions can my students are learning the be the most important strategy to use. It material and it seems that these is dependent on the lesson itself and strategies, when combined, makes whether it is new material, practice, or learning less challenging review. EL- Multiple strategies work for most CS-- I try to use strategies that reach and EL Students need to work and learning styles. engage the most students, since I have learn together students with a wide range of reading level, language skill, math skill, etc CS- With project-based, however, the CS- With project-based, however, the CS-I believe students learn best teacher must break the project up in parts - teacher must break the project up in by working hands on with the s caffolding and demonstration - parts - scaffolding and demonstration - material. to make sure all students have a clear direction. CS- Students need a variety of strategies, CS- I use structured student talk 2-3 CS- With project-based, however, vocabulary, prior knowledge, visuals, hands- times a day the teacher must break the project on, modeling, etc. up in parts - scaffolding and demonstration - to make sure all students have a clear direction. CS- Repetition is the mother of learning, and CS- choose the strategies that will reach CS- talking to each other about the best way to learn programming is to the majority of the students tasks rather than listening to me practice, practice, practice gives them a much more authentic learning experience and mimics problem solving in the real world. 112 HOW WHEN WHY CS- Projects require a lot of small pieces CS- I choose to implement these CS- Seeing the application or combined together to actually succeed, and strategies daily as they have, in my results make an impression on students are presented material in multiple experience, the most impact on student them for two reasons. Students formats so that it is more accessible. learning can visualize the purpose and their mistakes become more evident with visual aids to guide their progress. CS- it really takes time and effort to CS- practicing talking about their authentically tie them into the curriculum. learning is invaluable, and doing it in a partner is less stressful for them as they try, and master language patterns CS- In Exploring Computer Science we CS- I can connect them to the real work to include inquiry and equity strategies world to make the content accessible to all students. CS- I use a blended learning model and CS- break large projects into cohort collaborative groups smaller pieces so students do not become frustrated and feel the task is not possible. CS- I teach 7th and 8th grade students and CS- In order to address the employ a gamification model for 50% of the multiple learning styles of my class objectives. students. CS- I also like to incorporate a vocabulary CS- I like students to see real- journal, as most students, EL or not, have world relevance in what they are not been exposed to many words in the doing. computer science field. CS- flipped/hybrid model CS- I choose the strategies that will reach the majority of the 113 HOW WHEN WHY students and then choose strategies that will bring up the slow learners CS- Demonstrating the technique, procedure, CS- Collaboration requires concept, or terms with visual aids facilitates students to work together concrete application for the students CS- Break projects into smaller pieces CS- Collaboration is effective for all students, allowing for peer interaction and feedback from all students CS- - Dual coding techniques, I use Blog CS- Collaborating with peers is a Writing to summarize the week's learning. great way to learn in computer science and to see examples/visual aids of the possible products that can be created. CS- I do a lot of I do (direct instruction), we CS- By receiving the information do (partners), you do! in a visual format AND an auditory format, students can process it better CS- Collaboration requires students to work together. This is helped by visuals, and breaking projects down! 114 REFERENCES CITED Aquino-Sterling, C., Rodríguez-Valls, F., & Zahner, W. (2016). Fostering a culture of discourse in secondary mathematics classrooms: Equity approaches in teaching and teacher education for emergent bilingual students. Revista Internacional de Educasion Para La Justica Social, 5(2), 87–107. https://doi.org/10.15366/riejs2016.5.2.005 August, D., Branum-Martin, L., Cardenas-Hagan, E., & Francis, D. J. (2009). The impact of an instructional intervention on the science and language learning of middle grade English langauge learners. Journal of Research on Educational Effectiveness, 2(4), 345–376. August, D., Branum-Martin, L., Cárdenas-Hagan, E., Francis, D. J., Powell, J., Moore, S., & Haynes, E. F. (2014). Helping ELLs meet the Common Core State Standards for literacy in science: The impact of an instructional intervention focused on academic language. Journal of Research on Educational Effectiveness, 7(1), 54–82. https://doi.org/10.1080/19345747.2013.836763 August, D., & Shanahan, T. (2017). Developing literacy in second-language learners: Report of the National Literacy Panel on Language-Minority Children and Youth (First). New York: Routledge. https://doi.org/10.1080/10862960903340165 Babbie, E. (2013). The basics of social research. Boston: Cengage Learning. Baker, S., Lesaux, N., Jayanthi, M., Dimino, J., Proctor, C. P., Morris, J., … Newman- Gonchar, R. (2014). Teaching academic content and literacy to English language learners in elementary and middle School. What Works Clearinghouse, 1–108. Retrieved from http://ies.ed.gov/ncee/wwc/publications_reviews.aspx. Barr, V., & Stephenson, C. (2011). Bringing computational thinking to K-12. ACM Inroads, 2(1), 48. https://doi.org/10.1145/1929887.1929905 Bartram, D. (2007). Increasing validity with forced-choice criterion measurement formats. International Journal of Selection and Assessment, 15(3), 263–272. https://doi.org/10.1111/j.1468-2389.2007.00386.x Barwell, R. (2006). Integrating language and content: Issues from the mathematics classroom. Linguistics and Education, 16(2), 205–218. https://doi.org/10.1016/j.linged.2006.01.002 Ben-Ari, M. (1998). Constructivism in computer science education. ACM SIGCSE Bulletin, 30(1), 257–261. https://doi.org/10.1145/274790.274308 115 Borgioli, G. (2008). Equity for English langauge learners in mathmatics classrooms. National Council of Teachers of Mathmatics, 15(3), 185–191. Bravo, M. A., & Cervetti, G. N. (2014). Attending to the language and literacy needs of English learners in science. Equity and Excellence in Education, 47(2), 230–245. https://doi.org/10.1080/10665684.2014.900418 Brown, A., & Maydeu-Olivares, A. (2011). Item response modeling of forced-choice questionnaires. Educational and Psychological Measurement, 71(3), 460–502. https://doi.org/10.1177/0013164410375112 Calderon, M., Slavin, R. E., & Sanchez, M. (2011). Effective instruction for English learners. The Future of Children, 21(1), 103–127. Callahan, R., Wilkinson, L., Muller, C., & Frisco, M. (2009). ESL placement and schools: Effects on immigrant achievement. Educational Policy, 23(2), 355–384. https://doi.org/10.1177/0895904807310034 Carrier, K. A. (2003). Improving high school English language learners’ second language listening through strategy instruction. Bilingual Research Journal Fall, 27(3), 383– 408. Case, R. E. (2002). The intersection of language, education, and content: Science instruction for ESL students. Clearing House, 76(2), 71–74. https://doi.org/10.1080/00098650209604952 Cervetti, G. N., Kulikowich, J. M., & Bravo, M. A. (2015). The effects of educative curriculum materials on teachers’ use of instructional strategies for English language learners in science and on student learning. Contemporary Educational Psychology. https://doi.org/10.1016/j.cedpsych.2014.10.005 Chang, M., & Kim, S. (2009). Computer access and computer use for science performance of racial and linguistic minority students. Journal of Educational Computing Research, 40(4), 469–501. https://doi.org/10.2190/EC.40.4.e Code.org. (2014). Support K-12 computer science education in Oregon. Seattle, WA. Retrieved from https://code.org/advocacy/state-facts/OR.pdf College Board. (2017). Course and exam description: AP computer science principles. New York, NY. Retrieved from https://apcentral.collegeboard.org/pdf/ap-computer- science-principles-course-and-exam-description.pdf?course=ap-computer-science- principles Corder, G. W., & Foreman, D. I. (2014). Nonparametric statistics: A step-by-step approach (Second). Hoboken, New Jersey: John Wiley & Sons. 116 Creswell, J., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Thousand Oaks, CA: Sage Publications. Daya, S. (2002). Fisher exact test. Evidence-Based Obstetrics and Gynecology, 4(1), 3–4. https://doi.org/10.1054/ebog.26 Denner, J., Werner, L., Campe, S., & Ortiz, E. (2014). Pair Programming: Under what conditions is it advantageous for middle school students ? Journal of Research on Technology in Education, 46(3), 277–296. Dillman, D., Smyth, J., & Christian, L. M. (2014). Internet, phone, mail, and mixed-mode surveys: The tailored design methd (4th ed.). Hoboken: Wiley. Doolittle, P. E., & Camp, W. G. (1999). Constructivism: The career and technical education perspective. Journal of Career and Technical Education, 16(1). https://doi.org/10.21061/jcte.v16i1.706 Dworkin, S. L. (2012). Sample size policy for qualitative studies using in-depth interviews. Archives of Sexual Behavior, 41, 1319–1320. https://doi.org/10.1007/s10508-012-0016-6 Earle, J., & Maynard, R. (2013). Common guidelines for education research and development. National Science Foundation. https://doi.org/10.1080/00220973.2013.813364 Echevarria, J., Richards-Tutor, C., Canges, R., & Francis, D. (2011). Using the SIOP model to promote the acquisition of language and science concepts with English learners. Bilingual Research Journal. https://doi.org/10.1080/15235882.2011.623600 Echevarría, J., Vogt, M., & Short, D. J. (2004). Making content comprehensible for English learners: The SIOP model (2nd ed.). Boston: Allyn and Bacon. Fessakis, G., Gouli, E., & Mavroudi, E. (2013). Problem solving by 5-6 years old kindergarten children in a computer programming environment: A case study. Computers and Education, 63, 87–97. https://doi.org/10.1016/j.compedu.2012.11.016 Fincham, J. E. (2008). Response rates and responsiveness for surveys, standards and the Journal. American Journal of Pharmaceutical Education, 72(2), 43. Fitzner, K. (2007). Reliability and validity: A quick review. The Diabetes Educator, 33(5), 775–780. https://doi.org/10.1177/0145721707308172 117 Flanigan, A. E., Peteranetz, M. S., Shell, D. F., & Soh, L.-K. (2017). Implicit intelligence beliefs of computer science students: Exploring change across the semester. Contemporary Educational Psychology, 48, 179–196. https://doi.org/10.1016/j.cedpsych.2016.10.003 Fradd, S. H., & Lee, O. (1999). Teachers’ roles in promoting science inquiry with students from diverse language backgrounds. Educational Researcher, 28(6), 14–42. https://doi.org/10.3102/0013189X028006014 Fronza, I., Ioini, N. El, & Corral, L. (2017). Teaching computational thinking using Agile software engineering methods. ACM Transactions on Computing Education, 17(4), 1–28. https://doi.org/10.1145/3055258 Garza, E. V., Kennedy, K., & Arreguín-Anderson, M. G. (2014). ESL/SSL strategies that bridge content and language in science: Experiential learning in an environmental education workshop. Journal of Language Teaching and Research, 5(3), 498–504. https://doi.org/10.4304/jltr.5.3.498-504 Genesee, F., Lindholm-Leary, K., Saunders, W., & Christian, D. (2005). English language learners in U.S. schools: An overview of research findings. Journal of Education for Students Placed at Risk, 10(4), 363–385. https://doi.org/10.1207/s15327671espr1004_2 Genesee, F., Lindholm-Leary, K., Saunders, W., & Christian, D. (2006). Educating English language learners: A synthesis of research evidence. Cambridge, UK: Cabridge University Press. Gersten, R., & Baker, S. (2000). What we know about effective instructional practices for English-language learners. Exceptional Children, 66(4), 454–470. Gibbons, B. A. (2003). Supporting elementary science education for English learners: A constructivist evaluation instrument. The Journal of Educational Research, 96(6), 371–380. Retrieved from http://www.jstor.org/stable/27542455 Goldenberg, C., & Coleman, R. (2006). Promoting academic achievement among English learners: A guide to the research. Thousand Oaks, CA: Sage Press. Google. (2015). Images of computer science. Retrieved from https://services.google.com/fh/files/misc/images-of-computer-science-report.pdf Grant, C., & Osanloo, A. (2014). Understanding, selecting, and integrating a theoretical framework in dissertation research: Creating the blueprint for your “house.” Administrative Issues Journal Education Practice and Research, 12–26. https://doi.org/10.5929/2014.4.2.9 118 Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer science course for middle school students. Computer Science Education, 25(2), 199–237. https://doi.org/10.1080/08993408.2015.1033142 Guzdial, M. (2017). Balancing teaching CS efficiently with motivating students. Communications Of The Acm, 60(6), 10–11. https://doi.org/10.1145/3077227 Guzdial, M., & Morrison, B. (2016). Education growing computer science education into a STEM education discipline. Communications Of The Acm, 59(11), 31–33. https://doi.org/10.1145/3000612 Israel, M., Pearson, J. N., Tapia, T., Wherfel, Q. M., & Reese, G. (2015). Supporting all learners in school-wide computational thinking: A cross-case qualitative analysis. Computers and Education, 82, 263–279. https://doi.org/10.1016/j.compedu.2014.11.022 ISTE, & CSTA. (2011). Operational definition of computational thinking. Report, 1030054. Johnson, C. C. (2005). Making instruction relevant to language minority students at the middle level. Middle School Journal, 37(2), 10–14. Retrieved from http://www.jstor.org/stable/23043961 K-12 Computer science framework. (2016). Retrieved from www.k12cs.org Kanno, Y., & Kangas, S. E. N. (2014). “I’m Not Going to Be, Like, for the AP.” American Educational Research Journal, 51(5), 848–878. https://doi.org/10.3102/0002831214544716 Kent State University Libraries. (2019, March 18). SPSS Tutorials: Crosstabs. Retrieved from https://libguides.library.kent.edu/SPSS Kessler, S. (2017). You probably should have majored in computer science. Retrieved October 8, 2018, from https://qz.com/929275/you-probably-should-have-majored- in-computer-science/ Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121–1134. Margolis, J., Estrella, R., Goode, J., Jellison Holme, J., & Nao, K. (2008). Stuck in the shallow end: Education, race and computing. Cambridge, MA: MIT Press. Margolis, Jane, Goode, J., Chapman, G., & Ryoo, J. J. (2014). That classroom “magic.” Communications of the ACM, 57(7), 31–33. https://doi.org/10.1145/2618107 119 Marlowe, D., & Crowne, D. P. (1961). Social desirability and response to perceived situational demands. Journal of Consulting Psychology, 25(2), 109–115. Messick, S. (1996). Validity of performance assessments. In G. Phillips (Ed.), Technical issues in large-scale performance assessment (pp. 1–18). Washington, DC. Morgan, R., & Klaric, J. (2007). AP students in college: An analysis of five-year academic careers. New York, NY. Retrieved from collegeboard.com Moschkovich, J. (1999). Supporting the participation of English language learners in mathematical discussions. For the Learning of Mathematics, 19(1), 11–19. Moughamian, A., Rivera, M., & Francis, D. (2009). Instructional models and strategies for teaching English language learners. Portsmouth, NH. Retrieved from http://eric.ed.gov/?id=ED517794 Mustafa, E., & Fatma, E. N. (2013). Instructional technology as a tool in creating constructivist classrooms. Procedia - Social and Behavioral Sciences, 93(2004), 1441–1445. https://doi.org/10.1016/j.sbspro.2013.10.060 Nulty, D. D. (2008). The adequacy of response rates to online and paper surveys: What can be done? Assessment and Evaluation in Higher Education, 33(3), 301–314. https://doi.org/10.1080/02602930701293231 Papert, S., & Harel, I. (1991). Situating constructionism. In S. Papert & I. Harel (Eds.), Constructionism. New York: Ablex Publishing Corporation. Retrieved from http://www.papert.org/articles/SituatingConstructionism.html Parker, R. M. (1990). Power, control and validity in research. Journal of Learning Disabilities, 23(10), 613–620. Przybylla, M., & Romeike, R. (2014). Physical computing and its scope--Towards a constructionist computer science curriculum with physical computing. Informatics in Education, 13(2), 241–254. https://doi.org/10.15388/infedu.2014.05 Pudyastuti, Z. E., Palandi, J. F., & Kom, M. (2014). A correlation between students’ English proficiency and their computer programming mastery. In The 3rd UAD TEFL International Conference (Vol. 3, pp. 1219–1228). Retrieved from https://www.researchgate.net/publication/277097932 Remler, D. K., & Van Ryzin, G. G. (2015). Research methods in practice: Strategies for description and causation (2nd ed.). Thousand Oaks, CA: SAGE Publications. Resnik, M. (2007). Sowing the seeds for a more creative society. Learning & Leading with Technology, 35(4), 18–22. 120 Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26(3), 482–496. https://doi.org/10.1016/j.tate.2009.06.007 Sengupta, P., Kinnebrew, J. S., Basu, S., Biswas, G., & Clark, D. (2013). Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Education and Information Technologies, 18(2), 351–380. https://doi.org/10.1007/s10639-012-9240-x Sentance, S., & Csizmadia, A. (2017). Computing in the curriculum: Challenges and strategies from a teacher’s perspective. Education and Information Technologies, 22(2), 469–495. https://doi.org/10.1007/s10639-016-9482-0 Shan, G., & Gerstenberger, S. (2017). Fisher’s exact approach for post hoc analysis of a chi-squared test. PLoS ONE, 12(12), 1–12. https://doi.org/e0188709 Short, D. J., Echevarría, J., & Richards-Tutor, C. (2011). Research on academic literacy development in sheltered instruction classrooms. Language Teaching Research, 15(3), 363–380. https://doi.org/10.1177/1362168811401155 Smith, M. (2016, January 30). Computer science for all. Retrieved from https://obamawhitehouse.archives.gov/blog/2016/01/30/computer-science-all Soh, L.-K., Samal, A., & Nugent, G. (2007). An integrated framework for improved computer science education: Strategies, implementations, and results. Computer Science Education, 17(772450343), 59–83. https://doi.org/10.1080/08993400701203782 Staples, M. E., & Truxaw, M. P. (2012). An initial framework for the language of higher- order thinking mathematics practices. Mathematics Education Research Journal, 24(3), 257–281. https://doi.org/10.1007/s13394-012-0038-3 Star, J., Chen, J., Taylor, M., Durkin, K., Dede, C., & Chao, T. (2014). Studying technology-based strategies for enhancing motivation in mathematics. International Journal of STEM Education, 1(1), 1–19. https://doi.org/10.1186/2196-7822-1-7 State tracking 9 policies (Public). (n.d.). Retrieved October 8, 2018, from https://docs.google.com/spreadsheets/d/1YtTVcpQXoZz0IchihwGOihaCNeqCz2Hy LwaXYpyb2SQ/pubhtml Takanishi, R., & Le Menestrel, S. (Eds.). (2017). Promoting the educational success of children and youth learning English. Washington, DC: National Academies Press. Tan, M. (2011). Mathematics and science teachers’ beliefs and practices regarding the teaching of language in content learning. Language Teaching Research, 15(3), 325– 342. https://doi.org/10.1177/1362168811401153 121 The hidden half: Uncovering the invisible computing workforce. (2015). Retrieved from https://www.ecs.org/wp-content/uploads/TheHiddenHalf_infographic.pdf Tindal, G. A., & Marston, D. B. (1990). Classroom-based assessment: Evaluating instructional outcomes. Columbus: Merrill Publishing Company. U.S. Department of Education. (2017). Our nation’s English learners. Retrieved from https://www2.ed.gov/datastory/el-characteristics/index.html#two U.S. Department of Education Office of English Language Acqusition. (2016). English Learner tool kit (Rev. ed.). Washington, DC: Author. Umansky, I. M. (2016). Leveled and exclusionary tracking. American Educational Research Journal, 53(6), 1792–1833. https://doi.org/10.3102/0002831216675404 Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33. https://doi.org/10.1145/1118178.1118215 Wright, T. S., & Cervetti, G. N. (2016). A systematic review of the research on vocabulary instruction that impacts text comprehension. Reading Research Quarterly, 52(2), 203–226. https://doi.org/10.1002/rrq.163 Yadav, A., Gretter, S., Hambrusch, S., & Sands, P. (2017). Expanding Computer Science Education in Schools: Understanding Teacher Experiences and Challenges. Computer Science Education, 26(4), 235–254. https://doi.org/10.1080/08993408.2016.1257418 Yadav, A., Mayfield, C., Zhou, N., Hambrusch, S., & Korb, J. T. (2014). Computational thinking in elementary and secondary teacher education. ACM Transactions on Computing Education, 14(1), 1–16. https://doi.org/10.1145/2576872 Zahner, W., Velazquez, G., Moschkovich, J., Vahey, P., & Lara-Meloy, T. (2012). Mathematics teaching practices with technology that support conceptual understanding for Latino / a students. Journal of Mathematical Behavior, 31(4), 431–446. https://doi.org/10.1016/j.jmathb.2012.06.002 Zendler, A., & Klaudt, D. (2015). Instructional methods to computer science education as investigated by computer science teachers. Journal of Computer Science, 11(8), 915–927. https://doi.org/10.3844/jcssp.2015.915.927 122