SCREENING PRE-KINDERGARTEN STUDENTS FOR APPROPRIATE LITERACY INSTRUCTION by THOMAS J. CHARBONEAU A DISSERTATION Presented to the Educational Methodology, Policy, and Leadership Department and the Graduate School of the University of Oregon in partial fulfillment of the requirements for the degree of Doctor of Education June 2020 DISSERTATION APPROVAL PAGE Student: Thomas J. Charboneau Title: Screening Pre-Kindergarten Students for Appropriate Literacy Instruction This dissertation has been accepted and approved in partial fulfillment of the requirements for the Doctor of Education degree in the Educational Methodology, Policy, and Leadership Department by: Gerald Tindal Chairperson/Advisor Leilani Sáez Core Member S. Phillip Irvin Core Member Lillian Duran Institutional Representative and Kate Mondloch Interim Vice Provost and Dean of the Graduate School Original approval signatures are on file with the University of Oregon Graduate School. Degree awarded June 2020 ii © 2020 Thomas J. Charboneau iii DISSERTATION ABSTRACT Thomas J. Charboneau Doctor of Education Educational Methodology, Policy, and Leadership June 2020 Title: Screening Pre-Kindergarten Students for Appropriate Literacy Instruction Current literacy risk identification assessments at the beginning of kindergarten are typically unable to distinguish students who may need additional intense literacy interventions to learn foundational reading skills from those who will make typical growth in literacy as a result of classroom instruction. In contrast, considerable research has attempted to identify literacy risk in pre-kindergarten students, but the assessments are time and resource consuming to administer. In this study, 75 preschool students were assessed using a benchmark preschool assessment that focused on literacy and then predictive validity was explored through the middle of kindergarten. Children’s literacy assessment performance was correlated with their results on later kindergarten assessments to determine if the preschool assessment predicted later risk in literacy. This study provides evidence that the measures are valid and reliable as an early and efficient identifier of students who may benefit from early literacy interventions as they enter kindergarten. iv CURRICULUM VITAE NAME OF AUTHOR: Thomas J. Charboneau GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene California State University, San Bernardino University of California, Riverside DEGREES AWARDED: Doctor of Education, 2020, University of Oregon Master of Science, Educational Leadership, 2004, University of Oregon Bachelor of Arts, History and Law & Society, 2002, University of California, Riverside PROFESSIONAL EXPERIENCE: Elementary School Principal, Salem-Keizer Public Schools, 2016-current Elementary School Assistant Principal, Oro Grande School District, 2012-2016 Senior Middle School Teacher, Oro Grande School District, 2010-2012 Middle School Teacher, Oro Grande School District, 2008-2010 Middle School Teacher, Adelanto School District 2005-2008 v ACKNOWLEDGMENTS I wish to acknowledge everyone that has assisted me in the preparation of this manuscript. In no specific order, my family, advisor, friends, cohort members, colleagues, and students. My family first and foremost, especially my wonderful wife Ashley who not only supported me in my decision to complete my education, but continuing to encourage me to complete what I set out to do, In addition, a special thanks goes to Dr. Gerald Tindal, my advisor on two different occasions, about 15 years apart and his amazing brilliance in connecting my passions to a pathway for success. All of my friends who have been supportive and took my mind off of things at appropriate times. The wonderful fellow educators in my cohort: We did it! Thank you for 3 years of Zoom classes and collective student efficacy. To all the educators and students with whom I work: thank you for dealing with my craziness, my inattention to detail (sorry), and buckling down to do the hard work of teaching and learning. Finally, a giant thank you to Salem Keizer Public Schools and Dr. Kraig Sproles, who supported me in carrying out research on some of our youngest students. vi This manuscript is dedicated to the prekindergarten and kindergarten students who do what they do because it’s fun, talk the truth because they have no filter, and do their best every day because they don’t know any other way. vii TABLE OF CONTENTS Chapter Page I. INTRODUCTION .................................................................................................... 1 Overview ................................................................................................................ 3 Literature Synthesis ............................................................................................... 5 Kindergarten Skill Assessments ...................................................................... 6 Importance of Foundational Reading Skills .................................................... 7 Decoding .................................................................................................... 9 Language Comprehension ......................................................................... 10 Early Benchmark Assessment Hurdles in a Response to Intervention System ..................................................................... 10 Pre-Kindergarten Assessment .......................................................................... 12 Research Questions .......................................................................................... 13 II. METHODS.............................................................................................................. 15 Research Type and Unit of Analysis ............................................................... 15 Participants ....................................................................................................... 15 Assessments ..................................................................................................... 17 Analysis .....................................................................................................................19 Test of distribution comparability .................................................................... 19 Receiver Operating Characteristic (ROC) analyses ......................................... 20 Linear regression .............................................................................................. 21 III. RESULTS .............................................................................................................. 23 Research Question One – Score Distribution of the LRA ............................... 23 viii Chapter Page Research Question Two – ROC Analyses ....................................................... 24 Research Question Three – Linear Regression ................................................ 26 IV. DISCUSSION ........................................................................................................ 27 Summary of Findings ............................................................................................. 27 Findings Relative to the Research Literature ................................................... 27 Research Question One .................................................................................... 29 Research Question Two ................................................................................... 30 Research Question Three ................................................................................. 33 Study Limitations ............................................................................................. 34 Threats to internal validity ......................................................................... 35 History.................................................................................................. 35 Testing ................................................................................................. 35 Instrumentation .................................................................................... 36 Differential selection ............................................................................ 36 Future Research .............................................................................................. 37 Practical Applications ...................................................................................... 37 Conclusion ...................................................................................................... 39 REFERENCES CITED ................................................................................................ 40 ix LIST OF FIGURES Figure Page 1. Distribution of LRA scores with a normal distribution curve displayed for reference ........................................................................................... 24 2. AUC Comparison of being identified as having some or high risk on Winter kindergarten reading benchmark to LRA score ..................................................... 25 x LIST OF TABLES Table Page 1. Demographic data of study participants ................................................................ 17 2. Descriptive Statistics for Literacy Portion of LRA Assessment ............................ 23 3. Crosstabulation of scores below 10th, 15th, 20th percentile and optimal cut score (14.5) with Winter Kindergarten easyCBM® risk level ............ 26 4. Regression Summary Table for Kindergarten Literacy Risk and LRA score ....... 26 xi CHAPTER I INTRODUCTION Implementation of the Common Core State Standards (CCSS) by 48 states raised the rigor of student learning outcomes, even at early elementary grades and kindergarten now more closely mirrors the other elementary school grades (Bassok & Latham, 2017). CCSS requires kindergarten students to complete a level of curriculum that was previously primarily covered in first grade (Bassok, Latham, & Rorem 2016; Carmichael, Martino, Porter-Magee, & Wilson, 2010). Furthermore, full-day kindergarten in the United States increased from 56% in 1998 to 76% in 2012 (Flanagan & McPhee, 2009; Child Trends, 2013), creating comparable amounts of instructional time for kindergarten students with other elementary grades. With the increased academic demands and instructional time, kindergarten instruction has become increasingly focused on academics and monitoring student growth. Monitoring student growth in kindergarten begins with identifying children’s incoming abilities and skill levels. Knowing the abilities of the students prior to instruction also allows school staff to make decisions to best support learning. Knowing student skill set allows school staff to allocate appropriate resources in the forms of staffing and time prior to instruction beginning. When instruction begins, knowing the skill set of the students in the classroom allows for educators to use appropriate curriculum to best meet the students’ needs. It also allows teachers to strategically group students in instructional grouping that maximize students learning. This allows for teachers and other school staff to provide instruction that appropriately builds upon prior knowledge or provide remediate instruction in underlying skills that students may be missing. As soon as school begins in fall, kindergarten students begin learning foundational reading skills and applying those skills to begin word reading, since literacy is arguably the most important skill in education (Whitehurst, & Lonigan 2001). Therefore, identifying the extent of underlying, foundational reading skills a student possesses prior to or at the onset of kindergarten gives school staff necessary information to purposefully plan instruction for the student. This is a challenge for teachers because, prior to kindergarten, children experience a variety of educational settings, ranging from formalized, highly academic-focused preschool to being at home with family or attending care facilities with other children. According to the Developing Early Literacy Report published in 2008 by the National Early Learning Panel, about 25% of children of preschool age attended a formal preschool. Many educators who support and educate kindergarteners are frustrated with the lack of available academic data at the onset of the school year. State kindergarten entry assessments are not designed for elementary decision-making purposes and benchmark assessments do not have an ample distribution of scores to create meaningful decisions. Therefore, on top of the required state assessments and the district-mandated district assessments, kindergarten educators resort to using time-consuming, administration- intense assessments in order to have an understanding of the skills that students enter kindergarten with. This practice reduces the amount of instruction time during a critical period in the school year. Other, more efficient, assessment systems exist that could provide sufficient data for educators to make decisions at the beginning of kindergarten. 2 However, the performance of those assessments and their correlation with results of other assessments that are given later in the year are an area that needs to be researched and be sufficient prior to usage in a pre-kindergarten setting. Overview Instruction in literacy begins at the onset of kindergarten with implementation of CCSS; however many students continue to struggle with learning the skill of reading. The ability to read fluently in later elementary grades and beyond is required for students to continue to grow academically, meet grade-level standards, and increase their likelihood of high school graduation (Annie E. Casey Foundation [AEC], 2013). Fletcher, Lyon, Fuchs, and Barnes (2018) found that the longer a student takes to learn to read, the more difficult it is for them to become a successful reader so addressing potential difficulties early in kindergarten is important. Sixty-five to seventy-five percent of children designated as reading disabled during their elementary education continue to read poorly throughout their school years and beyond (Scarborough, 2001; Whitehurst & Lonigan, 2001). Once an elementary school student can fluently read, only about 5% to 10% struggle to read later (Whitehurst & Lonigan, 2001). Being able to correctly identify students who are struggling with reading or may struggle in the future can lead to additional instruction and practice, focused interventions, and progress monitoring to evaluate the impact and effectiveness of instruction, interventions, and curriculum. With the increased focus on system-wide accountability in education and the implementation of multi-tiered systems of support (MTSS), using early, preventative steps like targeted interventions to catch students up to grade level in reading is preferred to them being identified as behind on state assessments 3 at the end of third grade, when any chance of recovery is minimal (AEC, 2013). MTSS implementation models rely on seasonal benchmark assessments throughout the instructional year to monitor student learning, identify students who need more intense instruction, and gauge the effectiveness of instruction and interventions (Jimerson, Burns, & VanDerHeyden, 2007). The benchmark assessments that are used in MTSS do not result in a proficiency score (like end of year state assessment systems), but rather a percentile score comparing the results of that student to national results for students at that point in time in that specific grade level. Current benchmark assessments, often used as part of a Response to Intervention (RTI) system for identifying students at risk and given to students at the beginning of kindergarten result in most students performing poorly (Tindal, Irvin, Nese, & Slater, 2015). This can be attributed to the variety of prior settings and the academic or lack of academic focus that the students are exposed to prior to kindergarten. For example, when those benchmark assessments are given later in the school year, after word-reading instruction has begun, it is easier to identify students who are lagging behind their peers. The problem is that many months pass between seasonal assessment windows and those months represent valuable time that educators could have provided additional, targeted instruction. Lonigan, Burgess, and Anthony, (2000) found that a large component of children’s reading skills can be found in the preschool period and research on the identification of students with struggling literacy skills has been occurring in preschool settings for quite some time (Lonigan, et al., 2000); National Institute of Child Health & Human Development, 2000). These components, which were found to have relationships 4 with later reading abilities, include letter knowledge, letter sound knowledge, and phonological sensitivity. Therefore, it may be beneficial for elementary educators to consider using an assessment similar to those assessments given in preschool settings in order to identify early reading difficulties. Additionally, findings from longitudinal research studies beginning in preschool show predictive correlations into kindergarten and beyond and suggest which early literacy skills to assess them prior to or immediately upon entering kindergarten (Whitehurst, & Lonigan, 2001). Correct identification of students at risk for later reading difficulties in preschool or at the beginning of kindergarten needs to be multi-faceted and longitudinal in design (Shanahan, 2018). Literature Synthesis In this section, I explain the process used to search for relevant references as well as how search results were narrowed down to ensure that pertinent information was included. I began my literature search by using the University of Oregon’s online library portal, which accessed the following resources: ERIC, PSYCNet, Academic OneFile, Academic Search Premier, SAGE Complete, and ProQuest Educational Journals. In addition, I conducted an ancestral search from many articles on assessing, preventing, and teaching students with reading difficulties or dyslexia. I narrowed the results to include articles published since 1980, given that many of the foundational reading studies around a framework to read occurred in and after the mid-1980s. Additionally, I limited the results to peer-reviewed studies occurring in the United States. I used the following search terms: (a) reading skills or reading skills development and (b) early assessment or intervention or prevention and (c) kindergarten and (d) risk factor or contributing factors or predisposing factors and a total of 59 articles were found. 5 After reviewing the results and reading the abstracts, I excluded any articles that included “special education.” I did so because the articles that focused on special education did not include a population of students who also received general education instruction. This selection process resulted in 23 articles that I read more thoroughly to determine their applicability to my study, resulting in a total of 19 articles. Kindergarten Skill Assessments In this section, I describe the assessment systems being used in kindergarten, particularly Oregon, how these assessment systems were developed, and how the assessment systems don’t produce results that can be used to guide early instruction. I then focus on the purpose of these assessments and conclude that they are not designed, nor intended, to identify students who will not respond typically to instruction. Many states require assessments in kindergarten to identify deficits in foundational reading and mathematics skills. These assessments are shaped by knowledge of child development, social values, and learning targets for what children should be able to do (Zubrzycki, 2011). This movement to improve student achievement through greater accountability is a significant development in the field of education (Zubrzycki, 2011). In 2012, the Oregon legislature passed House Bill 4165, which directed the Early Learning Council and the Department of Education to create a kindergarten assessment that would be given to all Oregon kindergarten students starting in the fall of 2013. The Kindergarten Readiness Assessment (KRA), which was created as a result, is a composite assessment that includes an academic battery of early literacy fluency measures in phoneme segmenting, letter names, and letter sounds (Rowley, 2015). The KRA was adapted by the Oregon Department of Education from existing easyCBM© measures (Rowley, 2015). 6 In addition to using kindergarten entry assessments, many school districts assess all students multiple times over the course of a school year to monitor learning in literacy or mathematics, including kindergarten students. These assessments, called benchmark assessments, are aligned with developmentally appropriate foundational reading skills. The results of these assessments have many different uses, but one of the main objectives of the assessment is to identify students who are not making typical academic gains and may require different forms of instruction, curriculum, or additional instruction to acquire those skills (Jimerson, Burns, & VanDerHeyden, 2007). These assessments typically evolve over the course of a student’s educational career to include skills that are relevant and developmentally appropriate, leading to more advanced reading skills later. Many of the seasonal benchmark assessments in the primary grades (kindergarten, first, and second grades) have included a specific assessment of phonological awareness based on research findings. For example, Wagner, Torgesen, and Rashotte (1994) and Hogan, Catts, and Little (2005) found that phonological awareness in kindergarten was predictive of later decoding and word reading skills, particularly in second grade. Importance of Foundational Reading Skills In this section, I describe the components of literacy in depth and how the different skills follow a typical path to becoming literate. Finally, I explain the research regarding deficits in some of these skill areas and how they lead to substantial reading difficulties later. The goal of reading is reading comprehension, defined by Snow (2002) as “the process of simultaneously extracting and constructing meaning through interaction and involvement with written language.” Reading requires two primary skills: decoding and 7 language comprehension (Gough & Tunmer, 1986). Gough and Tunmer coined the term “Simple View” of reading comprehension as a student’s ability to decode (D) and comprehend spoken language (C). Gough noted that Reading Comprehension (R) = D x C, with values of C and D as either 0 or 1. If a student has no difficulty understanding spoken language, that student would have a score of 1. But, if they cannot decode text, they would have a decoding score of 0. Multiply those two numbers together and you get 0, meaning that this student does not have the skills for reading comprehension. On the flip side, if a student can fluently decode (D = 1) but is unable to comprehend language (C = 0), the words would carry no meaning and there is no comprehension of what was decoded. Therefore, the framework for reading comprehension rests on both decoding and language comprehension. This dissertation focuses on decoding skill development, which I describe in more detail below. Ehri’s (1992) model also provides a similar approach to word recognition development. For example, Ehri stated that word reading acquisition has multiple stages: pre-alphabetic, partial alphabetic, full alphabetic, and consolidated alphabetic. The focus of early reading instruction is the partial alphabetic stage, in which readers learn to decode, or form connections between letters in spellings and sounds in pronunciations of words (Ehri, 1992, 1998). Ehri (2005) emphasized that a critical early step is when readers know how to distinguish the separate phonemes in pronouncing words (Ehri 2005). Different models of decoding skills agree on a few different points: learning to read involves different skills that develop over time; the first skills involve letter identification, letter sound correspondence, and phoneme awareness. 8 Decoding. This skill is frequently referred to as word reading and comprises the ability to accurately pronounce written words without presumption or prior familiarity (Kilpatrick, 2015). Unlike language comprehension, decoding is finite: once a student has mastered this skill, they are able to decode any word in text. Decoding is made up of smaller skills that work together in a coordinated effort. According to the framework presented by Wren (2000), decoding relies foundationally on phoneme awareness and concepts about print. Phoneme awareness is the skill of knowing how to “consciously… recognize and manipulate the units of the spoken word” (Wren, p. 17, 2000). Phoneme awareness allows a reader to distinguish words based on changes to letter sounds, or phonemes, in words. Phoneme awareness has been identified as the only skill that plays a causal role in learning to read (Scarborough, 2001), while also being a key skill that relates to later differences in reading skills (Raz & Bryant, 1990). Once a student has a foundational understanding of phoneme awareness and concepts about print, which is the basic understanding that letters forms words and that the process of reading moves in a particular direction (e.g. left to right, top to bottom), they develop knowledge of the alphabetic principle and lexical exceptions to the rule. These skills allow readers to read words that don’t follow the normal systematic relationship between letters and phonemes used in decoding. Examples are words like “stomach” or “colonel,” where the reader must recognize words that they know from context and through the course of language acquisition (Webb, 2008). The ability of a reader to use decoding skills along with word recognition for exceptions culminates in a fluency. 9 Brain imaging studies have demonstrated that reading begins primarily as a phonological process and when a child begins to read, the brain changes in ways that permit meditation and word recognition (Goswami, 2008; Flethcher, Lyon, Fuchs, & Barnes, 2018). Kilpatrick (2018), found that a deficit in this skill was the primary reason why students struggle with word-level reading. Children who have difficulty with the automatic connection between print and speech have required much more practice to achieve a normal level of sight word learning (Ehri & Saltmarsh, 1995; Reitsma, 1983). Language comprehension. The definition used in education includes the understanding of vocabulary, background knowledge, verbal reasoning, and literacy knowledge to acquire new knowledge (Carroll, 1971; Kilpatrick, 2015). Most students acquire the basics of language comprehension through oral language comprehension prior to entering kindergarten. They continue to acquire more advanced language skills naturally and through formal education and over the course of their life (Scarborough, 2001). Early Benchmark Assessment Hurdles in a Response to Intervention System In this section, I explain how results on assessments given at the beginning of kindergarten do not allow for students to be identified who need additional instructional supports. Finally, I describe a recently developed assessment system, created to identify students who need additional instructional support in early childhood education. Benchmark assessments are used to identify students who are struggling regardless of typical instruction in a content area so that they can receive additional focused instruction, evaluate the effectiveness of interventions, and gauge the effectiveness of the instruction in those content areas. The easyCBM© system includes a 10 curriculum-based benchmark assessment that was developed at the University of Oregon as a formative assessment system to monitor the progress students are making towards grade-level proficiency in key skill and content areas and identify students who are at risk of not meeting grade level standards (Lai, Nese, Jamgochian, Alonzo, & Tindal, 2010). The easyCBM© uses a progression of key skills which begins in kindergarten with letter identification, letter sound identification, and phoneme segmenting. By the end of kindergarten, the assessments have changed to reflect the instruction occurring in the classroom and word reading fluency is assessed. However, the literacy measure that is given in the Fall of the kindergarten year, which focuses on a student’s ability to be fluent in letter identification, letter sound identification, and phoneme segmenting typically has asymmetric distributions, in which a majority of students are low performing and are identified as being high risk of not meeting grade level standards at the end of kindergarten (Tindal, Irvin, Nese, & Slater, 2015). While low performance on an entry assessment presents an opportunity for many students to show significant improvement as a result of instruction, students who need additional supports to be successful in literacy cannot be easily distinguished from typically-achieving students (Catts, Petscher, Schatschneider, Bridges, & Mendoza, 2009). Catts et. al (2009) explains that while floor effects in basic skills are not unusual, they impede the ability for educators to establish risk factors early in the school year. For meaningful cut scores to be established, which could be predictive of future risk, greater variance in the kindergarten-entry literacy measure results are needed. Without score variance, cut score ranges have little to no difference between scores considered “at risk” 11 versus little or no risk. This can result in many students being misidentified as not needing additional support when in fact, they need them (false negatives) or as needing additional support, when in fact, they do not (false positives). Later in the kindergarten year, particularly at the middle of the kindergarten, a normal distribution of scores occurs on literacy benchmark assessments and students who are not making sufficient progress and who may need additional targeted instruction in literacy are able to be identified (Lai, et al., 2010). Pre-Kindergarten Assessment Foundational literacy skills develop and can be assessed in preschool or early childhood education programs, including print knowledge, alphabet knowledge, and phonological awareness (Justice, 2006). Pre-kindergarten assessments have identified students who are at risk of persistent reading disabilities by measuring a student’s “literacy receptiveness” (Irvin, Sáez, Pilger, Alonzo, Squires, Twombly, & Tindal, 2018). Many different assessment systems assess foundational literacy skills, including Phonological Awareness Literacy Screening (PALS) PreK, Get Ready to Read, Individual Growth and Development Indicators (IGDIS), IStation, and Fast Bridge learning, which all vary in the training needed to for delivery and the length of time to administer the assessment or assessments (Gischlar, & Shapiro, 2014). The assessment used in this study, the Learning Receptiveness Assessment (LRA), was developed to be used to quickly identify students who were exhibiting difficulties in literacy, math, and working memory that would hamper later learning (Irvin, et. al, 2018). Interventions targeting these critical components of early literacy skills have led to increased school readiness (Koutsoftas, Harmon, & Gray, 2009). 12 However, the connection between the LRA results and later benchmark fluency assessments in kindergarten has not yet been established. In other words, its predictive validity is unknown. The assessment, compared to other pre-kindergarten assessments, required the least amount of time to administer and, because it is delivered via a tablet, required the least amount of test administration training (Sáez, & Irvin, 2020). Because of the LRA’s efficiency to measure early reading skills, it may provide useful information to guide kindergarten reading instruction, resource allocation, and reading curriculum. Research Questions Current literacy assessment practices at the beginning of kindergarten do not provide results useful to educators for the purpose of planning instruction. By answering these research questions, results of a prekindergarten assessment will be analyzed scientifically to determine if it is able to be used as a tool to make educational decisions that impact the learning of students. I present the results of the LRA and the relationship between results on the LRA with subsequent easyCBM® assessments to answer the following research question: 1. Does the literacy portion of the Learning Receptiveness Assessment have normal score distributions (descriptive, distributional, and correlational properties) reflecting the performance of preschool students in the spring prior to entering Kindergarten? 2. Does the LRA predict students who are needing further evaluation on kindergarten easyCBM® winter fundamental skills assessment? 13 3. What is the predictive concordant relation between the prekindergarten emergent literacy test and performance on kindergarten easyCBM® winter fundamental skills tests (letter sound, phoneme segmenting, and word reading fluency)? 14 CHAPTER II METHODS This study aimed to document three outcomes associated with the prekindergarten LRA measure: (a) the distribution of scores in the spring prior to kindergarten; (b) establishment of cut scores for identifying students who may be at risk of performing presenting reading development difficulties in kindergarten; and (c) documentation of the relationship with EasyCBM® benchmark literacy assessments administered in the middle of kindergarten. Research Type and Unit of Analysis This study used data that were collected from the same students using different literacy assessments with a non-experimental design that allowed for data to provide generalizability of a population examined over a period of time (Babbie, 2013). The unit of analysis for this research was student performance from two different assessments. The first research question focused on student scores obtained from the LRA literacy measure. The second research question focused on the extent to which cut scores could be determined to identify adequate levels of risk. Finally, the last research question examined the predictive relation of LRA literacy scores to later reading performance. The data were aggregated to make generalizations about cut scores that predict risk at the group level (Babbie, 2010) and future performance. Participants A convenience sample was used that included seventy-five (N = 75) students who concluded preschool and entered kindergarten in the spring and fall of 2019, respectively. Creswell and Creswell (2018) state that it is important to “purposefully select participants 15 or sites what will best help the researcher understand the problem and answer the research question” (p.278) Therefore, preschool students in diverse preschool programs were selected as the sample, see Table 1. The participants attended three different preschool programs in the boundaries of the second largest school district in Oregon. Participants were included in the study if they attended kindergarten at a school in the area’s public school district in the subsequent school year until the middle of kindergarten, when they took the Winter easyCBM® benchmark assessment. The preschools, although in proximity to each other, served different populations of students and varied widely in their programming. Two of the preschool programs were offered by the local school district. One of the programs was tuition based, while the second was based on income level and is free for the students who qualify. The tuition- based preschool operated by the school district had multiple half-day “classes” throughout the week, meeting with classes on alternating days (Monday, Wednesday, and Friday, or Tuesday and Thursday). The school-district operated, income-based preschool was a five-day-a-week, full day program. The other preschool program was operated by a community organization serving children throughout the area and operated a daily, full day preschool. 16 Table 1 Demographic Data of Study Participants Demographic and N Total Preschool 1 Preschool 2 Preschool 3 background characteristics Percent (Tuition- (Income- (Community based) based) programs) Total students 75 38 14 23 Gender identification Male 31 41% 37% 36% 52% Female 44 59% 63% 64% 48% Racial/ethnic identification Hispanic/LatinX 25 33% 29% 71% 17% White 45 60% 63% 29% 74% Other 5 7% 8% 0% 9% English Language Learner Yes 9 12% 3% 57% 0% No 66 88% 97% 43% 100% SPED/504 identification Yes 8 11% 8% 7% 13% No 67 89% 92% 93% 87% Assessments The Learning Receptiveness Assessment, developed by Behavior Research and Teaching (BRT) at the University of Oregon, is a measure to assess children’s literacy, behavioral, and working memory processing skills to improve learning (Irvin, et al., 2018; Sáez, & Irvin, 2020). The assessment is a “tablet-based screening tool that preschool classrooms can use to quickly identify children at greatest risk for exhibiting… difficulties that would hamper learning if not addressed” (Irvin, et al., p. 4, 2018). The 17 tablet delivery of the measure allows the questions and answer choices to be administered in a standardized delivery method (Sáez, & Irvin, 2020). For this research study, only the literacy portion of the assessment was used. The early literacy skills portion of the assessment focused on three domains: letter identification, letter sound identification, and phonological sensitivity (a rudimentary stage of phonological awareness. Each of the domains were assessed by orally and visually presenting the student with multiple pictured options and having the student select the correct choice. Each domain consisted of eight different items, with the phonological sensitivity domain having items consisting of beginning sound identification, multiple sounds blending, and phoneme-segmented word identification. The literacy portion of the measure had relatively high internal consistency in the spring of preschool, calculated using Cronbach’s alpha (α = .76) (Sáez, & Irvin, 2020). The assessment was administered to students in small group settings with a test administrator being a teacher, instructional assistant, or researcher. Administration time varied, with some students completing the assessment in one session and others taking two or more. The easyCBM® assessment, also developed by researchers at BRT is an assessment system designed to assess and monitor growth in students from grades K through Grade 8 as part of a Response to Intervention (RTI) system. For the purposes of this research study, only the classification of risk results from the benchmark assessment were used. There are three different levels of risk classification on the easyCBM®, determined by percentile ranking: high risk (10th percentile or lower, some risk (11th to 49th percentile), and no risk (50th percentile and higher). All kindergarten easyCBM® benchmark assessments demonstrated strong classification accuracy, with sensitivity and 18 specificity within normal ranges (AUC > .75) (Alonzo, & Anderson, 2018). The kindergarten easyCBM® measures are highly correlated with other measures (DIBELS and CTOPP Elision subtest) used to measure foundational reading skills. For example, the correlation between the easyCBM® phoneme segmenting (PS) and the DIBELS phoneme segmenting fluency was high for kindergarten (rs = .86). The kindergarten easyCBM® letter sounds measures showed moderate correlations with the DIBELS letter sound and nonsense word fluency, with rs = .55 (Lai, Alonzo, & Tindal, 2013). At the winter benchmark in kindergarten, students were assessed using three different fluency measures: letter sounds, phoneme segmenting, and word reading. All measures were administered in a one-on-one setting with a test administrator at their school of attendance. The student had one minute to respond after being visually prompted (letter sounds and word reading) or by the test administrator reading (phoneme segmenting) and being asked to identify the sound, word, or phonemes. Analysis Three different analytic strategies were used to answer the research questions presented in this study, including distribution comparability, receiving operating characteristic (ROC), and linear regression. Tests of Distribution Comparability To answer the first research questions, the distribution from the LRA in each of the domains of the literacy receptiveness portion and the overall result were analyzed using the Shapiro-Wilk (SW) and Lilliefors Significance Corrected Kolmogorov – Smirnov (KS) normality tests; descriptive statistics of the measures were also documented. These non-parametric normality tests compared the distribution of two 19 samples. They measured the maximum difference between an empirical and a hypothetical cumulative distribution (Massey, 1951). Each test was based on the full data set from the LRA literacy measure results. Normality results from the WS and KS tests were analyzed, a non-significant result (p > .05) represented a distribution of scores that were not significantly different from a normal distribution and therefore had a typical distribution. When the distribution of the scores were evaluated based on the descriptive statistics, a skewness value of less than 2.0 and kurtosis value less than 7.0 indicated that the data was non-linear and normally distributed (Fabrigar, Wegener, MacCallum, and Strahan, 1999). Receiver Operating Characteristic (ROC) Analyses To answer the second research question about the establishment of a cut score to more precisely determine possible early literacy risk, classification accuracy of different established cut scores were analyzed by conducting a Sensitivity/Specificity ROC analysis. The establishment of a cut score is intended to determine if a student needs further evaluation for being at risk of not making typical progress as a result of kindergarten instruction or if there is little or no risk of the student not making typical literacy progress in kindergarten. Three different cut scores were tested, one representing a cut score at the 10th percentile rank, another representing a cut at the 15th percentile rank, and finally the cut score representing the 20th percentile rank. These cut scores have typically been used in education when determining risk from benchmark assessments (Alonzo, & Anderson, 2018). For each cut score, the “at risk” classification on the easyCBM® winter kindergarten assessment was used as the outcome. 20 The cut scores were analyzed with the intent to maximize both sensitivity and specificity, with an increased emphasis on sensitivity, as outlined by Silberglitt and Hintze (2005). Each cut score was compared to each other, with the optimal cut score having the highest sensitivity and specificity statistics. As a threshold for answering the research question, the highest sensitivity and specificity statistics would ideally be above 0.8. However, if no cut score resulted in both sensitivity and specificity being above 0.8, a sensitivity score above 0.8 while specificity is above 0.7 was thought to be acceptable, to reduce false negatives when identifying students who need additional instruction. Results for each cut score for have been presented in the Results section with corresponding crosstabulation statistics. Linear Regression To answer the third research question about the strength of correlation between the LRA and the easyCBM® a linear regression analysis was conducted to predict the value of a dependent variable (EasyCBM® score) from the LRA results. To measure the strength of the correlation and the amount of variance accounted by the LRA, results were regressed on the dependent variable (winter EasyCBM® literacy risk classification), and both r and R2, respectively, were calculated. For this study, the dependent variable was the risk outcome on the Winter EasyCBM® literacy assessment and the predictor was the total score on the LRA prekindergarten assessment. To determine the strength of correlation, a significant value (p < .05) showed a strong correlation between the results of the two assessments. The amount of variance accounted for by the results on the LRA literacy assessment on the winter EasyCBM® literacy assessment was determined by calculating R2. To calculate 21 this result, the data from the prekindergarten assessment were fitted against a regression line to see how much variability existed between the results. A positive correlation between the measures demonstrated a positive relationship; the results showed a range of 0.3 to 0.6, which is typical for this type of study. 22 CHAPTER III RESULTS This section is organized by research questions that focus on the distribution of scores, prediction with cut scores, and the correlation between results on the two assessments. Research Question One – Score Distribution of the LRA The descriptive statistics from the literacy portion of the LRA assessment is presented in Table 2. A histogram displaying student scores are presented in Figure 1. The Shapiro-Wilk test of normality did not indicate significant differences between the LRA literacy score distribution compared to a normal score distribution (p = .057), while the Lilliefors Significance Corrected KS test indicated a slightly significant difference (p = .046). Table 2. Descriptive Statistics for Literacy Portion of LRA Assessment N Mean Median Min. Max SD Skewness Kurtosis Literacy 24 13.2 13 5 21 4.5 -.25 -.76 23 Figure 1. Distribution of LRA scores with normal distribution curve displayed for reference Research Question Two – ROC Analyses Using a ROC Analyses, the Area Under the Curve (AUC) for the literacy portion of the LRA assessment was examined to test the accuracy of classifying children with later reading difficulty risk based on the classification derived from the Winter kindergarten easyCBM® reading measurement. The AUC found was = .80 and is displayed in Figure 2. Invernizzi, et. al (2004) considered scores similar to this a good accuracy. Using the AUC, the optimal cut score from the ROC analyses was 14.5, having a sensitivity = .76 and specificity = .69. Table 3 shows the results of the three different cut scores on the LRA literacy measure and the optimal cut score that were crosstabulated with the three levels of risk using the combined kindergarten Winter easyCBM® reading measures. When LRA literacy cut sores at the 10th, 15th, and 20th percentile were used, no students with scores below these cut points were later classified with low risk in kindergarten. However, 25% 24 of student or fewer were considered as later having some risk in kindergarten using the 10th, 15th, and 20th percentile scores. Forty percent of scores from the LRA literacy measure that fell at or below the 10th percentile were later classified as high risk in kindergarten. Using a more liberal cut score, 70% of students whose scores fell at or below the 20th percentile were later classified with high risk in kindergarten. In contrast, the optimal cut score (OCS) of 14.5 accurately identified 31% of students later classified with low risk, 69% of students later classified with some risk, and 100% of students later classified as high risk. Figure 2. AUC Comparison of being identified as having some or high risk on Winter kindergarten reading benchmark to LRA score. 25 Table 3. Crosstabulation of Scores Below 10th, 15th, 20th Percentile and Optimal Cut Score (14.5) with Winter Kindergarten easyCBM® Risk Level Low Low Some Some High High Risk Risk Risk Risk Risk Risk % Count % Count % Count ≤ 10th Percentile 0 0 5 14 4 40 ≤ 15th Percentile 0 0 7 19 5 50 ≤ 20th Percentile 0 0 9 25 7 70 OCS (<14.5) 9 31 36 69 10 100 Research Question Three – Linear Regression The correlation strength was calculated between the LRA literacy score and the at risk (both some and high risk) outcome on the Winter easyCBM® literacy assessment using linear regression. There was a significant, large correlation between the results of the assessment (r = .58, p < .05). The LRA assessment accounted for 34% of the variance in the winter EasyCBM® assessment scores, see table 4 for all results of the linear regression. Table 4. Regression Summary Table for Kindergarten Literacy Risk and LRA score N r R2 Adj. R2 STD Error of B β p Estimate LRA Lit. Score 75 .58 .34 .33 .56 -.10 -.58 < .05 26 CHAPTER IV DISCUSSION This section includes a review and interpretation of the results and a discussion of the study limitations. Further analysis of the findings and future research as well as implications for practice are made, discussion regarding validity of the study and limitations, followed by a conclusion of the study. Summary of Findings The purpose of this study was to determine the normality of the results from a prekindergarten assessment, establish whether predetermined cut scores were able to accurately identify students later classified with reading difficulty risk, and measure and evaluate the strength of prediction for later identified reading risk. The summary of findings and interpretation of the results will be first summarized in terms of the literature and then organized by research question. Finally, I consider the limitations and implications for future research. Findings Relative to the Research Literature This study confirms the suggestion by Zubrzycki (2011) that early assessment results can reflect a child’s academic development, social values, and what skills they are able to do. More specifically, these assessment must identify students at risk of not making normal academic gains so that different forms of instruction, curriculum, or additional instruction to acquire those skills can be implemented as part of their education (Jimerson, Burns, & VanDerHeyden, 2007). In my study, the distributions were normal in the spring prior to kindergarten, showing student’s different levels of preliteracy skills which could be used to customize instruction upon entering kindergarten. However, the 27 ROC analysis was less than optimal in the eventual identification of poorly performing students, with many false positives/negatives. Misidentification of student skills affects the efficiency of providing the proper amount and level of instruction and support to all students. This results in students receiving instruction that is too difficult or is not challenging and therefore is not beneficial to their learning. Although the goal of reading is reading comprehension (Snow, 2002), such a leap from early entry skills to specific later outcomes is likely too distant. Therefore, in my study, I used a more proximal outcome with easyCBM® (word reading that emphasized decoding). The rather significant finding of a positive correlation (with a significant amount of variance accounted for) between having preliteracy skills, identified by Wren (2000) with phonemic awareness and Ehri’s (1992) approach to word recognition development, and later more advanced decoding skills later in the student’s education. Given the findings are consistent with theories of reading development, the only issue remaining is the timing of interventions for developing early literacy skills (Whitehurst, & Lonigan, 2001). Finally, considering all three questions as part of a Response to Intervention (RTI) system, the findings reflect this system can be value added but only if all three findings are considered holistically (Lai, Nese, Jamgochian, Alonzo, & Tindal, 2010). In this model, benchmark assessments are used to identify students at risk and need progress monitoring. This requires a distribution with tails at both ends (not skewed), a cut score that neither burdens the resources or results in under-identifying students, and the ability to make predictions into the future. The success of this study improves the efficiency for educators in all three aspects: identifying student skills quickly and giving educators 28 insight into which students are most likely to need different or more extensive instruction early in their education so that they are able to be successful readers later. Particularly as more students are being served in full day kindergarten (Flanagan & McPhee, 2009; Child Trends, 2013), such efficiencies become important. Research Question One The first research question focused on the distribution of scores on the literacy portion of the LRA. Using the Shapiro-Wilk test of normality, the results for the distribution was not significantly different than a normal distribution. The Lilliefors significance corrected KS test had different results, with a slightly significant difference (p > .05). According to Yap and Sim (2011), a small sample size, similar to this study, could result in inaccurate results. Furthermore, the Shapiro-Wilk test has been considered the most powerful of all tests of normality, performing over all types of distributions. The assessment, having a total of 24 items, had no results where a student responded correctly to all items (maximum score = 21, N = 1) nor a student who responded incorrectly to all items (minimum score = 5, N =2). Furthermore, both the mean (13.2) and median (13) are nearly equal, indicating a normal distribution (Fabrigar, Wegener, MacCallum, & Strahan, 1999). Both the skewness and the kurtosis fell within acceptable parameters and are similar to those that have been seen on other preschool assessments, such as the PALS-PreK and IGDIS assessments (Gischlar, & Shapiro, 2014), which each had skewness and kurtosis that were less than 2.0 and 7.0 respectively, less than the thresholds determined by Fabrigar, et. al (1999) to show a normal, linear distribution of results. 29 The distribution of literacy scores shows that, as described in the literature review, preschool or early childhood skills in foundational literacy areas can be assessed (Justice, 2006) and a typical distribution can occur. The LRA literacy results are contrary to the Oregon kindergarten Fall easyCBM® benchmark literacy assessment results where many students exhibit rather low performance (Tindal, et al., 2015) and are atypically distributed. This distribution of variance among prekindergarten LRA scores could allow educators to create meaningful instructional groupings. The normal distribution, lacking a floor or ceiling effect, let alone students with all of the items either incorrect or correct, allows for the results of this assessment to be used for instructional purposes, as discussed by Catts et. al (2009). Additionally, the outcome could be particularly helpful to kindergarten educators to use the results of the screener when doing initial evaluations of incoming students prior to enrollment or class placement. The assessment results have the necessary variance to determine which students may not need instruction beyond typical kindergarten instruction and which students may need additional instruction in foundational literacy skills. By creating classrooms that have a students with a diverse group of skills, educators can provide a full scope of the instruction around the standards and the peer grouping to create an effective educational environment for all students can be diverse, supportive of all students, and close the achievement gap. Research Question Two The second research question focused on the determination of cut scores to maximize both specificity and sensitivity as measured by risk results on the later kindergarten Winter easyCBM® literacy assessment. These findings, although not 30 sufficient to create clear cut scores, are similar to those that were described by Invernizzi, Sullivan, Meir, and Swank (2004) who found that some students identified by the PALS PreK as being below the developmental range were found to be successful readers at a later time. Therefore, it is important that students identified as performing in the lowest percentiles continue to receive literacy instruction and additional monitors of progress so that they instruction in literacy matches the skill set need of the learner. It is also important that educators practice flexibility in their groupings of students and adjust the placement of students based on the skill level that is shown. Overlooking students who need additional support (sensitivity) and identification of students for support who don’t need it (specificity) based on the results of one assessment can be minimized by encouraging flexible and frequent evaluation of student learning needs and the use of other assessment results to confirm skill level. When using a LRA literacy cut score at the 20th percentile, 30% of “at early risk” students were later classified at high risk on the kindergarten assessment. In a further examination of these results, other factors were found to be possible reasons for inconsistent assessment results. These results on the easyCBM® occur when a student has behaviors that interfere with the accuracy of results or when a Spanish-speaking student is given an English assessment and does not understand the task enough to show their knowledge. However, all students considered low risk on the kindergarten assessment scored above the 20th percentile on the LRA. In addition to the percentile cut scores, the optimum cut score was also analyzed with cross tabulation. This score had sensitivity and specificity values less than optimal (< .7), indicating that several students fell within the “early risk” zone in prekindergarten 31 but on the later kindergarten assessment were deemed at low risk (i.e. reading at grade level expectations). However, when the Optimal Cut Score was used, all students at high risk on the later kindergarten assessment were identified. An overlap between two scores indicates where many students fall: those who score below 14.5 (the optimal cut score as determined by the AUC) as high or some risk and those above 9.5 (the 20th percentile) as not at risk on the kindergarten assessment. These students who score between the optimal cut score and above the 20th percentile need to have their basic literacy skill development closely monitored for appropriate growth during kindergarten. The results of this study around establishing cut scores to identify students who may lack foundational preliteracy skills are consistent with other studies (Invernizzi, et al., 2004). Numerous participants in the Invernizzi (2004) study performed poorly on the prekindergarten assessment, but later performed at or above grade level on the kindergarten assessment. There were examples of the contrary as well: students who scored above the optimal cut score on the prekindergarten assessment, but on the kindergarten benchmark assessment given after multiple months of instruction, scored below typical for a kindergarten student. However, the largest group of students are those who scored between the optimal cut score but above the 20th percentile on the LRA assessment. These students can be characterized as having some preliteracy skills prior to entering kindergarten but need reinforcement of the foundational skills in letter names, sounds, and phoneme segmenting to be a successful reader. 32 Research Question Three To determine the strength of the correlation between the LRA and the literacy portion of the Winter easyCBM® risk score, a linear regression was calculated to determine if the results between the two assessments had a significant relationship. The r and R2 values were calculated to measure the strength of the correlation and the amount of variance that could be accounted by the outcome on the LRA assessment. The relationship between the two assessments showed a significant result (p < .05) that was moderate and positive (r = .58) and the LRA accounted for 34% of the variance (R2 = .34) on the later EasyCBM® assessment. These results are typical with other predictive correlation studies that have been conducted over time. For example, Invernizzi, et. al (2004) had similar results (r = .53, p < .01, R2 = .305) when the PALS- PreK foundational literacy assessments were given to 3,106 students in the Spring of 2002 and reassessed with the PALS-K in the Spring of 2003. A second longitudinal data analysis was conducted by Invernizzi et. al (2004) on 2,574 children who were assessed with the Pals-PreK in Spring 2002 and again a year and a half later with the PALS 1-3 first grade assessment and had similar results (r = .56, p <.01, R2 = .342). The results from this study and other studies of similar nature show a comparable amount of variance that can be predicted on primary elementary reading assessments by assessments that are given prior to kindergarten (Invernizzi, et. al, 2004; Koutsoftas, et al. 2009). The results from this study and the other studies show that there is a significant relationship between having early foundational skills and performing at grade level on later benchmark assessments. These mixed results provide an important message to elementary educators: while some of the variance in the performance of students can be 33 accounted by the skills that students come into school with, the majority of the performance on early literacy assessments is based on other factors (that also may need support in order for the student to reach their learning potential). These factors include quality and methods of instruction, which are controlled by the classroom teacher and the tools and methods of instruction that are implemented. Other factors, such as socioeconomic status, race, language barriers, and trauma are factors that also play a role, but the group of students from diverse settings included in this study provide some generalizability of results to students in different settings. While the results of this study do show that there are many factors that account for a student’s success early in education, no tool for predicting reading skills prior to kindergarten yet exists. The moderate correlation strength between the assessments does prove that there is a connection between having preliteracy skills prior to kindergarten The relationships between the assessments represent a connection between having foundational skills in literacy at an early age and being able to carry those skills over as students into kindergarten, as described by Irvin, et al. (2018). This study showed that there is enough evidence from administering a tablet-delivered benchmark assessment with relatively few items prior to kindergarten to identify students who may need additional literacy support in order for them to be considered “at grade level” during kindergarten. Study Limitations Although the purpose of this study was to evaluate the use of a pre-kindergarten assessment to determine its utility to make educational decisions, there are numerous limitations that affect this study. 34 One of the biggest threats to both internal and external validity of this study was the small sample size (N = 75). Although the number of participants sufficiently provide the power necessary to do the statistical calculations, more participants may have led to different results and different study findings. Threats to internal validity. In this study there were at least three other potential threats to internal validity to be considered (a) history, (b) testing, (c) instrumentation, and (d) differential selection (Creswell, 2014). History. One of the greatest threats to this study was history, because the students attended three different preschool programs and kindergarten at 27 different elementary schools, there are exponential other factors that may have impacted their learning, performance, or progress. The preschool programs varied in the curriculum being used, experience of teachers, size of class, and the number of school days and duration of the academic day. Each elementary school, although in the same district, have different teachers and some schools offer different learning programs (bilingual, dual language immersion) that may have had a substantial impact on the children’s kindergarten literacy results. Numerous study participants were the only student at their and the largest group of students at one elementary school accounted for only 16% (N = 12) of the total number of participants. While this helps to generalize the results from this study across different kindergarten settings, it also increases the possibility of other factors having sizeable impacts on the results of the kindergarten assessment. Testing. Although the LRA’s tablet delivery controls many aspects of test administration (e.g., standardized directions that limits potential “noisy” testing effects on validity), a possibility exists that the other assessment, easyCBM®, could have been 35 administered in a manner that did not produce fair results. Some students (N =5) did not take the English easyCBM® assessments, but rather took the Spanish measures because of the language program in which they were enrolled. While the outcome of the easyCBM® Spanish measures is still a risk predictor (low, some, high), the assessment is different than the English easyCBM® assessment that was given to the other participants in the winter of kindergarten. Also, some students with scores of zero across the easyCBM® literacy measures, indicating their high risk, they had results from the LRA that were not comparable. Instrumentation. The easyCBM® measure is a well-researched measure with technical adequacy to measure the constructs in this study (Alonzo & Anderson, 2018; Lai, et al., 2013; Tindal, et al., 2015). However, the LRA is a new assessment with limited published research regarding the internal and external validity as a literacy measure construct (Sáez, & Irvin, 2020). Future studies and increased usage of the LRA will minimize this limitation, but at the time of this study, the validity is preliminary and different results in later studies could question the validity of the results from this study. Differential selection. The purpose of the study was to determine if a pre- kindergarten literacy assessment for use in identifying risk of problems in learning to read and serve as a predictor of later outcomes. The subjects, however, were a convenience sample and not a random sample of pre-kindergarten students. All participants were in a preschool setting and were receiving pre-literacy instruction. The different skills taught around pre-literacy skills may have varied and there is a likelihood that little to no instruction around skills such as phonemic awareness, were delivered. 36 Future Research The need for continued use of the LRA assessment in all pre-kindergarten settings, both preschool and students that don’t attend school until kindergarten, needs to continue so that technical adequacy of assessments used for decision-making can be validated for use concerning risk and academic growth. The results of the LRA literacy measure had a typical distribution prior to kindergarten and research around skills or lack of specific skills when entering kindergarten could be examined using other assessments. In the school district where the participants attended, other assessments (Kindergarten Readiness Assessment, easyCBM® Fall benchmark, formative classroom assessments) could be administered and the results analyzed to provide an in-depth understanding of how to identify kindergarten students who are going to struggle with reading. This would allow educators to make decisions about providing early additional resources to those students and the effectiveness of the resources that are being implemented. Practical Applications Benchmark assessments are becoming one of the most useful formative assessment tools as multi-tiered systems of support continue to be implemented in school districts across the country and abroad. The research behind the implementation of such programs shows that to be able to provide appropriate instruction to students, an understanding of their underlying skill set is needed. The results from a benchmark assessment allow educators to make short term decisions about instruction, interventions, and resource allocation. The need for benchmark results to identify groups of students who have varying need is critical for the best usage of those resources. Resources in education are finite: there are simply not enough to give every student the full plethora of 37 resources available; nor is it necessary: most students will have adequate learning with typical classroom instruction by a highly qualified classroom teacher. The results of this study fit into the implementation of multi-tiered systems of support and the need to have an entry assessment into kindergarten that provides results with a typical distribution so educators can make short term decisions at the beginning of Kindergarten. The results from this assessment had similar statistical properties and performance as other valid assessments given to students at the same point in their education. The use of this assessment in and prior to kindergarten could allow educators to determine the level of support needed in kindergarten classrooms and more importantly: where and who to allocate those resources to. In schools that have more than one kindergarten class, the use of spring prekindergarten LRA literacy results at kindergarten entry could allow leveling of need across kindergarten classrooms, so that the educational needs of classrooms are similar and teachers are able to plan and build the most elective teacher efficacy, which is one of the strongest determining factors of student outcomes (Hattie, 2012). The results from this study could also be used in professional development as educators gain knowledge around what a quality assessment is and how to use an assessment for the strict purpose that the assessment was created. Educators frequently discuss the need to be fluid with the decision-making process because the results from the assessments don’t always paint a clear picture of a student’s abilities. This, in statistical terms, is sensitivity and specificity and empowering educators to recognize that with higher values in this area, the preciseness of an assessment will be improved and there will be less misidentification and lack of identification of students. 38 There are also applications from this study for future benchmark assessment creation. Unlike many of the other assessments that are created to assess foundational literacy skills, the results from this study mirrored many other studies in terms of normality of distribution, usage of cut scores, and strength of correlation with later literacy assessments. However, this assessment was delivered on a tablet and consisted of only twenty-four items. The results of this study show that benchmark assessments can perform and produce similar results to other, more lengthy, foundational literacy assessments. Conclusion Entering kindergarten with some foundational literacy skills most likely leads to sooner literacy outcomes in kindergarten and beyond. The use of the Learning Receptiveness Assessment as a benchmark literacy skills screener prior to kindergarten instruction has the potential to help educators identify which students may benefit from additional instruction in foundational literacy skills to be better readers later in their kindergarten year and beyond. This study examined accuracy of early risk identification using the literacy portion of the Learning Receptiveness Assessment, a pre-kindergarten literacy benchmark and the association between the results on that assessment to later literacy assessment scores. Results from this study found that it is possible to have a normal distribution of scores on a tablet-based literacy screener. However, the ability to establish highly accurate cut scores to determine later outcomes is less clear. Emphasis needs to continue a focus on providing a quality education for all students when they enter kindergarten in literacy if they are going to be successful readers later in their education and lives. 39 REFERENCES CITED Annie E. Casey Foundation (2013). Early warning! Why reading by the end of third grade matters. Special report. Annie E. Casey Foundation, Baltimore, MD. http://www.aecf.org/resources/early-warning-why-reading-by-theend-of-third- grade-matters/ Alonzo, J. & Anderson, D. (2018). Classification accuracy of the easyCBM® kindergarten – grade 2 reading measures (Technical Report No. 1801). Eugene, OR: Behavioral Research and Teaching, University of Oregon. Bassok, D., & Latham, S. (2017). Kids today: The rise in children’s academic skills at kindergarten entry. Educational Researcher, 46, 7-20. Bassok, D., Latham, S., & Rorem, A. (2016) Is kindergarten the new first grade? AERA Open, 2. doi:10.1177/2332858415616358 Carmichael, S. B., Martino, G., Porter-Magee, K., & Wilson, W. S. (2010). The state of state standards—and the Common Core – in 2010. Washington DC: Thomas B. Fordham Institute. Carroll, J. B. (1971) Defining language comprehension: Some speculations. Princeton, NJ: Educational Testing Service. Catts, H. W., Petscher, Y., Schatschneider, C., Bridges, M. S., & Mendoza, K. (2009). Floor effects associated with universal screening and their impact on the early identification of reading disabilities. Journal of Learning Disabilities, 42, 163- 176. doi:10.177/002221940 8326219 Child Trends. (2013). Full-day kindergarten: Indicators on children and youth (Updated ed.). Retrieved from http://www.childtrends.org/ wp- content/uploads/2013/06/102_Full-day-kindergarten.pdf Ehri, L. (1992). Reconceptualizing the development of sight word reading and its relationship to recoding. In P. Gough, L. Ehri, & R. Treiman (Eds.), Reading acquisition. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Ehri, L. (1998). Grapheme-phoneme knowledge is essential for learning to read words in English. In J. Metsala & L. Ehri (Eds.), Word recognition in beginning literacy. Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Ehri, L. (2005). Learning to read words: Theory, findings, and issues. Scientific Studies Of Reading, 9, 167-188. 40 Ehri, L., & Saltmarsh, J. (1995). Beginning readers outperform older disabled readers in learning to read words by sight. Reading and Writing: An Interdisciplinary Journal, 7, 295-326. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 272-299. doi: 10.1037/1082-989X.4.3.272. Flanagan, K. D., & McPhee, C. (2009). The children born in 2001 at kindergarten entry: First findings from the kindergarten data collections of the Early Childhood Longitudinal Study, Birth Cohort (ECLS-B) NCES 2010-00. Washington, DC: U.S. Department of Education. Fletcher, J. M., Lyon, G. R., Fuchs, L. S., & Barnes, M. A. (2018). Learning disabilities: From identification to intervention. New York, NY: Guilford. Gischlar, K. L., & Shapiro, E. S. (2014). Efficiency in assessing emergent literacy skills in students attending Head Start. NHSA Dialog, 17, (2) 1-16. Goswami, U. (2008) Reading, dyslexia, and the brain. Educational Research, 50, 135- 148. Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7, 6-10. Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. Routledge London; New York. Hogan, T. P., Catts, H. W., & Little, T. D. (2005). The relationship between phonological awareness and reading. Language, speech, and hearing services in schools, 36, 285-293. doi:10.1044/0161-1461(2005/029) Jimerson S.R., Burns M.K., VanDerHeyden A.M. (2007) Response to Intervention at School: The Science and Practice of Assessment and Intervention. In: Jimerson S.R., Burns M.K., VanDerHeyden A.M. (eds) Handbook of Response to Intervention. Springer, Boston, MA Invernizzi, M., Sullivan, A., Meir, J., & Swank, L., (2004). Phonological awareness literacy screening for preschool (PALS-PreK). Pre-K teachers' manual. Charlottesville, VA: University of Virginia Printing Services. 41 Irvin, P. S., Sáez, L., Pilger, M., Alonzo, J., Squires, J., Twombly, L., & Tindal, G., (2018). Project ICEBERG exploration: Using implementation science to guide preschool reading disabilities prevention (Technical Report No. 1802). Eugene, OR: Behavioral Research and Teaching, University of Oregon. Justice, L. (2006). Evidence based practice response to intervention, and the prevention of reading difficulties. Language, Speech, and Hearing Services in Schools, 37, 284–297. Kilpatrick, D. A. (2015). Essentials of assessing, preventing, and overcoming reading difficulties. Hoboken, NJ: John Wiley & Sons. Kilpatrick, D. A. (2018). Genetics, the environment, and poor instruction as contributors to word-level reading difficulties: Does it matter for early identification and instruction? Perspectives on Language and Literacy, 44(3), 25-28. Koutsoftas, A. D., Harmon, M. T., & Gray, S. (2009). The effect of tier 2 intervention for phonemic awareness in a response-to-intervention model in low-income preschool classrooms. Language, Speech, and Hearing Services in Schools, 40, 116-130. Lai, C. F., Alonzo, J., & Tindal, G. (2013). easyCBM® reading criterion related validity evidence: Grades K-1 (Technical Report No. 1309). Eugene, OR: Behavioral Research and Teaching, University of Oregon. Lai, C. F., Nese, J. F. T., Jamgochian, E. M., Alonzo, J., & Tindal, G. (2010). Technical adequacy of the easyCBM® primary-level reading measures: Grades K-1, 2009- 2010 version (Technical Report No. 1003). Eugene, OR: Behavioral Research and Teaching, University of Oregon. Lonigan, C. J., Burgess, S. R., & Anthony, J. L. (2000). Development of emergent literacy and early reading skills in preschool children: evidence from a latent- variable longitudinal study. Developmental psychology, 36, 596. Massey Jr, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American statistical association, 46(253), 68-78. National Institute of Child Health & Human Development. (2000). Report of the National Reading Panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. (NIH Publication No. 00-4769). Washington, DC: U. S. Government Printing Office. 42 Raz, I. T., & Bryant, P. (1990). Social background, phonological awareness and children's reading. British Journal of Developmental Psychology, 8, 209-225. Reitsma, P. (1983). Printed word learning in beginning readers. Journal of Experimental Child Psychology, 36, 321-339. Rowley, B., & Tindal, G. (2015). Kindergarten Assessment: Analysis of the Child Behavioral Rating Scale (CBRS). Eugene, OR: University of Oregon. Sáez, L. & Irvin, P. S. (2020). Development of the Prekindergarten Learning Receptiveness Assessment (LRA) Greenhouse: Process and Preliminary Findings (Technical Report No. 2001) Eugene, OR: Behavioral Research and Teaching, University of Oregon. Scarborough, H. S. (2001). Connecting early language and literacy to later reading (dis)abilities: Evidence, theory, and practice. In Neuman, S. & Dickinson, D. (Eds.), Handbook for research in early literacy (pp. 97–110). New York, NY: Guilford Press. Silberglitt, B., & Hintze, J. (2005). Formative assessment using CBM-R cut scores to track progress toward success on state-mandated achievement tests: A comparison of methods. Journal of Psychoeducational Assessment, 23, 304-325. doi: 10.1177/073428290502300402 Tindal, G., Irvin, P. S., Nese, J. F. T., & Slater, S. (2015). Skills for children entering kindergarten. Educational Assessment, 20, 297-319. doi: 10.1080/10627197.2015.10 93929 Wagner, R. K., & Torgesen, J. K. (1987). The nature of phonological processing and its causal role in the acquisition of reading skills. Psychological Bulletin, 101, 192– 212. Webb, S. (2008). The effects of context on incidental vocabulary learning. Reading in a Foreign Language, 20, 232-245. Whitehurst, G. J., & Lonigan, C. J. (2001). Emergent literacy: Development from pre- readers to readers. In S. B. Neuman & D. K. Dickinson (Eds.), Handbook of early literacy research (pp. 11–29). New York, NY: Guilford. Wren, S. (2000). The cognitive foundations of learning to read: A framework. Washington, DC: Southwest Educational Research Laboratory. 43 Zubrzycki, J. (2011). Common core poses challenges for preschools. Education Week, 1, 20-21. 44