Heterogeneity in Early Mathematics Screening: Investigating the Role of Intervention Effects on Screening Accuracy
Loading...
Date
2024-01-09
Authors
Ives, Christopher
Journal Title
Journal ISSN
Volume Title
Publisher
University of Oregon
Abstract
This study explores the heterogeneity in screening accuracy of the Assessing Student Proficiency in Early Number Sense (ASPENS) across schools within the context of a randomized control trial (RCT) for Fusion, a first-grade early math intervention. Students were assigned to one of three conditions: a business-as-usual (BAU) control group, a two-student Fusion group (2-Fusion), and a five-student Fusion group (5-Fusion). Two research questions were addressed: 1) To what extent does the observed screening accuracy of ASPENS meaningfully differ between students randomly assigned to the Fusion intervention conditions compared to the BAU condition?; and 2) To what extent is heterogeneity in screening accuracy reduced when is ASPENS is administered concurrently with its criterion, rather than at different times of the year? Data were analyzed using generalized linear mixed models to jointly model sensitivity and specificity at the participant level, using the 20th percentile on the Test of Early Mathematics Ability – 3rd Edition as the reference criterion. As hypothesized, findings indicated that specificity was significantly affected by treatment conditions, with the 2-Fusion condition exhibiting lower specificity than the BAU condition. 5-Fusion also demonstrated lower specificity than BAU, but this difference was not statistically significant. Furthermore, heterogeneity in screening accuracy across treatment groups was no longer evident when assessments were administered concurrently. The findings of this study underscore the challenges of prognostic screening frameworks and have implications for the use of publisher-recommended cut-scores, the development and validation of academic screening measures, and guiding best practices in utilizing screening assessments within multi-tiered systems of support.