The numeric understanding measures: Developing and validating adaptive and nonadaptive numeracy scales
Loading...
Date
2023-06-28
Authors
Silverstein, Michael C.
Bjälkebring, Pär
Shoots-Reinhard, Brittany
Peters, Ellen
Journal Title
Journal ISSN
Volume Title
Publisher
Cambridge University Press
Abstract
Numeracy—the ability to understand and use numeric information—is linked to good decision-making. Several
problems exist with current numeracy measures, however. Depending on the participant sample, some existing
measures are too easy or too hard; also, established measures often contain items well-known to participants.
The current article aimed to develop new numeric understanding measures (NUMs) including a 1-item (1-NUM),
4-item (4-NUM), and 4-item adaptive measure (A-NUM). In a calibration study, 2 participant samples (n = 226 and
264 from Amazon’s Mechanical Turk [MTurk]) each responded to half of 84 novel numeracy items.We calibrated
items using 2-parameter logistic item response theory (IRT) models. Based on item parameters, we developed the
3 new numeracy measures. In a subsequent validation study, 600 MTurk participants completed the new numeracy
measures, the adaptive Berlin Numeracy Test, and the Weller Rasch-Based Numeracy Test, in randomized order.
To establish predictive and convergent validities, participants also completed judgment and decision tasks, Raven’s
progressive matrices, a vocabulary test, and demographics. Confirmatory factor analyses suggested that the
1-NUM, 4-NUM, and A-NUM load onto the same factor as existing measures. The NUM scales also showed
similar association patterns to subjective numeracy and cognitive ability measures as established measures. Finally,
they effectively predicted classic numeracy effects. In fact, based on power analyses, the A-NUM and 4-NUM
appeared to confer more power to detect effects than existing measures. Thus, using IRT, we developed 3 brief
numeracy measures, using novel items and without sacrificing construct scope. The measures can be downloaded
as Qualtrics files (https://osf.io/pcegz/).
Description
19 pages
Keywords
Numeracy, Numeric literacy, Numeric reasoning, Adaptive test, Validation, Decision-making
Citation
Silverstein, M., Bjälkebring, P., Shoots-Reinhard, B., & Peters, E. (2023). The numeric understanding measures: Developing and validating adaptive and nonadaptive numeracy scales. Judgment and Decision Making, 18, E19. doi:10.1017/jdm.2023.17