Throwing the Baby Out With the Bathwater? The What Works Clearinghouse Criteria for Group Equivalence
Loading...
Date
2013-12-13
Authors
Stockard, Jean
Journal Title
Journal ISSN
Volume Title
Publisher
National Institute for Direct Instruction (NIFDI)
Abstract
The What Works Clearinghouse (WWC) provides summary reports of the effectiveness
of educational programs and practices, but only a small proportion of studies considered for
review meet their selection criteria. A common reason for rejecting studies from
consideration regards the WWC’s standard for equivalence of the intervention and
comparison groups. This paper examines the criteria related to these decisions. Calculations
based on the Central Limit Theorem illustrate how the probability of meeting the WWC
criteria declines sharply when studies include multiple measures and/or comparisons
(which is common in more sophisticated studies) and/or when sample sizes are smaller
(which is common in highly controlled experimental designs). Descriptions of two welldesigned
studies rejected for inclusion in recent WWC reports illustrate the nature of the
problem. Implications and recommendations for policy change are discussed.
Description
11 pages
Keywords
Central Limit Theorum, Criteria Examination, Technical Report