View in EDS

Small-Sample Adjustments for Tests of Moderators and Model Fit in Robust Variance Estimation in Meta-Regression

Saved in:
Bibliographic Details
Title: Small-Sample Adjustments for Tests of Moderators and Model Fit in Robust Variance Estimation in Meta-Regression
Language: English
Authors: Tipton, Elizabeth, Pustejovsky, James E., Society for Research on Educational Effectiveness (SREE)
Source: Society for Research on Educational Effectiveness. 2015.
Availability: Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; Fax: 202-640-4401; e-mail: inquiries@sree.org; Web site: http://www.sree.org
Peer Reviewed: Y
Page Count: 9
Publication Date: 2015
Document Type: Reports - Research
Descriptors: Randomized Controlled Trials, Sample Size, Effect Size, Hypothesis Testing, Regression (Statistics), Robustness (Statistics), Mathematical Models, Simulation, Statistical Bias, Sampling, Economics, Hierarchical Linear Modeling, Meta Analysis
Abstract: Randomized experiments are commonly used to evaluate the effectiveness of educational interventions. The goal of the present investigation is to develop small-sample corrections for multiple contrast hypothesis tests (i.e., F-tests) such as the omnibus test of meta-regression fit or a test for equality of three or more levels of a categorical moderator. Drawing on work that addresses related, simpler problems and special cases of cluster-robust variance estimation, the authors develop three small-sample tests based on different approximations to the distribution of a robust Wald test statistic. In the remainder of the study, the authors describe their modeling assumptions, proposed tests, and some initial simulation result. These approximations are drawn from a wide array of areas within statistics, ranging from econometrics to survey sampling. The paper presents both new analytic work describing these small-sample corrected test statistics and the results of a large simulation study that compares these potential solutions, as well as a discussion of the implications of our findings for practice. Tables and figures are appended.
Abstractor: ERIC
Number of References: 8
Entry Date: 2015
Accession Number: ED562265
Database: ERIC
Description
Abstract:Randomized experiments are commonly used to evaluate the effectiveness of educational interventions. The goal of the present investigation is to develop small-sample corrections for multiple contrast hypothesis tests (i.e., F-tests) such as the omnibus test of meta-regression fit or a test for equality of three or more levels of a categorical moderator. Drawing on work that addresses related, simpler problems and special cases of cluster-robust variance estimation, the authors develop three small-sample tests based on different approximations to the distribution of a robust Wald test statistic. In the remainder of the study, the authors describe their modeling assumptions, proposed tests, and some initial simulation result. These approximations are drawn from a wide array of areas within statistics, ranging from econometrics to survey sampling. The paper presents both new analytic work describing these small-sample corrected test statistics and the results of a large simulation study that compares these potential solutions, as well as a discussion of the implications of our findings for practice. Tables and figures are appended.