Hold the Bets! Should Quasi-Experiments Be Preferred to True Experiments When Causal Generalization Is the Goal?
Saved in:
| Title: | Hold the Bets! Should Quasi-Experiments Be Preferred to True Experiments When Causal Generalization Is the Goal? |
|---|---|
| Language: | English |
| Authors: | Andrew P. Jaciw (ORCID |
| Source: | American Journal of Evaluation. 2025 46(1):90-127. |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 38 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Descriptive |
| Education Level: | Elementary Education |
| Descriptors: | Elementary School Students, Elementary School Teachers, Generalization, Test Bias, Test Construction, Quasiexperimental Design, Research Methodology, Randomized Controlled Trials, Causal Models, Comparative Testing, Test Selection, Test Use, Test Validity |
| Geographic Terms: | Tennessee |
| DOI: | 10.1177/10982140241246208 |
| ISSN: | 1098-2140 1557-0878 |
| Abstract: | By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset by confounded selection into locations. This work shows that this tradeoff can lead to situations where estimates from QEs are less-biased from selection than are estimates from uncompromised XPs when drawing causal generalizations. This work establishes the conditions theoretically, demonstrates the idea empirically, and discusses the implications of the results. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1465063 |
| Database: | ERIC |
| Abstract: | By design, randomized experiments (XPs) rule out bias from confounded selection of participants into conditions. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from XPs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset by confounded selection into locations. This work shows that this tradeoff can lead to situations where estimates from QEs are less-biased from selection than are estimates from uncompromised XPs when drawing causal generalizations. This work establishes the conditions theoretically, demonstrates the idea empirically, and discusses the implications of the results. |
|---|---|
| ISSN: | 1098-2140 1557-0878 |
| DOI: | 10.1177/10982140241246208 |
