The Internal Validity of the School-Level Comparative Interrupted Time Series Design: Evidence from Four New Within-Study Comparisons
Saved in:
| Title: | The Internal Validity of the School-Level Comparative Interrupted Time Series Design: Evidence from Four New Within-Study Comparisons |
|---|---|
| Language: | English |
| Authors: | Sims, Sam (ORCID |
| Source: | Journal of Research on Educational Effectiveness. 2022 15(4):876-897. |
| Availability: | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals |
| Peer Reviewed: | Y |
| Page Count: | 22 |
| Publication Date: | 2022 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Elementary Secondary Education |
| Descriptors: | Validity, Comparative Analysis, Statistical Analysis, Intervention, Evaluation Methods, Institutional Characteristics, Outcomes of Education, Elementary Secondary Education, Foreign Countries, Databases, Student Records, Teaching Methods, Mathematics Achievement |
| Geographic Terms: | United Kingdom (England) |
| DOI: | 10.1080/19345747.2022.2051652 |
| ISSN: | 1934-5747 1934-5739 |
| Abstract: | Comparative interrupted time series (CITS) designs evaluate impact by modeling the relative deviation from trends among a treatment and comparison group after an intervention. The broad applicability of the design means it is widely used in education research. Like all non-experimental evaluation methods however, the internal validity of a given CITS evaluation depends on assumptions that cannot be directly verified. We provide an empirical test of the internal validity of CITS by conducting four within-study comparisons of school-level interventions previously evaluated using randomized controlled trials. Our estimate of bias across these four studies is 0.03 school-level (or 0.01 pupil-level) standard deviations. The results suggest well-conducted CITS evaluations of similar school-level education interventions are likely to display limited bias. |
| Abstractor: | As Provided |
| Entry Date: | 2023 |
| Accession Number: | EJ1367853 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Comparative interrupted time series (CITS) designs evaluate impact by modeling the relative deviation from trends among a treatment and comparison group after an intervention. The broad applicability of the design means it is widely used in education research. Like all non-experimental evaluation methods however, the internal validity of a given CITS evaluation depends on assumptions that cannot be directly verified. We provide an empirical test of the internal validity of CITS by conducting four within-study comparisons of school-level interventions previously evaluated using randomized controlled trials. Our estimate of bias across these four studies is 0.03 school-level (or 0.01 pupil-level) standard deviations. The results suggest well-conducted CITS evaluations of similar school-level education interventions are likely to display limited bias. |
|---|---|
| ISSN: | 1934-5747 1934-5739 |
| DOI: | 10.1080/19345747.2022.2051652 |
