Student Usage Predicts Treatment Effect Heterogeneity in the Cognitive Tutor Algebra I Program
Saved in:
| Title: | Student Usage Predicts Treatment Effect Heterogeneity in the Cognitive Tutor Algebra I Program |
|---|---|
| Language: | English |
| Authors: | Sales, Adam C., Wilks, Asa, Pane, John F. |
| Source: | International Educational Data Mining Society. 2016. |
| Availability: | International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: http://www.educationaldatamining.org |
| Peer Reviewed: | Y |
| Page Count: | 8 |
| Publication Date: | 2016 |
| Sponsoring Agency: | National Science Foundation (NSF) Institute of Education Sciences (ED) |
| Contract Number: | DRL1420374 R305B100012 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Education Level: | High Schools Secondary Education Middle Schools Junior High Schools |
| Descriptors: | Algebra, Mathematics Instruction, Educational Technology, Technology Uses in Education, Intelligent Tutoring Systems, High School Students, Secondary School Mathematics, Program Effectiveness, Middle School Students |
| Abstract: | The Cognitive Tutor Algebra I (CTAI) curriculum, which includes both textbook and online components, has been shown to boost student learning by about 0.2 standard deviations in a randomized effectiveness trial. Students who were assigned to the experimental condition varied substantially in how, and how much, the used the online component of CTAI, but original analyses of the experimental data focused on estimating average effects, and did not examine whether the CTAI treatment effect varied by the amount of style of usage. This study leverages log data from the experiment to present a more nuanced analysis. It uses the framework of Principal Stratification, which estimates the varying CTAI treatment effect as a function of "potential" usage--either how students used the program, or how they would have used it had they been assigned to the treatment condition. With experimental data, Principal Stratification does not require that we assume that all relevant variables have been measured. With this framework, we find that students who receive a medium amount of assistance from the software (in the form of hints and error feedback) experience the largest effects, with lower effects for students who receive a lot or a little; and evidence that students who do not follow the curriculum order experience smaller treatment effects. [For the full proceedings, see ED592609.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2019 |
| Accession Number: | ED592734 |
| Database: | ERIC |
| Abstract: | The Cognitive Tutor Algebra I (CTAI) curriculum, which includes both textbook and online components, has been shown to boost student learning by about 0.2 standard deviations in a randomized effectiveness trial. Students who were assigned to the experimental condition varied substantially in how, and how much, the used the online component of CTAI, but original analyses of the experimental data focused on estimating average effects, and did not examine whether the CTAI treatment effect varied by the amount of style of usage. This study leverages log data from the experiment to present a more nuanced analysis. It uses the framework of Principal Stratification, which estimates the varying CTAI treatment effect as a function of "potential" usage--either how students used the program, or how they would have used it had they been assigned to the treatment condition. With experimental data, Principal Stratification does not require that we assume that all relevant variables have been measured. With this framework, we find that students who receive a medium amount of assistance from the software (in the form of hints and error feedback) experience the largest effects, with lower effects for students who receive a lot or a little; and evidence that students who do not follow the curriculum order experience smaller treatment effects. [For the full proceedings, see ED592609.] |
|---|
