LOOL: Towards Personalization with Flexible & Robust Estimation of Heterogeneous Treatment Effects
Saved in:
| Title: | LOOL: Towards Personalization with Flexible & Robust Estimation of Heterogeneous Treatment Effects |
|---|---|
| Language: | English |
| Authors: | Duy M. Pham, Kirk P. Vanacore, Adam C. Sales, Johann A. Gagnon-Bartsch |
| Source: | Grantee Submission. 2024Paper presented at the International Conference on Educational Data Mining (17th, Atlanta, GA, Jul 2024). |
| Peer Reviewed: | Y |
| Page Count: | 9 |
| Publication Date: | 2024 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305D210031 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Education Level: | Junior High Schools Middle Schools Secondary Education |
| Descriptors: | Middle School Students, Middle School Teachers, Middle School Mathematics, Algebra, Mathematics Instruction, Individualized Instruction, Educational Technology, Computer Assisted Testing, Student Characteristics, Predictive Measurement, Prior Learning, English (Second Language), Computer Managed Instruction |
| DOI: | 10.5281/zenodo.12729840 |
| Abstract: | Effective personalization of education requires knowing how each student will perform under certain conditions, given their specific characteristics. Thus, the demand for interpretable and precise estimation of heterogeneous treatment effects is ever-present. This paper outlines a new approach to this problem based on the Leave-One-Out Potential Outcomes (LOOP) Estimator, which unbiasedly estimates individual treatment effects (ITE) from experiments. By regressing these estimates on a set of moderators, we obtain parameterized and easily interpretable estimates of conditional average treatment effects (CATE) that allow us to understand which individuals will likely benefit from each condition. We implement this approach with real-world data from an efficacy study that included four experimental conditions for instructing middle-school algebra. Our models indicate that treatment effect heterogeneity is significantly associated with students' prior subject knowledge and whether English is their native language. We then discuss possibilities for applications to enhance personalized assignments. [This paper was published in: "Proceedings of the 17th International Conference on Educational Data Mining," edited by B. PaaBen and C. D. Epp, International Educational Data Mining Society, 2024, pp. 376-84.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2025 |
| Accession Number: | ED670843 |
| Database: | ERIC |
| Abstract: | Effective personalization of education requires knowing how each student will perform under certain conditions, given their specific characteristics. Thus, the demand for interpretable and precise estimation of heterogeneous treatment effects is ever-present. This paper outlines a new approach to this problem based on the Leave-One-Out Potential Outcomes (LOOP) Estimator, which unbiasedly estimates individual treatment effects (ITE) from experiments. By regressing these estimates on a set of moderators, we obtain parameterized and easily interpretable estimates of conditional average treatment effects (CATE) that allow us to understand which individuals will likely benefit from each condition. We implement this approach with real-world data from an efficacy study that included four experimental conditions for instructing middle-school algebra. Our models indicate that treatment effect heterogeneity is significantly associated with students' prior subject knowledge and whether English is their native language. We then discuss possibilities for applications to enhance personalized assignments. [This paper was published in: "Proceedings of the 17th International Conference on Educational Data Mining," edited by B. PaaBen and C. D. Epp, International Educational Data Mining Society, 2024, pp. 376-84.] |
|---|---|
| DOI: | 10.5281/zenodo.12729840 |
