LOOL: Towards Personalization with Flexible & Robust Estimation of Heterogeneous Treatment Effects
Saved in:
| Title: | LOOL: Towards Personalization with Flexible & Robust Estimation of Heterogeneous Treatment Effects |
|---|---|
| Language: | English |
| Authors: | Duy M. Pham, Kirk P. Vanacore, Adam C. Sales, Johann A. Gagnon-Bartsch |
| Source: | International Educational Data Mining Society. 2024. |
| Availability: | International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/ |
| Peer Reviewed: | Y |
| Page Count: | 9 |
| Publication Date: | 2024 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305D210031 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Education Level: | Junior High Schools Middle Schools Secondary Education |
| Descriptors: | Middle School Students, Middle School Teachers, Middle School Mathematics, Algebra, Mathematics Instruction, Individualized Instruction, Educational Technology, Student Characteristics, Prior Learning, English (Second Language), Computer Assisted Instruction |
| Abstract: | Effective personalization of education requires knowing how each student will perform under certain conditions, given their specific characteristics. Thus, the demand for interpretable and precise estimation of heterogeneous treatment effects is ever-present. This paper outlines a new approach to this problem based on the Leave-One-Out Potential Outcomes (LOOP) Estimator, which unbiasedly estimates individual treatment effects (ITE) from experiments. By regressing these estimates on a set of moderators, we obtain parameterized and easily interpretable estimates of conditional average treatment effects (CATE) that allow us to understand which individuals will likely benefit from each condition. We implement this approach with real-world data from an efficacy study that included four experimental conditions for instructing middle-school algebra. Our models indicate that treatment effect heterogeneity is significantly associated with students' prior subject knowledge and whether English is their native language. We then discuss possibilities for applications to enhance personalized assignments. [For the complete proceedings, see ED675485.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2025 |
| Accession Number: | ED675644 |
| Database: | ERIC |
| Abstract: | Effective personalization of education requires knowing how each student will perform under certain conditions, given their specific characteristics. Thus, the demand for interpretable and precise estimation of heterogeneous treatment effects is ever-present. This paper outlines a new approach to this problem based on the Leave-One-Out Potential Outcomes (LOOP) Estimator, which unbiasedly estimates individual treatment effects (ITE) from experiments. By regressing these estimates on a set of moderators, we obtain parameterized and easily interpretable estimates of conditional average treatment effects (CATE) that allow us to understand which individuals will likely benefit from each condition. We implement this approach with real-world data from an efficacy study that included four experimental conditions for instructing middle-school algebra. Our models indicate that treatment effect heterogeneity is significantly associated with students' prior subject knowledge and whether English is their native language. We then discuss possibilities for applications to enhance personalized assignments. [For the complete proceedings, see ED675485.] |
|---|
