Beyond Accuracy: Embracing Meaningful Parameters in Educational Data Mining
Saved in:
| Title: | Beyond Accuracy: Embracing Meaningful Parameters in Educational Data Mining |
|---|---|
| Language: | English |
| Authors: | Napol Rachatasumrit, Paulo F. Carvalho, Kenneth R. Koedinger |
| Source: | International Educational Data Mining Society. 2024. |
| Availability: | International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/ |
| Peer Reviewed: | Y |
| Page Count: | 8 |
| Publication Date: | 2024 |
| Sponsoring Agency: | National Science Foundation (NSF) |
| Contract Number: | 2301130 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Descriptors: | Data Analysis, Models, Prediction, Accuracy, Artificial Intelligence, Technology Uses in Education, Performance, Statistical Analysis |
| Abstract: | What does it mean for a model to be a better model? One conceptualization, indeed a common one in Educational Data Mining, is that a better model is the one that fits the data better, that is, higher prediction accuracy. However, oftentimes, models that maximize prediction accuracy do not provide meaningful parameter estimates, making them less useful for building theory and practice. Here we argue that models that provide meaningful parameters are better models and, indeed, often also provide higher prediction accuracy. To illustrate our argument, we investigate the Performance Factor Analysis (PFA) model and the Additive Factors Model (AFM). PFA often has higher prediction accuracy than the AFM. However, PFA's parameter estimates are ambiguous and confounded. We propose more interpretable models (AFMh and PFAh) designed to address the confounded parameters and use synthetic data to demonstrate PFA's parameter interpretability issues. The results from the experiment with 27 real-world datasets also support our claims and show that more interpretable models will also produce better predictions. [For the complete proceedings, see ED675485.] |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | ED675541 |
| Database: | ERIC |
| Abstract: | What does it mean for a model to be a better model? One conceptualization, indeed a common one in Educational Data Mining, is that a better model is the one that fits the data better, that is, higher prediction accuracy. However, oftentimes, models that maximize prediction accuracy do not provide meaningful parameter estimates, making them less useful for building theory and practice. Here we argue that models that provide meaningful parameters are better models and, indeed, often also provide higher prediction accuracy. To illustrate our argument, we investigate the Performance Factor Analysis (PFA) model and the Additive Factors Model (AFM). PFA often has higher prediction accuracy than the AFM. However, PFA's parameter estimates are ambiguous and confounded. We propose more interpretable models (AFMh and PFAh) designed to address the confounded parameters and use synthetic data to demonstrate PFA's parameter interpretability issues. The results from the experiment with 27 real-world datasets also support our claims and show that more interpretable models will also produce better predictions. [For the complete proceedings, see ED675485.] |
|---|
