View in EDS

Identifying Courses for Targeted Review Using GAP Analysis and Machine Learning.

Saved in:
Bibliographic Details
Title: Identifying Courses for Targeted Review Using GAP Analysis and Machine Learning.
Authors: Joseph, Kishore1 (AUTHOR) kishore.joseph@siu.edu, Calvert, Wesley C.2 (AUTHOR), Keys Jr., Oliver3 (AUTHOR), McCrocklin, Shannon M.4 (AUTHOR)
Source: Education Sciences. May2026, Vol. 16 Issue 5, p806. 24p.
Subject Terms: *Course evaluation (Education), *Academic achievement, *Machine learning, *Educational quality, Artificial neural networks, Deviation (Statistics), Resource allocation, Risk assessment
Abstract: We examined the limitations of observed course DFWI rates (% of D and F grades, withdrawals, and incompletes) as evaluation metrics, which obscure student characteristics, course design, instruction, structure, and latent factors, posing challenges in identifying courses that need improvement. An artificial neural network (ANN) was trained using student data to model risk, accounting for variations in student characteristics. The model's predictions on test data were averaged at the course level, producing expected DFWI rates based on student composition. Courses with high observed DFWI rates and large deviations between observed and predicted DFWI rates (the GAP) were ranked and prioritized for review, as they may reflect aspects of course design, structure, or instructional practices warranting further qualitative evaluation. Our predictions are non-causal, and modeling calibration varies across subgroups; therefore, the original GAP rankings, robust to a post-hoc calibration check, are presented as risk-adjusted indicators for prioritizing courses for further review rather than as definitive causal measures of course quality. Rankings based on observed DFWI rates differ substantially from risk-adjusted GAP rankings, indicating that relying on observed DFWI rates alone may misidentify high-risk courses. Our methodology can assist educators and administrators in making fair resource allocation decisions and improving student outcomes. [ABSTRACT FROM AUTHOR]
Copyright of Education Sciences is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Education Research Complete
Description
Abstract:We examined the limitations of observed course DFWI rates (% of D and F grades, withdrawals, and incompletes) as evaluation metrics, which obscure student characteristics, course design, instruction, structure, and latent factors, posing challenges in identifying courses that need improvement. An artificial neural network (ANN) was trained using student data to model risk, accounting for variations in student characteristics. The model's predictions on test data were averaged at the course level, producing expected DFWI rates based on student composition. Courses with high observed DFWI rates and large deviations between observed and predicted DFWI rates (the GAP) were ranked and prioritized for review, as they may reflect aspects of course design, structure, or instructional practices warranting further qualitative evaluation. Our predictions are non-causal, and modeling calibration varies across subgroups; therefore, the original GAP rankings, robust to a post-hoc calibration check, are presented as risk-adjusted indicators for prioritizing courses for further review rather than as definitive causal measures of course quality. Rankings based on observed DFWI rates differ substantially from risk-adjusted GAP rankings, indicating that relying on observed DFWI rates alone may misidentify high-risk courses. Our methodology can assist educators and administrators in making fair resource allocation decisions and improving student outcomes. [ABSTRACT FROM AUTHOR]
ISSN:22277102
DOI:10.3390/educsci16050806