Sensemaking of Process Data from Evaluation Studies of Educational Games: An Application of Cross-Classified Item Response Theory Modeling
Saved in:
| Title: | Sensemaking of Process Data from Evaluation Studies of Educational Games: An Application of Cross-Classified Item Response Theory Modeling |
|---|---|
| Language: | English |
| Authors: | Tianying Feng (ORCID |
| Source: | Journal of Educational Measurement. 2026 63(1). |
| Availability: | Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us |
| Peer Reviewed: | Y |
| Page Count: | 37 |
| Publication Date: | 2026 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305D210032 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Educational Games, Item Response Theory, Models, Misconceptions, Item Analysis, Game Based Learning, Pretests Posttests |
| DOI: | 10.1111/jedm.12396 |
| ISSN: | 0022-0655 1745-3984 |
| Abstract: | Process information collected from educational games can illuminate how students approach interactive tasks, complementing assessment outcomes routinely examined in evaluation studies. However, the two sources of information are historically analyzed and interpreted separately, and diagnostic process information is often underused. To tackle these issues, we present a new application of cross-classified item response theory modeling, using indicators of knowledge misconceptions and item-level assessment data collected from a multisite game-based randomized controlled trial. This application addresses (a) the joint modeling of students' pretest and posttest item responses and game-based processes described by indicators of misconceptions; (b) integration of gameplay information when gauging the intervention effect of an educational game; (c) relationships among game-based misconception, pretest initial status, and pre-to-post change; and (d) nesting of students within schools, a common aspect in multisite research. We also demonstrate how to structure the data and set up the model to enable our proposed application, and how our application compares to three other approaches to analyzing gameplay and assessment data. Lastly, we note the implications for future evaluation studies and for using analytic results to inform learning and instruction. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2026 |
| Accession Number: | EJ1501291 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Process information collected from educational games can illuminate how students approach interactive tasks, complementing assessment outcomes routinely examined in evaluation studies. However, the two sources of information are historically analyzed and interpreted separately, and diagnostic process information is often underused. To tackle these issues, we present a new application of cross-classified item response theory modeling, using indicators of knowledge misconceptions and item-level assessment data collected from a multisite game-based randomized controlled trial. This application addresses (a) the joint modeling of students' pretest and posttest item responses and game-based processes described by indicators of misconceptions; (b) integration of gameplay information when gauging the intervention effect of an educational game; (c) relationships among game-based misconception, pretest initial status, and pre-to-post change; and (d) nesting of students within schools, a common aspect in multisite research. We also demonstrate how to structure the data and set up the model to enable our proposed application, and how our application compares to three other approaches to analyzing gameplay and assessment data. Lastly, we note the implications for future evaluation studies and for using analytic results to inform learning and instruction. |
|---|---|
| ISSN: | 0022-0655 1745-3984 |
| DOI: | 10.1111/jedm.12396 |
