Measuring Upper-Elementary Students' Understanding of AI Concepts -- A Rasch Model Analysis
Saved in:
| Title: | Measuring Upper-Elementary Students' Understanding of AI Concepts -- A Rasch Model Analysis |
|---|---|
| Language: | English |
| Authors: | Srijita Chakraburty, Krista D. Glazewski, Cindy E. Hmelo-Silver, Dubravka Svetina Valdivia, Anne Ottenbreit-Leftwich, Bradford Mott, James Lester |
| Source: | Information and Learning Sciences. 2025 126(7-8):445-471. |
| Availability: | Emerald Publishing Limited. Howard House, Wagon Lane, Bingley, West Yorkshire, BD16 1WA, UK. Tel: +44-1274-777700; Fax: +44-1274-785201; e-mail: emerald@emeraldinsight.com; Web site: http://www.emerald.com/insight |
| Peer Reviewed: | Y |
| Page Count: | 27 |
| Publication Date: | 2025 |
| Sponsoring Agency: | National Science Foundation (NSF), Division of Research on Learning in Formal and Informal Settings (DRL) |
| Contract Number: | 1934128 1934153 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Elementary Education Early Childhood Education Grade 3 Primary Education Grade 4 Intermediate Grades Grade 5 Middle Schools |
| Descriptors: | Elementary School Students, Knowledge Level, Artificial Intelligence, Student Evaluation, Test Reliability, Concept Formation, Developmentally Appropriate Practices, Grade 3, Grade 4, Grade 5 |
| DOI: | 10.1108/ILS-10-2023-0164 |
| ISSN: | 2398-5348 2398-5356 |
| Abstract: | Purpose: This paper aims to introduce a novel AI learning progression for upper-elementary students and aligns assessment items across levels of each construct to gather evidence of understanding. It also validates this quantitative measure by examining these items as two subscales for psychometric properties using the Rasch model. Design/methodology/approach: Conducting a cognitive analysis of diverse data sources, including the AI4K12 big ideas (Touretzky et al., 2019), student performance on assessment items, and classroom activities from prior implementations of an AI curriculum intervention (Glazewski et al., 2022), and drawing insights from subject matter experts, this paper outlines the design of the learning progression. The second section delves into the refinement and mapping of assessment items and an evaluation of their psychometric properties to ensure the reliable placement of students within the progression. Findings: This project identified key starting points for students and outlined how their understanding of core AI concepts should develop. The validation of the two subscales resulted in a reliable tool for accurately assessing students' AI abilities. This tool helps educators match assessment questions to students' current understanding and guide their progression through the learning journey. Originality/value: This learning progression offers a unique framework for teaching AI to younger students, addressing a gap in K-12 education. It provides a roadmap for progressively teaching AI concepts, allowing educators to design lessons and assessments that are appropriate for students' developmental stages. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1489034 |
| Database: | ERIC |
| Abstract: | Purpose: This paper aims to introduce a novel AI learning progression for upper-elementary students and aligns assessment items across levels of each construct to gather evidence of understanding. It also validates this quantitative measure by examining these items as two subscales for psychometric properties using the Rasch model. Design/methodology/approach: Conducting a cognitive analysis of diverse data sources, including the AI4K12 big ideas (Touretzky et al., 2019), student performance on assessment items, and classroom activities from prior implementations of an AI curriculum intervention (Glazewski et al., 2022), and drawing insights from subject matter experts, this paper outlines the design of the learning progression. The second section delves into the refinement and mapping of assessment items and an evaluation of their psychometric properties to ensure the reliable placement of students within the progression. Findings: This project identified key starting points for students and outlined how their understanding of core AI concepts should develop. The validation of the two subscales resulted in a reliable tool for accurately assessing students' AI abilities. This tool helps educators match assessment questions to students' current understanding and guide their progression through the learning journey. Originality/value: This learning progression offers a unique framework for teaching AI to younger students, addressing a gap in K-12 education. It provides a roadmap for progressively teaching AI concepts, allowing educators to design lessons and assessments that are appropriate for students' developmental stages. |
|---|---|
| ISSN: | 2398-5348 2398-5356 |
| DOI: | 10.1108/ILS-10-2023-0164 |
