Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills
Saved in:
| Title: | Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills |
|---|---|
| Authors: | Andrews-Todd, Jessica, Steinberg, Jonathan (ORCID |
| Source: | Journal of Intelligence. 2022 10. |
| Availability: | MDPI AG. Klybeckstrasse 64, 4057 Basel, Switzerland. e-mail: indexing@mdpi.com; e-mail: jintelligence@mdpi.com; Web site: https://www.mdpi.com/journal/jintelligence |
| Peer Reviewed: | Y |
| Page Count: | 24 |
| Publication Date: | 2022 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305A170432 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Automation, Classification, Cooperative Learning, Problem Solving, College Students, Competence, Artificial Intelligence, Documentation |
| ISSN: | 2079-3200 |
| Abstract: | Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals' CPS competency are necessary, as traditional assessment types such as multiple-choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals' CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students' display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts. [For the corresponding grantee submission, see ED621737.] |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2022 |
| Accession Number: | EJ1354059 |
| Database: | ERIC |
| Abstract: | Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals' CPS competency are necessary, as traditional assessment types such as multiple-choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals' CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students' display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts. [For the corresponding grantee submission, see ED621737.] |
|---|---|
| ISSN: | 2079-3200 |
