View in EDS

Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills

Saved in:
Bibliographic Details
Title: Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills
Authors: Andrews-Todd, Jessica, Steinberg, Jonathan (ORCID 0000-0002-6957-7735), Flor, Michael, Forsyth, Carolyn M.
Source: Journal of Intelligence. 2022 10.
Availability: MDPI AG. Klybeckstrasse 64, 4057 Basel, Switzerland. e-mail: indexing@mdpi.com; e-mail: jintelligence@mdpi.com; Web site: https://www.mdpi.com/journal/jintelligence
Peer Reviewed: Y
Page Count: 24
Publication Date: 2022
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305A170432
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Automation, Classification, Cooperative Learning, Problem Solving, College Students, Competence, Artificial Intelligence, Documentation
ISSN: 2079-3200
Abstract: Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals' CPS competency are necessary, as traditional assessment types such as multiple-choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals' CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students' display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts. [For the corresponding grantee submission, see ED621737.]
Abstractor: As Provided
IES Funded: Yes
Entry Date: 2022
Accession Number: EJ1354059
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1354059
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1354059
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Andrews-Todd%2C+Jessica%22">Andrews-Todd, Jessica</searchLink><br /><searchLink fieldCode="AR" term="%22Steinberg%2C+Jonathan%22">Steinberg, Jonathan</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-6957-7735">0000-0002-6957-7735</externalLink>)<br /><searchLink fieldCode="AR" term="%22Flor%2C+Michael%22">Flor, Michael</searchLink><br /><searchLink fieldCode="AR" term="%22Forsyth%2C+Carolyn+M%2E%22">Forsyth, Carolyn M.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Intelligence%22"><i>Journal of Intelligence</i></searchLink>. 2022 10.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: MDPI AG. Klybeckstrasse 64, 4057 Basel, Switzerland. e-mail: indexing@mdpi.com; e-mail: jintelligence@mdpi.com; Web site: https://www.mdpi.com/journal/jintelligence
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305A170432
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Cooperative+Learning%22">Cooperative Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Competence%22">Competence</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Documentation%22">Documentation</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2079-3200
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Competency in skills associated with collaborative problem solving (CPS) is critical for many contexts, including school, the workplace, and the military. Innovative approaches for assessing individuals' CPS competency are necessary, as traditional assessment types such as multiple-choice items are not well suited for such a process-oriented competency. In a move to computer-based environments to support CPS assessment, innovative computational approaches are also needed to understand individuals' CPS behaviors. In the current study, we describe the use of a simulation-based task on electronics concepts as an environment for higher education students to display evidence of their CPS competency. We further describe computational linguistic methods for automatically characterizing students' display of various CPS skills in the task. Comparisons between such an automated approach and an approach based on human annotation to characterize student CPS behaviors revealed above average agreement. These results give credence to the potential for automated approaches to help advance the assessment of CPS and to circumvent the time-intensive human annotation approaches that are typically used in these contexts. [For the corresponding grantee submission, see ED621737.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2022
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1354059
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1354059
RecordInfo BibRecord:
  BibEntity:
    PhysicalDescription:
      Pagination:
        PageCount: 24
    Subjects:
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Cooperative Learning
        Type: general
      – SubjectFull: Problem Solving
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Competence
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Documentation
        Type: general
    Titles:
      – TitleFull: Exploring Automated Classification Approaches to Advance the Assessment of Collaborative Problem Solving Skills
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Andrews-Todd, Jessica
      – PersonEntity:
          Name:
            NameFull: Steinberg, Jonathan
      – PersonEntity:
          Name:
            NameFull: Flor, Michael
      – PersonEntity:
          Name:
            NameFull: Forsyth, Carolyn M.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-electronic
              Value: 2079-3200
          Numbering:
            – Type: volume
              Value: 10
          Titles:
            – TitleFull: Journal of Intelligence
              Type: main
ResultId 1