View in EDS

Improving the Generalizability of Models of Collaborative Discourse

Saved in:
Bibliographic Details
Title: Improving the Generalizability of Models of Collaborative Discourse
Language: English
Authors: Chelsea Chandler, Rohit Raju, Jason G. Reitman, William R. Penuel, Monica Ko, Jeffrey B. Bush, Quentin Biddy, Sidney K. D’Mello
Source: International Educational Data Mining Society. 2025.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 13
Publication Date: 2025
Sponsoring Agency: National Science Foundation (NSF), Division of Research on Learning in Formal and Informal Settings (DRL)
Contract Number: 2019805
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Junior High Schools
Middle Schools
Secondary Education
Higher Education
Postsecondary Education
Descriptors: Artificial Intelligence, Models, Natural Language Processing, Discourse Analysis, Classification, Generalization, Middle School Students, College Students, Cooperative Learning
Abstract: We investigated methods to enhance the generalizability of large language models (LLMs) designed to classify dimensions of collaborative discourse during small group work. Our research utilized five diverse datasets that spanned various grade levels, demographic groups, collaboration settings, and curriculum units. We explored different model training techniques with RoBERTa and Mistral LLMs, including traditional fine-tuning, data augmentation paired with fine-tuning, and prompting. Our findings revealed that traditionally fine-tuning RoBERTa on a single dataset (serving as our baseline) led to overfitting, with the model failing to generalize beyond the training data's specific curriculum and language patterns. In contrast, fine-tuning RoBERTa with embedding augmented data led to significant improvements in generalization, as did pairing Mistral embeddings with a support vector machine classifier. However, fine-tuning and few-shot prompting Mistral did not yield similar improvements. Our findings highlight scalable alternatives to the resource-intensive process of curating labeled datasets for each new application, offering practical strategies to enhance model adaptability in diverse educational settings. [For the complete proceedings, see ED675583.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED675588
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED675588
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED675588
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Improving the Generalizability of Models of Collaborative Discourse
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Chelsea+Chandler%22">Chelsea Chandler</searchLink><br /><searchLink fieldCode="AR" term="%22Rohit+Raju%22">Rohit Raju</searchLink><br /><searchLink fieldCode="AR" term="%22Jason+G%2E+Reitman%22">Jason G. Reitman</searchLink><br /><searchLink fieldCode="AR" term="%22William+R%2E+Penuel%22">William R. Penuel</searchLink><br /><searchLink fieldCode="AR" term="%22Monica+Ko%22">Monica Ko</searchLink><br /><searchLink fieldCode="AR" term="%22Jeffrey+B%2E+Bush%22">Jeffrey B. Bush</searchLink><br /><searchLink fieldCode="AR" term="%22Quentin+Biddy%22">Quentin Biddy</searchLink><br /><searchLink fieldCode="AR" term="%22Sidney+K%2E+D%27Mello%22">Sidney K. D’Mello</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 13
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: National Science Foundation (NSF), Division of Research on Learning in Formal and Informal Settings (DRL)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: 2019805
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Junior+High+Schools%22">Junior High Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Middle+Schools%22">Middle Schools</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Models%22">Models</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Discourse+Analysis%22">Discourse Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Generalization%22">Generalization</searchLink><br /><searchLink fieldCode="DE" term="%22Middle+School+Students%22">Middle School Students</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Cooperative+Learning%22">Cooperative Learning</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: We investigated methods to enhance the generalizability of large language models (LLMs) designed to classify dimensions of collaborative discourse during small group work. Our research utilized five diverse datasets that spanned various grade levels, demographic groups, collaboration settings, and curriculum units. We explored different model training techniques with RoBERTa and Mistral LLMs, including traditional fine-tuning, data augmentation paired with fine-tuning, and prompting. Our findings revealed that traditionally fine-tuning RoBERTa on a single dataset (serving as our baseline) led to overfitting, with the model failing to generalize beyond the training data's specific curriculum and language patterns. In contrast, fine-tuning RoBERTa with embedding augmented data led to significant improvements in generalization, as did pairing Mistral embeddings with a support vector machine classifier. However, fine-tuning and few-shot prompting Mistral did not yield similar improvements. Our findings highlight scalable alternatives to the resource-intensive process of curating labeled datasets for each new application, offering practical strategies to enhance model adaptability in diverse educational settings. [For the complete proceedings, see ED675583.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED675588
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED675588
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Models
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Discourse Analysis
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Generalization
        Type: general
      – SubjectFull: Middle School Students
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Cooperative Learning
        Type: general
    Titles:
      – TitleFull: Improving the Generalizability of Models of Collaborative Discourse
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Chelsea Chandler
      – PersonEntity:
          Name:
            NameFull: Rohit Raju
      – PersonEntity:
          Name:
            NameFull: Jason G. Reitman
      – PersonEntity:
          Name:
            NameFull: William R. Penuel
      – PersonEntity:
          Name:
            NameFull: Monica Ko
      – PersonEntity:
          Name:
            NameFull: Jeffrey B. Bush
      – PersonEntity:
          Name:
            NameFull: Quentin Biddy
      – PersonEntity:
          Name:
            NameFull: Sidney K. D’Mello
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1