View in EDS

Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback

Saved in:
Bibliographic Details
Title: Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback
Language: English
Authors: Victor-Alexandru Padurean, Tung Phung, Nachiket Kotalwar, Michael Liut, Juho Leinonen, Paul Denny, Adish Singla
Source: International Educational Data Mining Society. 2025.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 8
Publication Date: 2025
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Automation, Student Writing Models, Feedback (Response), Programming, Computer Science Education, Artificial Intelligence, Natural Language Processing, Writing Evaluation, College Students, Higher Education, Foreign Countries
Geographic Terms: New Zealand
Abstract: The growing need for automated and personalized feedback in programming education has led to recent interest in leveraging generative AI for feedback generation. However, current approaches tend to rely on prompt engineering techniques in which predefined prompts guide the AI to generate feedback. This can result in rigid and constrained responses that fail to accommodate the diverse needs of students and do not reflect the style of human-written feedback from tutors or peers. In this study, we explore learnersourcing as a means to fine-tune language models for generating feedback that is more similar to that written by humans, particularly peer students. Specifically, we asked students to act in the flipped role of a tutor and write feedback on programs containing bugs. We collected approximately 1,900 instances of student-written feedback on multiple programming problems and buggy programs. To establish a baseline for comparison, we analyzed a sample of 300 instances based on correctness, length, and how the bugs are described. Using this data, we fine-tuned open-access generative models, specifically Llama3 and Phi3. Our findings indicate that fine-tuning models on learnersourced data not only produces feedback that better matches the style of feedback written by students, but also improves accuracy compared to feedback generated through prompt engineering alone, even though some student-written feedback is incorrect. This surprising finding highlights the potential of student-centered finetuning to improve automated feedback systems in programming education. [For the complete proceedings, see ED675583.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED675665
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED675665
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED675665
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Victor-Alexandru+Padurean%22">Victor-Alexandru Padurean</searchLink><br /><searchLink fieldCode="AR" term="%22Tung+Phung%22">Tung Phung</searchLink><br /><searchLink fieldCode="AR" term="%22Nachiket+Kotalwar%22">Nachiket Kotalwar</searchLink><br /><searchLink fieldCode="AR" term="%22Michael+Liut%22">Michael Liut</searchLink><br /><searchLink fieldCode="AR" term="%22Juho+Leinonen%22">Juho Leinonen</searchLink><br /><searchLink fieldCode="AR" term="%22Paul+Denny%22">Paul Denny</searchLink><br /><searchLink fieldCode="AR" term="%22Adish+Singla%22">Adish Singla</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 8
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Writing+Models%22">Student Writing Models</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Programming%22">Programming</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+Education%22">Computer Science Education</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22New+Zealand%22">New Zealand</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: The growing need for automated and personalized feedback in programming education has led to recent interest in leveraging generative AI for feedback generation. However, current approaches tend to rely on prompt engineering techniques in which predefined prompts guide the AI to generate feedback. This can result in rigid and constrained responses that fail to accommodate the diverse needs of students and do not reflect the style of human-written feedback from tutors or peers. In this study, we explore learnersourcing as a means to fine-tune language models for generating feedback that is more similar to that written by humans, particularly peer students. Specifically, we asked students to act in the flipped role of a tutor and write feedback on programs containing bugs. We collected approximately 1,900 instances of student-written feedback on multiple programming problems and buggy programs. To establish a baseline for comparison, we analyzed a sample of 300 instances based on correctness, length, and how the bugs are described. Using this data, we fine-tuned open-access generative models, specifically Llama3 and Phi3. Our findings indicate that fine-tuning models on learnersourced data not only produces feedback that better matches the style of feedback written by students, but also improves accuracy compared to feedback generated through prompt engineering alone, even though some student-written feedback is incorrect. This surprising finding highlights the potential of student-centered finetuning to improve automated feedback systems in programming education. [For the complete proceedings, see ED675583.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED675665
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED675665
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 8
    Subjects:
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Student Writing Models
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Programming
        Type: general
      – SubjectFull: Computer Science Education
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Higher Education
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: New Zealand
        Type: general
    Titles:
      – TitleFull: Humanizing Automated Programming Feedback: Fine-Tuning Generative Models with Student-Written Feedback
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Victor-Alexandru Padurean
      – PersonEntity:
          Name:
            NameFull: Tung Phung
      – PersonEntity:
          Name:
            NameFull: Nachiket Kotalwar
      – PersonEntity:
          Name:
            NameFull: Michael Liut
      – PersonEntity:
          Name:
            NameFull: Juho Leinonen
      – PersonEntity:
          Name:
            NameFull: Paul Denny
      – PersonEntity:
          Name:
            NameFull: Adish Singla
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1