View in EDS

Comparison of AI-Generated and Instructor Feedback: No Significant Difference in Perceived Feedback Quality and Neither on Performance

Saved in:
Bibliographic Details
Title: Comparison of AI-Generated and Instructor Feedback: No Significant Difference in Perceived Feedback Quality and Neither on Performance
Language: English
Authors: Seyma Çaglar-Özhan, Perihan Tekeli, Selay Arkün-Kocadere
Source: Journal of Computer Assisted Learning. 2025 41(5).
Availability: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Peer Reviewed: Y
Page Count: 11
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Feedback (Response), Artificial Intelligence, Teacher Role, Technology Uses in Education, Program Effectiveness, Student Projects, College Students, Proposal Writing
DOI: 10.1111/jcal.70134
ISSN: 0266-4909
1365-2729
Abstract: Background: Feedback is an essential part of the educational process as it enriches students' learning experiences, provides information about their current performance, shows them what is lacking in achieving goals, and provides guidance on the strategies needed to achieve those goals. Teachers, especially in crowded classrooms, often have difficulty allocating enough time to provide feedback. ChatGPT has the potential to deliver feedback more efficiently and effectively. Objective: This study aims to investigate the effectiveness of ChatGPT-generated feedback in students' project proposal development process. Therefore, ChatGPT-generated and instructor's feedback are compared in terms of perceived feedback quality and performance of students. Method: In the study, a post-test control group experimental design was used. A total of 48 students enrolled in a blended course at a state university participated in the study for 6 weeks. The study was conducted using a randomised post-test-only control group experimental design. Students were asked to prepare a project proposal and feedback was given on each task. While the control group received instructor feedback, the experimental group received ChatGPT feedback. Data were collected via the Formative Feedback Perceptions Scale, and the students' project proposals were graded based on the rubric to evaluate their performance. Results and Conclusion: According to the Mann-Whitney U test analysis, there was no statistically significant difference between the experimental and the control group regarding both students' perceived feedback quality and performance. In conclusion, it can be argued that when guided by the appropriate prompts and asked to examine assignments of similar complexity to those in this study, ChatGPT can provide feedback of similar quality and equivalence to the feedback provided by instructors. In this way, it seems possible to improve students' learning as much as an instructor with individualised feedback by ChatGPT. Finally, it should be noted that these findings are within the limitations of the specific context of project proposal development, reliance on self-reported data, and the experimental nature of the study; it is recommended to evaluate the effectiveness of AI-generated feedback in different contexts.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1484357
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1484357
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Comparison of AI-Generated and Instructor Feedback: No Significant Difference in Perceived Feedback Quality and Neither on Performance
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Seyma+Çaglar-Özhan%22">Seyma Çaglar-Özhan</searchLink><br /><searchLink fieldCode="AR" term="%22Perihan+Tekeli%22">Perihan Tekeli</searchLink><br /><searchLink fieldCode="AR" term="%22Selay+Arkün-Kocadere%22">Selay Arkün-Kocadere</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Computer+Assisted+Learning%22"><i>Journal of Computer Assisted Learning</i></searchLink>. 2025 41(5).
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 11
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Role%22">Teacher Role</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Program+Effectiveness%22">Program Effectiveness</searchLink><br /><searchLink fieldCode="DE" term="%22Student+Projects%22">Student Projects</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Proposal+Writing%22">Proposal Writing</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1111/jcal.70134
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0266-4909<br />1365-2729
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Feedback is an essential part of the educational process as it enriches students' learning experiences, provides information about their current performance, shows them what is lacking in achieving goals, and provides guidance on the strategies needed to achieve those goals. Teachers, especially in crowded classrooms, often have difficulty allocating enough time to provide feedback. ChatGPT has the potential to deliver feedback more efficiently and effectively. Objective: This study aims to investigate the effectiveness of ChatGPT-generated feedback in students' project proposal development process. Therefore, ChatGPT-generated and instructor's feedback are compared in terms of perceived feedback quality and performance of students. Method: In the study, a post-test control group experimental design was used. A total of 48 students enrolled in a blended course at a state university participated in the study for 6 weeks. The study was conducted using a randomised post-test-only control group experimental design. Students were asked to prepare a project proposal and feedback was given on each task. While the control group received instructor feedback, the experimental group received ChatGPT feedback. Data were collected via the Formative Feedback Perceptions Scale, and the students' project proposals were graded based on the rubric to evaluate their performance. Results and Conclusion: According to the Mann-Whitney U test analysis, there was no statistically significant difference between the experimental and the control group regarding both students' perceived feedback quality and performance. In conclusion, it can be argued that when guided by the appropriate prompts and asked to examine assignments of similar complexity to those in this study, ChatGPT can provide feedback of similar quality and equivalence to the feedback provided by instructors. In this way, it seems possible to improve students' learning as much as an instructor with individualised feedback by ChatGPT. Finally, it should be noted that these findings are within the limitations of the specific context of project proposal development, reliance on self-reported data, and the experimental nature of the study; it is recommended to evaluate the effectiveness of AI-generated feedback in different contexts.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1484357
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1484357
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/jcal.70134
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
    Subjects:
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Teacher Role
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Program Effectiveness
        Type: general
      – SubjectFull: Student Projects
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Proposal Writing
        Type: general
    Titles:
      – TitleFull: Comparison of AI-Generated and Instructor Feedback: No Significant Difference in Perceived Feedback Quality and Neither on Performance
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Seyma Çaglar-Özhan
      – PersonEntity:
          Name:
            NameFull: Perihan Tekeli
      – PersonEntity:
          Name:
            NameFull: Selay Arkün-Kocadere
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 0266-4909
            – Type: issn-electronic
              Value: 1365-2729
          Numbering:
            – Type: volume
              Value: 41
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: Journal of Computer Assisted Learning
              Type: main
ResultId 1