View in EDS

A Comparison of Generative AI Solutions and Textbook Solutions in an Introductory Programming Course

Saved in:
Bibliographic Details
Title: A Comparison of Generative AI Solutions and Textbook Solutions in an Introductory Programming Course
Language: English
Authors: Ernst Bekkering, Patrick Harrington
Source: Information Systems Education Journal. 2025 23(1):4-22.
Availability: Information Systems and Computing Academic Professionals. Box 488, Wrightsville Beach, NC 28480. e-mail: publisher@isedj.org; Web site: http://isedj.org
Peer Reviewed: Y
Page Count: 19
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Higher Education, Artificial Intelligence, Programming, Textbook Evaluation, Comparative Analysis, Technology Uses in Education, Technology Integration, Technological Advancement, Computer Science Education, Problem Solving
ISSN: 1545-679X
Abstract: Generative AI has recently gained the ability to generate computer code. This development is bound to affect how computer programming is taught in higher education. We used past programming assignments and solutions for textbook exercises in our introductory programming class to analyze how accurately one of the leading models, ChatGPT, generates solutions. We selected the ChatGPT-4 available through the Bing search engine for our testing. We used a one-tailed test to calculate success percentage of the textbook versus ChatGPT solutions to determine if there was a statistically significant difference. Neither the book nor ChatGPT provided perfect solutions. Analysis of the results showed that the generated code does not always meet the programming requirements, but also that instructions for generative AI coding and for traditional programming can be improved. We conclude with recommendations for incorporating generative AI in programming classes. [Note: The publication year (2024) shown in the citation on the PDF is incorrect. The correct publication year is 2025.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1462116
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1462116
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1462116
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A Comparison of Generative AI Solutions and Textbook Solutions in an Introductory Programming Course
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Ernst+Bekkering%22">Ernst Bekkering</searchLink><br /><searchLink fieldCode="AR" term="%22Patrick+Harrington%22">Patrick Harrington</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Information+Systems+Education+Journal%22"><i>Information Systems Education Journal</i></searchLink>. 2025 23(1):4-22.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Information Systems and Computing Academic Professionals. Box 488, Wrightsville Beach, NC 28480. e-mail: publisher@isedj.org; Web site: http://isedj.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 19
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Programming%22">Programming</searchLink><br /><searchLink fieldCode="DE" term="%22Textbook+Evaluation%22">Textbook Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Integration%22">Technology Integration</searchLink><br /><searchLink fieldCode="DE" term="%22Technological+Advancement%22">Technological Advancement</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+Education%22">Computer Science Education</searchLink><br /><searchLink fieldCode="DE" term="%22Problem+Solving%22">Problem Solving</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1545-679X
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Generative AI has recently gained the ability to generate computer code. This development is bound to affect how computer programming is taught in higher education. We used past programming assignments and solutions for textbook exercises in our introductory programming class to analyze how accurately one of the leading models, ChatGPT, generates solutions. We selected the ChatGPT-4 available through the Bing search engine for our testing. We used a one-tailed test to calculate success percentage of the textbook versus ChatGPT solutions to determine if there was a statistically significant difference. Neither the book nor ChatGPT provided perfect solutions. Analysis of the results showed that the generated code does not always meet the programming requirements, but also that instructions for generative AI coding and for traditional programming can be improved. We conclude with recommendations for incorporating generative AI in programming classes. [Note: The publication year (2024) shown in the citation on the PDF is incorrect. The correct publication year is 2025.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1462116
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1462116
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 19
        StartPage: 4
    Subjects:
      – SubjectFull: Higher Education
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Programming
        Type: general
      – SubjectFull: Textbook Evaluation
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Technology Integration
        Type: general
      – SubjectFull: Technological Advancement
        Type: general
      – SubjectFull: Computer Science Education
        Type: general
      – SubjectFull: Problem Solving
        Type: general
    Titles:
      – TitleFull: A Comparison of Generative AI Solutions and Textbook Solutions in an Introductory Programming Course
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ernst Bekkering
      – PersonEntity:
          Name:
            NameFull: Patrick Harrington
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-electronic
              Value: 1545-679X
          Numbering:
            – Type: volume
              Value: 23
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Information Systems Education Journal
              Type: main
ResultId 1