View in EDS

Anchoring Concepts Influence Essay Conceptual Structure and Test Performance

Saved in:
Bibliographic Details
Title: Anchoring Concepts Influence Essay Conceptual Structure and Test Performance
Language: English
Authors: Roy B. Clariana, Ryan Solnosky
Source: International Association for Development of the Information Society. 2023.
Availability: International Association for the Development of the Information Society. e-mail: secretariat@iadis.org; Web site: http://www.iadisportal.org
Peer Reviewed: Y
Page Count: 8
Publication Date: 2023
Document Type: Speeches/Meeting Papers
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Comparative Analysis, Writing Evaluation, Essays, Word Frequency, Undergraduate Students, Engineering Education, Architectural Education, Textbooks, Lecture Method, Teaching Methods, Laboratories, Benchmarking, Multiple Choice Tests, Cues, Concept Formation, Computer Software, Artificial Intelligence, Computational Linguistics, Specialists, Networks, Peer Groups, Writing Instruction
Abstract: This quasi-experimental study seeks to improve the conceptual quality of summary essays by comparing two conditions, essay prompts with or without a list of 13 broad concepts, the concepts were selected across a continuum of the 100 most frequent words in the lesson materials. It is anticipated that only the most central concepts will be used as "anchors" when writing. Participants (n = 90) in an Architectural Engineering undergraduate course read the assigned lesson textbook chapter and attended lectures and labs, then in a final lab session were asked to write a 300-word summary of the lesson content. Data consists of the essays converted to networks and the end-of-unit multiple choice test. Compared to the expert network benchmark, the essay networks of those receiving the broad concepts in the writing prompt were not significantly different from those who did not receive these concepts. However those receiving the broad concepts were significantly more like peer essay networks (mental model convergence) and like the networks of the two PowerPoint lectures but neither were like the textbook chapter. Further, those receiving the broad concepts performed significantly better on the end-of-unit test than those not receiving the concepts. Term frequency analysis of the essays indicates as expected that the most network-central concepts had a greater frequency in essays, the other terms frequencies were remarkably the same for both the terms and no terms groups, suggesting a similar underlying conceptual mental model of this lesson content. To further explore the influence of anchoring concepts in summary writing prompts, essays were generated with the same two summary writing prompts using OpenAI (ChatGPT) and Google Bard, plus a new prompt that used the 13 most central concepts from the expert's network. The quality of the essay networks for both AI systems were equivalent to the students' essay networks for the broad concepts and for the no concept treatments. However, the AI essays derived with the 13 most central concepts were significantly better (more like the expert network) than the students and AI essays derived with broad concepts or no concepts treatments. In addition, Bard and OpenAI used several of the same concepts at a higher frequency than the students suggesting that the two AI systems have more similar knowledge graphs of this content. In sum, adding 13 broad conceptual terms to a summary writing prompt improved both structural and declarative knowledge outcomes, but adding 13 most central concepts may be even better. More research is needed to understand how including concepts and other terms in a writing prompt influences students' essay conceptual structure and subsequent test performance. [For the full proceedings, see ED636095.]
Abstractor: As Provided
Entry Date: 2024
Accession Number: ED636499
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED636499
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED636499
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Anchoring Concepts Influence Essay Conceptual Structure and Test Performance
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Roy+B%2E+Clariana%22">Roy B. Clariana</searchLink><br /><searchLink fieldCode="AR" term="%22Ryan+Solnosky%22">Ryan Solnosky</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Association+for+Development+of+the+Information+Society%22"><i>International Association for Development of the Information Society</i></searchLink>. 2023.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Association for the Development of the Information Society. e-mail: secretariat@iadis.org; Web site: http://www.iadisportal.org
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 8
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Evaluation%22">Writing Evaluation</searchLink><br /><searchLink fieldCode="DE" term="%22Essays%22">Essays</searchLink><br /><searchLink fieldCode="DE" term="%22Word+Frequency%22">Word Frequency</searchLink><br /><searchLink fieldCode="DE" term="%22Undergraduate+Students%22">Undergraduate Students</searchLink><br /><searchLink fieldCode="DE" term="%22Engineering+Education%22">Engineering Education</searchLink><br /><searchLink fieldCode="DE" term="%22Architectural+Education%22">Architectural Education</searchLink><br /><searchLink fieldCode="DE" term="%22Textbooks%22">Textbooks</searchLink><br /><searchLink fieldCode="DE" term="%22Lecture+Method%22">Lecture Method</searchLink><br /><searchLink fieldCode="DE" term="%22Teaching+Methods%22">Teaching Methods</searchLink><br /><searchLink fieldCode="DE" term="%22Laboratories%22">Laboratories</searchLink><br /><searchLink fieldCode="DE" term="%22Benchmarking%22">Benchmarking</searchLink><br /><searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Cues%22">Cues</searchLink><br /><searchLink fieldCode="DE" term="%22Concept+Formation%22">Concept Formation</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Software%22">Computer Software</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+Linguistics%22">Computational Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Specialists%22">Specialists</searchLink><br /><searchLink fieldCode="DE" term="%22Networks%22">Networks</searchLink><br /><searchLink fieldCode="DE" term="%22Peer+Groups%22">Peer Groups</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+Instruction%22">Writing Instruction</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This quasi-experimental study seeks to improve the conceptual quality of summary essays by comparing two conditions, essay prompts with or without a list of 13 broad concepts, the concepts were selected across a continuum of the 100 most frequent words in the lesson materials. It is anticipated that only the most central concepts will be used as "anchors" when writing. Participants (n = 90) in an Architectural Engineering undergraduate course read the assigned lesson textbook chapter and attended lectures and labs, then in a final lab session were asked to write a 300-word summary of the lesson content. Data consists of the essays converted to networks and the end-of-unit multiple choice test. Compared to the expert network benchmark, the essay networks of those receiving the broad concepts in the writing prompt were not significantly different from those who did not receive these concepts. However those receiving the broad concepts were significantly more like peer essay networks (mental model convergence) and like the networks of the two PowerPoint lectures but neither were like the textbook chapter. Further, those receiving the broad concepts performed significantly better on the end-of-unit test than those not receiving the concepts. Term frequency analysis of the essays indicates as expected that the most network-central concepts had a greater frequency in essays, the other terms frequencies were remarkably the same for both the terms and no terms groups, suggesting a similar underlying conceptual mental model of this lesson content. To further explore the influence of anchoring concepts in summary writing prompts, essays were generated with the same two summary writing prompts using OpenAI (ChatGPT) and Google Bard, plus a new prompt that used the 13 most central concepts from the expert's network. The quality of the essay networks for both AI systems were equivalent to the students' essay networks for the broad concepts and for the no concept treatments. However, the AI essays derived with the 13 most central concepts were significantly better (more like the expert network) than the students and AI essays derived with broad concepts or no concepts treatments. In addition, Bard and OpenAI used several of the same concepts at a higher frequency than the students suggesting that the two AI systems have more similar knowledge graphs of this content. In sum, adding 13 broad conceptual terms to a summary writing prompt improved both structural and declarative knowledge outcomes, but adding 13 most central concepts may be even better. More research is needed to understand how including concepts and other terms in a writing prompt influences students' essay conceptual structure and subsequent test performance. [For the full proceedings, see ED636095.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED636499
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED636499
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 8
    Subjects:
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Writing Evaluation
        Type: general
      – SubjectFull: Essays
        Type: general
      – SubjectFull: Word Frequency
        Type: general
      – SubjectFull: Undergraduate Students
        Type: general
      – SubjectFull: Engineering Education
        Type: general
      – SubjectFull: Architectural Education
        Type: general
      – SubjectFull: Textbooks
        Type: general
      – SubjectFull: Lecture Method
        Type: general
      – SubjectFull: Teaching Methods
        Type: general
      – SubjectFull: Laboratories
        Type: general
      – SubjectFull: Benchmarking
        Type: general
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Cues
        Type: general
      – SubjectFull: Concept Formation
        Type: general
      – SubjectFull: Computer Software
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Computational Linguistics
        Type: general
      – SubjectFull: Specialists
        Type: general
      – SubjectFull: Networks
        Type: general
      – SubjectFull: Peer Groups
        Type: general
      – SubjectFull: Writing Instruction
        Type: general
    Titles:
      – TitleFull: Anchoring Concepts Influence Essay Conceptual Structure and Test Performance
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Roy B. Clariana
      – PersonEntity:
          Name:
            NameFull: Ryan Solnosky
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Titles:
            – TitleFull: International Association for Development of the Information Society
              Type: main
ResultId 1