View in EDS

Automated Text Analysis of Organic Chemistry Students' Written Hypotheses

Saved in:
Bibliographic Details
Title: Automated Text Analysis of Organic Chemistry Students' Written Hypotheses
Language: English
Authors: Amber J. Dood (ORCID 0000-0003-4572-1402), Field M. Watts (ORCID 0000-0002-1800-1816), Megan C. Connor (ORCID 0000-0003-3266-4162), Ginger V. Shultz (ORCID 0000-0002-7285-748X)
Source: Journal of Chemical Education. 2024 101(3):807-818.
Availability: Division of Chemical Education, Inc. and ACS Publications Division of the American Chemical Society. 1155 Sixteenth Street NW, Washington, DC 20036. Tel: 800-227-5558; Tel: 202-872-4600; e-mail: eic@jce.acs.org; Web site: http://pubs.acs.org/jchemeduc
Peer Reviewed: Y
Page Count: 12
Publication Date: 2024
Document Type: Journal Articles
Reports - Research
Descriptors: Automation, Organic Chemistry, Writing (Composition), Scientific Concepts, Content Analysis, Hypothesis Testing, Feedback (Response), Artificial Intelligence, Technology Uses in Education, Science Education
DOI: 10.1021/acs.jchemed.3c00757
ISSN: 0021-9584
1938-1328
Abstract: Generating a testable hypothesis is a necessary skill for engaging in science, requiring both general reasoning skills and specific content knowledge of the phenomenon being investigated. While many students have the reasoning skills necessary for developing testable hypotheses in a general science context, it can be challenging for students to apply these skills to content areas where the phenomena being studied are unobservable, such as problems in organic chemistry. Generating hypotheses for experiments is a skill that students are expected to apply in chemistry teaching laboratories. Students can benefit from receiving real-time feedback to support developing this ability. However, providing feedback on writing can be challenging for instructors, especially in large-enrollment courses. For this study, we explored and compared the performance of several machine learning algorithms to classify organic chemistry students' written hypotheses for a science-general scenario and an organic chemistry-specific scenario. These models were trained to analyze students' written hypotheses for the presence of five features: (1) a testable prediction in a given situation; (2) a predicted change in the independent variable that is related to the dependent variable; (3) scientific content addressing the research question provided in the prompt; (4) a correct and completely defined independent variable; and (5) a correct and completely defined dependent variable. This work has implications for guiding future research on the feasibility of providing instructors with information about students' hypothesis-generating abilities that can support the delivery of real-time feedback.
Abstractor: As Provided
Entry Date: 2024
Accession Number: EJ1452930
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1452930
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Automated Text Analysis of Organic Chemistry Students' Written Hypotheses
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Amber+J%2E+Dood%22">Amber J. Dood</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-4572-1402">0000-0003-4572-1402</externalLink>)<br /><searchLink fieldCode="AR" term="%22Field+M%2E+Watts%22">Field M. Watts</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-1800-1816">0000-0002-1800-1816</externalLink>)<br /><searchLink fieldCode="AR" term="%22Megan+C%2E+Connor%22">Megan C. Connor</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-3266-4162">0000-0003-3266-4162</externalLink>)<br /><searchLink fieldCode="AR" term="%22Ginger+V%2E+Shultz%22">Ginger V. Shultz</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0002-7285-748X">0000-0002-7285-748X</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Journal+of+Chemical+Education%22"><i>Journal of Chemical Education</i></searchLink>. 2024 101(3):807-818.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Division of Chemical Education, Inc. and ACS Publications Division of the American Chemical Society. 1155 Sixteenth Street NW, Washington, DC 20036. Tel: 800-227-5558; Tel: 202-872-4600; e-mail: eic@jce.acs.org; Web site: http://pubs.acs.org/jchemeduc
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 12
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Automation%22">Automation</searchLink><br /><searchLink fieldCode="DE" term="%22Organic+Chemistry%22">Organic Chemistry</searchLink><br /><searchLink fieldCode="DE" term="%22Writing+%28Composition%29%22">Writing (Composition)</searchLink><br /><searchLink fieldCode="DE" term="%22Scientific+Concepts%22">Scientific Concepts</searchLink><br /><searchLink fieldCode="DE" term="%22Content+Analysis%22">Content Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Hypothesis+Testing%22">Hypothesis Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Technology+Uses+in+Education%22">Technology Uses in Education</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Education%22">Science Education</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1021/acs.jchemed.3c00757
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0021-9584<br />1938-1328
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Generating a testable hypothesis is a necessary skill for engaging in science, requiring both general reasoning skills and specific content knowledge of the phenomenon being investigated. While many students have the reasoning skills necessary for developing testable hypotheses in a general science context, it can be challenging for students to apply these skills to content areas where the phenomena being studied are unobservable, such as problems in organic chemistry. Generating hypotheses for experiments is a skill that students are expected to apply in chemistry teaching laboratories. Students can benefit from receiving real-time feedback to support developing this ability. However, providing feedback on writing can be challenging for instructors, especially in large-enrollment courses. For this study, we explored and compared the performance of several machine learning algorithms to classify organic chemistry students' written hypotheses for a science-general scenario and an organic chemistry-specific scenario. These models were trained to analyze students' written hypotheses for the presence of five features: (1) a testable prediction in a given situation; (2) a predicted change in the independent variable that is related to the dependent variable; (3) scientific content addressing the research question provided in the prompt; (4) a correct and completely defined independent variable; and (5) a correct and completely defined dependent variable. This work has implications for guiding future research on the feasibility of providing instructors with information about students' hypothesis-generating abilities that can support the delivery of real-time feedback.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2024
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1452930
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1452930
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1021/acs.jchemed.3c00757
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 807
    Subjects:
      – SubjectFull: Automation
        Type: general
      – SubjectFull: Organic Chemistry
        Type: general
      – SubjectFull: Writing (Composition)
        Type: general
      – SubjectFull: Scientific Concepts
        Type: general
      – SubjectFull: Content Analysis
        Type: general
      – SubjectFull: Hypothesis Testing
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Technology Uses in Education
        Type: general
      – SubjectFull: Science Education
        Type: general
    Titles:
      – TitleFull: Automated Text Analysis of Organic Chemistry Students' Written Hypotheses
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Amber J. Dood
      – PersonEntity:
          Name:
            NameFull: Field M. Watts
      – PersonEntity:
          Name:
            NameFull: Megan C. Connor
      – PersonEntity:
          Name:
            NameFull: Ginger V. Shultz
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 03
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 0021-9584
            – Type: issn-electronic
              Value: 1938-1328
          Numbering:
            – Type: volume
              Value: 101
            – Type: issue
              Value: 3
          Titles:
            – TitleFull: Journal of Chemical Education
              Type: main
ResultId 1