View in EDS

Meta-Learning for Better Learning: Using Meta-Learning Methods to Automatically Label Exam Questions with Detailed Learning Objectives

Saved in:
Bibliographic Details
Title: Meta-Learning for Better Learning: Using Meta-Learning Methods to Automatically Label Exam Questions with Detailed Learning Objectives
Language: English
Authors: Zur, Amir, Applebaum, Isaac, Nardo, Jocelyn Elizabeth, DeWeese, Dory, Sundrani, Sameer, Salehi, Shima
Source: International Educational Data Mining Society. 2023.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 10
Publication Date: 2023
Document Type: Speeches/Meeting Papers
Reports - Research
Descriptors: Equal Education, Prior Learning, Educational Objectives, Chemistry, Physics, Textbooks, Benchmarking, Science Instruction, Feedback (Response), Computer Assisted Testing, Classification, Networks, Computational Linguistics, Test Items
Abstract: Detailed learning objectives foster an effective and equitable learning environment by clarifying what instructors expect students to learn, rather than requiring students to use prior knowledge to infer these expectations. When questions are labeled with relevant learning goals, students understand which skills are tested by those questions. Labeling also helps instructors provide personalized feedback based on the learning objectives each student struggles to master. However, developing detailed learning objectives is time-consuming, making many instructors unable to pursue it. Labeling course questions with learning objectives can be even more time-intensive. To address this challenge, we develop a benchmark for automatically labeling questions with learning objectives. The benchmark comprises 4,875 questions and 1,267 expert-verified learning objectives from college physics and chemistry textbooks. This dataset provides a large library of learning objectives, and, to the best of our knowledge, is the first benchmark to measure performance on labeling questions with learning objectives. We use meta-learning methods to train classifiers and test them against our benchmark in a few-shot classification setting. These classifiers achieve acceptable performance on a test set with previously unseen questions (AUC 0.84), as well as a course with previously unseen questions and unseen learning objectives (AUC 0.84). Our work facilitates labeling questions with learning objectives to help instructors provide better feedback and create equitable learning environments. [For the complete proceedings, see ED630829.]
Abstractor: As Provided
Notes: https://github.com/AmirZur/smartstem-ai
Entry Date: 2023
Accession Number: ED630880
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED630880
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED630880
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Meta-Learning for Better Learning: Using Meta-Learning Methods to Automatically Label Exam Questions with Detailed Learning Objectives
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Zur%2C+Amir%22">Zur, Amir</searchLink><br /><searchLink fieldCode="AR" term="%22Applebaum%2C+Isaac%22">Applebaum, Isaac</searchLink><br /><searchLink fieldCode="AR" term="%22Nardo%2C+Jocelyn+Elizabeth%22">Nardo, Jocelyn Elizabeth</searchLink><br /><searchLink fieldCode="AR" term="%22DeWeese%2C+Dory%22">DeWeese, Dory</searchLink><br /><searchLink fieldCode="AR" term="%22Sundrani%2C+Sameer%22">Sundrani, Sameer</searchLink><br /><searchLink fieldCode="AR" term="%22Salehi%2C+Shima%22">Salehi, Shima</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2023.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 10
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Equal+Education%22">Equal Education</searchLink><br /><searchLink fieldCode="DE" term="%22Prior+Learning%22">Prior Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Educational+Objectives%22">Educational Objectives</searchLink><br /><searchLink fieldCode="DE" term="%22Chemistry%22">Chemistry</searchLink><br /><searchLink fieldCode="DE" term="%22Physics%22">Physics</searchLink><br /><searchLink fieldCode="DE" term="%22Textbooks%22">Textbooks</searchLink><br /><searchLink fieldCode="DE" term="%22Benchmarking%22">Benchmarking</searchLink><br /><searchLink fieldCode="DE" term="%22Science+Instruction%22">Science Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Feedback+%28Response%29%22">Feedback (Response)</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Assisted+Testing%22">Computer Assisted Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Networks%22">Networks</searchLink><br /><searchLink fieldCode="DE" term="%22Computational+Linguistics%22">Computational Linguistics</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Detailed learning objectives foster an effective and equitable learning environment by clarifying what instructors expect students to learn, rather than requiring students to use prior knowledge to infer these expectations. When questions are labeled with relevant learning goals, students understand which skills are tested by those questions. Labeling also helps instructors provide personalized feedback based on the learning objectives each student struggles to master. However, developing detailed learning objectives is time-consuming, making many instructors unable to pursue it. Labeling course questions with learning objectives can be even more time-intensive. To address this challenge, we develop a benchmark for automatically labeling questions with learning objectives. The benchmark comprises 4,875 questions and 1,267 expert-verified learning objectives from college physics and chemistry textbooks. This dataset provides a large library of learning objectives, and, to the best of our knowledge, is the first benchmark to measure performance on labeling questions with learning objectives. We use meta-learning methods to train classifiers and test them against our benchmark in a few-shot classification setting. These classifiers achieve acceptable performance on a test set with previously unseen questions (AUC 0.84), as well as a course with previously unseen questions and unseen learning objectives (AUC 0.84). Our work facilitates labeling questions with learning objectives to help instructors provide better feedback and create equitable learning environments. [For the complete proceedings, see ED630829.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: Note
  Label: Notes
  Group: Note
  Data: https://github.com/AmirZur/smartstem-ai
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED630880
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED630880
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
    Subjects:
      – SubjectFull: Equal Education
        Type: general
      – SubjectFull: Prior Learning
        Type: general
      – SubjectFull: Educational Objectives
        Type: general
      – SubjectFull: Chemistry
        Type: general
      – SubjectFull: Physics
        Type: general
      – SubjectFull: Textbooks
        Type: general
      – SubjectFull: Benchmarking
        Type: general
      – SubjectFull: Science Instruction
        Type: general
      – SubjectFull: Feedback (Response)
        Type: general
      – SubjectFull: Computer Assisted Testing
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Networks
        Type: general
      – SubjectFull: Computational Linguistics
        Type: general
      – SubjectFull: Test Items
        Type: general
    Titles:
      – TitleFull: Meta-Learning for Better Learning: Using Meta-Learning Methods to Automatically Label Exam Questions with Detailed Learning Objectives
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Zur, Amir
      – PersonEntity:
          Name:
            NameFull: Applebaum, Isaac
      – PersonEntity:
          Name:
            NameFull: Nardo, Jocelyn Elizabeth
      – PersonEntity:
          Name:
            NameFull: DeWeese, Dory
      – PersonEntity:
          Name:
            NameFull: Sundrani, Sameer
      – PersonEntity:
          Name:
            NameFull: Salehi, Shima
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2023
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1