View in EDS

Natural Language-Driven Teacher Gesture Recognition

Saved in:
Bibliographic Details
Title: Natural Language-Driven Teacher Gesture Recognition
Language: English
Authors: Yu Xiong, Shengyi Chen, Ting Cai, Lulu Chen, Jun Li
Source: International Educational Data Mining Society. 2025.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 7
Publication Date: 2025
Document Type: Speeches/Meeting Papers
Reports - Research
Descriptors: Artificial Intelligence, Natural Language Processing, Nonverbal Communication, Classroom Communication, Teacher Behavior, Video Technology, Classification, Classroom Environment, Physical Environment
Abstract: Teacher gesture recognition aims to identify and interpret teacher gestures within academic settings. It has been applied in domains such as teaching performance evaluation, the optimization of online education, and special needs education. However, the background similarity of teacher gestures, the inter-class similarity, and the intra-class variability limit the recognition capabilities of visual neural networks. In this paper, a Natural Language-Driven Teacher Gesture Recognition (NLD-TGR) framework is proposed. To mitigate the effects of background similarity, textual descriptions for each frame are generated using GPT-4o, guided by prompts specifically designed to describe the teacher's hand posture in the frames. Then, we combine video features with text features mapped to a high-dimensional space to create semantically-enhanced fused features. To overcome the limitations of one-hot labels in capturing inter-class and intra-class relationships, we embed semantically interpreted category names into a textual feature space. Gesture classification is then performed by computing the similarity between these textual embeddings and the fused feature representations. The experimental results validate the effectiveness of the proposed method, which achieves state-of-the-art performance with an accuracy of 93.7% on the TBU-G teacher gesture benchmark. [For the complete proceedings, see ED675583.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED675624
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED675624
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED675624
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Natural Language-Driven Teacher Gesture Recognition
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Yu+Xiong%22">Yu Xiong</searchLink><br /><searchLink fieldCode="AR" term="%22Shengyi+Chen%22">Shengyi Chen</searchLink><br /><searchLink fieldCode="AR" term="%22Ting+Cai%22">Ting Cai</searchLink><br /><searchLink fieldCode="AR" term="%22Lulu+Chen%22">Lulu Chen</searchLink><br /><searchLink fieldCode="AR" term="%22Jun+Li%22">Jun Li</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 7
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Artificial+Intelligence%22">Artificial Intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Natural+Language+Processing%22">Natural Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Nonverbal+Communication%22">Nonverbal Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Classroom+Communication%22">Classroom Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Behavior%22">Teacher Behavior</searchLink><br /><searchLink fieldCode="DE" term="%22Video+Technology%22">Video Technology</searchLink><br /><searchLink fieldCode="DE" term="%22Classification%22">Classification</searchLink><br /><searchLink fieldCode="DE" term="%22Classroom+Environment%22">Classroom Environment</searchLink><br /><searchLink fieldCode="DE" term="%22Physical+Environment%22">Physical Environment</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Teacher gesture recognition aims to identify and interpret teacher gestures within academic settings. It has been applied in domains such as teaching performance evaluation, the optimization of online education, and special needs education. However, the background similarity of teacher gestures, the inter-class similarity, and the intra-class variability limit the recognition capabilities of visual neural networks. In this paper, a Natural Language-Driven Teacher Gesture Recognition (NLD-TGR) framework is proposed. To mitigate the effects of background similarity, textual descriptions for each frame are generated using GPT-4o, guided by prompts specifically designed to describe the teacher's hand posture in the frames. Then, we combine video features with text features mapped to a high-dimensional space to create semantically-enhanced fused features. To overcome the limitations of one-hot labels in capturing inter-class and intra-class relationships, we embed semantically interpreted category names into a textual feature space. Gesture classification is then performed by computing the similarity between these textual embeddings and the fused feature representations. The experimental results validate the effectiveness of the proposed method, which achieves state-of-the-art performance with an accuracy of 93.7% on the TBU-G teacher gesture benchmark. [For the complete proceedings, see ED675583.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED675624
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED675624
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
    Subjects:
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Natural Language Processing
        Type: general
      – SubjectFull: Nonverbal Communication
        Type: general
      – SubjectFull: Classroom Communication
        Type: general
      – SubjectFull: Teacher Behavior
        Type: general
      – SubjectFull: Video Technology
        Type: general
      – SubjectFull: Classification
        Type: general
      – SubjectFull: Classroom Environment
        Type: general
      – SubjectFull: Physical Environment
        Type: general
    Titles:
      – TitleFull: Natural Language-Driven Teacher Gesture Recognition
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Yu Xiong
      – PersonEntity:
          Name:
            NameFull: Shengyi Chen
      – PersonEntity:
          Name:
            NameFull: Ting Cai
      – PersonEntity:
          Name:
            NameFull: Lulu Chen
      – PersonEntity:
          Name:
            NameFull: Jun Li
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1