View in EDS

Multimodal, Multi-Class Bias Mitigation for Predicting Speaker Confidence

Saved in:
Bibliographic Details
Title: Multimodal, Multi-Class Bias Mitigation for Predicting Speaker Confidence
Language: English
Authors: Andrew Emerson, Arti Ramesh, Patrick Houghton, Vinay Basheerabad, Navaneeth Jawahar, Chee Wee Leong
Source: International Educational Data Mining Society. 2024.
Availability: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
Peer Reviewed: Y
Page Count: 7
Publication Date: 2024
Document Type: Speeches/Meeting Papers
Reports - Research
Descriptors: Self Esteem, Prediction, Cues, Bias, Adults, Speech Communication, Nonverbal Communication
Abstract: Projecting confidence during conversation or presentation is a critical skill. To effectively display confidence, speakers must employ a blend of verbal and non-verbal signals. A predictive model that leverages rich multimodal cues to measure a speaker's confidence must also mitigate biases that develop through data labelling practices, inherent imbalances in the demographic distribution, or biases introduced into the model during the training process. Fairly predicting the confidence of speakers across differing backgrounds enables more accurate and actionable feedback to a larger population of speakers. This paper introduces a set of approaches for bias mitigation for multimodal, multi-class confidence prediction of adult speakers in a work-like setting. We evaluate the extent to which bias mitigation techniques improve the performance of a multimodal confidence classifier with a dataset of 233 2-minute videos. Experimental results suggest that by bounding the loss across perceived races, genders, accents, and ages, multimodal models can significantly outperform unmitigated baselines. The implications, including automated feedback of speaker confidence, are discussed. [For the complete proceedings, see ED675485.]
Abstractor: As Provided
Entry Date: 2025
Accession Number: ED675542
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED675542
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED675542
AccessLevel: 3
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Multimodal, Multi-Class Bias Mitigation for Predicting Speaker Confidence
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Andrew+Emerson%22">Andrew Emerson</searchLink><br /><searchLink fieldCode="AR" term="%22Arti+Ramesh%22">Arti Ramesh</searchLink><br /><searchLink fieldCode="AR" term="%22Patrick+Houghton%22">Patrick Houghton</searchLink><br /><searchLink fieldCode="AR" term="%22Vinay+Basheerabad%22">Vinay Basheerabad</searchLink><br /><searchLink fieldCode="AR" term="%22Navaneeth+Jawahar%22">Navaneeth Jawahar</searchLink><br /><searchLink fieldCode="AR" term="%22Chee+Wee+Leong%22">Chee Wee Leong</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22International+Educational+Data+Mining+Society%22"><i>International Educational Data Mining Society</i></searchLink>. 2024.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 7
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2024
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Speeches/Meeting Papers<br />Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Self+Esteem%22">Self Esteem</searchLink><br /><searchLink fieldCode="DE" term="%22Prediction%22">Prediction</searchLink><br /><searchLink fieldCode="DE" term="%22Cues%22">Cues</searchLink><br /><searchLink fieldCode="DE" term="%22Bias%22">Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Adults%22">Adults</searchLink><br /><searchLink fieldCode="DE" term="%22Speech+Communication%22">Speech Communication</searchLink><br /><searchLink fieldCode="DE" term="%22Nonverbal+Communication%22">Nonverbal Communication</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Projecting confidence during conversation or presentation is a critical skill. To effectively display confidence, speakers must employ a blend of verbal and non-verbal signals. A predictive model that leverages rich multimodal cues to measure a speaker's confidence must also mitigate biases that develop through data labelling practices, inherent imbalances in the demographic distribution, or biases introduced into the model during the training process. Fairly predicting the confidence of speakers across differing backgrounds enables more accurate and actionable feedback to a larger population of speakers. This paper introduces a set of approaches for bias mitigation for multimodal, multi-class confidence prediction of adult speakers in a work-like setting. We evaluate the extent to which bias mitigation techniques improve the performance of a multimodal confidence classifier with a dataset of 233 2-minute videos. Experimental results suggest that by bounding the loss across perceived races, genders, accents, and ages, multimodal models can significantly outperform unmitigated baselines. The implications, including automated feedback of speaker confidence, are discussed. [For the complete proceedings, see ED675485.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED675542
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED675542
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 7
    Subjects:
      – SubjectFull: Self Esteem
        Type: general
      – SubjectFull: Prediction
        Type: general
      – SubjectFull: Cues
        Type: general
      – SubjectFull: Bias
        Type: general
      – SubjectFull: Adults
        Type: general
      – SubjectFull: Speech Communication
        Type: general
      – SubjectFull: Nonverbal Communication
        Type: general
    Titles:
      – TitleFull: Multimodal, Multi-Class Bias Mitigation for Predicting Speaker Confidence
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Andrew Emerson
      – PersonEntity:
          Name:
            NameFull: Arti Ramesh
      – PersonEntity:
          Name:
            NameFull: Patrick Houghton
      – PersonEntity:
          Name:
            NameFull: Vinay Basheerabad
      – PersonEntity:
          Name:
            NameFull: Navaneeth Jawahar
      – PersonEntity:
          Name:
            NameFull: Chee Wee Leong
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2024
          Titles:
            – TitleFull: International Educational Data Mining Society
              Type: main
ResultId 1