Multimodal, Multi-Class Bias Mitigation for Predicting Speaker Confidence
Saved in:
| Title: | Multimodal, Multi-Class Bias Mitigation for Predicting Speaker Confidence |
|---|---|
| Language: | English |
| Authors: | Andrew Emerson, Arti Ramesh, Patrick Houghton, Vinay Basheerabad, Navaneeth Jawahar, Chee Wee Leong |
| Source: | International Educational Data Mining Society. 2024. |
| Availability: | International Educational Data Mining Society. e-mail: admin@educationaldatamining.org; Web site: https://educationaldatamining.org/conferences/ |
| Peer Reviewed: | Y |
| Page Count: | 7 |
| Publication Date: | 2024 |
| Document Type: | Speeches/Meeting Papers Reports - Research |
| Descriptors: | Self Esteem, Prediction, Cues, Bias, Adults, Speech Communication, Nonverbal Communication |
| Abstract: | Projecting confidence during conversation or presentation is a critical skill. To effectively display confidence, speakers must employ a blend of verbal and non-verbal signals. A predictive model that leverages rich multimodal cues to measure a speaker's confidence must also mitigate biases that develop through data labelling practices, inherent imbalances in the demographic distribution, or biases introduced into the model during the training process. Fairly predicting the confidence of speakers across differing backgrounds enables more accurate and actionable feedback to a larger population of speakers. This paper introduces a set of approaches for bias mitigation for multimodal, multi-class confidence prediction of adult speakers in a work-like setting. We evaluate the extent to which bias mitigation techniques improve the performance of a multimodal confidence classifier with a dataset of 233 2-minute videos. Experimental results suggest that by bounding the loss across perceived races, genders, accents, and ages, multimodal models can significantly outperform unmitigated baselines. The implications, including automated feedback of speaker confidence, are discussed. [For the complete proceedings, see ED675485.] |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | ED675542 |
| Database: | ERIC |
| Abstract: | Projecting confidence during conversation or presentation is a critical skill. To effectively display confidence, speakers must employ a blend of verbal and non-verbal signals. A predictive model that leverages rich multimodal cues to measure a speaker's confidence must also mitigate biases that develop through data labelling practices, inherent imbalances in the demographic distribution, or biases introduced into the model during the training process. Fairly predicting the confidence of speakers across differing backgrounds enables more accurate and actionable feedback to a larger population of speakers. This paper introduces a set of approaches for bias mitigation for multimodal, multi-class confidence prediction of adult speakers in a work-like setting. We evaluate the extent to which bias mitigation techniques improve the performance of a multimodal confidence classifier with a dataset of 233 2-minute videos. Experimental results suggest that by bounding the loss across perceived races, genders, accents, and ages, multimodal models can significantly outperform unmitigated baselines. The implications, including automated feedback of speaker confidence, are discussed. [For the complete proceedings, see ED675485.] |
|---|
