Federated Learning for Heterogeneous Multimodal Emotion Recognition on Edge Devices.
Saved in:
| Title: | Federated Learning for Heterogeneous Multimodal Emotion Recognition on Edge Devices. |
|---|---|
| Authors: | Sharma, Bhawana1, Saxena, Komal1 ksaxena1@amity.edu |
| Source: | International Journal of Performability Engineering. Mar2026, Vol. 22 Issue 3, p149-157. 9p. |
| Subjects: | Federated learning, Affective computing, Language models, Transformer models, Artificial intelligence, Data distribution, Computer peripherals |
| Abstract: | The rapid proliferation of artificial intelligence in mental health applications, particularly in digital journaling and emotion tracking, has accumulated significant privacy concerns regarding the centralization of sensitive user data. Although Federated Learning offers a decentralized alternative by training models locally on edge devices, existing frameworks predominantly rely on the assumption of Independent and Identically Distributed unimodal data. This approach fails to address the inherent diversity of real-world user interactions where client inputs vary dynamically between text-only entries and multimodal content. To bridge this gap, we present MobileFedFusion, a resource-efficient and privacy-preserving Federated Learning architecture designed for heterogeneous edge environments. We propose a novel Modality Masking mechanism that enables a unified global model to aggregate gradients from diverse clients, seamlessly integrating text-only and multimodal contributors without architectural fragmentation. The system leverages a lightweight fusion of MobileBERT and MobileViT that is specifically engineered to operate within the computational constraints of mobile hardware. Experimental validation on a non-IID partition of the Reddit GoEmotions and Memotion 3.0 datasets demonstrates that our approach achieves a Global Macro F1 score of 0.68. The results indicate that the model effectively converges to state-of-the-art performance while ensuring that sensitive personal data never leaves the local device. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Performability Engineering is the property of Totem Publisher, Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 192187930 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Federated Learning for Heterogeneous Multimodal Emotion Recognition on Edge Devices. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Sharma%2C+Bhawana%22">Sharma, Bhawana</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Saxena%2C+Komal%22">Saxena, Komal</searchLink><relatesTo>1</relatesTo><i> ksaxena1@amity.edu</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Performability+Engineering%22">International Journal of Performability Engineering</searchLink>. Mar2026, Vol. 22 Issue 3, p149-157. 9p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Federated+learning%22">Federated learning</searchLink><br /><searchLink fieldCode="DE" term="%22Affective+computing%22">Affective computing</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Transformer+models%22">Transformer models</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Data+distribution%22">Data distribution</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+peripherals%22">Computer peripherals</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: The rapid proliferation of artificial intelligence in mental health applications, particularly in digital journaling and emotion tracking, has accumulated significant privacy concerns regarding the centralization of sensitive user data. Although Federated Learning offers a decentralized alternative by training models locally on edge devices, existing frameworks predominantly rely on the assumption of Independent and Identically Distributed unimodal data. This approach fails to address the inherent diversity of real-world user interactions where client inputs vary dynamically between text-only entries and multimodal content. To bridge this gap, we present MobileFedFusion, a resource-efficient and privacy-preserving Federated Learning architecture designed for heterogeneous edge environments. We propose a novel Modality Masking mechanism that enables a unified global model to aggregate gradients from diverse clients, seamlessly integrating text-only and multimodal contributors without architectural fragmentation. The system leverages a lightweight fusion of MobileBERT and MobileViT that is specifically engineered to operate within the computational constraints of mobile hardware. Experimental validation on a non-IID partition of the Reddit GoEmotions and Memotion 3.0 datasets demonstrates that our approach achieves a Global Macro F1 score of 0.68. The results indicate that the model effectively converges to state-of-the-art performance while ensuring that sensitive personal data never leaves the local device. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of International Journal of Performability Engineering is the property of Totem Publisher, Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=192187930 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.23940/ijpe.26.03.p4.149157 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 9 StartPage: 149 Subjects: – SubjectFull: Federated learning Type: general – SubjectFull: Affective computing Type: general – SubjectFull: Language models Type: general – SubjectFull: Transformer models Type: general – SubjectFull: Artificial intelligence Type: general – SubjectFull: Data distribution Type: general – SubjectFull: Computer peripherals Type: general Titles: – TitleFull: Federated Learning for Heterogeneous Multimodal Emotion Recognition on Edge Devices. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Sharma, Bhawana – PersonEntity: Name: NameFull: Saxena, Komal IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 03 Text: Mar2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 09731318 Numbering: – Type: volume Value: 22 – Type: issue Value: 3 Titles: – TitleFull: International Journal of Performability Engineering Type: main |
| ResultId | 1 |
