Federated Learning for Heterogeneous Multimodal Emotion Recognition on Edge Devices.
Saved in:
| Title: | Federated Learning for Heterogeneous Multimodal Emotion Recognition on Edge Devices. |
|---|---|
| Authors: | Sharma, Bhawana1, Saxena, Komal1 ksaxena1@amity.edu |
| Source: | International Journal of Performability Engineering. Mar2026, Vol. 22 Issue 3, p149-157. 9p. |
| Subjects: | Federated learning, Affective computing, Language models, Transformer models, Artificial intelligence, Data distribution, Computer peripherals |
| Abstract: | The rapid proliferation of artificial intelligence in mental health applications, particularly in digital journaling and emotion tracking, has accumulated significant privacy concerns regarding the centralization of sensitive user data. Although Federated Learning offers a decentralized alternative by training models locally on edge devices, existing frameworks predominantly rely on the assumption of Independent and Identically Distributed unimodal data. This approach fails to address the inherent diversity of real-world user interactions where client inputs vary dynamically between text-only entries and multimodal content. To bridge this gap, we present MobileFedFusion, a resource-efficient and privacy-preserving Federated Learning architecture designed for heterogeneous edge environments. We propose a novel Modality Masking mechanism that enables a unified global model to aggregate gradients from diverse clients, seamlessly integrating text-only and multimodal contributors without architectural fragmentation. The system leverages a lightweight fusion of MobileBERT and MobileViT that is specifically engineered to operate within the computational constraints of mobile hardware. Experimental validation on a non-IID partition of the Reddit GoEmotions and Memotion 3.0 datasets demonstrates that our approach achieves a Global Macro F1 score of 0.68. The results indicate that the model effectively converges to state-of-the-art performance while ensuring that sensitive personal data never leaves the local device. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Performability Engineering is the property of Totem Publisher, Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
Be the first to leave a comment!
