A Hierarchical Cross-Modal Interaction and Dynamic Gating Fusion Framework for Enhanced Multimodal Sentiment Analysis.

Saved in:

Bibliographic Details
Title:	A Hierarchical Cross-Modal Interaction and Dynamic Gating Fusion Framework for Enhanced Multimodal Sentiment Analysis.
Authors:	Zheng, Wei¹ weizheng@djtu.edu.cn, Gao, Yibo¹ 2524845783@qq.com, Li, Jing¹ 41821456@qq.com
Source:	IAENG International Journal of Computer Science. May2026, Vol. 53 Issue 5, p1740-1749. 10p.
Subjects:	Affective computing, Attention, Multimodal user interfaces, Machine learning, Contrastive learning
Abstract:	Image-text-based sentiment analysis leverages the complementarity of image and text information to improve multimodal sentiment classification accuracy, thus becoming a key research topic in affective computing. However, limited extraction of inter-modal interactive semantics and insufficient context-adaptability in selecting key sentiment cues during high-level fusion leave substantial room for improvement. To address these issues, this study proposes a hierarchical cross-modal interaction and dynamic gating fusion framework. Firstly, to enable deep interactions between text and image modalities, a hierarchical interaction mechanism combining intra-modal self-attention and cross-modal cross-attention is introduced, strengthening inter-modal semantic correlations and complementarity. Secondly, a learnable gating mechanism assigns element-wise weights to high-level semantics from different modalities, enhancing salient information and suppressing noise for context-adaptive fusion. Additionally, the model integrates supervised contrastive learning to constrain the fused feature space and sharpen decision boundaries. Finally, a multi-task joint training strategy with auxiliary classifiers improves uni-modal and fused representations, boosting sentiment classification performance. Experiments on three public multimodal sentiment datasets show that the proposed model achieves superior results, demonstrating strong generalization and broad applicability. [ABSTRACT FROM AUTHOR]
	Copyright of IAENG International Journal of Computer Science is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	Image-text-based sentiment analysis leverages the complementarity of image and text information to improve multimodal sentiment classification accuracy, thus becoming a key research topic in affective computing. However, limited extraction of inter-modal interactive semantics and insufficient context-adaptability in selecting key sentiment cues during high-level fusion leave substantial room for improvement. To address these issues, this study proposes a hierarchical cross-modal interaction and dynamic gating fusion framework. Firstly, to enable deep interactions between text and image modalities, a hierarchical interaction mechanism combining intra-modal self-attention and cross-modal cross-attention is introduced, strengthening inter-modal semantic correlations and complementarity. Secondly, a learnable gating mechanism assigns element-wise weights to high-level semantics from different modalities, enhancing salient information and suppressing noise for context-adaptive fusion. Additionally, the model integrates supervised contrastive learning to constrain the fused feature space and sharpen decision boundaries. Finally, a multi-task joint training strategy with auxiliary classifiers improves uni-modal and fused representations, boosting sentiment classification performance. Experiments on three public multimodal sentiment datasets show that the proposed model achieves superior results, demonstrating strong generalization and broad applicability. [ABSTRACT FROM AUTHOR]
ISSN:	1819656X