A Hierarchical Cross-Modal Interaction and Dynamic Gating Fusion Framework for Enhanced Multimodal Sentiment Analysis.
Saved in:
| Title: | A Hierarchical Cross-Modal Interaction and Dynamic Gating Fusion Framework for Enhanced Multimodal Sentiment Analysis. |
|---|---|
| Authors: | Zheng, Wei1 weizheng@djtu.edu.cn, Gao, Yibo1 2524845783@qq.com, Li, Jing1 41821456@qq.com |
| Source: | IAENG International Journal of Computer Science. May2026, Vol. 53 Issue 5, p1740-1749. 10p. |
| Subjects: | Affective computing, Attention, Multimodal user interfaces, Machine learning, Contrastive learning |
| Abstract: | Image-text-based sentiment analysis leverages the complementarity of image and text information to improve multimodal sentiment classification accuracy, thus becoming a key research topic in affective computing. However, limited extraction of inter-modal interactive semantics and insufficient context-adaptability in selecting key sentiment cues during high-level fusion leave substantial room for improvement. To address these issues, this study proposes a hierarchical cross-modal interaction and dynamic gating fusion framework. Firstly, to enable deep interactions between text and image modalities, a hierarchical interaction mechanism combining intra-modal self-attention and cross-modal cross-attention is introduced, strengthening inter-modal semantic correlations and complementarity. Secondly, a learnable gating mechanism assigns element-wise weights to high-level semantics from different modalities, enhancing salient information and suppressing noise for context-adaptive fusion. Additionally, the model integrates supervised contrastive learning to constrain the fused feature space and sharpen decision boundaries. Finally, a multi-task joint training strategy with auxiliary classifiers improves uni-modal and fused representations, boosting sentiment classification performance. Experiments on three public multimodal sentiment datasets show that the proposed model achieves superior results, demonstrating strong generalization and broad applicability. [ABSTRACT FROM AUTHOR] |
| Copyright of IAENG International Journal of Computer Science is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 193482030 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: A Hierarchical Cross-Modal Interaction and Dynamic Gating Fusion Framework for Enhanced Multimodal Sentiment Analysis. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Zheng%2C+Wei%22">Zheng, Wei</searchLink><relatesTo>1</relatesTo><i> weizheng@djtu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Gao%2C+Yibo%22">Gao, Yibo</searchLink><relatesTo>1</relatesTo><i> 2524845783@qq.com</i><br /><searchLink fieldCode="AR" term="%22Li%2C+Jing%22">Li, Jing</searchLink><relatesTo>1</relatesTo><i> 41821456@qq.com</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22IAENG+International+Journal+of+Computer+Science%22">IAENG International Journal of Computer Science</searchLink>. May2026, Vol. 53 Issue 5, p1740-1749. 10p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Affective+computing%22">Affective computing</searchLink><br /><searchLink fieldCode="DE" term="%22Attention%22">Attention</searchLink><br /><searchLink fieldCode="DE" term="%22Multimodal+user+interfaces%22">Multimodal user interfaces</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Contrastive+learning%22">Contrastive learning</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Image-text-based sentiment analysis leverages the complementarity of image and text information to improve multimodal sentiment classification accuracy, thus becoming a key research topic in affective computing. However, limited extraction of inter-modal interactive semantics and insufficient context-adaptability in selecting key sentiment cues during high-level fusion leave substantial room for improvement. To address these issues, this study proposes a hierarchical cross-modal interaction and dynamic gating fusion framework. Firstly, to enable deep interactions between text and image modalities, a hierarchical interaction mechanism combining intra-modal self-attention and cross-modal cross-attention is introduced, strengthening inter-modal semantic correlations and complementarity. Secondly, a learnable gating mechanism assigns element-wise weights to high-level semantics from different modalities, enhancing salient information and suppressing noise for context-adaptive fusion. Additionally, the model integrates supervised contrastive learning to constrain the fused feature space and sharpen decision boundaries. Finally, a multi-task joint training strategy with auxiliary classifiers improves uni-modal and fused representations, boosting sentiment classification performance. Experiments on three public multimodal sentiment datasets show that the proposed model achieves superior results, demonstrating strong generalization and broad applicability. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of IAENG International Journal of Computer Science is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=193482030 |
| RecordInfo | BibRecord: BibEntity: Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 10 StartPage: 1740 Subjects: – SubjectFull: Affective computing Type: general – SubjectFull: Attention Type: general – SubjectFull: Multimodal user interfaces Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Contrastive learning Type: general Titles: – TitleFull: A Hierarchical Cross-Modal Interaction and Dynamic Gating Fusion Framework for Enhanced Multimodal Sentiment Analysis. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Zheng, Wei – PersonEntity: Name: NameFull: Gao, Yibo – PersonEntity: Name: NameFull: Li, Jing IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 1819656X Numbering: – Type: volume Value: 53 – Type: issue Value: 5 Titles: – TitleFull: IAENG International Journal of Computer Science Type: main |
| ResultId | 1 |
