PDSM: Progressive Dynamic Strategy with Mixture-of-Experts for Multimodal Video Summarization.
Saved in:
| Title: | PDSM: Progressive Dynamic Strategy with Mixture-of-Experts for Multimodal Video Summarization. |
|---|---|
| Authors: | Chen, Rongjun1 chenrongjun@gpnu.edu.cn, Liang, Yaoxin2 liangyaoxin@gpnu.edu.cn, Wang, Leijun3 wangleijun@gpnu.edu.cn, Hu, Xianglei3 huxianglei@gpnu.edu.cn, Yuan, Jun4 yuanjun@gpnu.edu.cn, Zeng, Xianxian4 zengxianxian@gpnu.edu.cn, Li, Jiawen4 lijiawen@gpnu.edu.cn |
| Source: | Engineering Letters. Apr2026, Vol. 34 Issue 4, p1311-1321. 11p. |
| Subjects: | Video summarization, Ensemble learning |
| Abstract: | Video summarization, which extracts representative keyframes or clips from long-form videos, plays a critical role in facilitating efficient browsing, accurate content retrieval, and enhanced comprehension. In practice, human perception often integrates video and subtitle cues to perform thematic and detail-oriented analysis and identify salient segments within a video. However, most existing approaches focus primarily on a single modality, such as visual features of video frames or semantic information from textual subtitles, thus neglecting the inherent synergistic potential of multimodal data. While some methods incorporate multiple modalities, they often fail to fully exploit the deeper semantic associations between video and text, particularly the interplay between global (summary-level descriptions) and local (detailed descriptions) features. Consequently, generated summaries suffer from reduced precision, potentially omitting critical events or including redundant information, which ultimately diminishes the overall effectiveness of the summarization process. To address these limitations, we propose a Progressive Dynamic Strategy with Mixture-of-Experts (PDSM) framework for multimodal video summarization. Specifically, PDSM employs a Mixture-of-Experts model and a cross-modal progressive alignment strategy to assess the complementary strengths of different information modalities and to highlight the most relevant segments for the desired summary. Extensive experiments demonstrate the superior performance of the proposed method against state-of-the-art benchmark approaches across commonly used evaluation metrics. This work thus opens a novel direction for multimodal video summarization. [ABSTRACT FROM AUTHOR] |
| Copyright of Engineering Letters is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 192720692 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: PDSM: Progressive Dynamic Strategy with Mixture-of-Experts for Multimodal Video Summarization. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Chen%2C+Rongjun%22">Chen, Rongjun</searchLink><relatesTo>1</relatesTo><i> chenrongjun@gpnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Liang%2C+Yaoxin%22">Liang, Yaoxin</searchLink><relatesTo>2</relatesTo><i> liangyaoxin@gpnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Wang%2C+Leijun%22">Wang, Leijun</searchLink><relatesTo>3</relatesTo><i> wangleijun@gpnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Hu%2C+Xianglei%22">Hu, Xianglei</searchLink><relatesTo>3</relatesTo><i> huxianglei@gpnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Yuan%2C+Jun%22">Yuan, Jun</searchLink><relatesTo>4</relatesTo><i> yuanjun@gpnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Zeng%2C+Xianxian%22">Zeng, Xianxian</searchLink><relatesTo>4</relatesTo><i> zengxianxian@gpnu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Li%2C+Jiawen%22">Li, Jiawen</searchLink><relatesTo>4</relatesTo><i> lijiawen@gpnu.edu.cn</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Engineering+Letters%22">Engineering Letters</searchLink>. Apr2026, Vol. 34 Issue 4, p1311-1321. 11p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Video+summarization%22">Video summarization</searchLink><br /><searchLink fieldCode="DE" term="%22Ensemble+learning%22">Ensemble learning</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Video summarization, which extracts representative keyframes or clips from long-form videos, plays a critical role in facilitating efficient browsing, accurate content retrieval, and enhanced comprehension. In practice, human perception often integrates video and subtitle cues to perform thematic and detail-oriented analysis and identify salient segments within a video. However, most existing approaches focus primarily on a single modality, such as visual features of video frames or semantic information from textual subtitles, thus neglecting the inherent synergistic potential of multimodal data. While some methods incorporate multiple modalities, they often fail to fully exploit the deeper semantic associations between video and text, particularly the interplay between global (summary-level descriptions) and local (detailed descriptions) features. Consequently, generated summaries suffer from reduced precision, potentially omitting critical events or including redundant information, which ultimately diminishes the overall effectiveness of the summarization process. To address these limitations, we propose a Progressive Dynamic Strategy with Mixture-of-Experts (PDSM) framework for multimodal video summarization. Specifically, PDSM employs a Mixture-of-Experts model and a cross-modal progressive alignment strategy to assess the complementary strengths of different information modalities and to highlight the most relevant segments for the desired summary. Extensive experiments demonstrate the superior performance of the proposed method against state-of-the-art benchmark approaches across commonly used evaluation metrics. This work thus opens a novel direction for multimodal video summarization. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Engineering Letters is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=192720692 |
| RecordInfo | BibRecord: BibEntity: Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 11 StartPage: 1311 Subjects: – SubjectFull: Video summarization Type: general – SubjectFull: Ensemble learning Type: general Titles: – TitleFull: PDSM: Progressive Dynamic Strategy with Mixture-of-Experts for Multimodal Video Summarization. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Chen, Rongjun – PersonEntity: Name: NameFull: Liang, Yaoxin – PersonEntity: Name: NameFull: Wang, Leijun – PersonEntity: Name: NameFull: Hu, Xianglei – PersonEntity: Name: NameFull: Yuan, Jun – PersonEntity: Name: NameFull: Zeng, Xianxian – PersonEntity: Name: NameFull: Li, Jiawen IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 04 Text: Apr2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 1816093X Numbering: – Type: volume Value: 34 – Type: issue Value: 4 Titles: – TitleFull: Engineering Letters Type: main |
| ResultId | 1 |
