PDSM: Progressive Dynamic Strategy with Mixture-of-Experts for Multimodal Video Summarization.

Saved in:

Bibliographic Details
Title:	PDSM: Progressive Dynamic Strategy with Mixture-of-Experts for Multimodal Video Summarization.
Authors:	Chen, Rongjun¹ chenrongjun@gpnu.edu.cn, Liang, Yaoxin² liangyaoxin@gpnu.edu.cn, Wang, Leijun³ wangleijun@gpnu.edu.cn, Hu, Xianglei³ huxianglei@gpnu.edu.cn, Yuan, Jun⁴ yuanjun@gpnu.edu.cn, Zeng, Xianxian⁴ zengxianxian@gpnu.edu.cn, Li, Jiawen⁴ lijiawen@gpnu.edu.cn
Source:	Engineering Letters. Apr2026, Vol. 34 Issue 4, p1311-1321. 11p.
Subjects:	Video summarization, Ensemble learning
Abstract:	Video summarization, which extracts representative keyframes or clips from long-form videos, plays a critical role in facilitating efficient browsing, accurate content retrieval, and enhanced comprehension. In practice, human perception often integrates video and subtitle cues to perform thematic and detail-oriented analysis and identify salient segments within a video. However, most existing approaches focus primarily on a single modality, such as visual features of video frames or semantic information from textual subtitles, thus neglecting the inherent synergistic potential of multimodal data. While some methods incorporate multiple modalities, they often fail to fully exploit the deeper semantic associations between video and text, particularly the interplay between global (summary-level descriptions) and local (detailed descriptions) features. Consequently, generated summaries suffer from reduced precision, potentially omitting critical events or including redundant information, which ultimately diminishes the overall effectiveness of the summarization process. To address these limitations, we propose a Progressive Dynamic Strategy with Mixture-of-Experts (PDSM) framework for multimodal video summarization. Specifically, PDSM employs a Mixture-of-Experts model and a cross-modal progressive alignment strategy to assess the complementary strengths of different information modalities and to highlight the most relevant segments for the desired summary. Extensive experiments demonstrate the superior performance of the proposed method against state-of-the-art benchmark approaches across commonly used evaluation metrics. This work thus opens a novel direction for multimodal video summarization. [ABSTRACT FROM AUTHOR]
	Copyright of Engineering Letters is the property of International Association of Engineers (IAENG) and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	Video summarization, which extracts representative keyframes or clips from long-form videos, plays a critical role in facilitating efficient browsing, accurate content retrieval, and enhanced comprehension. In practice, human perception often integrates video and subtitle cues to perform thematic and detail-oriented analysis and identify salient segments within a video. However, most existing approaches focus primarily on a single modality, such as visual features of video frames or semantic information from textual subtitles, thus neglecting the inherent synergistic potential of multimodal data. While some methods incorporate multiple modalities, they often fail to fully exploit the deeper semantic associations between video and text, particularly the interplay between global (summary-level descriptions) and local (detailed descriptions) features. Consequently, generated summaries suffer from reduced precision, potentially omitting critical events or including redundant information, which ultimately diminishes the overall effectiveness of the summarization process. To address these limitations, we propose a Progressive Dynamic Strategy with Mixture-of-Experts (PDSM) framework for multimodal video summarization. Specifically, PDSM employs a Mixture-of-Experts model and a cross-modal progressive alignment strategy to assess the complementary strengths of different information modalities and to highlight the most relevant segments for the desired summary. Extensive experiments demonstrate the superior performance of the proposed method against state-of-the-art benchmark approaches across commonly used evaluation metrics. This work thus opens a novel direction for multimodal video summarization. [ABSTRACT FROM AUTHOR]
ISSN:	1816093X