View in EDS

DeST: A Decoupled Spatio-Temporal Framework for Action Segmentation.

Saved in:
Bibliographic Details
Title: DeST: A Decoupled Spatio-Temporal Framework for Action Segmentation.
Authors: Li, Yunheng1 (AUTHOR) yunhengli@mail.nankai.edu.cn, Li, Zhong-Yu1 (AUTHOR) lizhongyu@mail.nankai.edu.cn, Gao, Shanghua1 (AUTHOR), Wang, Qilong2 (AUTHOR), Hou, Qibin1,3 (AUTHOR) houqb@nankai.edu.cn, Cheng, Ming-Ming1,3 (AUTHOR) cmm@nankai.edu.cn
Source: International Journal of Computer Vision. May2026, Vol. 134 Issue 5, p1-21. 21p.
Subjects: Spatiotemporal processes, Movement sequences, Computer science, Software frameworks
Abstract: Effectively modeling discriminative spatio-temporal information is essential for segmenting activities in long action sequences. However, we observe that existing methods are limited in weak spatio-temporal modeling capability due to two forms of coupled modeling: (i) Cascaded interaction couples spatial and temporal modeling, which over-smooths motion modeling over the long sequence, and (ii) Joint-shared temporal modeling adopts shared weights to model each joint, ignoring the distinct motion patterns of different joints. In this paper, we present a Decoupled Spatio-Temporal Framework (DeST) to address the above issues. Firstly, we decouple the cascaded spatio-temporal interaction to avoid stacking multiple spatio-temporal blocks, while achieving sufficient spatio-temporal interaction. Specifically, DeST performs once unified spatial modeling and divides the spatial features into different groups of sub-features, which then adaptively interact with temporal features from different layers. Since the different sub-features contain distinct spatial semantics, the model could learn better interaction patterns at each layer. Meanwhile, inspired by the fact that different joints move at different speeds, we propose joint-decoupled temporal modeling, which employs independent trainable weights to capture distinctive temporal features of each joint. On four large-scale benchmarks of different scenes, DeST significantly outperforms current state-of-the-art methods with less computational complexity. Our code is available at: https://github.com/lyhisme/DeST. [ABSTRACT FROM AUTHOR]
Copyright of International Journal of Computer Vision is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Effectively modeling discriminative spatio-temporal information is essential for segmenting activities in long action sequences. However, we observe that existing methods are limited in weak spatio-temporal modeling capability due to two forms of coupled modeling: (i) Cascaded interaction couples spatial and temporal modeling, which over-smooths motion modeling over the long sequence, and (ii) Joint-shared temporal modeling adopts shared weights to model each joint, ignoring the distinct motion patterns of different joints. In this paper, we present a Decoupled Spatio-Temporal Framework (DeST) to address the above issues. Firstly, we decouple the cascaded spatio-temporal interaction to avoid stacking multiple spatio-temporal blocks, while achieving sufficient spatio-temporal interaction. Specifically, DeST performs once unified spatial modeling and divides the spatial features into different groups of sub-features, which then adaptively interact with temporal features from different layers. Since the different sub-features contain distinct spatial semantics, the model could learn better interaction patterns at each layer. Meanwhile, inspired by the fact that different joints move at different speeds, we propose joint-decoupled temporal modeling, which employs independent trainable weights to capture distinctive temporal features of each joint. On four large-scale benchmarks of different scenes, DeST significantly outperforms current state-of-the-art methods with less computational complexity. Our code is available at: https://github.com/lyhisme/DeST. [ABSTRACT FROM AUTHOR]
ISSN:09205691
DOI:10.1007/s11263-026-02797-0