A multi-view 3D perception network based on spatio-temporal fusion.

Saved in:

Bibliographic Details
Title:	A multi-view 3D perception network based on spatio-temporal fusion.
Authors:	LI, He^1,2, CHEN, Pintong¹, YU, Rong¹, TAN, Beihai³
Source:	Computer Engineering & Science / Jisuanji Gongcheng yu Kexue. Nov2025, Vol. 47 Issue 11, p2019-2028. 10p.
Subjects:	Autonomous vehicles, Temporal integration, Object recognition (Computer vision), Metadata, Depth perception, Safety, Computer performance
Abstract:	As a critical component of autonomous driving, the perception system directly influences a vehicle's comprehension of its surrounding environment and serves as the foundation for achieving safe and reliable autonomous driving. Traditional 2D image detection perception techniques can only provide limited information. While 3D perception offers richer perceptual data, it faces key challenges, including insufficient fusion of spatial information and inadequate utilization of temporal information. This paper proposes and designs a multi-view 3D perception network that integrates spatio-temporal information. This network comprises a multi-view surround 3D perception network and a spatio-temporal fusion network MVSPNet. The multi-view surround perception network efficiently fuses multi-camera image data through precise spatial perspective transformation, constructing a unified bird's eye view spatial representation. This achieves spatial alignment and fusion of data from multiple cameras. Compared to the current advanced monocular baseline model FCOS3D, it achieves a mean average precision (mAP) of 0.343, representing a performance improvement of 14.7%. The spatio-temporal fusion network MVSPNet enables the temporal fusion of multi-view images by integrating multi-frame data. This further significantly enhances the network's performance, and fusing 2 frames of temporal data results in an additional mAP improvement of 10.2%. The experiment results fully demonstrate the advancement of the designed network in effectively fusing multi-view spatial information and temporal information. This study provides an effective solution for enhancing 3D perception of autonomous driving systems in dynamic and complex scenarios, holding significant implications for advancing the development of safe and reliable autonomous driving technology. [ABSTRACT FROM AUTHOR]
	Copyright of Computer Engineering & Science / Jisuanji Gongcheng yu Kexue is the property of Computer Engineering & Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database:	Engineering Source

Description
Abstract:	As a critical component of autonomous driving, the perception system directly influences a vehicle's comprehension of its surrounding environment and serves as the foundation for achieving safe and reliable autonomous driving. Traditional 2D image detection perception techniques can only provide limited information. While 3D perception offers richer perceptual data, it faces key challenges, including insufficient fusion of spatial information and inadequate utilization of temporal information. This paper proposes and designs a multi-view 3D perception network that integrates spatio-temporal information. This network comprises a multi-view surround 3D perception network and a spatio-temporal fusion network MVSPNet. The multi-view surround perception network efficiently fuses multi-camera image data through precise spatial perspective transformation, constructing a unified bird's eye view spatial representation. This achieves spatial alignment and fusion of data from multiple cameras. Compared to the current advanced monocular baseline model FCOS3D, it achieves a mean average precision (mAP) of 0.343, representing a performance improvement of 14.7%. The spatio-temporal fusion network MVSPNet enables the temporal fusion of multi-view images by integrating multi-frame data. This further significantly enhances the network's performance, and fusing 2 frames of temporal data results in an additional mAP improvement of 10.2%. The experiment results fully demonstrate the advancement of the designed network in effectively fusing multi-view spatial information and temporal information. This study provides an effective solution for enhancing 3D perception of autonomous driving systems in dynamic and complex scenarios, holding significant implications for advancing the development of safe and reliable autonomous driving technology. [ABSTRACT FROM AUTHOR]
ISSN:	1007130X
DOI:	10.3969/j.issn.1007-130X.2025.11.012