View in EDS

Bridging local and global representations for self-supervised monocular depth estimation.

Saved in:
Bibliographic Details
Title: Bridging local and global representations for self-supervised monocular depth estimation.
Authors: Lin, Meiling1,2 (AUTHOR) linmeiling@ime.ac.cn, Li, Gongyan1 (AUTHOR), Hao, Yuexing1,2 (AUTHOR)
Source: Engineering Applications of Artificial Intelligence. Jul2024:Part C, Vol. 133, pN.PAG-N.PAG. 1p.
Subjects: Convolutional neural networks, Monoculars, Bridges, Feature extraction, Transformer models, Stereo image
Abstract: Monocular depth estimation is a challenging problem, especially in a self-supervised manner without relying on the depth ground truth. The self-supervised approach places higher demands on the global and local feature extraction capabilities of the network. Based on this view, we propose to perform efficient hybrid feature extraction by exploiting the capacity of the Transformer and convolutional neural network to model long-range dependencies and local correlations at the same time. A bowknot-type fuser is designed to align features extracted from different sources as well as bridge global and local semantic representations. To obtain higher-quality pseudo-labels from the extracted features, we suggest the pseudo-label smoothing technique to fully utilize the multi-scale features, consequently boosting the effect of self-distillation loss as an auxiliary supervision to train the neural network. In addition, we propose pixel adaptive smoothness loss to refine the predicted depth map by introducing the image's textural and spatial information. The suggested method is trained on the KITTI benchmark using stereo image pairs and achieves competitive depth estimation performance in contrast with previous approaches. The code and models are available at https://github.com/MaylingLin/BLGR-Depth. • A self-supervised monocular depth estimation framework BLGR-Depth achieving results comparable to the current SOTA methods. • A bowknot-type fuser aligns semantic information between different features and capitalizes on the global and local features. • A selective post-processing method with the pseudo-label smoothing strategy to generate pseudo-labels according to channel-wise attention. • A pixel adaptive smoothing loss to refine the predicted depth map during training. [ABSTRACT FROM AUTHOR]
Copyright of Engineering Applications of Artificial Intelligence is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Monocular depth estimation is a challenging problem, especially in a self-supervised manner without relying on the depth ground truth. The self-supervised approach places higher demands on the global and local feature extraction capabilities of the network. Based on this view, we propose to perform efficient hybrid feature extraction by exploiting the capacity of the Transformer and convolutional neural network to model long-range dependencies and local correlations at the same time. A bowknot-type fuser is designed to align features extracted from different sources as well as bridge global and local semantic representations. To obtain higher-quality pseudo-labels from the extracted features, we suggest the pseudo-label smoothing technique to fully utilize the multi-scale features, consequently boosting the effect of self-distillation loss as an auxiliary supervision to train the neural network. In addition, we propose pixel adaptive smoothness loss to refine the predicted depth map by introducing the image's textural and spatial information. The suggested method is trained on the KITTI benchmark using stereo image pairs and achieves competitive depth estimation performance in contrast with previous approaches. The code and models are available at https://github.com/MaylingLin/BLGR-Depth. • A self-supervised monocular depth estimation framework BLGR-Depth achieving results comparable to the current SOTA methods. • A bowknot-type fuser aligns semantic information between different features and capitalizes on the global and local features. • A selective post-processing method with the pseudo-label smoothing strategy to generate pseudo-labels according to channel-wise attention. • A pixel adaptive smoothing loss to refine the predicted depth map during training. [ABSTRACT FROM AUTHOR]
ISSN:09521976
DOI:10.1016/j.engappai.2024.108277