Bridging local and global representations for self-supervised monocular depth estimation.
Saved in:
| Title: | Bridging local and global representations for self-supervised monocular depth estimation. |
|---|---|
| Authors: | Lin, Meiling1,2 (AUTHOR) linmeiling@ime.ac.cn, Li, Gongyan1 (AUTHOR), Hao, Yuexing1,2 (AUTHOR) |
| Source: | Engineering Applications of Artificial Intelligence. Jul2024:Part C, Vol. 133, pN.PAG-N.PAG. 1p. |
| Subjects: | Convolutional neural networks, Monoculars, Bridges, Feature extraction, Transformer models, Stereo image |
| Abstract: | Monocular depth estimation is a challenging problem, especially in a self-supervised manner without relying on the depth ground truth. The self-supervised approach places higher demands on the global and local feature extraction capabilities of the network. Based on this view, we propose to perform efficient hybrid feature extraction by exploiting the capacity of the Transformer and convolutional neural network to model long-range dependencies and local correlations at the same time. A bowknot-type fuser is designed to align features extracted from different sources as well as bridge global and local semantic representations. To obtain higher-quality pseudo-labels from the extracted features, we suggest the pseudo-label smoothing technique to fully utilize the multi-scale features, consequently boosting the effect of self-distillation loss as an auxiliary supervision to train the neural network. In addition, we propose pixel adaptive smoothness loss to refine the predicted depth map by introducing the image's textural and spatial information. The suggested method is trained on the KITTI benchmark using stereo image pairs and achieves competitive depth estimation performance in contrast with previous approaches. The code and models are available at https://github.com/MaylingLin/BLGR-Depth. • A self-supervised monocular depth estimation framework BLGR-Depth achieving results comparable to the current SOTA methods. • A bowknot-type fuser aligns semantic information between different features and capitalizes on the global and local features. • A selective post-processing method with the pseudo-label smoothing strategy to generate pseudo-labels according to channel-wise attention. • A pixel adaptive smoothing loss to refine the predicted depth map during training. [ABSTRACT FROM AUTHOR] |
| Copyright of Engineering Applications of Artificial Intelligence is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 177604642 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Bridging local and global representations for self-supervised monocular depth estimation. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Lin%2C+Meiling%22">Lin, Meiling</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> linmeiling@ime.ac.cn</i><br /><searchLink fieldCode="AR" term="%22Li%2C+Gongyan%22">Li, Gongyan</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Hao%2C+Yuexing%22">Hao, Yuexing</searchLink><relatesTo>1,2</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Engineering+Applications+of+Artificial+Intelligence%22">Engineering Applications of Artificial Intelligence</searchLink>. Jul2024:Part C, Vol. 133, pN.PAG-N.PAG. 1p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Convolutional+neural+networks%22">Convolutional neural networks</searchLink><br /><searchLink fieldCode="DE" term="%22Monoculars%22">Monoculars</searchLink><br /><searchLink fieldCode="DE" term="%22Bridges%22">Bridges</searchLink><br /><searchLink fieldCode="DE" term="%22Feature+extraction%22">Feature extraction</searchLink><br /><searchLink fieldCode="DE" term="%22Transformer+models%22">Transformer models</searchLink><br /><searchLink fieldCode="DE" term="%22Stereo+image%22">Stereo image</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Monocular depth estimation is a challenging problem, especially in a self-supervised manner without relying on the depth ground truth. The self-supervised approach places higher demands on the global and local feature extraction capabilities of the network. Based on this view, we propose to perform efficient hybrid feature extraction by exploiting the capacity of the Transformer and convolutional neural network to model long-range dependencies and local correlations at the same time. A bowknot-type fuser is designed to align features extracted from different sources as well as bridge global and local semantic representations. To obtain higher-quality pseudo-labels from the extracted features, we suggest the pseudo-label smoothing technique to fully utilize the multi-scale features, consequently boosting the effect of self-distillation loss as an auxiliary supervision to train the neural network. In addition, we propose pixel adaptive smoothness loss to refine the predicted depth map by introducing the image's textural and spatial information. The suggested method is trained on the KITTI benchmark using stereo image pairs and achieves competitive depth estimation performance in contrast with previous approaches. The code and models are available at https://github.com/MaylingLin/BLGR-Depth. • A self-supervised monocular depth estimation framework BLGR-Depth achieving results comparable to the current SOTA methods. • A bowknot-type fuser aligns semantic information between different features and capitalizes on the global and local features. • A selective post-processing method with the pseudo-label smoothing strategy to generate pseudo-labels according to channel-wise attention. • A pixel adaptive smoothing loss to refine the predicted depth map during training. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Engineering Applications of Artificial Intelligence is the property of Pergamon Press - An Imprint of Elsevier Science and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=177604642 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.engappai.2024.108277 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 1 StartPage: N.PAG Subjects: – SubjectFull: Convolutional neural networks Type: general – SubjectFull: Monoculars Type: general – SubjectFull: Bridges Type: general – SubjectFull: Feature extraction Type: general – SubjectFull: Transformer models Type: general – SubjectFull: Stereo image Type: general Titles: – TitleFull: Bridging local and global representations for self-supervised monocular depth estimation. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Lin, Meiling – PersonEntity: Name: NameFull: Li, Gongyan – PersonEntity: Name: NameFull: Hao, Yuexing IsPartOfRelationships: – BibEntity: Dates: – D: 15 M: 07 Text: Jul2024:Part C Type: published Y: 2024 Identifiers: – Type: issn-print Value: 09521976 Numbering: – Type: volume Value: 133 Titles: – TitleFull: Engineering Applications of Artificial Intelligence Type: main |
| ResultId | 1 |
