SDFA-Net: Urban Manhole Cover Detection via Fisheye Image and LiDAR Point Cloud Fusion.
Saved in:
| Title: | SDFA-Net: Urban Manhole Cover Detection via Fisheye Image and LiDAR Point Cloud Fusion. |
|---|---|
| Authors: | Lan, Qiuping1 (AUTHOR), Hu, Shuwen1,2 (AUTHOR), Li, Jia1,3 (AUTHOR), Huang, Yijie1,2,4 (AUTHOR), Li, Zikuan1,3,4 (AUTHOR), Fan, Qiang2,3,4 (AUTHOR) fanqiang09@nudt.edu.cn |
| Source: | Remote Sensing. Sep2026, Vol. 18 Issue 18, p3206. 30p. |
| Subjects: | Point cloud, Multisensor data fusion |
| Abstract: | Highlights: What are the main findings? SDFA-Net combines a semi-dense depth-supervised view transformation module (SDS-VT) with a fidelity-aware attention fusion module (FA-Fusion) to resolve spatial misalignment between fisheye cameras and LiDAR, achieving accurate and lightweight cross-modal fusion for manhole cover detection under wide-angle distortion. An anchor-free, geometry-aware detection head (Geo-Head) jointly predicts 2D localization and cover-to-road elevation difference in a single forward pass, extending conventional detection to quantitative, millimeter-level 3D defect diagnosis across five condition categories. What are the implications of the main findings? The proposed framework achieves robust, spatially aligned, and fidelity-adaptive fusion between fisheye-camera imagery and LiDAR point clouds under wide-angle distortion and cross-modal sparsity, providing a reliable geometric foundation for wearable multimodal perception in urban environments. This work provides an extensible paradigm for wearable mobile mapping-based infrastructure inspection, demonstrating the potential of image and LiDAR fusion methods in automated municipal asset monitoring, road-network-scale defect surveying, and intelligent urban infrastructure management. Automated manhole cover detection, a key task in urban infrastructure inspection, is hindered by modality-specific limitations: conventional monocular image-based detectors do not directly quantify cover-to-road elevation differences, LiDAR-based methods suffer from sparse sampling that limits recall for small, distant targets, and existing fusion approaches suffer from cross-modal spatial misalignment and lack 3D defect quantification. This paper proposes SDFA-Net, an end-to-end fisheye image and LiDAR point cloud fusion framework for manhole cover detection and diagnosis in wearable mobile mapping scenarios. The semi-dense depth-supervised view transformation (SDS-VT) module constructs semi-dense depth ground truth from multi-frame accumulated point clouds to explicitly supervise image-to-bird's-eye-view (BEV) feature projection, addressing cross-modal spatial registration under wide-angle fisheye distortion. The fidelity-adaptive fusion (FA-Fusion) module builds a physical fidelity map by combining point-cloud projection density with depth-prediction entropy, then performs quality-aware adaptive fusion via cascaded channel-spatial attention. The geometry-aware decoupled detection head (Geo-Head) adopts an anchor-free decoupled architecture to jointly predict 2D localization, physical dimensions, and millimeter-level relative elevation in a single forward pass. A multimodal dataset of 3500 spatiotemporally aligned frames is constructed using the NavVis VLX wearable platform, covering five defect categories, with Random Sample Consensus (RANSAC)-derived relative-elevation reference labels independently validated by digital leveling. Experiments show that SDFA-Net achieves 92.9% mAP@0.5 and 63.5% mAP@0.5:0.95 with 18.5 M parameters, improving over the best single-modal baseline by 4.6, 11.0, and 8.7 percentage points in mAP@0.5, mAP@0.5:0.95, and Recall, respectively. Compared with BEVFusion, SDFA-Net achieves higher detection accuracy while maintaining a smaller overall model size and an inference speed of 35 FPS. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? SDFA-Net combines a semi-dense depth-supervised view transformation module (SDS-VT) with a fidelity-aware attention fusion module (FA-Fusion) to resolve spatial misalignment between fisheye cameras and LiDAR, achieving accurate and lightweight cross-modal fusion for manhole cover detection under wide-angle distortion. An anchor-free, geometry-aware detection head (Geo-Head) jointly predicts 2D localization and cover-to-road elevation difference in a single forward pass, extending conventional detection to quantitative, millimeter-level 3D defect diagnosis across five condition categories. What are the implications of the main findings? The proposed framework achieves robust, spatially aligned, and fidelity-adaptive fusion between fisheye-camera imagery and LiDAR point clouds under wide-angle distortion and cross-modal sparsity, providing a reliable geometric foundation for wearable multimodal perception in urban environments. This work provides an extensible paradigm for wearable mobile mapping-based infrastructure inspection, demonstrating the potential of image and LiDAR fusion methods in automated municipal asset monitoring, road-network-scale defect surveying, and intelligent urban infrastructure management. Automated manhole cover detection, a key task in urban infrastructure inspection, is hindered by modality-specific limitations: conventional monocular image-based detectors do not directly quantify cover-to-road elevation differences, LiDAR-based methods suffer from sparse sampling that limits recall for small, distant targets, and existing fusion approaches suffer from cross-modal spatial misalignment and lack 3D defect quantification. This paper proposes SDFA-Net, an end-to-end fisheye image and LiDAR point cloud fusion framework for manhole cover detection and diagnosis in wearable mobile mapping scenarios. The semi-dense depth-supervised view transformation (SDS-VT) module constructs semi-dense depth ground truth from multi-frame accumulated point clouds to explicitly supervise image-to-bird's-eye-view (BEV) feature projection, addressing cross-modal spatial registration under wide-angle fisheye distortion. The fidelity-adaptive fusion (FA-Fusion) module builds a physical fidelity map by combining point-cloud projection density with depth-prediction entropy, then performs quality-aware adaptive fusion via cascaded channel-spatial attention. The geometry-aware decoupled detection head (Geo-Head) adopts an anchor-free decoupled architecture to jointly predict 2D localization, physical dimensions, and millimeter-level relative elevation in a single forward pass. A multimodal dataset of 3500 spatiotemporally aligned frames is constructed using the NavVis VLX wearable platform, covering five defect categories, with Random Sample Consensus (RANSAC)-derived relative-elevation reference labels independently validated by digital leveling. Experiments show that SDFA-Net achieves 92.9% mAP@0.5 and 63.5% mAP@0.5:0.95 with 18.5 M parameters, improving over the best single-modal baseline by 4.6, 11.0, and 8.7 percentage points in mAP@0.5, mAP@0.5:0.95, and Recall, respectively. Compared with BEVFusion, SDFA-Net achieves higher detection accuracy while maintaining a smaller overall model size and an inference speed of 35 FPS. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18183206 |
