V3Reg: Model Integrating Visual Information for Extreme Low Overlap Point Cloud Registration.
Saved in:
| Title: | V3Reg: Model Integrating Visual Information for Extreme Low Overlap Point Cloud Registration. |
|---|---|
| Authors: | Li, Yaxiong1 (AUTHOR), Hou, Yifan1 (AUTHOR), Yang, Qisong1 (AUTHOR), Guan, Dongdong1 (AUTHOR) gdd@whu.edu.cn |
| Source: | Remote Sensing. Jun2026, Vol. 18 Issue 12, p2050. 17p. |
| Subjects: | Multisensor data fusion, Point cloud, Image fusion, Depth maps (Digital image processing) |
| Abstract: | Highlights: What are the main findings? We propose V3Reg, a visual–geometric point cloud registration framework that maps DINOv3 patch-level image representations onto 3D points to strengthen matching when geometric overlap is extremely limited. A task-aware channel-wise gated fusion module is developed to dynamically balance visual and geometric cues; on RGBD-ZeroMatch, V3Reg maintains 50.2% registration recall at only 5% overlap. What are the implications of the main findings? The proposed framework shows that foundation-model visual features can provide reliable complementary constraints when pure geometric correspondences are too sparse or ambiguous. The adaptive fusion strategy and RGBD-ZeroMatch benchmark support more robust evaluation and development of RGB-D registration methods for real-world low-overlap scenarios. Extremely low overlap leads to severely scarce local geometric correspondences across frame pairs. Pure geometric descriptors—encoding merely low-level shape signatures—inherently fail to impose sufficient constraints for reliable transformation estimation when matches become critically sparse, rendering registration fundamentally fragile. While recent red-green-blue-depth (RGB-D) attempts have explored visual augmentation, they predominantly rely on low-level chromatic statistics or shallow convolutional neural network (CNN) features, underutilizing the rich hierarchical semantics inherent in RGB imagery. We present V3Reg, a robust registration framework that pioneers the integration of large-scale vision foundation models (DINOv3) with adaptive cross-modal fusion. Specifically, we extract mid-to-deep semantic features (Layer 11) from DINOv3 to transcend low-level texture limitations, and propose a Task-Aware Channel-Wise Gated Adaptive Fusion (TACGAF) module that dynamically calibrates geometric-visual contributions via registration-error-guided channel-wise gating. To rigorously evaluate ultra-low-overlap robustness, we reconstruct RGBD-ZeroMatch, a benchmark with controllable overlap ratios ranging from 1% to 20%. Extensive experiments demonstrate that V3Reg achieves 99.6% Feature Matching Recall and 96.3% Registration Recall on standard benchmarks. Notably, it maintains 50.2% Registration Recall at merely 5% overlap, outperforming prior methods by over 18 percentage points. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? We propose V3Reg, a visual–geometric point cloud registration framework that maps DINOv3 patch-level image representations onto 3D points to strengthen matching when geometric overlap is extremely limited. A task-aware channel-wise gated fusion module is developed to dynamically balance visual and geometric cues; on RGBD-ZeroMatch, V3Reg maintains 50.2% registration recall at only 5% overlap. What are the implications of the main findings? The proposed framework shows that foundation-model visual features can provide reliable complementary constraints when pure geometric correspondences are too sparse or ambiguous. The adaptive fusion strategy and RGBD-ZeroMatch benchmark support more robust evaluation and development of RGB-D registration methods for real-world low-overlap scenarios. Extremely low overlap leads to severely scarce local geometric correspondences across frame pairs. Pure geometric descriptors—encoding merely low-level shape signatures—inherently fail to impose sufficient constraints for reliable transformation estimation when matches become critically sparse, rendering registration fundamentally fragile. While recent red-green-blue-depth (RGB-D) attempts have explored visual augmentation, they predominantly rely on low-level chromatic statistics or shallow convolutional neural network (CNN) features, underutilizing the rich hierarchical semantics inherent in RGB imagery. We present V3Reg, a robust registration framework that pioneers the integration of large-scale vision foundation models (DINOv3) with adaptive cross-modal fusion. Specifically, we extract mid-to-deep semantic features (Layer 11) from DINOv3 to transcend low-level texture limitations, and propose a Task-Aware Channel-Wise Gated Adaptive Fusion (TACGAF) module that dynamically calibrates geometric-visual contributions via registration-error-guided channel-wise gating. To rigorously evaluate ultra-low-overlap robustness, we reconstruct RGBD-ZeroMatch, a benchmark with controllable overlap ratios ranging from 1% to 20%. Extensive experiments demonstrate that V3Reg achieves 99.6% Feature Matching Recall and 96.3% Registration Recall on standard benchmarks. Notably, it maintains 50.2% Registration Recall at merely 5% overlap, outperforming prior methods by over 18 percentage points. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18122050 |
