Consistency-Guided Distillation from Vision Foundation Models for Zero-Shot Airborne Point Cloud Segmentation.
Saved in:
| Title: | Consistency-Guided Distillation from Vision Foundation Models for Zero-Shot Airborne Point Cloud Segmentation. |
|---|---|
| Authors: | Gao, Yuan1,2,3 (AUTHOR), Zhao, Jindong2,4 (AUTHOR), Xia, Shaobo3,4 (AUTHOR), Nie, Sheng1,2,3,4 (AUTHOR), Wang, Cheng1,2,3 (AUTHOR), Xi, Xiaohuan1,2,3 (AUTHOR) xixh@aircas.ac.cn |
| Source: | Remote Sensing. Jun2026, Vol. 18 Issue 12, p1875. 20p. |
| Subjects: | Semantics, Software frameworks |
| Abstract: | Highlights: What are the main findings? The proposed dual-branch framework effectively identifies and filters out "semantic bleeding" hallucinations by cross-validating 2D open-vocabulary semantics with 3D elevation-constrained geometric instances. Our method successfully rescues submerged small-scale targets (e.g., vehicles and street elements), achieving substantial mIoU improvements of up to 29 absolute percentage points over existing zero-shot baselines on the H3D and Turin3D datasets. What is the implication of the main finding? The research establishes a highly efficient, label-free paradigm for large-scale airborne point cloud interpretation, eliminating the reliance on costly and labor-intensive 3D manual annotations. It demonstrates that integrating authentic 3D geometric topological priors is crucial for correcting cross-modal semantic noise and unleashing the true potential of 2D Vision Foundation Models in 3D geospatial applications. Semantic segmentation of large-scale airborne point clouds traditionally relies on labor-intensive 3D manual annotations. While recent zero-shot methods attempt to alleviate this burden by distilling knowledge from 2D Vision–Language Models (VLMs) via 2D-to-3D projection, they suffer from performance degradation in complex urban environments. Specifically, lacking 3D geometric awareness, 2D VLMs frequently exhibit "semantic bleeding", where large-scale background categories (e.g., ground) erroneously submerge small-scale targets (e.g., vehicles and street elements). To address this issue, we propose a geometry-constrained pseudo-label generation and purification framework. Our approach tackles the problem through a dual-branch design: extracting open-vocabulary semantics via SAM3-based multi-view projection while simultaneously deriving sharp, class-agnostic instances using SAM2 on Gamma-transformed elevation maps. By introducing a geometric–semantic consistency module, we evaluate the internal semantic purity and external spatial homogeneity of these instances, detecting and filtering out semantic misclassifications. The purified pseudo-labels are then used to supervise a 3D sparse convolutional network via a Masked Cross-Entropy Loss. Experiments on the H3D and Turin3D datasets demonstrate that our method recovers small-scale targets that are prone to being submerged, outperforming existing zero-shot baselines by improving mIoU from 52.15% to 63.45% on H3D and from 29.52% to 58.51% on Turin3D, thereby narrowing the performance gap with fully-supervised approaches. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? The proposed dual-branch framework effectively identifies and filters out "semantic bleeding" hallucinations by cross-validating 2D open-vocabulary semantics with 3D elevation-constrained geometric instances. Our method successfully rescues submerged small-scale targets (e.g., vehicles and street elements), achieving substantial mIoU improvements of up to 29 absolute percentage points over existing zero-shot baselines on the H3D and Turin3D datasets. What is the implication of the main finding? The research establishes a highly efficient, label-free paradigm for large-scale airborne point cloud interpretation, eliminating the reliance on costly and labor-intensive 3D manual annotations. It demonstrates that integrating authentic 3D geometric topological priors is crucial for correcting cross-modal semantic noise and unleashing the true potential of 2D Vision Foundation Models in 3D geospatial applications. Semantic segmentation of large-scale airborne point clouds traditionally relies on labor-intensive 3D manual annotations. While recent zero-shot methods attempt to alleviate this burden by distilling knowledge from 2D Vision–Language Models (VLMs) via 2D-to-3D projection, they suffer from performance degradation in complex urban environments. Specifically, lacking 3D geometric awareness, 2D VLMs frequently exhibit "semantic bleeding", where large-scale background categories (e.g., ground) erroneously submerge small-scale targets (e.g., vehicles and street elements). To address this issue, we propose a geometry-constrained pseudo-label generation and purification framework. Our approach tackles the problem through a dual-branch design: extracting open-vocabulary semantics via SAM3-based multi-view projection while simultaneously deriving sharp, class-agnostic instances using SAM2 on Gamma-transformed elevation maps. By introducing a geometric–semantic consistency module, we evaluate the internal semantic purity and external spatial homogeneity of these instances, detecting and filtering out semantic misclassifications. The purified pseudo-labels are then used to supervise a 3D sparse convolutional network via a Masked Cross-Entropy Loss. Experiments on the H3D and Turin3D datasets demonstrate that our method recovers small-scale targets that are prone to being submerged, outperforming existing zero-shot baselines by improving mIoU from 52.15% to 63.45% on H3D and from 29.52% to 58.51% on Turin3D, thereby narrowing the performance gap with fully-supervised approaches. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18121875 |
