Generalized Robot Vision-Language Model via Linguistic Foreground-Aware Contrast.
Saved in:
| Title: | Generalized Robot Vision-Language Model via Linguistic Foreground-Aware Contrast. |
|---|---|
| Authors: | Liu, Kangcheng1,2 (AUTHOR) kcliu@mae.cuhk.edu.hk, Wang, Chaoqun3 (AUTHOR), Han, Xiaodong4 (AUTHOR), Liu, Yong-Jin5 (AUTHOR), Chen, Baoquan6 (AUTHOR) |
| Source: | International Journal of Computer Vision. Jun2025, Vol. 133 Issue 6, p3481-3518. 38p. |
| Subjects: | Artificial intelligence, Linguistic models, Point cloud, Image processing, Image representation |
| Abstract: | Contrastive learning has recently demonstrated great potential for unsupervised pre-training in 3D scene understanding tasks. However, most existing work randomly selects point features as anchors while building contrast, leading to a clear bias toward background points that often dominate in 3D scenes. Also, object awareness and foreground-to-background discrimination are neglected, making contrastive learning less effective. To tackle these issues, we propose a general foreground-aware feature contrast FAC++ framework to learn more effective point cloud representations in pre-training. FAC++ consists of two novel contrast designs to construct more effective and informative contrast pairs. The first is building positive pairs within the same foreground segment where points tend to have the same semantics. The second is that we prevent over-discrimination between 3D segments/objects and encourage grouped foreground-to-background distinctions at the segment level with adaptive feature learning in a Siamese correspondence network, which adaptively learns feature correlations within and across point cloud views effectively. Our proposed approach enhances both the local coherence as well as the overall feature discrimination. Moreover, we have designed the linguistic foreground-aware regional point sampling to enhance more balanced foreground-aware learning, which is termed FAC++. Visualization with point activation maps shows that our contrast pairs capture clear correspondences among foreground regions during pre-training. Quantitative experiments also show that FAC++ achieves superior knowledge transfer and data efficiency in various downstream 3D semantic segmentation, instance segmentation as well as object detection tasks. All codes, data, and models are available at: (https://github.com/KangchengLiu/FAC_Foreground_Aware_Contrast). [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Computer Vision is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 185069262 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Generalized Robot Vision-Language Model via Linguistic Foreground-Aware Contrast. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Liu%2C+Kangcheng%22">Liu, Kangcheng</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> kcliu@mae.cuhk.edu.hk</i><br /><searchLink fieldCode="AR" term="%22Wang%2C+Chaoqun%22">Wang, Chaoqun</searchLink><relatesTo>3</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Han%2C+Xiaodong%22">Han, Xiaodong</searchLink><relatesTo>4</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Liu%2C+Yong-Jin%22">Liu, Yong-Jin</searchLink><relatesTo>5</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Chen%2C+Baoquan%22">Chen, Baoquan</searchLink><relatesTo>6</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Computer+Vision%22">International Journal of Computer Vision</searchLink>. Jun2025, Vol. 133 Issue 6, p3481-3518. 38p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink><br /><searchLink fieldCode="DE" term="%22Linguistic+models%22">Linguistic models</searchLink><br /><searchLink fieldCode="DE" term="%22Point+cloud%22">Point cloud</searchLink><br /><searchLink fieldCode="DE" term="%22Image+processing%22">Image processing</searchLink><br /><searchLink fieldCode="DE" term="%22Image+representation%22">Image representation</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Contrastive learning has recently demonstrated great potential for unsupervised pre-training in 3D scene understanding tasks. However, most existing work randomly selects point features as anchors while building contrast, leading to a clear bias toward background points that often dominate in 3D scenes. Also, object awareness and foreground-to-background discrimination are neglected, making contrastive learning less effective. To tackle these issues, we propose a general foreground-aware feature contrast FAC++ framework to learn more effective point cloud representations in pre-training. FAC++ consists of two novel contrast designs to construct more effective and informative contrast pairs. The first is building positive pairs within the same foreground segment where points tend to have the same semantics. The second is that we prevent over-discrimination between 3D segments/objects and encourage grouped foreground-to-background distinctions at the segment level with adaptive feature learning in a Siamese correspondence network, which adaptively learns feature correlations within and across point cloud views effectively. Our proposed approach enhances both the local coherence as well as the overall feature discrimination. Moreover, we have designed the linguistic foreground-aware regional point sampling to enhance more balanced foreground-aware learning, which is termed FAC++. Visualization with point activation maps shows that our contrast pairs capture clear correspondences among foreground regions during pre-training. Quantitative experiments also show that FAC++ achieves superior knowledge transfer and data efficiency in various downstream 3D semantic segmentation, instance segmentation as well as object detection tasks. All codes, data, and models are available at: (https://github.com/KangchengLiu/FAC_Foreground_Aware_Contrast). [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of International Journal of Computer Vision is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=185069262 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1007/s11263-024-02340-z Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 38 StartPage: 3481 Subjects: – SubjectFull: Artificial intelligence Type: general – SubjectFull: Linguistic models Type: general – SubjectFull: Point cloud Type: general – SubjectFull: Image processing Type: general – SubjectFull: Image representation Type: general Titles: – TitleFull: Generalized Robot Vision-Language Model via Linguistic Foreground-Aware Contrast. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Liu, Kangcheng – PersonEntity: Name: NameFull: Wang, Chaoqun – PersonEntity: Name: NameFull: Han, Xiaodong – PersonEntity: Name: NameFull: Liu, Yong-Jin – PersonEntity: Name: NameFull: Chen, Baoquan IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 06 Text: Jun2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 09205691 Numbering: – Type: volume Value: 133 – Type: issue Value: 6 Titles: – TitleFull: International Journal of Computer Vision Type: main |
| ResultId | 1 |
