Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation.
Saved in:
| Title: | Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation. |
|---|---|
| Authors: | Lin, Meiling1,2 (AUTHOR) linmeiling@ime.ac.cn, Li, Gongyan1 (AUTHOR), Xu, Shaoyun1 (AUTHOR), Hao, Yuexing1,2 (AUTHOR), Zhang, Shu1 (AUTHOR) |
| Source: | Neurocomputing. Dec2024, Vol. 610, pN.PAG-N.PAG. 1p. |
| Subjects: | Confidence regions (Mathematics), Data mining, Algorithms, Heuristic |
| Abstract: | Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods. • A novel weakly supervised semantic segmentation framework named DeFB-SV. • A Siamese network consisting of two ViT branches yielding fine-grained pseudo-labels. • A semantically heuristic patch scorer generating mixed-resolution image patches. • A multi-confidence-region strategy achieving finer segmentation results adaptively. [ABSTRACT FROM AUTHOR] |
| Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 180629562 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Lin%2C+Meiling%22">Lin, Meiling</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<i> linmeiling@ime.ac.cn</i><br /><searchLink fieldCode="AR" term="%22Li%2C+Gongyan%22">Li, Gongyan</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Xu%2C+Shaoyun%22">Xu, Shaoyun</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Hao%2C+Yuexing%22">Hao, Yuexing</searchLink><relatesTo>1,2</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Zhang%2C+Shu%22">Zhang, Shu</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Neurocomputing%22">Neurocomputing</searchLink>. Dec2024, Vol. 610, pN.PAG-N.PAG. 1p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Confidence+regions+%28Mathematics%29%22">Confidence regions (Mathematics)</searchLink><br /><searchLink fieldCode="DE" term="%22Data+mining%22">Data mining</searchLink><br /><searchLink fieldCode="DE" term="%22Algorithms%22">Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Heuristic%22">Heuristic</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods. • A novel weakly supervised semantic segmentation framework named DeFB-SV. • A Siamese network consisting of two ViT branches yielding fine-grained pseudo-labels. • A semantically heuristic patch scorer generating mixed-resolution image patches. • A multi-confidence-region strategy achieving finer segmentation results adaptively. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=180629562 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1016/j.neucom.2024.128540 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 1 StartPage: N.PAG Subjects: – SubjectFull: Confidence regions (Mathematics) Type: general – SubjectFull: Data mining Type: general – SubjectFull: Algorithms Type: general – SubjectFull: Heuristic Type: general Titles: – TitleFull: Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Lin, Meiling – PersonEntity: Name: NameFull: Li, Gongyan – PersonEntity: Name: NameFull: Xu, Shaoyun – PersonEntity: Name: NameFull: Hao, Yuexing – PersonEntity: Name: NameFull: Zhang, Shu IsPartOfRelationships: – BibEntity: Dates: – D: 28 M: 12 Text: Dec2024 Type: published Y: 2024 Identifiers: – Type: issn-print Value: 09252312 Numbering: – Type: volume Value: 610 Titles: – TitleFull: Neurocomputing Type: main |
| ResultId | 1 |
