View in EDS

Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation.

Saved in:
Bibliographic Details
Title: Decoupling foreground and background with Siamese ViT networks for weakly-supervised semantic segmentation.
Authors: Lin, Meiling1,2 (AUTHOR) linmeiling@ime.ac.cn, Li, Gongyan1 (AUTHOR), Xu, Shaoyun1 (AUTHOR), Hao, Yuexing1,2 (AUTHOR), Zhang, Shu1 (AUTHOR)
Source: Neurocomputing. Dec2024, Vol. 610, pN.PAG-N.PAG. 1p.
Subjects: Confidence regions (Mathematics), Data mining, Algorithms, Heuristic
Abstract: Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods. • A novel weakly supervised semantic segmentation framework named DeFB-SV. • A Siamese network consisting of two ViT branches yielding fine-grained pseudo-labels. • A semantically heuristic patch scorer generating mixed-resolution image patches. • A multi-confidence-region strategy achieving finer segmentation results adaptively. [ABSTRACT FROM AUTHOR]
Copyright of Neurocomputing is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:Due to the coarse granularity of information extraction in image-level annotation-based weakly supervised semantic segmentation algorithms, there exists a significant gap between the generated pseudo-labels and the real pixel-level labels. In this paper, we propose the DeFB-SV framework, which consists of a dual-branch Siamese network structure. This framework separates the foreground and background of images by generating unified resolution and mixed resolution class activation maps, which are then fused to obtain pseudo-labels. The mixed-resolution class activation maps are produced by a new mixed-resolution patch partition method, where we introduce a semantically heuristic patch scorer to divide the image into patches of different sizes based on semantics. Additionally, a novel multi-confidence region division mechanism is proposed to enable the adaptive extraction of the effective parts of pseudo-labels, further enhancing the accuracy of weakly supervised semantic segmentation algorithms. The proposed semantic segmentation framework, DeFB-SV, is evaluated on the PASCAL VOC 2012 and MS COCO 2014 datasets, demonstrating comparable segmentation performance with state-of-the-art methods. • A novel weakly supervised semantic segmentation framework named DeFB-SV. • A Siamese network consisting of two ViT branches yielding fine-grained pseudo-labels. • A semantically heuristic patch scorer generating mixed-resolution image patches. • A multi-confidence-region strategy achieving finer segmentation results adaptively. [ABSTRACT FROM AUTHOR]
ISSN:09252312
DOI:10.1016/j.neucom.2024.128540