Class-Aware Visual Prompt Learning for Vision Language Models.
Saved in:
| Title: | Class-Aware Visual Prompt Learning for Vision Language Models. |
|---|---|
| Authors: | Sihui Zhang1, Zhijiang Li1 lizhijiang@whu.edu.cn |
| Source: | Journal of Imaging Science & Technology. May/Jun2025, Vol. 69 Issue 3, p1-7. 7p. |
| Subjects: | Language models, Visual learning, Recognition (Psychology), Task performance, Generalization |
| Abstract: | Vision-language pre-trained (VLP) models, such as CLIP, have exhibited remarkable performance in downstream tasks with excellent generalization capabilities. Meanwhile, textual and visual prompt learning have been widely adopted to enhance VLP model performance in downstream tasks. However, a challenging issue in visual prompt learning is the inferior ability on few-shot recognition tasks, the inability to capture specific class information. Thus, we propose a class-aware visual prompt learning method to enhance the perceptual abilities of VLP model with an independent class prompting module, which consists of trainable prompts for each class. As class-aware prompts tend to be inaccurate in the training process, we developed an intra-class compactness loss and inter-class dispersion loss to enhance the intra-class consistency. Finally, we introduced attention-based adapter layers to tackle the prompt selection issue. Extensive experiments demonstrated that our method achieved superior efficiency and effectiveness, surpassing previous visual prompting methods in a series of downstream datasets. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Imaging Science & Technology is the property of International Society for Imaging Science & Technology and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 186634227 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Class-Aware Visual Prompt Learning for Vision Language Models. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Sihui+Zhang%22">Sihui Zhang</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Zhijiang+Li%22">Zhijiang Li</searchLink><relatesTo>1</relatesTo><i> lizhijiang@whu.edu.cn</i> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Journal+of+Imaging+Science+%26+Technology%22">Journal of Imaging Science & Technology</searchLink>. May/Jun2025, Vol. 69 Issue 3, p1-7. 7p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Visual+learning%22">Visual learning</searchLink><br /><searchLink fieldCode="DE" term="%22Recognition+%28Psychology%29%22">Recognition (Psychology)</searchLink><br /><searchLink fieldCode="DE" term="%22Task+performance%22">Task performance</searchLink><br /><searchLink fieldCode="DE" term="%22Generalization%22">Generalization</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Vision-language pre-trained (VLP) models, such as CLIP, have exhibited remarkable performance in downstream tasks with excellent generalization capabilities. Meanwhile, textual and visual prompt learning have been widely adopted to enhance VLP model performance in downstream tasks. However, a challenging issue in visual prompt learning is the inferior ability on few-shot recognition tasks, the inability to capture specific class information. Thus, we propose a class-aware visual prompt learning method to enhance the perceptual abilities of VLP model with an independent class prompting module, which consists of trainable prompts for each class. As class-aware prompts tend to be inaccurate in the training process, we developed an intra-class compactness loss and inter-class dispersion loss to enhance the intra-class consistency. Finally, we introduced attention-based adapter layers to tackle the prompt selection issue. Extensive experiments demonstrated that our method achieved superior efficiency and effectiveness, surpassing previous visual prompting methods in a series of downstream datasets. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Journal of Imaging Science & Technology is the property of International Society for Imaging Science & Technology and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=186634227 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.2352/J.ImagingSci.Technol.2025.69.3.030415 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 7 StartPage: 1 Subjects: – SubjectFull: Language models Type: general – SubjectFull: Visual learning Type: general – SubjectFull: Recognition (Psychology) Type: general – SubjectFull: Task performance Type: general – SubjectFull: Generalization Type: general Titles: – TitleFull: Class-Aware Visual Prompt Learning for Vision Language Models. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Sihui Zhang – PersonEntity: Name: NameFull: Zhijiang Li IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May/Jun2025 Type: published Y: 2025 Identifiers: – Type: issn-print Value: 10623701 Numbering: – Type: volume Value: 69 – Type: issue Value: 3 Titles: – TitleFull: Journal of Imaging Science & Technology Type: main |
| ResultId | 1 |
