View in EDS HTML Full Text PDF Full Text

SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image Synthesis.

Saved in:
Bibliographic Details
Title: SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image Synthesis.
Authors: Cheng, Zhenwei1 (AUTHOR) 202135253@mail.sdu.edu.cn, Wu, Lei1 (AUTHOR) i_lily@sdu.edu.cn, Li, Xiang1 (AUTHOR) xiangli_@mail.sdu.edu.cn, Meng, Xiangxu1 (AUTHOR) mxx@sdu.edu.cn
Source: Computer Graphics Forum. Sep2024, Vol. 43 Issue 6, p1-13. 13p.
Subjects: Video processing, Class differences
Abstract: Freehand sketch‐to‐image (S2I) is a challenging task due to the individualized lines and the random shape of freehand sketches. The multi‐class freehand sketch‐to‐image synthesis task, in turn, presents new challenges for this research area. This task requires not only the consideration of the problems posed by freehand sketches but also the analysis of multi‐class domain differences in the conditions of a single model. However, existing methods often have difficulty learning domain differences between multiple classes, and cannot generate controllable and appropriate textures while maintaining shape stability. In this paper, we propose a style‐guided multi‐class freehand sketch‐to‐image synthesis model, SMFS‐GAN, which can be trained using only unpaired data. To this end, we introduce a contrast‐based style encoder that optimizes the network's perception of domain disparities by explicitly modelling the differences between classes and thus extracting style information across domains. Further, to optimize the fine‐grained texture of the generated results and the shape consistency with freehand sketches, we propose a local texture refinement discriminator and a Shape Constraint Module, respectively. In addition, to address the imbalance of data classes in the QMUL‐Sketch dataset, we add 6K images by drawing manually and obtain QMUL‐Sketch+ dataset. Extensive experiments on SketchyCOCO Object dataset, QMUL‐Sketch+ dataset and Pseudosketches dataset demonstrate the effectiveness as well as the superiority of our proposed method. [ABSTRACT FROM AUTHOR]
Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 179808113
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image Synthesis.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Cheng%2C+Zhenwei%22">Cheng, Zhenwei</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> 202135253@mail.sdu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Wu%2C+Lei%22">Wu, Lei</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> i_lily@sdu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Li%2C+Xiang%22">Li, Xiang</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> xiangli_@mail.sdu.edu.cn</i><br /><searchLink fieldCode="AR" term="%22Meng%2C+Xiangxu%22">Meng, Xiangxu</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> mxx@sdu.edu.cn</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Computer+Graphics+Forum%22">Computer Graphics Forum</searchLink>. Sep2024, Vol. 43 Issue 6, p1-13. 13p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Video+processing%22">Video processing</searchLink><br /><searchLink fieldCode="DE" term="%22Class+differences%22">Class differences</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Freehand sketch‐to‐image (S2I) is a challenging task due to the individualized lines and the random shape of freehand sketches. The multi‐class freehand sketch‐to‐image synthesis task, in turn, presents new challenges for this research area. This task requires not only the consideration of the problems posed by freehand sketches but also the analysis of multi‐class domain differences in the conditions of a single model. However, existing methods often have difficulty learning domain differences between multiple classes, and cannot generate controllable and appropriate textures while maintaining shape stability. In this paper, we propose a style‐guided multi‐class freehand sketch‐to‐image synthesis model, SMFS‐GAN, which can be trained using only unpaired data. To this end, we introduce a contrast‐based style encoder that optimizes the network's perception of domain disparities by explicitly modelling the differences between classes and thus extracting style information across domains. Further, to optimize the fine‐grained texture of the generated results and the shape consistency with freehand sketches, we propose a local texture refinement discriminator and a Shape Constraint Module, respectively. In addition, to address the imbalance of data classes in the QMUL‐Sketch dataset, we add 6K images by drawing manually and obtain QMUL‐Sketch+ dataset. Extensive experiments on SketchyCOCO Object dataset, QMUL‐Sketch+ dataset and Pseudosketches dataset demonstrate the effectiveness as well as the superiority of our proposed method. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Computer Graphics Forum is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=179808113
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1111/cgf.15190
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 13
        StartPage: 1
    Subjects:
      – SubjectFull: Video processing
        Type: general
      – SubjectFull: Class differences
        Type: general
    Titles:
      – TitleFull: SMFS‐GAN: Style‐Guided Multi‐class Freehand Sketch‐to‐Image Synthesis.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Cheng, Zhenwei
      – PersonEntity:
          Name:
            NameFull: Wu, Lei
      – PersonEntity:
          Name:
            NameFull: Li, Xiang
      – PersonEntity:
          Name:
            NameFull: Meng, Xiangxu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 09
              Text: Sep2024
              Type: published
              Y: 2024
          Identifiers:
            – Type: issn-print
              Value: 01677055
          Numbering:
            – Type: volume
              Value: 43
            – Type: issue
              Value: 6
          Titles:
            – TitleFull: Computer Graphics Forum
              Type: main
ResultId 1