Cross-Project Generalization Challenges in Transformer-Based Code Smell Detection: An Empirical Study.
Saved in:
| Title: | Cross-Project Generalization Challenges in Transformer-Based Code Smell Detection: An Empirical Study. |
|---|---|
| Authors: | Burra, Bhavana Chowdary1 burrabhavana@gmail.com, Shukla, Seema1, Goyal, Mayank Kumar1 |
| Source: | International Journal of Performability Engineering. Jun2026, Vol. 22 Issue 6, p318-330. 13p. |
| Subjects: | Transformer models, Machine learning, Computer software quality control, Maintainability (Engineering), Optimization algorithms, Skewness (Probability theory) |
| Abstract: | Detecting code smells is very important for increasing software maintainability and lowering the technical debt of large-scale software systems. Traditional machine learning methods rely heavily on manually engineered features and, as a result, can struggle to generalize across projects due to domain differences and class imbalance in the datasets. However, although transformer-based pre-trained models have shown great promise in understanding the semantics of source code, there has been limited investigation into how well they perform across different datasets, particularly balanced versus imbalanced ones. In this study, we compare the performance of baseline machine learning models and transformer-based models for detecting multiple types of code smells on two heterogeneous datasets with different distribution properties. From the analysis, we see that the degree of imbalance in the datasets and the differences between the two domains significantly affect the performance and generalization of the various models. Our experimental results show that whilst transformer-based models outperform baseline machine learning models, the extent of their advantage varies with dataset characteristics; therefore, transformer-based models do not generalize well across projects. We have also found that providing domain-specific fine-tuning strategies can improve adaptability and detection performance in real-world use. This study provides insights into dataset characteristics, model behavior across domains, and the need for adaptive learning approaches to develop robust, generalized code smell detection systems. [ABSTRACT FROM AUTHOR] |
| Copyright of International Journal of Performability Engineering is the property of Totem Publisher, Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Links: – Type: pdflink Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 194978494 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Cross-Project Generalization Challenges in Transformer-Based Code Smell Detection: An Empirical Study. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Burra%2C+Bhavana+Chowdary%22">Burra, Bhavana Chowdary</searchLink><relatesTo>1</relatesTo><i> burrabhavana@gmail.com</i><br /><searchLink fieldCode="AR" term="%22Shukla%2C+Seema%22">Shukla, Seema</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Goyal%2C+Mayank+Kumar%22">Goyal, Mayank Kumar</searchLink><relatesTo>1</relatesTo> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22International+Journal+of+Performability+Engineering%22">International Journal of Performability Engineering</searchLink>. Jun2026, Vol. 22 Issue 6, p318-330. 13p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Transformer+models%22">Transformer models</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+software+quality+control%22">Computer software quality control</searchLink><br /><searchLink fieldCode="DE" term="%22Maintainability+%28Engineering%29%22">Maintainability (Engineering)</searchLink><br /><searchLink fieldCode="DE" term="%22Optimization+algorithms%22">Optimization algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Skewness+%28Probability+theory%29%22">Skewness (Probability theory)</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Detecting code smells is very important for increasing software maintainability and lowering the technical debt of large-scale software systems. Traditional machine learning methods rely heavily on manually engineered features and, as a result, can struggle to generalize across projects due to domain differences and class imbalance in the datasets. However, although transformer-based pre-trained models have shown great promise in understanding the semantics of source code, there has been limited investigation into how well they perform across different datasets, particularly balanced versus imbalanced ones. In this study, we compare the performance of baseline machine learning models and transformer-based models for detecting multiple types of code smells on two heterogeneous datasets with different distribution properties. From the analysis, we see that the degree of imbalance in the datasets and the differences between the two domains significantly affect the performance and generalization of the various models. Our experimental results show that whilst transformer-based models outperform baseline machine learning models, the extent of their advantage varies with dataset characteristics; therefore, transformer-based models do not generalize well across projects. We have also found that providing domain-specific fine-tuning strategies can improve adaptability and detection performance in real-world use. This study provides insights into dataset characteristics, model behavior across domains, and the need for adaptive learning approaches to develop robust, generalized code smell detection systems. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of International Journal of Performability Engineering is the property of Totem Publisher, Inc. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=194978494 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.23940/ijpe.26.06.p3.318330 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 318 Subjects: – SubjectFull: Transformer models Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Computer software quality control Type: general – SubjectFull: Maintainability (Engineering) Type: general – SubjectFull: Optimization algorithms Type: general – SubjectFull: Skewness (Probability theory) Type: general Titles: – TitleFull: Cross-Project Generalization Challenges in Transformer-Based Code Smell Detection: An Empirical Study. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Burra, Bhavana Chowdary – PersonEntity: Name: NameFull: Shukla, Seema – PersonEntity: Name: NameFull: Goyal, Mayank Kumar IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 06 Text: Jun2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 09731318 Numbering: – Type: volume Value: 22 – Type: issue Value: 6 Titles: – TitleFull: International Journal of Performability Engineering Type: main |
| ResultId | 1 |
