Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software Systems.
Saved in:
| Title: | Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software Systems. |
|---|---|
| Authors: | Mühlbauer, Stefan1, Sattler, Florian2, Kaltenecker, Christian2, Dorn, Johannes1, Apel, Sven2, Siegmund, Norbert3 |
| Source: | ICSE: International Conference on Software Engineering. 2023, p2085-2097. 13p. |
| Subjects: | Workload of computer networks, Machine learning, Software engineering, Reliability in engineering, Reliability engineering software |
| Abstract: | Modern software systems often exhibit numerous configuration options to tailor them to user requirements, including the system's performance behavior. Performance models derived via machine learning are an established approach for estimating and optimizing configuration-dependent software performance. Most existing approaches in this area rely on software performance measurements conducted with a single workload (i.e., input fed to a system). This single workload, however, is often not representative of a software system's real-world application scenarios. Understanding to what extent configuration and workload---individually and combined---cause a software system's performance to vary is key to understand whether performance models are generalizable across different configurations and workloads. Yet, so far, this aspect has not been systematically studied. To fill this gap, we conducted a systematic empirical study across 25 258 configurations from nine real-world configurable software systems to investigate the effects of workload variation at system-level performance and for individual configuration options. We explore driving causes for workload-configuration interactions by enriching performance observations with option-specific code coverage information. Our results demonstrate that workloads can induce substantial performance variation and interact with configuration options, often in non-monotonous ways. This limits not only the generaliz-ability of single-workload models, but also challenges assumptions for existing transfer-learning techniques. As a result, workloads should be considered when building performance prediction models to maintain and improve representativeness and reliability. [ABSTRACT FROM AUTHOR] |
| Copyright of ICSE: International Conference on Software Engineering is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| FullText | Links: – Type: pdflink Text: Availability: 1 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 185196146 AccessLevel: 6 PubType: Conference PubTypeId: conference PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software Systems. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Mühlbauer%2C+Stefan%22">Mühlbauer, Stefan</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Sattler%2C+Florian%22">Sattler, Florian</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Kaltenecker%2C+Christian%22">Kaltenecker, Christian</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Dorn%2C+Johannes%22">Dorn, Johannes</searchLink><relatesTo>1</relatesTo><br /><searchLink fieldCode="AR" term="%22Apel%2C+Sven%22">Apel, Sven</searchLink><relatesTo>2</relatesTo><br /><searchLink fieldCode="AR" term="%22Siegmund%2C+Norbert%22">Siegmund, Norbert</searchLink><relatesTo>3</relatesTo> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22ICSE%3A+International+Conference+on+Software+Engineering%22">ICSE: International Conference on Software Engineering</searchLink>. 2023, p2085-2097. 13p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Workload+of+computer+networks%22">Workload of computer networks</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Software+engineering%22">Software engineering</searchLink><br /><searchLink fieldCode="DE" term="%22Reliability+in+engineering%22">Reliability in engineering</searchLink><br /><searchLink fieldCode="DE" term="%22Reliability+engineering+software%22">Reliability engineering software</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Modern software systems often exhibit numerous configuration options to tailor them to user requirements, including the system's performance behavior. Performance models derived via machine learning are an established approach for estimating and optimizing configuration-dependent software performance. Most existing approaches in this area rely on software performance measurements conducted with a single workload (i.e., input fed to a system). This single workload, however, is often not representative of a software system's real-world application scenarios. Understanding to what extent configuration and workload---individually and combined---cause a software system's performance to vary is key to understand whether performance models are generalizable across different configurations and workloads. Yet, so far, this aspect has not been systematically studied. To fill this gap, we conducted a systematic empirical study across 25 258 configurations from nine real-world configurable software systems to investigate the effects of workload variation at system-level performance and for individual configuration options. We explore driving causes for workload-configuration interactions by enriching performance observations with option-specific code coverage information. Our results demonstrate that workloads can induce substantial performance variation and interact with configuration options, often in non-monotonous ways. This limits not only the generaliz-ability of single-workload models, but also challenges assumptions for existing transfer-learning techniques. As a result, workloads should be considered when building performance prediction models to maintain and improve representativeness and reliability. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of ICSE: International Conference on Software Engineering is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=185196146 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1109/ICSE48619.2023.00176 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 13 StartPage: 2085 Subjects: – SubjectFull: Workload of computer networks Type: general – SubjectFull: Machine learning Type: general – SubjectFull: Software engineering Type: general – SubjectFull: Reliability in engineering Type: general – SubjectFull: Reliability engineering software Type: general Titles: – TitleFull: Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software Systems. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Mühlbauer, Stefan – PersonEntity: Name: NameFull: Sattler, Florian – PersonEntity: Name: NameFull: Kaltenecker, Christian – PersonEntity: Name: NameFull: Dorn, Johannes – PersonEntity: Name: NameFull: Apel, Sven – PersonEntity: Name: NameFull: Siegmund, Norbert IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: 2023 Type: published Y: 2023 Titles: – TitleFull: ICSE: International Conference on Software Engineering Type: main |
| ResultId | 1 |
