View in EDS PDF Full Text

On The Limits of Mutation Reduction Strategies.

Saved in:
Bibliographic Details
Title: On The Limits of Mutation Reduction Strategies.
Authors: Gopinath, Rahul1 gopinath@eecs.orst.edu, Alipour, Mohammad Amin1 alipour@eecs.orst.edu, Ahmed, Iftekhar1 ahmedi@onid.orst.edu, Jensen, Carlos1 cjensen@eecs.orst.edu, Groce, Alex1 agroce@gmail.com
Source: ICSE: International Conference on Software Engineering. 5/14/2016, p511-522. 12p.
Subjects: Mutation testing of computer software, Computer software testing, Statistics, Software engineering, Artificial intelligence
Abstract: Although mutation analysis is considered the best way to evaluate the effectiveness of a test suite, hefty computational cost often limits its use. To address this problem, various mutation reduction strategies have been proposed, all seeking to reduce the number of mutants while maintaining the representativeness of an exhaustive mutation analysis. While research has focused on the reduction achieved, the effectiveness of these strategies in selecting representative mutants, and the limits in doing so have not been investigated, either theoretically or empirically. We investigate the practical limits to the effectiveness of mutation reduction strategies, and provide a simple theoretical framework for thinking about the absolute limits. Our results show that the limit in improvement of effectiveness over random sampling for real-world open source programs is a mean of only 13.078%. Interestingly, there is no limit to the improvement that can be made by addition of new mutation operators. Given that this is the maximum that can be achieved with perfect advance knowledge of mutation kills, what can be practically achieved may be much worse. We conclude that more effort should be focused on enhancing mutations than removing operators in the name of selective mutation for questionable benefit. [ABSTRACT FROM AUTHOR]
Copyright of ICSE: International Conference on Software Engineering is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Links:
  – Type: pdflink
Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 125519959
AccessLevel: 6
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: On The Limits of Mutation Reduction Strategies.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Gopinath%2C+Rahul%22">Gopinath, Rahul</searchLink><relatesTo>1</relatesTo><i> gopinath@eecs.orst.edu</i><br /><searchLink fieldCode="AR" term="%22Alipour%2C+Mohammad+Amin%22">Alipour, Mohammad Amin</searchLink><relatesTo>1</relatesTo><i> alipour@eecs.orst.edu</i><br /><searchLink fieldCode="AR" term="%22Ahmed%2C+Iftekhar%22">Ahmed, Iftekhar</searchLink><relatesTo>1</relatesTo><i> ahmedi@onid.orst.edu</i><br /><searchLink fieldCode="AR" term="%22Jensen%2C+Carlos%22">Jensen, Carlos</searchLink><relatesTo>1</relatesTo><i> cjensen@eecs.orst.edu</i><br /><searchLink fieldCode="AR" term="%22Groce%2C+Alex%22">Groce, Alex</searchLink><relatesTo>1</relatesTo><i> agroce@gmail.com</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22ICSE%3A+International+Conference+on+Software+Engineering%22">ICSE: International Conference on Software Engineering</searchLink>. 5/14/2016, p511-522. 12p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Mutation+testing+of+computer+software%22">Mutation testing of computer software</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+software+testing%22">Computer software testing</searchLink><br /><searchLink fieldCode="DE" term="%22Statistics%22">Statistics</searchLink><br /><searchLink fieldCode="DE" term="%22Software+engineering%22">Software engineering</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Although mutation analysis is considered the best way to evaluate the effectiveness of a test suite, hefty computational cost often limits its use. To address this problem, various mutation reduction strategies have been proposed, all seeking to reduce the number of mutants while maintaining the representativeness of an exhaustive mutation analysis. While research has focused on the reduction achieved, the effectiveness of these strategies in selecting representative mutants, and the limits in doing so have not been investigated, either theoretically or empirically. We investigate the practical limits to the effectiveness of mutation reduction strategies, and provide a simple theoretical framework for thinking about the absolute limits. Our results show that the limit in improvement of effectiveness over random sampling for real-world open source programs is a mean of only 13.078%. Interestingly, there is no limit to the improvement that can be made by addition of new mutation operators. Given that this is the maximum that can be achieved with perfect advance knowledge of mutation kills, what can be practically achieved may be much worse. We conclude that more effort should be focused on enhancing mutations than removing operators in the name of selective mutation for questionable benefit. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of ICSE: International Conference on Software Engineering is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=125519959
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1145/2884781.2884787
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 12
        StartPage: 511
    Subjects:
      – SubjectFull: Mutation testing of computer software
        Type: general
      – SubjectFull: Computer software testing
        Type: general
      – SubjectFull: Statistics
        Type: general
      – SubjectFull: Software engineering
        Type: general
      – SubjectFull: Artificial intelligence
        Type: general
    Titles:
      – TitleFull: On The Limits of Mutation Reduction Strategies.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Gopinath, Rahul
      – PersonEntity:
          Name:
            NameFull: Alipour, Mohammad Amin
      – PersonEntity:
          Name:
            NameFull: Ahmed, Iftekhar
      – PersonEntity:
          Name:
            NameFull: Jensen, Carlos
      – PersonEntity:
          Name:
            NameFull: Groce, Alex
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 14
              M: 05
              Text: 5/14/2016
              Type: published
              Y: 2016
          Titles:
            – TitleFull: ICSE: International Conference on Software Engineering
              Type: main
ResultId 1