View in EDS

Effect sizes and their variance for AB/BA crossover design studies.

Saved in:
Bibliographic Details
Title: Effect sizes and their variance for AB/BA crossover design studies.
Authors: Madeyski, Lech1 Lech.Madeyski@pwr.edu.pl, Kitchenham, Barbara2 b.a.kitchenham@keele.ac.uk
Source: Empirical Software Engineering. Aug2018, Vol. 23 Issue 4, p1982-2017. 36p.
Abstract: Vegas et al. IEEE Trans Softw Eng 42(2):120:135 (2016) raised concerns about the use of AB/BA crossover designs in empirical software engineering studies. This paper addresses issues related to calculating standardized effect sizes and their variances that were not addressed by the Vegas et al.’s paper. In a repeated measures design such as an AB/BA crossover design each participant uses each method. There are two major implication of this that have not been discussed in the software engineering literature. Firstly, there are potentially two different standardized mean difference effect sizes that can be calculated, depending on whether the mean difference is standardized by the pooled within groups variance or the within-participants variance. Secondly, as for any estimated parameters and also for the purposes of undertaking meta-analysis, it is necessary to calculate the variance of the standardized mean difference effect sizes (which is not the same as the variance of the study). We present the model underlying the AB/BA crossover design and provide two examples to demonstrate how to construct the two standardized mean difference effect sizes and their variances, both from standard descriptive statistics and from the outputs of statistical software. Finally, we discuss the implication of these issues for reporting and planning software engineering experiments. In particular we consider how researchers should choose between a crossover design or a between groups design. [ABSTRACT FROM AUTHOR]
Copyright of Empirical Software Engineering is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
FullText Text:
  Availability: 0
Header DbId: egs
DbLabel: Engineering Source
An: 130276623
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Effect sizes and their variance for AB/BA crossover design studies.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Madeyski%2C+Lech%22">Madeyski, Lech</searchLink><relatesTo>1</relatesTo><i> Lech.Madeyski@pwr.edu.pl</i><br /><searchLink fieldCode="AR" term="%22Kitchenham%2C+Barbara%22">Kitchenham, Barbara</searchLink><relatesTo>2</relatesTo><i> b.a.kitchenham@keele.ac.uk</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22Empirical+Software+Engineering%22">Empirical Software Engineering</searchLink>. Aug2018, Vol. 23 Issue 4, p1982-2017. 36p.
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Vegas et al. IEEE Trans Softw Eng 42(2):120:135 (<xref>2016</xref>) raised concerns about the use of AB/BA crossover designs in empirical software engineering studies. This paper addresses issues related to calculating standardized effect sizes and their variances that were not addressed by the Vegas et al.’s paper. In a repeated measures design such as an AB/BA crossover design each participant uses each method. There are two major implication of this that have not been discussed in the software engineering literature. Firstly, there are potentially two different standardized mean difference effect sizes that can be calculated, depending on whether the mean difference is standardized by the pooled within groups variance or the within-participants variance. Secondly, as for any estimated parameters and also for the purposes of undertaking meta-analysis, it is necessary to calculate the variance of the standardized mean difference effect sizes (which is not the same as the variance of the study). We present the model underlying the AB/BA crossover design and provide two examples to demonstrate how to construct the two standardized mean difference effect sizes and their variances, both from standard descriptive statistics and from the outputs of statistical software. Finally, we discuss the implication of these issues for reporting and planning software engineering experiments. In particular we consider how researchers should choose between a crossover design or a between groups design. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of Empirical Software Engineering is the property of Springer Nature and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=130276623
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s10664-017-9574-5
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 36
        StartPage: 1982
    Titles:
      – TitleFull: Effect sizes and their variance for AB/BA crossover design studies.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Madeyski, Lech
      – PersonEntity:
          Name:
            NameFull: Kitchenham, Barbara
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 08
              Text: Aug2018
              Type: published
              Y: 2018
          Identifiers:
            – Type: issn-print
              Value: 13823256
          Numbering:
            – Type: volume
              Value: 23
            – Type: issue
              Value: 4
          Titles:
            – TitleFull: Empirical Software Engineering
              Type: main
ResultId 1