View in EDS HTML Full Text PDF Full Text

Testing time limits in screener questions for online surveys with programmers.

Saved in:
Bibliographic Details
Title: Testing time limits in screener questions for online surveys with programmers.
Authors: Danilova, Anastasia1 danilova@cs.uni-bonn.de, Horstmann, Stefan1 Stefan.Horstmann@gmx.net, Smith, Matthew1 smith@cs.uni-bonn.de, Naiakshina, Alena1 naiakshi@cs.uni-bonn.de
Source: ICSE: International Conference on Software Engineering. 2022, p2080-2090. 11p.
Subjects: Computer surveys, Computer programming, Qualtrics Inc., Benchmarking (Management), Methodology
Abstract: Recruiting study participants with programming skill is essential for researchers. As programming is not a common skill, recruiting programmers as participants in large numbers is challenging. Platforms like Amazon MTurk or Qualtrics offer to recruit participants with programming knowledge. As this is self-reported, participants without programming experience could still take part, either due to a misunderstanding or to obtain the study compensation. If these participants are not detected, the data quality will suffer. To tackle this, Danilova et al. [11] developed and tested screening tasks to detect non-programmers. Unfortunately, the most reliable screeners were also those that took the most time. Since screeners should take as little time as possible, we examine whether the introduction of time limits allows us to create more efficient (i.e., quicker but still reliable) screeners. Our results show that this is possible and we extend the pool of screeners and make recommendations on how to improve the process. [ABSTRACT FROM AUTHOR]
Copyright of ICSE: International Conference on Software Engineering is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Full text is not displayed to guests.
FullText Links:
  – Type: pdflink
Text:
  Availability: 1
Header DbId: egs
DbLabel: Engineering Source
An: 185195864
AccessLevel: 6
PubType: Conference
PubTypeId: conference
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Testing time limits in screener questions for online surveys with programmers.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Danilova%2C+Anastasia%22">Danilova, Anastasia</searchLink><relatesTo>1</relatesTo><i> danilova@cs.uni-bonn.de</i><br /><searchLink fieldCode="AR" term="%22Horstmann%2C+Stefan%22">Horstmann, Stefan</searchLink><relatesTo>1</relatesTo><i> Stefan.Horstmann@gmx.net</i><br /><searchLink fieldCode="AR" term="%22Smith%2C+Matthew%22">Smith, Matthew</searchLink><relatesTo>1</relatesTo><i> smith@cs.uni-bonn.de</i><br /><searchLink fieldCode="AR" term="%22Naiakshina%2C+Alena%22">Naiakshina, Alena</searchLink><relatesTo>1</relatesTo><i> naiakshi@cs.uni-bonn.de</i>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%22ICSE%3A+International+Conference+on+Software+Engineering%22">ICSE: International Conference on Software Engineering</searchLink>. 2022, p2080-2090. 11p.
– Name: Subject
  Label: Subjects
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computer+surveys%22">Computer surveys</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+programming%22">Computer programming</searchLink><br /><searchLink fieldCode="DE" term="%22Qualtrics+Inc%2E%22">Qualtrics Inc.</searchLink><br /><searchLink fieldCode="DE" term="%22Benchmarking+%28Management%29%22">Benchmarking (Management)</searchLink><br /><searchLink fieldCode="DE" term="%22Methodology%22">Methodology</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Recruiting study participants with programming skill is essential for researchers. As programming is not a common skill, recruiting programmers as participants in large numbers is challenging. Platforms like Amazon MTurk or Qualtrics offer to recruit participants with programming knowledge. As this is self-reported, participants without programming experience could still take part, either due to a misunderstanding or to obtain the study compensation. If these participants are not detected, the data quality will suffer. To tackle this, Danilova et al. [11] developed and tested screening tasks to detect non-programmers. Unfortunately, the most reliable screeners were also those that took the most time. Since screeners should take as little time as possible, we examine whether the introduction of time limits allows us to create more efficient (i.e., quicker but still reliable) screeners. Our results show that this is possible and we extend the pool of screeners and make recommendations on how to improve the process. [ABSTRACT FROM AUTHOR]
– Name: AbstractSuppliedCopyright
  Label:
  Group: Ab
  Data: <i>Copyright of ICSE: International Conference on Software Engineering is the property of Association for Computing Machinery and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=185195864
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1145/3510003.3510223
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 2080
    Subjects:
      – SubjectFull: Computer surveys
        Type: general
      – SubjectFull: Computer programming
        Type: general
      – SubjectFull: Qualtrics Inc.
        Type: general
      – SubjectFull: Benchmarking (Management)
        Type: general
      – SubjectFull: Methodology
        Type: general
    Titles:
      – TitleFull: Testing time limits in screener questions for online surveys with programmers.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Danilova, Anastasia
      – PersonEntity:
          Name:
            NameFull: Horstmann, Stefan
      – PersonEntity:
          Name:
            NameFull: Smith, Matthew
      – PersonEntity:
          Name:
            NameFull: Naiakshina, Alena
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 05
              Text: 2022
              Type: published
              Y: 2022
          Titles:
            – TitleFull: ICSE: International Conference on Software Engineering
              Type: main
ResultId 1