View in EDS

Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study. Research Report. ETS RR-22-01

Saved in:
Bibliographic Details
Title: Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study. Research Report. ETS RR-22-01
Language: English
Authors: Guo, Hongwen, Rios, Joseph A., Ling, Guangming, Wang, Zhen, Gu, Lin, Yang, Zhitong, Liu, Lydia O.
Source: ETS Research Report Series. Dec 2022.
Availability: Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/
Peer Reviewed: Y
Page Count: 22
Publication Date: 2022
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Test Format, Test Wiseness, Test Items, Item Response Theory, Difficulty Level, Scores, Effect Size, College Students, Foreign Countries, Reaction Time
Geographic Terms: United States, Canada
ISSN: 2330-8516
Abstract: Different variants of the selected-response (SR) item type have been developed for various reasons (i.e., simulating realistic situations, examining critical-thinking and/or problem-solving skills). Generally, the variants of SR item format are more complex than the traditional multiple-choice (MC) items, which may be more challenging to test takers and thus may discourage their test engagement on low-stakes assessments. Low test-taking effort has been shown to distort test scores and thereby diminish score validity. We used data collected from a large-scale assessment to investigate how variants of the SR item format may impact test properties and test engagement. Results show that the studied variants of SR item format were generally harder and more time consuming compared to the traditional MC item format, but they did not show negative impact on test-taking effort. However, item position had a dominating influence on nonresponse rates and rapid-guessing rates in a cumulative fashion, even though the effect sizes were relatively small in the studied data.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1366810
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=EJ1366810
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: EJ1366810
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study. Research Report. ETS RR-22-01
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Guo%2C+Hongwen%22">Guo, Hongwen</searchLink><br /><searchLink fieldCode="AR" term="%22Rios%2C+Joseph+A%2E%22">Rios, Joseph A.</searchLink><br /><searchLink fieldCode="AR" term="%22Ling%2C+Guangming%22">Ling, Guangming</searchLink><br /><searchLink fieldCode="AR" term="%22Wang%2C+Zhen%22">Wang, Zhen</searchLink><br /><searchLink fieldCode="AR" term="%22Gu%2C+Lin%22">Gu, Lin</searchLink><br /><searchLink fieldCode="AR" term="%22Yang%2C+Zhitong%22">Yang, Zhitong</searchLink><br /><searchLink fieldCode="AR" term="%22Liu%2C+Lydia+O%2E%22">Liu, Lydia O.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22ETS+Research+Report+Series%22"><i>ETS Research Report Series</i></searchLink>. Dec 2022.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: RDweb@ets.org; Web site: https://www.ets.org/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 22
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2022
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Test+Format%22">Test Format</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Wiseness%22">Test Wiseness</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Effect+Size%22">Effect Size</searchLink><br /><searchLink fieldCode="DE" term="%22College+Students%22">College Students</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Reaction+Time%22">Reaction Time</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22United+States%22">United States</searchLink><br /><searchLink fieldCode="DE" term="%22Canada%22">Canada</searchLink>
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 2330-8516
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Different variants of the selected-response (SR) item type have been developed for various reasons (i.e., simulating realistic situations, examining critical-thinking and/or problem-solving skills). Generally, the variants of SR item format are more complex than the traditional multiple-choice (MC) items, which may be more challenging to test takers and thus may discourage their test engagement on low-stakes assessments. Low test-taking effort has been shown to distort test scores and thereby diminish score validity. We used data collected from a large-scale assessment to investigate how variants of the SR item format may impact test properties and test engagement. Results show that the studied variants of SR item format were generally harder and more time consuming compared to the traditional MC item format, but they did not show negative impact on test-taking effort. However, item position had a dominating influence on nonresponse rates and rapid-guessing rates in a cumulative fashion, even though the effect sizes were relatively small in the studied data.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1366810
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1366810
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 22
    Subjects:
      – SubjectFull: Test Format
        Type: general
      – SubjectFull: Test Wiseness
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Effect Size
        Type: general
      – SubjectFull: College Students
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Reaction Time
        Type: general
      – SubjectFull: United States
        Type: general
      – SubjectFull: Canada
        Type: general
    Titles:
      – TitleFull: Influence of Selected-Response Format Variants on Test Characteristics and Test-Taking Effort: An Empirical Study. Research Report. ETS RR-22-01
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Guo, Hongwen
      – PersonEntity:
          Name:
            NameFull: Rios, Joseph A.
      – PersonEntity:
          Name:
            NameFull: Ling, Guangming
      – PersonEntity:
          Name:
            NameFull: Wang, Zhen
      – PersonEntity:
          Name:
            NameFull: Gu, Lin
      – PersonEntity:
          Name:
            NameFull: Yang, Zhitong
      – PersonEntity:
          Name:
            NameFull: Liu, Lydia O.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 12
              Type: published
              Y: 2022
          Identifiers:
            – Type: issn-electronic
              Value: 2330-8516
          Titles:
            – TitleFull: ETS Research Report Series
              Type: main
ResultId 1