View in EDS

Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence from a National Test

Saved in:
Bibliographic Details
Title: Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence from a National Test
Language: English
Authors: Lions, Séverin (ORCID 0000-0003-0697-7974), Dartnell, Pablo, Toledo, Gabriela, Godoy, María Inés, Córdova, Nora, Jiménez, Daniela, Lemarié, Julie
Source: Educational and Psychological Measurement. Oct 2023 83(5):861-884.
Availability: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
Peer Reviewed: Y
Page Count: 24
Publication Date: 2023
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Secondary Education
Descriptors: Multiple Choice Tests, Test Items, Item Analysis, Responses, Error of Measurement, High Stakes Tests, Standardized Tests, Language Processing, Interference (Learning), Academic Achievement, Foreign Countries, College Entrance Examinations, Accuracy, Difficulty Level, Comparative Analysis, Reading Processes, Secondary School Students
Geographic Terms: Chile
DOI: 10.1177/00131644221132335
ISSN: 0013-1644
1552-3888
Abstract: Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both correct response and distractors. Since multiple-choice tests' outcomes can be strikingly consequential and option position effects constitute a potential source of measurement error, these effects should be clarified. In this study, two experiments in which the position of correct response and distractors was carefully manipulated were performed within a Chilean national high-stakes standardized test, responded by 195,715 examinees. Results show small but clear and systematic effects of options position on examinees' responses in both experiments. They consistently indicate that a five-option item is slightly easier when the correct response is in A rather than E and when the most attractive distractor is after and far away from the correct response. They clarify and extend previous findings, showing that the appeal of all options is influenced by position. The existence and nature of a potential interference phenomenon between the options' processing are discussed, and implications for test development are considered.
Abstractor: As Provided
Entry Date: 2023
Accession Number: EJ1390939
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1390939
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence from a National Test
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Lions%2C+Séverin%22">Lions, Séverin</searchLink> (ORCID <externalLink term="https://orcid.org/0000-0003-0697-7974">0000-0003-0697-7974</externalLink>)<br /><searchLink fieldCode="AR" term="%22Dartnell%2C+Pablo%22">Dartnell, Pablo</searchLink><br /><searchLink fieldCode="AR" term="%22Toledo%2C+Gabriela%22">Toledo, Gabriela</searchLink><br /><searchLink fieldCode="AR" term="%22Godoy%2C+María+Inés%22">Godoy, María Inés</searchLink><br /><searchLink fieldCode="AR" term="%22Córdova%2C+Nora%22">Córdova, Nora</searchLink><br /><searchLink fieldCode="AR" term="%22Jiménez%2C+Daniela%22">Jiménez, Daniela</searchLink><br /><searchLink fieldCode="AR" term="%22Lemarié%2C+Julie%22">Lemarié, Julie</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Educational+and+Psychological+Measurement%22"><i>Educational and Psychological Measurement</i></searchLink>. Oct 2023 83(5):861-884.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: https://sagepub.com
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 24
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2023
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink><br /><searchLink fieldCode="EL" term="%22Secondary+Education%22">Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Responses%22">Responses</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22High+Stakes+Tests%22">High Stakes Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Standardized+Tests%22">Standardized Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Processing%22">Language Processing</searchLink><br /><searchLink fieldCode="DE" term="%22Interference+%28Learning%29%22">Interference (Learning)</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22College+Entrance+Examinations%22">College Entrance Examinations</searchLink><br /><searchLink fieldCode="DE" term="%22Accuracy%22">Accuracy</searchLink><br /><searchLink fieldCode="DE" term="%22Difficulty+Level%22">Difficulty Level</searchLink><br /><searchLink fieldCode="DE" term="%22Comparative+Analysis%22">Comparative Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Reading+Processes%22">Reading Processes</searchLink><br /><searchLink fieldCode="DE" term="%22Secondary+School+Students%22">Secondary School Students</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Chile%22">Chile</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/00131644221132335
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 0013-1644<br />1552-3888
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Even though the impact of the position of response options on answers to multiple-choice items has been investigated for decades, it remains debated. Research on this topic is inconclusive, perhaps because too few studies have obtained experimental data from large-sized samples in a real-world context and have manipulated the position of both correct response and distractors. Since multiple-choice tests' outcomes can be strikingly consequential and option position effects constitute a potential source of measurement error, these effects should be clarified. In this study, two experiments in which the position of correct response and distractors was carefully manipulated were performed within a Chilean national high-stakes standardized test, responded by 195,715 examinees. Results show small but clear and systematic effects of options position on examinees' responses in both experiments. They consistently indicate that a five-option item is slightly easier when the correct response is in A rather than E and when the most attractive distractor is after and far away from the correct response. They clarify and extend previous findings, showing that the appeal of all options is influenced by position. The existence and nature of a potential interference phenomenon between the options' processing are discussed, and implications for test development are considered.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2023
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1390939
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1390939
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/00131644221132335
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 24
        StartPage: 861
    Subjects:
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Responses
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
      – SubjectFull: High Stakes Tests
        Type: general
      – SubjectFull: Standardized Tests
        Type: general
      – SubjectFull: Language Processing
        Type: general
      – SubjectFull: Interference (Learning)
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: College Entrance Examinations
        Type: general
      – SubjectFull: Accuracy
        Type: general
      – SubjectFull: Difficulty Level
        Type: general
      – SubjectFull: Comparative Analysis
        Type: general
      – SubjectFull: Reading Processes
        Type: general
      – SubjectFull: Secondary School Students
        Type: general
      – SubjectFull: Chile
        Type: general
    Titles:
      – TitleFull: Position of Correct Option and Distractors Impacts Responses to Multiple-Choice Items: Evidence from a National Test
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Lions, Séverin
      – PersonEntity:
          Name:
            NameFull: Dartnell, Pablo
      – PersonEntity:
          Name:
            NameFull: Toledo, Gabriela
      – PersonEntity:
          Name:
            NameFull: Godoy, María Inés
      – PersonEntity:
          Name:
            NameFull: Córdova, Nora
      – PersonEntity:
          Name:
            NameFull: Jiménez, Daniela
      – PersonEntity:
          Name:
            NameFull: Lemarié, Julie
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Type: published
              Y: 2023
          Identifiers:
            – Type: issn-print
              Value: 0013-1644
            – Type: issn-electronic
              Value: 1552-3888
          Numbering:
            – Type: volume
              Value: 83
            – Type: issue
              Value: 5
          Titles:
            – TitleFull: Educational and Psychological Measurement
              Type: main
ResultId 1