View in EDS

Is the Use of Multiple-Choice Items and a Holistically-Scored Paragraph Translation Task Fair? Examining a Large-Scale Translation Subtest

Saved in:
Bibliographic Details
Title: Is the Use of Multiple-Choice Items and a Holistically-Scored Paragraph Translation Task Fair? Examining a Large-Scale Translation Subtest
Language: English
Authors: Zhiqiang Yang, Chengyuan Yu (ORCID 0000-0001-8797-3902)
Source: Asia Pacific Education Review. 2025 26(2):493-502.
Availability: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
Peer Reviewed: Y
Page Count: 10
Publication Date: 2025
Document Type: Journal Articles
Reports - Research
Education Level: Higher Education
Postsecondary Education
Descriptors: Multiple Choice Tests, Test Items, Scoring, Translation, Language Tests, Second Language Instruction, English (Second Language), Foreign Countries, Test Bias, Art Education, Humanities Instruction, Social Sciences, STEM Education, Majors (Students), Objective Tests, Sex Fairness
Geographic Terms: China
DOI: 10.1007/s12564-024-09993-y
ISSN: 1598-1037
1876-407X
Abstract: This study investigated the test fairness of the translation section of a large-scale English test in China by examining its Differential Test Functioning (DTF) and Differential Item Functioning (DIF) across gender and major. Regarding DTF, the entire translation section exhibits partial strong measurement invariance across female and male test takers, while exhibiting full measurement invariance across test takers in (1) arts & humanities and social sciences (A&HSS) and (2) science, technology, engineering or mathematics (STEM) majors. No major-based DIF was detected in this study. Objective test items tend to favor male test takers, while direct translation test task was more favorable to females. Combining the DIF and DTF results, there may be a cancelation effect in our case. However, the effect size of DIF is either negligible or slight to moderate, indicating minimal impact on the overall fairness of the translation test task. This study further discusses the necessity of exploring the source of DIF and the importance of combining DIF and DTF for test fairness research.
Abstractor: As Provided
Entry Date: 2025
Accession Number: EJ1487618
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: EJ1487618
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Is the Use of Multiple-Choice Items and a Holistically-Scored Paragraph Translation Task Fair? Examining a Large-Scale Translation Subtest
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Zhiqiang+Yang%22">Zhiqiang Yang</searchLink><br /><searchLink fieldCode="AR" term="%22Chengyuan+Yu%22">Chengyuan Yu</searchLink> (ORCID <externalLink term="http://orcid.org/0000-0001-8797-3902">0000-0001-8797-3902</externalLink>)
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Asia+Pacific+Education+Review%22"><i>Asia Pacific Education Review</i></searchLink>. 2025 26(2):493-502.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: Y
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 10
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Journal Articles<br />Reports - Research
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Higher+Education%22">Higher Education</searchLink><br /><searchLink fieldCode="EL" term="%22Postsecondary+Education%22">Postsecondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Multiple+Choice+Tests%22">Multiple Choice Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Scoring%22">Scoring</searchLink><br /><searchLink fieldCode="DE" term="%22Translation%22">Translation</searchLink><br /><searchLink fieldCode="DE" term="%22Language+Tests%22">Language Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Second+Language+Instruction%22">Second Language Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22English+%28Second+Language%29%22">English (Second Language)</searchLink><br /><searchLink fieldCode="DE" term="%22Foreign+Countries%22">Foreign Countries</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Bias%22">Test Bias</searchLink><br /><searchLink fieldCode="DE" term="%22Art+Education%22">Art Education</searchLink><br /><searchLink fieldCode="DE" term="%22Humanities+Instruction%22">Humanities Instruction</searchLink><br /><searchLink fieldCode="DE" term="%22Social+Sciences%22">Social Sciences</searchLink><br /><searchLink fieldCode="DE" term="%22STEM+Education%22">STEM Education</searchLink><br /><searchLink fieldCode="DE" term="%22Majors+%28Students%29%22">Majors (Students)</searchLink><br /><searchLink fieldCode="DE" term="%22Objective+Tests%22">Objective Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Sex+Fairness%22">Sex Fairness</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22China%22">China</searchLink>
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1007/s12564-024-09993-y
– Name: ISSN
  Label: ISSN
  Group: ISSN
  Data: 1598-1037<br />1876-407X
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: This study investigated the test fairness of the translation section of a large-scale English test in China by examining its Differential Test Functioning (DTF) and Differential Item Functioning (DIF) across gender and major. Regarding DTF, the entire translation section exhibits partial strong measurement invariance across female and male test takers, while exhibiting full measurement invariance across test takers in (1) arts & humanities and social sciences (A&HSS) and (2) science, technology, engineering or mathematics (STEM) majors. No major-based DIF was detected in this study. Objective test items tend to favor male test takers, while direct translation test task was more favorable to females. Combining the DIF and DTF results, there may be a cancelation effect in our case. However, the effect size of DIF is either negligible or slight to moderate, indicating minimal impact on the overall fairness of the translation test task. This study further discusses the necessity of exploring the source of DIF and the importance of combining DIF and DTF for test fairness research.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2025
– Name: AN
  Label: Accession Number
  Group: ID
  Data: EJ1487618
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=EJ1487618
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1007/s12564-024-09993-y
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 10
        StartPage: 493
    Subjects:
      – SubjectFull: Multiple Choice Tests
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Scoring
        Type: general
      – SubjectFull: Translation
        Type: general
      – SubjectFull: Language Tests
        Type: general
      – SubjectFull: Second Language Instruction
        Type: general
      – SubjectFull: English (Second Language)
        Type: general
      – SubjectFull: Foreign Countries
        Type: general
      – SubjectFull: Test Bias
        Type: general
      – SubjectFull: Art Education
        Type: general
      – SubjectFull: Humanities Instruction
        Type: general
      – SubjectFull: Social Sciences
        Type: general
      – SubjectFull: STEM Education
        Type: general
      – SubjectFull: Majors (Students)
        Type: general
      – SubjectFull: Objective Tests
        Type: general
      – SubjectFull: Sex Fairness
        Type: general
      – SubjectFull: China
        Type: general
    Titles:
      – TitleFull: Is the Use of Multiple-Choice Items and a Holistically-Scored Paragraph Translation Task Fair? Examining a Large-Scale Translation Subtest
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Zhiqiang Yang
      – PersonEntity:
          Name:
            NameFull: Chengyuan Yu
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Type: published
              Y: 2025
          Identifiers:
            – Type: issn-print
              Value: 1598-1037
            – Type: issn-electronic
              Value: 1876-407X
          Numbering:
            – Type: volume
              Value: 26
            – Type: issue
              Value: 2
          Titles:
            – TitleFull: Asia Pacific Education Review
              Type: main
ResultId 1