Is the Use of Multiple-Choice Items and a Holistically-Scored Paragraph Translation Task Fair? Examining a Large-Scale Translation Subtest
Saved in:
| Title: | Is the Use of Multiple-Choice Items and a Holistically-Scored Paragraph Translation Task Fair? Examining a Large-Scale Translation Subtest |
|---|---|
| Language: | English |
| Authors: | Zhiqiang Yang, Chengyuan Yu (ORCID |
| Source: | Asia Pacific Education Review. 2025 26(2):493-502. |
| Availability: | Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link.springer.com/ |
| Peer Reviewed: | Y |
| Page Count: | 10 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Higher Education Postsecondary Education |
| Descriptors: | Multiple Choice Tests, Test Items, Scoring, Translation, Language Tests, Second Language Instruction, English (Second Language), Foreign Countries, Test Bias, Art Education, Humanities Instruction, Social Sciences, STEM Education, Majors (Students), Objective Tests, Sex Fairness |
| Geographic Terms: | China |
| DOI: | 10.1007/s12564-024-09993-y |
| ISSN: | 1598-1037 1876-407X |
| Abstract: | This study investigated the test fairness of the translation section of a large-scale English test in China by examining its Differential Test Functioning (DTF) and Differential Item Functioning (DIF) across gender and major. Regarding DTF, the entire translation section exhibits partial strong measurement invariance across female and male test takers, while exhibiting full measurement invariance across test takers in (1) arts & humanities and social sciences (A&HSS) and (2) science, technology, engineering or mathematics (STEM) majors. No major-based DIF was detected in this study. Objective test items tend to favor male test takers, while direct translation test task was more favorable to females. Combining the DIF and DTF results, there may be a cancelation effect in our case. However, the effect size of DIF is either negligible or slight to moderate, indicating minimal impact on the overall fairness of the translation test task. This study further discusses the necessity of exploring the source of DIF and the importance of combining DIF and DTF for test fairness research. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1487618 |
| Database: | ERIC |
| Abstract: | This study investigated the test fairness of the translation section of a large-scale English test in China by examining its Differential Test Functioning (DTF) and Differential Item Functioning (DIF) across gender and major. Regarding DTF, the entire translation section exhibits partial strong measurement invariance across female and male test takers, while exhibiting full measurement invariance across test takers in (1) arts & humanities and social sciences (A&HSS) and (2) science, technology, engineering or mathematics (STEM) majors. No major-based DIF was detected in this study. Objective test items tend to favor male test takers, while direct translation test task was more favorable to females. Combining the DIF and DTF results, there may be a cancelation effect in our case. However, the effect size of DIF is either negligible or slight to moderate, indicating minimal impact on the overall fairness of the translation test task. This study further discusses the necessity of exploring the source of DIF and the importance of combining DIF and DTF for test fairness research. |
|---|---|
| ISSN: | 1598-1037 1876-407X |
| DOI: | 10.1007/s12564-024-09993-y |
