A DQF-MQM Evaluation of Machine Translation in English-Arabic Customs Law.
Saved in:
| Title: | A DQF-MQM Evaluation of Machine Translation in English-Arabic Customs Law. |
|---|---|
| Authors: | Damseh, Karam1, Ghassemiazghandi, Mozhgan1 mozhgan.ghassemi@gmail.com |
| Source: | Journal of Language Teaching & Research. May2026, Vol. 17 Issue 3, p1019-1027. 9p. |
| Subject Terms: | *Translating & interpreting, Tariff laws, Machine translating |
| Reviews & Products: | Google Translate (Web resource) |
| Abstract: | This study presents a corpus-based diagnostic evaluation of Google Translate's performance in translating the Revised Kyoto Convention (RKC) from English into Arabic. Building on the "Adequacy-Fluency Tradeoff" observed in recent literature, researchers employ an explanatory sequential triangulation design to assess whether high-resource NMT models can satisfy the strict compliance requirements of Customs law. The methodology applies the COMET metric to a stratified test suite of the RKC corpus (N = 1,000 segments) to establish a quantitative baseline. To diagnose potential metric discordance, researchers conducted a targeted human evaluation: a Likert-scale assessment of the highest-scoring segments (n = 100) to verify the "performance ceiling," followed by a granular DQF-MQM error analysis of the lowest-scoring segments (n = 200) conducted by a panel of Experts. The analysis identifies a profound discordance within the RKC corpus: while the system achieved a high mean COMET score of 87.70, the detailed analysis of the "performance floor" revealed 1,579 discrete errors. The resulting penalty score (1,292.5) exceeded the professional acceptance threshold for this text type by a factor of 5.7. These findings indicate that, within this specific legal corpus, Google Translate exhibits "specification-blindness," systematically failing to disambiguate polysemous "Terms of Art" required for international trade compliance. The research concludes that for the RKC, automated metrics do not yet serve as a reliable proxy for legal review, and that a Human-in-the-Loop framework remains an absolute prerequisite for the deployment of Google Translate in Customs administration. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Language Teaching & Research is the property of Academy Publication Co., LTD and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Education Research Complete |
Be the first to leave a comment!
