Exploring user perspectives
Student evaluation of the evolution in neural machine translation from English to Korean and its implications
Published online: 25 April 2024
https://doi.org/10.1075/forum.00036.shi
https://doi.org/10.1075/forum.00036.shi
Abstract
Since the advent of Google NMT in 2016, human translators have been overwhelmed by the concern about being
replaced by machine translation. Although professional translators argue that the machine translation output is not refined enough
to surpass human translators, their claims are sometimes emotional and based on incorrect perceptions, without verification and
substantiation from user evaluation and specific quality evaluation data. Therefore, this study examines the evolution of NMT
output from English to Korean diachronically and provides specific user evaluation data that can verify and substantiate the
claims. Despite the steady improvement in NMT performance observed in recent years, it has been recognized that there is still a
significant gap that must be bridged for NMT to achieve parity with human translation. Nevertheless, as the collaboration between
NMT and human translators is expected to increase, it is essential to reinforce relevant systems, such as technological and legal
support systems.
Résumé
Depuis l’avènement de Google NMT en 2016, les traducteurs humains ont été accablés par la crainte d’être remplacés
par la traduction automatique. Bien que les traducteurs professionnels affirment que la traduction automatique n’est pas encore
suffisamment raffinée pour surpasser les capacités des traducteurs humains, leurs assertions sont parfois teintées d’émotions et
reposent sur des perceptions erronées, dépourvues de vérification ou de justification issues d’évaluations utilisateurs et de
données spécifiques concernant la qualité. C’est pourquoi cette étude examine l’évolution chronologique des résultats de la
traduction automatique de l’anglais vers le coréen et fournit des données spécifiques d’évaluation par les utilisateurs,
lesquelles peuvent confirmer et étayer ces affirmations. Malgré les améliorations constantes des performances de la NMT observées
ces dernières années, il est reconnu qu’un fossé significatif persiste avant que la NMT puisse atteindre la parité avec la
traduction humaine. Néanmoins, étant donné que la collaboration entre les NMT et les traducteurs humains devrait s’intensifier, il
est essentiel de renforcer les systèmes pertinents, tels que les systèmes de soutien technologique et juridique.
Article outline
- 1.Introduction
- 2.Fear spreading among human translators
- 3.NMT user evaluation process
- 3.1Overview of the evaluation
- 3.2Evaluators
- 3.3Procedures
- 3.4Source texts for NMT
- 4.Results of quantitative evaluation of NMT output
- 5.Results of qualitative evaluation of NMT output
- 6.Discussions and conclusion
- Notes
References
References (22)
Alonso, Elisa, and Lucas Nunes Vieira. 2017. “The
Translator’s Amanuensis 2020.” JoSTrans: The Journal of Specialized
Translation (28): 345–361.
Cadwell, Patrick, Sharon O’Brien, and Carlos S. C. Teixeira. 2018. “Resistance
and Accommodation: Factors for the (Non-) Adoption of Machine Translation among Professional
Translators.” Perspectives 26(3): 301–321.
Calude, Andreea S. 2004. “Machine Translation of Various
Text Genres.” The New Zealand Linguistic Society
Journal (46): 67–94.
Google Translate. Accessed August 16, 2019. [URL]
. Accessed September 1, 2021. [URL]
Ha, Seonyoung. 2017. “‘Human
vs AI’ Translation Battle...Human Beats the Machine. The Korea
Daily, February 21. [URL]
Johnson. 2017. “Why Translators Have
the Blues. A Profession under Pressure.” The
Economist, May 24. [URL]
Kang, Insun. 2018. “In
the Age of Artificial Intelligence and Robots...Translator/Casher will Disappear, Entertainer/Writer will
Survive.” Maeil Business News
Korea, April 2. [URL]
Marcus, G. 2018. The
deepest problem with deep learning. Medium. Available
at [URL]
Melby, Alan K. 2020. “Future of Machine
Translation.” In The Routledge Handbook of Translation and
Technology, edited by Minako O’Hagan. 419–436. London and New York: Routledge.
Moorkens, Joss. 2017. “Under
Pressure: Translation in Times of
Austerity.” Perspectives 25 (3): 464–477.
O’Thomas, Mark. 2017. “Humanum
ex Machina: Translation in the Post-global, Posthuman
World.” Target 29(2): 284–300.
Papago. Accessed August 16, 2019. [URL]
. Accessed September 1m 2021. [URL]
Park, Hyewon. 2022. “What
is ‘This Job’ That is not Taken away by AI and Robots? Chosun
Ilbo, April 15, [URL]
Pym, Anthony. 2020. “Translation,
Risk Management, and Cognition.” In The Routledge Handbook of
Translation and Cognition, edited by Arnt Lykke Jakobsen and Fabio Alves. 445–458. London and New York: Routledge.
Seo, Bo-Hyun, and Soonyoung Kim. 2018. “An
Analysis of Errors in Machine Translation.” Journal of Translation
Studies 19(1): 99–116.
Tonja, Atnafu Lambebo, Olga Kolesnikova, Alexander Gelbukh, and Grigori Sidorov. 2023. “Low-Resource
Neural Machine Translation Improvement Using Source-Side Monolingual Data.” Applied
Sciences 13(2): 1–20.
Vilar, David, Jia Xu, Luis Fernando D’Haro, and Hermann Ney. 2006. “Error
analysis of Statistical Machine Translation Output.” The 5th International Conference on
Language Resources and Evaluation, Genoa, Italy
Cited by (1)
Cited by one other publication
This list is based on CrossRef data as of 8 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.
