Cover not available

Article published In: Translation and Interpreting Studies
Vol. 19:2 (2024) ► pp.277–301

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

MT error detection and correction by Chinese language learners

ORCiD logo with link | Dublin City UniversityDublin
ORCiD logo with link | University College DublinDublin
ORCiD logo with link | Dublin City UniversityDublin
Published online: 22 March 2024
DOI logo with link
https://doi.org/10.1075/tis.22092.zha
Abstract
This article analyzes texts that have been generated by machine translation (MT) and post-edited by native English-speaking trainee translators (English>Chinese) who are also Chinese language learners enrolled in a four-year undergraduate translation program. The project examines the work product of trainee translators to categorize 122 errors that are (un)noticed and (un)corrected by them. MT errors in the Accuracy category were best identified and corrected, followed by those in the Lexicon and Fluency categories. Trainee translators who were advanced language learners outperformed the intermediate group in MT error detection and correction, especially in the Lexicon category. This study sheds light upon the use of raw MT output as meaningful input for trainee translators who are in the process of learning Chinese. Its findings provide information regarding the type of exercises needed in language learning and translation training for students with different levels of language proficiency.
Article outline
  • Introduction
  • Pedagogical uses of post-editing
  • Research design
    • Typology adopted in the current study
    • Participants
    • Choice of source text (ST)
    • Research procedure
  • Data analysis
    • Quantitative analysis
    • The factor of language proficiency
    • Text level
    • Qualitative analysis
    • Fluency
    • Accuracy
    • Lexicon
  • Discussion
  • Limitations
  • Conclusion
  • Acknowledgements
  • Notes
  • References
References (56)
References
Baayen, R. H., D. J. Davidson, and D. M. Bates. 2008. “Mixed-effects modeling with crossed random effects for subjects and items.” Journal of Memory and Language 59(4): 390–412. Google Scholar logo with link to Google Scholar
Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting linear mixed-effects models using Lme4.” Journal of Statistical Software 671: 1–48. Google Scholar logo with link to Google Scholar
Bellassen, J., and Pengpeng Zhang. 1997. A Key to Chinese Speech and Writing. Beijing: Sinolingua.Google Scholar logo with link to Google Scholar
Bowker, Lynne, and Jairo Buitrago Ciro. 2019. Machine Translation and Global Research: Towards Improved Machine Translation Literacy in the Scholarly Community. Bingley, UK: Emerald. Google Scholar logo with link to Google Scholar
Chang, Chia-chien, and Masaru Yamada. 2021. “Translation tasks for learning collocations: Effects of machine translation plus post-editing and sight translation.” English Teaching & Learning 45(1): 27–44. Google Scholar logo with link to Google Scholar
Chang, Pi-Chuan, Michel Galley, and Christopher D. Manning. 2008. “Optimizing Chinese word segmentation for machine translation performance.” In Proceedings of the Third Workshop on Statistical Machine Translation – StatMT ’08, 224–32. Columbus, OH: Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Chen, Hsin-hsi. 1994. “Contextual analysis of Chinese sentences with punctuation marks.” Literary and Linguistic Computing 9(4): 281–89. Google Scholar logo with link to Google Scholar
Clifford, Joan, Lisa Merschel, and Joan Munné. 2013. “Surveying the landscape – What is the role of machine translation in language learning.” @tic. Revista d’innovació Educativa 101: 108–21.Google Scholar logo with link to Google Scholar
Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. New York: Routledge. Google Scholar logo with link to Google Scholar
Correa, Maite. 2014. “Leaving the “peer” out of peer-editing: Online translators as a pedagogical tool in the Spanish as a second language classroom.” Latin American Journal of Content and Language Integrated Learning 7(1): 1–20. Google Scholar logo with link to Google Scholar
Cui, Songren, and Kuo-ming Sung. 2022. “Directional complement.” In A Reference Grammar for Teaching Chinese: Syntax and Discourse, ed. by Songren Cui and Kuo-ming Sung, 447–89. Singapore: Springer. Google Scholar logo with link to Google Scholar
Depraetere, Ilse. 2010. “What counts as useful advice in a university post-editing training context? Report on a case study.” In Proceedings of the 14th Annual Conference of the European Association for Machine Translation. St Raphaël, France. [URL]
Dorst, Aletta G., Susana Valdez, and Heather Bouman. 2022. “Machine translation in the multilingual classroom: How, when and why do humanities students at a Dutch University use machine translation?’ Translation and Translanguaging in Multilingual Contexts 8(1): 49–66. Google Scholar logo with link to Google Scholar
ELIS Research. 2023. “European Language Industry Survey 2023: Trends, Expectations and Concerns of the European Language Industry.” 10. ELIS. ELIA, EMT, EUATC, FIT Europe, GALA, LIND, Women in Localization. [URL]
Fonseca, Norma, and Fabio Alves. 2016. “Assessing complexity and difficulty levels of machine-translated texts.” Letras & Letras 32(1): 306–337. Google Scholar logo with link to Google Scholar
García, Ignacio, and María Isabel Pena. 2011. “Machine translation-assisted language learning: Writing for beginners.” Computer Assisted Language Learning 24(5): 471–87. Google Scholar logo with link to Google Scholar
Groves, Mike, and Klaus Mundt. 2021. “A ghostwriter in the machine? Attitudes of academic staff towards machine translation use in internationalised higher education.” Journal of English for Academic Purposes 501: article 100957. Google Scholar logo with link to Google Scholar
Guerberof-Arenas, Ana, and Joss Moorkens. 2019. “Machine translation and post-editing training as part of a master’s programme.” The Journal of Specialised Translation 311: 217–38.Google Scholar logo with link to Google Scholar
Hu, Ke, and Patrick Cadwell. 2016. “A comparative study of post-editing guidelines.” Baltic Journal of Modern Computing 4 (2): 346–53.Google Scholar logo with link to Google Scholar
ISO 18587. 2017. Translation Services – Post-Editing of Machine Translation Output – Requirements. Geneva: ISO.Google Scholar logo with link to Google Scholar
Jensen, Kristian T. H. 2009. “Indicators of text complexity.” In Behind the Mind: Methods, Models and Results in Translation Process Research, ed. by Susanne Göpferich, Arnt Lykke Jakobsen, and Inger M. Mees, 61–80. Copenhagen: Samfundslitteratur.Google Scholar logo with link to Google Scholar
Jia, Yanfang, Michael Carl, and Xiangling Wang. 2019a. “How does the post-editing of neural machine translation compare with from-scratch translation.” The Journal of Specialised Translation 311: 60–86.Google Scholar logo with link to Google Scholar
. 2019b. “Post-editing neural machine translation versus phrase-based machine translation for English–Chinese.” Machine Translation 33(1–2): 9–29. Google Scholar logo with link to Google Scholar
Jia, Yanfang, and Binghan Zheng. 2022. “The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study.” Across Languages and Cultures 23(1): 36–55. Google Scholar logo with link to Google Scholar
Klimova, Blanka, Marcel Pikhart, Alice Delorme Benites, Caroline Lehr, and Christina Sanchez-Stockhammer. 2023. “Neural machine translation in foreign language teaching and learning: A systematic review.” Education and Information Technologies 281: 663–682. Google Scholar logo with link to Google Scholar
Läubli, Samuel, Chantal Amrhein, Patrick Düggelin, Beatriz Gonzalez, Alena Zwahlen, and Martin Volk. 2019. “Post-editing productivity with neural machine translation: An empirical assessment of speed and quality in the banking and finance domain.” arXiv. [URL]
Li, Xiaoshi. 2010. “Sociolinguistic variation in the speech of learners of Chinese as a second language.” Language Learning 60(2): 366–408. Google Scholar logo with link to Google Scholar
Liu, Baolin, Zhongning Wang, and Zhixing Jin. 2010. “The effects of punctuations in Chinese sentence comprehension: An ERP study.” Journal of Neurolinguistics 23(1): 66–80. Google Scholar logo with link to Google Scholar
Lommel, Arle. 2018. “Metrics for translation quality assessment: A case for standardising error typologies.” In Translation Quality Assessment: From Principles to Practice, ed. by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 109–27. Cham: Springer International. Google Scholar logo with link to Google Scholar
Mellinger, Christopher D. 2017. “Translators and machine translation: Knowledge and skills gaps in translator pedagogy.” The Interpreter and Translator Trainer 11(4): 280–93. Google Scholar logo with link to Google Scholar
Mellinger, Christopher D., and Thomas A. Hanson. 2017. Quantitative Research Methods in Translation and Interpreting Studies. London: Routledge.Google Scholar logo with link to Google Scholar
Miyata, Rei, and Atsushi Fujita. 2021. “Understanding pre-editing for black-box neural machine translation.” In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 1539–50. EACL. Google Scholar logo with link to Google Scholar
Moorkens, Joss. 2018. “What to expect from neural machine translation: A practical in-class translation evaluation exercise.” The Interpreter and Translator Trainer 12(4): 375–87. Google Scholar logo with link to Google Scholar
Niño, Ana. 2008. “Evaluating the use of machine translation post-editing in the foreign language class.” Computer Assisted Language Learning 21(1): 29–49. Google Scholar logo with link to Google Scholar
. 2009. “Machine translation in foreign language learning: Language learners’ and tutors’ perceptions of its advantages and disadvantages.” ReCALL 21(2): 241–58. Google Scholar logo with link to Google Scholar
. 2020. “Exploring the use of online machine translation for independent language learning.” Research in Learning Technology 281: article 2402. Google Scholar logo with link to Google Scholar
O’Brien, Sharon. 2021. “Post-editing.” In Handbook of Translation Studies, vol. 51, ed. by Yves Gambier and Luc van Doorslaer, 178–84. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
. 2022. “How to deal with errors in machine translation: Post-editing.” In Machine Translation for Everyone: Empowering Users in the Age of Artificial Intelligence, ed. by Dorothy Kenny, 105–20. Berlin: Language Science Press.Google Scholar logo with link to Google Scholar
Plitt, Mirko, and François Masselot. 2010. “A productivity test of statistical machine translation post-editing in a typical localisation context.” The Prague Bulletin of Mathematical Linguistics 931: 7–16. Google Scholar logo with link to Google Scholar
R Core Team. 2023. “R: The R Project for Statistical Computing.” 2023. [URL]
Rico Pérez, Celia. 2024. “Re-thinking machine translation post-editing guidelines.” The Journal of Specialised Translation 411: 26–47. Google Scholar logo with link to Google Scholar
Rico, Celia, and Diana González Pastor. 2022. “The role of machine translation in translation education: A thematic analysis of translator educators’ beliefs.” Translation & Interpreting 14(1): 177–197. Google Scholar logo with link to Google Scholar
Sánchez-Gijón, Pilar, and Olga Torres-Hostench. 2014. “MT post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by translation trainees.” In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas, 5–19. Vancouver, Canada: AMTA.Google Scholar logo with link to Google Scholar
Screen, Ben. 2017. “Machine translation and Welsh: Analysing free statistical machine translation for the professional.” The Journal of Specialised Translation 281: 317–44.Google Scholar logo with link to Google Scholar
Somers, H. L. 2000. “Machine translation in the classroom.” In Computers and Translation: A Translator’s Guide, ed. by H. L. Sommers, 319–340. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Sun, Chaofen. 2006. Chinese: A Linguistic Introduction. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
TAUS, and CNGL. 2010. “Machine translation postediting guidelines.” [URL]
Toral, Antonio, and Andy Way. 2018. “What level of quality can neural machine translation attain on literary text?’ In Translation Quality Assessment: From Principles to Practice, ed. by Joss Moorkens, Sheila Castilho, Federico Gaspari, and Stephen Doherty, 263–87. Cham: Springer International. Google Scholar logo with link to Google Scholar
Vardaro, Jennifer, Moritz Schaeffer, and Silvia Hansen-Schirra. 2019. “Translation quality and error recognition in professional neural machine translation post-editing.” Informatics 6(3): article 41. Google Scholar logo with link to Google Scholar
Wang, Xiangling, Tingting Wang, Ricardo Muñoz Martín, and Yanfang Jia. 2021. “Investigating usability in postediting neural machine translation: Evidence from translation trainees’ self-perception and performance.” Across Languages and Cultures 22(1): 100–123. Google Scholar logo with link to Google Scholar
Wang, Yining, Long Zhou, Jiajun Zhang, and Chengqing Zong. 2017. “Word, subword or character? An empirical study of granularity in Chinese-English NMT.” In Machine Translation, ed. by Derek F. Wong and Deyi Xiong, 30–42. Singapore: Springer. Google Scholar logo with link to Google Scholar
Yamada, Masaru. 2019. “The impact of google neural machine translation on post-editing by student translators.” The Journal of Specialised Translation 311: 81–106.Google Scholar logo with link to Google Scholar
Yang, Jizhou, ed. 2016. Chinese Course. 3rd ed. Beijing: Beijing Language and Culture University Press.Google Scholar logo with link to Google Scholar
Zhang, Hong, and Olga Torres-Hostench. 2022. “Training in machine translation post-editing for foreign language students.” Language Learning & Technology 26(1): 1–17.Google Scholar logo with link to Google Scholar
Zhang, Qi, and Zhouxiang Lu. 2014. “The writing of Chinese characters by CFL learners: Can writing on Facebook and using machine translation help?’ Language Learning in Higher Education 4(2): 441–467. Google Scholar logo with link to Google Scholar
Zhen, Yu-Yao, Ya-Ning Wu, Guang-Ming Yu, and Chun-Ping Zheng. 2021. “A review study of the application of machine translation in education from 2011 to 2020.” In Proceedings of the 29th International Conference on Computers in Education, ed. by Maria Mercedes T. Rodrigo, Sridhar Iyer, and Antonija Mitrovic, 17–24. Bangkok, Thailand: Asia-Pacific Society for Computers in Education (APSCE).Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue