Cover not available

Article published In: Target
Vol. 31:1 (2019) ► pp.125–149

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Measuring the difficulty of text translation

The combination of text-focused and translator-oriented approaches

 | Shandong University of Finance and Economics
ORCiD logo with link | Durham University
 | Durham University
Published online: 8 January 2019
DOI logo with link
https://doi.org/10.1075/target.18036.zhe
Abstract
This paper explores the impact of text complexity on translators’ subjective perception of translation difficulty and on their cognitive load. Twenty-six MA translation students from a UK university were asked to translate three English texts with different complexity into Chinese. Their eye movements were recorded by an eye-tracker, and their cognitive load was self-assessed with a Likert scale before translation and NASA-TLX scales after translation. The results show that: (i) the intrinsic complexity measured by readability, word frequency and non-literalness was in line with the results received from informants’ subjective assessment of translation difficulty; (ii) moderate and positive correlations existed between most items in the self-assessments and the indicator (fixation and saccade durations) obtained by the eye-tracking measurements; and (iii) the informants’ cognitive load as indicated by fixation and saccade durations (but not for pupil size) increased significantly in two of the three texts along with the increase in source text complexity.
Article outline
  • 1.Introduction
  • 2.Measuring cognitive load
  • 3.Research design
    • 3.1Informants
    • 3.2Materials
    • 3.3Experimental settings
    • 3.4Experimental procedure
  • 4.Results
    • 4.1Quality assessment of eye-tracking data
    • 4.2Subjective measurements
      • 4.2.1Pre-translation rating
      • 4.2.2Post-translation rating
    • 4.3Physiological measurements
      • 4.3.1Fixation and saccade duration (FSD)
      • 4.3.2Pupil dilation
    • 4.4The link between subjective and physiological measurements
  • 5.Discussion
    • 5.1Quantitative text measurements
    • 5.2Physiological measurements
    • 5.3Subjective judgements
  • 6.Conclusion
  • Acknowledgements
  • Notes
  • References
References (49)
References
Akbari, Alireza, and Winibert Segers. 2017. “Translation Difficulty: How to Measure and What to Measure.” Lebende Sprachen 62 (1): 3–29. Google Scholar logo with link to Google Scholar
Ashby, Jane, Keith Rayner, and Charles Clifton. 2005. “Eye Movements of Highly Skilled and Average Readers: Differential Effects of Frequency and Predictability.” The Quarterly Journal of Experimental Psychology Section A 58 (6): 1065–1086. Google Scholar logo with link to Google Scholar
Cain, Brad. 2007. A Review of the Mental Workload Literature. Technical Report, Defence Research and Development Canada Toronto.Google Scholar logo with link to Google Scholar
Carpenter, Patricia A., and Marcel A. Just. 1989. “The Role of Working Memory in Language Comprehension.” In Complex Information Processing: The Impact of Herbert A. Simon, edited by David Klahr and Kenneth Kotovsk, 31–68. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers.Google Scholar logo with link to Google Scholar
Chaffin, Roger, Robin K. Morris, and Rachel E. Seely. 2001. “Learning New Word Meanings from Context: A Study of Eye Movements.” Journal of Experimental Psychology: Learning, Memory, and Cognition 27 (1): 225–235.Google Scholar logo with link to Google Scholar
Ehrlich, Susan F., and Keith Rayner. 1981. “Contextual Effects on Word Perception and Eye Movements during Reading.” Journal of Verbal Learning and Verbal Behavior 20 (6): 641–655. Google Scholar logo with link to Google Scholar
Evans, James D. 1996. Straightforward Statistics for the Behavioral Sciences. Pacific Grove, CA: Brooks/Cole Publishing Co.Google Scholar logo with link to Google Scholar
Gibbs Jr, Raymond W. 1990. “Comprehending Figurative Referential Descriptions.” Journal of Experimental Psychology: Learning, Memory and Cognition 16 (1): 56–66.Google Scholar logo with link to Google Scholar
Hart, Sandra G., and Lowell E. Staveland. 1988. “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research.” In Human Mental Workload, edited by Peter A. Hancock and Najmedin Meshkati, 139–183. Amsterdam: North-Holland. Google Scholar logo with link to Google Scholar
Hess, Eckhard H., and James M. Polt. 1964. “Pupil Size in Relation to Mental Activity during Simple Problem-Solving.” Science 143 (3611): 1190–1192. Google Scholar logo with link to Google Scholar
Hvelplund, Kristian Tangsgaard. 2011. Allocation of Cognitive Resources in Translation: An Eye-tracking and Key-logging Study. PhD diss. Copenhagen Business School.Google Scholar logo with link to Google Scholar
. 2014. “Eye Tracking and the Translation Process: Reflections on the Analysis and Interpretation of Eye-tracking Data.” In Minding Translation / Con la traducción en mente, edited by Ricardo Muñoz Martín, 201–224. San Vicente del Raspeig: Publicaciones de la Universidad de Alicante.Google Scholar logo with link to Google Scholar
Hyönä, Jukka, Jorma Tommola, and Anna-Mari Alaja. 1995. “Pupil Dilation as a Measure of Processing Load in Simultaneous Interpretation and Other Language Tasks.” The Quarterly Journal of Experimental Psychology 48 (3): 598–612. Google Scholar logo with link to Google Scholar
Inhoff, Albrecht Werner. 1984. “Two Stages of Word Processing during Eye Fixations in the Reading of Prose.” Journal of Verbal Learning & Verbal Behavior 23 (5): 612–624. Google Scholar logo with link to Google Scholar
Iqbal, Shamsi T., Xianjun Sam Zheng, and Brian P. Bailey. 2004. “Task-evoked Pupillary Response to Mental Workload in Human-Computer Interaction.” In CHI’04 Extended Abstracts on Human Factors in Computing Systems, edited by Elizabeth Dykstra-Erickson and Manfred Tscheligi, 1477–1480. Vienna.Google Scholar logo with link to Google Scholar
Irwin, David E. 2004. “Fixation Location and Fixation Duration as Indices of Cognitive Processing.” In The Interface of Language, Vision, and Action: Eye Movements and Visual World, edited by John Henderson and Fernanda Ferreira, 105–134. New York: Psychology Press.Google Scholar logo with link to Google Scholar
Jensen, Kristian T. H. 2009. “Indicators of Text Complexity.” In Behind the Mind: Methods, Models and Results in Translation Process Research, edited by Susanne Göpferich, Arnt L. Jakobsen, and Inger M. Mees, 61–80. Copenhagen: Samfundslitteratur.Google Scholar logo with link to Google Scholar
Johannsen, Gunnar. 1979. “Workload and Workload Measurement.” In Mental Workload: Its Theory and Measurement, edited by Neville Moray, 3–11. New York: Springer Science & Business Media. Google Scholar logo with link to Google Scholar
Just, Marcel A., and Patricia A. Carpenter. 1980. “A Theory of Reading: From Eye Fixations to Comprehension.” Psychological Review 87 (4): 329–354. Google Scholar logo with link to Google Scholar
. 1993. “The Intensity Dimension of Thought: Pupillometric Indices of Sentence Processing.” Canadian Journal of Experimental Psychology 47 (2): 310–339. Google Scholar logo with link to Google Scholar
LeBreton, James M., and Jenell L. Senter. 2008. “Answers to 20 Questions about Interrater Reliability and Interrater Agreement.” Organizational Research Methods 11 (4): 815–852. Google Scholar logo with link to Google Scholar
Liu, Minhua, and Yu-Hsien Chiu. 2011. “Assessing Source Material Difficulty for Consecutive Interpreting.” In Interpreting Chinese, Interpreting China, edited by Robin Setton, 135–156. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Mishra, Abhijit, Pushpak Bhattacharyya, and Michael Carl. 2013. “Automatically Predicting Sentence Translation Difficulty.” In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 346–351. Sofia.Google Scholar logo with link to Google Scholar
O’Brien, Sharon. 2006. “Eye-tracking and Translation Memory Matches.” Perspectives 14 (3): 185–205.Google Scholar logo with link to Google Scholar
Paas, Fred G. W. C. 1992. “Training Strategies for Attaining Transfer of Problem-Solving Skill in Statistics: A Cognitive-Load Approach.” Journal of Educational Psychology 84 (4): 429–434. Google Scholar logo with link to Google Scholar
Paas, Fred G. W. C., and Jeroen J. G. van Merriënboer. 1994a. “Instructional Control of Cognitive Load in the Training of Complex Cognitive Tasks.” Educational Psychology Review 6 (4): 351–371. Google Scholar logo with link to Google Scholar
. 1994b. “Variability of Worked Examples and Transfer of Geometrical Problem-Solving Skills: A Cognitive-Load Approach.” Journal of Educational Psychology 86 (1): 122–133. Google Scholar logo with link to Google Scholar
Pavlović, Nataša, and Kristian Jensen. 2009. “Eye Tracking Translation Directionality.” In Translation Research Projects 21, edited by Anthony Pym and Alexander Perekrestenko, 93–109. Tarragona: Intercultural Studies Group.Google Scholar logo with link to Google Scholar
Pomplun, Marc, and Sindhura Sunkara. 2003. “Pupil Dilation as an Indicator of Cognitive Workload in Human-Computer Interaction.” In Proceedings of the 10th International Conference on HCI (Vol. 31), 542–546. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.Google Scholar logo with link to Google Scholar
Rayner, Keith. 1998. “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin 124 (3): 372–422. Google Scholar logo with link to Google Scholar
Rayner, Keith, and Arnold D. Well. 1996. “Effects of Contextual Constraint on Eye Movements in Reading: A Further Examination.” Psychonomic Bulletin & Review 3 (4): 504–509. Google Scholar logo with link to Google Scholar
Rayner, Keith, and Susan A. Duffy. 1986. “Lexical Complexity and Fixation Times in Reading: Effects of Word Frequency, Verb Complexity, and Lexical Ambiguity.” Memory & Cognition 14 (3):191–201. Google Scholar logo with link to Google Scholar
Rayner, Keith, and Martin H. Fischer. 1996. “Mindless Reading Revisited: Eye Movements during Reading and Scanning Are Different.” Perception & Psychophysics 58 (5): 734–747. Google Scholar logo with link to Google Scholar
Rayner, Keith, and Gary E. Raney. 1996. “Eye Movement Control in Reading and Visual Search: Effects of Word Frequency.” Psychonomic Bulletin & Review 3 (2): 245–248. Google Scholar logo with link to Google Scholar
Rayner, Keith, Sara C. Sereno, and Gary E. Raney. 1996. “Eye Movement Control in Reading: A Comparison of Two Types of Models.” Journal of Experimental Psychology: Human Perception and Performance 22 (5): 1188–1200.Google Scholar logo with link to Google Scholar
Schotter, Elizabeth R., and Keith Rayner. 2012. “Eye Movements in Reading: Implications for Reading Subtitles.” In Eye Tracking in Audiovisual Translation, edited by Elisa Perego, 83–104. Roma: Aracne Editrice.Google Scholar logo with link to Google Scholar
Schultheis, Holger, and Anthony Jameson. 2004. “Assessing Cognitive Load in Adaptive Hypermedia Systems: Physiological and Behavioural Methods.” In Adaptive Hypermedia and Adaptive Web-based Systems, edited by Wolfgang Nejdl and Paul De Bra, 225–234. Berlin: Springer. Google Scholar logo with link to Google Scholar
Sereno, Sara C., Patrick J. O’Donnell, and Keith Rayner. 2006. “Eye Movements and Lexical Ambiguity Resolution: Investigating the Subordinate-Bias Effect.” Journal of Experimental Psychology: Human Perception and Performance 32 (2): 335–350.Google Scholar logo with link to Google Scholar
Sharmin, Selina, Oleg Spakov, Kari-Jouko Räihä, and Arnt L. Jakobsen. 2008. “Where on the Screen Do Translation Students Look While Translating, and for How Long?” In Looking at Eyes: Eye-Tracking Studies of Reading and Translation Processing, edited by Arnt L. Jakobsen, Susanne Göpferich, and Inger M. Mees, 31–51. Copenhagen: Samfundslitteratur.Google Scholar logo with link to Google Scholar
Staub, Adrian, and Keith Rayner. 2007. “Eye Movements and On-Line Comprehension Processes.” In The Oxford Handbook of Psycholinguistics, edited by M. Gareth Gaskell and Gerry Altmann, 327–342. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Sun, Sanjun. 2015. “Measuring Translation Difficulty: Theoretical and Methodological Considerations.” Across Languages and Cultures 16 (1): 29–54. Google Scholar logo with link to Google Scholar
Sun, Sanjun, and Gregory M. Shreve. 2014. “Measuring Translation Difficulty: An Empirical Study.” Target 26 (1): 98–127. Google Scholar logo with link to Google Scholar
Sweller, John. 2010. “Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load.” Educational Psychology Review 22 (2): 123–138. Google Scholar logo with link to Google Scholar
Sweller, John, Jeroen J. G. Van Merrienboer, and Fred G. W. C. Paas. 1998. “Cognitive Architecture and Instructional Design.” Educational Psychology Review 10 (3): 251–296. Google Scholar logo with link to Google Scholar
Vauras, Marja, Jukka Hyönä, and Pekka Niemi. 1992. “Comprehending Coherent and Incoherent Texts: Evidence from Eye Movement Patterns and Recall Performance.” Journal of Research in Reading 15 (1): 39–54. Google Scholar logo with link to Google Scholar
Williams, Rihana, and Robin Morris. 2004. “Eye Movements, Word Familiarity, and Vocabulary Acquisition.” European Journal of Cognitive Psychology 16 (1–2): 312–339. Google Scholar logo with link to Google Scholar
Zola, David. 1984. “Redundancy and Word Perception during Reading.” Perception & Psychophysics 36 (3): 277–284. Google Scholar logo with link to Google Scholar
Cited by (21)

Cited by 21 other publications

Chen, Shirong, Jia Feng, Michael Carl & Muhammad Zammad Aslam
2025. Exploring preparatory reading in bidirectional sight and written translation through clustering analysis of eye-tracking data. PLOS One 20:8 ► pp. e0329858 ff. DOI logo
Chmiel, Agnieszka, Nicoletta Spinolo, Paweł Korpal, Christian Olalla-Soler, Paulina Rozkrut, Marta Kajzer-Wietrzny & Serena Ghiselli
2025. The impact of remote interpreting settings on interpreter experience and performance. Translation and Interpreting Studies DOI logo
Jiang, Xinlei, Yue Jiang & Xiaopeng Zhang
2025. Assessing effects of source text complexity on L2 learners’ interpreting performance: a dependency-based approach. International Review of Applied Linguistics in Language Teaching 63:4 ► pp. 2717 ff. DOI logo
Yang, Wenting, Ricardo Muñoz Martín & Xiangling Wang
2025. Revisiting Grammatical Metaphor in Translation: New Insights from the English–Chinese Pair. Metaphor and Symbol 40:3 ► pp. 206 ff. DOI logo
Liu, Xiao & Ying Cui
2024. The impact of text type on Chinese-English translation effort: an investigation with reference to translation entropy. Humanities and Social Sciences Communications 11:1 DOI logo
Wu, Hao
2024. Neural networks application based on language features in the classification of complex English textbooks granularity. Intelligent Decision Technologies 18:2 ► pp. 855 ff. DOI logo
Yan, Hao, Yi Zhang, Yanqin Feng, Yang Li, Yueting Zhang, Yujun Lee, Maoqing Chen, Zijuan Shi, Yuan Liang, Yuqin Hei & Xu Duan
2024. Assessing mental demand in consecutive interpreting: Insights from an fNIRS study. Acta Psychologica 243 ► pp. 104132 ff. DOI logo
A.S. Mohammed, Tawffeek
2023. Translating Islamic Media Discourse from Arabic into English: An Analysis of Translation Process. In Translatology, Translation and Interpretation - Toward a New Scientific Endeavor, DOI logo
Chmiel, Agnieszka & Nicoletta Spinolo
2022. Testing the impact of remote interpreting settings on interpreter experience and performance. Translation, Cognition & Behavior 5:2 ► pp. 250 ff. DOI logo
Jia, Yanfang & Binghan Zheng
2022. The interaction effect between source text complexity and machine translation quality on the task difficulty of NMT post-editing from English to Chinese: A multi-method study. Across Languages and Cultures 23:1 ► pp. 36 ff. DOI logo
Kuang, Huolingxiao & Binghan Zheng
2022. How does interpreting performance correlate with note-taking process, note-taking product and note-reading process? An eye-tracking and pen-recording study. Across Languages and Cultures 23:2 ► pp. 167 ff. DOI logo
Liu, Yanmei & Binghan Zheng
2022. Comparability of difficulty levels of translation tasks in CET-6 parallel test forms: evidence from product and process-based data. The Interpreter and Translator Trainer 16:4 ► pp. 428 ff. DOI logo
Yang, Chengsong, Yimeng Wang & Na Fan
2022. Are parallel translation tasks parallel in difficulty? an eye-tracking study. Perspectives 30:4 ► pp. 711 ff. DOI logo
Yuan, Rongjie
2022. Material development for beginner student interpreters: how does text structure contribute to the difficulty of consecutive interpreting?. The Interpreter and Translator Trainer 16:1 ► pp. 58 ff. DOI logo
Zhou, Xiangyan, Xiangling Wang & Xiaodong Liu
2022. The impact of task complexity and translating self-efficacy belief on students’ translation performance: Evidence from process and product data. Frontiers in Psychology 13 DOI logo
Cui, Yixiao & Binghan Zheng
2021. Consultation behaviour with online resources in English-Chinese translation: an eye-tracking, screen-recording and retrospective study. Perspectives 29:5 ► pp. 740 ff. DOI logo
Gieshoff, Anne Catherine, Caroline Lehr & Andrea Hunziker Heeb
2021. Stress, cognitive, emotional and ergonomic demands in interpreting and translation. Cognitive Linguistic Studies 8:2 ► pp. 404 ff. DOI logo
Hu, Zhenming, Binghan Zheng & Xiangling Wang
2021. The impact of a metacognitive self-regulation inventory in translator self-training: a pre-post study with English-Chinese translation students. The Interpreter and Translator Trainer 15:4 ► pp. 430 ff. DOI logo
Vanroy, Bram, Orphée De Clercq, Arda Tezcan, Joke Daems & Lieve Macken
2021. Metrics of Syntactic Equivalence to Assess Translation Difficulty. In Explorations in Empirical Translation Process Research [Machine Translation: Technologies and Applications, 3], ► pp. 259 ff. DOI logo
Jiang, Xinlei & Yue Jiang
2020. Effect of dependency distance of source text on disfluencies in interpreting. Lingua 243 ► pp. 102873 ff. DOI logo
Su, Wenchao
2020. Introduction. In Eye-Tracking Processes and Styles in Sight Translation [New Frontiers in Translation Studies, ], ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 4 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue