Article published In: Developments in Cognitive Translation and Interpreting Studies
Edited by Kairong Xiao and Sandra L. Halverson
[Cognitive Linguistic Studies 8:2] 2021
► pp. 378–403
Dialogue interpreting, self-revision in translation and post-editing
Investigating post-editing effort
Does directionality play a role?
Published online: 22 November 2021
https://doi.org/10.1075/cogls.00083.sta
https://doi.org/10.1075/cogls.00083.sta
Abstract
The working environment of translators has changed significantly in recent decades, with post-editing (PE)
emerging as a new trend in the human translation workflow, particularly following the advent of neural machine translation (NMT)
and the improvement of the quality of the machine translation (MT) raw output especially at the level of fluency. In addition, the
directionality axiom is increasingly being questioned with translators working from and into their first language both in the
context of translation (Buchweitz, A., & Alves, F. (2006). Cognitive
adaptation in translation: an interface between language direction, time, and recursiveness in target text
production. Letras de
Hoje, 41(2), 241–272.; Pavlović, N., & Jensen, K. T. H. (2009). Eye
tracking translation directionality. In A. Pym & A. Perekrestenko (Eds.), Translation
research
projects 21 (Vol. 21, pp. 93–109). Tarragona: Intercultural Studies Group.; Fonseca, D. L., & Barbosa, N. (2015). Directionality
in translation: Investigating prototypical patterns in editing procedures. Translation &
Interpreting, 7(1), 111–125.; Hunziker Heeb, A. (2015). Does
professional translation into L2 involve more effort than into
L1? In Translation Process Research: Workshop
4 (n.p.). Las Palmas, Spain.; Ferreira, A. (2013). Direcionalidade
em tradução: O papel da subcompetência bilíngue em tarefas de tradução L1 e
L2. (Unpublished doctoral dissertation, Federal University of Minas Gerais). Retrieved from [URL], (2014). Analyzing
recursiveness patterns and retrospective protocols of professional translators in L1 and L2 translation
tasks. Translation and Interpreting
Studies, 9(1), 109–127. ; Ferreira, A., Schwieter, J. W., Gottardo, A., & Jones, J. (2016). Cognitive
effort in direct and inverse translation performance: insight from eye-tracking
technology. Cadernos de
Tradução, 36(3), 60–80. ; Feng, J. (2017). Comparing
cognitive load in L1 and L2 translation: Evidence from eye-tracking. Foreign Languages in China (《中国外语》), 781, 79–91.) and in the context of PE (Garcia, I. (2011). Translating
by post-editing: is it the way forward? Machine
Translation, 25(3), 217–237. ; Sánchez-Gijón, P., & Torres-Hostench, O. (2014). MT
Post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by
translation trainees. In S. O’Brien, M. Simard & L. Specia (Eds.), Proceedings
of the Third Workshop on Post-Editing Technology and Practice (WPTP-3)-AMTA
Workshop (pp. 15–19). Vancouver, Canada: Association for Machine Translation in the Americas.; Da Silva, I. A. L., Alves, F., Schmaltz, M., Pagano, A., Wong, D., Chao, L., Leal, A. L., Quaresma, P., Garcia, C., & Da Silva, G. E. (2017). Translation,
post-editing and directionality: A study of effort in the Chinese-Portuguese language
pair. In A. L. Jakobsen & B. Mesa-Lao (Eds.), Translation
in transition. Between cognition, computing and
technology (pp. 108–134). Amsterdam: John Benjamins. ; Toledo Báez, M. C. (2018). Machine
translation and post-editing: Impact of training and directionality on quality and
productivity. Revista Tradumàtica. Technologies de la
Traducció, 161, 24–34. ). In this study we employ product- and
process-oriented approaches to investigate directionality in PE in the English-Greek language pair. In particular, we compare the
cognitive, temporal, and technical effort expended by translators for the full PE of NMT output in L1 (Greek) with the effort
required for the full PE of NMT output in L2 (English), while we also analyze the quality of the final translation product. Our
findings reveal that PE in L2, i.e., inverse PE, is less demanding than PE in L1, i.e., direct PE, in terms of the time and
keystrokes required, and the cognitive load exerted on translators. Finally, our research shows that directionality does not imply
differences in quality.
Article outline
- 1.Introduction
- 2.Methodology
- 2.1Participants
- 2.2STs and MT outputs
- 2.3Process and product analysis
- 3.Findings and discussion
- 3.1Process analysis: Measuring PE effort
- 3.1.1Temporal effort
- 3.1.2Technical effort
- 3.1.3Cognitive effort
- 3.2Product analysis
- 3.1Process analysis: Measuring PE effort
- 4.Conclusion
- Acknowledgements
- Notes
References
References (84)
Akbari, A., & Segers, W. (2017). Translation
difficulty: How to measure and what to measure. Lebende
Sprachen, 62(1), 3–29.
Alves, F., Pagano, A., Neumann, S., Steiner, E., & Hansen-Schirra, S. (2010). Translation
units and grammatical shifts: Towards an integration of product- and process-based translation
research. In G. M. Shreve & E. Angelone (Eds.), Translation
and
cognition (pp. 109–142). Amsterdam: John Benjamins.
Beeby, L. A. (2001). Direction
of translation (directionality). In M. Baker (Ed.), Routledge
encyclopedia of translation
studies (pp. 63–67). London: Routledge.
Blum-Kulka, S. (2000). Shifts
of cohesion and coherence in translation. In L. Venuti (Ed.), The
translation studies
reader (pp. 298–312). London: Routledge.
Buchweitz, A., & Alves, F. (2006). Cognitive
adaptation in translation: an interface between language direction, time, and recursiveness in target text
production. Letras de
Hoje, 41(2), 241–272.
Carl, M. (2012). Translog–II:
A program for recording user activity data for empirical reading and writing
research. In Proceedings of the 8th international conference on
language resources and evaluation
(LREC’12) (pp. 4108–4112). Istanbul: European Language Resources Association (ELRA).
Carl, M., & Buch-Kromann, M. (2010). Correlating
translation product and translation process data of professional and student
translators. In F. Yvon & V. Hansen (Eds.), Proceedings
of the 14th annual conference of the European Association for Machine
Translation (n.p.). Saint-Raphaël, France: European Association for Machine Translation.
Carl, M., Dragsted, B., Elming, J., Hardt, D., & Jakobsen, A. L. (2011). The
process of post-editing: a pilot study. Copenhagen Studies in
Language, 411, 131–142.
Castilho, S., Moorkens, J., Gaspari, F., Calixto, I., Tinsley, J., & Way, A. (2017a). Is
neural machine translation the new state of the art? The Prague Bulletin of Mathematical
Linguistics, 108(1), 109–120.
Castilho, S., Moorkens, J., Gaspari, F., Sennrich, R., Sosoni, V., Georgakopoulou, Y., Lohar, P., Way, A., Miceli Barone, A., & Gialama, M. (2017b). A
Comparative quality evaluation of PBSMT and NMT using professional
translators. In Proceedings of machine translation summit
XVI (pp. 116–131). Nagoya, Japan.
Cohen, J. (1960). A
coefficient of agreement for nominal scales. Educational and Psychological
Measurement, 20(1), 37–46.
Conklin, K., Pellicer-Sánchez, A., & Carrol, G. (2018). Eye-tracking:
A guide for applied linguistics
research. Cambridge: Cambridge University Press.
Da Silva, I. A. L., Alves, F., Schmaltz, M., Pagano, A., Wong, D., Chao, L., Leal, A. L., Quaresma, P., Garcia, C., & Da Silva, G. E. (2017). Translation,
post-editing and directionality: A study of effort in the Chinese-Portuguese language
pair. In A. L. Jakobsen & B. Mesa-Lao (Eds.), Translation
in transition. Between cognition, computing and
technology (pp. 108–134). Amsterdam: John Benjamins.
Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017). Identifying
the machine translation error types with the greatest impact on post-editing effort. Frontiers
in Psychology, 81, 1282.
De Almeida, G. (2013). Translating
the post-editor: An investigation of post-editing changes and correlations with professional experience across two romance
languages. (Unpublished doctoral dissertation, Dublin City University). Retrieved from [URL]
Densmer, L. (2014, August 19). Light
and full MT post-editing explained [RWS Moravia blog]. Retrieved from [URL]
Depraetere, I. (2010). What
counts as useful advice in a university post-editing training context? Report on a case
study. In F. Yvon & V. Hansen (Eds.), Proceedings
of the 14th annual conference of the European Association for Machine
Translation (n.p.). Saint-Raphaël, France: European Association for Machine Translation.
Do Carmo, F. (2019). Edit
distances do not describe editing, but they can be useful for translation process
research. In M. Carl & S. Hansen-Schirra (Eds.), Proceedings
of the 2nd MEMENTO workshop on Modelling Parameters of Cognitive Effort in Translation
Production (pp. 1–2). Dublin, Ireland: European Association for Machine Translation.
Dodds, J. M. (1999). Friends,
false friends and foes or back to basics in L1 to L2
translation. In G. Anderman & M. Rogers (Eds.), Word,
text, translation: Liber Amicorum for Peter
Newmark (pp. 56–65). Clevedon: Multilingual.
Doherty, S., & Kenny, D. (2014). The
design and evaluation of a statistical machine translation syllabus for translation
students. The Interpreter and Translator
Trainer, 8(2), 295–315.
Doherty, S., Kenny, D., & Way, A. (2012). Taking
statistical machine translation to the student
translator. In Proceedings of the tenth biennial conference of the
Association for Machine Translation in the Americas (n.p.). San Diego, USA: Association for Machine Translation in the Americas.
Doherty, S., O’Brien, S., & Carl, M. (2010). Eye
tracking as an automatic MT evaluation technique. Machine
Translation, 24(1), 1–13.
Elming, J., Balling, L. W., & Carl, M. (2014). Investigating
user behaviour in post-editing and translation using the CASMACAT
workbench. In S. O’Brien, L. Winther Balling, M. Carl, M. Simard & L. Specia (Eds.), Post-editing
of machine
translation (pp.147–169). Newcastle: Cambridge Scholars Publishing.
Feng, J. (2017). Comparing
cognitive load in L1 and L2 translation: Evidence from eye-tracking. Foreign Languages in China (《中国外语》), 781, 79–91.
Ferreira, A. (2013). Direcionalidade
em tradução: O papel da subcompetência bilíngue em tarefas de tradução L1 e
L2. (Unpublished doctoral dissertation, Federal University of Minas Gerais). Retrieved from [URL]
(2014). Analyzing
recursiveness patterns and retrospective protocols of professional translators in L1 and L2 translation
tasks. Translation and Interpreting
Studies, 9(1), 109–127.
Ferreira, A., & Schwieter, J. W. (2017). Directionality
in translation. In J. W. Schwieter & A. Ferreira (Eds.), The
handbook of translation and
cognition (pp. 90–105). Hoboken, NJ: Wiley-Blackwell.
Ferreira, A., Schwieter, J. W., Gottardo, A., & Jones, J. (2016). Cognitive
effort in direct and inverse translation performance: insight from eye-tracking
technology. Cadernos de
Tradução, 36(3), 60–80.
Firbas, J. (1992). Functional
sentence perspective in written and spoken communication (Studies in English
Language). Cambridge: Cambridge University Press.
Flanagan, M., & Christensen, T. P. (2014). Testing
post-editing guidelines: how translation trainees interpret them and how to tailor them for translator training
purposes. The Interpreter and Translator
Trainer, 8(2), 257–275.
Fonseca, D. L., & Barbosa, N. (2015). Directionality
in translation: Investigating prototypical patterns in editing procedures. Translation &
Interpreting, 7(1), 111–125.
Fridman, L., Langhans, P., Lee, J., & Reimer, B. (2016). Driver
gaze region estimation without use of eye movement. IEEE Intelligent
Systems, 31(3), 49–56.
Garcia, I. (2011). Translating
by post-editing: is it the way forward? Machine
Translation, 25(3), 217–237.
Gouadec, D. (2007). Translator
as a profession. Amsterdam: John Benjamins.
Grosman, M., Kadrič, M., Kovačič, I., & Snell-Hornby, M. (Eds.). (2000). Translation
into non-mother tongues in professional practice and
training. Tübingen: Stauffenburg.
Guerberof, A., & Moorkens, J. (2019). Machine
translation and post-editing training as part of a master’s programme. Jostrans: The Journal of
Specialised
Translation, 311, 217–238.
Halverson, S. L. (2017). Multimethod
approaches. In J. W. Schwieter & A. Ferreira (Eds.), The
handbook of translation and
cognition (pp. 195–212). Hoboken, NJ: Wiley-Blackwell.
Hu, K., & Cadwell, P. (2016). A
comparative study of post-editing guidelines. Baltic Journal of Modern
Computing, 4(2), 346–353.
Hunziker Heeb, A. (2015). Does
professional translation into L2 involve more effort than into
L1? In Translation Process Research: Workshop
4 (n.p.). Las Palmas, Spain.
Hvelplund, K. T. (2011). Allocation
of cognitive resources in translation: An eye-tracking and key-logging study. (Doctoral
dissertation, Copenhagen Business School). Frederiksberg: Samfundslitteratur.
Jia, Y., Carl, M., & Wang, X. (2019). Post-editing
neural machine translation versus phrase-based machine translation for English-Chinese. Machine
Translation, 33(1–2), 9–29.
Kenny, D., & Doherty, S. (2014). Statistical
machine translation in the translation curriculum: Overcoming obstacles and empowering
translators. The Interpreter and Translator
Trainer, 8(2), 276–294.
Koponen, M. (2012). Comparing
human perceptions of post-editing effort with post-editing
operations. In Proceedings of the seventh workshop on statistical
machine
translation (pp. 181–190). Montréal, Canada: Association for Computational Linguistics.
(2015). How
to teach machine translation post-editing? Experiences from a post-editing
course. In S. O’Brien & M. Simard (Eds.), Proceedings
of the 4th workshop on post-editing technology and practice
(WPTP4) (pp. 2–15). Miami, FA: Association for Machine Translation in the Americas.
Koponen, M., & Salmi, L. (2017). Post-editing
quality: Analysing the correctness and necessity of post-editor corrections. Linguistica
Antverpiensia, New Series: Themes in Translation
Studies, 161, 137–148.
Koponen, M., Salmi, L., & Nikulin, M. (2019). A
product and process analysis of post-editor corrections on neural, statistical and rule-based machine translation
output. Machine
Translation, 331, 61–90.
Krings, Η. (2001). Repairing
texts: Empirical investigations of machine translation post-editing processes. Kent, OH: Kent State University Press.
Lacruz, I., & Shreve, G. M. (2014). Pauses
and cognitive effort in post-editing. In S. O’Brien, L. Winther Balling, M. Carl, M. Simard, & L. Specia (Eds.), Post-editing
of machine
translation (pp. 246–272). Newcastle: Cambridge Scholars Publishing.
Lavie, A., & Agarwal, A. (2007). METEOR:
An Automatic Metric for MT Evaluation with High Levels of Correlation with Human
Judgments. In Proceedings of the Second Workshop on Statistical
Machine Translation. Association for Computational
Linguistics (pp. 228–231). Prague, Czech Republic: Association for Computational Linguistics.
Massardo, I., van der Meer, J., O’Brien, S., Hollowood, F., Aranberri, N., & Drescher, K. (2016). MT
post-editing guidelines. The Netherlands: TAUS Signature Editions.
Mesa-Lao, B. (2012). The
next generation translator’s workbench: post-editing in CASMACAT v.
1.0. In Proceedings of the 34th Translating and the Computer
Conference (n.p.). London, UK: ASLIB
(2014). Gaze
behaviour on source texts: An exploratory study comparing translation and
post-editing. In S. O’Brien, L. Winther Balling, M. Carl, M. Simard, & L. Specia (Eds.), Post-editing
of machine
translation (pp. 219–245). Newcastle: Cambridge Scholars Publishing.
Monti, J., Seretan, V., Corpas-Pastor, G., & Mitkov, R. (2018). Multiword
units in machine translation and translation technology. In R. Mitkov, J. Monti, G. C. Pastor & V. Seretan (Eds.), Multiword
units in machine translation and translation
technology (pp. 1–40). Amsterdam: John Benjamins.
Moorkens, J., & O’Brien, S. (2015). Post-editing
evaluations: trade-offs between novice and professional
participants. In Proceedings of the 18th Annual Conference of the
European Association for Machine
Translation (pp. 75–81). Antalya, Turkey: European Association for Machine Translation.
Moorkens, J., O’Brien, S., da Silva, I. A. L., Fonseca, N., & Alves, F. (2015). Correlations
of perceived post-editing effort with measurements of actual effort. Machine
Translation, 291, 267–284.
Mráček, D. (2018). Inverse
translation: the more challenging direction. Linguistica
Pragensia, 28(2), 202–221.
Nitzke, J. (2019). Problem
solving activities in post-editing and translation from scratch: A multi-method
study. Berlin: Language Science Press.
O’Brien, S. (2002). Teaching
post-editing: A proposal for course content. In Proceedings of the
6th EAMT Workshop: Teaching machine
translation (pp. 99–106). Manchester, UK: European Association for Machine Translation.
(2009). Eye
tracking in translation process research: methodological challenges and solutions. Methodology,
technology and innovation in translation process
research, 381, 251–266.
(2010). Introduction
to post-editing: Who, what, how and where to next. In The Ninth
Conference of the Association for Machine Translation in the Americas. Denver, Colorado: Association for Machine Translation in the Americas.
O’Brien, S., & Simard, M. (2014). Introduction
to special issue on post-editing. Machine
Translation, 28(3), 159–164.
Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002). BLEU:
a method for automatic evaluation of machine
translation. In Proceedings of the 40th annual meeting on association
for computational
linguistics (311–318). Philadelphia, Pennsylvania, USA: Association for Computational Linguistics.
Pavlović, N. (2007). Directionality
in Collaborative Translation Processes. A Study of Novice Translators. (Unpublished doctoral
dissertation, Universitat Rovira i Virgili). Retrieved
from [URL]
Pavlović, N., & Jensen, K. T. H. (2009). Eye
tracking translation directionality. In A. Pym & A. Perekrestenko (Eds.), Translation
research
projects 21 (Vol. 21, pp. 93–109). Tarragona: Intercultural Studies Group.
Pokorn, N. (2005). Challenging
the traditional axioms: Translation into a non-mother
tongue. Amsterdam: John Benjamins.
Popovic, M., Arčan, M., & Lommel, A. (2016). Potential
and limits of using post-edits as reference translations for mt evaluation. Baltic Journal of
Modern
Computing 2 (Vol. 4, No. 2, pp. 218–229). Riga, Latvia.
Raptis, S., & Giagkou, M. (2016). From
capturing to generating human behavior: closing the interaction loop at the hubic
lab. In Proceedings of the 20th panhellenic conference on informatics
(PCI) (pp. 1–6). Partas, Greece: ACM Digital Library, International Conference Proceedings Series.
Sánchez-Gijón, P., & Torres-Hostench, O. (2014). MT
Post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output post-edited by
translation trainees. In S. O’Brien, M. Simard & L. Specia (Eds.), Proceedings
of the Third Workshop on Post-Editing Technology and Practice (WPTP-3)-AMTA
Workshop (pp. 15–19). Vancouver, Canada: Association for Machine Translation in the Americas.
Schmitt, P. A. (1998). Marktsituation
der Übersetzer. In M. Snell-Hornby, H. G. Hönig, P. Kussmaul & P. A. Schmitt (Eds.), Handbuch
translation (pp. 5–13). Tübingen: Stauffenburg.
Sharmin, S., Spakov, O., Räihä, K. J., & Jakobsen, A. L. (2008). Where
on the screen do translation students look while translating, and for how long? Copenhagen
studies in
language, 361, 31–51.
Shreve, G., & Angelone, Ε. (Εds.). (2010). Translation
and cognition. Amsterdam: John Benjamins.
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., & Makhoul, J. (2006). A
study of translation edit rate with targeted human
annotation. In Proceedings of the 7th Conference of the Association
for Machine Translation in the
Americas (Vol. 200. No. 6, pp. 223–231). Cambridge, Massachusetts, USA: The Association for Machine Translation in the Americas.
Talaván, N., & Rodríguez-Arancón, P. (2014). The
use of reverse subtitling as an online collaborative language learning tool. The Interpreter
and Translator
Trainer, 8(1), 84–101.
Toledo Báez, M. C. (2018). Machine
translation and post-editing: Impact of training and directionality on quality and
productivity. Revista Tradumàtica. Technologies de la
Traducció, 161, 24–34.
Vidal, E. (1997). Finite-state
speech-to-speech translation. In 1997 IEEE International Conference
on Acoustics, Speech, and Signal
Processing (Vol. 11, pp. 111–114). Munich, Germany: IEEE Computer Society.
Vieira, L. N., Alonso, E., & Bywood, L. (2019). Introduction:
Post-editing in practice – Process, product and networks. Jostrans: The Journal of Specialised
Translation, 311, 2–13.
Wang, B. (2011). Translation
practices and the issue of directionality in
China. Meta, 56(4), 896–914.
Whyatt, B. (2019). In
search of directionality effects in the translation process and in the end
product. Translation, Cognition and
Behaviour, 2(1), 79–100.
Xiao, K. (2013). Cognitive
approaches to translation process: Current trends, challenges and future
development. In R. A. Awang, A. A. G. Ghani & L. A. Ayob (Eds.), Translator
and Interpreter Education and Training: Innovation, Assessment and
Recognition (pp.432–448). Kuala Lumpur: Malaysian Translators Association.
Cited by (3)
Cited by three other publications
Robert, Isabelle S.
This list is based on CrossRef data as of 8 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.
