Cover not available

Article published In: Translation, Cognition & Behavior
Vol. 1:1 (2018) ► pp.119–146

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

The effect of explanatory captions on the reception of foreign audiovisual products

A study drawing on eyetracking data and retrospective interviews

 | Durham University
 | Durham University
Published online: 2 March 2018
DOI logo with link
https://doi.org/10.1075/tcb.00006.zhe
Abstract
The present research triangulates questionnaire, retrospective interview and eyetracking data, aiming to investigate how Explanatory Captions (ECs) are received by different viewers with varied educational backgrounds, and whether or not the presence of ECs improves their understanding of the AV content. The results show that the provision of ECs, for a subtitled video in a foreign language, greatly increased positive cognitive effects on the viewers. Viewers tend to reduce time spent on viewing images, but invest additional processing effort on the ECs, although their allocation of processing effort on subtitles experienced little change. Furthermore, the eyetracking data suggest that most participants adopted a fixed reading pattern on subtitles and ECs when they appeared simultaneously, which could balance some of the negative impact of ECs on their viewing experience. The findings gained through this experimental research will provide some guidance and suggestions for subtitlers when preparing subtitles and ECs.
Article outline
  • 1.Introduction
  • 2.Theoretical framework
    • 2.1Relevance theory
    • 2.2Positive cognitive effect (PCEs) and processing effort
  • 3.Eye-tracking research on subtitle reception
  • 4.Methods
    • 4.1Participants
    • 4.2Stimuli
    • 4.3Eye movement recording and analysis
    • 4.4Questionnaire and retrospective interviews
    • 4.5Procedure
    • 4.6Quality assessment of eyetracking data
  • 5.Results
    • 5.1Positive Cognitive Effects (PCEs)
      • 5.1.1Self-evaluated PCEs
      • 5.1.2Tested PCEs
    • 5.2Processing efforts
      • 5.2.1Questionnaire and retrospection results
      • 5.2.2Eye-tracking results
  • 6.Discussion
  • 7.Conclusion
  • Notes
  • References
References (28)
References
Bisson, Marie-Josée, Walter J. B. Van Heuven, Kathy Conklin, and Richard J. Tunney. 2014. “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics 35 (2): 399–418. Google Scholar logo with link to Google Scholar
Caffrey, Colm. 2012. “Using an Eye–tracking Tool to Measure the Effects of Experimental Subtitling Procedures on Viewer Perception of Subtitled AV Content.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 135–156. Roma: Aracne.Google Scholar logo with link to Google Scholar
Dwyer, Tessa. 2015. “From Subtitles to SMS: Eye Tracking, Texting and Sherlock .” Refractory: A Journal of Entertainment Media, 251. Accessed 11 Nov 2017: [URL]
d’Ydewalle, Géry, and Ingrid Gielen. 1992. “Attention Allocation with Overlapping Sound, Image, and Text.” In Eye Movements and Visual Cognition: Scene Perception and Reading. Edited by K. Rayner, 415–427. New York: Springer. Google Scholar logo with link to Google Scholar
d’Ydewalle, Géry, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults while Reading Television Subtitles.” European Psychologist 12 (3): 196–205. Google Scholar logo with link to Google Scholar
d’Ydewalle, Géry, Johan van Rensbergen, and Joris Pollet. 1987. “Reading a Message When the Same Message is Available Auditorily in Another Language: The Case of Subtitling.” In Eye Movements: From Physiology to Cognition. Edited by J. K. O’Regan and A. Lévy—Schoen, 313–321. Amsterdam: North—Holland. Google Scholar logo with link to Google Scholar
de Linde, Zoe, and Neil M. Kay. 1999. The Semiotics of Subtitling. Manchester: St. Jerome.Google Scholar logo with link to Google Scholar
Desilla, Louisa. 2012. “Implicatures in Film: Construal and Functions in Bridget Jones Romantic Comedies.” Journal of Pragmatics 441: 30–53. Google Scholar logo with link to Google Scholar
Ghia, Elisa. 2012. “The Impact of Translation Strategies on Subtitle Reading.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 157–182. Roma: Aracne.Google Scholar logo with link to Google Scholar
Hegarty, Mary. 1992. “The Mechanics of Comprehension and Comprehension of Mechanics.” In Eye Movements and Visual Cognition: Scene Perception and Reading. Edited by K. Rayner, 428–443. New York: Springer. Google Scholar logo with link to Google Scholar
Hsiao, Chi-hua. 2014. The Cultural Translation of U.S. Television Programs and Movies: Subtitle Groups as Cultural Brokers in China. PhD Thesis, University of California, Los Angeles.Google Scholar logo with link to Google Scholar
Hvelplund, Kristian Tangsgaard. 2011. Allocation of Cognitive Resources in Translation: An Eye-tracking and Key-logging Study. PhD thesis, Copenhagen Business School.Google Scholar logo with link to Google Scholar
. 2014. “Eye Tracking and the Translation Process: Reflections on the Analysis and Interpretation of Eye-tracking Data. In Minding Translation. Edited by R. Muñoz, 201–223. MonTI, special issue 1. Alicante: Universidad de Alicante.Google Scholar logo with link to Google Scholar
Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew. 2013. “Measuring the Impact of Subtitles on Cognitive Load: Eye Tracking and Dynamic Audiovisual Texts.” Proceedings of the 2013 Conference on Eye Tracking South Africa, 62–66. New York: ACM. Google Scholar logo with link to Google Scholar
Lautenbacher, Olli Philippe. 2012. “From Still Pictures to Moving Pictures.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 135–155. Roma: Aracne.Google Scholar logo with link to Google Scholar
Moran, Siobhan. 2012. “The Effect of Linguistic Variation on Subtitle Reception.” In Eye-tracking in Audiovisual Translation. Edited by E. Perego, 183–222. Roma: Aracne.Google Scholar logo with link to Google Scholar
National Standard of the People’s Republic of China. 2005. Target Text Quality Requirements for Translation Services. Beijing: Standards Press of China.Google Scholar logo with link to Google Scholar
Perego, Elisa. 2012. Eye Tracking in Audiovisual Translation. Roma: Aracne.Google Scholar logo with link to Google Scholar
Rayner, Keith. 1998. “Eye Movements in Reading and Information Processing: 20 Years of Research.” Psychological Bulletin 124 (3): 372–422. Google Scholar logo with link to Google Scholar
Rayner, Keith, and Alexander Pollatsek. 1989. The Psychology of Reading. Englewood Cliffs, NJ: Prentice Hall.Google Scholar logo with link to Google Scholar
Sperber, Dan, and Deirdre Wilson. 1995. Relevance: Communication and Cognition. 2nd ed. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
Szarkowska, Agnieszka and Jan-Louis Kruger. 2015. “Subtitles on the Moving Image: An Overview of Eye Tracking Studies.” Refractory: a Journal of Entertainment Media, 251. Accessed 11 Nov 2017: [URL]
Tannen, Deborah. 1989. Talking Voices: Repetition, Dialogue and Imagery in Conversational Discourse. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Thomas, Jenny. 1995. Meaning in Interaction: An Introduction to Pragmatics. London: Longman.Google Scholar logo with link to Google Scholar
Wilson, Deirdre, and Dan Sperber. 2002. “Relevance Theory.” UCL Working papers in Linguistics 141: 249–287.Google Scholar logo with link to Google Scholar
Yus Ramos, Francisco B. 2008. “Alterations of Relevance in Cyber—Media.” Universitas Psychologica 7 (3): 629–642.Google Scholar logo with link to Google Scholar
Zheng, Binghan, and Xia Xiang. 2014. “The Impact of Cultural Background Knowledge in the Processing of Metaphorical Expressions.” Translation and Interpreting Studies 9 (1): 5–24. Google Scholar logo with link to Google Scholar
Cited by (7)

Cited by seven other publications

Jiang, Qihang & Stephen Doherty
2024. An eye-tracking study on the cognitive processing, reception, and perception of swearword translation in films. Translation Spaces 13:2 ► pp. 293 ff. DOI logo
Nettelbeck, Hugh Wilson
2024. Are subtitling norms evidence-based? A narrative comparison of guidelines and subtitling advice with empirical evidence. Perspectives ► pp. 1 ff. DOI logo
Hu, Bei
2023. Flowing to the reception side. In Translation Flows [Benjamins Translation Library, 163], ► pp. 183 ff. DOI logo
Wu, Zhiwei & Zhuojia Chen
2021. A systematic review of experimental research in audiovisual translation 1992–2020. Translation, Cognition & Behavior 4:2 ► pp. 281 ff. DOI logo
Abdel Latif, Muhammad M. M.
2020. Translation and Interpreting Assessment Research. In Translator and Interpreter Education Research [New Frontiers in Translation Studies, ], ► pp. 61 ff. DOI logo
Abdel Latif, Muhammad M.M.
2019. Eye-tracking in recent L2 learner process research: A review of areas, issues, and methodological approaches. System 83 ► pp. 25 ff. DOI logo
Sun, Sanjun & Kairong Xiao
2019. Chinese scholarship in Cognitive Translation Studies. Translation, Cognition & Behavior 2:1 ► pp. 125 ff. DOI logo

This list is based on CrossRef data as of 5 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue