Cover not available

In:Linking up with Video: Perspectives on interpreting practice and research
Edited by Heidi Salaets and Geert Brône
[Benjamins Translation Library 149] 2020
► pp. 203–233

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

Chapter 8
Eye-tracking in interpreter-mediated talk

From research to practice

 | University of Ghent
 | KU Leuven
Published online: 13 January 2020
DOI logo with link
https://doi.org/10.1075/btl.149.09vra
Abstract
A starting point for a multimodal analysis of interpreter-mediated interaction is the discussion on the pros and cons of new technologies especially for new forms of distant, remote or offsite interpreting. More specifically, in this contribution is argued that empirical multimodal analyses of participants’ and interpreters’ behavior in real-life interpreting settings may provide much-needed groundwork that helps to fuel this discussion. As one of the most central points of criticism raised by interpreting practitioners is that distant forms of interpreting lack certain characteristics that are typical for on-site or face-to-face interpreting (i.e. distant solutions are not true-to-life, Van Rotterdam & van den Hoogen 2011). It becomes increasingly important to pinpoint what these typical and natural characteristics are. Empirically grounded insights into these characteristics may then, in a research-technology-application loop, feed into innovative technologies, rendering novel interpreting solutions that are more true-to-life.
In this empirical analysis we focus on one specific phenomenon that has received increased attention in the recent literature in multimodal interaction analysis as well as interpreting studies, viz. eye gaze as a semiotic resource employed by speakers as well as hearers. The main questions here are how interpreters typically manage the turn-taking process in real-life interaction, among others by means of gaze, and how important visual access is to the co-participants for a smooth exchange. In order to answer the research questions, the researchers make use of eye-tracking technology to gain detailed information on interpreters’ and primary participants’ gaze behavior.
Article outline
  • 1.Face-to-face vs. remote interpreting
  • 2.Multimodal interaction: A brief technological state-of-the-art
  • 3.Interpreting: Multimodality & eye gaze
  • 4.Dataset & methodological considerations
  • 5.Empirical analysis
    • 5.1Multimodal feedback patterns in interpreter-mediated talk
      • a.Interpreter-directed feedback
      • b.Primary directed feedback
      • c.Dual feedback
    • 5.2Turn-taking in interpreter-mediated interactions
      • Adjacency pairs
      • Multiple turns
  • 6.Conclusion
  • Notes
  • References
  • Appendix. Transcription conventions
References (65)
References
Auer, Peter. 2005. “Projection in interaction and projection in grammar.” Text 25 (1): 7–36. Google Scholar logo with link to Google Scholar
Bavelas, Janet., Linda Coates, and Trudy Johnson. 2002. “Listener responses as a collaborative process: The role of gaze.” Journal of Communication, 566–580. Google Scholar logo with link to Google Scholar
Bot, Hanneke. 2005. Dialogue Interpreting in Mental Health. Amsterdam and New York: Rodopi Publishers.Google Scholar logo with link to Google Scholar
Braun Sabine. 2013. “Keep your distance? Remote interpreting in legal proceedings: A critical assessment of a growing practice.” Interpreting: international journal of research and practice in interpreting, 15 (2), pp.200–228. Google Scholar logo with link to Google Scholar
Braun, Sabine. 2015. “Remote interpreting.” In Routledge Encyclopedia of Interpreting Studies, ed. by Franz Pöchhacker, 346–348. London/New York: Routledge.Google Scholar logo with link to Google Scholar
. 2017. “What a micro-analytical investigation of additions and expansions in remote interpreting can tell us about interpreters’ participation in a shared virtual space.” Journal of Pragmatics 107, 165–177. Google Scholar logo with link to Google Scholar
Braun, Sabine, and L. Judith Taylor,. 2012. Videoconference and Remote Interpreting in Criminal Proceedings. Cambridge/Antwerp: Intersentia.Google Scholar logo with link to Google Scholar
Brenger, Bela, and Irene Mittelberg. (2015). “Shakes, nods and tilts. Motion-capture data profiles of speakers’ and listeners’ head gestures.” In: Proceedings of the 3rd Gesture and Speech in Interaction (GESPIN) Conference, Sept. 2015, Nantes. 43–48.Google Scholar logo with link to Google Scholar
Brône, Geert and Bert Oben. 2015. “InSight Interaction. A multimodal and multifocal dialogue corpus.” Language Resources and Evaluation 49: 195–214. Google Scholar logo with link to Google Scholar
Brône, Geert, Bert Oben , Annelies Jehoul , Jelena Vranjes, and, Kurt Feyaerts. 2017. “Eye gaze and viewpoint in multimodal interaction management.” Cognitive Linguistics 28 (3): 449–484. Google Scholar logo with link to Google Scholar
Clayman, Steven E. 2013. “Turn-Constructional Units and the Transition-Relevance-Place.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell, and Tanya Stivers, 150–166, Wiley-Blackwell.Google Scholar logo with link to Google Scholar
Clark, Herbert. 1996. Using Language. Cambridge University press: Cambridge. Google Scholar logo with link to Google Scholar
Clark, Herbert, and Susan Brennan. 1991. “Grounding in communication.” In Perspectives on socially shared cognition, ed. by Lauren B. Resnick, John M. Levine, and Stephanie D. Teasley, Washington: APA Books. Google Scholar logo with link to Google Scholar
Davidson, Brad. 2002. “A model for the construction of conversational common ground in interpreted discourse.” Journal of Pragmatics 34: 1273–1300. Google Scholar logo with link to Google Scholar
. 2015. “Gaze.” In Routledge Encyclopedia of Interpreting Studies, ed. by Franz Pöchhacker, Nadja Grbic, Peter Mead, and Robin Setton, 168–168, New York: Routledge.Google Scholar logo with link to Google Scholar
Davitti, Elena, and Sergio Pasquandrea. 2017. “Embodied participation: What multimodal analysis can tell us about interpreter-mediated encounters in pedagogical settings.” Elsevier Journal of Pragmatics 107: 105–128. Google Scholar logo with link to Google Scholar
Deppermann, Arnulf. 2013. “Turn-design at turn-beginnings: Multimodal resources to deal with tasks of turn-construction in German.” Journal of Pragmatics 46 (1): 91–121. Google Scholar logo with link to Google Scholar
Ellis, S. Ronald. 2004. “Videoconferencing in refugee hearings.” Ellis Report to the Immigration and Refugee Board Audit and Evaluation Committee. [URL].Google Scholar logo with link to Google Scholar
Englund Dimitrova, Birgitta. 1997. “Degree of Interpreter Responsibility in the Interaction Process in Community Interpreting.” In The Critical Link: Interpreters in the Community, ed. by Silvana E. Carr, 147–164, Amsterdam/Philadelphia: Benjamins.Google Scholar logo with link to Google Scholar
Esteban Causo, Jose. 2012. “Conference interpreting with information and communication technologies. Experiences from the European Commission DG Interpretation.” In Videoconference and remote interpreting in criminal proceedings, ed. by Sabine Braun, and Judith Taylor, 227–232 . Antwerp: Intersentia.Google Scholar logo with link to Google Scholar
Ford, Cecilia, Barbara Fox, and A. Sandra Thompson. 1996. “Practices in the construction of turns: The TCU revisted.” Pragmatics 6 ( 3 ): 427–454 . Google Scholar logo with link to Google Scholar
Gerwing, Jennfer, & Bavelas Janet (2013). “The social interactive nature of gestures: Theory, assumptions, methods, and findings. In Body-language-communication, Volume I, Contemporary approaches, ed. by Cornelia Müller, Allan Cienki, Ellen Fricke, Silva Ladewig, David McNeil & Sedinha Tessendorf. 816-831. Berlin: Mouton De Gruyter.Google Scholar logo with link to Google Scholar
Goffman, Erving. 1981. Forms of talk. Oxford: Basil Blackwell.Google Scholar logo with link to Google Scholar
Goodwin, Charles. 1981. Conversational Organization. Interaction between Speakers and Hearers. New York, London: Academic.Google Scholar logo with link to Google Scholar
Hayashi, Makoto. 2013. “Turn Allocation and Turn Sharing.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell and Tanya Stivers, 167–190, Wiley-Blackwell.Google Scholar logo with link to Google Scholar
Heath, Christian. 1986. Body Movement and Speech in Medical Interaction. Cambridge, U.K.: Cambridge University Press. Google Scholar logo with link to Google Scholar
Holler, Judith, Kobin H.,Kendrick, Marisa Casillas, and Stephen, C. Levinson . 2016. Turn-Taking in Human Communicative Interaction. Lausanne: Frontiers Media. .Google Scholar logo with link to Google Scholar
Holler, Judith, and Kobin. H. Kendrick. 2015.“Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency.” Frontiers in Psychology 6: 98. Google Scholar logo with link to Google Scholar
Jokinen, Kristiina. 2010. “Non-verbal signals for turn-taking & feedback.” Proc. of 7th Int. Conf. on Language Resources & Evaluation (LREC) International Universal Communication Symposium.Google Scholar logo with link to Google Scholar
Kendon, Adam. 1967. “Some functions of gaze-direction in social interaction.” Acta Psychologica 26: 22–63. Google Scholar logo with link to Google Scholar
Kipp, Michael. 2014. “ANVIL: A Universal Video Research Tool.” In Handbook of Corpus Phonology, ed. by Jacques Durand, Ulrike Gut, and Gjert Kristofferson, 420–436. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Lang, Ranier. 1978. “Behavioral aspects of liaison interpreters in Papua New Guinea: some preliminary observations.” In Language Interpretation and Communication, ed. by David Gerver, and H. Wallace Sinaiko, 231–244. New Yotk/London: Plenum Press. Google Scholar logo with link to Google Scholar
Lerner, Gene. H. 2003. “Selecting next speaker: The context-sensitive operation of a context-free organization.” Language in Society 32: 177–201. Google Scholar logo with link to Google Scholar
Llewellyn-Jones, Peter, and Robert. G. Lee, 2014. Redefining the Role of the Community Interpreter: The Concept of role-space. Lincoln: SLI Press.Google Scholar logo with link to Google Scholar
Li, Shuangyu. 2015. “Nine Types of Turn-taking in Interpreter-mediated GP Consultations.” Applied Linguistics Review 6 (1): 73–96. Google Scholar logo with link to Google Scholar
Licoppe, Christian, and Clair-Antoine Veyrier,. 2017. “How to show the interpreter on screen? The normatice organization of visual ecologies in multilingual courtrooms with video links.” Elsevier Journal of Pragmatics 107: 147–164. Google Scholar logo with link to Google Scholar
Linell, Per, Linda Jönsson, and Cecilia Wadensjö. 1992. “Establishing Communicative Contact through a Dialogue Interpreter.” In LSP Communication/Fachsprachliche Kommunikation, ed. by Annette Grindsted, and Johannes Wagner, 125–142, Tübingen: Narr Verlag.Google Scholar logo with link to Google Scholar
Mason, Ian. 2012. “Gaze, positioning and identity in interpreter-mediated dialogues.” In Coordinating Participation in Dialogue Interpreting, ed. by Claudio. Baraladi and Laura Gavioli, 177–199, Amsterdam/Philadelphia: Benjamins Publishing. Google Scholar logo with link to Google Scholar
Merlini, Raffaela, and Roberta Favaron. 2005. “Examining the “voice of interpreting” in speech pathology.” Interpreting 7 (2): 263–302. Google Scholar logo with link to Google Scholar
Mondada, Lorenza. 2007. “Multimodal resources for turn-taking: pointing and emergence of next speakers.” Discourse Studies: 194–225. Google Scholar logo with link to Google Scholar
. 2013. “Video as a tool in the social sciences.” In Body – Language – Communication: An International Handbook on Multimodality in Human Interaction , Vol. 1, ed. by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, and Sedinha Teßendorf, 982–992. Berlin: De Gruyter Mouton.Google Scholar logo with link to Google Scholar
. 2016. “Challenges of multimodality: Language and the body in social interaction.” Journal of Sociolinguistics 20–3, 336–366. Google Scholar logo with link to Google Scholar
Moser-Mercer, Barbara. 2003. “Remote interpreting: assessment of human factors and performance parameters’” Communicate! Summer 2003. Available at [URL].Google Scholar logo with link to Google Scholar
. 2005. “Remote interpreting: issues of multi-sensory integration in a multilingual task,” Meta, 50 (2), 727–738. Google Scholar logo with link to Google Scholar
Oben, Bert, and Geert Brône. 2015. “What you see is what you do: On the relation between gaze and gesture in multimodal alignment.” Language and Cognition 7 (4): 546–562. Google Scholar logo with link to Google Scholar
Pasquandrea, Sergio. 2011. “Managing multiple actions through multimodality: Doctors’ involvement in interpreter-mediated interactions.” Language in Society 40 (4): 455–481. Google Scholar logo with link to Google Scholar
Pitsch Karola, Bernhard-Andreas Brüning, Christian Schnier, Holger Dierker, and Sven Wachsmuth. 2010. Linking conversation analysis and motion capturing: How to robustly track multiple participants. In Proceedings of the LREC Workshop on Multimodal Corpora: Advances in Capturing, Coding and Analyzing Multimodality (MMC 2010), ed. by : Michael Kipp, Jean-Claude Martin, Patrizia Paggio, and Dirk Heylen, 63–69.Google Scholar logo with link to Google Scholar
Price, Erika, Eliseo Pérez-Stable, Dana Nickleach, Monica López, and Leah Karliner. 2012. “Interpreter perspectives of in-person, telephonic, and videoconferencing medical interpretation in clinical encounters.” Patient Education & Counseling, 87 (2), 226–232. Google Scholar logo with link to Google Scholar
Rossano, Federico. 2013. “Gaze in Conversation.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell, and Tanya Stivers, 308–329. Oxford: Wiley-Blackwell.Google Scholar logo with link to Google Scholar
Roziner, Ilan, and Miriam Shlesinger. 2010. “Much ado about something remote: Stress and performance in remote interpreting.” Interpreting 12 (2): 214–247. Google Scholar logo with link to Google Scholar
Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson. 1974. “A simplest systematics for the organization of turn-taking in conversation.” Language 50: 696–735. Google Scholar logo with link to Google Scholar
Schmitt, Reinhold. 2005.“Zur multimodalen Struktur von turn-taking.” Gesprächsforschung 6: 17–61.Google Scholar logo with link to Google Scholar
Selting, Margaret. et al. 2009.“Gesprächsanalytisches Transkriptionssystem 2 (GAT 2).” Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion 10: 353–402. [[URL]]Google Scholar logo with link to Google Scholar
Sloetjes, Han, and Peter Wittenburg 2008. “Annotation by category – ELAN and ISO DCR.” In: Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008).Google Scholar logo with link to Google Scholar
Stivers, Tania and Federico Rossano. 2010. “Mobilizing Response.” Research on Language and Social Interaction 43 (1): 3–31. Google Scholar logo with link to Google Scholar
Vandemoortele, Sarah, Stijn De Beugher, Geert Brône, Kurt Feyaerts, Toon Goedemé, Thomas De Baets, and Stijn Vervliet. 2016. Into the Wild. Musical Communication in Ensemble Playing. Leuven, Acco.Google Scholar logo with link to Google Scholar
Van Rotterdam, Peter, and Ronald van den Hoogen. 2011. “True-to-life requirements for using videoconferencing in legal proceedings.” In Videoconference and remote interpreting in criminal proceedings, ed. by Sabine Braun, and Judith L. Taylor, 187–197. Guildford: University of Surrey.Google Scholar logo with link to Google Scholar
Vranjes, Jelena, Geert Brône, and Kurt Feyaerts. 2018. “Dual feedback in interpreter-mediated interactions: on the role of gaze in the production of listener responses.” Journal of Pragmatics 134: 15–30. Google Scholar logo with link to Google Scholar
Vranjes, Jelena, Hanneke Bot, Kurt Feyaerts, and Geert Brône. (2019). “Displaying recipiency in an interpreter-mediated dialogue: an eye-tracking study. In Eye-tracking in interaction. Studies on the role of eye gaze in dialogue, ed. by Bert Oben and G. Brône . Amsterdam: Benjamins Publishing.Google Scholar logo with link to Google Scholar
Vranjes, Jelena. 2017. “Dual feedback and the management of intersubjectivity in interpreter-mediated talk. Mapping Multimodal Dialogue (MaMuD 4).” Frankfurt/Oder, Germany, 23–24 November 2017.Google Scholar logo with link to Google Scholar
Wadensjö, Cecilia. 1998. Interpreting as interaction. London and New York: Longman.Google Scholar logo with link to Google Scholar
Cited by (5)

Cited by five other publications

Jerkovic, Tiana
2024. Space, body and presence. Interpreting. International Journal of Research and Practice in Interpreting 26:2 ► pp. 201 ff. DOI logo
Argaman, Einav
2023. Neither speaker nor recipient: The middle-distance look of unaddressed participants. Discourse Studies 25:5 ► pp. 573 ff. DOI logo
Salaets, Heidi & Katalin Balogh
2023. Are interpreters and interpreting technology ready for the post-Covid era?. In Introducing New Hypertexts on Interpreting (Studies) [Benjamins Translation Library, 160], ► pp. 254 ff. DOI logo
Zhang, Xiaojun, Gloria Corpas Pastor & Jing Zhang
2023. Videoconference interpreting goes multimodal. In Interpreting Technologies – Current and Future Trends [IVITRA Research in Linguistics and Literature, 37], ► pp. 169 ff. DOI logo
Doherty, Stephen, Natalie Martschuk, Jane Goodman-Delahunty & Sandra Hale
2022. An Eye-Movement Analysis of Overt Visual Attention During Consecutive and Simultaneous Interpreting Modes in a Remotely Interpreted Investigative Interview. Frontiers in Psychology 13 DOI logo

This list is based on CrossRef data as of 3 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue