Cover not available

In:Mobile Eye Tracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 1–21

Get fulltext
PDF logo with download linkDownload Chapter PDF
EPUB logo with download linkDownload Chapter EPUB

Chapter 1
Introduction

Mobile eye tracking for the study of gaze in social interaction

 | University of Heidelberg
 | University of Freiburg
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 13 May 2025
DOI logo with link
https://doi.org/10.1075/pbns.351.01stu
Article outline
  • 1.Gaze in social interaction: A multifunctional resource
  • 2.The advent of mobile eye tracking: A turning point in Conversation Analysis?
  • 3.The chapters of this volume
  • Acknowledgements
  • Note
  • References
References (104)
References
Argyle, Michael and Cook, Mark. 1976. Gaze and Mutual Gaze. London: Cambridge University Press.Google Scholar logo with link to Google Scholar
Attardo, Salvatore and Pickering, Lucy. 2022. Eye Tracking in Linguistics. London: Bloomsbury Academic.Google Scholar logo with link to Google Scholar
Auer, Peter. 2018. “Gaze, Addressee Selection and Turn-Taking in Three-Party Interaction.” In Eye-Tracking in Interaction: Studies on the Role of Eye Gaze in Dialogue, ed. by Geert Brône and Bert Oben. 197–231. Amsterdam, Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
. 2021a. “Turn-allocation and gaze: A multimodal revision of the “current-speaker-selects-next” rule of the turn-taking system of conversation analysis.” Discourse Studies 23(2), 117–140. Google Scholar logo with link to Google Scholar
Auer, Peter & Zima, Elisabeth. 2021. “On word searches, gaze, and co-participation.” Gesprächsforschung — Online — Zeitschrift zur verbalen Interaktion 22, 390–425.Google Scholar logo with link to Google Scholar
Auer, Peter, Laner, Barbara, Pfeiffer, Martin, Botsch, Kerstin. 2024. “A multimodal investigation of the format ‘perception imperative + exclamative’ based on mobile eye-tracking data.” In New Perspectives in Interactional Linguistic Research. ed. by Margret Selting and Dagmar Barth-Weingarten. 242–271. Amsterdam: Benjamins. Google Scholar logo with link to Google Scholar
Auer, Peter and Laner, Barbara. This volume. “Laughter and gaze among talkers on a walk.”
Balantani, Angeliki. 2022. “Transitioning between activities in joint projects: The case of German “so”.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 23, 343–370.Google Scholar logo with link to Google Scholar
Balantani, Angeliki and Lázaro, Stefani. 2021. “Joint attention and reference construction: The role of pointing and “so”.” Language and Communication 79(6), 33–52. Google Scholar logo with link to Google Scholar
Barthel, Mathias and Rühlemann, Christoph. This volume. “Pupil size indicates planning effort at turn transitions in natural conversation”.
Bavelas, Janet, Coates, Linda and Johnson, Trudy. 2002. “Listener responses as a collaborative process: The role of gaze.” Journal of Communication 52, 566–580. Google Scholar logo with link to Google Scholar
Beattie, Geoffrey W. 1978. “Floor apportionment and gaze in conversational dyads.” British Journal of Social & Clinical Psychology 17(1), 7–15. Google Scholar logo with link to Google Scholar
1979. “Planning units in spontaenous speech: Some evidence from hesitations in speech and speaker gaze direction in conversation.” Linguistics 17, 61–78. Google Scholar logo with link to Google Scholar
Brône, Geert and Oben, Bert. 2015. “InSight Interaction: A multimodal and multifocal dialogue corpus.” Language Resources and Evaluation 49(1), 195–214. Google Scholar logo with link to Google Scholar
Brooks, Rechele and Meltzoff, Andrew N. 2014. “Gaze following: A mechanism for building social connections between infants and adults.” In Mechanisms of social connection: From brain to group. ed. by Mario Mikulincer and Phillip R. Shaver. 167–183. American Psychological Association. Google Scholar logo with link to Google Scholar
Carpenter, Malinda, Katherine Nagell, Michael Tomasello, George Butterworth, and Chris Moore. 1998. “Social Cognition, Joint Attention, and Communicative Competence from 9 to 15 Months of Age.” Monographs of the Society for Research in Child Development 63(4): 1–166. Google Scholar logo with link to Google Scholar
Clark, Herbet H. 1996. Using language. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
2021. “Common ground.” In International Encyclopedia of Linguistic Anthropology, ed. by James Stanlaw. New York: Wiley.
Degutyte, Ziedune and Astell, Arlene. 2021. “The Role of Eye Gaze in Regulating Turn Taking in Conversations: A Systematized Review of Methods and Findings.” Frontiers in Psychology 12: 616471. Google Scholar logo with link to Google Scholar
Deppermann, Arnulf. 2015. “Pragmatik revisited”. In Sprachwissenschaft im Fokus. Positionsbestimmungen und Perspektiven. ed. by Ludwig M. Eichinger. 323–352. Berlin/Boston: de Gruyter. Google Scholar logo with link to Google Scholar
Deppermann, Arnulf, Streeck, Jürgen. (Eds.). 2018. Time in Embodied Interaction. Amsterdam, Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
De Stefani, Elwys. 2014. “Rearranging (in) space: On mobility and its relevance for the study of face-to-face interaction”. In Space in Language and Linguistics: Geographical, Interactional, and Cognitive Perspectives, ed. by Peter Auer, Martin Hilpert, Anja Stukenbrock and Benedikt Szmrecsanyi. 434–463. Berlin, Boston: De Gruyter.Google Scholar logo with link to Google Scholar
. 2021. “If-clauses, their grammatical consequents, and their embodied consequence: making people turn around in guided tours.” Frontiers in Communication 6:661165. Google Scholar logo with link to Google Scholar
Duchowski, Andrew T. 2017. Eye Tracking Methodology: Theory and Practice. London: Springer International Publishing. Google Scholar logo with link to Google Scholar
Duncan, Starkey. 1972. “Some signals and rules for taking speaking turns in conversations.” Journal of Personality and Social Psychology 23, 283–292. Google Scholar logo with link to Google Scholar
Enfield, Nick. 2009. The anatomy of meaning. Speech, gesture, and composite utterances. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Eriksson, Mats. 2009. “Referring as interaction: On the interplay between linguistic and bodily practices.” Journal of Pragmatics 41(2), 240–262. Google Scholar logo with link to Google Scholar
Flom, R., Lee, K., Muir, D. (Eds.). 2007. Gaze-Following. Its Development and Significance. Lawrence Erlbaum, Mahwah, NJ.Google Scholar logo with link to Google Scholar
Goffman, Erving. 1963. Behaviour in Public Places: Notes on the Social Organization of Gatherings. New York: Free Press.Google Scholar logo with link to Google Scholar
. 1981. “Footings.” In: Forms of talk. ed. by Erving Goffman. 124–159. Philadelphia: University of Pennsylvania Press.Google Scholar logo with link to Google Scholar
. 1986. Frame Analysis. An Essay on the Organization of Experience. Boston: Northeastern University Press.Google Scholar logo with link to Google Scholar
Goodwin, Charles. 1980. “Restarts, Pauses, and the Achievement of a State of Mutual Gaze at Turn-Beginning.” Sociological Inquiry 50 (3–4), 272–302. Google Scholar logo with link to Google Scholar
. 1981. Conversational organization: Interaction between speakers and hearers. Academic Press.Google Scholar logo with link to Google Scholar
. 1984. “Notes on story structure and the organization of participation.” In Structures of social action: Studies in Conversation Analysis, ed. by J. Maxwell Atkinson and John Heritage. 225–246. London: Cambridge University Press.Google Scholar logo with link to Google Scholar
. 1986. “Between and within: Alternative sequential treatments of continuers and assessments” Human Studies 9(2–3), 205–217. Google Scholar logo with link to Google Scholar
Goodwin, Marjorie. 1980. “Processes of Mutual Monitoring Implicated in the Production of Description Sequences.” Sociological Inquiry 50(3–4), 303–317. Google Scholar logo with link to Google Scholar
Goodwin, Charles. 2003. “Pointing as situated practice.” In Pointing: Where Language, Culture, and Cognition Meet. ed. by Satoro Kita. 217–241. Lawrence Erlbaum, Mahwah, NJ.Google Scholar logo with link to Google Scholar
. 1994. “Professional Vision.” American Anthropologist. 96(3), 606–633. Google Scholar logo with link to Google Scholar
. 1996. “Transparent Vision.” In: Interaction and Grammar. ed. by Elinor Ochs, Emanuel A. Schegloff und Sandra A. Thompson. 370–404. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Goodwin, Marjorie and Goodwin, Charles. 1986. “Gesture and coparticipation in the activity of searching for a word.” Semiotica 62, 51–75.Google Scholar logo with link to Google Scholar
Hindmarsh, Jon and Heath, Christian. 2000. “Embodied reference: A study of deixis in workplace interaction.” Journal of Pragmatics 32(12), 1855–1878. Google Scholar logo with link to Google Scholar
Holler, Judith and Kendrick, Kobin. 2015. “Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency.” Frontiers in Psychology 6: 98. Google Scholar logo with link to Google Scholar
Holler, Judith and Levinson, Stephen C. 2019. “Multimodal Language Processing in Human Communication.” Trends in Cognitive Science 23(8), 639–652. Google Scholar logo with link to Google Scholar
Holmqvist, Kenneth, Nyström, Marcus, Andersson, Richard, Dewhurst, Richard, Halszka, Jarodzka, and Joost van de Weijer (eds). 2011. Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Jehoul, Annelies, Brône, Geert and Feyaerts, Kurt. 2017. “Gaze patterns and fillers. Empirical data on the difference between Dutch ‘euh’ and ‘euhm’.” Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016), 43–50.Google Scholar logo with link to Google Scholar
Jucker, Andreas H., Schneider, Klaus P., Bublitz, Woldfram. Eds. 2018. Methods in Pragmatics. De Gruyter Mouton, Berlin, Boston. Google Scholar logo with link to Google Scholar
Jucker, Andreas H., Taavitsainen, Irma (Eds.). 2010. Historical Pragmatics. Berlin, Bosotn: De Gruyter Mouton, Berlin, Boston. Google Scholar logo with link to Google Scholar
Kendon, Adam. 1967. “Some functions of gaze direction in social interaction.” Acta Psychologica 26: 22–63. Google Scholar logo with link to Google Scholar
. 1990. “Movement coordination in social interaction: some examples described.” In Conducting interaction. Patterns of behavior in focused encounters. ed. by Adam Kendon. 91–115. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Kendrick, Kobin H. and Holler, Judith. 2017. “Gaze Direction Signals Response Preference in Conversation.” Research on Language and Social Interaction 50(1), 12–32. Google Scholar logo with link to Google Scholar
Kristiansen, Elisabeth Dalby and Rasmussen, Gitte. 2021. “Eye-tracking Recordings as Data in EMCA Studies: Exploring Possibilities and Limitations. Social Interaction.” Video-Based Studies of Human Sociality 4(4). Google Scholar logo with link to Google Scholar
Krug, Maximilian. 2020. “Erzählen inszenieren. Ein Theatermonolog als multimodale Leistung des Interaktionsensembles auf der Probebühne.” Linguistik online 104 (4), 59–81. Google Scholar logo with link to Google Scholar
. 2022. Gleichzeitigkeit in der Interaktion. Strukturelle (In)Kompatibilität bei Multiaktivitäten in Theaterproben. Berlin: De Gruyter. Google Scholar logo with link to Google Scholar
. This volume. “Gaze aversion as a marker of disalignment in interactions.”
Locher, Miriam A., Graham, Sage L. Eds. 2010. Interpersonal Pragmatics. Berlin, Boston: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Masuch, Johanna. This volume. “In the relationship between gaze and the German recipient token hm_hm.”
Mondada, Lorenza. 2013. “Conversation analysis: Talk and bodily resources for the organization of social interaction”. In: Body — Language — Communication. Volume 1. ed. by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill and Sedinha Tessendorf, 218–226. Berlin, Boston: De Gruyter Mouton. Google Scholar logo with link to Google Scholar
. 2014. “Bodies in action: multimodal analysis of walking and talking.” Language and Dialogue 4 (3): 357–403. Google Scholar logo with link to Google Scholar
. 2019. “Contemporary Issues in Conversation Analysis: Embodiment and Materiality, Multimodality and Multisensoriality in Social Interaction.” Journal of Pragmatics 145: 47–62. Google Scholar logo with link to Google Scholar
Nishizaka, Aug. 2017. “The Perceived Body and Embodied Vision in Interaction”. Mind, Culture, and Activity 24(2), 110–128. Google Scholar logo with link to Google Scholar
Oben, Bert. 2018. “Gaze as a predictor for lexical and gestural alignment.” In Eye-Tracking in interaction. Studies on the role of eye gaze in dialogue. ed by Geert Brône and Bert Oben. 233–264. Amsterdam/Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Oben, Bert, Brône, Geert. 2016. “Explaining interactive alignment: A multimodal and multifactorial account.” Journal of Pragmatics, 104: 32–51. Google Scholar logo with link to Google Scholar
Oben, Bert, Clarissa de Vries, and Geert Brône. This volume. “Mobile eye-tracking and mixed-methods approaches to interaction analysis.”
Pfeiffer, Martin and Weiss, Clarissa. 2022. “Reenactments during tellings: Using gaze for initiating reenactments, switching roles, and representing events.” Journal of Pragmatics 189, 92–113. Google Scholar logo with link to Google Scholar
Płużyczka, Monika. 2018. “The First Hundred Years: a History of Eye Tracking as a Research Method.” Applied Linguistics Papers. 25/4, 101–116. Google Scholar logo with link to Google Scholar
Rasmussen, Gitte and Kristiansen, Elisabeth Dalby. This volume. “The influence of the specificities of gaze behavior on emerging and ensuing interaction”.
Rossano, Federico. 2012a. Gaze behaviour in face-to-face conversation [Doctoral dissertation]. Radboud University Nijmegen.
. 2012b. “Gaze in Conversation.” The Handbook of Conversation Analysis. ed. by Jack Sidnell and Tanya Stivers: 308–329. Oxford, U.K.: Wiley Blackwell. Google Scholar logo with link to Google Scholar
Rühlemann, Christoph. 2022. “How is emotional resonance achieved in storytellings of sadness/distress?” Frontiers in Psychology Volume 13 — 2022. Google Scholar logo with link to Google Scholar
Rühlemann, Christoph, and Alexander Ptak. 2023. “Reaching beneath the Tip of the Iceberg: A Guide to the Freiburg Multimodal Interaction Corpus.” Open Linguistics 9 (1). Google Scholar logo with link to Google Scholar
Sacks, Harvey. 1992. Lectures on conversation. Vol. 1 & 2 Oxford: Basil Blackwell.Google Scholar logo with link to Google Scholar
Scaife, Michael and Bruner, Jerome S. 1975. “The capacity for joint visual attention in the infant.” Nature 253(5489), 265–666. Google Scholar logo with link to Google Scholar
Schegloff, Emanuel A. 1982. “Discourse as an interactional achievement: Some uses of ‘uh huh’ and other things that come between sentences.” In Analyzing Discourse: Text and Talk, ed. by Deborah Tannen, 71–93. Georgetown University Press.Google Scholar logo with link to Google Scholar
Schütz, Alfred. 1953. “Common-sense and Scientific Interpretation of Human Action.” Philosophy and Phenomenological Research 14 (1): 1–38. Google Scholar logo with link to Google Scholar
Selting, Margret, Auer, Peter, Barth-Weingarten, Dagmar, Bergmann, Jörg, Bergmann, Pia, Birkner, Karin, Couper-Kuhlen, Elizabeth, Deppermann, Arnulf, Gilles, Peter, Günthner, Susanne, Hartung, Martin, Kern, Frederike, Mertzlufft, Christine, Meyer, Christian, Morek, Miriam, Oberzaucher, Frank, Peters, Jörg, Quasthoff, Uta, Schutte, Wilfried, Stukenbrock, Anja, and Uhmann, Susanne. 2009. “Gesprächsanalytisches Transkriptionssystem 2 (GAT 2).” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 10: 353–402.Google Scholar logo with link to Google Scholar
Stivers, Tanya. 2010. “An overview of the question–response system in American English conversation.” Journal of Pragmatics 42(10), 2772–2781. Google Scholar logo with link to Google Scholar
Stivers, Tanya and Federico Rossano. 2010. “Mobilising response.” Research on Language and Social Interaction 43: 3–31. Google Scholar logo with link to Google Scholar
Stivers, Tanya, Enfield, Nick., Brown, Penelope, Englert, Christona, Hayashi, Makoto, Heinemann, Trine, Hoymann, Gertie, Rossano, Frederico, De Ruiter, Jan P., Yoon, Kyong-Eun and Levinson, Steven C. 2009. “Universals and cultural variation in turn-taking in conversation.” Proceedings of the National Academy of Sciences 106(26), 10587–10592. Google Scholar logo with link to Google Scholar
Streeck, Jürgen. 2014. “Mutual gaze and recognition: Revisiting Kendon’s ‘Gaze direction on two-person interaction’” In From Gesture in Conversation to Gesture as Visible Utterance: Essays in Honor of Adam Kendon. ed. by Mandana Seyfeddinipur and Marianne Gullberg. 35–55. Amsterdam: Benjamins.Google Scholar logo with link to Google Scholar
Streeck, Jürgen, Goodwin, Charles, LeBaron, Curtis. Eds. (2011). Embodied Interaction: Language and Body in the Material World. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Stukenbrock, Anja. 2009. “Referenz durch Zeigen. Zur Theorie der Deixis.” Deutsche Sprache 37, 289–316. Google Scholar logo with link to Google Scholar
. 2014a. “Take the words out of my mouth: Verbal instructions as embodied practices.” Journal of Pragmatics 65, 80–102. Google Scholar logo with link to Google Scholar
. 2014b. “Pointing to an ‘empty’ space: Deixis am Phantasma in face-to-face interaction.” Journal of Pragmatics 74, 70–93. Google Scholar logo with link to Google Scholar
. 2015. Deixis in der face-to-face-Interaktion. Berlin, Boston: De Gruyter. Google Scholar logo with link to Google Scholar
. 2018a. “Mobile dual eye-tracking in face-to-face interaction. The case of deixis and joint attention.” In Eye Tracking in Interaction. Studies on the Role of Eye Gaze in Dialogue. ed. by Geert Brône and Bert Oben. 265–301. Amsterdam: Benjamins. Google Scholar logo with link to Google Scholar
. 2018b. “Blickpraktiken von Sprechern und Hörern bei der Lokaldeixis: Mobile Eye Tracking-Analysen.” Gesprächsforschung. Online-Zeitschrift zur verbalen Interaktion, 132–168.Google Scholar logo with link to Google Scholar
. 2020. “Deixis, Meta-Perceptive Gaze Practices, and the Interactional Achievement of Joint Attention.” Frontiers in Psychology 11, 1779–1779. Google Scholar logo with link to Google Scholar
. 2023. “Temporality and the cooperative infrastructure of human communication: Noticings to delay and to accelerate onward movement in mobile interaction.” Language & Communication 92, 33–54. Google Scholar logo with link to Google Scholar
Stukenbrock, Anja and Balantani, Angeliki. This volume. “When the establishment of joint attention becomes problematic: How participants manage divergent and competing foci of attention.”
Stukenbrock, Anja and Dao, Anh Nhi. 2019. “Joint Attention in Passing: What Dual Mobile Eye Tracking Reveals About Gaze in Coordinating Embodied Activities at a Market: Social Encounters in Time and Space”. In Embodied Activities in Face-to-face and Mediated Settings, ed. by Elisabeth Reber and Cornelia Gerhardt. 177–213. Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Sweetser, Eve and Stec, Kashmiri. 2016. “Maintaining multiple viewpoints with gaze.” In: Viewpoint and the Fabric of Meaning: Form and Use of Viewpoint Tools across Languages and Modalities. ed. by Barbara Dancygier, Wie-lun Lu and Arie Verhagen. 237–258. Berlin: de Gruyter Mouton. Google Scholar logo with link to Google Scholar
Tomasello Michael, Hare, Brian, Lehmann Hagen, and Call, Josep. 2007. “Reliance on head versus eyes in the gaze following of great apes and human infants: the cooperative eye hypothesis.” Journal of Human Evolution 52(3), 314–20. Google Scholar logo with link to Google Scholar
Vranjes, Jelena, Brône, Geert and Feyaerts, Kurt. 2018. “Dual feedback in interpreter-mediated interactions: On the role of gaze in the production of listener responses.” Journal of Pragmatics 134, 15–30. Google Scholar logo with link to Google Scholar
Wade, Nicholas and Tatler, Benjamin. 2009. “Did Javal measure eye movements during reading?” Journal of Eye Movement Research 5, 5–7. Google Scholar logo with link to Google Scholar
Weiß, Clarissa. 2018. “When gaze-selected next speakers do not take the turn.” Journal of Pragmatics 133, 28–44. Google Scholar logo with link to Google Scholar
. 2019. “Blickverhalten des nicht-blickselegierten Sprechers während Korrekturen und Elaborierungen.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 20, 1–28.Google Scholar logo with link to Google Scholar
Weiß. 2020. Blick und Turn-Taking in Face-to-Face-Interaktionen: Multimodale Interaktionsanalysen triadischer Gesprächssituationen mit Hilfe von Eye-Tracking. Verlag für Gesprächsforschung. [URL]
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A. & H. Sloetjes. 2006. “ELAN: a Professional Framework for Multimodality Research.” Proceedings of LREC 2006, Fifth International Conference on Language Resources and Evaluation, 1556–1559.Google Scholar logo with link to Google Scholar
Zima, Elisabeth. 2018. “Multimodale Mittel der Rederechtsaushandlung im gemeinsamen Erzählen.” [Multimodal means to negotiate turn-taking in joint storytellings] Gesprächsforschung — Online — Zeitschrift zur verbalen Interaktion (18), 241–273.Google Scholar logo with link to Google Scholar
. 2020. “Gaze and recipient feedback in triadic storytelling activities.” Discourse Processes 57 (9), 725–748. Google Scholar logo with link to Google Scholar
Zima, Elisabeth, Weiß, Clarissa and Geert Brône. 2019. “Gaze and overlap resolution in triadic interactions.” Journal of Pragmatics 140, 49–69. Google Scholar logo with link to Google Scholar
Zima, Elisabeth, Auer, Peter and Rühlemann, Christoph. This volume. “Why research on gaze in social interaction needs mobile eye tracking.”
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue