Cover not available

In:Mobile Eye Tracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 100–127

Get fulltext
PDF logo with download linkDownload Chapter PDF
EPUB logo with download linkDownload Chapter EPUB

Chapter 4
Mobile eye-tracking and mixed-methods approaches to interaction analysis

 | University of Leuven
 | University of Leuven
 | University of Leuven
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 13 May 2025
DOI logo with link
https://doi.org/10.1075/pbns.351.04obe
Abstract
The integration of mobile eye-tracking technology in linguistic research has catalyzed a surge of investigations across diverse linguistic subdisciplines. This chapter advocates for a mixed-methods approach in analyzing eye gaze behaviour during face-to-face interactions. Through two case studies, examining eye gaze in interactional irony and gaze synchronization, we demonstrate how this approach can help enhance our understanding of conversational eye gaze behaviour.
Article outline
  • 1.Introduction
  • 2.Defining and refining units of analysis: A case study on irony in interaction
  • 3.Mutual gaze during face-to-face interaction: A second case study
    • 3.1Data and method
    • 3.2Results
    • 3.3Discussion of the quantitative results
    • 3.4Further explaining the observed synchronisation in qualitative observations
    • 3.5Functional quantification
  • 4.Conclusion
  • Notes
  • References
References (65)
References
Argyle, Michael and Cook, Mark. 1976. Gaze and Eye contact. London: Cambridge University Press.Google Scholar logo with link to Google Scholar
Auer, Peter. 2005. “Projection in Interaction and Projection in Grammar.” Text — Interdisciplinary Journal for the Study of Discourse 25(1), 7–36. Google Scholar logo with link to Google Scholar
. 2021. “Turn-allocation and gaze: A multimodal revision of the “current-speaker-selects-next” rule of the turn-taking system of conversation analysis.” Discourse Studies 23(2), 117–140. Google Scholar logo with link to Google Scholar
Auer, Peter & Zima, Elisabeth. 2021. “On word searches, gaze, and co-participation.” Gesprächsforschung — Online — Zeitschrift zur verbalen Interaktion 22, 390–425.Google Scholar logo with link to Google Scholar
Barnden, John A. 2017. “Irony, pretence and fictively-elaborating hyperbole.“ In Irony in language use and communication, ed. by Anegliki Athanasiadou and Herbert L. Colston, 145–178. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Bavelas, Janet, Coates, Linda and Johnson, Trudy. 2002. Listener responses as a collaborative process: The role of gaze. Journal of Communication 52, 566–580. Google Scholar logo with link to Google Scholar
Bishop, Laura, Cancino-Chacon, Carlos, and Goebl, Werer. 2021. “Beyond synchronization: Body gestures and gaze direction in duo performance.” In Together in Music: Participation, Co-Ordination, and Creativity in Ensembles. ed. by Renee Timmers, Freya Bailes, and Helena Daffern, 182–188. Oxford: Oxford University. Google Scholar logo with link to Google Scholar
Brennan, Susan, Chen, Xin, Dickinson, Christopher A., Neider, Mark B., and Zelinsky, Gregory. 2008. “Coordinating cognition: The costs and benefits of shared gaze during collaborative search.” Cognition 106, 1465–1477. Google Scholar logo with link to Google Scholar
Brône, Geert. 2008. “Hyper- and misunderstanding in interactional humour”. Journal of Pragmatics 40(12), 2027–2061. Google Scholar logo with link to Google Scholar
. 2021. “The multimodal negotiation of irony and humour in interaction: On the role of eye gaze in joint pretense”. In Figurative Thought and Language (Vol. 11), ed. by Augusto Soares da Silva, 109–136. Amsterdam: John Benjamins Publishing Company.Google Scholar logo with link to Google Scholar
Brône, Geert and Oben, Bert. 2015. “InSight Interaction: A multimodal and multifocal dialogue corpus.” Language Resources and Evaluation 49(1), 195–214. Google Scholar logo with link to Google Scholar
Brône, Geeert and Oben, Bert (Eds.) 2018. Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue. (Advances in Interaction Studies (AIS)). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Brône, Geert and Oben, Bert. 2022. “Monitoring the pretence. Intersubjective grounding, gaze and irony.“ In Cognitive Sociolinguistics Revisited. ed. by Gitte Kristiansen, Karlien Franco, Stefano De Pascale, Laura Rosseel and Weiwei Zhang, 544–556. Berlin: De Gruyter Mouton.Google Scholar logo with link to Google Scholar
Brône, Geert & Oben, Bert. 2023. Mobile eye-tracking for multimodal interaction analysis. In The Routledge Handbook of Experimental Linguistics. ed. by Sandrine Zufferey and Pascal Gygax, 283–298. London: Routledge. Google Scholar logo with link to Google Scholar
Chartrand, Tanya L. and Bargh, John A. 1999. “The chameleon effect: The perception behaviour link and social interaction.” Journal of Personality and Social Psychology 76(6), 893–910. Google Scholar logo with link to Google Scholar
Clark, Herbert H. 1996. Using Language. Cambridge University Press. Google Scholar logo with link to Google Scholar
2016. “Depiction as a method of communication.” Psychological Review 123–3, 324–347. Google Scholar logo with link to Google Scholar
Clark, Herbert H. & Gerrig, Richard. 1984. On the pretense theory of irony. Journal of Experimental Psychology: General 113, 121–126. Google Scholar logo with link to Google Scholar
Coco, Moreno I. and Dale, Rick. 2014. “Cross-recurrence quantification analysis of categorical and continuous time series: an R package.” Frontiers in Psychology 5, 510. Google Scholar logo with link to Google Scholar
Coulson, Seana. 2005. “Sarcasm and the space-structuring model.” In The Literal and Nonliteral in Language and Thought. ed. by Seana Coulson and Barbara Lewandowska-Tomaszczyk. 129–144. Frankfurt: Peter Lang.Google Scholar logo with link to Google Scholar
Dale, Rick, Kirkham, Natasha and Richardson, Daniel. 2011. “How two people become a tangram recognition system.” Proceedings of the European Conference on Computer-Supported Cooperative Work. Berlin: Springer Verlag.Google Scholar logo with link to Google Scholar
de Ruiter, Jan, Mitterer, Holger and Enfield, Nick. 2006. “Predicting the end of a speaker’s turn: A cognitive cornerstone of conversation.” Language 82, 515–535. Google Scholar logo with link to Google Scholar
De Stefani, Elwys. 2021. “Embodied responses to questions-in-progress: silent nods as affirmative answers.” Discourse Processes 58(4), 353–371. Google Scholar logo with link to Google Scholar
de Vries, Clarissa, Oben, Bert and Brône, Geert. 2021. “Exploring the role of the body in communicating ironic stance.” Languages and Modalities 1(1), 65–80. Google Scholar logo with link to Google Scholar
ELAN (Version 6.4) [Computer software]. 2023. Nijmegen: Max Planck Institute for Psycholinguistics, The Language Archive. Retrieved from [URL]
Frischen, Alexandra, Bayliss, Andrew P. and Tipper, Steven. 2007. “Gaze cueing of attention: Visual attention, social cognition, and individual differences.” Psychological Bulletin 133(4), 694–724. Google Scholar logo with link to Google Scholar
Fusaroli, Riccardo and Tylén, Kristian. 2016. “Investigating Conversational Dynamics: Interactive Alignment, Interpersonal Synergy, and Collective Task Performance.” Cognitive Science 40(1), 145–71. Google Scholar logo with link to Google Scholar
Fusaroli, Riccardo, Konvalinka, Ivana and Wallot, Sebastian. 2014. “Analyzing social interactions: The promises and challenges of using cross recurrence quantification analysis.” Springer Proceedings in Mathematics and Statistics 103, 137–155. Google Scholar logo with link to Google Scholar
Gibbs, Raymond W. 2000. Irony in Talk Among Friends. Metaphor and Symbol 15(1). 5–27. Google Scholar logo with link to Google Scholar
Gironzetti, Elisa, Pickering, Lucy, Huang, Meichan, Zhang, Ying, Menjo, Shigehito and Attardo, Salvatore. 2016. “Smiling synchronicity and gaze patterns in dyadic humourous conversations.” Humour: International Journal of Humour Research 29(2), 301–324.Google Scholar logo with link to Google Scholar
Goodwin, Charles. 1981. Conversational Organization: Interaction Between Speakers and Hearers. Cambridge, MA: Academic Press.Google Scholar logo with link to Google Scholar
González-Fuente, Santoago, Escandell-Vidal, Victoria and Prieto, Pilar. 2015. “Gestural codas pave the way to the understanding of verbal irony.” Journal of Pragmatics 90, 26–47. Google Scholar logo with link to Google Scholar
Goodwin, Marjorie and Goodwin, Charles. 1986. “Gesture and coparticipation in the activity of searching for a word.” Semiotica 62, 51–75.Google Scholar logo with link to Google Scholar
Hadelich, Kerstin and Crocker, Matthew. 2006. “Gaze alignment of interlocutors in conversational dialogues.” Proceedings of the 2006 Symposium on Eye Tracking Research and Applications, 38. Google Scholar logo with link to Google Scholar
Heller, Vivien. 2021. Embodied displays of “Doing Thinking.” Epistemic and interactive functions of thinking displays in children’s argumentative activities. Frontiers in Psychology 12: 636671. Google Scholar logo with link to Google Scholar
Jehoul, Annelies, Brône, Geert and Feyaerts, Kurt. 2017. “Gaze patterns and fillers. Empirical data on the difference between Dutch ‘euh’ and ‘euhm’.” Proceedings of the 4th European and 7th Nordic Symposium on Multimodal Communication (MMSYM 2016), 43–50.Google Scholar logo with link to Google Scholar
Jokinen, Kristiina, Furukawa, Hirohisa, Nishida, Masafumi, Yamamoto, Seiichi. 2013. Gaze and turn-taking behavior in casual conversational interactions. ACM Transactions on Interactive Intelligent Systems 3 (2), Article 12, 30 pages. Google Scholar logo with link to Google Scholar
Kendon, Adam. 1967. “Some functions of gaze-direction in social interaction.” Acta Psychologica 26, 22–63. Google Scholar logo with link to Google Scholar
Kendrick, Kobin H. and Holler, Judith. 2017. “Gaze Direction Signals Response Preference in Conversation.” Research on Language and Social Interaction 50(1), 12–32. Google Scholar logo with link to Google Scholar
Lachat, Fanny, Conty, Laurence, Hugueville, Laurent and George, Nathalie. 2012. “Gaze cueing effect in a face-to-face situation.” Journal of Nonverbal Behaviour 36, 177–190. Google Scholar logo with link to Google Scholar
Louwerse, Max, Dale, Rick, Bard, Ellen and Jeuniaux, Patrick. 2012. “Behaviour matching in multimodal communication is synchronized.” Cognitive Science 36, 1404–1426. Google Scholar logo with link to Google Scholar
Mondada, Lorenza. 2014a. “The local constitution of multimodal resources for social interaction.” Journal of Pragmatics 65, 137–156. Google Scholar logo with link to Google Scholar
. 2014b. “Pointing, talk and the bodies: reference and joint attention as embodied interactional achievements” In From Gesture in Conversation to Visible Utterance in Action, ed. by Mandana Seyfeddinipur and Marianne Gullberg, 95–124. Amsterdam: John Benjamins, 95–124. Google Scholar logo with link to Google Scholar
Nakano, Tamami and Kitazawa, Shigeru. 2010. “Eyeblink entrainment at breakpoints of speech.” Experimental Brain Research 205, 577–581. Google Scholar logo with link to Google Scholar
Neider, Mark B., Chen, Xin, Dickinson, Christopher, Brennan, Susan and Zelinsky, Gregory. 2010. “Coordinating spatial referencing using shared gaze.” Psychonomic Bulletin & Review 17, 718–724. Google Scholar logo with link to Google Scholar
Oertel, Chatarine, Wlodarczak Marcin, Edlund Jens, Wagner Petra and Gustafson Joakim. 2012. “Gaze patterns in turn-taking.” Proceedings of Interspeech, 2247–2250. Google Scholar logo with link to Google Scholar
Pfeiffer, Martin & Clarissa Weiss. 2022. Reenactments during tellings: Using gaze for initiating reenactments, switching roles and representing events. Journal of Pragmatics 189. 92–113. Google Scholar logo with link to Google Scholar
Richardson, Daniel and Dale, Rick. 2005. “Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension.” Cognitive Science 29, 1045–1060. Google Scholar logo with link to Google Scholar
Richardson, Daniel, Dale, Rick and Tomlinson, John. 2009. “Conversation, gaze coordination, and beliefs about visual context.” Cognitive Science 33, 1468–1482. Google Scholar logo with link to Google Scholar
Sacks, Harvey, Schegloff, Emanuel. and Jefferson, Gail. 1974. “A simplest systematics for the organization of turn-taking for conversation.” Language 50, 696–735. Google Scholar logo with link to Google Scholar
Shockley, Kevin, Santana, Marie-Vee and Fowler, Carol. 2003. “Mutual interpersonal postural constraints are involved in cooperative conversation.” Journal of Experimental Psychology: Human Perception and Performance 29, 326–332.Google Scholar logo with link to Google Scholar
Sidnell, Jack. 2006. Coordinating Gesture, Talk, and Gaze in Reenactments. Research on Language and Social Interaction 39(4). 377–409. Google Scholar logo with link to Google Scholar
Sperber, Dan and Wilson, Deirdre. 1981. “Irony and the use-mention distinction.” In Radical Pragmatics. ed. by Peter Cole. 295–318. New York: Academic Press.Google Scholar logo with link to Google Scholar
Stukenbrock, Anja. 2020. “Deixis, Meta-Perceptive Gaze Practices, and the Interactional Achievement of Joint Attention.” Frontiers in Psychology 11: 1779. Google Scholar logo with link to Google Scholar
Tabacaru, Sabina and Lemmens, Maarten. 2014. “Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding.” The European Journal of Humour Research 2(2), 11–31. Google Scholar logo with link to Google Scholar
Ten Bosch, Louis, Oostdijk, Nelleke and Boves, Lou. 2005. “On temporal aspects of turn taking in conversational dialogues.” Speech Communication 47(1), 80–86. Google Scholar logo with link to Google Scholar
Thompson, Sandra A. & Ryoko Suzuki. 2014. Reenactments in conversation: Gaze and recipiency. Discourse Studies 16(6). 816–846. Google Scholar logo with link to Google Scholar
Tobin, Vera. 2016. Performance, Irony, and Viewpoint in Language. In Theatre, Performance and Cognition: Languages, Bodies and Ecologies. ed. by Amy Cook and Ronda Blair. 54–67. London: Bloomsbury Methuen. Google Scholar logo with link to Google Scholar
Vandemoortele, Sarah, Feyaerts, Kurt, Reybrouck, Mark, De Bièvre, Geert, Brône, Geert, De Baets, Thomas. 2018. “Gazing at the partner in musical trios: a mobile eye-tracking study.” Journal of Eye Movement Research 11 (2), 1–13. Google Scholar logo with link to Google Scholar
Vrzakova, Hana, Amon, Mary J., Stewart, Angela and D’Mello, Sidney. 2019. “Dynamics of Visual Attention in Multiparty Collaborative Problem Solving using Multidimensional Recurrence Quantification Analysis.” Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ‘19), Paper 342, 1–14. Google Scholar logo with link to Google Scholar
Wallot, Sebastian. 2017. “Recurrence quantification analysis of processes and products of discourse: A tutorial in R.” Discourse Processes 54(5–6), 382–405. Google Scholar logo with link to Google Scholar
Xu, Tian, de Barbaro, Kaya, Abney, Drew and Cox, Ralf. 2020. “Finding Structure in Time: Visualizing and Analyzing Behavioural Time Series.” Frontiers in Psychology 11, 1457. Google Scholar logo with link to Google Scholar
Zima, Elisabeth. 2020. Gaze and Recipient Feedback in Triadic Storytelling Activities. Discourse Processes 57(9). 725–748. Google Scholar logo with link to Google Scholar
Zima, Elisabeth, Weiß, Clarissa and Geert Brône. 2019. “Gaze and overlap resolution in triadic interactions.” Journal of Pragmatics 140, 49–69. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue