Cover not available

In:Mobile Eye Tracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 243–276

Get fulltext
PDF logo with download linkDownload Chapter PDF
EPUB logo with download linkDownload Chapter EPUB

Chapter 9
When the establishment of joint attention becomes problematic

How participants manage divergent and competing foci of attention

 | University of Heidelberg
 | University of Lausanne
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 13 May 2025
DOI logo with link
https://doi.org/10.1075/pbns.351.09stu
Abstract
In the past decades, a substantial amount of research has studied how joint attention is collaboratively accomplished in social interaction. By contrast, divergent and competing foci of attention have remained largely unexplored. Our study investigates how participants establish, or refrain from establishing, joint attention in the face of attentional divergence and competition. When participants summon their co-participants’ attention on an object, the preferred response is to reorient and share attention. However, for various reasons, addressees may not always follow the invitation to share attention. One of the instances in which they may not (immediately) respond by reorienting is when they are themselves engrossed in something and prefer not to give it up for the sake of attention sharing.
Using the methodological principles of Conversation Analysis and a corpus of naturally occurring interactions recorded with video cameras and mobile eye tracking glasses, we examine the use of deictics and embodied practices to invite joint attention in open states of talk when the co-participant’s attention is diverging. The recordings enable us to zoom in on how gaze (eye tracking data) and embodied orientation (data from external cameras) index and contribute to how sequences of divergent, competing, and joint attention unfold. Preliminary observations suggest, first, that the participants’ spatial configuration contributes to how the problem of competing foci of attention is handled, and second, that participants deploy different verbal and embodied practices to pursue joint attention in the face of competing sites of interest. These practices are sensitive to, and reflexively constitute the participants’ spatial configuration and range on a continuum from less to more response mobilising.
Article outline
  • 1.Introduction
  • 2.Background: On joint attention and how it is accomplished in social interaction
  • 3.Data and methodology
  • 4.How participants manage divergent and competing foci of attention
    • 4.1Sequential resolution: Both objects attended to
    • 4.2Sequential resolution: Sharing attention on one object and abandoning the other
    • 4.3Lack of attention sharing
  • 5.Discussion
  • Notes
  • References
References (71)
References
Anna, Marina, and Martin Pfeiffer. 2021. “Die frühkindliche Verwendung des Exklamativformats oh+X: Noticings und Problemmitteilungen.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 22: 1–35.Google Scholar logo with link to Google Scholar
Auer, Peter, Barbara Laner, Martin Pfeiffer, and Kerstin Botsch. 2024. “Noticing and Assessing Nature: A Multimodal Investigation of the Format “Perception Imperative + Exclamative” based on Mobile Eye-Tracking Data.” In New Perspectives in Interactional Linguistic Research, ed. by Margret Selting, and Dagmar Barth-Weingarten, 245–275. Amsterdam, Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
Auer, Peter, and Anja Stukenbrock. 2022. “Deictic Reference in Space.” In Pragmatics of Space, ed. by Andreas H. Jucker, and Heiko Hausendorf, 23–61. Berlin, Boston: Mouton De Gruyter. Google Scholar logo with link to Google Scholar
Balantani, Angeliki. 2021. “Reference Construction in Interaction: The Case of Type-indicative “so”.” Journal of Pragmatics 181: 241–258. Google Scholar logo with link to Google Scholar
Balantani, Angeliki, and Stefanie Lázaro. 2021. “Joint Attention and Reference Construction: The Role of Pointing and “so”.” Language and Communication 79: 33–52. Google Scholar logo with link to Google Scholar
Botsch, Kerstin, Peter Auer, Barbara Laner and Martin Pfeiffer. This volume. “Joint attention without language? On intersubjectivity and the joint experience of nature.”
Clark, Herbert H. 1996. Using Language. Cambridge, MA: Cambridge University Press. Google Scholar logo with link to Google Scholar
2003. “Pointing and Placing.” In Pointing. Where Language, Culture, and Cognition Meet, ed. by Sotaro Kita, 243–268. Hillsdale NJ: Erlbaum.Google Scholar logo with link to Google Scholar
Clark, Herbert H., and Meredyth A. Krych. 2004. “Speaking while Monitoring Addressees for Understanding.” Journal of Memory and Language 50 (1): 62–81. Google Scholar logo with link to Google Scholar
Clark, Herbert H., and Catherine R. Marshall. 1981. “Definite Reference and Mutual Knowledge.” In Elements of Discourse Understanding, ed. by Aravind K. Joshi, Bonnie L. Webber, and Ivan A. Sag B, 10–63. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Clark, Herbert H., Robert Schreuder, and Samuel Buttrick. 1983. “Common Ground and the Understanding of Demonstrative Reference.” Journal of Verbal Learning and Verbal Behaviour 22: 245–258. Google Scholar logo with link to Google Scholar
Couper-Kuhlen, Elizabeth, and Dagmar Barth-Weingarten. 2011. “A System for Transcribing Talk-in-Interaction, Translated and Adapted for English by Elizabeth Couper-Kuhlen and Dagmar Barth-Weingarten.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 12: 1–51.Google Scholar logo with link to Google Scholar
Couper-Kuhlen, Elizabeth, and Margret Selting. 2018. Interactional Linguistics. Studying Language in Social Interaction. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Deppermann, Arnulf. 2013. “Multimodal Interaction from a Conversation Analytic Perspective.” Journal of Pragmatics 46 (1): 1–7. Google Scholar logo with link to Google Scholar
De Stefani, E. 2014. “Establishing Joint Orientation towards Commercial Objects in a Self-service Store: How Practices of Categorisation Matter.” In Interacting with Objects, ed. by Maurice Nevile, Pentti Haddington, Trine Heinemann, and Mirka Rauniomaa, 271–294. Amsterdam, Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
De Stefani, Elwys, and Arnulf Deppermann. 2021. “Les gestes de pointage dans un environnement changeant et éphémère: les leçons de conduit.” Langage et Societé 2: 141–166. Google Scholar logo with link to Google Scholar
Diessel, Holger. 2006. “Demonstratives, Joint Attention, and the Emergence of Grammar.” Cognitive Linguistics 17: 463–489. Google Scholar logo with link to Google Scholar
Diessel, Holger, and Kenny R. Coventry. 2020. “Demonstratives in Spatial Language and Social Interaction: An Interdisciplinary Review. Frontier in Psychology 11. Google Scholar logo with link to Google Scholar
Eriksson, Mats. 2009. “Referring as Interaction: On the Interplay between Linguistic and Bodily Practices. Journal of Pragmatics 41 (2): 240–262. Google Scholar logo with link to Google Scholar
Fillmore, Charles. 1997[1971]. Lectures on Deixis. Stanford, CA: University of Chicago Press.Google Scholar logo with link to Google Scholar
Fricke, Ellen. 2007. Origo, Geste und Raum. Berlin, New York: De Gruyter. Google Scholar logo with link to Google Scholar
Goffman, Erving. 1963. Behaviour in Public Places. Notes on the Social Organization of Gathering. New York: Free Press.Google Scholar logo with link to Google Scholar
. 1978. “Response Cries.” Language 54 (4): 787–815. Google Scholar logo with link to Google Scholar
. 1981. Forms of Talk. Philadelphia: University of Pennsylvania Press.Google Scholar logo with link to Google Scholar
Goodwin, Charles. 1981. Conversational Organization: Interaction between Speakers and Hearers. New York: Academic Press.Google Scholar logo with link to Google Scholar
. 2003. “Pointing as Situated Practice.” In Pointing. Where Language, Culture, and Cognition Meet, ed. by Sotaro Kita, 217–241. Mahwah, NJ: Erlbaum. Google Scholar logo with link to Google Scholar
. 2007. “Environmentally Coupled Gestures.” In Gesture and the Dynamic Dimension of Language. Essays in Honor of David McNeill, ed. by Susan D. Duncan, Justine Cassell, and Elena T. Levy, 195–212. Amsterdam, Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
. 2017. Co-Operative Action. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Goodwin, Marjorie Harness, and Charles Goodwin. 2012. “Car Talk: Integrating Texts, Bodies, and Changing Landscapes.” Semiotica 191 (1): 257–286. Google Scholar logo with link to Google Scholar
Hausendorf, Heiko. 2013. “On the Interactive Achievement of Space — and its Possible Meanings.” In Space in Language and Linguistics. Geographical, Interactional, and Cognitive Perspectives, ed. by Peter Auer, Martin Hilpert, Anja Stukenbrock, and Benedikt Szmrecsanyi, 276–303. Berlin, Boston: De Gruyter. Google Scholar logo with link to Google Scholar
Hausendorf, Heiko, Lorenza Mondada, and Reinhold Schmitt (eds). 2012. Raum als interaktive Ressource. Tübingen: Narr Francke Attempto.Google Scholar logo with link to Google Scholar
Heritage, John. 1984. “A Change of State Token and Aspects of Its Sequential Placement.” In Structures of Social Action, ed. by J. Maxwell Atkinson, and John Heritage, 299–345. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Hindmarsh, Jon, and Christian Heath. 2000. “Embodied Reference: a Study of Deixis in Workplace Interaction.” Journal of Pragmatics 32 (12): 1855–1878. Google Scholar logo with link to Google Scholar
Hindmarsh, Jon, Christian Heath, and Mike Fraser. 2006. “(Im)Materiality, Virtual Reality and Interaction: Grounding the ‘Virtual’ in Studies of Technology in Action.” The Sociological Review 54 (4): 795–817. Google Scholar logo with link to Google Scholar
Keating, Elizabeth. 2015. “The Role of the Body and Space in Digital Multimodality.” In The Routledge Handbook of Language and Digital Communication, ed. by Alexandra Georgakopoulou, and Tereza Spilioti, 259–272. London: Routledge.Google Scholar logo with link to Google Scholar
Kendon, Adam. 1967. “Some Functions of Gaze-direction in Social Interaction.” Acta Psychologica 26: 22–63. Google Scholar logo with link to Google Scholar
. 1990. Conducting Interaction. Patterns of Behaviour in Focused Encounters. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
. 2004. Gesture: Visible Action as Utterance. Cambridge University Press, Cambridge. Google Scholar logo with link to Google Scholar
Kita, Sotaro (ed.) 2003. Pointing: Where Language, Culture and Cognition Meet. Erlbaum, Mahwah, NJ. Google Scholar logo with link to Google Scholar
Laner, Barbara. 2022. “Guck mal der Baum” — Zur Verwendung von Wahrnehmungsimperativen mit und ohne ‘mal’.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 23: 1–35.Google Scholar logo with link to Google Scholar
Luff, Paul, Christian Heath, Hideaki Kuzuoka, Jon Hindmarsh, Keiichi Yamazaki, and Oyama Shinya. 2003. “Fractured Ecologies: Creating Environments for Collaboration.” Human-Computer Interaction 18 (1–2): 51–85. Google Scholar logo with link to Google Scholar
Mondada, Lorenza. 2012. “Deixis: An Integrated Interactional Multimodal Analysis.” In Prosody and Embodiment in Interactional Grammar, ed. by Pia Bergmann, Jana Brenning, Martin Pfeiffer, and Elisabeth Reber, 173–206. Berlin, Boston: De Gruyter. Google Scholar logo with link to Google Scholar
. 2013a. “Interactional Space and the Study of Embodied Talk-in-Interaction.” In Space in Language and Linguistics. Geographical, Interactional, and Cognitive Perspectives, ed. by Peter Auer, Martin Hilpert, Anja Stukenbrock, and Benedikt Szmrecsanyi, 247–275. Berlin, Boston: De Gruyter. Google Scholar logo with link to Google Scholar
. 2013b. “The Conversation Analytic Approach to Data Collection.” In The Handbook of Conversation Analysis, ed. by Jack Sidnell, and Tanya Stivers, 32–56. Wiley-Blackwell, Chichester. Google Scholar logo with link to Google Scholar
. 2014a. “The Local Constitution of Multimodal Resources for Social Interaction.” Journal of Pragmatics 65: 137–156. Google Scholar logo with link to Google Scholar
. 2014b. “Instructions in the Operating Room: How the Surgeon Directs their Assistant’s Hands.” Discourse Studies 16: 131–161. Google Scholar logo with link to Google Scholar
. 2019. “Conventions for Transcribing Multimodality.” Accessed 13 March 2023. Available at: [URL]
Mondada, Lorenza, and Marja-Leena Sorjonen. 2021. “OKAY in Closings and Transitions.” In OKAY across Languages: Toward a Comparative Approach to its Use in Talk-in-Interaction, ed. by Emma Betz, Arnulf Deppermann, Lorenza Mondada, and Marja-Leena Sorjonen, 94–127. Amsterdam, John Benjamins. Google Scholar logo with link to Google Scholar
Moore, Chris, and Philip. J. Dunham (eds). 1995. Joint Attention: Its Origins and Role in Development. Lawrence Erlbaum Associates, Inc.Google Scholar logo with link to Google Scholar
Pfeiffer, Martin. 2016. “The Deictic Dimension of Exclamations: On the Use of wh-exclamatives in German Face-to-Face Interaction.” Revue de Sémantique et Pragmatique 40: 35–57. Google Scholar logo with link to Google Scholar
Quine, Willard van Orman. 1960. Word and Object. Cambridge, MA: Massachusetts Institute of Technology Press.Google Scholar logo with link to Google Scholar
Raymond, Geoffrey. 2003. “Grammar and Social Organization: Yes/No Interrogatives and the Structure of Responding.” American Sociological Review 68 (6): 939–967. Google Scholar logo with link to Google Scholar
Sacks, Harvey. 1992. Lectures on Conversation, Vol. I, ed. by Gail Jefferson. Oxford: Basil Blackwell.Google Scholar logo with link to Google Scholar
Schegloff, Emanuel A. 1998. “Body Torque.” Social Research 65, 535–596.Google Scholar logo with link to Google Scholar
2007. Sequence Organization in Interaction: A Primer in Conversation Analysis. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Selting, Margret, Peter Auer, Dagmar Barth-Weingarten, Jörg R. Bergmann, Pia Bergmann, Karin Birkner, Elizabeth Couper-Kuhlen, Arnulf Deppermann, Peter Gilles, Susanne Günthner, Martin Hartung, Friederike Kern, Christine Mertzlufft, Christian Meyer, Miriam Morek, Frank Oberzaucher, Jörg Peters, Uta Quasthoff, Wilfried Schütte, Anja Stukenbrock, and Susanne Uhmann. 2009. “Gesprächsanalytisches Transkriptionssystem 2 (GAT 2)” Gesprächsforschung 10: 353–402. Available at: [URL]
Streeck, Jürgen. 2002. “Grammars, Words, and Embodied Meanings: on the Uses and Evolution of so and like.” Journal of Communication 52: 581–596. Google Scholar logo with link to Google Scholar
. 2009. Gesturecraft: The manu-facture of meaning. Amsterdam / Philadelphia, John Benjamins Publishing. Google Scholar logo with link to Google Scholar
Streeck, Jürgen. 2014. “Mutual Gaze and Recognition: Revisiting Kendon’s ’Gaze Direction in Two-person Conversation.” From Gesture in Conversation to Visible Action as Utterance: Essays in Honor of Adam Kendon, ed. by Mandana Seyfeddinipur, and Marianne Gullberg, 35–56. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Streeck, Jürgen, Charles Goodwin, and Curtis LeBaron (eds). 2011. Embodied Interaction: Language and Body in the Material World. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Stukenbrock, Anja. 2014. “Take the Words out of my Mouth: Verbal Instructions as Embodied Practices.” Journal of Pragmatics 65: 80–102. Google Scholar logo with link to Google Scholar
. 2015. Deixis in der face-to-face-Interaktion. Berlin, Boston: De Gruyter. Google Scholar logo with link to Google Scholar
. 2018a. “Mobile Dual Eye-Tracking in Face-to-Face Interaction: The Case of Deixis and Joint Attention.” In Advances in Interaction Studies, ed. by Geert Brône, and Bert Oben, 265–300. Amsterdam, Philadelphia: Benjamins.Google Scholar logo with link to Google Scholar
. 2018b. “Forward-looking: Where Do We Go with Multimodal Projections?.” In Time in Embodied Interaction. Synchronicity and Sequentiality of Multimodal Resources, ed. by Arnulf Deppermann, and Jürgen Streeck, 31–68. Amsterdam, Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
. 2020. “Deixis, Meta-Perceptive Gaze Practices, and the Interactional Achievement of Joint Attention.” Frontiers in Psychology 11:1779. Google Scholar logo with link to Google Scholar
. 2023. “Temporality and the Cooperative Infrastructure of Human Communication: Noticings to Delay and to Accelerate Onward Movement in Mobile Interaction.” Language and Communication 92: 33–54. Google Scholar logo with link to Google Scholar
Stukenbrock, Anja, and Anh Nhi Dao. 2019. “Joint Attention in Passing. What Dual Mobile Eye-Tracking Reveals about Gaze in Coordinating Embodied Activities at the Market.” In Embodied Activities in Face-to-Face and Mediated Settings, ed. by Cornelia Gerhardt, and Elisabeth Reber, 177–213. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Tomasello, Michael. 2008. Origins of Human Communication. Cambridge, MA: MIT Press. Google Scholar logo with link to Google Scholar
Vom Lehn, Dirk. 2013. “Withdrawing from Exhibits: The Interactional Organisation of Museum Visits.” In Interaction and Mobility: Language and the Body in Motion, ed. by Pentti Haddington, Lorenza Mondada, and Maurice Nevile, 65–90. Berlin, Boston: De Gruyter. Google Scholar logo with link to Google Scholar
Wittenburg, Peter, Hennie Brugman, Albert Russel, Alex Kalssmann, and Han Sloetjes. 2006. “ELAN: a Professional Framework for Multimodality Research.” In Proceedings of the Fifth International Conference on Language Resources and Evaluation, 1556–1559.Google Scholar logo with link to Google Scholar
Cited by (1)

Cited by one other publication

Zima, Elisabeth, Peter Auer & Christoph Rühlemann
2025. Why research on gaze in social interaction needs mobile eye tracking. In Mobile Eye Tracking [Pragmatics & Beyond New Series, 351], ► pp. 24 ff. DOI logo

This list is based on CrossRef data as of 14 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue