Cover not available

In:Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 233–264

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

Chapter 10
Gaze as a predictor for lexical and gestural alignment

Published online: 13 November 2018
DOI logo with link
https://doi.org/10.1075/ais.10.10obe
Abstract
This chapter provides evidence for the role gaze might have on behaviour at another multimodal level of behaviour, viz. alignment (or copying behaviour) of hand gestures and lexical items. In a corpus of dyadic interactions we demonstrate that gaze behaviour affects alignment behaviour differently at the lexical than at the gestural level: if a speaker is looking at an addressee’s face while uttering a target word, this significantly increases the probability that the addressee will use that same word later in the conversation. If a speaker is looking at an addressee’s face while performing a target gesture, there is no correlation with subsequent gesture production by that addressee. However, if an addressee looked at a gesture made by a speaker, this gesture was significantly more often used by that addressee later in the conversation (compared to situations in which the addressee was not looking at that gesture). We argue that the difference in gaze behaviour between lexical and gestural alignment cases might be explained by the dual function of eye gaze in interaction, viz. gaze for visual perception and gaze for signalling meaning.
Article outline
  • 1.Introduction
    • 1.1Eye gaze during conversation
    • 1.2Alignment during conversation
  • 2.Case study 1: Gaze and lexical alignment
    • 2.1Method
    • 2.2Analysis
    • 2.3Results
    • 2.4Discussion
  • 3.Case study 2: Gaze and gestural alignment
    • 3.1Method & analysis
    • 3.2Results
    • 3.3Discussion
  • 4.Conclusion & future work
  • Note
  • References
References (70)
References
Argyle, M. & Cook, M. (1976). Gaze and Eye contact. London: Cambridge University Press.Google Scholar logo with link to Google Scholar
Argyle, M. & Ingham, R. (1972). Gaze, mutual gaze, and proximity. Semiotica, 6, 32–49.Google Scholar logo with link to Google Scholar
Bateson, M., Nettle, D. & Roberts, G. (2006). Cues of being watched enhance cooperation in real-world setting. Biology Letters, 2, 412–414.Google Scholar logo with link to Google Scholar
Beckner, C., Blythe, R., Bybee, J., Christiansen, M., Croft, W., Ellis, N., Holland, J., Ke, J., Larsen-Freeman, D. & Schoenemann, T. (2009). Language Is a Complex Adaptive System: Position Paper. Language Learning, 59, 1–26.Google Scholar logo with link to Google Scholar
Bergmann, K. & Kopp, S. (2012). Gestural alignment in natural dialogue. Proceedings of the 34th Annual Conference of the Cognitive Science Society, 1326–1331.Google Scholar logo with link to Google Scholar
Branigan, H., Pickering, M. & Cleland, A. (2000). Syntactic co-ordination in dialogue. Cognition, 75, 13–25.Google Scholar logo with link to Google Scholar
Brennan S., Chen X., Dickinson C., Neider M. & Zelinsky G. (2008). Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition, 106, 1465–1477.Google Scholar logo with link to Google Scholar
Brennan, S. & Clark, H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 1482–1493.Google Scholar logo with link to Google Scholar
Brône, G., Oben, B. (2015). InSight interaction: A multimodal and multifocal dialogue corpus. Language Resources and Evaluation, 49(1), 195–214.Google Scholar logo with link to Google Scholar
Clark, H. (1996). Using Language. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Chartrand, T. & Bargh, J. (1999). The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology, 76, 893–910.Google Scholar logo with link to Google Scholar
Cummins, F. (2011). Gaze and blinking in dyadic conversation: A study in coordinated behaviour among individuals. Language and Cognitive Processes, 27, 1525–1549.Google Scholar logo with link to Google Scholar
Dale, R., Fusaroli, R., Duran, N. & Richardson, D. (2014). The self-organization of human interaction. In B. Ross (Ed.), Psychology of Learning and Motivation (pp.43–95). Academic Press.Google Scholar logo with link to Google Scholar
Dale, R., Kirkham, N. & Richardson, D. (2011). How two people become a tangram recognition system. Proceedings of the European Conference on Computer-Supported Cooperative Work. Berlin: Springer Verlag.Google Scholar logo with link to Google Scholar
Dale, R. & Spivey, M. (2006). Unraveling the dyad: Using recurrence analysis to explore patterns of syntactic coordination between children and caregivers in conversation. Language Learning, 56, 391–430.Google Scholar logo with link to Google Scholar
Dominey, P. (2004). Situation alignment and routinization in language acquisition. Behavioral and Brain Sciences, 27, 195.Google Scholar logo with link to Google Scholar
Duchowsky, A. (2007). Eye Tracking Methodology. Theory and Practice. Berlin: Springer.Google Scholar logo with link to Google Scholar
Emery, N. (2000). The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews, 24, 581–604.Google Scholar logo with link to Google Scholar
Frischen, A., Bayliss, A., & Tipper, S. (2007). Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychological Bulletin, 133, 694–724.Google Scholar logo with link to Google Scholar
Fusaroli, R., Bahrami, B., Olsen, K., Roepstorff, A., Rees, G., Frith, C. & Tylén, K. (2012). [URL] . Psychological Science, 23, 931–939.Google Scholar logo with link to Google Scholar
Fusaroli, R., Konvalinka, I. & Wallot, S. (2014). Analyzing social interactions: The promises and challenges of using cross recurrence quantification analysis. Springer Proceedings in Mathematics and Statistics, 103, 137–155.Google Scholar logo with link to Google Scholar
Fusaroli, R. & Tylén, K. (2015). Investigating conversational dynamics: interactive alignment, interpersonal synergy and collective task performance. Cognitive Science, 40, 145–171.Google Scholar logo with link to Google Scholar
Garrod, S. & Anderson, A. (1987). Saying what you mean in dialogue: A study in conceptual and semantic co-ordination. Cognition, 27, 181–218.Google Scholar logo with link to Google Scholar
Giles, H. & Powesland, P. (1975). Speech styles and social evaluation. New York: Academic Press.Google Scholar logo with link to Google Scholar
Gries, S. (2005). Syntactic Priming: A Corpus-based Approach. Journal of Psycholinguistic Research, 34, 365–399.Google Scholar logo with link to Google Scholar
Gullberg, M. & Kita, S. (2009). Attention to speech-accompanying gestures: Eye movements and information uptake. Journal of Nonverbal Behaviour, 33, 251–277.Google Scholar logo with link to Google Scholar
Hadelich, K. & Crocker, M. (2006). Gaze alignment of interlocutors in conversational dialogues. Proceedings of the 2006 Symposium on Eye Tracking Research and Applications, 38.Google Scholar logo with link to Google Scholar
Jakobson, R. (1960). Linguistics and Poetics. In T. Sebeok (Ed.), Style in Language (pp.350–377). Cambridge MA: M.I.T. Press.Google Scholar logo with link to Google Scholar
James, W. (1878). Brute and human intellect. The Journal of Speculative Philosphy, 12, 236–276.Google Scholar logo with link to Google Scholar
Jones, B., DeBruine, L., Little, A., Conway, C. & Feinberg, D. (2006). Integrating gaze direction and expression in preferences for attractive faces. Psychological Science, 17, 588–591.Google Scholar logo with link to Google Scholar
Kendon, A. (1967) Some functions of gaze-direction in social interaction. Acta Psychologica, 26, 22–63.Google Scholar logo with link to Google Scholar
Kimbara, I. (2008). Gesture Form Convergence in Joint Description. Journal of Nonverbal Behavior, 32, 123–131.Google Scholar logo with link to Google Scholar
Lachat, F., Conty, L., Hugueville, L. & George, N. (2012). Gaze cueing effect in a face-to-face situation. Journal of Nonverbal Behaviour, 36, 177–190.Google Scholar logo with link to Google Scholar
Langton, S., Watt, R. & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends in cognitive sciences, 4, 50–58.Google Scholar logo with link to Google Scholar
Lewandowski, N. (2012). Talent in nonnative phonetic convergence (Unpublished doctoral dissertation). Universität Stuttgart, Stuttgart.Google Scholar logo with link to Google Scholar
Louwerse, M., Dale, R., Bard, E. & Jeuniaux, P. (2012). Behavior matching in multimodal communication is synchronized. Cognitive Science, 36, 1404–1426.Google Scholar logo with link to Google Scholar
Manson, J., Bryant, G., Gervais, M. & Kline, M. (2013). Convergence of speech rate in conversation predicts cooperation. Evolution and Human Behavior, 34, 419–426.Google Scholar logo with link to Google Scholar
McNeill, D. (2006). Gesture, Gaze, and Ground. Lecture Notes in Computer Science 3869, 1-14.Google Scholar logo with link to Google Scholar
Mathews, A., Fox, E., Yiend, J. & Calder, A. (2003). The face of fear: Effects of eye gaze and emotion on visual attention. Visual Cognition, 10, 823–835.Google Scholar logo with link to Google Scholar
Mol, L., Krahmer, E., Maes, A. & Swerts, M. (2012). Adaptation in gesture: Converging hands or converging minds? Journal of Memory and Language, 66, 249–264.Google Scholar logo with link to Google Scholar
Muir, L. & Richardson, I. (2005). Perception of sign language and its application to visual communications for deaf people. Journal of Deaf Studies and Deaf Education, 10, 390–340.Google Scholar logo with link to Google Scholar
Nielsen, G. (1962). Studies in self confrontation. Copenhagen: Munksgaard.Google Scholar logo with link to Google Scholar
Neider, M., Chen, X., Dickinson, C., Brennan, S. & Zelinsky, G. (2010). Coordinating spatial referencing using shared gaze. Psychonomic Bulletin & Review, 17, 718–724.Google Scholar logo with link to Google Scholar
Oben, B. (2015). Modelling interactive alignment: a multimodal and temporal account. [unpublished doctoral thesis]Google Scholar logo with link to Google Scholar
Oben B., Brône G. (2015). What you see is what you do: On the relation between gaze and gesture in multimodal alignment. Language and Cognition, 7(4), 546–562.Google Scholar logo with link to Google Scholar
Paxton, A. & Dale, R. (2013). Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45, 329–343.Google Scholar logo with link to Google Scholar
Paxton, A., Abney, D, Kello, C. & Dale, R. (2014). Network analysis of multimodal, multiscale coordination in dyadic problem solving. In P. Bello, M. Guarini, M. McShane & B. Scassellati (Eds.), Proceedings of the 36th Annual Meeting of the Cognitive Science Society (pp.2735–2740). Austin: Cognitive Science Society.Google Scholar logo with link to Google Scholar
Piaget, J. (1932). The moral judgment of the child. Glencoe: The Free Press.Google Scholar logo with link to Google Scholar
Piazza, J., Bering, J. & Ingram, G. (2011). Princess Alice is watching you: Children’s belief in an invisible person inhibits cheating. Journal of Experimental Child Psychology, 109, 311–320.Google Scholar logo with link to Google Scholar
Pickering, M. & Garrod, S. (2004). Towards a Mechanistic Psychology of Dialogue. Behavioural and Brain Sciences, 27, 169–225.Google Scholar logo with link to Google Scholar
(2006). Alignment as the Basis for Successful Communication. Research on Language and Communication, 4, 203–288.Google Scholar logo with link to Google Scholar
Costa, A., Pickering, M. & Sorace, A. (2008) Alignment in second language dialogue. Language and Cognitive Processes, 23, 528-556.Google Scholar logo with link to Google Scholar
Posner, M., Snyder, C. & Davidson, B. (1980). Attention and the detection of signals. Journal of Experimental Psychology, 109, 160–174.Google Scholar logo with link to Google Scholar
Postma, M., Brunninkhuis, N. & Postma, E. (2013). Eye Gaze Affects Vocal Intonation Mimicry. Proceedings of the 35th Annual Conference of the Cognitive Science Society, 1139–1144.Google Scholar logo with link to Google Scholar
Powell, L., Roberts, G. & Nettle, D. (2012). Eye images increase charitable donations: Evidence from an opportunistic field experiment in a supermarket. Ethology, 118, 1–6.Google Scholar logo with link to Google Scholar
Reitter, D., Moore, D. & Keller, F. (2006). Priming of syntactic rules in task-oriented dialogue and spontaneous conversation. Proceedings of the 28th Annual Conference of the Cognitive Science Society, 685–690.Google Scholar logo with link to Google Scholar
Richardson, D. & Dale, R. (2005). Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive Science, 29, 1045–1060.Google Scholar logo with link to Google Scholar
Richardson, D., Dale, R., & Kirkham, N. (2007). The art of conversation is coordination. Common ground and the coupling of eye movements during dialogue. Psychological Science, 18, 407–413.Google Scholar logo with link to Google Scholar
Richardson, D., Dale, R. & Tomlinson, J. (2009). Conversation, gaze coordination, and beliefs about visual context. Cognitive Science, 33, 1468–1482.Google Scholar logo with link to Google Scholar
Roche, J., Dale, R. & Caucci, G. (2010). Doubling up on double meanings: Pragmatic alignment. Language and Cognitive Processes, 27, 1–24.Google Scholar logo with link to Google Scholar
Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.Google Scholar logo with link to Google Scholar
Shockley, K., Santana, M. & Fowler, C. (2003). Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29, 326–332.Google Scholar logo with link to Google Scholar
Szczepek Reed, B. (2010). Prosody and alignment: A sequential perspective. Cultural Studies of Science Education, 5, 859–867.Google Scholar logo with link to Google Scholar
Tollefsen, D. & Dale, R. (2012). Naturalizing joint action: A process-based approach. Philosophical Psychology, 25, 385–407.Google Scholar logo with link to Google Scholar
Van Engen, K., Baese-Berk, M., Baker, R., Choi, A., Kim, M. & Bradlow, A. (2010). The Wildcat corpus of native- and foreign-accented English: Communicative efficiency across conversational dyads with varying language alignment profiles. Language and speech, 53, 510–540.Google Scholar logo with link to Google Scholar
Wang, Y., Newport, R. & Hamilton, A. (2011). Eye contact enhances mimicry of intransitive hand movements. Biology Letters, 7, 7–10.Google Scholar logo with link to Google Scholar
Wang, Y. & Hamilton, A. (2014). Why does gaze enhance mimicry? Placing gaze-mimicry effects in relation to other gaze phenomena. The Quarterly Journal of Experimental Psychology, 67, 747–762.Google Scholar logo with link to Google Scholar
Zima, E. (2013). Kognition in der Interaktion. Eine kognitiv-linguistische und gesprächsanalytische Studie dialogischer Resonanz in österreichischen Parlamentsdebatten. Heidelberg: Universitätsverlag Winter.Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Stukenbrock, Anja & Elisabeth Zima
2025. Introduction. In Mobile Eye Tracking [Pragmatics & Beyond New Series, 351], ► pp. 1 ff. DOI logo
Pleyer, Michael
2023. The role of interactional and cognitive mechanisms in the evolution of (proto)language(s). Lingua 282 ► pp. 103458 ff. DOI logo
Rasenberg, Marlou, Asli Özyürek & Mark Dingemanse
2020. Alignment in Multimodal Interaction: An Integrative Framework. Cognitive Science 44:11 DOI logo

This list is based on CrossRef data as of 14 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue