Cover not available

In:Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 47–66

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

Chapter 3
Effects of a speaker’s gaze on language comprehension and acquisition

Published online: 13 November 2018
DOI logo with link
https://doi.org/10.1075/ais.10.03kno
Abstract
This chapter discusses the role of a speaker’s gaze on a listener’s language processing, recall of information, and on child language learning. Speaker gaze facilitates performance in all of these domains, which suggests that it plays an important role in communication. Indeed, the findings indicate that speaker gaze can facilitate not just referential but also compositional processes such as syntactic structuring and thematic role assignment in listeners. Speaker gaze even guided comprehenders’ attention to a target object more rapidly than other aspects of the visual context (action event depictions); but unlike the action depictions, it had no beneficial effects on comprehenders’ post-experiment memory in socially non-interactive settings (when comprehenders inspect the speaker on a computer display). In socially interactive settings, however, speaker gaze seemed to affect memory in that joint object-based attention between a caregiver and child was beneficial for language learning. We conclude that speaker gaze effects are important for both language processing and learning, and are potentially boosted in a socially interactive context.
Article outline
  • 1.Introduction
  • 2.The role of speaker eye gaze: Syntactic structuring and thematic role assignment
  • 3.Immediate attention and later recall: Speaker gaze vs. action depictions
  • 4.Speaker gaze and joint attention in child language processing and learning
  • 5.The role of speaker gaze: Social and pragmatic considerations
  • Acknowledgements
  • References
References (67)
References
Baldwin, D. (1993a). Infants’ ability to consult the speaker for clues to word reference. Journal of Child Language, 2, 395–418.Google Scholar logo with link to Google Scholar
(1993b). Early referential understanding: Young children’s ability to recognize referential acts for what they are. Developmental Psychology, 29, 1–12.Google Scholar logo with link to Google Scholar
Baldwin, D. A. (2000). Interpersonal understanding fuels knowledge acquisition. Current Directions in Psychological Science, 9, 40–45.Google Scholar logo with link to Google Scholar
Brooks, R., & Meltzoff, A. N. (2005). The development of gaze following and its relation to language. Developmental Science, 8, 535–543.Google Scholar logo with link to Google Scholar
Carpenter, M., Nagell, K., & Tomasello, M. (1998). Social cognition, joint attention, and communicative competence from 9 to 15 months of age. Monographs of the Society for Research in Child Development, 63, 1–174. Google Scholar logo with link to Google Scholar
Chambers, C. G., Tanenhaus, M. K., & Magnuson, J. S. (2004). Actions and affordances in syntactic ambiguity resolution. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 687–696.Google Scholar logo with link to Google Scholar
Conboy, B. T., Brooks, R., Meltzoff, A. N., & Kuhl, P. K. (2015) Social Interaction in Infants’ Learning of Second-Language Phonetics: An Exploration of Brain-Behavior Relations, Developmental Neuropsychology, 40, 216–229.Google Scholar logo with link to Google Scholar
Cooper, R. M. (1974). The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology, 6, 84–107.Google Scholar logo with link to Google Scholar
Dahan, D., & Tanenhaus, M. (2005). Looking at the rope when looking for the snake: Conceptually mediated eye movements during spoken-word recognition. Psychonomic Bulletin & Review, 12, 453–459.Google Scholar logo with link to Google Scholar
Doherty-Sneddon, G., & Phelps, F. G. (2005). Gaze aversion: A response to cognitive or social difficulty? Memory & Cognition, 33, 727–733.Google Scholar logo with link to Google Scholar
(2007). Teachers’ responses to children’s eye gaze. Educational Psychology, 27, 93–109.Google Scholar logo with link to Google Scholar
Driver, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S. (1999). Gaze perception triggers reflexive visuospatial orienting. Visual Cognition, 6, 509–540.Google Scholar logo with link to Google Scholar
Farroni, T., Csibra, G., Simion, F., & Johnson, M. H. (2002). Eye contact detection in humans from birth. Proceedings of the National Academy of Sciences of the United States of America, 99, 9602–9605. Google Scholar logo with link to Google Scholar
Griffin, Z. M., & Bock, K. (2000). What the eyes say about speaking. Psychological Science, 11, 274–279.Google Scholar logo with link to Google Scholar
Garrod, S., & Pickering, M. J. (2009). Joint action, interactive alignment, and dialog. Topics in Cognitive Science, 1, 292–304.Google Scholar logo with link to Google Scholar
Glenberg, A. M., Shroeder, J. L., & Robertson, D. A. (1998). Averting the gaze disengages the environment and facilitates remembering. Memory & Cognition, 26, 651–658.Google Scholar logo with link to Google Scholar
Gleitman. L. R., (1990). The structural sources of verb meaning. Language Acquisition 1, 3–55.Google Scholar logo with link to Google Scholar
Hanna, J., and Brennan, S. (2007). Speaker’s eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 57, 596–615.Google Scholar logo with link to Google Scholar
Huettig, F., Rommers, J., & Meyer, A. S. (2011). Using the visual world paradigm to study language processing: A review and critical evaluation. Acta psychologica, 137, 151–171.Google Scholar logo with link to Google Scholar
Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye- movements in depicted events. Cognition, 95, 95–127.Google Scholar logo with link to Google Scholar
Knoeferle, P., & Crocker, M. W. (2007). The influence of recent scene events on spoken comprehension: evidence from eye movements. Journal of Memory and Language, 57, 519–543.Google Scholar logo with link to Google Scholar
(2006). The coordinated interplay of scene, utterance, and world knowledge: evidence from eye-tracking. Cognitive Science, 30, 481–529.Google Scholar logo with link to Google Scholar
Knoeferle, P. & Guerra, E. (2012). What’s non-linguistic visual context? – A view from language comprehension. In: Rita Finkbeiner, Jörg Meibauer, & Petra Schuhmacher (eds) What is a context? Linguistic approaches and challenges (pp.129–149). Amsterdam: John Benjamins.Google Scholar logo with link to Google Scholar
(2016). Visually situated language comprehension. Language and Linguistics Compass, 10, 66–82.Google Scholar logo with link to Google Scholar
Knoeferle, P., Habets, B., Crocker, M. W., Muente, T. F. (2008). Visual scenes trigger immediate syntactic reanalysis: evidence from ERPs during situated spoken comprehension. Cerebral Cortex, 18, 789–795.Google Scholar logo with link to Google Scholar
Knoeferle, P. & Kreysa, H. (2012). Effects of speaker gaze on syntactic structuring. Frontiers in Psychology, 2:376. .Google Scholar logo with link to Google Scholar
Knoeferle, P., Urbach, T, & Kutas, M (2011). Comprehending visual context influences on incremental sentence comprehension: insights from ERPs and picture-sentence verification. Psychophysiology, 48, 495–506.Google Scholar logo with link to Google Scholar
Knoeferle, P., Urbach, T., & Kutas, M. (2014). Different mechanisms for role relations versus verb-action congruence effects: Evidence from ERPs in picture-sentence verification. Acta Psychologica, 152, 133–148.Google Scholar logo with link to Google Scholar
Kreysa, H., & Knoeferle, P. (2011). Effects of speaker gaze on spoken language comprehension: Task matters. In: L. Carlson, C. Hölscher & T.F. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp.1557–1562), Austin, TX: The Cognitive Science Society.Google Scholar logo with link to Google Scholar
Kreysa, H., Knoeferle, P., & Nunnemann, E. (2014). Effects of speaker gaze versus depicted actions on visual attention during sentence comprehension. In: Paul Bello, Marcello Guarini, Marjorie McShane & Brian Scassellati (Eds). Proceedings of the 36th Annual Meeting of the Cognitive Science Society, (pp.2513–2518). Austin, TX: Cognitive Science Society.Google Scholar logo with link to Google Scholar
Kreysa, H., Nunnemann, E., & Knoeferle, P. (2018). Distinct effects of different visual cues on sentence comprehension and later recall: The case of speaker gaze versus depicted actions. Acta Psychologica. Google Scholar logo with link to Google Scholar
Kreysa, H., & Pickering M. J. (2011). Eye movements in dialogue. In S. P. Liversedge, I. D. Gilchrist, & S. Everling (Eds.), The Oxford Handbook of Eye Movements (p.943–959). Oxford: OUP.Google Scholar logo with link to Google Scholar
Kuhl, P. K., Tsao, F. -M., & Liu, H. -M. (2003). Foreign-language experience in infancy: Effects of short-term exposure and social interaction on phonetic learning. Proceedings of the National Academy of Sciences, 100, 9096–9101.Google Scholar logo with link to Google Scholar
Kuhl, P. K. (2007). Is speech learning “gated” by the social brain? Developmental Science, 10, 110–120.Google Scholar logo with link to Google Scholar
(2011). Social mechanisms in early language acquisition: Understanding integrated brain systems supporting language. In J. Decety & J. Cacioppo (Eds.), The Oxford handbook of social neuroscience (pp.649–667). Oxford, UK: Oxford University Press.Google Scholar logo with link to Google Scholar
Langton, S. R. H. and Bruce, V. (2000). You must see the point: automatic processing of cues to the direction of social attention. Journal of Experimental Psychology: HPP, 26, 747–757.Google Scholar logo with link to Google Scholar
Langton, S., Watt, R. J., & Bruce, V. (2000). Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Sciences, 4, 50–59.Google Scholar logo with link to Google Scholar
Macdonald, R. G., & Tatler, B. W. (2013). Do as eye say: Gaze cueing and language in a real-world social interaction. Journal of Vision, 13, 1–12.Google Scholar logo with link to Google Scholar
Meyer, A. S., Sleiderink, A. M., & Levelt, W. J. M. (1998). Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66, B25–B33.Google Scholar logo with link to Google Scholar
Münster K. and Knoeferle P. (2018) Extending Situated Language Comprehension (Accounts) with Speaker and Comprehender Characteristics: Toward Socially Situated Interpretation. Frontiers in Psychology. 8: 2267. .Google Scholar logo with link to Google Scholar
Nappa, R., Wessel, A., McEldoon, K. L., Gleitman, L., & Trueswell, J. C. (2009). Use of Speaker’s Gaze and Syntax in Verb Learning. Language Learning and Development, 5, 203–234.Google Scholar logo with link to Google Scholar
Neider, M. B., Chen, X., Dickinson, C. A., Brennan, S., & Zelinsky, G. (2010). Coordinating spatial referencing using shared gaze. Psychological Bulletin & Review, 17, 718–724.Google Scholar logo with link to Google Scholar
Phillips A. T., Wellman H. M., & Spelke E. S. (2002). Infants’ ability to connect gaze and emotional expression to intentional action. Cognition, 85, 53–78.Google Scholar logo with link to Google Scholar
Pickering, M. J., & Garrod, S. (2004). Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27, 169–225.Google Scholar logo with link to Google Scholar
Pickering, M., & Garrod, S. (2007). Do people use language production to make predictions during comprehension? Trends in Cognitive Sciences, 11, 105–110.Google Scholar logo with link to Google Scholar
Pickering, M. J., & Garrod, S. (2009). Language, interaction and embodiment. European Journal of Social Psychology, 39, 1178–1179.Google Scholar logo with link to Google Scholar
Ricciardelli, P., Bricolo, E., Aglioti, S., & Chelazzi, L. (2002). My eyes want to look where your eyes are looking: Exploring the tendency to imitate another individual’s gaze. NeuroReport, 13, 2259–2264.Google Scholar logo with link to Google Scholar
Rosenthal Rollins, P., & Snow, C. (1998). Shared attention and grammatical development in typical children and children with autism. Journal of Child Language, 25, 653–673.Google Scholar logo with link to Google Scholar
Samson, D., Apperly, I. A., Braithwaite, J. J., Andrews, B. J., & Bodley Scott, S. E. (2010). Seeing it their way: evidence for rapid and involuntary computation of what other people see. Journal of Experimental Psychology: Human Perception and Performance, 36, 1255–1266.Google Scholar logo with link to Google Scholar
Santiesteban, I., Catmur, C., Hopkins, S. C., Bird, G., & Heyes, C. (2014). Avatars and arrows: Implicit mentalizing or domain-general processing? Journal of Experimental Psychology: Human Perception and Performance, 40, 929–937.Google Scholar logo with link to Google Scholar
Saxton, M. (2010). Child Language: Acquisition and Development. London: Sage Publications.Google Scholar logo with link to Google Scholar
Scaife, M., & Bruner, J. S. (1975). The capacity for joint visual attention in the infant. Nature, 253, 265–266.Google Scholar logo with link to Google Scholar
Spivey, M., Tanenhaus, M., Eberhard, K., & Seidvy, J. (2002). Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45, 447–481.Google Scholar logo with link to Google Scholar
Staudte, M., and Crocker, M. W. (2011). Investigating joint attention mechanisms through spoken human–robot interaction. Cognition 120, 268–291.Google Scholar logo with link to Google Scholar
Staudte, M., Crocker, M. W., Heloir, A., & Kipp, M. (2014). The influence of speaker gaze on listener comprehension: Contrasting visual versus intentional accounts, Cognition, 133, 317–328.Google Scholar logo with link to Google Scholar
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 632–634.Google Scholar logo with link to Google Scholar
Tomasello, M. (2003). Constructing a language: A usage-based theory of language acquisition. Cambridge, MA: Harvard University Press.Google Scholar logo with link to Google Scholar
Tomasello, M., Carpenter, M., Call, J., Behne, T., & Moll, H. (2005). Understanding and sharing intentions: The origins of cultural cognition. Behavioral and Brain Sciences, 28, 675–691.Google Scholar logo with link to Google Scholar
Tomasello, M., & Farrar, J. (1986). Joint attention and early language. Child Development, 57, 1454–1463.Google Scholar logo with link to Google Scholar
Tomasello, M. & Todd, J. (1983). Joint attention and lexical acquisition style. First Language, 4, 197–211. Google Scholar logo with link to Google Scholar
Trueswell, J. C., Sekerina, I., Hill, N. M., & Logrip, M. L. (1999). The kindergarten-path effect: Studying on-line sentence processing in young children. Cognition, 73, 89–13.Google Scholar logo with link to Google Scholar
Woodward, A., & Sommerville, J. (2000). Twelve-month-old infants interpret actions in context. Psychological Science, 11, 73–77.Google Scholar logo with link to Google Scholar
Woodward, A. L. (2003). Infants’ developing understanding of the link between looker and object. Developmental Science, 6, 297–311.Google Scholar logo with link to Google Scholar
Woodward, A. (1998). Infants selectively encode the goal object of an actor’s reach. Cognition, 69, 1–34.Google Scholar logo with link to Google Scholar
Wu, R., Gopnik, A., Richardson, D. C., & Kirkham, N. Z. (2011). Infants learn about objects from statistics and people. Developmental Psychology, 47, 1220–1229.Google Scholar logo with link to Google Scholar
Yu, C., Ballard, D. H., & Aslin, R. N. (2005). The role of embodied intention in early lexical acquisition. Cognitive Science, 29, 961–1005.Google Scholar logo with link to Google Scholar
Zhang, Lu & Knoeferle, P. (2012). Visual Context Effects on Thematic Role Assignment in Children versus Adults: Evidence from Eye Tracking in German. In: Naomi Miyake, David Peebles, & Richard P. Cooper, Proceedings of the Annual Meeting of the Cognitive Science Society (pp.2593–2598), Boston, USA: The Cognitive Science Society.Google Scholar logo with link to Google Scholar
Cited by (1)

Cited by one other publication

Janjić, Paula, Gozdem Arikan, Harmen B. Gudde, Joseph J.C. Murphy, Laksha Sivaram & Kenny R. Coventry
2025. The role of pointing gestures and eye gaze in second language vocabulary learning. Discourse Processes 62:8-9 ► pp. 673 ff. DOI logo

This list is based on CrossRef data as of 13 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue