Cover not available

In:Mobile Eye Tracking: New avenues for the study of gaze in social interaction
Edited by Elisabeth Zima and Anja Stukenbrock
[Pragmatics & Beyond New Series 351] 2025
► pp. 188–205

Get fulltext
PDF logo with download linkDownload Chapter PDF
EPUB logo with download linkDownload Chapter EPUB

Chapter 7
Pupil size indicates planning effort at turn transitions in natural conversation

 | Leibniz-Institute for the German LanguageMannheim
 | University of Freiburg
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 13 May 2025
DOI logo with link
https://doi.org/10.1075/pbns.351.07bar
Abstract
The study investigates the cognitive demands of speech planning in unrestricted, natural conversation. Focusing on question-answer sequences in triadic interactions, we analyse whether answerers, compared to not-answerers, exhibit increased cognitive effort during turn transitions. Using pupil size data from the Freiburg Multimodal Interaction Corpus, we find that answerers indeed show greater pupil dilation than not-answerers, suggesting heightened processing load during speech planning at transition-relevance places. This finding supports the hypothesis that speech planning is a primary contributor to increased cognitive effort during turn transitions, highlighting the value of pupillometry in the study of naturalistic conversation. The findings offer insights into the cognitive dynamics of multiparty social interaction, bridging the gap between controlled experiments and ecologically valid conversational settings.
Article outline
  • 1.Introduction
  • 2.Methods
    • 2.1Data collection
    • 2.2Data Pre-processing and statistical analysis
  • 3.Results
  • 4.Discussion
  • 5.Conclusion
  • Notes
  • References
  • Appendix
References (66)
References
. 2021b. “Turn-Allocation and Gaze: A Multimodal Revision of the ‘Current-Speaker-Selects-next’ Rule of the Turn-Taking System of Conversation Analysis.” Discourse Studies 23 (2): 117–40. Google Scholar logo with link to Google Scholar
Barthel, Mathias. 2020. “Speech Planning in Dialogue — Psycholinguistic Studies of the Timing of Turn Taking.” PhD Thesis, Nijmegen: Radboud University Nijmegen. [URL]
. 2021. “Speech Planning Interferes with Language Comprehension: Evidence from Semantic Illusions in Question-Response Sequences.” In Proceedings of the 25th Workshop on the Semantics and Pragmatics of Dialogue, 1–14. Potsdam, Germany.Google Scholar logo with link to Google Scholar
Barthel, Mathias, and Stephen C. Levinson. 2020. “Next Speakers Plan Word Forms in Overlap with the Incoming Turn: Evidence from Gaze-Contingent Switch Task Performance.” Language, Cognition and Neuroscience 35 (9): 1183–1202. Google Scholar logo with link to Google Scholar
Barthel, Mathias, Antje S. Meyer, and Stephen C. Levinson. 2017. “Next Speakers Plan Their Turn Early and Speak after Turn-Final ‘Go-Signals.’” Frontiers in Psychology 8: 393. Google Scholar logo with link to Google Scholar
Barthel, Mathias, and Sebastian Sauppe. 2019. “Speech Planning at Turn Transitions in Dialog Is Associated with Increased Processing Load.” Cognitive Science 43 (7): e12768. Google Scholar logo with link to Google Scholar
Barthel, Mathias, Sebastian Sauppe, Stephen C. Levinson, and Antje S. Meyer. 2016. “The Timing of Utterance Planning in Task-Oriented Dialogue: Evidence from a Novel List-Completion Paradigm.” Frontiers in Psychology 7: 1858. Google Scholar logo with link to Google Scholar
Bates, Douglas, Martin Mächler, Ben Bolker, and Steve Walker. 2015. “Fitting Linear Mixed-Effects Models Using Lme4.” Journal of Statistical Software 67 (1). Google Scholar logo with link to Google Scholar
Beatty, Jackson. 1982. “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources.” Psychological Bulletin 91 (2): 276–92. Google Scholar logo with link to Google Scholar
Boersma, Paul, and David Weenink. 2015. Praat: Doing Phonetics by Computer [Computer Program]. Version 5.3.56, [URL]
Bögels, Sara. 2020. “Neural Correlates of Turn-Taking in the Wild: Response Planning Starts Early in Free Interviews.” Cognition 203 (October): 104347. Google Scholar logo with link to Google Scholar
Bögels, Sara, and Stephen C. Levinson. 2017. “The Brain Behind the Response: Insights Into Turn-Taking in Conversation From Neuroimaging.” Research on Language and Social Interaction 50 (1): 71–89. Google Scholar logo with link to Google Scholar
Bögels, Sara, Lilla Magyari, and Stephen C. Levinson. 2015. “Neural Signatures of Response Planning Occur Midway through an Incoming Question in Conversation.” Scientific Reports 5 (12881): 1–11. Google Scholar logo with link to Google Scholar
Boiteau, Timothy W., Patrick S. Malone, Sara A. Peters, and Amit Almor. 2014. “Interference between Conversation and a Concurrent Visuomotor Task.” Journal of Experimental Psychology: General 143 (1): 295–311. Google Scholar logo with link to Google Scholar
Bradley, Margaret M., Laura Miccoli, Miguel A. Escrig, and Peter J. Lang. 2008. “The Pupil as a Measure of Emotional Arousal and Autonomic Activation.” Psychophysiology 45 (4): 602–7. Google Scholar logo with link to Google Scholar
Drews, Frank A., Monisha Pasupathi, and David L. Strayer. 2008. “Passenger and Cell Phone Conversations in Simulated Driving.” Journal of Experimental Psychology: Applied 14 (4): 392–400. Google Scholar logo with link to Google Scholar
Engelhardt, Paul E., Fernanda Ferreira, and Elena G. Patsenko. 2010. “Pupillometry Reveals Processing Load during Spoken Language Comprehension.” Quarterly Journal of Experimental Psychology 63 (4): 639–45. Google Scholar logo with link to Google Scholar
Ford, Cecilia E., Barbara A. Fox, and Sandra A. Thompson. 1996. “Practices in the Construction of Turns: The ‘TCU’ Revisited.” Pragmatics 6 (3): 427–54.Google Scholar logo with link to Google Scholar
Ford, Cecilia E., and Sandra A. Thompson. 1996. “Interactional Units in Conversation: Syntactic, Intonational, and Pragmatic Recources for the Management of Turns.” In Interaction and Grammar, edited by Elinor Ochs, Emanuel A. Schegloff, and Sandra A. Thompson, 134–84. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Fox Tree, Jean E. 2002. “Interpreting Pauses and Ums at Turn Exchanges.” Discourse Processes 34 (1): 37–55. Google Scholar logo with link to Google Scholar
Hayashi, Makoto. 2012. “Turn Allocation and Turn Sharing.” In The Handbook of Conversation Analysis, edited by Jack Sidnell and Tanya Stivers, 1st ed., 167–90. Wiley. Google Scholar logo with link to Google Scholar
Heldner, Mattias. 2011. “Detection Thresholds for Gaps, Overlaps, and No-Gap-No-Overlaps.” The Journal of the Acoustical Society of America 130 (1): 508–13. Google Scholar logo with link to Google Scholar
Heldner, Mattias, and Jens Edlund. 2010. “Pauses, Gaps and Overlaps in Conversations.” Journal of Phonetics 38 (4): 555–68. Google Scholar logo with link to Google Scholar
Henetz, Tania. 2017. “Don’t Hesitate! The Length of Inter-Turn Gaps Influences Observers’ Interactional Attributions.” PhD Thesis, Stanford: Stanford University.
Heritage, John. 1984. Garfinkel and Ethnomethodology. Repr. Cambridge: Polity Press.Google Scholar logo with link to Google Scholar
Hess, E. H., and J. M. Polt. 1964. “Pupil Size in Relation to Mental Activity during Simple Problem-Solving.” Science 143 (3611): 1190–92. Google Scholar logo with link to Google Scholar
Holler, Judith, Phillip M. Alday, Caitlin Decuyper, Mareike Geiger, Kobin H. Kendrick, and Antje S. Meyer. 2021. “Competition Reduces Response Times in Multiparty Conversation.” Frontiers in Psychology 12: 693124. Google Scholar logo with link to Google Scholar
Indefrey, Peter, and Willem J. M. Levelt. 2004. “The Spatial and Temporal Signatures of Word Production Components.” Cognition 92 (1–2): 101–44. Google Scholar logo with link to Google Scholar
Jefferson, Gail. 2004. “Glossary of Transcript Symbols with an Introduction.” In Pragmatics & Beyond New Series, edited by Gene H. Lerner, 125:13–31. Amsterdam: John Benjamins Publishing Company. Google Scholar logo with link to Google Scholar
Jescheniak, Jörg D., and Willem J. M. Levelt. 1994. “Word Frequency Effects in Speech Production: Retrieval of Syntactic Information and of Phonological Form.” Journal of Experimental Psychology: Learning, Memory, and Cognition 20 (4): 824–43. Google Scholar logo with link to Google Scholar
Just, Marcel A., and Patricia A. Carpenter. 1993. “The Intensity Dimension of Thought: Pupillometric Indices of Sentence Processing.” Canadian Journal of Experimental Psychology 47 (2): 310–39. Google Scholar logo with link to Google Scholar
Kahneman, Daniel, and Jackson Beatty. 1966. “Pupil Diameter and Load on Memory.” Science 154 (3756): 1583–85. Google Scholar logo with link to Google Scholar
Kendon, Adam. 1967. “Some Functions of Gaze-Direction in Social Interaction.” Acta Psychologica 26 (January): 22–63. Google Scholar logo with link to Google Scholar
Koch, Xaver, and Esther Janse. 2016. “Speech Rate Effects on the Processing of Conversational Speech across the Adult Life Span.” The Journal of the Acoustical Society of America 139 (4): 1618–36. Google Scholar logo with link to Google Scholar
Kubose, Tate T., Kathryn Bock, Gary S. Dell, Susan M. Garnsey, Arthur F. Kramer, and Jeff Mayhugh. 2006. “The Effects of Speech Production and Speech Comprehension on Simulated Driving Performance.” Applied Cognitive Psychology 20 (1): 43–63. Google Scholar logo with link to Google Scholar
Kuznetsova, Alexandra, Per B. Brockhoff, and Rune H. B. Christensen. 2017. “LmerTest Package: Tests in Linear Mixed Effects Models.” Journal of Statistical Software 82 (13). Google Scholar logo with link to Google Scholar
Laeng, Bruno, Sylvain Sirois, and Gustaf Gredebäck. 2012. “Pupillometry: A Window to the Preconscious?” Perspectives on Psychological Science 7 (1): 18–27. Google Scholar logo with link to Google Scholar
Lerner, Gene H. 2003. “Selecting next Speaker: The Context-Sensitive Operation of a Context-Free Organization.” Language in Society 32 (2): 177–201. Google Scholar logo with link to Google Scholar
Levelt, Willem J. M., Ardi Roelofs, and Antje S. Meyer. 1999. “A Theory of Lexical Access in Speech Production.” Behavioural and Brain Sciences 22 (01): 1–75. Google Scholar logo with link to Google Scholar
Levinson, Stephen C., and Francisco Torreira. 2015. “Timing in Turn-Taking and Its Implications for Processing Models of Language.” Frontiers in Psychology 6 (731): 10–26. Google Scholar logo with link to Google Scholar
Mathôt, Sebastiaan. 2018. “Pupillometry: Psychology, Physiology, and Function.” Journal of Cognition 1 (1): 16. Google Scholar logo with link to Google Scholar
Papesh, Megan H., and Stephen D. Goldinger. 2012. “Pupil-BLAH-Metry: Cognitive Effort in Speech Planning Reflected by Pupil Dilation.” Attention, Perception, & Psychophysics 74 (4): 754–65. Google Scholar logo with link to Google Scholar
Roberts, Felicia, and Alexander L. Francis. 2013. “Identifying a Temporal Threshold of Tolerance for Silent Gaps after Requests.” The Journal of the Acoustical Society of America 133 (6): EL471–77. Google Scholar logo with link to Google Scholar
Roberts, Felicia, Alexander L. Francis, and Melanie Morgan. 2006. “The Interaction of Inter-Turn Silence with Prosodic Cues in Listener Perceptions of ‘Trouble’ in Conversation.” Speech Communication 48 (9): 1079–93. Google Scholar logo with link to Google Scholar
Roberts, Seán G., Francisco Torreira, and Stephen C. Levinson. 2015. “The Effects of Processing and Sequence Organization on the Timing of Turn Taking: A Corpus Study.” Frontiers in Psychology 6. Google Scholar logo with link to Google Scholar
Rühlemann, Christoph. 2024. “Gaze alternation predicts inclusive next-speaker selection: Evidence from eyetracking.” Frontiers in Communication (Sec. Multimodality of Communication) 9:1396925. Google Scholar logo with link to Google Scholar
Rühlemann, Christoph, and Mathias Barthel. 2024. “Word frequency and cognitive effort in turns-at-talk: Turn structure affects processing load in natural conversation.” Frontiers in Psychology (Sec. Psychology of Language) 15. Google Scholar logo with link to Google Scholar
. Under revision. “Speech planning depends on next-speaker selection: Evidence from pupillometry in question-answer sequences in naturalistic triadic conversation.”
Rühlemann, Christoph, and Alexander Ptak. 2023. “Reaching beneath the Tip of the Iceberg: A Guide to the Freiburg Multimodal Interaction Corpus.” Open Linguistics 9 (1). Google Scholar logo with link to Google Scholar
Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson. 1974. “A Simplest Systematics for the Organization of Turn-Taking for Conversation.” Language 50 (4): 696–735. Google Scholar logo with link to Google Scholar
Sauppe, Sebastian. 2017. “Symmetrical and Asymmetrical Voice Systems and Processing Load: Pupillometric Evidence from Sentence Production in Tagalog and German.” Language 93 (2): 288–313. Google Scholar logo with link to Google Scholar
Schmidtke, Jens. 2014. “Second Language Experience Modulates Word Retrieval Effort in Bilinguals: Evidence from Pupillometry.” Frontiers in Psychology 5: 137. Google Scholar logo with link to Google Scholar
Selting, Margret. 2000. “The Construction of Units in Conversational Talk.” Language in Society 29: 477–517. Google Scholar logo with link to Google Scholar
Sevilla, Yamila, Mora Maldonado, and Diego E. Shalóm. 2014. “Pupillary Dynamics Reveal Computational Cost in Sentence Planning.” Quarterly Journal of Experimental Psychology 67 (6): 1041–52. Google Scholar logo with link to Google Scholar
Sirois, Sylvain, and Julie Brisson. 2014. “Pupillometry.” Wiley Interdisciplinary Reviews: Cognitive Science 5 (6): 679–92. Google Scholar logo with link to Google Scholar
Stivers, Tanja, Nick J. Enfield, Penelope Brown, C. Englert, Makoto Hayashi, Trine Heinemann, Gertie Hoymann, et al. 2009. “Universals and Cultural Variation in Turn-Taking in Conversation.” Proceedings of the National Academy of Sciences 106 (26): 10587–92. Google Scholar logo with link to Google Scholar
Stivers, Tanya, and Federico Rossano. 2010. “Mobilizing Response.” Research on Language & Social Interaction 43 (1): 3–31. Google Scholar logo with link to Google Scholar
The Language Archive. 2023. ELAN (version 6.7). Nijmegen: Max Planck Institute for Psycholinguistics. [URL]
Tromp, Johanne, Peter Hagoort, and Antje S. Meyer. 2016. “Pupillometry Reveals Increased Pupil Size during Indirect Request Comprehension.” Quarterly Journal of Experimental Psychology 69 (6): 1093–1108. Google Scholar logo with link to Google Scholar
Walker, M. B., and C. Trimboli. 1984. “The Role of Nonverbal Signals in Co-Ordinating Speaking Turns.” Journal of Language and Social Psychology 3 (4): 257–72. Google Scholar logo with link to Google Scholar
Wel, Pauline van der, and Henk van Steenbergen. 2018. “Pupil Dilation as an Index of Effort in Cognitive Control Tasks: A Review.” Psychonomic Bulletin & Review 25 (6): 2005–15. Google Scholar logo with link to Google Scholar
Wu, Esther Xiu Wen, Bruno Laeng, and Svein Magnussen. 2012. “Through the Eyes of the Own-Race Bias: Eye-Tracking and Pupillometry during Face Recognition.” Social Neuroscience 7 (2): 202–16. Google Scholar logo with link to Google Scholar
Zekveld, Adriana A., Sophia E. Kramer, and Joost M. Festen. 2010. “Pupil Response as an Indication of Effortful Listening: The Influence of Sentence Intelligibility:” Ear and Hearing 31 (4): 480–90. Google Scholar logo with link to Google Scholar
Zima, Elisabeth, Clarissa Weiß, and Geert Brône. 2019. “Gaze and Overlap Resolution in Triadic Interactions.” Journal of Pragmatics 140 (January): 49–69. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue