Cover not available

Article published In: Interaction Studies
Vol. 25:3 (2024) ► pp.257–280

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB

Changes in the topical structure of explanations are related to explainees’ multimodal behaviour

ORCiD logo with link | Paderborn University
ORCiD logo with link | Paderborn University
ORCiD logo with link | Paderborn University
Open Access logo
Available under the Creative Commons Attribution-NonCommercial (CC BY-NC) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

This article was made Open Access under a CC BY-NC 4.0 license through payment of an APC by or on behalf of the authors.

Published online: 27 June 2025
DOI logo with link
https://doi.org/10.1075/is.23033.laz
Abstract
Everyday explanations are interactive processes with the aim to provide a less knowledgeable person with reasonable information about other people, objects, or events. Because explanations are interactive communicative processes, the topical structure of an explanation may vary dynamically depending on the immediate feedback of the explainee. In this paper, we analyse topical transitions in medical explanations organised by different physicians (explainers) related to different forms of multimodal behaviour of caregivers (explainees) attending an explanation about the procedures of an upcoming surgery of a child. The analyses reveal that explainees’ multimodal behaviour with gaze shifts (and particularly gaze aversion) can predict a transition from an elaborated topic to a new one, whereas explainees’ forms of multimodal behaviour with static gaze cannot be related to changes of the topical structure.
Article outline
  • 1.Introduction
  • 2.Theoretical background
    • 2.1Monitoring listeners’ behaviour
    • 2.2(Non-)verbal forms of behaviour
      • 2.2.1Gaze behaviour
      • 2.2.2Head nodding and backchannelling
    • 2.3Explanation structure
  • 3.The current study
    • 3.1Participants
    • 3.2Procedure
    • 3.3Data coding
      • 3.3.1Explainees’ (non-)verbal behaviour
      • 3.3.2Forms of multimodal behaviour
      • 3.3.3Explainers’ explanation structure
      • 3.3.4Temporal threshold
      • 3.3.5Reliability
    • 3.4Data analysis
      • 3.4.1Conditional probabilities
      • 3.4.2Generalised linear mixed model (GLMM)
  • 4.Results
    • 4.1Conditional probabilities across participants
    • 4.2Analysis of fixed and random effects
    • 4.3Summary of results
  • 5.Discussion
    • 5.1Caregivers’ static gaze behaviour with or without other modalities
    • 5.2Caregivers’ gaze shifts and gaze aversions with or without other modalities
    • 5.3Concluding remarks
  • Acknowledgements
  • References
References (40)
References
Allwood, J., Nivre, J., & Ahlsén, E. (1992). On the Semantics and Pragmatics of Linguistic Feedback. Journal of Semantics, 9(1), 1–26. Google Scholar logo with link to Google Scholar
Allwood, J. & Cerrato, L. (2003). A study of gestural feedback expressions. Proceedings of the 1st Nordic Symposium on Multimodal Communication. Copenhagen, 7–22.Google Scholar logo with link to Google Scholar
Argyle, M., & Cook, M. (1976). Gaze and mutual gaze. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Arnold, K. (2012). Humming along. Contemporary Psychoanalysis, 48(1), 100–117. Google Scholar logo with link to Google Scholar
Bavelas, J. B., Black, A., Lemery, C. R., & Mullett, J. (1986). “I show you how you feel”: Motor mimicry as a communicative act. Journal of Personality and Social Psychology, 501, 322–329. Google Scholar logo with link to Google Scholar
Bavelas, J. B., Coates, L., & Johnson, T. (2002). Listener responses as a collaborative process: the role of gaze. Journal of Communication, 52(3), 566–580. Google Scholar logo with link to Google Scholar
Brooks, M. E., Kristensen, K., Van Benthem, K. J., Magnússon, Á., Berg, C. W., Nielsen, A., Skaug, H. J., Mächler, M., & Bolker, B. M. (2017). GLMMTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modeling. R Journal, 9(2), 378–400. Google Scholar logo with link to Google Scholar
Clark, H. & Brennan, S. A. (1991). Grounding in Communication. In Resnick, L. B., Levine, J. M. & Teasley, S. D. (Eds.) Perspectives on socially shared cognition (pp. 127–142). APA Books. Google Scholar logo with link to Google Scholar
Clark, H. H. (2003). Pointing and placing. In S. Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 243–268). Lawrence Erlbaum. Google Scholar logo with link to Google Scholar
Clark, H. H., & Krych, M. A. (2004). Speaking while monitoring addressees for understanding. Journal of Memory and Language, 501, 62–81. Google Scholar logo with link to Google Scholar
Degutyte, Z., & Astell, A. (2021). The role of eye gaze in regulating turn taking in conversations: A systematized review of methods and findings. Frontiers in Psychology, 121, 616471. Google Scholar logo with link to Google Scholar
Dekking, F. M., Kraaikamp, C., Lopuhaä, H. P., & Meester, L. E. (2005). A modern introduction to probability and statistics: Understanding why and how. Springer London. Google Scholar logo with link to Google Scholar
Dingemanse, M., Roberts, S. G., Baranova, J., Blythe, J., Drew, P., Floyd, S., et al. (2015). Universal Principles in the Repair of Communication Problems. PLoS ONE 10(9): e0136100. Google Scholar logo with link to Google Scholar
Doherty-Sneddon, G., & Phelps, F. G. (2007). Teacher’s responses to children’s eye gaze. Educational Psychology, 27(1), 93–109. Google Scholar logo with link to Google Scholar
Eshghi, A., Howes, C., Gregoromichelaki, E., Hough, J., & Purver, M. (2015). Feedback in conversation as incremental semantic update. Proceedings of the 11th International Conference on Computational Semantics. London, 261–271.Google Scholar logo with link to Google Scholar
Gander, A. G., & Gander, P. (2020). Micro-feedback as cues to understanding in communication. Dialogue and Perception — Extended Papers from DaP2018. In C. Howes, S. Dobnik, & E. Breitholtz (Eds.) CLASP Papers in Computational Linguistics (pp. 1–11). Gothenburg University.Google Scholar logo with link to Google Scholar
Glenberg, A. M., Schroeder, J. L., & Robertson, D. A. (1998). Averting the gaze disengages the environment and facilitates remembering. Memory & cognition, 26(4), 651–658. Google Scholar logo with link to Google Scholar
Goodwin, Ch. (1981). Conversational organisation between speakers and hearers. Academic Press.Google Scholar logo with link to Google Scholar
Gravano, A., Benus, S., Chávez, H., Hirschberg, J., & Wilcox, L. (2007). On the role of context and prosody in the interpretation of ’okay’. Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics. Prague, 800–807.Google Scholar logo with link to Google Scholar
Hömke, P., Holler, J., & Levinson, S. C. (2017). Eye blinking as addressee feedback in face-to-face conversation. Research on Language and Social Interaction, 50(1), 54–70. Google Scholar logo with link to Google Scholar
Hömke, P., Levinson, S. C., & Holler, J. (2022). Eyebrow movements as signals of communicative problems in human face-to-face interaction. PsyArXiv, [URL].
Ismail, N. M., & Syahputri, V. N. (2022). “I Mean You Can Stop. I Already Understand You”: Head Tilts during Conversations. Lingua Didaktika: Jurnal Bahasa dan Pembelajaran Bahasa, 16(1), 1–11. Google Scholar logo with link to Google Scholar
Kendon, A. (1967). Some functions of gaze-direction in social attention. Acta Psychologica, 261, 22–63. Google Scholar logo with link to Google Scholar
Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Departmental Papers (ASC), 30(3), 1–16. Google Scholar logo with link to Google Scholar
(2013). Content Analysis: An Introduction to Its Methodology (3rd ed). Sage Publications.Google Scholar logo with link to Google Scholar
Lai, C. (2010). What do you mean, you’re uncertain?: The interpretation of cue words and rising intonation in dialogue. Proceedings of Interspeech 2010, Makuhari, 1413–1416. Google Scholar logo with link to Google Scholar
Morency, L. P., Christoudias, C. M., & Darell, T. (2006). Recognizing gaze aversion gestures in embodied conversational discourse. Proceedings of the 8th International Conference on Multimodal Interfaces, 298–294. Google Scholar logo with link to Google Scholar
Nota, N., Trujillo, J. P., & Holler, J. (2021). Facial signals and social actions in multimodal face-to-face interaction. Brain Sciences, 11(8), 1017. Google Scholar logo with link to Google Scholar
Park, H. W., Gelsomini, M., Lee, J. J., & Breazeal, C. (2017). Telling Stories to Robots: The Effect of Backchannelling on a Child’s Storytelling. Proceedings of the 2017 ACM/IEEE International Conference in Human-Robot Interaction. Vienna, Austria, 100–108. Google Scholar logo with link to Google Scholar
Phelps, F. G., Doherty-Sneddon, G., & Warnock, H. (2006). Helping children think: Gaze aversion and teaching. British Journal of Developmental Psychology, 24(3), 577–588. Google Scholar logo with link to Google Scholar
Rohlfing, K. J., Cimiano, P., Scharlau, I., Matzner, T., Buhl, H. M., Buschmeier, H., Esposito, E., Grimminger, A., Hammer, B., Häb-Umbach, R., Horwath, I., Hüllermeier, E., Kern, F., Kopp, S., Thommes, K., Ngonga Ngomo, A.-C., Schulte, C., Wachsmuth, H., Wagner, P., & Wrede, B. (2021). Explanation as a Social Practice: Toward a Conceptual Framework for the Social Design of AI Systems. IEEE Transactions on Cognitive and Developmental Systems, 13(3), 717–728. Google Scholar logo with link to Google Scholar
Roscoe, R. & Chi, M. T. H. (2008). Tutor learning: the role of explaining and responding to questions. Instructional Science, 36(4), 321–350. Google Scholar logo with link to Google Scholar
Rossano, F. (2005). When it’s over is it really over? On the effects of sustained gaze vs. gaze withdrawal at sequence possible completion. International Pragmatic Association, Riva del Garda, July.Google Scholar logo with link to Google Scholar
(2013). Gaze in conversation. In J. Sidnell, & T. Stivers (Eds.), The handbook of conversation analysis (pp. 308–329). Malden, MA: Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Sacks, H., Schegloff, E., & Jefferson, G. (1974). A simplest systematics for the organization of turn taking in conversation. Language, 501, 696–735. Google Scholar logo with link to Google Scholar
Schegloff, E. A. (1982). Discourse as an interactional achievement: Some uses of ‘uh huh’ and other things that come between sentences. In Tannen, D. (Ed.) Analysing discourse: Text and talk. Georgetown University Roundtable on Languages and Linguistics 1981 (pp. 71–93). Georgetown University Press.Google Scholar logo with link to Google Scholar
Tang, B., Frye, H. A., Gelfand, A. E. et al. (2023). Zero-Inflated Beta Distribution Regression Modeling. Journal of Agricultural, Biological and Environmental Statistics, 281, 117–137. Google Scholar logo with link to Google Scholar
Ward, N., & Tsukahara, W. (2000). Prosodic features which cue back-channel responses in English and Japanese. Journal of Pragmatics, 32(8), 1177–1207. Google Scholar logo with link to Google Scholar
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H. (2006). ELAN: a Professional Framework for Multimodality Research. Proceedings of the 5th international conference on language resources and evaluation (LREC), 1556–1559.Google Scholar logo with link to Google Scholar
Yngve, V. H. (1970). On getting a word in edgewise. Papers from the sixth regional meeting Chicago Linguistic Society, April 16–18, 1970, Chicago Linguistic Society, Chicago (pp. 567–578).Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue