Cover not available

In:Eye-tracking in Interaction: Studies on the role of eye gaze in dialogue
Edited by Geert Brône and Bert Oben
[Advances in Interaction Studies 10] 2018
► pp. 67–90

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

Chapter 4
Weaving oneself into others

Coordination in conversational systems

Published online: 13 November 2018
DOI logo with link
https://doi.org/10.1075/ais.10.04dal
Abstract
We review a range of findings that show how eye movements (and other body movements) exhibit correlated behavior across two or more people during natural interactions. We then synthesize these different results into a more general account of how people’s cognitive, sensory and motor systems become coordinated with one another during natural dialogue. We argue that treating conversants as parts of one integrated system is a useful explanatory strategy for understanding interaction. We end by describing explicit quantitative conditions for seeking “systemhood” in human interaction. These conditions motivate future research questions on social eye movements and other behaviors.
Article outline
  • 1.Introduction: The eyes of social systems
    • 1.1Three types of system
  • 2.Collecting samples from the temporal dynamics of a cognitive process
    • 2.1Devise a cyclic motor behavior
    • 2.2A series of experimental trials as dynamics
    • 2.3Dense-sampling methods
  • 3.Eyetracking as a dense-sampling measure of human interaction
    • 3.1Human interaction as systemic coupling
  • 4.Concluding discussion: Criteria for systemhood
    • 4.1Implications for research: Weak and strong conditions for systemhood
    • 4.2Conclusion
  • References
References (102)
References
Adams, F., & Aizawa, K. (2001). The bounds of cognition. Philosophical psychology, 14(1), 43–64.Google Scholar logo with link to Google Scholar
Allopenna, P. D., Magnuson, J. S., & Tanenhaus, M. K. (1998). Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of memory and language, 38(4), 419–439.Google Scholar logo with link to Google Scholar
Altmann, G. T., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73(3), 247–264.Google Scholar logo with link to Google Scholar
Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American Psychologist, 51(4), 355.Google Scholar logo with link to Google Scholar
Ashby, W. R. (1956). An introduction to cybernetics. London: Chapman & Hall.Google Scholar logo with link to Google Scholar
Bakeman, R., & Quera, V. (2011). Sequential analysis and observational methods for the behavioral sciences. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Bechtel, W., & Abrahamsen, A. (1991). Connectionism and the Mind. Backwell, Cambridge, MA.Google Scholar logo with link to Google Scholar
(2010). Dynamic mechanistic explanation: Computational modeling of circadian rhythms as an exemplar for cognitive science. Studies in History and Philosophy of Science Part A, 41(3), 321–333.Google Scholar logo with link to Google Scholar
Beer, R. D., & Williams, P. L. (2015). Information processing and dynamics in minimally cognitive agents. Cognitive science, 39(1), 1–38.Google Scholar logo with link to Google Scholar
Brennan, S. E., Galati, A., & Kuhlen, A. K. (2010). Two minds, one dialog: Coordinating speaking and understanding. Psychology of Learning and Motivation, 53, 301–344.Google Scholar logo with link to Google Scholar
Brown-Schmidt, S., Campana, E., & Tanenhaus, M. K. (2005). Real-time reference resolution by naïve participants during a task-based unscripted conversation. In J. Trueswell & M. Tanenhaus (Eds.), World Situated Language Use: Psycholinguistic, Linguistic and Computational Perspectives on Bridging the Product and Action Traditions, (pp.153–171) Cambridge, MA: MIT Press.Google Scholar logo with link to Google Scholar
Brown-Schmidt, S., Yoon, S. O., & Ryskin, R. A. (2015). People as contexts in conversation. Psychology of Learning and Motivation, 62, 59–99.Google Scholar logo with link to Google Scholar
Brown-Schmidt, S., & Tanenhaus, M. K. (2008). Real-time investigation of referential domains in unscripted conversation: A targeted language game approach. Cognitive Science, 32(4), 643–684.Google Scholar logo with link to Google Scholar
Chemero, A. (2011). Radical embodied cognitive science. MIT press.Google Scholar logo with link to Google Scholar
Chemero, A., & Turvey, M. T. (2008). Autonomy and hypersets. Biosystems,91(2), 320–330.Google Scholar logo with link to Google Scholar
Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22(1), 1–39.Google Scholar logo with link to Google Scholar
Coupland, N. (1985). ‘Hark, hark, the lark’: social motivations for phonological style-shifting. Language & Communication, 5(3), 153–171.Google Scholar logo with link to Google Scholar
Crosby, J. R., Monin, B. & Richardson, D. C. (2008). Where Do We Look During Potentially Offensive Behavior? Psychological Science, 19 (3), 226–228.Google Scholar logo with link to Google Scholar
Crutchfield, J. P. (1994). The calculi of emergence: computation, dynamics and induction. Physica D: Nonlinear Phenomena, 75(1), 11–54.Google Scholar logo with link to Google Scholar
Dale, R., Kirkham, N. Z. & Richardson, D. C. (2011). The dynamics of reference and shared visual attention. Frontiers in Psychology, 2.Google Scholar logo with link to Google Scholar
Dotov, D. G., Nie, L., & Chemero, A. (2010). A demonstration of the transition from ready-to-hand to unready-to-hand. PLoS One, 5(3), e9433.Google Scholar logo with link to Google Scholar
Eberhard, K. M., Spivey-Knowlton, M. J., Sedivy, J. C., & Tanenhaus, M. K. (1995). Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of psycholinguistic research, 24(6), 409–436.Google Scholar logo with link to Google Scholar
Edelman, S. (2008). On the nature of minds, or: truth and consequences.Journal of Experimental & Theoretical Artificial Intelligence, 20(3), 181–196.Google Scholar logo with link to Google Scholar
Eliasmith, C. (1996). The third contender: A critical examination of the dynamicist theory of cognition. Philosophical Psychology, 9(4), 441–463.Google Scholar logo with link to Google Scholar
(2012). The complex systems approach: rhetoric or revolution.Topics in cognitive science, 4(1), 72–77.Google Scholar logo with link to Google Scholar
Enfield, N. J. (2013). Relationship thinking: Agency, enchrony, and human sociality. Oxford University Press.Google Scholar logo with link to Google Scholar
Farmer, T. A., Cargill, S. A., Hindy, N. C., Dale, R., & Spivey, M. J. (2007). Tracking the continuity of language comprehension: Computer mouse trajectories suggest parallel syntactic processing. Cognitive Science, 31(5), 889–909.Google Scholar logo with link to Google Scholar
Ferreira, F., & Clifton, C. (1986). The independence of syntactic processing. Journal of memory and language, 25(3), 348–368.Google Scholar logo with link to Google Scholar
Fitneva, S. A. & Spivey, M. J. (2005). Context and language processing: The effect of authorship. In J. Trueswell & M. Tanenhaus (Eds.), World Situated Language Use: Psycholinguistic, Linguistic and Computational Perspectives on Bridging the Product and Action Traditions, (pp.317–328) Cambridge, MA: MIT Press.Google Scholar logo with link to Google Scholar
Frazier, L. (1995). Constraint satisfaction as a theory of sentence processing. Journal of Psycholinguistic Research, 24(6), 437–468.Google Scholar logo with link to Google Scholar
Fusaroli, R., & Tylén, K. (2015). Investigating conversational dynamics: Interactive alignment, Interpersonal synergy, and collective task performance. Cognitive Science, 40, 145–171.Google Scholar logo with link to Google Scholar
Fusaroli, R., Bahrami, B., Olsen, K., Roepstorff, A., Rees, G., Frith, C., & Tylén, K. (2012). Coming to terms quantifying the benefits of linguistic coordination. Psychological Science, 23(8), 931–939.Google Scholar logo with link to Google Scholar
Fusaroli, R., Bjørndahl, J. S., Roepstorff, A., & Tylén, K. (2016). A Heart for Interaction: Shared Physiological Dynamics and Behavioral Coordination in a Collective, Creative Construction Task. Journal of Experimental Psychology: Human Perception & Performance, 42(9), 1297–1310.Google Scholar logo with link to Google Scholar
Fusaroli, R., Rączaszek-Leonardi, J., & Tylén, K. (2014). Dialog as interpersonal synergy. New Ideas in Psychology, 32, 147–157.Google Scholar logo with link to Google Scholar
Gallagher, S., & Crisafi, A. (2009). Mental institutions. Topoi, 28(1), 45–51.Google Scholar logo with link to Google Scholar
Gilden, D. L. (2001). Cognitive emissions of 1/f noise. Psychological review, 108(1), 33–56.Google Scholar logo with link to Google Scholar
Gow, D. W. & Olson, B. B. (2015): Sentential influences on acoustic- phonetic processing: a Granger causality analysis of multimodal imaging data, Language, Cognition and Neuroscience. Google Scholar logo with link to Google Scholar
Haken, H. (1977). Synergetics. Physics Bulletin , 28(9), 412.Google Scholar logo with link to Google Scholar
Haken, H., Kelso, J. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological cybernetics, 51(5), 347–356.Google Scholar logo with link to Google Scholar
Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., & Malach, R. (2004). Intersubject synchronization of cortical activity during natural vision. science, 303(5664), 1634-1640.Google Scholar logo with link to Google Scholar
Hove, M. J., & Keller, P. E. (2015). Impaired movement timing in neurological disorders: rehabilitation and treatment strategies. Annals of the New York Academy of Sciences, 1337(1), 111–117.Google Scholar logo with link to Google Scholar
Hove, M. J., & Risen, J. L. (2009). It’s all in the timing: Interpersonal synchrony increases affiliation. Social Cognition, 27(6), 949–960.Google Scholar logo with link to Google Scholar
Hove, M. J., Spivey, M. J., & Krumhansl, C. L. (2010). Compatibility of motion facilitates visuomotor synchronization. Journal of Experimental Psychology: Human Perception and Performance, 36(6), 1525–1534.Google Scholar logo with link to Google Scholar
Hutchins, E. (1995). Cognition in the Wild. MIT press.Google Scholar logo with link to Google Scholar
(1995). How a cockpit remembers its speeds. Cognitive science,19(3), 265–288.Google Scholar logo with link to Google Scholar
Jaeger, T. F. (2010). Redundancy and reduction: Speakers manage syntactic information density. Cognitive psychology, 61(1), 23–62.Google Scholar logo with link to Google Scholar
Jordan, J. S., & Knoblich, G. (2004). Spatial perception and control. Psychonomic Bulletin & Review, 11(1), 54–59.Google Scholar logo with link to Google Scholar
Kello, C. T., Anderson, G. G., Holden, J. G., & Van Orden, G. C. (2008). The pervasiveness of 1/f scaling in speech reflects the metastable basis of cognition. Cognitive Science, 32(7), 1217–1231.Google Scholar logo with link to Google Scholar
Kello, C. T., Brown, G. D., Ferrer-i-Cancho, R., Holden, J. G., Linkenkaer-Hansen, K., Rhodes, T., & Van Orden, G. C. (2010). Scaling laws in cognitive sciences. Trends in Cognitive Sciences, 14(5), 223–232.Google Scholar logo with link to Google Scholar
Kelso, J. A. S. (1984). Phase transitions and critical behavior in human bimanual coordination. American Journal of Physiology-Regulatory, Integrative and Comparative Physiology, 246(6), R1000–R1004.Google Scholar logo with link to Google Scholar
(1997). Dynamic patterns: The self-organization of brain and behavior. MIT press.Google Scholar logo with link to Google Scholar
Kelso, J. S., Tuller, B., Vatikiotis-Bateson, E., & Fowler, C. A. (1984). Functionally specific articulatory cooperation following jaw perturbations during speech: evidence for coordinative structures. Journal of Experimental Psychology: Human Perception and Performance, 10(6), 812.Google Scholar logo with link to Google Scholar
Kloos, H., & Van Orden, G. (2010). Voluntary behavior in cognitive and motor tasks. Mind and Matter, 8(1), 19–43.Google Scholar logo with link to Google Scholar
Knoblich, G., & Jordan, J. S. (2003). Action coordination in groups and individuals: learning anticipatory control. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(5), 1006–1016.Google Scholar logo with link to Google Scholar
Knoeferle, P., & Crocker, M. W. (2006). The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking. Cognitive Science, 30(3), 481–529.Google Scholar logo with link to Google Scholar
Konvalinka, I., Xygalatas, D., Bulbulia, J., Schjødt, U., Jegindø, E. M., Wallot, S., Van Orden, G., & Roepstorff, A. (2011). Synchronized arousal between performers and related spectators in a fire-walking ritual. Proceedings of the National Academy of Sciences, 108(20), 8514–8519.Google Scholar logo with link to Google Scholar
Kuhlen, A. K., Allefeld, C., & Haynes, J. D. (2012). Content-specific coordination of listeners’ to speakers’ EEG during communication. Frontiers in Human Neuroscience, 6 , Article 266.Google Scholar logo with link to Google Scholar
Lakin, J. L., & Chartrand, T. L. (2003). Using nonconscious behavioral mimicry to create affiliation and rapport. Psychological science, 14(4), 334–339.Google Scholar logo with link to Google Scholar
Latash, M. L., Scholz, J. P., & Schöner, G. (2002). Motor control strategies revealed in the structure of motor variability. Exercise and sport sciences reviews, 30(1), 26–31.Google Scholar logo with link to Google Scholar
Louwerse, M. M., Dale, R., Bard, E. G., & Jeuniaux, P. (2012). Behavior matching in multimodal communication is synchronized. Cognitive Science,36(8), 1404–1426.Google Scholar logo with link to Google Scholar
Manson, J. H., Bryant, G. A., Gervais, M. M., & Kline, M. A. (2013). Convergence of speech rate in conversation predicts cooperation. Evolution and Human Behavior, 34(6), 419–426.Google Scholar logo with link to Google Scholar
Mitchell, M. (2009). Complexity: A guided tour. Oxford University Press.Google Scholar logo with link to Google Scholar
Mønster, D., Håkonsson, D. D., Eskildsen, J. K., & Wallot, S. (2016). Physiological evidence of interpersonal dynamics in a cooperative production task. Physiology & Behavior, 156, 24–34.Google Scholar logo with link to Google Scholar
Neisser, U. (1976). Cognition and reality: Principles and implications of cognitive psychology. San Francisco, California: W. H. Freeman.Google Scholar logo with link to Google Scholar
Newell, A. (1980). Physical symbol systems*. Cognitive science, 4(2), 135–183.Google Scholar logo with link to Google Scholar
Noë, A. (2009). Out of our heads: Why you are not your brain, and other lessons from the biology of consciousness. Macmillan.Google Scholar logo with link to Google Scholar
Paxton, A., & Dale, R. (2013). Argument disrupts interpersonal synchrony. The Quarterly Journal of Experimental Psychology, 66(11), 2092–2102.Google Scholar logo with link to Google Scholar
(2013a). Frame-differencing methods for measuring bodily synchrony in conversation. Behavior Research Methods, 45(2), 329–343.Google Scholar logo with link to Google Scholar
(2013b). Argument disrupts interpersonal synchrony. The Quarterly Journal of Experimental Psychology, 66(11), 2092–2102.Google Scholar logo with link to Google Scholar
Reali, F., Spivey, M. J., Tyler, M. J., & Terranova, J. (2006). Inefficient conjunction search made efficient by concurrent spoken delivery of target identity. Perception & psychophysics, 68(6), 959–974.Google Scholar logo with link to Google Scholar
Repp, B. H. (2005). Sensorimotor synchronization: A review of the tapping literature. Psychonomic bulletin & review, 12(6), 969–992.Google Scholar logo with link to Google Scholar
Richardson, D. C., & Dale, R. (2005). Looking to understand: The coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cognitive science, 29(6), 1045–1060.Google Scholar logo with link to Google Scholar
Richardson, D. C., Dale, R. & Kirkham, N. (2007). The art of conversation is coordination: common ground and the coupling of eye movements during dialogue. Psychological Science, 18, 407–413.Google Scholar logo with link to Google Scholar
Richardson, D. C., Street, C. N. H., Tan, J. Y. M., Kirkham, N. Z., Hoover, M. A., & Ghane Cavanaugh, A. (2012).Joint perception: gaze and social context. Frontiers in Human Neuroscience, 6. Google Scholar logo with link to Google Scholar
Richardson, M. J., Dale, R. & Marsh, K. (2014). Complex dynamical systems in social and personality psychology: theory, modeling and analysis. In H. T. Reis & C. M. Judd (Eds.), Handbook of Research Methods in Social and Personality Psychology (pp.253–282). Cambridge University Press.Google Scholar logo with link to Google Scholar
Richardson, M. J., Marsh, K. L., & Schmidt, R. C. (2005). Effects of visual and verbal interaction on unintentional interpersonal coordination. Journal of Experimental Psychology: Human Perception and Performance, 31(1), 62–79.Google Scholar logo with link to Google Scholar
Riley, M. A., Richardson, M. J., Shockley, K., & Ramenzoni, V. C. (2011). Interpersonal synergies. Frontiers in psychology, 2, 38.Google Scholar logo with link to Google Scholar
Risko, E. F., & Kingstone, A. (2011). Eyes wide shut: implied social presence, eye tracking and attention. Attention, Perception, & Psychophysics, 73(2), 291–296.Google Scholar logo with link to Google Scholar
Risko, E. F., Richardson, D. C., & Kingstone, A. (2016). Breaking the Fourth Wall of Cognitive Science Real-World Social Attention and the Dual Function of Gaze. Current Directions in Psychological Science, 25(1), 70–74.Google Scholar logo with link to Google Scholar
Rupert, R. D. (2011). Cognitive systems and the supersized mind. Philosophical Studies, 152(3), 427–436.Google Scholar logo with link to Google Scholar
Schmidt, R. C., Carello, C., & Turvey, M. T. (1990). Phase transitions and critical fluctuations in the visual coordination of rhythmic movements between people. Journal of experimental psychology: human perception and performance, 16(2), 227–247.Google Scholar logo with link to Google Scholar
Sebanz, N., Knoblich, G., & Prinz, W. (2003). Representing others’ actions: just like one’s own?. Cognition, 88(3), B11–B21.Google Scholar logo with link to Google Scholar
Sebanz, N., Knoblich, G., Prinz, W., & Wascher, E. (2006). Twin peaks: An ERP study of action planning and control in coacting individuals. Journal of Cognitive Neuroscience, 18(5), 859–870.Google Scholar logo with link to Google Scholar
Shenhar, A. (1990). On system properties and systemhood. International Journal Of General System, 18(2), 167–174.Google Scholar logo with link to Google Scholar
Shockley, K., Baker, A. A., Richardson, M. J., & Fowler, C. A. (2007). Articulatory constraints on interpersonal postural coordination. Journal of Experimental Psychology: Human Perception and Performance, 33(1), 201.Google Scholar logo with link to Google Scholar
Shockley, K., Santana, M. V., & Fowler, C. A. (2003). Mutual interpersonal postural constraints are involved in cooperative conversation. Journal of Experimental Psychology: Human Perception and Performance, 29(2), 326–332.Google Scholar logo with link to Google Scholar
Simon, H. A. (1992). What is an “explanation” of behavior?. Psychological Science, 3(3), 150–161.Google Scholar logo with link to Google Scholar
Spivey-Knowlton, M. J. (1996). Integration of visual and linguistic information: Human data and model simulations. Ph.D. Dissertation, U. Rochester, NY.Google Scholar logo with link to Google Scholar
Spivey, M. (2008). The continuity of mind. Oxford University Press.Google Scholar logo with link to Google Scholar
Spivey, M. J. (2012). The spatial intersection of minds. Cognitive Processing, 13, 343–346.Google Scholar logo with link to Google Scholar
(2016). Semantics influences speech perception: commentary on Gow and Olson (2015). Language, Cognition, and Neuroscience, 31(7), 856-859.Google Scholar logo with link to Google Scholar
Strogatz, S. (2003). Sync: The emerging science of spontaneous order. New York, NY: Hyperion.Google Scholar logo with link to Google Scholar
Taleb, N. N. (2012). Antifragile: Things that gain from disorder (Vol. 3). Random House Incorporated.Google Scholar logo with link to Google Scholar
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217), 1632–1634.Google Scholar logo with link to Google Scholar
Tollefsen, D. P. (2006). From extended mind to collective mind. Cognitive systems research, 7(2), 140–150.Google Scholar logo with link to Google Scholar
Tollefsen, D. & Dale, R. (2012). Naturalizing joint action: a process-based approach. Philosophical Psychology, 25, 385–407.Google Scholar logo with link to Google Scholar
Van Gelder, T. (1998). The dynamical hypothesis in cognitive science.Behavioral and brain sciences, 21(05), 615–628.Google Scholar logo with link to Google Scholar
Van Orden, G. C., & Holden, J. G. (2002). Intentional contents and self-control. Ecological Psychology, 14(1–2), 87–109.Google Scholar logo with link to Google Scholar
Van Orden, G. C., Holden, J. G., & Turvey, M. T. (2003). Self-organization of cognitive performance. Journal of Experimental Psychology: General, 132(3), 331.Google Scholar logo with link to Google Scholar
Van Orden, G., & Stephen, D. G. (2012). Is cognitive science usefully cast as complexity science?. Topics in cognitive science, 4(1), 3–6.Google Scholar logo with link to Google Scholar
Wegner, D. M. (1987). Transactive memory: A contemporary analysis of the group mind. In Theories of group behavior (pp.185–208). Springer New York.Google Scholar logo with link to Google Scholar
Wiener, N. (1961). Cybernetics or Control and Communication in the Animal and the Machine (Vol. 25). MIT press.Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Haraped, Lucas, D. Jacob Gerlofs, Olive Chung‐Hui Huang, Cam Hickling, Walter F. Bischof, Pierre Sachse & Alan Kingstone
2025. Coordinating Attention in Face‐to‐Face Collaboration: The Dynamics of Gaze, Pointing, and Verbal Reference. Cognitive Science 49:10 DOI logo
Spivey, Michael J.
2023. Cognitive Science Progresses Toward Interactive Frameworks. Topics in Cognitive Science 15:2 ► pp. 219 ff. DOI logo
Dale, Rick, Anne S. Warlaumont & Kerri L. Johnson
2022. The fundamental importance of method to theory. Nature Reviews Psychology 2:1 ► pp. 55 ff. DOI logo

This list is based on CrossRef data as of 13 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue