Cover not available

Article published In: Interaction Studies
Vol. 25:2 (2024) ► pp.167–189

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Embodiment matters when establishing eye contact with a robot

ORCiD logo with link | Istituto Italiano di Tecnologia | Technical University of DenmarkKongens Lyngby
 | Istituto Italiano di Tecnologia
 | Istituto Italiano di Tecnologia
Published online: 7 February 2025
DOI logo with link
https://doi.org/10.1075/is.22060.kom
Abstract
Eye contact constitutes a strong social signal in humans and affects various attentional processes. However, eye contact with another human evokes different responses compared with a direct gaze of an image on a screen. The question of interest is whether this holds also for eye contact with a robot. Previous experiments with physically present iCub humanoid robot showed that eye contact affects participants’ orienting of attention. In the present study, we investigated whether a robot’s eye contact on the screen could show similar effects. Specifically, in two experiments we examined the impact of eye contact on the gaze-cueing effect (orienting of attention in response to a directional gaze shift) while we varied the timing of the events within a trial sequence. Our results showed that the robot’s eye contact did not modulate the gaze-cueing effect (gaze-cueing effect present in all conditions), thereby suggesting that eye contact gaze presented in a 2D format on the screen has less impact on observers than its 3D embodied version in a physically present robot. Overall, our findings stress the importance of embodied interactions for understanding the mechanisms of social cognition.
Article outline
  • 1.Introduction
    • 1.1Embodied vs dis-embodied eye contact
    • 1.2Embodied protocols for social cognition
    • 1.3Theoretical background: The eye contact effect on joint attention
  • 2.Aim of the study
  • 3.Experiment 1
    • 3.1Methods
      • 3.1.1Participants
      • 3.1.2Stimuli and material
      • 3.1.3Procedure
    • 3.2Analysis
      • Reaction times
    • 3.3Results
      • Reaction times
      • Engagement ratings
    • 3.4Discussion
  • 4.Experiment 2
    • 4.1Methods
      • 4.1.1Participants
      • 4.1.2Stimuli and apparatus
    • 4.2Analysis
      • Reaction times
    • 4.3Results
      • Reaction times
      • Engagement ratings
    • 4.4Discussion
  • 5.General discussion
  • 6.Conclusions
  • Competing interests
  • Data repository
  • Notes
  • References
References (57)
References
Abubshait, A., Weis, P., & Wiese, E. (2018). Effects of embodiment on social attention mechanisms in human-robot interaction. Frontiers in Human Neuroscience, 121, Google Scholar logo with link to Google Scholar
Admoni, H., & Scassellati, B. (2017). Social eye gaze in human-robot interaction: a review. Journal of Human-Robot Interaction, 6(1), 25–63, Google Scholar logo with link to Google Scholar
Belkaid, M., Kompatsiari, K., De Tommaso, D., Zablith, I., & Wykowska, A. (2021). Mutual gaze with a robot affects human neural activity and delays decision-making processes. Science Robotics, 6(58), Google Scholar logo with link to Google Scholar
Cabibihan, J. J., Javed, H., Ang, M., & Aljunied, S. M. (2013). Why robots? A survey on the roles and benefits of social robots in the therapy of children with autism. International journal of social robotics, 5(4), 593–618, Google Scholar logo with link to Google Scholar
Chen, Z., McCrackin, S. D., Morgan, A., & Itier, R. J. (2021). The gaze cueing effect and its enhancement by facial expressions are impacted by task demands: direct comparison of target localization and discrimination tasks. Frontiers in Psychology, 121, Google Scholar logo with link to Google Scholar
Chevalier, P., Kompatsiari, K., Ciardo, F., & Wykowska, A. (2020). Examining joint attention with the use of humanoid robots-A new approach to study fundamental mechanisms of social cognition. Psychonomic bulletin & review, 1–20, Google Scholar logo with link to Google Scholar
Ciardo, F., & Wykowska, A. (2020). La robotica assistenziale sociale come strumento per promuovere lo sviluppo socio-cognitivo: vantaggi, limiti e prospettive future. Sistemi intelligenti, 32(1), 9–25, Google Scholar logo with link to Google Scholar
(2022). Robot’s Social Gaze Affects Conflict Resolution but not Conflict Adaptations. Journal of Cognition, 5(1), 2, Google Scholar logo with link to Google Scholar
Conty, L., George, N., & Hietanen, J. K. (2016). Watching eyes effects: When others meet the self. Consciousness and cognition, 451, 184–197, Google Scholar logo with link to Google Scholar
Cousineau, D. (2005). Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson’s method. Tutorials in quantitative methods for psychology, 1(1), 42–45, Google Scholar logo with link to Google Scholar
Dalmaso, M., Castelli, L., & Galfano, G. (2021). Increased gaze cueing of attention during COVID-19 lockdown. Iscience, 24(11), Google Scholar logo with link to Google Scholar
Davidson, R. J. (2004). What does the prefrontal cortex “do” in affect: perspectives on frontal EEG asymmetry research. Biological psychology, 67(1–2), 219–234, Google Scholar logo with link to Google Scholar
Driver IV, J., Davis, G., Ricciardelli, P., Kidd, P., Maxwell, E., & Baron-Cohen, S. (1999). Gaze perception triggers reflexive visuospatial orienting. Visual cognition, 6(5), 509–540, Google Scholar logo with link to Google Scholar
Edwards, S. G., Stephenson, L. J., Dalmaso, M., & Bayliss, A. P. (2015). Social orienting in gaze leading: a mechanism for shared attention. Proceedings of the Royal Society B: Biological Sciences, 282(1812), Google Scholar logo with link to Google Scholar
Emery, N. J. (2000). The eyes have it: the neuroethology, function and evolution of social gaze. Neuroscience & biobehavioral reviews, 24(6), 581–604, Google Scholar logo with link to Google Scholar
Friesen, C. K., & Kingstone, A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic bulletin & review, 5(3), 490–495, Google Scholar logo with link to Google Scholar
Frischen, A., Bayliss, A. P., & Tipper, S. P. (2007). Gaze cueing of attention: visual attention, social cognition, and individual differences. Psychological bulletin, 133(4), 694, Google Scholar logo with link to Google Scholar
Ghiglino, D., Chevalier, P., Floris, F., Priolo, T., & Wykowska, A. (2021). Follow the white robot: Efficacy of robot-assistive training for children with autism spectrum disorder. Research in Autism Spectrum Disorders, 861, 101822, Google Scholar logo with link to Google Scholar
Hamilton, A. F. D. C. (2016). Gazing at me: the importance of social meaning in understanding direct-gaze cues. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1686), Google Scholar logo with link to Google Scholar
Harmon-Jones, E. (2003). Clarifying the emotive functions of asymmetrical frontal cortical activity. Psychophysiology, 40(6), 838–848, Google Scholar logo with link to Google Scholar
Hietanen, J. K., Leppänen, J. M., Peltola, M. J., Linna-Aho, K., & Ruuhiala, H. J. (2008). Seeing direct and averted gaze activates the approach–avoidance motivational brain systems. Neuropsychologia, 46(9), 2423–2430, Google Scholar logo with link to Google Scholar
Hietanen, J. K., Myllyneva, A., Helminen, T. M., & Lyyra, P. (2016). The effects of genuine eye contact on visuospatial and selective attention. Journal of Experimental Psychology: General, 145(9), 1102, Google Scholar logo with link to Google Scholar
Kajopoulos, J., Cheng, G., Kise, K., Müller, H. J., & Wykowska, A. (2021). Focusing on the face or getting distracted by social signals? The effect of distracting gestures on attentional focus in natural interaction. Psychological Research, 85(2), 491–502, Google Scholar logo with link to Google Scholar
Kiilavuori, H., Sariola, V., Peltola, M. J., & Hietanen, J. K. (2021). Making eye contact with a robot: Psychophysiological responses to eye contact with a human and with a humanoid robot. Biological Psychology, 1581, 107989, Google Scholar logo with link to Google Scholar
Kompatsiari, K., Ciardo, F., Tikhanoff, V., Metta, G., & Wykowska, A. (2018). On the role of eye contact in gaze cueing. Scientific reports, 8(1), 1–10, Google Scholar logo with link to Google Scholar
Kompatsiari, K., Pérez-Osorio, J., De Tommaso, D., Metta, G., & Wykowska, A. (2018, October). Neuroscientifically-grounded research for improved human-robot interaction. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3403–3408), Google Scholar logo with link to Google Scholar
Kompatsiari, K., Bossi, F., & Wykowska, A. (2021). Eye contact during joint attention with a humanoid robot modulates oscillatory brain activity. Social cognitive and affective neuroscience, 16(4), 383–392, Google Scholar logo with link to Google Scholar
Kompatsiari, K., Ciardo, F., & Wykowska, A. (2022). To follow or not to follow your gaze: The interplay between strategic control and the eye contact effect on gaze-induced attention orienting. Journal of Experimental Psychology: General. Advance online publication. Google Scholar logo with link to Google Scholar
Kuhn, G., Vacaityte, I., D’Souza, A. D., Millett, A. C., & Cole, G. G. (2018). Mental states modulate gaze following, but not automatically. Cognition, 1801, 1–9. Google Scholar logo with link to Google Scholar
Lachat, F., Hugueville, L., Lemaréchal, J. D., Conty, L., & George, N. (2012). Oscillatory brain correlates of live joint attention: a dual-EEG study. Frontiers in human neuroscience, 61, 156. Google Scholar logo with link to Google Scholar
Macrae, C. N., Hood, B. M., Milne, A. B., Rowe, A. C., & Mason, M. F. (2002). Are you looking at me? Eye gaze and person perception. Psychological science, 13(5), 460–464, Google Scholar logo with link to Google Scholar
McCrackin, S. D., & Itier, R. J. (2019). Perceived gaze direction differentially affects discrimination of facial emotion, attention, and gender–An ERP study. Frontiers in neuroscience, 131, 517, Google Scholar logo with link to Google Scholar
Metta, G., Natale, L., Nori, F., Sandini, G., Vernon, D., Fadiga, L., … & Montesano, L. (2010). The iCub humanoid robot: An open-systems platform for research in cognitive development. Neural networks, 23(8–9), 1125–1134, Google Scholar logo with link to Google Scholar
Pan, X., & Hamilton, A. F. D. C. (2018). Why and how to use virtual reality to study human social interaction: The challenges of exploring a new research landscape. British Journal of Psychology, 109(3), 395–417, Google Scholar logo with link to Google Scholar
Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., … & Lindeløv, J. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior research methods, 51(1), 195–203, Google Scholar logo with link to Google Scholar
Pönkänen, L. M., Peltola, M. J., & Hietanen, J. K. (2011). The observer observed: Frontal EEG asymmetry and autonomic responses differentiate between another person’s direct and averted gaze when the face is seen live. International Journal of Psychophysiology, 82(2), 180–187, Google Scholar logo with link to Google Scholar
Posner, M. I. (1980). Orienting of attention. Quarterly journal of experimental psychology, 32(1), 3–25, Google Scholar logo with link to Google Scholar
Prinsen, J., & Alaerts, K. (2019). Eye contact enhances interpersonal motor resonance: comparing video stimuli to a live two-person action context. Social cognitive and affective neuroscience, 14(9), 967–976, Google Scholar logo with link to Google Scholar
Risko, E. F., Laidlaw, K. E., Freeth, M., Foulsham, T., & Kingstone, A. (2012). Social attention with real versus reel stimuli: toward an empirical approach to concerns about ecological validity. Frontiers in human neuroscience, 61, 143, Google Scholar logo with link to Google Scholar
Risko, E. F., Richardson, D. C., & Kingstone, A. (2016). Breaking the fourth wall of cognitive science: Real-world social attention and the dual function of gaze. Current Directions in Psychological Science, 25(1), 70–74, Google Scholar logo with link to Google Scholar
Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., & Vogeley, K. (2013). Toward a second-person neuroscience 1. Behavioral and brain sciences, 36(4), 393–414, Google Scholar logo with link to Google Scholar
Schilbach, L. (2014). On the relationship of online and offline social cognition. Frontiers in human neuroscience, 81, 278, Google Scholar logo with link to Google Scholar
Senju, A., & Hasegawa, T. (2005). Direct gaze captures visuospatial attention. Visual cognition, 12(1), 127–144, Google Scholar logo with link to Google Scholar
Senju, A., & Johnson, M. H. (2009). The eye contact effect: mechanisms and development. Trends in cognitive sciences, 13(3), 127–134, Google Scholar logo with link to Google Scholar
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of experimental psychology, 18(6), 643, Google Scholar logo with link to Google Scholar
Team, J. A. S. P. (2016). Jasp. Version 0.8. 0.0. software.Google Scholar logo with link to Google Scholar
Varela, F. J., Rosch, E., & Thompson, E. (1992). The embodied mind: Cognitive science and human experience. MIT press, Google Scholar logo with link to Google Scholar
Wagenmakers, E. J. (2007). A practical solution to the pervasive problems of p values. Psychonomic bulletin & review, 14(5), 779–804, Google Scholar logo with link to Google Scholar
Wiese, E., Wykowska, A., Zwickel, J., & Müller, H. J. (2012). I see what you mean: How attentional selection is shaped by ascribing intentions to others. PLoS ONE, 71, e45391, Google Scholar logo with link to Google Scholar
Wiese, E., Zwickel, J., & Müller, H. J. (2013). The importance of context information for the spatial specificity of gaze cueing. Attention, Perception, & Psychophysics, 75(5), 967–982, Google Scholar logo with link to Google Scholar
Wiese, E., Metta, G., & Wykowska, A. (2017). Robots as intentional agents: using neuroscientific methods to make robots appear more social. Frontiers in psychology, 81, 1663, Google Scholar logo with link to Google Scholar
Wykowska, A., Wiese, E., Prosser, A., & Müller, H. J. (2014). Beliefs about the minds of others influence how we process sensory information. PloS one, 9(4), Google Scholar logo with link to Google Scholar
Wykowska, A., Kajopoulos, J., Ramirez-Amaro, K., & Cheng, G. (2015). Autistic traits and sensitivity to human-like features of robot behavior. Interaction Studies, 16(2), 219–248. Google Scholar logo with link to Google Scholar
Wykowska, A., Chaminade, T., & Cheng, G. (2016). Embodied artificial agents for understanding human social cognition. Philosophical Transactions of the Royal Society B: Biological Sciences, 371(1693), 20150375, Google Scholar logo with link to Google Scholar
Wykowska, A. (2020). Social robots to test flexibility of human social cognition. International Journal of Social Robotics, 12(6), 1203–1211, Google Scholar logo with link to Google Scholar
Xu, S., Zhang, S., & Geng, H. (2018). The effect of eye contact is contingent on visual awareness. Frontiers in psychology, 91, 93, Google Scholar logo with link to Google Scholar
Zhang, X., Dalmaso, M., Galfano, G., & Castelli, L. (2022). Tuning social modulations of gaze cueing via contextual factors. Psychonomic Bulletin & Review, 1–7.2, Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue