Cover not available

Article published In: Interaction Studies
Vol. 18:2 (2017) ► pp.254–275

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

An experimental approach to study the physiology of natural social interactions

 | Institut de Neurosciences de la Timone |
Published online: 8 December 2017
DOI logo with link
https://doi.org/10.1075/is.18.2.06gry
The classical experimental methodology is ill-suited for the investigation of the behavioral and physiological correlates of natural social interactions. A new experimental approach combining a natural conversation between two persons with control conditions is proposed in this paper. Behavior, including gaze direction and speech, and physiology, including electrodermal activity, are recorded during a discussion between two participants through videoconferencing. Control for the social aspect of the interaction is provided by the use of an artificial agent and of videoed conditions. A cover story provides spurious explanations for the purpose of the experiment and for the recordings, as well as a controlled and engaging topic of discussion. Preprocessing entails transforming raw measurements into boxcar and delta functions time series indicating when a certain behaviour or physiological event is present. The preliminary analysis presented here consists in finding statistically significant differences between experimental conditions in the temporal associations between behavioral and physiological time series. Significant results validate the experimental approach and further developments including more elaborate analysis and adaptation of the paradigm to functional MRI are discussed.
Article outline
  • 1.Introduction
    • 2.Summary of the experimental approach
  • 3.Methods
    • 3.1Participants
    • 3.2Experimental paradigm
      • 3.2.1Cover story
      • 3.2.2Experimental conditions
    • 3.3Experimental setup
      • 3.3.1Embodied conversational agent
      • 3.3.2Physical setup
    • 3.4Experimental recording
    • 3.5Data preprocessing
      • 3.5.1Electrodermal activity
      • 3.5.2Eyetracking
      • 3.5.3Conversation behaviour
    • 3.6Statistical analysis
  • 4.Results
    • 4.1Effect of temporal resolution of the analysis
    • 4.2Relation with verbal behaviour
    • 4.3Relation with oculomotor behaviours
  • 5.Discussion
    • 5.1Current findings
    • 5.2Further developments
  • Acknowledgements
  • References
References (32)
References
Bach, D. R., & Friston, K. J.. (2013). Model-based analysis of skin conductance responses: Towards causal models in psychophysiology. Psychophysiology, 50(1), 15–22. Google Scholar logo with link to Google Scholar
Benedek, M., & Kaernbach, C.. (2010). Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology, 47(4), 647–658.Google Scholar logo with link to Google Scholar
Bigi, B., Watanabe, T., & Prévot, L.. (2014). Representing multimodal linguistics annotated data. Proceedings of 9th International conference on Language Resources and Evaluation (LREC) , Reykjavik (Iceland).
Chaminade, T., Hodgins, J., & Kawato, M.. (2007). Anthropomorphism influences perception of computer-animated characters' actions. Social Cognitive and Affective Neuroscience, 2(3), 206–216. Google Scholar logo with link to Google Scholar
Chaminade, T., & Cheng, G.. (2009). Social cognitive neuroscience and humanoid robotics. Journal of Physiology-Paris, 103(3-5), 286–295. Google Scholar logo with link to Google Scholar
Chaminade, T., Rosset, D., Fonseca, D. D., Nazarian, B., Lutcher, E., Cheng, G., & Deruelle, C.. (2012). How do we think machines think? An fMRI study of alleged competition with an artificial intelligence. Frontiers in Human Neuroscience, 6, 103.Google Scholar logo with link to Google Scholar
Chaminade, T., Fonseca, D., Rosset, D., Cheng, G., & Deruelle, C.. (2015). Atypical modulation of hypothalamic activity by social context in ASD. Research in Autism Spectrum Disorders, 101, 41–50. Google Scholar logo with link to Google Scholar
Chartrand, T. L., & Bargh, J. A.. (1999). The chameleon effect: The perception–behavior link and social interaction. Journal of personality and social psychology, 76(6), 893–910. Google Scholar logo with link to Google Scholar
Dawson, M. E., Schell, A. M., & Filion, D. L.. (2007). The electrodermal system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.). Handbook of psychophysiology (3rd ed.; pp. 159–181). Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Dennett, D. C.. (1996). The Intentional Stance (6th printing). Cambridge, MA, USA: The MIT Press.Google Scholar logo with link to Google Scholar
Diehl, J. J., Schmitt, L. M., Villano, M., & Crowell, C. R.. (2012). The clinical use of robots for individuals with Autism Spectrum Disorders: A critical review. Research in Autism Spectrum Disorders, 6(1), 249–262. Google Scholar logo with link to Google Scholar
French, D. P., & Sutton, S.. (2010). Reactivity of measurement in health psychology: How much of a problem is it? What can be done about it? British Journal of Health Psychology, 15(3), 453–468. Google Scholar logo with link to Google Scholar
Frith, C. D., & Allen, H. A.. (1983). The skin conductance orienting response as an index of attention. Biological psychology, 17(1), 27–39. Google Scholar logo with link to Google Scholar
Hauber, J., Regenbrecht, H., Hills, A., Cockburn, A., & Billinghurst, M.. (2005). Social presence in two-and three-dimensional videoconferencing. Proceedings of 8th Annual International Workshop on Presence , London (UK), 189–198.
Khalfa, S., Isabelle, P., Jean-Pierre, B., & Manon, R.. (2002). Event-related skin conductance responses to musical emotions in humans. Neuroscience letters, 328(2), 145–149. Google Scholar logo with link to Google Scholar
Kilpatrick, D. G.. (1972). Differential responsiveness of two electrodermal indices to psychological stress and performance of a complex cognitive task. Psychophysiology, 9(2), 218–226. Google Scholar logo with link to Google Scholar
Krach, S., Hegel, F., Wrede, B., Sagerer, G., Binkofski, F., & Kircher, T.. (2008). Can Machines Think? Interaction and Perspective Taking with Robots Investigated via fMRI. PLoS ONE, 3(7), e2597. Google Scholar logo with link to Google Scholar
Kuster, C., Popa, T., Bazin, J.-C., Gotsman, C., & Gross, M.. (2012). Gaze correction for home video conferencing. ACM Transactions on Graphics, 31(6), 174. Google Scholar logo with link to Google Scholar
Laming, D. R. J.. (1968). Information theory of choice-reaction times. London: Academic Press.Google Scholar logo with link to Google Scholar
Nomura, T., Suzuki, T., Kanda, T., & Kato, K.. (2006). Measurement of negative attitudes toward robots. Interaction Studies, 7(3), 437–454 Google Scholar logo with link to Google Scholar
Ochs, M., Niewiadomski, R., Brunet, P., & Pelachaud, C.. (2012). Smiling virtual agent in social context. Cognitive Processing, 13(S2), 519–532. Google Scholar logo with link to Google Scholar
Pelachaud, C.. (2009). Modelling multimodal expression of emotion in a virtual agent. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3539–3548. Google Scholar logo with link to Google Scholar
Pfungst, O.. (1911). Clever Hans (The horse of Mr. von Osten): A contribution to experimental animal and human psychology (Trans. C. L. Rahn). New York: Henry Holt. Google Scholar logo with link to Google Scholar
Riek, L. D.. (2012). Wizard of oz studies in hri: A systematic review and new reporting guidelines. Journal of Human-Robot Interaction 11.Google Scholar logo with link to Google Scholar
Rolison, M. J., Naples, A. J., & McPartland, J. C.. (2015). Interactive Social Neuroscience to Study Autism Spectrum Disorder. The Yale Journal of Biology and Medicine, 88(1), 17–24.Google Scholar logo with link to Google Scholar
Roth, W. T., Dawson, M. E., & Filion, D. L.. (2012). Publication recommendations for electrodermal measurements. Psychophysiology, 491, 1017–1034. Google Scholar logo with link to Google Scholar
Schilbach, L.. (2010). A second-person approach to other minds. Nature Reviews Neuroscience, 11(6), 449–449. Google Scholar logo with link to Google Scholar
Schilbach, L., Timmermans, B., Reddy, V., Costall, A., Bente, G., Schlicht, T., & Vogeley, K.. (2013). Toward a second-person neuroscience. Behavioral And Brain Sciences, 36(4), 393–414. Google Scholar logo with link to Google Scholar
Senju, A., & Johnson, M. H.. (2009). The eye contact effect: Mechanisms and development. Trends in Cognitive Sciences, 13(3), 127–134. Google Scholar logo with link to Google Scholar
Turing, A.. (1950), Computing machinery and intelligence. Mind, 59(236), 433–460. Google Scholar logo with link to Google Scholar
Wykowska, A., Chaminade, T., & Cheng, G.. (2016). Embodied artificial agents for understanding human social cognition. Philosophical Transactions of the Royal Society Biological Sciences, 371(1693), 20150375. Google Scholar logo with link to Google Scholar
Xiong, X., & De la Torre, F.. (2013). Supervised descent method and its application to face alignment. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar logo with link to Google Scholar
Cited by (13)

Cited by 13 other publications

Weinstein, Netta, Guy Itzchakov & Michael R. Maniaci
2025. Exploring the connecting potential of AI: Integrating human interpersonal listening and parasocial support into human-computer interactions. Computers in Human Behavior: Artificial Humans 4 ► pp. 100149 ff. DOI logo
Hmamouche, Youssef, Magalie Ochs, Laurent Prévot, Thierry Chaminade & Kiyoshi Nakahara
2024. Interpretable prediction of brain activity during conversations from multimodal behavioral signals. PLOS ONE 19:3 ► pp. e0284342 ff. DOI logo
Chaminade, Thierry & Nicolas Spatola
2022. Perceived facial happiness during conversation correlates with insular and hypothalamus activity for humans, not robots. Frontiers in Psychology 13 DOI logo
Datteri, Edoardo, Thierry Chaminade & Donato Romano
2022. Going Beyond the “Synthetic Method”: New Paradigms Cross-Fertilizing Robotics and Cognitive Neuroscience. Frontiers in Psychology 13 DOI logo
De Castro Martins, Caio, Thierry Chaminade & Marc Cavazza
2022. Causal Analysis of Activity in Social Brain Areas During Human-Agent Conversation. Frontiers in Neuroergonomics 3 DOI logo
Hogenhuis, Ann & Ruud Hortensius
2022. Domain‐specific and domain‐general neural network engagement during human–robot interactions. European Journal of Neuroscience 56:10 ► pp. 5902 ff. DOI logo
Spatola, Nicolas & Thierry Chaminade
2022. Precuneus brain response changes differently during human–robot and human–human dyadic social interaction. Scientific Reports 12:1 DOI logo
Rauchbauer, Birgit, Bruno Nazarian, Morgane Bourhis, Magalie Ochs, Laurent Prévot & Thierry Chaminade
2019. Brain activity during reciprocal social interaction investigated using conversational robots as control condition. Philosophical Transactions of the Royal Society B: Biological Sciences 374:1771 ► pp. 20180033 ff. DOI logo
Riou, Matthieu, Bassam Jabaian, Stéphane Huet & Fabrice Lefèvre
2019. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ► pp. 3072 ff. DOI logo
Riou, Matthieu, Bassam Jabaian, Stéphane Huet & Fabrice Lefèvre
2019. Lilia, A Showcase for Fast Bootstrap of Conversation-Like Dialogues Based on a Goal-Oriented System. In Statistical Language and Speech Processing [Lecture Notes in Computer Science, 11816], ► pp. 31 ff. DOI logo
Rudnicki, Konrad, Carolyn Declerck, Charlotte De Backer & Mario Berth
2019. Physiological changes during first encounters and their role in determining the perceived interaction quality. Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems 20:2 ► pp. 275 ff. DOI logo
Nicolas Spatola
2019. L’interaction Homme-Robot, de l’anthropomorphisme à l’humanisation. L’Année psychologique Vol. 119:4 ► pp. 515 ff. DOI logo
Chaminade, Thierry, Birgit Rauchbauer, Bruno Nazarian, Morgane Bourhis, Magalie Ochs & Laurent Prévot
2018. Proceedings of the 20th International Conference on Multimodal Interaction: Adjunct, ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 17 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue