Cover not available

Article published In: Interaction Studies
Vol. 16:2 (2015) ► pp.272–302

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

The effects of culture and context on perceptions of robotic facial expressions

 | School of Informatics and Computing, Indiana University | Department of Informatics, Centerstone Research Institute |
 | Department of Informatics, Centerstone Research Institute |
Published online: 26 November 2015
DOI logo with link
https://doi.org/10.1075/is.16.2.11ben
We report two experimental studies of human perceptions of robotic facial expressions while systematically varying context effects and the cultural background of subjects (n = 93). Except for Fear, East Asian and Western subjects were not significantly different in recognition rates, and, while Westerners were better at judging affect from mouth movement alone, East Asians were not any better at judging affect based on eye/brow movement alone. Moreover, context effects appeared capable of over-riding such cultural differences, most notably for Fear. The results seem to run counter to previous theories of cultural differences in facial expression based on emoticons and eye fixation patterns. We connect this to broader research in cognitive science – suggesting the findings support a dynamical systems view of social cognition as an emergent phenomenon. The results here suggest that, if we can induce appropriate context effects, it may be possible to create culture-neutral models of robots and affective interaction.
Article outline
  • 1.Introduction
    • 1.1Background
    • 1.2Related work
    • 1.3The role of human-robot interaction
  • 2.Methods
    • 2.1General overview/subjects
    • 2.2Robotic face
    • 2.3Experimental design
    • 2.4Analysis
  • 3.Experiments
    • 3.1Experiment 1a
      • 3.1.1Experiment 1a – Methods
      • 3.1.2Experiment 1a – Analysis
      • 3.1.3Experiment 1a – Results
    • 3.2Experiment 1b
      • 3.2.1Experiment 1b – Methods
      • 3.2.2Experiment 1b – Analysis
      • 3.2.3Experiment 1b – Results
    • 3.3Experiment 2
      • 3.3.1Experiment 2 – Methods
      • 3.3.2Experiment 2 – Analysis
      • 3.3.3Experiment 2 – Results
  • 4.Discussion
    • 4.1General discussion
    • 4.2Implications
    • 4.3Limitations
    • 4.4Future directions
  • Acknowledgements
  • References
References (84)
References
Adams, B., Breazeal, C., Brooks, R.A., & Scassellati, B. (2000). Humanoid robots: A new kind of tool. IEEE Intelligent Systems and their Applications, 15(4), 25–31. Google Scholar logo with link to Google Scholar
Arizpe, J., Kravitz, D.J., Yovel, G., & Baker, C.I. (2012). Start position strongly influences fixation patterns during face processing: Difficulties with eye movements as a measure of information use. PLoS ONE, 7(2), e31106. Google Scholar logo with link to Google Scholar
Asada, M., Hosoda, K., Kuniyoshi, Y., Ishiguro, H., Inui, T., Yoshikawa, Y., et al. (2009). Cognitive developmental robotics: A survey. IEEE Transactions on Autonomous Mental Development, 1(1), 12–34. Google Scholar logo with link to Google Scholar
Auvray, M., Lenay, C., & Stewart, J. (2009). Perceptual interactions in a minimalist virtual environment. New Ideas in Psychology, 27(1), 32–47. Google Scholar logo with link to Google Scholar
Barrett, L.F., Mesquita, B., & Gendron, M. (2011). Context in emotion perception. Current Directions in Psychological Science, 20(5), 286–290. Google Scholar logo with link to Google Scholar
Barsalou, L.W., Breazeal, C., & Smith L.B. (2007). Cognition as coordinated non-cognition. Cognitive Processing, 8(2), 79–91. Google Scholar logo with link to Google Scholar
Bartneck, C. & Okada, M. (2001). Robotic user interfaces. Proceedings of the Human and Computer Conference (HC2001) , 130–140.
Bartneck, C., Suzuki, T., Kanda, T., & Nomura, T. (2007). The influence of people’s culture and prior experiences with Aibo on their attitude towards robots. AI & Society, 21(1–2), 217–30. Google Scholar logo with link to Google Scholar
Bartneck, D., Kulic, E., Croft, M., & Zoghbi, S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics, 1(1), 71–81. Google Scholar logo with link to Google Scholar
Bazo, D., Vaidyanathan, R., Lentz, A., & Melhuish, C. (2010). Design and testing of a hybrid expressive face for a humanoid robot. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 5317–5322.
Becker-Asano, C. & Ishiguro, H. (2011). Evaluating facial displays of emotion for the android robot Geminoid F. Proceedings of the IEEE Workshop on Affective Computational Intelligence (WACI) , 1–8.
Beer, R.D. (1995). A dynamical systems perspective on agent-environment interaction. Artificial Intelligence, 72(1–2), 173–215. Google Scholar logo with link to Google Scholar
. (2000). Dynamical approaches to cognitive science. Trends in Cognitive Sciences, 4(3), 91–99. Google Scholar logo with link to Google Scholar
Bennett, C.C., & Šabanović, S. (2013). Perceptions of affective expression in a minimalist robotic face. Proceedings of the ACM/IEEE Conference on Human-Robot Interaction (HRI) , 81–82.
. (2014). Deriving minimal features for human-like facial expressions in robotic faces. International Journal of Social Robotics, 6(3), 367–381. Google Scholar logo with link to Google Scholar
Bennett, C.C., Šabanović, S., Fraune, M., & Shaw, K. (2014). Context congruency and robotic facial expressions: Do effects on human perceptions vary across culture? Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) , 465–470.
Biehl, M., Matsumoto, D., Ekman, P., Hearn, V., Heider, K., Kudoh, T., & Ton, V. (1997). Matsumoto and Ekman’s Japanese and Caucasian Facial Expressions of Emotion (JACFEE): Reliability data and cross-national differences. Journal of Nonverbal Behavior, 21(1), 3–21. Google Scholar logo with link to Google Scholar
Blais, C., Roy, C., Fiset, D., Arguin, M., & Gosselin, F. (2012). The eyes are not the window to basic emotions. Neuropsychologia, 50(12), 2830–2838. Google Scholar logo with link to Google Scholar
Breazeal, C. (2003). Emotion and sociable humanoid robots. International Journal of Human-Computer Studies, 59(1–2), 119–155. Google Scholar logo with link to Google Scholar
Boiger, M., & Mesquita, B. (2012). The construction of emotion in interactions, relationships, and cultures. Emotion Review, 4(3), 221–229. Google Scholar logo with link to Google Scholar
Bosse, T., Pontier, M., & Treur, J. (2010). A computational model based on Gross’ emotion regulation theory. Cognitive Systems Research, 11(3), 211–230. Google Scholar logo with link to Google Scholar
Canamero, L. (2005). Emotion understanding from the perspective of autonomous robots research. Neural Networks, 18(4), 445–455. Google Scholar logo with link to Google Scholar
Canamero, L., & Fredslund, J. (2001). I show you how I like you – can you read it in my face? IEEE Transactions on Systems Man and Cybernetics A: Systems and Humans, 31(5), 454–459. Google Scholar logo with link to Google Scholar
Caporael, L.R. (1997). The evolution of truly social cognition: The core configuration model. Personality and Social Psychology Review, 1(4), 276–298. Google Scholar logo with link to Google Scholar
Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3), 181–204. Google Scholar logo with link to Google Scholar
Cohn, J.F. (2010). Advances in behavioral science using automated facial image analysis and synthesis. IEEE Signal Process Magazine, 27(6), 128–133. Google Scholar logo with link to Google Scholar
Darwin, C. (1872). The expression of the emotions in man and animals. London, UK: John Murray. Google Scholar logo with link to Google Scholar
Davies, I.R., Sowden, P.T., Jerrett, D.T., Jerret,t T., & Corbett, G.G. (1998). A cross-cultural study of English and Setswana speakers on a colour triads task : A test of the Sapir-Whorf hypothesis. British Journal of Psychology, 89(1), 1–15. Google Scholar logo with link to Google Scholar
De Gelder, B. (2009). Why bodies? Twelve reasons for including bodily expressions in affective neuroscience. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 364(1535), 3475–3484. Google Scholar logo with link to Google Scholar
De Jaegher, H., Di Paolo, E., & Gallagher, S. (2010). Can social interaction constitute social cognition? Trends in Cognitive Sciences, 14(10), 441–447. Google Scholar logo with link to Google Scholar
Ekman, P. (1971). Universals and cultural differences in facial expressions of emotion. Nebraska Symposium on Motivation, 191, 207–283.Google Scholar logo with link to Google Scholar
. (2009). Darwin’s contributions to our understanding of emotional expressions. ­Philosophical Transactions of the Royal Society of London B: Biological Sciences, 364(1535), 3449–3451. Google Scholar logo with link to Google Scholar
Ekman, P., & Friesen, W.V. (2003). Unmasking the face: A guide to recognizing emotions from facial clues. Los Altos, CA, USA: Malor Books.Google Scholar logo with link to Google Scholar
Elfenbein, H.A. (2013). Nonverbal dialects and accents in facial expressions of emotion. Emotion Review, 5(1), 90–96. Google Scholar logo with link to Google Scholar
Embgen, S., Luber, M., Becker-Asano, C., Ragni, M., Evers, V., & Arras, K.O. (2012). Robot-specific social cues in emotional body language. Proceedings of the IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) , 1019–1025.
Friesen, W.V. (1973). Cultural differences in facial expressions in a social situation: An experimental test on the concept of display rules. Dissertation Abstracts International, 33(8-B), 3976–3977.Google Scholar logo with link to Google Scholar
Froese, T., & Di Paolo, E.A. (2011). The enactive approach: Theoretical sketches from cell to society. Pragmatics & Cognition, 19(1), 1–36. Google Scholar logo with link to Google Scholar
Froese, T., & Ziemke, T. (2009). Enactive artificial intelligence: Investigating the systemic organization of life and mind. Artificial Intelligence, 173(3–4), 466–500. Google Scholar logo with link to Google Scholar
Fugate, J.M.B. (2013). Categorical perception for emotional faces. Emotion Review, 5(1), 84–89. Google Scholar logo with link to Google Scholar
Geraci, R.M. (2006). Spiritual robots: Religion and our scientific view of the natural world. Theology and Science, 4(3), 229–246. Google Scholar logo with link to Google Scholar
Gibson, J.J. (1979). The ecological approach to visual perception. Boston, MA, USA: Houghlin Mifflin. Google Scholar logo with link to Google Scholar
Gross, J.J. & Levenson, R.W. (1995). Emotion elicitation using films. Cognition and Emotion, 9(1), 87–108. Google Scholar logo with link to Google Scholar
Hall, E.T. (1977). Beyond culture. New York, NY, USA: Anchor Books.Google Scholar logo with link to Google Scholar
Ikegami, T. & Suzuki, K. (2008). From a homeostatic to a homeodynamic self. BioSystems, 91(2), 388–400. Google Scholar logo with link to Google Scholar
Jack, R.E., Blais, C., Scheepers, C., Schyns, P.G., & Caldara, R. (2009). Cultural confusions show that facial expressions are not universal. Current Biology, 19(18), 1543–1548. Google Scholar logo with link to Google Scholar
Jack, R.E., Garrod, O.G., Yu, H., Caldara, R., & Schyns, P.G. (2012). Facial expressions of emotion are not culturally universal. Proceeedings of the National Acadamy of Sciences USA , 109(19), 7241–7244.
Kaplan, F. (2004). Who is afraid of the humanoid? Investigating cultural differences in the acceptance of robots. International Journal of Humanoid Robotics, 1(3), 465–480. Google Scholar logo with link to Google Scholar
Kleinsmith, A., De Silva, P.R., & Bianchi-Berthouze, N. (2006). Cross-cultural differences in recognizing affect from body posture. Interacting with Computers, 18(6), 1371–89. Google Scholar logo with link to Google Scholar
Kobayashi, H., Ichikawa, Y., Senda, M., & Shiiba, T. (2003). Realization of realistic and rich facial expressions by face robot. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 1123–1128.
Koda, T., Ruttkay, Z., Nakagawa, Y., & Tabuchi, K. (2010). Cross-cultural study on facial regions as cues to recognize emotions of virtual agents. In T. Ishida (Ed.), Lecture Notes in Computer Science (Culture and Computing) (pp. 16–27). Springer, Berlin. Google Scholar logo with link to Google Scholar
Kozima, H., Michalowski, M.P., & Nakagawa, C. (2009). Keepon: A playful robot for research, therapy, and entertainment. International Journal of Social Robotics, 1(1), 3–18. Google Scholar logo with link to Google Scholar
Krumhuber, E.G., Kappas, A., & Manstead, A.S.R. (2013). Effects of dynamic aspects of facial expressions: A review. Emotion Review, 5(1), 41–46. Google Scholar logo with link to Google Scholar
Lee, T.H., Choi, J.S., & Cho, Y.S. (2012). Context modulation of facial emotion perception differed by individual difference. PLoS ONE, 7(3), e32987. Google Scholar logo with link to Google Scholar
Lee, H.R., & Sabanović, S. (2014). Culturally variable preferences for robot design and use in South Korea, Turkey, and the United States. Proceedings of the ACM/IEEE Conference on Human-Robot Interaction (HRI) , 17–24.
Lindquist, K.A. & Gendron, M. (2013). What’s in a word? Language constructs emotion perception. Emotion Review, 5(1), 66–71. Google Scholar logo with link to Google Scholar
MacDorman, K.F., Vasudevan, S.K., & Ho, C.C. (2009). Does Japan really have robot mania? Comparing attitudes by implicit and explicit measures. AI & Society, 23(4), 485–510. Google Scholar logo with link to Google Scholar
Matsumoto, D. (1992). American-Japanese cultural differences in the recognition of universal facial expressions. Journal of Cross-Cultural Psychology, 23(1), 72–84. Google Scholar logo with link to Google Scholar
Matsumoto, N., Fujii, H., & Okada, M. (2006). Minimal design for human-agent communication. Artificial Life and Robotics, 10(1), 49–54. Google Scholar logo with link to Google Scholar
McGann. M., De Jaegher, H., & Di Paolo, E. (2013). Enaction and psychology. Review of General Psychology, 17(2), 203–209. Google Scholar logo with link to Google Scholar
Miwa, H., Itoh, K., Matsumoto, M., Zecca, M., Takanobu, H., Rocella, S., et al. (2004). Effective emotional expressions with expression humanoid robot we-4rii: Integration of humanoid robot hand rch-1. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2203–2208.
Mori, M. (1970). Bukimi no tani[The uncanny valley]. Energy, 7(4), 33–35. [URL]Google Scholar logo with link to Google Scholar
Nelson, N.L., & Russell, J.A. (2013). Universality revisited. Emotion Review, 5(1), 8–15. Google Scholar logo with link to Google Scholar
Nisbett, R.E., Peng, K., Choi, I., & Norenzayan, A. (2001). Culture and systems of thought: Holistic versus analytic cognition. Psychological Review, 108(2), 291–310. Google Scholar logo with link to Google Scholar
Nisbett, R.E. (2003). The geography of thought: How Asians and Westerners think differently. New York, NY, USA: Free Press. Google Scholar logo with link to Google Scholar
Nomura, T., & Kanda, T. (2003). On proposing the concept of robot anxiety and considering measurement of it. Proceedings of 12th IEEE International Symposium on Robot and Human interactive Communication (RO-MAN) , 373–378.
Nomura, T., Suzuki, T., Kanda, T., & Kato, K. (2006). Measurement of negative attitudes toward robots. Interaction Studies, 7(3), 437–454. Google Scholar logo with link to Google Scholar
Pantic, M. (2009). Machine analysis of facial behaviour: Naturalistic and dynamic behaviour. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 364(1535), 3505–3513. Google Scholar logo with link to Google Scholar
Perlovsky, L. (2009). Language and emotions: Emotional Sapir-Whorf hypothesis. Neural Networks, 22(5–6), 518–526. Google Scholar logo with link to Google Scholar
Peterson, M.F., & Eckstein, M.P. (2012). Looking just below the eyes is optimal across face recognition tasks. Proceeedings of the National Acadamy of Sciences USA , 109(48), E3314–3323.
Picard, R.W. (1997). Affective computing. Boston, MA, USA: MIT Press. Google Scholar logo with link to Google Scholar
Righart, R., & de Gelder, B. (2008). Recognition of facial expressions is influenced by emotional scene gist. Cognitive Affective & Behavavioral Neuroscience, 8(3), 264–272. Google Scholar logo with link to Google Scholar
Russell, J.A., & Fernández-Dols, J.M. (1997). The psychology of facial expression. Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Ruttkay, Z. (2009). Cultural dialects of real and synthetic emotional facial expressions. AI & Society, 24(3), 307–315. Google Scholar logo with link to Google Scholar
Šabanović, S., Michalowski, M.P., & Simmons R. (2006). Robots in the wild: Observing human-robot social interaction outside the lab. 9th IEEE International Workshop on Advanced Motion Control , 596–601.
Šabanović, S. (2010). Emotion in robot cultures: Cultural models of affect in social robot design. Proceedings of the Conference on Design & Emotion (D&E2010) , Chicago, IL.
. (2014). Inventing Japan’s “robotics culture”: The repeated assembly of science, technology, and culture in social robotics. Social Studies of Science, 44(3), 342–367. Available online: [URL]. Google Scholar logo with link to Google Scholar
Saldien, J., Goris, K., Vanderborght, B., Vanderfaeillie, J., & Lefeber, D. 2010. Expressing emotions with the social robot Probo. International Journal of Social Robotics, 2(4), 377–389. Google Scholar logo with link to Google Scholar
Scassellati, B. (2006). How developmental psychology and robotics complement each other. Technical report, Massachusetts Institute of Technology, CSAIL.
Shore, B. (1996). Culture in mind: Cognition, culture, and the problem of meaning. Oxford, UK: Oxford University Press.Google Scholar logo with link to Google Scholar
Sosnowski, S., Bittermann, A., Kuhnlenz, K., & Buss, M. (2006). Design and evaluation of emotion-display EDDIE. Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 3113–3118.
Sterelny, K. (2007). Social intelligence, human intelligence and niche construction. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1480), 719–730. Google Scholar logo with link to Google Scholar
Yuki, M., Maddux, W.W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43(2), 303–311. Google Scholar logo with link to Google Scholar
Zhang, J., & Sharkey, A.J. (2011). Contextual recognition of robot emotions. In R. Groß et al. (Ed.), Towards autonomous robotic systems (pp. 78–89). Berlin, Germany: Springer. Google Scholar logo with link to Google Scholar
Cited by (10)

Cited by ten other publications

Fernandes, Rafaella, Larissa Costa, Priscila Machado & Judith Kelner
2026. An Exploratory Study on Emotions Conveyed Through Facial Expressions in Screen-Face Social Robots: Cultural Perception and Design Implications in Developing Countries. In Human-Computer Interaction – INTERACT 2025 [Lecture Notes in Computer Science, 16108], ► pp. 319 ff. DOI logo
Fujii, Ayaka, Carlos Toshinori Ishi, Kurima Sakai, Tomo Funayama, Ritsuko Iwai, Yusuke Takahashi, Takatsune Kumada & Takashi Minato
2025. Evaluating human perceptions of android robot facial expressions based on variations in instruction styles. Frontiers in Robotics and AI 12 DOI logo
Li, Ranran, Halle Quang, Michaela Filipčíková, Yi Xu, Fiona Kumfor, Branka Spehar & Skye McDonald
2025. Why Do Cultures Affect Facial Emotion Perception? A Systematic Review. Journal of Cross-Cultural Psychology 56:6 ► pp. 680 ff. DOI logo
Li, Mingming, Fu Guo, Chen Fang & Fengxiang Li
2023. Multisensory integration effect of humanoid robot appearance and voice on users’ affective preference and visual attention. Behaviour & Information Technology 42:14 ► pp. 2387 ff. DOI logo
Penčić, Marko, Maja Čavić, Dragana Oros, Petar Vrgović, Kalman Babković, Marko Orošnjak & Dijana Čavić
2022. Anthropomorphic Robotic Eyes: Structural Design and Non-Verbal Communication Effectiveness. Sensors 22:8 ► pp. 3060 ff. DOI logo
Robinson, Fraser & Goldie Nejat
2022. An analysis of design recommendations for socially assistive robot helpers for effective human-robot interactions in senior care. Journal of Rehabilitation and Assistive Technologies Engineering 9 DOI logo
Wullenkord, Ricarda, Jacqueline Bellon, Bruno Gransche, Sebastian Nähr-Wagener & Friederike Eyssel
2022. Social appropriateness in HMI. Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems 23:3 ► pp. 360 ff. DOI logo
Ahn, Yunjeong & Jieun Lee
2021. The Role of Anthropomorphic Messengers in Sustainable Participatory Corporate Social Responsibility: Focusing on Messenger’s Facial Expression and Participation Effort. Sustainability 13:8 ► pp. 4365 ff. DOI logo
Bennett, Casey
2021. Emergent Robotic Personality Traits via Agent-Based Simulation of Abstract Social Environments. Information 12:3 ► pp. 103 ff. DOI logo
Chesher, Chris & Fiona Andreallo
2021. Robotic Faciality: The Philosophy, Science and Art of Robot Faces. International Journal of Social Robotics 13:1 ► pp. 83 ff. DOI logo

This list is based on CrossRef data as of 16 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue