Cover not available

Article In: Group Dynamics in Human–Robot Interaction
Edited by Alessandra Sciutti, Dario Pasquali, Giulia Belgiovine and Linda Lastrico
[Interaction Studies 26:3] 2025
► pp. 477–505

If they disagree, will you conform?

Exploring the role of robots’ value awareness in a decision-making task

ORCiD logo with link | Italian Institute of TechnologyGenoa
 | Ghent University
 | Italian Institute of TechnologyGenoa
 | Ghent University
 | Italian Institute of TechnologyGenoa
This content is being prepared for publication; it may be subject to changes.
Abstract
This study investigates whether the opinions of robotic agents are more likely to influence human decision-making when the robots are perceived as value-aware (i.e., when they display an understanding of human principles). We designed an experiment in which participants interacted with two Furhat robots — one programmed to be Value-Aware and the other Non-Value-Aware — during a labeling task for images representing human values. Results indicate that participants distinguished the Value-Aware robot from the Non-Value-Aware one. Although their explicit choices did not indicate a clear preference for one robot over the other, participants directed their gaze more toward the Value-Aware robot. Additionally, the Value-Aware robot was perceived as more loyal, suggesting that value awareness in a social robot may enhance its perceived commitment to the group. Finally, when both robots disagreed with the participant, conformity occurred in about one out of four trials, and participants took longer to confirm their responses, suggesting that two robots expressing dissent may introduce hesitation in decision-making. On one hand, this highlights the potential risk that robots, if misused, could manipulate users for unethical purposes. On the other hand, it reinforces the idea that social robots might encourage reflection in ambiguous situations and help users avoid scams.
Article outline
  • 1.Introduction & motivation
  • 2.Related work
    • 2.1Values in human-robot interaction
    • 2.2Conformity and influence in human-robot teams
  • 3.Hypotheses and structure of the study
  • 4.Stimuli selection
    • 4.1Participants
    • 4.2Questionnaire
    • 4.3Data analysis & selection criteria
  • 5.Experiments with robotic agents
    • 5.1Participants
    • 5.2Setup
    • 5.3Experimental task description
    • 5.4Furhat robots
    • 5.5Software & data acquisition
    • 5.6Data analysis
      • 5.6.1Final questionnaire
      • 5.6.2Responses data in the images-keyword association
      • 5.6.3Face data in the description task
  • 6.Results
    • 6.1Final questionnaire
    • 6.2Responses in the images-keyword association
      • 6.2.1Gaze in the description task
  • 7.Discussion
  • 8.Conclusion
  • Acknowledgements
  • Note
  • References
References (44)
References
Abate, Andrea F., Carmen Bisogni, Lucia Cascone, Aniello Castiglione, Gerardo Costabile & Ilenia Mercuri. 2020. Social robot interactions for social engineering: Opportunities and open issues. In 2020 ieee intl conf on dependable, autonomic and secure computing, intl conf on pervasive intelligence and computing, intl conf on cloud and big data computing, intl conf on cyber science and technology congress (dasc/picom/cbdcom/cyberscitech), 539–547. IEEE. Google Scholar logo with link to Google Scholar
Abbo, Giulio Antonio & Tony Belpaeme. 2023. Users’ perspectives on value awareness in social robots. In Hri 2023, the 18th acm/ieee international conference on human-robot interaction, 1–5.Google Scholar logo with link to Google Scholar
Abbo, Giulio Antonio, Tony Belpaeme & Micol Spitale. 2025. Concerns and values in humanrobot interactions: A focus on social robotics. arXiv preprint arXiv:2501.05628.Google Scholar logo with link to Google Scholar
Alves-Oliveira, Patricia, Pedro Sequeira & Ana Paiva. 2016. The role that an educational robot plays. In 2016 25th ieee international symposium on robot and human interactive communication (ro-man), 817–822. IEEE. Google Scholar logo with link to Google Scholar
Andtfolk, Malin, Linda Nyholm, Hilde Eide & Lisbeth Fagerström. 2022. Humanoid robots in the care of older persons: A scoping review. Assistive Technology 34(5). 518–526. Google Scholar logo with link to Google Scholar
Aroyo, Alexander Mois, Francesco Rea, Giulio Sandini & Alessandra Sciutti. 2018. Trust and social engineering in human robot interaction: Will a robot make you disclose sensitive information, conform to its recommendations or gamble? IEEE Robotics and Automation Letters 3(4). 3701–3708. Google Scholar logo with link to Google Scholar
Asch, Solomon E. 1951. Effects of group pressure on the modification and distortion of judgments. In Harold Guetzkow (ed.), Groups, leadership and men, 177–190. Carnegie Press.Google Scholar logo with link to Google Scholar
Belpaeme, Tony, James Kennedy, Aditi Ramachandran, Brian Scassellati & Fumihide Tanaka. 2018. Social robots for education: A review. Science robotics 3(21). Google Scholar logo with link to Google Scholar
Belpaeme, Tony & Fumihide Tanaka. 2021. Social robots as educators. In OECD Digital Education Outlook 2021 Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, 1431. OECD Publishing Paris. Google Scholar logo with link to Google Scholar
Bhat, Shreyas, Joseph B. Lyons, Cong Shi & X Jessie Yang. 2024. Evaluating the impact of personalized value alignment in human-robot interaction: Insights into trust and team performance outcomes. In Proceedings of the 2024 acm/ieee international conference on human-robot interaction, 32–41. Google Scholar logo with link to Google Scholar
Brandstetter, Jurgen, Péter Rácz, Clay Beckner, Eduardo B. Sandoval, Jennifer Hay & Christoph Bartneck. 2014. A peer pressure experiment: Recreation of the asch conformity experiment with robots. In 2014 ieee/rsj international conference on intelligent robots and systems, 1335–1340. IEEE. Google Scholar logo with link to Google Scholar
Chevalier, Pauline, Bob R. Schadenberg, Amir Aly, Angelo Cangelosi & Adriana Tapus. 2022. Context-awareness in human-robot interaction: Approaches and challenges. In 2022 17th ACM/IEEE international conference on human-robot interaction (hri), 1241–1243. IEEE. Google Scholar logo with link to Google Scholar
Chyung, Seung Youn, Katherine Roberts, Ieva Swanson & Andrea Hankinson. 2017. Evidencebased survey design: The use of a midpoint on the likert scale. Performance improvement 56(10). 15–23. Google Scholar logo with link to Google Scholar
Cialdini, Robert B. & Noah J. Goldstein. 2004. Social influence: Compliance and conformity. Annu. Rev. Psychol. 55(1). 591–621. Google Scholar logo with link to Google Scholar
Cifuentes, Carlos A., Maria J. Pinto, Nathalia Céspedes & Marcela Múnera. 2020. Social robots in therapy and care. Current Robotics Reports 11. 59–74. Google Scholar logo with link to Google Scholar
Ciupinska, Kinga, Serena Marchesi, Giulio Antonio Abbo, Tony Belpaeme & Agnieszka Wykowska. 2024a. Awareprompt : using diffusion models to create methods for measuring value-aware ai architectures. In Ana Paula Rocha, Luc Steels & Jaap van den Herik (eds.), Proceedings of the 16th international conference on agents and artificial intelligence : volume 3, 1436–1443. SCITEPRESS. Google Scholar logo with link to Google Scholar
. 2024b. Stimuli from: Awareprompt: Using diffusion models to create methods for measuring value-aware ai architectures. Google Scholar logo with link to Google Scholar
Ciupińska, Kinga, Agnieszka Wykowska & Davide De Tommaso. 2024. From labs to living rooms: Evaluating humans’ perception of value-laden decisions made by humanoid robot. Preprint. Google Scholar logo with link to Google Scholar
Cocchella, Francesca, Giulia Pusceddu, Giulia Belgiovine, Michela Bogliolo, Linda Lastrico, Maura Casadio, Francesco Rea & Alessandra Sciutti. 2023. At school with a robot: Italian students’ perception of robotics during an educational program. In 2023 32nd ieee international conference on robot and human interactive communication (ro-man), 1413–1419. IEEE. Google Scholar logo with link to Google Scholar
Foddy, Margaret. 1978. Patterns of gaze in cooperative and competitive negotiation. Human relations 31(11). 925–938. Google Scholar logo with link to Google Scholar
Graham, Jesse, Jonathan Haidt, Sena Koleva, Matt Motyl, Ravi Iyer, Sean P. Wojcik & Peter H. Ditto. 2013. Moral foundations theory: The pragmatic validity of moral pluralism. In Advances in experimental social psychology, vol. 471, 55–130. Elsevier.Google Scholar logo with link to Google Scholar
Graham, Jesse, Brian A. Nosek, Jonathan Haidt, Ravi Iyer, Koleva Spassena & Peter H. Ditto. 2008. Moral foundations questionnaire. Journal of Personality and Social Psychology.Google Scholar logo with link to Google Scholar
Grishchenko, Ivan, Geng Yan, Eduard Gabriel Bazavan, Andrei Zanfir, Nikolai Chinaev, Karthik Raveendran, Matthias Grundmann & Cristian Sminchisescu. 2023. Blendshapes ghum: Realtime monocular facial blendshape prediction. arXiv preprint arXiv:2309.05782.Google Scholar logo with link to Google Scholar
Hancock, Peter A., Deborah R. Billings, Kristin E. Schaefer, Jessie Y. C. Chen, Ewart J De Visser & Raja Parasuraman. 2011. A meta-analysis of factors affecting trust in human-robot interaction. Human factors 53(5). 517–527. Google Scholar logo with link to Google Scholar
Hertz, Nicholas, Tyler Shaw, Ewart J de Visser & Eva Wiese. 2019. Mixing it up: how mixed groups of humans and machines modulate conformity. Journal of Cognitive Engineering and Decision Making 13(4). 242–257. Google Scholar logo with link to Google Scholar
Hou, Yoyo Tsung-Yu, Wen-Ying Lee & Malte Jung. 2023. “should i follow the human, or follow the robot?” — robots in power can have more influence than humans on decision-making. In Proceedings of the 2023 chi conference on human factors in computing systems, 1–13. Google Scholar logo with link to Google Scholar
Leiner, Dominik J. 2024. Sosci survey (version 3.5.01). Computer software. [URL]
Normoyle, Aline, Jeremy B. Badler, Teresa Fan, Norman I. Badler, Vinicius J. Cassol & Soraia R. Musse. 2013. Evaluating perceived trust from procedurally animated gaze. In Proceedings of motion on games, 141–148. Google Scholar logo with link to Google Scholar
Pasquali, Dario, Austin Kothig, Alexander Mois Aroyo, John Edison Munoz Cadorna, Kerstin Dautenhahn, Stefano Bencetti, Rea Francesco & Alessandra Sciutti. 2023. That’s not a good idea: A robot changes your behavior against social engineering. In Proceedings of the 11th international conference on human-agent interaction, 63–71. Google Scholar logo with link to Google Scholar
Podell, Dustin, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Muller, Joe Penna & Robin Rombach. 2023. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952.Google Scholar logo with link to Google Scholar
Pusceddu, G., M. Sangineto, F. Cocchella, M. Bogliolo, G. Belgiovine, L. Lastrico, M. Casadio, F. Rea, C. Gena & S. Sciutti. 2025. Exploring children’s strategies in response to robot’s advice during a group task with icub and nao. In Proceedings of the international conference on social robotics. Google Scholar logo with link to Google Scholar
Qin, Xin, Chen Chen, Kai Chi Yam, Limei Cao, Wanlu Li, Jian Guan, Puchu Zhao, Xiaowei Dong & Yiqiang Lin. 2022. Adults still can’t resist: A social robot can induce normative conformity. Computers in Human Behavior 1271. 107041. Google Scholar logo with link to Google Scholar
Qiu, Liang, Yizhou Zhao, Jinchao Li, Pan Lu, Baolin Peng, Jianfeng Gao & Song-Chun Zhu. 2022. Valuenet: A new dataset for human value driven dialogue system. Proceedings of the AAAI Conference on Artificial Intelligence 36(10). 11183–11191. . [URL]
Rassin, Eric, Peter Muris, Ingmar Franken, Maartje Smit & Maggie Wong. 2007. Measuring general indecisiveness. Journal of Psychopathology and Behavioral Assessment 291. 60–67. Google Scholar logo with link to Google Scholar
Rokeach, Milton. 1973. The nature of human values. Free press.Google Scholar logo with link to Google Scholar
Salomons, Nicole, Michael Van Der Linden, Sarah Strohkorb Sebo & Brian Scassellati. 2018. Humans conform to robots: Disambiguating trust, truth, and conformity. In Proceedings of the 2018 acm/ieee international conference on human-robot interaction, 187–195. Google Scholar logo with link to Google Scholar
Schwartz, Shalom H. 2012. An overview of the schwartz theory of basic values. Online readings in Psychology and Culture 2(1). 11. Google Scholar logo with link to Google Scholar
Sembroski, Catherine E., Marlena R. Fraune & Selma Šabanović. 2017. He said, she said, it said: Effects of robot group membership and human authority on people’s willingness to follow their instructions. In 2017 26th ieee international symposium on robot and human interactive communication (ro-man), 56–61. IEEE. Google Scholar logo with link to Google Scholar
Sharkey, Amanda & Noel Sharkey. 2021. We need to talk about deception in social robotics! Ethics and Information Technology 231. 309–316. Google Scholar logo with link to Google Scholar
Shiomi, Masahiro & Norihiro Hagita. 2016. Do synchronized multiple robots exert peer pressure? In Proceedings of the fourth international conference on human agent interaction, 27–33. Google Scholar logo with link to Google Scholar
Twyman, Matt, Nigel Harvey & Clare Harries. 2008. Trust in motives, trust in competence: Separate factors determining the effectiveness of risk communication. Judgment and Decision Making 3(1). 111–120. Google Scholar logo with link to Google Scholar
Van Eecke, Paul, Lara Verheyen, Tom Willaert & Katrien Beuls. 2023. The candide model: How narratives emerge where observations meet beliefs. In The 61st annual meeting of the association for computational linguistics, 48–57. Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Vollmer, AnnaLisa, Robin Read, Dries Trippas & Tony Belpaeme. 2018. Children conform, adults resist: A robot group induced peer pressure on normative social conformity. Science robotics 3(21). eaat7111. Google Scholar logo with link to Google Scholar
Zhang, Lixuan, Iryna Pentina & Yuhong Fan. 2021. Who do you choose? comparing perceptions of human vs robo-advisor in the context of financial services. Journal of Services Marketing 35(5). 634–646. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue