Cover not available

Article published In: Interaction Studies
Vol. 23:1 (2022) ► pp.116–142

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Interacting with an embodied interface

Effects of embodied agent and voice command on smart TV interface

 | Nanyang Technological University
 | Hallym University
 | Sogang University
Published online: 20 October 2022
DOI logo with link
https://doi.org/10.1075/is.20030.lee
Abstract
Despite their potential for facilitating interaction between a user and computer, an embodied agent and voice command have not been examined enough for their matching effects. The current study proposes that an embodied agent and voice command generate positive evaluative outcomes, particularly when they are accompanied by each other. To test this prediction, we conducted a 2 (visual output: embodied agent vs. geometric figure) × 2 (input modality: voice command vs. remote controller) between-subjects experiment (N = 52), and examined whether visual output and input modality jointly influence participants’ social attribution (i.e., anthropomorphism, animacy, likability, and perceived intelligence), social presence, and satisfaction. Results show that voice command does facilitate users’ social attribution and social presence, but only when an embodied agent was presented. Also, the effects of voice command on social presence and satisfaction were mediated by anthropomorphism and perceived intelligence respectively, but only when the interface displayed an embodied agent. The present study evidences the holistic nature of human-computer interaction, revealing the importance of matches in the input and output interface.
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Effects of embodied agents on a computer interface
    • 2.2Input modality of a smart TV
    • 2.3Contingent effect of visual output and input modality
  • 3.Method
    • 3.1Design and participants
    • 3.2Procedure
    • 3.3Material
    • 3.4Manipulation
    • 3.5Measures
      • Attribution of social characters
      • Social presence
      • Satisfaction
  • 4.Results
    • 4.1Effects on social attribution
    • 4.2Effects on social presence and satisfaction
    • 4.3Conditional indirect effect of input modality
  • 5.Discussion
    • 5.1Summary of findings
    • 5.2Implications
    • 5.3Limitations and directions for further research
  • Disclosure statement
  • References
References (47)
References
Allison, F., Luger, E., & Hofmann, K. (2017). Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 2337–2344. Google Scholar logo with link to Google Scholar
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics, 1(1), 71–81. Google Scholar logo with link to Google Scholar
Baylor, A. L., & Ryu, J. (2003). The effects of image and animation in enhancing pedagogical agent persona. Journal of Educational Computing Research, 281, 373–394. Google Scholar logo with link to Google Scholar
Brave, S., Nass, C., & Hutchinson, K. (2005). Computers that care: Investigating the effects of orientation of emotion exhibited by an embodied computer agent. International Journal of Human-Computer Studies, 621, 161–178. Google Scholar logo with link to Google Scholar
Brennan, S. E., & Hulteen, E. A. (1995). Interaction and feedback in a spoken language system: A theoretical framework. Knowledge-Based Systems, 81, 143–151. Google Scholar logo with link to Google Scholar
Clark, H. H. (1999). How do real people communicate with virtual partners. Proceedings of 1999 AAAI Fall Symposium, Psychological Models of Communication in Collaborative Systems, 43–47. North Falmouth, MA.Google Scholar logo with link to Google Scholar
Cohen, P. R., & Oviatt, S. L. (1995). The role of voice input for human-machine communication. Proceedings of the National Academy of Sciences, 921, 9921–9927. Retrieved from [URL].
Cordasco, G., Esposito, M., Masucci, F., Riviello, M. T., Esposito, A., Chollet, G., … Pelosi, G. (2014). Assessing voice user interfaces: The vAssist system prototype. 5th IEEE Conference on Cognitive Infocommunications, 91–96.
Dehn, D. M., & Van Mulken, S. (2000). The impact of animated interface agents: A review of empirical research. International Journal of Human-Computer Studies, 521, 1–22. Google Scholar logo with link to Google Scholar
Dworkin, M., Chakraborty, A., Lee, S., Monahan, C., Hightow-Weidman, L., Garofalo, R., … Jimenez, A. (2018). A realistic talking human embodied agent mobile phone intervention to promote hiv medication adherence and retention in care in young HIV-positive african american men who have sex with men: Qualitative study. JMIR MHealth and UHealth, 61, e10211. Google Scholar logo with link to Google Scholar
Fogg, B., & Nass, C. (1997). How users reciprocate to computers: An experiment that demonstrates behavior change. Proceedings of the CHI ’97 Extended Abstracts on Human Factors in Computing Systems, 331–332. Google Scholar logo with link to Google Scholar
Hasegawa, D., Cassell, J., & Araki, K. (2010). The role of embodiment and perspective in direction-giving systems. 2010 AAAI Fall Symposium. Retrieved from [URL]
Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY, US: Guilford Press.Google Scholar logo with link to Google Scholar
Hong, W., Hess, T. J., & Hardin, A. (2013). When filling the wait makes it feel longer: A paradigm shift perspective for managing online delay. MIS Quarterly, 371, 383–406. Retrieved from [URL].
Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985). Direct manipulation interfaces. Human–Computer Interaction, 11, 311–338. Google Scholar logo with link to Google Scholar
Jeong, J.-W., & Lee, D.-H. (2014). Inferring search intents from remote control movement patterns: A new content search method for smart TV. IEEE Transactions on Consumer Electronics, 601, 92–98. Google Scholar logo with link to Google Scholar
Kätsyri, J., Förger, K., Mäkäräinen, M., & Takala, T. (2015). A review of empirical evidence on different uncanny valley hypotheses: Support for perceptual mismatch as one road to the valley of eeriness. Frontiers in Psychology, 61, 390. Google Scholar logo with link to Google Scholar
Kim, Yanghee, Thayne, J., & Wei, Q. (2017). An embodied agent helps anxious students in mathematics learning. Educational Technology Research and Development, 651, 219–235. Google Scholar logo with link to Google Scholar
Kim, Youjeong, & Sundar, S. S. (2012). Anthropomorphism of computers: Is it mindful or mindless? Computers in Human Behavior, 281, 241–250. Google Scholar logo with link to Google Scholar
Koda, T., & Maes, P. (1996). Agents with faces: The effect of personification. Proceedings of 5th International Workshop on Robot and Human Communication, 189–194. Google Scholar logo with link to Google Scholar
Lee, B., Isenberg, P., Riche, N. H., & Carpendale, S. (2012). Beyond Mouse and Keyboard: Expanding Design Considerations for Information Visualization Interactions. IEEE Transactions on Visualization and Computer Graphics, 181, 2689–2698. Google Scholar logo with link to Google Scholar
Lee, D., Sah, Y. J., & Lee, S. (2019). Improving usability perception of error-prone ai speakers: Elaborated feedback mitigates negative consequences of errors. International Journal of Human–Computer Interaction, 351, 1645–1652. Google Scholar logo with link to Google Scholar
Lee, K. M. (2004). Presence, explicated. Communication Theory, 14(1), 27–50. Google Scholar logo with link to Google Scholar
Lee, K. M., & Nass, C. (2005). Social-psychological origins of feelings of presence: Creating social presence with machine-generated voices. Media Psychology, 71, 31–45. Google Scholar logo with link to Google Scholar
Lee, K. M., Peng, W., Jin, S.-A., & Yan, C. (2006). Can robots manifest personality?: An empirical test of personality recognition, social responses, and social presence in human–robot interaction. Journal of Communication, 561, 754–772. Google Scholar logo with link to Google Scholar
Mitchell, W. J., Szerszen, K. A., Lu, A. S., Schermerhorn, P. W., Scheutz, M., & MacDorman, K. F. (2011). A mismatch in the human realism of face and voice produces an uncanny valley. I-Perception, 21, 10–12. Google Scholar logo with link to Google Scholar
Mori, M. (1970). The uncanny valley. Energy, 7(4), 33–35.Google Scholar logo with link to Google Scholar
Nass, C., & Gong, L. (2000). Speech interfaces from an evolutionary perspective. Communications of the ACM, 43(9), 36–43. Google Scholar logo with link to Google Scholar
Nass, C., & Moon, Y. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 561, 81–103. Google Scholar logo with link to Google Scholar
Nass, C., Moon, Y., & Carney, P. (1999). Are people polite to computers? Responses to computer-based interviewing systems. Journal of Applied Social Psychology, 291, 1093–1109. Google Scholar logo with link to Google Scholar
Nass, C., Moon, Y., & Green, N. (1997). Are machines gender neutral? Gender-stereotypic responses to computers with voices. Journal of Applied Social Psychology, 271, 864–876. Google Scholar logo with link to Google Scholar
Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 72–78. Google Scholar logo with link to Google Scholar
Pyae, A., & Joelsson, T. N. (2018). Investigating the usability and user experiences of voice user interface: A case of google home smart speaker. Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, 127–131. Google Scholar logo with link to Google Scholar
Rhee, E., Shin, I., Jung, Y., & Lee, H. (2013). Cloud-based 3D gaming system for smart TV. Proceedings of the 2013 International Conference on ICT Convergence (ICTC), 714–715. Google Scholar logo with link to Google Scholar
Rosenberg-Kima, R. B., Baylor, A. L., Plant, E. A., & Doerr, C. E. (2007). The importance of interface agent visual presence: Voice alone is less effective in impacting young women’s attitudes toward engineering. In Y. de Kort, W. IJsselsteijn, C. Midden, B. Eggen, & B. J. Fogg (Eds.), Persuasive Technology (pp. 214–222). Berlin: Springer. Google Scholar logo with link to Google Scholar
Sah, Y. J., & Peng, W. (2015). Effects of visual and linguistic anthropomorphic cues on social perception, self-awareness, and information disclosure in a health website. Computers in Human Behavior, 451, 392–401. Google Scholar logo with link to Google Scholar
Seyama, J., & Nagayama, R. S. (2007). The uncanny valley: Effect of realism on the impression of artificial human faces. Presence: Teleoperators and Virtual Environments, 161, 337–351. Google Scholar logo with link to Google Scholar
Shamekhi, A., Liao, Q. V., Wang, D., Bellamy, R. K. E., & Erickson, T. (2018). Face value? Exploring the effects of embodiment for a group facilitation agent. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1–13. Google Scholar logo with link to Google Scholar
Silvia, P. J., & Duval, T. S. (2001). Objective self-awareness theory: Recent progress and enduring problems. Personality and Social Psychology Review, 51, 230–241. Google Scholar logo with link to Google Scholar
Sproull, L., Subramani, M., Kiesler, S., Walker, J. H., & Waters, K. (1996). When the interface is a face. Human–Computer Interaction, 111, 97–124. Google Scholar logo with link to Google Scholar
Stafford, R. Q., MacDonald, B. A., Jayawardena, C., Wegner, D. M., & Broadbent, E. (2014). Does the robot have a mind? Mind perception and attitudes towards robots predict use of an eldercare robot. International Journal of Social Robotics, 61, 17–32. Google Scholar logo with link to Google Scholar
Walther, J. B. (1996). Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal interaction. Communication Research, 231, 3–43. Google Scholar logo with link to Google Scholar
(2011). Visual cues in computer-mediated communication: Sometimes less is more. In A. Kappas & N. C. Krämer (Eds.), Face-to-Face Communication over the Internet (pp. 17–38). Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Walther, J. B., Slovacek, C. L., & Tidwell, L. C. (2001). Is a picture worth a thousand words? Photographic images in long-term and short-term computer-mediated communication. Communication Research, 28(1), 105–134. Google Scholar logo with link to Google Scholar
Yee, N., Bailenson, J. N., & Rickertsen, K. (2007). A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1–10. Google Scholar logo with link to Google Scholar
Yi, J. S., Kang, Y. a, & Stasko, J. (2007). Toward a Deeper Understanding of the Role of Interaction in Information Visualization. IEEE Transactions on Visualization and Computer Graphics, 131, 1224–1231. Google Scholar logo with link to Google Scholar
Yu, E., Hong, A., & Hwang, J. (2016). A socio-technical analysis of factors affecting the adoption of smart TV in Korea. Computers in Human Behavior, 611, 89–102. Google Scholar logo with link to Google Scholar
Cited by (2)

Cited by two other publications

Mou, Yi, Xiaoyu Ye & Wenbin Ma
2026. Building and Repairing Trust in Chatbots: The Interplay Between Social Role and Performance During Interactions. Behavioral Sciences 16:1 ► pp. 118 ff. DOI logo
Jeon, Yongwoog Andrew, Yuhosua Ryoo, Sangwook Lee & Hye Jin Yoon
2025. How Small Talk and Co-Viewing with AI Companions Boost Advertising Effectiveness and Persuasion: Chat, Watch, and Buy. Journal of Advertising Research ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 17 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue