Cover not available

Article published In: Interaction Studies
Vol. 22:1 (2021) ► pp.110–140

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required
Groups in human-agent interaction

Robots as an interactive-social medium in storytelling to multiple children

 | Interaction Science Labs., ATR | Doshisha University
 | Interaction Science Labs., ATR
 | Interaction Science Labs., ATR | Keio University
 | Interaction Science Labs., ATR | Doshisha University
 | Doshisha University
 | Interaction Science Labs., ATR
Published online: 17 September 2021
DOI logo with link
https://doi.org/10.1075/is.18033.tam
Abstract
This paper investigates the effects of group interaction in a storytelling situation for children using two robots: a reader robot and a listener robot as a side-participant. We developed a storytelling system that consists of a reader robot, a listener robot, a display, a gaze model, a depth sensor, and a human operator who responds and provides easily understandable answers to the children’s questions. We experimentally investigated the effects of using a listener robot and either one or two children during a storytelling situation on the children’s preferences and their speech activities. Our experimental results showed that the children preferred storytelling with the listener robot. Although two children obviously produced more speech than one child, the listener robot discouraged the children’s speech regardless of whether one or two were listening.
Article outline
  • 1.Introduction
  • 2.Related work
    • 2.1Robot as an information-providing medium
    • 2.2Interactive storytelling-robot systems for children
  • 3.System overview
    • 3.1Human-tracking system
    • 3.2Gaze model
    • 3.3Robot
    • 3.4Details of implemented behavior
    • 3.5Operator
  • 4.Experiment
    • 4.1Hypotheses and predictions about children’s speech activities and the number of robots
    • 4.2Participants
    • 4.3Conditions
    • 4.4Environment
    • 4.5Procedure
    • 4.6Measurements
  • 5.Results
    • 5.1Observations
    • 5.2Verification of prediction: Speech activities based on the number of children and robots
    • 5.3Additional analysis of subjective impressions
    • 5.4Children’s behaviors analysis
      • 5.4.1Behavior category of children
      • 5.4.2Conversation topics of children
      • 5.4.3Touch behaviors between robots and children
  • 6.Discussion
    • 6.1Implications of experiment results
    • 6.2Design implications for application perspective
    • 6.3Toward (semi-) autonomous storytelling
    • 6.4Limitations
  • 7.Conclusion
  • Conflict of interest
  • Acknowledgements
  • References
References (59)
References
Alborzi, H., A. Druin, J. Montemayor, M. Platner, J. Porteous, L. Sherman, A. Boltman, G. Taxén, J. Best, and J. Hammer (2000). “Designing StoryRooms: interactive storytelling spaces for children,” in Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, pp. 95–104.
Arai, H., M. Kimoto, T. Iio, K. Shimohara, R. Matsumura, and M. Shiomi (2019). “How Can Robot’s Gaze Ratio and Body Direction Show an Awareness of Priority to the People with whom it is Interacting?,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 3798–3805. Google Scholar logo with link to Google Scholar
Bers, M. U., and J. Cassell (1998). “Interactive storytelling systems for children: Using technology to explore language and identity,” Journal of Interactive Learning Research, vol. 9, no. 2, pp. 183.Google Scholar logo with link to Google Scholar
Clark, H. H. (1996). Using Language: Cambridge University Press. Google Scholar logo with link to Google Scholar
Cohen, J. (1960). “A coefficient of agreement for nominal scales,” Educational and psychological measurement, vol. 20, no. 1, pp. 37–46. Google Scholar logo with link to Google Scholar
Collins, F. . (1999). “The use of traditional storytelling in education to the learning of literacy skills,” Early Child Development and Care, vol. 152, no. 1, pp. 77–108. Google Scholar logo with link to Google Scholar
Dahlbäck, N., A. Jönsson, and L. Ahrenberg (1993). “Wizard of Oz studies: why and how,” in Proceedings of the 1st international conference on Intelligent user interfaces, Orlando, Florida, USA, pp. 193–200.
Fails, J. A., A. Druin, and M. L. Guha (2014). “Interactive storytelling: Interacting with people, environment, and technology,” International Journal of Arts and Technology, vol. 7, no. 1, pp. 112–124. Google Scholar logo with link to Google Scholar
Fourati, N., A. Richard, S. Caillou, N. Sabouret, J.-C. Martin, E. Chanoni, and C. Clavel (2016). “Facial Expressions of Appraisals Displayed by a Virtual Storyteller for Children,” Intelligent Virtual Agents: 16th International Conference, IVA 2016, Los Angeles, CA, USA, September 20–23, 2016, Proceedings, D. Traum, W. Swartout, P. Khooshabeh et al., eds., pp. 234–244, Cham: Springer International Publishing. Google Scholar logo with link to Google Scholar
Fridin, M. (2014). “Storytelling by a kindergarten social assistive robot: A tool for constructive learning in preschool education,” Computers & education, vol. 701, pp. 53–64. Google Scholar logo with link to Google Scholar
Gharbi, M., P. V. Paubel, A. Clodic, O. Carreras, R. Alami, and J. M. Cellier (2015). “Toward a better understanding of the communication cues involved in a human-robot object transfer,” in Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE International Symposium on, pp. 319–324.
Glas, D. F., T. Kanda, H. Ishiguro, and N. Hagita (2011). “Teleoperation of multiple social robots,” IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 42, no. 3, pp. 530–544. Google Scholar logo with link to Google Scholar
Goffman, E. (1981). Forms of talk: University of Pennsylvania Press.Google Scholar logo with link to Google Scholar
Gordon, G., S. Spaulding, J. Kory Westlund, J. J. Lee, L. Plummer, M. Martinez, M. Das, and C. Breazeal (2016). “Affective Personalization of a Social Robot Tutor for Children’s Second Language Skills,” in Thirtieth AAAI Conference on Artificial Intelligence.Google Scholar logo with link to Google Scholar
Gross, H.-M., H. Boehme, C. Schroeter, S. Müller, A. König, E. Einhorn, C. Martin, M. Merten, and A. Bley (2009). “TOOMAS: interactive shopping guide robots in everyday use-final implementation and experiences from long-term field trials,” in Intelligent Robots and Systems, 2009. IROS 2009. The IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2005–2012. Google Scholar logo with link to Google Scholar
Hato, Y., S. Satake, T. Kanda, M. Imai, and N. Hagita (2010). “Pointing to space: modeling of deictic interaction referring to regions,” in Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on, pp. 301–308.
Huisman, G. (2017). “Social Touch Technology: A Survey of Haptic Technology for Social Touch,” IEEE Transactions on Haptics, vol. 10, no. 3, pp. 391–408. Google Scholar logo with link to Google Scholar
Iio, T., Y. Yoshikawa, and H. Ishiguro (2016). “Pre-scheduled Turn-Taking between Robots to Make Conversation Coherent,” in Proceedings of the Fourth International Conference on Human Agent Interaction, Biopolis, Singapore, pp. 19–25.
(2017). “Retaining Human-Robots Conversation: Comparing Single Robot to Multiple Robots in a Real Event,” Journal of Advanced Computational Intelligence Intelligent Informatics, vol. 21, no. 4, pp. 675–685. Google Scholar logo with link to Google Scholar
Isbell, R., J. Sobol, L. Lindauer, and A. Lowrance. (2004). “The effects of storytelling and story reading on the oral language complexity and story comprehension of young children,” Early childhood education journal, vol. 32, no. 3, pp. 157–163. Google Scholar logo with link to Google Scholar
Kanda, T., H. Ishiguro, T. Ono, M. Imai, and K. Mase (2002). “Multi-robot cooperation for human-robot communication,” in Robot and Human Interactive Communication, 2002. Proceedings. 11th IEEE International Workshop on, pp. 271–276.
Karatas, N., S. Yoshikawa, P. R. S. De Silva, and M. Okada (2015). “Namida: Multiparty conversation based driving agents in futuristic vehicle,” in International Conference on Human-Computer Interaction, pp. 198–207.
Kennedy, J., V. Lemaignan, C. Montassier, P. Lavalade, B. Irfan, F. Papadopoulos, E. Senft, and T. Belpaeme (2017, “Child Speech Recognition in Human-Robot Interaction: Evaluations and Recommendations,” in Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, pp. 82–90.
Komatsubara, T., M. Shiomi, T. Kanda, and H. Ishiguro (2017). “Can Using Pointing Gestures Encourage Children to Ask Questions?,” International Journal of Social Robotics, vol. 10, no. 4, pp. 387–399. Google Scholar logo with link to Google Scholar
Komatsubara, T., M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita (2014). “Can a social robot help children’s understanding of science in classrooms?,” in Proceedings of the second international conference on Human-agent interaction, Tsukuba, Japan, pp. 83–90.
Kory, J., and C. Breazeal (2014). “Storytelling with robots: Learning companions for preschool children’s language development,” in Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on, pp. 643–648.
Lee, M. K., S. Kiesler, and J. Forlizzi (2010). “Receptionist or information kiosk: how do people talk with a robot?,” in Proceedings of the 2010 ACM conference on Computer supported cooperative work, Savannah, Georgia, USA, pp. 31–40.
Leite, I., M. McCoy, M. Lohani, D. Ullman, N. Salomons, C. Stokes, S. Rivers, and B. Scassellati (2015). “Emotional Storytelling in the Classroom: Individual versus Group Interaction between Children and Robots,” in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, Portland, Oregon, USA, pp. 75–82.
(2017). “Narratives with robots: The impact of interaction context and individual Differences on story recall and emotional Understanding,” Frontiers in Robotics and AI, vol. 41, pp. 29. Google Scholar logo with link to Google Scholar
Lombard, M., J. Snyder-Duch, and C. C. Bracken (2002). “Content analysis in mass communication: Assessment and reporting of intercoder reliability,” Human Communication Research, vol. 28, no. 4, pp. 587–604. Google Scholar logo with link to Google Scholar
Matsuyama, Y., I. Akiba, A. Saito, and T. Kobayashi (2013). “A Four-Participant Group Facilitation Framework for Conversational Robots,” in Proceedings of the SIGDIAL 2013 Conference, pp. 284–293.
Matsumura, R., M. Shiomi, T. Miyashita, H. Ishiguro, and N. Hagita (2014). “Who is Interacting With me?; Identification of an Interacting Person Through Playful Interaction With a Small Robot,” IEEE Transactions on Human-Machine Systems, vol. 44, no. 2, pp. 169–179. Google Scholar logo with link to Google Scholar
(2015). “What kind of floor am I standing on? Floor surface identification by a small humanoid robot through full-body motions,” Advanced Robotics. Google Scholar logo with link to Google Scholar
Mutlu, B., J. Forlizzi, and J. Hodgins (2006). “A storytelling robot: Modeling and evaluation of human-like gaze behavior,” in The 6th IEEE-RAS international conference on Humanoid robots, pp. 518–523.
Mutlu, B., T. Kanda, J. Forlizzi, J. Hodgins, and H. Ishiguro (2012). “Conversational gaze mechanisms for human-like robots,” ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 1, no. 2, pp. 12.Google Scholar logo with link to Google Scholar
Mutlu, B., T. Shiwa, T. Kanda, H. Ishiguro, and N. Hagita (2009). “Footing in human-robot conversations: how robots might shape participant roles using gaze cues,” in Proceedings of the 4th ACM/IEEE international conference on Human robot interaction, pp. 61–68.
Nourbakhsh, I. R., C. Kunz, and T. Willeke (2003). “The mobot museum robot installations: A five year experiment,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003.(IROS 2003). pp. 3636–3641.
Park, H. W., M. Gelsomini, J. J. Lee, and C. Breazeal (2017). “Telling stories to robots: The effect of backchanneling on a child’s storytelling,” in Proceedings of the 2017 ACM/IEEE international conference on human-robot interaction, pp. 100–108.
Powers, A., S. Kiesler, S. Fussell, and C. Torrey (2007). “Comparing a computer agent with a humanoid robot,” in Human-Robot Interaction (HRI), 2007 2nd ACM/IEEE International Conference on, pp. 145–152.
Roussou, M. (2004). “Learning by doing and learning through play: an exploration of interactivity in virtual environments for children,” Computers in Entertainment (CIE), vol. 2, no. 1, pp. 10–10. Google Scholar logo with link to Google Scholar
Sakamoto, D., K. Hayashi, T. Kanda, M. Shiomi, S. Koizumi, H. Ishiguro, T. Ogasawara, and N. Hagita (2009). “Humanoid Robots as a Broadcasting Communication Medium in Open Public Spaces,” International Journal of Social Robotics, vol. 1, no. 2, pp. 157–169. Google Scholar logo with link to Google Scholar
Satake, S., K. Hayashi, K. Nakatani, and T. Kanda (2015). “Field trial of an information-providing robot in a shopping mall,” in The IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015) pp. 1832–1839.
Satake, S., T. Kanda, D. F. Glas, M. Imai, H. Ishiguro, and N. Hagita (2013). “A robot that approaches pedestrians,” IEEE Transactions on Robotics, vol. 29, no. 2, pp. 508–524. Google Scholar logo with link to Google Scholar
Shi, C., M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita (2015). “Measuring Communication Participation to Initiate Conversation in Human–Robot Interaction,” International Journal of Social Robotics, vol. 7, no. 5, pp. 889–910. Google Scholar logo with link to Google Scholar
Shi, C., M. Shiomi, C. Smith, T. Kanda, and H. Ishiguro (2013). “A Model of Distributional Handing Interaction for a Mobile Robot,” in Robotics: Science and Systems.Google Scholar logo with link to Google Scholar
Shinozawa, K., F. Naya, J. Yamato, and K. Kogure (2005). “Differences in effect of robot and screen agent recommendations on human decision-making,” International Journal of Human-Computer Studies, vol. 62, no. 2, pp. 267–279. Google Scholar logo with link to Google Scholar
Shiomi, M., T. Kanda, I. Howley, K. Hayashi, and N. Hagita (2015). “Can a Social Robot Stimulate Science Curiosity in Classrooms?,” International Journal of Social Robotics, vol. 7, no. 5, pp. 641–652. Google Scholar logo with link to Google Scholar
Shiomi, M., T. Kanda, H. Ishiguro, and N. Hagita (2007). “Interactive Humanoid Robots for a Science Museum,” IEEE Intelligent Systems, no. 2, pp. 25–32. Google Scholar logo with link to Google Scholar
Shiomi, M., D. Sakamoto, T. Kanda, C. T. Ishi, H. Ishiguro, and N. Hagita (2010). “Field Trial of a Networked Robot at a Train Station,” International Journal of Social Robotics, vol. 3, no. 1, pp. 27–40. Google Scholar logo with link to Google Scholar
Shiomi, M., K. Shinozawa, Y. Nakagawa, T. Miyashita, T. Sakamoto, T. Terakubo, H. Ishiguro, and N. Hagita (2013). “Recommendation Effects of a Social Robot for Advertisement-Use Context in a Shopping Mall,” International Journal of Social Robotics, vol. 5, no. 2, pp. 251–262. Google Scholar logo with link to Google Scholar
Short, E. S., K. Swift-Spong, H. Shim, K. M. Wisniewski, D. K. Zak, S. Wu, E. Zelinski, and M. J. Matarić (2017). “Understanding social interactions with socially assistive robotics in intergenerational family groups,” in Robot and Human Interactive Communication (RO-MAN), 2017 26th IEEE International Symposium on, pp. 236–241.
Tamura, Y., M. Kimoto, M. Shiomi, T. Iio, K. Shimohara, and N. Hagita. (2017). “Effects of a Listener Robot with Children in Storytelling,” in Proceedings of the 5th International Conference on Human Agent Interaction, Bielefeld, Germany, pp. 35–43.
Toh, L. P. E., A. Causo, P. W. Tzuo, I.-M. Chen, and S. H. Yeo (2016). “A Review on the Use of Robots in Education and Young Children,” Educational Technology & Society, vol. 19, no. 2, pp. 148–163.Google Scholar logo with link to Google Scholar
Van Gils, F. . (2005). “Potential applications of digital storytelling in education,” in 3rd Twente student conference on IT.
Vázquez, M., A. Steinfeld, S. E. Hudson, and J. Forlizzi (2014). “Spatial and other social engagement cues in a child-robot interaction: Effects of a sidekick,” in Proceedings of the 2014 ACM/IEEE international conference on Human-robot interaction, pp. 391–398.
Wada, K., D. F. Glas, M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita (2015). “Capturing Expertise: Developing Interaction Content for a Robot Through Teleoperation by Domain Experts,” International Journal of Social Robotics, vol. 7, no. 5, pp. 653–672. Google Scholar logo with link to Google Scholar
Yamaoka, F., T. Kanda, H. Ishiguro, and N. Hagita (2010). “A model of proximity control for information-presenting robots,” IEEE Transactions on Robotics, vol. 26, no. 1, pp. 187–195. Google Scholar logo with link to Google Scholar
Yamazaki, A., K. Yamazaki, Y. Kuno, M. Burdelski, M. Kawashima, and H. Kuzuoka (2008). “Precision timing in human-robot interaction: coordination of head movement and utterance,” in Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, Florence, Italy, pp. 131–140.
Yohanan, S., and K. E. MacLean (2012). “The role of affective touch in human-robot interaction: Human intent and expectations in touching the haptic creature,” International Journal of Social Robotics, vol. 4, no. 2, pp. 163–180. Google Scholar logo with link to Google Scholar
Cited by (17)

Cited by 17 other publications

Korpan, Raj, Khadeja Ahmar, Raitah A. Jinnat & Jackie Yee
2026. Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction, ► pp. 644 ff. DOI logo
Steinhaeusser, Sophia C., Sophia Maier & Birgit Lugrin
2026. Multimodal robotic storytelling integrating sound effects and background music. Computers in Human Behavior: Artificial Humans 7 ► pp. 100248 ff. DOI logo
Wróbel, Alicja, Paulina Zguda, Karolina Źróbek, Marie-Monique Schaper, Paweł Gajewski & Bipin Indurkhya
2025. ‘I Prefer Robot Cats!’ Reflections on Robot Animal-Like Morphology from an in-the-Wild Child-Robot Interaction Workshop. In Intelligent Human Computer Interaction [Lecture Notes in Computer Science, 15557], ► pp. 159 ff. DOI logo
Hatano, Yota, Jun Baba, Junya Nakanishi, Yuichiro Yoshikawa & Hiroshi Ishiguro
2024. Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, ► pp. 517 ff. DOI logo
Kimura, Yuki, Emi Anzai, Naoki Saiwaki & Masahiro Shiomi
2024. 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), ► pp. 485 ff. DOI logo
Kimura, Yuki, Emi Anzai, Naoki Saiwaki & Masahiro Shiomi
2025.  Two is better than one: investigating how explanation styles by multiple robots influence the feeling of kawaii toward objects . Advanced Robotics 39:19-20 ► pp. 1274 ff. DOI logo
Matsusaka, Ryuto, Masahiro Shiomi & Tetsuya Takiguchi
2024. 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), ► pp. 1029 ff. DOI logo
Ng, Spencer, Ting-Han Lin, You Li & Sarah Sebo
2024. Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, ► pp. 522 ff. DOI logo
Shiomi, Masahiro
2024. A systematic survey of multiple social robots as a passive- and interactive-social medium. Advanced Robotics 38:7 ► pp. 440 ff. DOI logo
Shiomi, Masahiro, Taichi Hirayama, Mitsuhiko Kimoto, Takamasa Iio & Katsunori Shimohara
2024. 2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN), ► pp. 351 ff. DOI logo
Shiomi, Masahiro, Taichi Hirayama, Mitsuhiko Kimoto, Takamasa Iio & Katsunori Shimohara
2025. Having too many robots for apologies is not beneficial: number and cultural effects of apologizing robots between Japan and the US. Advanced Robotics 39:24 ► pp. 1639 ff. DOI logo
Higashino, Kana, Mitsuhiko Kimoto, Takamasa Iio, Katsunori Shimohara & Masahiro Shiomi
2023. Is Politeness Better than Impoliteness? Comparisons of Robot's Encouragement Effects Toward Performance, Moods, and Propagation. International Journal of Social Robotics 15:5 ► pp. 717 ff. DOI logo
Okada, Yuka, Mitsuhiko Kimoto, Takamasa Iio, Katsunori Shimohara, Masahiro Shiomi & The Anh Han
2023. Two is better than one: Apologies from two robots are preferred. PLOS ONE 18:2 ► pp. e0281604 ff. DOI logo
Shiomi, Masahiro, Rina Hayashi, Hiroshi Nittono & Wojciech Trzebiński
2023. Is two cuter than one? number and relationship effects on the feeling of kawaii toward social robots. PLOS ONE 18:10 ► pp. e0290433 ff. DOI logo
Hirayama, Taichi, Yuka Okada, Mitsuhiko Kimoto, Takamasa Iio, Katsunori Shimohara & Masahiro Shiomi
2022. Proceedings of the 10th International Conference on Human-Agent Interaction, ► pp. 212 ff. DOI logo
Hirayama, Taichi, Yuka Okada, Mitsuhiko Kimoto, Takamasa Iio, Katsunori Shimohara & Masahiro Shiomi
2024. Speed effects in touching behaviours: impact on perceived relationships in robot-robot interactions. Advanced Robotics 38:7 ► pp. 492 ff. DOI logo
Šabanović, Selma, Malte Jung, Ana Paiva & Friederike Eyssel
2021. Special section editorial. Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems 22:1 ► pp. 81 ff. DOI logo

This list is based on CrossRef data as of 17 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue