Cover not available

Article published In: Interaction Studies
Vol. 20:2 (2019) ► pp.205–233

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Socio-emotional development in high functioning children with Autism Spectrum Disorders using a humanoid robot

 | University of Minho
 | University of Minho
 | University of Minho
 | University of Minho
 | University of Hertfordshire
 | University of Minho
Published online: 7 October 2019
DOI logo with link
https://doi.org/10.1075/is.15003.cos
Abstract
The use of robots had already been proven to encourage the promotion of social interaction and skills lacking in children with Autism Spectrum Disorders (ASD), who typically have difficulties in recognizing facial expressions and emotions. The main goal of this research is to study the influence of a humanoid robot to develop socio-emotional skills in children with ASD. The children’s performance in game scenarios aiming to develop facial expressions recognition skills is presented. Along the sessions, children who performed the game scenarios with the robot and the experimenter had a significantly better performance than the children who performed the game scenarios without the robot. The main conclusions of this research support that a humanoid robot is a useful tool to develop socio-emotional skills in the intervention of children with ASD, due to the engagement and positive learning outcome observed.
Article outline
  • 1.Introduction
  • 2.Background
    • 2.1Facial expressions displayed by robots
    • 2.2Emotion recognition difficulty
    • 2.3Research questions
  • 3.Methods
    • 3.1Ethics statement
    • 3.2Participants
    • 3.3Procedures
      • Performance task
      • Practice phase
    • 3.4Focus groups
    • 3.5The robot
    • 3.6Robot’s input and processing
    • 3.7Experimental setup
    • 3.8Evaluation tools
  • 4.Results
  • 4.1Comparison of the pre- and post-test data between G1, G2, and G3
  • 5.Discussion of the results
    • 5.1Summary of hypotheses and implications
  • 6.Conclusions and future work
  • Acknowledgements
  • References
References (41)
References
Association, A. (2013). Diagnostic and Statistical Manual of Mental Disorders, 5th Edition: DSM 5. book-pointUS. Google Scholar logo with link to Google Scholar
Bakeman, R. and Gottman, J. (1997). Observing interaction: An introduction to sequential analysis. Cambridge Univ Pr.. Google Scholar logo with link to Google Scholar
Baron-Cohen, S. (1991). The development of a theory of mind in autism: deviance and delay? Psychiatric Clinics of North America. Google Scholar logo with link to Google Scholar
Boucenna, S., Narzisi, A., Tilmont, E., Muratori, F., Pioggia, G., Cohen, D., and Chetouani, M. (2014). Interactive technologies for autistic children: A review. Cognitive Computation, pages 1–19.Google Scholar logo with link to Google Scholar
Cabibihan, J.-J., Javed, H., Jr., M. H. A., and Aljunied, S. M. (2013). Why robots? a survey on the roles and benefits of social robots in the therapy of children with autism. CoRR, abs/1311.0352.Google Scholar logo with link to Google Scholar
Celani, G., Battacchi, M., and Arcidiacono, L. (1999). The understanding of the emotional meaning of facial expressions in people with autism. Journal of Autism and Developmental Disorders, 29(1):57–66. Google Scholar logo with link to Google Scholar
Charman, T. and Stone, W. L. (2006). Social and communication development in autism spectrum disorders: Early identification, diagnosis, and intervention. Guilford Press.Google Scholar logo with link to Google Scholar
Clark, T. F., Winkielman, P., and Mcintosh, D. N. (2008). Autism and the extraction of emotion from briefly presented facial expressions: Stumbling at the first step of empathy. Emotion, 8(6):803. Google Scholar logo with link to Google Scholar
Costa, S., Soares, F., Pereira, A. P., Santos, C., and Hiolle, A. (2014a). Building a game scenario to encourage children with autism to recognize and label emotions using a humanoid robot. in Robot and Human Interactive Communication, 2014 RO-MAN: The 23rd IEEE International Symposium on, pages 820–825. IEEE.Google Scholar logo with link to Google Scholar
Costa, S., Soares, F., and Santos, C. (2013). Facial expressions and gestures to convey emotions with a hu-manoid robot. In Herrmann, G., Pearson, M., Lenz, A., Bremner, P., Spiers, A., and Leonards, U., editors, Social Robotics, volume 8239 of Lecture Notes in Computer Science, pages 542–551. Springer International Publishing.Google Scholar logo with link to Google Scholar
Costa, S., Soares, F. O., Pereira, A. P., Santos, C., and Hiolle, A. (2014b). A pilot study using imitation and storytelling scenarios as activities for labelling emotions by children with autism using a humanoid robot. In Proceeding of The 4th International Conference on Development and Learning and on Epigenetic Robotics (IEEE ICDL-EPIROB 2014). Google Scholar logo with link to Google Scholar
Dautenhahn, K. (1999). Robots as social actors: Aurora and the case of autism. In Proc. CT99, The Third International Cognitive Technology Conference, August, San Francisco, volume 3591, page 374.Google Scholar logo with link to Google Scholar
Diehl, J. J., Schmitt, L. M., Villano, M., and Crowell, C. R. (2012). The clinical use of robots for individuals with autism spectrum disorders: A critical review. Research in autism spectrum disorders, 6(1):249–262. Google Scholar logo with link to Google Scholar
Ekman, P. and Rosenberg, E. L. (1998). What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA.Google Scholar logo with link to Google Scholar
Filipe, C. (2012). Autismo: conceitos, mitos e preconceitos.Google Scholar logo with link to Google Scholar
Giullian, N., Ricks, D., Atherton, A., Colton, M., Goodrich, M., and Brinton, B. (2010). Detailed requirements for robots in autism therapy. In Systems Man and Cybernetics (SMC), 2010 IEEE International Conference on, pages 2595–2602. IEEE. Google Scholar logo with link to Google Scholar
Hanson, D., Baurmann, S., Riccio, T., Margolin, R., Dockins, T., Tavares, M., and Carpenter, K. (2009). Zeno: a cognitive character. In AI Magazine, and special Proc. of AAAI National Conference, Chicago.Google Scholar logo with link to Google Scholar
Hobson, R. P. (1986). The autistic child’s appraisal of expressions of emotion: A further study. Journal of Child Psychology and Psychiatry, 27(5):671–680. Google Scholar logo with link to Google Scholar
Huskens, B., Verschuur, R., Gillesen, J., Didden, R., and Barakova, E. (2013). Promoting question-asking in school-aged children with autism spectrum disorders: Effectiveness of a robot intervention compared to a human-trainer intervention. Developmental neurorehabilitation, 16(5):345–356. Google Scholar logo with link to Google Scholar
Johnson, C. P., Myers, S. M., et al. (2007). Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120(5):1183–1215. Google Scholar logo with link to Google Scholar
Kanade, T., Cohn, J. F., and Tian, Y. (2000). Comprehensive database for facial expression analysis. In Automatic Face and Gesture Recognition, 2000. Proceedings. Fourth IEEE International Conference on, pages 46–53. IEEE.Google Scholar logo with link to Google Scholar
Lee, J., Takehashi, H., Nagai, C., Obinata, G., and Stefanov, D. (2012). Which robot features can stimulate better responses from children with autism in robotassisted therapy? Int. J. Advanced Robotic Systems, 9(72).Google Scholar logo with link to Google Scholar
Lee, K. M., Jung, Y., Kim, J., and Kim, S. R. (2006). Are physically embodied social agents better than disembodied social agents?: The effects of physical embodiment, tactile interaction, and people’s loneliness in human-robot interaction. International Journal of Human-Computer Studies, 64(10):962–973. Google Scholar logo with link to Google Scholar
Lima, C. B. D. (2012). Perturbações do espectro do autismo: manual prático de intervençao. Lidel.Google Scholar logo with link to Google Scholar
Loveland, K. A., Tunali-Kotoski, B., Chen, Y. R., Ortegon, J., Pearson, D. A., Brelsford, K. A., and Gibbs, M. C. (1997). Emotion recognition in autism: Verbal and nonverbal information. Development and Psychopathology, null1:579–593. Google Scholar logo with link to Google Scholar
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on, pages 94–101. IEEE.Google Scholar logo with link to Google Scholar
Mazzei, D., Lazzeri, N., Billeci, L., Igliozzi, R., Mancini, A., Ahluwalia, A., Muratori, F., and De Rossi, D. (2011). Development and evaluation of a social robot platform for therapy in autism. In Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, pages 4515–4518. IEEE. Google Scholar logo with link to Google Scholar
Mazzei, D., Lazzeri, N., Hanson, D., and De Rossi, D. (2012). Hefes: an hybrid engine for facial expressions synthesis to control human-like androids and avatars. In BIOROB 2012 proceedings.Google Scholar logo with link to Google Scholar
Mesibov, G. B. and Howley, M. (2003). Accessing the curriculum for pupils with autistic spectrum disorders: Using the TEACCH programme to help inclusion. David Fulton Publishers.Google Scholar logo with link to Google Scholar
Noldus, L. (1991). The observer: A software system for collection and analysis of observational data. Behavior Research Methods, Instruments, & Computers, 23(3):415–429. Google Scholar logo with link to Google Scholar
Palmen, A., Didden, R., and Lang, R. (2012). A systematic review of behavioral intervention research on adaptive skill building in high-functioning young adults with autism spectrum disorder. Research in Autism Spectrum Disorders, 6(2):602–617. Google Scholar logo with link to Google Scholar
Pease, B. and Pease, A. (2008). The definitive book of body language. Random House LLC.Google Scholar logo with link to Google Scholar
Ramdoss, S., Lang, R., Mulloy, A., Franco, J., OReilly, M., Didden, R., and Lancioni, G. (2011). Use of computer-based interventions to teach communication skills to children with autism spectrum disorders: A systematic review. Journal of Behavioral Education, 20(1):55–76. Google Scholar logo with link to Google Scholar
Ramdoss, S., Machalicek, W., Rispoli, M., Mulloy, A., Lang, R., and OReilly, M. (2012). Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: A systematic review. Developmental neurorehabilitation, 15(2):119–135. Google Scholar logo with link to Google Scholar
Ricks, D. J. and Colton, M. B. (2010). Trends and considerations in robot-assisted autism therapy. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 4354–4359. IEEE. Google Scholar logo with link to Google Scholar
Russell, J. A. (1980). A circumplex model of affect. Journal of personality and social psychology, 39(6):1161. Google Scholar logo with link to Google Scholar
Saldien, J., Goris, K., Vanderborght, B., Vanderfaeillie, J., and Lefeber, D. (2010). Expressing emotions with the social robot probo. International Journal of Social Robotics, 2(4):377–389. Google Scholar logo with link to Google Scholar
Scassellati, B., Admoni, H., and Mataric, M. (2012). Robots for use in autism research. Annual Review of Biomedical Engineering, 141:275–294. Google Scholar logo with link to Google Scholar
Werry, I., Dautenhahn, K., Ogden, B., and Harwin, W. (2001). Can social interaction skills be taught by a social agent? the role of a robotic mediator in autism therapy. In Cognitive technology: instruments of mind, pages 57–74. Springer. Google Scholar logo with link to Google Scholar
Zwaigenbaum, L., Bryson, S., Rogers, T., Roberts, W., Brian, J., and Szatmari, P. (2005). Behavioral manifestations of autism in the first year of life. International Journal of Developmental Neuroscience, 23(2):143–152. Google Scholar logo with link to Google Scholar
Cited by (17)

Cited by 17 other publications

De la Hoz, Alejandro, Ester Ceballos & Javier Cubero
2026. Therapeutic Effect of Educational Robotics on Neurocognitive and Social Skills in Two Case Studies of Children with Neurodevelopmental Difficulties. Disabilities 6:1 ► pp. 13 ff. DOI logo
Saiko, Sabrina & Catherine Burns
2026. Bridging Research and Practice: Barriers to Social Robotics in Autism Therapy Stakeholders’ Point of View. In HCI International 2025 – Late Breaking Papers [Lecture Notes in Computer Science, 16331], ► pp. 393 ff. DOI logo
Cersosimo, Rita
2025. “To Teach is to Learn Twice”: Using Social Robots as Tutees in Inclusive Education. In Social Robots in Education [Studies in Computational Intelligence, 1194], ► pp. 221 ff. DOI logo
Holeva, Vasiliki, V. A. Nikopoulou, C. Lytridis, C. Bazinas, P. Kechayas, G. Sidiropoulos, M. Papadopoulou, M. D. Kerasidou, C. Karatsioras, N. Geronikola, G. A. Papakostas, V. G. Kaburlasos & A. Evangeliou
2024. Effectiveness of a Robot-Assisted Psychological Intervention for Children with Autism Spectrum Disorder. Journal of Autism and Developmental Disorders 54:2 ► pp. 577 ff. DOI logo
Hou, Shumeng, Pengpeng Cai, Lingling Yu, He Cui, Jiajia Hu & Zhen Wei
2024. Improving the social communication skills of preschoolers with autism spectrum disorder: robot-based intervention on Picture Exchange Communication System use. International Journal of Developmental Disabilities ► pp. 1 ff. DOI logo
Kewalramani, Sarika, Kelly-Ann Allen, Erin Leif & Andrea Ng
2024. A Scoping Review of the Use of Robotics Technologies for Supporting Social-Emotional Learning in Children with Autism. Journal of Autism and Developmental Disorders 54:12 ► pp. 4481 ff. DOI logo
Silva, Vinícius, Ana Paula Pereira, Filomena Soares, Celina P. Leão, Andrea Jurdi, João Sena Esteves & Juliana Hertzberg
2024. Social Stories for Promoting Social Communication with Children with Autism Spectrum Disorder Using a Humanoid Robot: Step-by-Step Study. Technology, Knowledge and Learning 29:2 ► pp. 735 ff. DOI logo
Alghamdi, Maha, Noura Alhakbani & Abeer Al-Nafjan
2023. Assessing the Potential of Robotics Technology for Enhancing Educational for Children with Autism Spectrum Disorder. Behavioral Sciences 13:7 ► pp. 598 ff. DOI logo
Neerincx, Anouk, Yanzhe Li, Kelvin van de Sande, Frank Broz, Mark Neerincx & Maartje de Graaf
2023. 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), ► pp. 1202 ff. DOI logo
Soleiman, Pegah, Hadi Moradi, Bijan Mehralizadeh, Hamed Ameri, Rosa I. Arriaga, Hamid Reza Pouretemad, Negin Baghbanzadeh & Leila Kashani Vahid
2023. Fully robotic social environment for teaching and practicing affective interaction: Case of teaching emotion recognition skills to children with autism spectrum disorder, a pilot study. Frontiers in Robotics and AI 10 DOI logo
Abdelmohsen, Maha & Yasmine Arafa
2021. Virtual Social Robot Enhances the Social Skills of Children with HFA. In Social Robotics [Lecture Notes in Computer Science, 13086], ► pp. 497 ff. DOI logo
Alimardani, Maryam, Lisa Neve & Anouk Verkaart
2021. Storytelling Robots for Training of Emotion Recognition in Children with Autism; Opinions from Experts. In HCI International 2021 - Late Breaking Posters [Communications in Computer and Information Science, 1498], ► pp. 223 ff. DOI logo
Chen, Xieling, Di Zou, Lucas Kohnke, Haoran Xie, Gary Cheng & Mohammed Saqr
2021. Affective states in digital game-based learning: Thematic evolution and social network analysis. PLOS ONE 16:7 ► pp. e0255184 ff. DOI logo
Lecciso, Flavia, Annalisa Levante, Rosa Angela Fabio, Tindara Caprì, Marco Leo, Pierluigi Carcagnì, Cosimo Distante, Pier Luigi Mazzeo, Paolo Spagnolo & Serena Petrocchi
2021. Emotional Expression in Children With ASD: A Pre-Study on a Two-Group Pre-Post-Test Design Comparing Robot-Based and Computer-Based Training. Frontiers in Psychology 12 DOI logo
Silva, Vinícius, Filomena Soares, João Sena Esteves, Cristina P. Santos & Ana Paula Pereira
2021. Fostering Emotion Recognition in Children with Autism Spectrum Disorder. Multimodal Technologies and Interaction 5:10 ► pp. 57 ff. DOI logo
Damianidou, Despoina, Ami Eidels & Michael Arthur-Kelly
2020. The Use of Robots in Social Communications and Interactions for Individuals with ASD: a Systematic Review. Advances in Neurodevelopmental Disorders 4:4 ► pp. 357 ff. DOI logo
Silva, Vinicius, Filomena Soares, João Sena Esteves & Ana Paula Pereira
2018. Building a Hybrid Approach for a Game Scenario Using a Tangible Interface in Human Robot Interaction. In Serious Games [Lecture Notes in Computer Science, 11243], ► pp. 241 ff. DOI logo

This list is based on CrossRef data as of 16 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue