Cover not available

Article published In: Gesture
Vol. 23:1/2 (2024) ► pp.97–118

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Hand-y scaffolding:

An examination of the impact of iconic gestures on comprehension and memory of L1 and L2 accented spoken narratives

 | University of Missouri
 | Fordham University
ORCiD logo with link | University of Missouri
Published online: 28 July 2025
DOI logo with link
https://doi.org/10.1075/gest.24037.you
Abstract
Currently, it is unclear whether observing iconic gestures benefits comprehension of second language (L2) accented narratives, and whether it benefits it to a different degree than it does for first language (L1) accented narratives. In this study, L1 American English speakers watched videos of two narratives, one of which featured L1 accented American English, and the other of which featured L2 French-accented English. One narrative featured iconic gestures complementing speech, and the other did not contain gestures. Despite a marginally greater benefit of iconic gestures for L1 than L2 accented narratives, observing iconic gestures had no significant impact on comprehension, free recall, and event memory for L1 and L2 accented narratives. These findings suggest that observing iconic gestures may not enhance comprehension or memory for L2 accented spoken narratives. Implications for improving communication between L1 and L2 accented speakers are discussed.
Article outline
  • Introduction
  • Method
    • Participants
    • Materials
    • Procedure
    • Coding
  • Results
    • Note taking
    • Free recall
    • Event memory
  • Discussion
  • Acknowledgements
  • Notes
  • References
References (70)
References
Algana, M., & Hardison, D. M. (2024). Variable effects of speakers’ visual cues and accent on L2 listening comprehension: A mixed-methods approach. Language Teaching Research, 13621688241246106. Google Scholar logo with link to Google Scholar
Anderson-Hsieh, J., & Koehler, K. (1988). The effect of foreign accent and speaking rate on native speaker comprehension. Language Learning, 38(4), 561–613. Google Scholar logo with link to Google Scholar
Austin, E. E., & Sweller, N. (2014). Presentation and production: The role of gesture in spatial communication. Journal of Experimental Child Psychology, 1221, 92–103. Google Scholar logo with link to Google Scholar
Austin, E. E., Sweller, N., & Van Bergen, P. (2018). Pointing the way forward: Gesture and adults’ recall of route direction information. Journal of Experimental Psychology: Applied, 24(4), 490–508.Google Scholar logo with link to Google Scholar
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. Google Scholar logo with link to Google Scholar
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using Ime4. Journal of Statistical Software, 67(1), 1–48. Google Scholar logo with link to Google Scholar
Bharadwaj, A., Dargue, N., & Sweller, N. (2022). A hands-on approach to learning: Gesture production during encoding and its effect on narrative recall. Cognitive Science, 46(12). Google Scholar logo with link to Google Scholar
Cameron, H., & Xu, X. (2011). Representational gesture, pointing gesture, and memory recall of preschool children. Journal of Nonverbal Behavior, 35(2), 155–171. Google Scholar logo with link to Google Scholar
Carlson, C., Jacobs, S. A., Perry, M., & Church, R. B. (2014). The effect of gestured instruction on the learning of physical causality problems. Gesture, 14(1), 26–45. Google Scholar logo with link to Google Scholar
Church, R. B., Ayman-Nolley, S., & Mahootian, S. (2004). The role of gesture in bilingual education: Does gesture enhance learning?. International Journal of Bilingual Education and Bilingualism, 7(4), 303–319. Google Scholar logo with link to Google Scholar
Church, R. B., Garber, P., & Rogalski, K. (2007). The role of gesture in memory and social communication. Gesture, 7(2), 137–158. Google Scholar logo with link to Google Scholar
Church, R. B., Kelly, S. D., & Lynch, K. (2000). Immediate memory for mismatched speech and representational gesture across development. Journal of Nonverbal Behavior, 241, 151–174. Google Scholar logo with link to Google Scholar
Clarke, C. M., & Garrett, M. F. (2004). Rapid adaptation to foreign-accented English. The Journal of the Acoustical Society of America, 116(6), 3647–3658. Google Scholar logo with link to Google Scholar
Cook, S. W., Duffy, R. G., & Fenn, K. M. (2013). Consolidation and transfer of learning after observing hand gesture. Child Development, 84(6), 1863–1871. Google Scholar logo with link to Google Scholar
Dahl, T. I., & Ludvigsen, S. (2014). How I see what you’re saying: The role of gestures in native and foreign language listening comprehension. The Modern Language Journal, 93(3). Google Scholar logo with link to Google Scholar
Dargue, N., & Sweller, N. (2018a). Donald Duck’s garden: The effects of observing iconic reinforcing and contradictory gestures on narrative comprehension. Journal of Experimental Child Psychology, 1751, 96–107. Google Scholar logo with link to Google Scholar
(2018b). Not all gestures are created equal: The effects of typical and atypical iconic gestures on narrative comprehension. Journal of Nonverbal Behavior, 42(3), 1–19. Google Scholar logo with link to Google Scholar
(2020). Learning stories through gesture: Gesture’s effects on child and adult narrative comprehension. Educational Psychology Review, 321, 249–276. Google Scholar logo with link to Google Scholar
Dargue, N., Sweller, N., & Jones, M. P. (2019). When our hands help us understand: A meta-analysis into the effects of gesture on comprehension. Psychological Bulletin, 145(8), 765–784. Google Scholar logo with link to Google Scholar
Drijvers, L., & Özyürek, A. (2017). Visual context enhanced: The joint contribution of iconic gestures and visible speech to degraded speech comprehension. Journal of Speech, Language, and Hearing Research, 60(1), 212–222. Google Scholar logo with link to Google Scholar
(2020). Non-native listeners benefit less from gestures and visible speech than native listeners during degraded speech comprehension. Language and Speech, 63(2), 209–220. Google Scholar logo with link to Google Scholar
Drijvers, L., Vaitonytė, J., & Özyürek, A. (2019). Degree of language experience modulates visual attention to visible speech and iconic gestures during clear and degraded speech comprehension. Cognitive Science, 43(10), e12789. Google Scholar logo with link to Google Scholar
Driskell, J. E., & Radtke, P. H. (2003). The effect of gesture on speech production and comprehension. Human Factors, 45(3), 445–454. Google Scholar logo with link to Google Scholar
Feyereisen, P. (2006). Further investigation on the mnemonic effect of gestures: Their meaning matters. European Journal of Cognitive Psychology, 18(2), 185–205. Google Scholar logo with link to Google Scholar
Goldin-Meadow, S. (1999). The role of gesture in communication and thinking. Trends in Cognitive Sciences, 3(11), 419–429. Google Scholar logo with link to Google Scholar
Grey, S., Cosgrove, A. L., & van Hell, J. G. (2020). Faces with foreign accents: An event-related potential study of accented sentence comprehension. Neuropsychologia, 1471. Google Scholar logo with link to Google Scholar
Hansen, K., Steffens, M. C., Rakić, T., & Wiese, Holger. (2017). When appearance does not match accent: Neural correlates of ethnicity-related expectancy violations. Social Cognitive and Affective Neuroscience, 12(3), 507–515. Google Scholar logo with link to Google Scholar
Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137(2), 297–315. Google Scholar logo with link to Google Scholar
Hubbard, A. L., Wilson, S. M., Callan, D. E., & Dapretto, M. (2009). Giving speech a hand: Gesture modulates activity in auditory cortex during speech perception. Human Brain Mapping, 30(3), 1028–1037. Google Scholar logo with link to Google Scholar
Ianì, F., & Bucciarelli, M. (2017). Mechanisms underlying the beneficial effect of a speaker’s gestures on the listener. Journal of Memory and Language, 961, 110–121. Google Scholar logo with link to Google Scholar
Kelly, S. D., Barr, D. J., Church, R. B., & Lynch, K. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 40(4), 577–592. Google Scholar logo with link to Google Scholar
Kelly, S. D., & Church, R. B. (1997). Can children detect conceptual information conveyed through other children's nonverbal behaviors. Cognition and Instruction, 15(1), 107–134. Google Scholar logo with link to Google Scholar
(1998). A comparison between children's and adults' ability to detect conceptual information conveyed through representational gestures. Child Development, 69(1), 85–93. Google Scholar logo with link to Google Scholar
Kelly, S. D., McDevitt, T., & Esch, M. (2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes, 24(2), 313–334. Google Scholar logo with link to Google Scholar
Kelly, S. D., & Ngo Tran, Q. A. (2023). Exploring the Emotional Functions of Co-Speech Hand Gesture in Language and Communication. Topics in Cognitive Science. Google Scholar logo with link to Google Scholar
Kelly, S. D., Özyürek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21(2). Google Scholar logo with link to Google Scholar
Koumoutsakis, T., Church, R. B., Alibali, M. W., Singer, M., & Ayman-Nolley, S. (2016). Gesture in instruction: Evidence from live and video lessons. Journal of Nonverbal Behavior, 401, 301–315. Google Scholar logo with link to Google Scholar
Krason, A., Fenton, R., Varley, R., & Vigliocco, G. (2021). The role of iconic gestures and mouth movements in face-to-face communication. Psychonomic Bulletin & Review, 29(2), 600–612. Google Scholar logo with link to Google Scholar
Krason, A., Varley, R., & Vigliocco, G. (2024). Understanding discourse in face-to-face settings: The impact of multimodal cues and listening conditions. Journal of Experimental Psychology: Learning, Memory, and Cognition, 51(5), 837–854. Google Scholar logo with link to Google Scholar
Krauss, R. M., Dushay, R. A., Chen, Y., & Rauscher, F. (1995). The communicative value of conversational hand gesture. Journal of Experimental Social Psychology, 31(6), 533–552. Google Scholar logo with link to Google Scholar
Krauss, R. M., Morrel-Samuels, P., & Colasante, C. (1991). Do conversational hand gestures communicate? Journal of Personality and Social Psychology, 61(5), 743. Google Scholar logo with link to Google Scholar
Kuznetsova, A., Brockhoff, P. B., Christensen, R. H. B. (2017). ImerTest package: Tests in linear mixed effects models. Journal of Statistical Software, 821, 1–26. Google Scholar logo with link to Google Scholar
Lin, Y. L. (2022). Speech-accompanying gestures in L1 and L2 conversational interaction by speakers of different proficiency levels. International Review of Applied Linguistics in Language Teaching, 60(2), 123–142. Google Scholar logo with link to Google Scholar
Llanes-Coromina, J., Vilà-Giménez, I., Kushch, O., Borràs-Comes, J., & Prieto, P. (2018). Beat gestures help preschoolers recall and comprehend discourse information. Journal of Experimental Child Psychology, 1721, 168–188. Google Scholar logo with link to Google Scholar
Macoun, A., & Sweller, N. (2016). Listening and watching: The effects of observing gesture on preschoolers’ narrative comprehension. Cognitive Development, 401, 68–81. Google Scholar logo with link to Google Scholar
Major, R. C., Fitzmaurice, S. F., Bunta, F., & Balasubramanian, C. (2002). The effects of nonnative accents on listening comprehension: Implications for ESL assessment. TESOL Quarterly, 36(2), 173–190. Google Scholar logo with link to Google Scholar
Maye, J., Aslin, R. N., & Tanenhaus, M. K. (2008). The weckud wetch of the wast: Lexical adaptation to a novel accent. Cognitive Science, 32(3), 543–562. Google Scholar logo with link to Google Scholar
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press.Google Scholar logo with link to Google Scholar
Morett, L. M. (2018). In hand and in mind: Effects of gesture production and viewing on second language word learning. Applied Psycholinguistics, 39(2), 355–381. Google Scholar logo with link to Google Scholar
(2023). Observing gesture at learning enhances subsequent phonological and semantic processing of L2 words: An N400 study. Brain and Language, 2461, 105327. Google Scholar logo with link to Google Scholar
Morett, L. M., & Chang, L. Y. (2015). Emphasizing sound and meaning: Pitch gestures enhance Mandarin lexical tone acquisition. Language, Cognition, and Neuroscience, 30(3), 347–353. Google Scholar logo with link to Google Scholar
Morett, L. M., Feiler, J. B., & Getz, L. M. (2022). Elucidating the influences of embodiment and conceptual metaphor on lexical and non-speech tone learning. Cognition, 2221, 105014. Google Scholar logo with link to Google Scholar
Morett, L. M., & Fraundorf, S. H. (2019). Listeners consider alternative speaker productions in discourse comprehension and memory: Evidence from beat gesture and pitch accenting. Memory & Cognition, 47(8), 1515–1530. Google Scholar logo with link to Google Scholar
Morett, L. M., Fraundorf, S. H., & McPartland, J. C. (2021). Eye see what you're saying: Contrastive use of beat gesture and pitch accent affects online interpretation of spoken discourse. Journal of Experimental Psychology: Learning, Memory, and Cognition, 47(9), 1494. Google Scholar logo with link to Google Scholar
Morett, L. M., Roche, J. M., Fraundorf, S. H., & McPartland, J. C. (2020). Contrast is in the eye of the beholder: Infelicitous beat gesture increases cognitive load during online spoken discourse comprehension. Cognitive Science, 44(10), e12912. Google Scholar logo with link to Google Scholar
Munro, M. J., & Derwing, T. M. (1995). Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Language Learning, 45(1), 73–97. Google Scholar logo with link to Google Scholar
Özer, D., Özyürek, A., & Göksun, T. (2025). Spatial working memory is critical for gesture processing: Evidence from gestures with varying semantic links to speech. Psychonomic Bulletin & Review, 1–15. Google Scholar logo with link to Google Scholar
Paivio, A. (1990). Mental representations: A dual coding approach. Oxford University Press. Google Scholar logo with link to Google Scholar
Ping, R., Church, R. B., Decatur, M. A., Larson, S. W., Zinchenko, E., & Goldin-Meadow, S. (2021). Unpacking the gestures of chemistry learners: What the hands tell us about correct and incorrect conceptions of stereochemistry. Discourse processes, 58(3), 213–232. Google Scholar logo with link to Google Scholar
Ping, R., Parrill, F., Church, R. B., & Goldin-Meadow, S. (2022). Teaching stereoisomers through gesture, action, and mental imagery. Chemistry Education Research and Practice, 23(3), 698–713. Google Scholar logo with link to Google Scholar
Rueckert, L., Church, R. B., Avila, A., & Trejo, T. (2017). Gesture enhances learning of a complex statistical concept. Cognitive Research: Principles and Implications, 21, 1–6. Google Scholar logo with link to Google Scholar
Shapiro, L. (2019). Embodied cognition. Routledge. Google Scholar logo with link to Google Scholar
Sueyoshi, A., & Hardison, D. M. (2005). The role of gestures and facial cues in second language listening comprehension. Language Learning, 55(4), 661–699. Google Scholar logo with link to Google Scholar
Tsunemoto, A., Lindberg, R., Trofimovich, P. , & McDonough, K. (2022). Visual cues and rater perceptions of second language comprehensibility, accentedness, and fluency. Studies in Second Language Acquisition, 44(3), 659–684. Google Scholar logo with link to Google Scholar
Valdiviejas, H., Koumoutsakis, T., Mistak, A., Mogil, A., Ayman-Nolley, S., & Church, R. B. (2022). Can gesture help English language learners understand the equal sign?. NABE Journal of Research and Practice, 12(3-4), 121–134. Google Scholar logo with link to Google Scholar
Wakefield, E., Novack, M. A., Congdon, E. L., Franconeri, S., & Goldin-Meadow, S. (2018). Gesture helps learners learn, but not merely by guiding their visual attention. Developmental Science, 21(6), e12664. Google Scholar logo with link to Google Scholar
Wheeler, P., & Saito, K. (2022). Second language speech intelligibility revisited: Differential roles of phonological accuracy, visual speech, and iconic gesture. The Modern Language Journal, 106(2), Google Scholar logo with link to Google Scholar
Witteman, M. J., Bardhan, N. P., Weber, A., & McQueen, J. M. (2015). Automaticity and stability of adaptation to a foreign-accented speaker. Language and Speech, 58(2), 168–189. Google Scholar logo with link to Google Scholar
Yeo, A., Ledesma, I., Nathan, M. J., Alibali, M. W., & Church, R. B. (2017). Teachers’ gestures and students’ learning: sometimes “hands off” is better. Cognitive Research: Principles and Implications, 21, 1–11. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue