Cover not available

Article published In: Gesture
Vol. 19:1 (2020) ► pp.41–71

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Emotion matters

The effect of hand gesture on emotionally valenced sentences

 | Colgate University
 | Colgate University
Published online: 18 May 2021
DOI logo with link
https://doi.org/10.1075/gest.19029.lev
Abstract
Recent theories and neural models of co-speech gesture have extensively considered its cognitive role in language comprehension but have ignored the emotional function. We investigated the integration of speech and co-speech gestures in memory for verbal information with different emotional connotations (either positive, negative, or neutral). In a surprise cued-recall task, gesture boosted memory for speech with all three emotional valences. Interestingly, gesture was more likely to become integrated into memory of neutrally and positively valenced speech than negatively valenced speech. The results suggest that gesture-speech integration is modulated by emotional valence of speech, which has implications for the emotional function of gesture in language comprehension.
Article outline
  • Cognitive function of gesture: Production
  • Cognitive function of gesture: Comprehension
  • Emotional valence and gesture
  • Present study
  • Method
    • Participants
    • Materials
      • Video stimulus
      • Stimulus norming
      • Filler task
      • Memory tests
    • Procedure
    • Coding
    • Design
  • Results
    • Cued-recall responses
      • Remembering of speech
      • Addition of information from gesture
      • Remembering speech and adding from gesture
  • Discussion
    • Gesture and memory
    • Emotional valence and memory
  • Conclusion
  • Acknowledgements
  • Notes
  • Cued-recall response coding manual
  • References
References (54)
References
Adelman, James S. & Estes, Zachary. (2013). Emotion and memory: A recognition advantage for positive and negative words independent of arousal. Cognition, 1291, 530–535. Google Scholar logo with link to Google Scholar
Baumeister, Roy F., Bratslavsky, Ellen, Finkenauer, Catrin, & Vohs, Kathleen D. (2001). Bad is stronger than good. Review of General Psychology, 5 (4), 323–370. Google Scholar logo with link to Google Scholar
Bohn, Annette & Berntsen, Dorthe. (2007). Pleasantness bias in flashbulb memories: Positive and negative flashbulb memories of the fall of the Berlin Wall among east and west Germans. Memory & Cognition, 35 (3), 565–577. Google Scholar logo with link to Google Scholar
Borg, Céline, Bouazza, Zineb, Godeau, Marielle, Getenet, Jean-Claude, & Chainay, Hanna. (2018). Effect of emotion and type of encoding on memory for actions: Verbal and subjected-performed tasks. Dementia and Geriatric Cognitive Disorders, 45 (3/4), 162–179. Google Scholar logo with link to Google Scholar
Bowen, Holly J., Kark, Sarah M., & Kensinger, Elizabeth A. (2018). NEVER forget: negative emotional valence enhances recapitulation. Psychonomic Bulletin & Review, 25 (3), 870–891. Google Scholar logo with link to Google Scholar
Broaders, Sara C. & Goldin-Meadow, Susan. (2010). Truth is at hand: How gesture adds information during investigative interviews. Psychological Science, 21 (5), 623–628. Google Scholar logo with link to Google Scholar
Charles, Susan T., Mather, Mara, & Carstensen, Laura L. (2003). Aging and emotional memory: The forgettable nature of negative images for older adults. Journal of Experimental Psychology: General, 132 (2), 310–324. Google Scholar logo with link to Google Scholar
Church, R. Breckinridge, Alibali, Martha W., & Kelly, Spencer D. (Eds.). (2017). Why gesture? How the hands function in speaking, thinking and communicating. Amsterdam & Philadelphia: John Benjamins Publishing Company. Google Scholar logo with link to Google Scholar
Church, R. Breckinridge, Garber, Phillip, & Rogalski, Kathryn. (2007). The role of gesture in memory and social communication. Gesture, 7 (2), 137–158. Google Scholar logo with link to Google Scholar
Cohen, Ronald L. (1989). Memory for action events: The power of enactment. Educational Psychology Review, 1 (1), 57–80. Google Scholar logo with link to Google Scholar
Darwin, Charles. (1872). The expression of the emotions in man and animals. London, England: John Murray. Google Scholar logo with link to Google Scholar
De Stefani, Elisa & De Marco, Doriana. (2019). Gesture, language and emotional communication: An embodied view of social interaction. Frontiers in Psychology, 101, 2063. Google Scholar logo with link to Google Scholar
Dewhurst, Stephen A. & Parry, Lisa A. (2000). Emotionality, distinctiveness, and recollective experience. European Journal of Cognitive Psychology, 12 (4), 541–551. Google Scholar logo with link to Google Scholar
Ekman, Paul & Friesen, Wallace V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1 (1), 49–98. Google Scholar logo with link to Google Scholar
(1972). Hand movements. Journal of Communication, 22 (4), 353–374. Google Scholar logo with link to Google Scholar
Feyereisen, Pierre. (2009). Enactment effects and integration processes in younger and older adults’ memory for actions. Memory, 17 (4), 374–385. Google Scholar logo with link to Google Scholar
Gallese, Vittorio. (2018). Embodied simulation and its role in cognition. Reti, saperi, linguaggi, 7 (13), 31–46.Google Scholar logo with link to Google Scholar
Goldin-Meadow, Susan & Morford, Marolyn. (1985). Gesture in early child language: Studies of deaf and hearing children. Merrill-Palmer Quarterly, 31 (2), 145–176.Google Scholar logo with link to Google Scholar
Hirata, Yukari, Kelly, Spencer D., Huang, Jessica, & Manansala, Michael. (2014). Effects of hand gestures on auditory learning of second-language vowel length contrasts. Journal of Speech, Language, and Hearing Research, 57 (6), 2090–2101. Google Scholar logo with link to Google Scholar
Holland, Alisha C. & Kensinger, Elizabeth A. (2012). Younger, middle-aged, and older adults’ memories for the 2008 Presidential election. Journal of Applied Research on Memory and Cognition, 1 (3), 163–170. Google Scholar logo with link to Google Scholar
Hostetter, Autumn B. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137 (2), 297–315. Google Scholar logo with link to Google Scholar
Hostetter, Autumn B. & Alibali, Martha W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15 (3), 495–514. Google Scholar logo with link to Google Scholar
(2018). Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review, 261, 721–752. Google Scholar logo with link to Google Scholar
Kelly, Spencer D. (2017). Exploring the boundaries of gesture-speech integration during language comprehension. In R. B. Church, M. W. Alibali, & S. D. Kelly (Eds.), Why gesture? How the hands function in speaking, thinking and communicating (pp. 243–266). Amsterdam & Philadelphia: John Benjamins Publishing Company. Google Scholar logo with link to Google Scholar
Kelly, Spencer D., Barr, Dale J., Church, R. Breckinridge, & Lynch, Katheryn. (1999). Offering a hand to pragmatic understanding: The role of speech and gesture in comprehension and memory. Journal of Memory and Language, 401, 577–592. Google Scholar logo with link to Google Scholar
Kelly, Spencer D., Hirata, Yukari, Manansala, Michael, & Huang, Jessica. (2014). Exploring the role of hand gestures in learning novel phoneme contrasts and vocabulary in a second language. Frontiers in Psychology, 5, 673, 1–11. Google Scholar logo with link to Google Scholar
Kelly, Spencer D. & Lee, Angela L. (2012). When actions speak too much louder than words: Hand gestures disrupt word learning when phonetic demands are high. Language and Cognitive Processes, 27 (6), 793–807. Google Scholar logo with link to Google Scholar
Kendon, Adam. (1972). Some relationships between body motion and speech. In Aron Wolfe Siegman & Benjamin Pope (Eds.), Studies in dyadic communication (pp. 177–210). New York: Pergamon Press. Google Scholar logo with link to Google Scholar
. (1980). Gesticulation and speech: Two Aspects of the. In Mary R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–227). The Hague: Mouton. Google Scholar logo with link to Google Scholar
Kensinger, Elizabeth A. (2009). Remembering the details: Effects of emotion. Emotion Review, 1 (2), 99–113. Google Scholar logo with link to Google Scholar
Kensinger, Elizabeth A. & Choi, Elizabeth S. (2009). When side matters: Hemispheric processing and the visual specificity of emotional memories. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35 (1), 247–253.Google Scholar logo with link to Google Scholar
Kensinger, Elizabeth A. & Corkin, Suzanne. (2004). Two routes to emotional memory: Distinct neural processes for valence and arousal. Proceedings of the National Academy of Sciences, 101 (9), 3310–3315. Google Scholar logo with link to Google Scholar
Kensinger, Elizabeth A., Garoff-Eaton, Rachel J., & Schacter, Daniel L. (2006). Memory for specific visual details can be enhanced by negative arousing content. Journal of Memory and Language, 541, 99–112. Google Scholar logo with link to Google Scholar
Kensinger, Elizabeth A. & Schacter, Daniel L. (2006). When the red sox shocked the yankees: Comparing negative and positive memories. Psychonomic Bulletin & Review, 13 (5), 757–763. Google Scholar logo with link to Google Scholar
Krahmer, Emiel & Swerts, Marc. (2007). The effects of visual beats on prosodic prominence: Acoustic analyses, auditory perception and visual perception. Journal of Memory and Language, 57 (3), 396–414. Google Scholar logo with link to Google Scholar
Levine, Linda J. & Bluck, Susan. (2004). How emotions fade: valence, appraisals, and the emotional impact of remembered events. In S. P. Shohov (Ed.), Advances in psychology research (Vol. 301, pp. 3–20). Hauppauge, NY: Nova Science Publishers.Google Scholar logo with link to Google Scholar
Macedonia, Manuela, Müller, Karsten, & Friederici, Angela D. (2011). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32 (6), 982–998. Google Scholar logo with link to Google Scholar
Matheson, Heath E. & Barsalou, Lawrence W. (2018). Embodiment and grounding in cognitive neuroscience. Stevens’ handbook of experimental psychology and cognitive neuroscience, 31, 1–27. Google Scholar logo with link to Google Scholar
McNeill, David. (1992). Hand and mind. Chicago: The University of Chicago Press.Google Scholar logo with link to Google Scholar
McNeill, David & Levy, Elena. (1982). Conceptual representations in language activity and gesture. In R. J. Jarvella & W. Klein (Eds.), Speech, place, and action: Studies in deixis and related topics (pp. 271–295). New York: John Wiley & Sons.Google Scholar logo with link to Google Scholar
McNeill, David, Pedelty, Laura L., & Levy, Elena T. (1990). Speech and gesture. Advances in Psychology, 701, 203–256. Google Scholar logo with link to Google Scholar
Nascimento, Juliana Maria Steffen do, Grüdtner Buratto, Luciano, Schaefer, Alexandre, & Milnitsky Stein, Lilian. (2016). Emotional valence and perceived event frequency affect memory accuracy for a personally relevant life event. Applied Cognitive Psychology, 301, 1020–1029. Google Scholar logo with link to Google Scholar
Nilsson, Lars-Göran & Bäckman, Lars. (1989). Implicit memory and the enactment of verbal instructions. In Stephan Lewandowsky, John C. Dunn, & Kim Kirsner (Eds.), Implicit memory: Theoretical issues (pp. 173–183). Hillsdale, NJ: Erlbaum.Google Scholar logo with link to Google Scholar
Ochsner, Kevin N. (2000). Are affective events richly recollected or simply familiar? The experience and process of recognizing feelings past. Journal of Experimental Psychology: General, 129 (2), 242–261. Google Scholar logo with link to Google Scholar
Post, Lysanne S., van Gog, Tamara, Paas, Fred, & Zwaan, Rolf A. (2013). Effects of simultaneously observing and making gestures while studying grammar animations on cognitive load and learning. Computers in Human Behavior, 29 (4), 1450–1455. Google Scholar logo with link to Google Scholar
Pouw, Wim T. J. L., de Nooijer, Jaqueline A., van Gog, Tamara, Zwaan, Rolf A., & Paas, Fred. (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Frontiers in Psychology, 5, 359, 1–14. Google Scholar logo with link to Google Scholar
Rozin, Paul & Royzman, Edward B. (2001). Negativity bias, negativity dominance, and contagion. Personality and social psychology review, 5 (4), 296–320. Google Scholar logo with link to Google Scholar
Sekine, Kazuki, Sowden, Hannah, & Kita, Sotaro. (2015). The development of the ability to semantically integrate information in speech and iconic gesture in comprehension. Cognitive Science, 391, 1855–1880. Google Scholar logo with link to Google Scholar
Skipper, Jeremy I. (2015). The NOLB model: A model of the natural organization of language and the brain. In Roel M. Willems (Ed.), Cognitive neuroscience of natural language use (pp. 101–134). Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Spranger, Tina, Schatz, Tanja R., & Knopf, Monika. (2008). Does action make you faster? A retrieval-based approach to investigating the origins of the enactment effect. Scandinavian Journal of Psychology, 49 (6), 487–495. Google Scholar logo with link to Google Scholar
Straube, Benjamin, Green, Antonia, Weis, Susanne, Chatterjee, Anjan, & Kircher, Tilo. (2009). Memory effects of speech and gesture binding: cortical and hippocampal activation in relation to subsequent memory performance. Journal of Cognitive Neuroscience, 21 (4), 821–836. Google Scholar logo with link to Google Scholar
Wang, Bo. (2013). Facial expression influences recognition memory for faces: Robust enhancement effect of fearful expression. Memory, 21 (3), 301–314. Google Scholar logo with link to Google Scholar
Wilson, Margaret. (2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 9 (4), 625–636. Google Scholar logo with link to Google Scholar
Yeo, Amelia, Ledesma, Iasmine, Nathan, Mitchell J., Alibali, Martha W., & Church, Ruth Breckinridge. (2017). Teachers’ gestures and students’ learning: Sometimes “hands off” is better. Cognitive Research: Principles and Implications, 2 (1), 411, 1–11. Google Scholar logo with link to Google Scholar
Cited by (6)

Cited by six other publications

Ceylan, Süleyman Can, Demet Özer & Tilbe Göksun
2025. Retellings of emotional videos: the dynamics of gesture characteristics in diverse emotion categories. Cognition and Emotion ► pp. 1 ff. DOI logo
Kelly, Spencer D. & Quang‐Anh Ngo Tran
2025. Exploring the Emotional Functions of Co‐Speech Hand Gesture in Language and Communication. Topics in Cognitive Science 17:3 ► pp. 586 ff. DOI logo
Tian, Lixian, Steven G. McCafferty & Man Zhu
2025. The cognitive-emotional dialectics in second language development: speech and co-speech gestures of Chinese learners of English. International Review of Applied Linguistics in Language Teaching 63:4 ► pp. 2633 ff. DOI logo
Aslan, Zeynep, Demet Özer & Tilbe Göksun
2024. Exploring Emotions Through Co-speech Gestures: The Caveats and New Directions. Emotion Review 16:4 ► pp. 265 ff. DOI logo
Rühlemann, Christoph & James Trujillo
2024. The effect of gesture expressivity on emotional resonance in storytelling interaction. Frontiers in Psychology 15 DOI logo
Thakore, Kavya, Trisha Das, Shamma Jahan & Naomi Sweller
2024. From Hands to Mind: How Gesture, Emotional Valence, and Individual Differences Impact Narrative Recall. Educational Psychology Review 36:4 DOI logo

This list is based on CrossRef data as of 9 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

video

video 1

video 2

Video clips

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue