Cover not available

Article published In: Pragmatics & Cognition
Vol. 31:1 (2024) ► pp.49–96

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Four-year-olds’ visuospatial cognitive abilities and their relation to observer‑viewpoint gestures across three communicative tasks

 | Paderborn University
 | Bielefeld University
 | Bielefeld University
 | Bielefeld University
 | Bielefeld University
 | Bielefeld University
 | Paderborn University
Published online: 11 October 2024
DOI logo with link
https://doi.org/10.1075/pc.00040.roh
Abstract
The gesture-as-simulated-action framework explains the occurrence of iconic gestures. Accordingly, simulated visual imagery gives rise to observer-viewpoint, whereas simulated motor imagery gives rise to character-viewpoint gestures. Because little is known about whether this relationship is either the product of becoming a competent speaker in different communicative tasks or exists from an early age, we investigated 4-year-olds. In the first session, 55 children performed three different communicative tasks. In the second session, we administered a SON-R non-verbal intelligence test to assess children’s cognitive abilities. Results revealed a strong association between the frequency with which children used observer-viewpoints gestures and their scores in visual processing abilities. Because the association between gestures and visuospatial cognitive abilities was found across three communicative tasks, we can propose that it is a general ability rather than a phenomenon of task-specific discourse competence. We discuss these findings in terms of gestural threshold and cognitive offloading.
Article outline
  • 1.Introduction
    • 1.1Iconic gestures and visual stimuli
    • 1.2Iconic gestures and individual differences in cognitive skills
    • 1.3Viewpoints in iconic gestures
    • 1.4Development of the integration of gesture, language, and event knowledge
    • 1.5The present study
  • 2.Methods
    • 2.1Pilot study
    • 2.2Participants
    • 2.3Procedure
    • 2.4Communicative tasks
      • 2.4.1Explanation
      • 2.4.2Retelling
      • 2.4.3Illustration
    • 2.5Stimuli
      • 2.5.1Explanation
      • 2.5.2Retelling
      • 2.5.3Illustration
    • 2.6Intelligence test
    • 2.7Coding
      • 2.7.1Speech
      • 2.7.2Gesture
      • 2.7.3Speech about event aspects related to gestures
    • 2.8Data analysis
  • 3.Results
    • 3.1Communicative tasks
    • 3.2Gestures and SON-R scores
      • 3.2.1Visual processing
      • 3.2.2Fluid intelligence
    • 3.3Speech and visual processing
    • 3.4Alignment between children’s and caregiver’s communication
  • 4.Discussion
    • 4.1Viewpoints across the communicative tasks
    • 4.2Children’s and caregivers’ interactive behavior
    • 4.3Children’s speech and their use of viewpoints
    • 4.4Children’s speech, viewpoints, and their visual processing scores
    • 4.5Generalizability of the results across tasks
    • 4.6Possible mechanisms underlying the relation of the frequency in viewpoint gestures and children’s scores in visual processing abilities
  • 5.Conclusion
  • Authors’ contribution
  • Acknowledgements
  • Supplementary items
    • A.Further analyses of the task conditions in relation to children’s gestures and IQ
    • B.Script for the game explanation
    • C.Analysis of the event aspects
  • References
References (79)
References
Abramov, Olga, Friederike Kern, Sofia Koutalidis, Ulrich Mertens, Katharina J. Rohlfing & Stefan Kopp. 2021. The relation between cognitive abilities and the distribution of semantic features across speech and gesture in 4-year-olds. Cognitive Science 45(7). e13012. Google Scholar logo with link to Google Scholar
Alamillo, Asela Reig, Jean-Marc Colletta & Michèle Guidetti. 2013. Gesture and language in narratives and explanations: The effects of age and communicative activity on late multimodal discourse development. Journal of Child Language 40(3). 511–538. Google Scholar logo with link to Google Scholar
Austin, Elizabeth E. & Naomi Sweller. 2014. Presentation and production: The role of gesture in spatial communication. Journal of Experimental Child Psychology 1221. 92–103. Google Scholar logo with link to Google Scholar
Baldwin, Dare A. & Jessica E. Kosie. 2021. How does the mind render streaming experience as events? Topics in Cognitive Science 13(1). 79–105. Google Scholar logo with link to Google Scholar
Bergmann, Kirsten & Stefan Kopp. 2012. Gestural alignment in natural dialogue. In Proceedings of the Annual Meeting of the Cognitive Science Society 341. 1326–1331.Google Scholar logo with link to Google Scholar
Capirci, Olga, Annarita Contaldo, Maria Cristina Caselli & Virginia Volterra. 2005. From action to language through gesture: A longitudinal perspective. Gesture 51. 155–177. Google Scholar logo with link to Google Scholar
Cartmill, Erica A., Sian Beilock & Susan Goldin-Meadow. 2012. A word in the hand: Action, gesture and mental representation in humans and non-human primates. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 367(1585). 129–143. Google Scholar logo with link to Google Scholar
Cartmill, Erica A., Lilia Rissman, Miriam A. Novack & Susan Goldin-Meadow. 2017. The development of iconicity in children’s co-speech gesture and homesign. Language, Interaction and Acquisition 8(1). 42–68. Google Scholar logo with link to Google Scholar
Cattell, Raymond B. 1963. Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology 54(1). 1–22. Google Scholar logo with link to Google Scholar
Chu, Mingyuan & Sotaro Kita. 2011. The nature of gestures’ beneficial role in spatial problem solving. Journal of Experimental Psychology 140(1). 102–116. Google Scholar logo with link to Google Scholar
Chu, Mingyuan, Antje Meyer, Lucy Foulkes & Sotaro Kita. 2014. Individual differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and empathy. Journal of Experimental Psychology 143(2). 694–709. Google Scholar logo with link to Google Scholar
Church, R. Breckinridge & Susan Goldin-Meadow. 1986. The mismatch between gesture and speech as an index of transitional knowledge. Cognition 231. 43–71. Google Scholar logo with link to Google Scholar
Church, R. Breckinridge, Spencer D. Kelly & Kathryn Lynch. 2000. Immediate memory for mismatched speech and representational gesture across development. Journal of Nonverbal Behavior 24(2). 151–174. Google Scholar logo with link to Google Scholar
Cohen, Jacob. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1). 37–46. Google Scholar logo with link to Google Scholar
Cook, Susan Wagner, Terina Kuangyi Yip & Susan Goldin-Meadow. 2012. Gestures, but not meaningless movements, lighten working memory load when explaining math. Language and Cognitive Processes 27(4). 594–610. Google Scholar logo with link to Google Scholar
Crawshaw, Camilla E., Friederike Kern, Ulrich Mertens & Katharina J. Rohlfing. 2020. Children’s narrative elaboration after reading a storybook versus viewing a video. Frontiers in Psychology 111. 569891. Google Scholar logo with link to Google Scholar
Debreslioska, Sandra & Marianne Gullberg. 2019. Discourse reference is bimodal: How information status in speech interacts with presence and viewpoint of gestures. Discourse Processes 56(1). 41–60. Google Scholar logo with link to Google Scholar
Demir, Özlem Ece, Susan C. Levine & Susan Goldin-Meadow. 2015. A tale of two hands: Children’s early gesture use in narrative production predicts later narrative structure in speech. Journal of Child Language 42(3). 662–681. Google Scholar logo with link to Google Scholar
Doskočilová, Hana & Zdeněk Miler. 1998/2013. Der Maulwurf und der Grüne Stern [The Mole and the Green Star]. Leipzig: LeiV.Google Scholar logo with link to Google Scholar
Ehrlich, Stacy B., Susan C. Levine & Susan Goldin-Meadow. 2006. The importance of gesture in children’s spatial reasoning. Developmental Psychology 42(6). 1259–1268. Google Scholar logo with link to Google Scholar
Fischer, Kurt W. & Louise Silvern. 1985. Stages and individual differences in cognitive development. Annual Review of Psychology 36(1). 613–648. Google Scholar logo with link to Google Scholar
Flanagan, Dawn P. & Shauna G. Dixon. 2014. The Cattell-Horn-Carroll theory of cognitive abilities. In Cecil R. Reynolds, Kimberly J. Vannest & Elaine Fletcher-Janzen (eds.), Encyclopedia of Special education. New Jersey: Wiley. Google Scholar logo with link to Google Scholar
Frederiksen, Anne Therese. 2017. Separating viewpoint from mode of representation in iconic co-speech gestures: Insights from Danish narratives. Language and Cognition 9(4). 677–708. Google Scholar logo with link to Google Scholar
Furman, Reyhan, Alin C. Küntay & Asli Özyürek. 2014. Early language-specificity of children’s event encoding in speech and gesture: Evidence from caused motion in Turkish. Language, Cognition and Neuroscience 291. 620–634. Google Scholar logo with link to Google Scholar
Goldin-Meadow, Susan, Howard Nusbaum, Spencer D. Kelly & Susan Wagner. 2001. Explaining math: Gesturing lightens the load. Psychological Science 12(6). 516–522. Google Scholar logo with link to Google Scholar
Goldin-Meadow, Susan & Martha W. Alibali. 2013. Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology 641. 257–283. Google Scholar logo with link to Google Scholar
Goodwyn, Susan W., Linda P. Acredolo & Catherine A. Brown. 2000. Impact of symbolic gesturing on early language development. Journal of Nonverbal Behavior 241. 80–104. Google Scholar logo with link to Google Scholar
Holler, Judith & Stephen C. Levinson. 2019. Multimodal language processing in human communication. Trends in Cognitive Sciences 23(8). 639–652. Google Scholar logo with link to Google Scholar
Hostetter, Autumn B. 2011. When do gestures communicate? A meta-analysis. Psychological Bulletin 137(2). 297–315. Google Scholar logo with link to Google Scholar
. 2008. Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review 15(3). 495–514. Google Scholar logo with link to Google Scholar
. 2019. Gesture as simulated action: Revisiting the framework. Psychonomic Bulletin & Review 26(3). 721–752. Google Scholar logo with link to Google Scholar
Hostetter, Autumn B. & William D. Hopkins. 2002. The effect of thought structure on the production of lexical movements. Brain and Language 82(1). 22–29. Google Scholar logo with link to Google Scholar
Hostetter, Autumn B. & Catherine J. Skirving. 2011. The effect of visual vs. verbal stimuli on gesture production. Journal of Nonverbal Behavior 35(3). 205–223. Google Scholar logo with link to Google Scholar
Iverson, Jana M. & Thal, D. J. 1998. Communicative transitions. There’s more to the hand than meets the eye. In Amy M. Wetherby, Steven F. Warren & Joe Reichle (eds.), Transitions in prelinguistic communication, 59–86. Baltimore: Paul H. Brookes.Google Scholar logo with link to Google Scholar
Kern, Friederike, Ulrich Boden, Anne Nemeth, Sofia Koutalidis, Olga Abramov, Stefan Kopp & Katharina J. Rohlfing. 2024. Preschool children’s discourse competence in different genres and how it relates to iconic gestures. Journal of Child Language 51(3). 656–680. Google Scholar logo with link to Google Scholar
Kidd, Evan, Seamus Donnelly & Morten H. Christiansen. 2018. Individual differences in language acquisition and processing. Trends in Cognitive Sciences 22(2). 154–169. Google Scholar logo with link to Google Scholar
Kita, Sotaro, Martha W. Alibali & Mingyuan Chu. 2017. How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis. Psychological Review 124(3). 245–266. Google Scholar logo with link to Google Scholar
Landis, J. Richard & Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33(1). 159. Google Scholar logo with link to Google Scholar
Laros, Jacob Arie & Peter Johannes Tellegen. 2015. Construction and validation of the SON-R 5 1/2–17, the Snijders-Oomen non-verbal intelligence test.
LeBaron, Curtis & Jürgen Streeck. 2000. Gestures, knowledge and the world. In David McNeill (ed.), Review of language and gesture: Window into thought and action, 118–138. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lohman, David F. 1994. Spatial ability. In Robert J. Sternberg (ed.), Encyclopedia of human intelligence, 1000–1007. London: Macmillan.Google Scholar logo with link to Google Scholar
Lüke, Carina, Ute Ritterfeld, Angela Grimminger, Katharina J. Rohlfing & Ulf Liszkowski. 2020. Integrated communication system: Gesture and language acquisition in typically developing children and children with LD and DLD. Frontiers in Psychology 111. 498187. Google Scholar logo with link to Google Scholar
Mandler, Jean M. 2000. Perceptual and conceptual processes in infancy. Journal of Cognition and Development 1(1). 3–36. Google Scholar logo with link to Google Scholar
Marstaller, Lars & Hana Burianová. 2013. Individual differences in the gesture effect on working memory. Psychonomic Bulletin & Review 20(3). 496–500. Google Scholar logo with link to Google Scholar
McNeill, David. 1992. Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
Mickley, Manfred & Gerolf Renner. 2010. Intelligenztheorie für die Praxis: Auswahl, Anwendung und Interpretation deutschsprachiger Testverfahren für Kinder und Jugendliche auf Grundlage der CHCTheorie. Klinische Diagnostik und Evaluation 31. 447–466.Google Scholar logo with link to Google Scholar
. 2019. Auswahl, Anwendung und Interpretation deutschsprachiger Intelligenztests für Kinder und Jugendliche auf Grundlage der CHC-Theorie: Update, Erweiterung und kritische Bewertung [Selection, use, and interpretation of German intelligence tests for children and adolescents based on CHC-theory: Update, extension, and critical discussion]. Praxis der Kinderpsychologie und Kinderpsychiatrie 68(4). 323–343. Google Scholar logo with link to Google Scholar
Mizuguchi, Takashi & Kuniaki Sugai. 2002. Object-related knowledge and the production of gestures with imagined objects by preschool children. Perceptual and Motor Skills 94(1). 71–79. Google Scholar logo with link to Google Scholar
Möhring, Wenke, Andrew D. Ribner, Robin Segerer, Melissa E. Libertus, Tobias Kahl, Larissa Maria Troesch & Alexander Grob. 2021. Developmental trajectories of children’s spatial skills: Influencing variables and associations with later mathematical thinking. Learning and Instruction 751. 101515. Google Scholar logo with link to Google Scholar
Morsella, Ezequiel & Robert M. Krauss. 2004. The role of gestures in spatial working memory and speech. The American Journal of Psychology 117(3). 411–7424. Google Scholar logo with link to Google Scholar
Mumford, Katherine H. & Sotaro Kita. 2014. Children use gesture to interpret novel verb meanings. Child Development 85(3). 1181–1189. Google Scholar logo with link to Google Scholar
Murgiano, Margherita, Yasamin Motamedi & Gabriella Vigliocco. 2021. Situating language in the real-world: The role of multimodal iconicity and indexicality. Journal of Cognition 4(1). 38. Google Scholar logo with link to Google Scholar
O’Reilly, Anne Watson, Kathleen M. Painter & Marc H. Bornstein. 1997. Relations between language and symbolic gesture development in early childhood. Cognitive Development 121. 185–197. Google Scholar logo with link to Google Scholar
Özer, Demet & Tilbe Göksun. 2020. Gesture use and processing: A review on individual differences in cognitive resources. Frontiers in Psychology 111. 573555. Google Scholar logo with link to Google Scholar
Özyürek, Asli. 2010. The role of iconic gestures in production and comprehension of language: Evidence from brain and behavior. In Stefan Kopp & Iple Wachsmuth (eds.), Gesture in embodied communication and human-computer interaction. New York: Springer. Google Scholar logo with link to Google Scholar
Parrill, Fey. 2010. Viewpoint in speech-gesture integration: Linguistic structure, discourse structure, and event structure. Language and Cognitive Processes 25(5). 650–668. Google Scholar logo with link to Google Scholar
Parrill, Fey, Brittany Lavanty, Austin Bennett, Alayna Klco & Ozlem Ece Demir-Lira. 2018. The relationship between character viewpoint gesture and narrative structure in children. Language and Cognition 10(3). 408–434. Google Scholar logo with link to Google Scholar
Perry, Michelle, R. Breckinridge Church & Susan Goldin-Meadow. 1992. Is gesture-speech mismatch a general index of transitional knowledge? Cognitive Development 7(1). 109–122. Google Scholar logo with link to Google Scholar
Quasthoff, Uta, Vivian Heller & Miriam Morek. 2017. On the sequential organization and genre-orientation of discourse units in interaction: An analytic framework. Discourse Studies 19(1). 84–110. Google Scholar logo with link to Google Scholar
Raven, John C. 1962. Advanced progressive matrices. London: Lewis.Google Scholar logo with link to Google Scholar
Rettig, Heike. 2017. Mit Beispielen argumentieren. Die Analyse von Beispielverwendung in der verbalen Interaktion. In Iris Meißner & Eva Lia Wyss (eds.), Begründen — Erklären — Argumentieren: Konzepte und Modellierungen in der Angewandten Linguistik, 159–188. Tübingen: Stauffenburg Verlag.Google Scholar logo with link to Google Scholar
Rohlfing, Katharina J. 2019. Language learning from the use of gestures. In Jessica Horst & Janne von Koss Torkildsen (eds.), International Handbook of Language Acquisition, 213–233. London: Routledge. Google Scholar logo with link to Google Scholar
Rohlfing, Katharina J., Ulrich Mertens, Sofia Koutalidis, Olga Abramov, Stefan Kopp & Friederike Kern. 2020. Through the lens of a communicative genre: Differences in use of iconic co-speech gestures by 4-years-old children. Poster presented at the online conference “Many Paths to Language”, Nijmegen.
Sassenberg, Uta, Manja Foth, Isabell Wartenburger & Elke Van Der Meer. 2011. Show your hands — Are you really clever? Reasoning, gesture production, and intelligence. Linguistics 49(1). 841. Google Scholar logo with link to Google Scholar
Sassenberg, Uta & Elke Van Der Meer. 2010. Do we really gesture more when it is more difficult? Cognitive Science 34(4). 643–664. Google Scholar logo with link to Google Scholar
Sekine, Kazuki, Catherine Wood & Sotaro Kita. 2018. Gestural depiction of motion events in narrative increases symbolic distance with age. Language, Interaction and Acquisition 9(1). 40–68. Google Scholar logo with link to Google Scholar
Selting, Margret, Peter Auer, Dagmar Barth-Weingarten, Jörg R. Bergmann, Pia Bergmann, Karin Birkner, Elizabeth Couper-Kuhlen, Arnulf Deppermann, Peter Gilles, Susanne Günthner, Martin Hartung, Friederike Kern, Christine Mertzlufft, Christian Meyer, Miriam Morek, Frank Oberzaucher, Jörg Peters, Uta Quastthoff, Willfried Schütte, Anja Stukenbrock & Susanne Uhmann. 2009. Gesprächsanalytisches Transkriptionssystem 2 (GAT 2). Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 101. 353–402. [URL]
Smithson, Lisa & Elena Nicoladis. 2014. Lending a hand to imagery? The impact of visuospatial working memory interference upon iconic gesture production in a narrative task. Journal of Nonverbal Behavior 38(2). 247–258. Google Scholar logo with link to Google Scholar
Stec, Kashmiri. 2012. Meaningful shifts. Gesture 12(3). 327–360. Google Scholar logo with link to Google Scholar
Streeck, Jürgen. 2008. Depicting by gesture. Gesture 8(3). 280–302. Google Scholar logo with link to Google Scholar
Stites, Lauren J. & Şeyda Özçalışkan. 2017. Who did what to whom? Children track story referents first in gesture. Journal of Psycholinguistic Research 46(4). 1019–1032. Google Scholar logo with link to Google Scholar
Tellegen, Peter J., Jacob A. Laros, & Franz Petermann. 2007. SON-R 2½-7 Non-verbaler Intelligenztest. Testmanual mit deutscher Normierung und Validierung. Boston: Hogrefe.Google Scholar logo with link to Google Scholar
Trofatter, Caroline, Carly Kontra, Sian Beilock & Susan Goldin-Meadow. 2015. Gesturing has a larger impact on problem-solving than action, even when action is accompanied by words. Language, Cognition and Neuroscience 30(3). 251–260. Google Scholar logo with link to Google Scholar
Vernon, Philip A. 1983. Speed of information processing and general intelligence. Intelligence 7(1). 53–70. Google Scholar logo with link to Google Scholar
Wood, David, Jerome S. Bruner & Gail Ross. 1976. The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, and Allied Disciplines 17(2). 89–100. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue