Cover not available

Article published In: Review of Cognitive Linguistics
Vol. 18:2 (2020) ► pp.289–315

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Gesturing in the wild

Evidence for a flexible mental timeline

 | University of Murcia
 | University of Murcia
 | University of Navarra
 | University of Murcia
Published online: 4 December 2020
DOI logo with link
https://doi.org/10.1075/rcl.00061.val
Abstract
Psycholinguistic evidence shows that spatial domains are automatically activated when processing temporal expressions. Speakers conceptualize time as a straight line deployed along different axes (mostly sagittal, though also vertical). The use of the lateral axis, which cannot be lexicalized in any language, has nonetheless been attested in temporal tasks in laboratories using a variety of experiments. This leads to the question of what axes are actually at work when conceptualizating time in oral communication.
The present study examines a great number of temporal expressions, taken from television shows, noting their associated co-speech gestures. Our results show that (1) speakers overwhelmingly use the lateral axis; (2) they are not performing simple space-to-time mappings, but are using instead a “timeline”, a material anchor which is a far more complex construct and that can explain some of the intricacies and contextual variations shown in the pattern of results.
Article outline
  • 1.Introduction
    • 1.1The lateral axis
    • 1.2Multimodality and the NewsScape Library of Television News
  • 2.Gesturing the timeline: A multimodal study
    • 2.1Research questions and hypotheses
    • 2.2Materials and methodology
      • 2.2.1Data collection
      • 2.2.2Filtering
      • 2.2.3Coding
  • 3.Results
    • 3.1Construal
    • 3.2Axis
    • 3.3Directionality
  • 4.Discussion
    • 4.1The lateral timeline: New evidence from spontaneous speech
    • 4.2Flexibility
  • 5.Conclusion
  • References
References (77)
References
Alibali, M. W., Bassok, M., Solomon, K. D., Syc, S. E., & Goldin-Meadow, S. (1999). Illuminating mental representations through speech and gesture. Psychological Science, 101, 327–333. Google Scholar logo with link to Google Scholar
Baldwin, D. A. (2002). The rise of intentional understanding: Analogies to the ontogenesis of language. In T. Givon & B. Malle (Eds.), The evolution of language out of prelanguage (pp. 285–305). Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Barrett, L. (2015). Beyond the brain: how body and environment shape animal and human minds. Princeton, NJ: Princeton University Press.Google Scholar logo with link to Google Scholar
Boroditsky, L., & Gaby, A. (2010). Remembrances of times east: Absolute spatial representations of time in an aboriginal Australian community. Psychological Science, 211, 1635–1639. 10.1177/0956797610386621. Google Scholar logo with link to Google Scholar
Boroditsky, L., Fuhrman, O., & McCormick, K. (2011). Do English and Mandarin speakers think about time differently? Cognition, 118(1), 126–132. Google Scholar logo with link to Google Scholar
Bressem, J. (2013). A linguistic perspective on the notation of form features in gestures. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body – language – communication. An international handbook on multimodality in human interaction (pp. 1079–1098). Berlin & New York: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Brewer, M. (2000). Research design and issues of validity. In H. Reis & C. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 11–26). Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Calbris, G. (1990). The semiotics of French gesture. Bloomington & Indianapolis: Indiana University Press.Google Scholar logo with link to Google Scholar
(2011). Elements of meaning in gesture. Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Casasanto, D. (2009). When is a linguistic metaphor a conceptual metaphor? In V. Evans & S. Pourcel (Eds.), New directions in Cognitive Linguistics (pp. 127–145). Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Casasanto, D., & Boroditsky, L. (2008). Time in the mind: using space to think about time. Cognition, 1061, 579–93. Google Scholar logo with link to Google Scholar
Casasanto, D., & Bottini, R. (2014). Mirror-reading can reverse the flow of time. Journal of Experimental Psychology: General, 143(2), 473–479. Google Scholar logo with link to Google Scholar
Casasanto, D., & Jasmin, K. (2012). The hands of time: Temporal gestures in English speakers. Cognitive Linguistics, 23(4), 643–674. Google Scholar logo with link to Google Scholar
Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J. -P. Koenig (Ed.), Discourse and cognition: bridging the gap (pp. 189–204). Stanford, CA: CSLI Publications.Google Scholar logo with link to Google Scholar
Cienki, A., & Müller, C. (2008). Metaphor, gesture and thought. In R. W. Gibbs (Ed), Cambridge handbook of metaphor and thought (pp. 483–501). Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Clark, A. (2008). Supersizing the mind: Embodiment, action and cognitive extension. Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
Clark, H. H. (1996). Using language. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Coulson, S., & Pagán-Cánovas, C. (2013). Understanding timelines: Conceptual metaphor and conceptual integration. Cognitive Semiotics, 5(1–2), 198–219. Google Scholar logo with link to Google Scholar
Dancygier, B. (2012). The language of stories. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Dancygier, B., & Sweetser, E. (Eds.). (2012). Viewpoint in language: A multimodal perspective. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Dirven, R., & Parings, R. (Eds.). (2002). Metaphor and metonymy in comparison and contrast. Berlin & New York: Mouton de Gruyter.Google Scholar logo with link to Google Scholar
Engberg-Pedersen, E. (1993). Space in Danish sign language: The semantics and morphosyntax of the use of space in a visual language. Hamburg: SIGNUM-Verlag.Google Scholar logo with link to Google Scholar
(1999). Space and time. In J. Allwood & P. Gardenfors (Eds.), Cognitive semantics: Meaning and cognition (pp. 131–152). Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Emmorey, K. (2001). Space on hand: The exploitation of signing space to illustrate abstract thought. In M. Gattis (Ed.), Spatial schemas and abstract thought (pp. 147–174). Cambridge: The MIT Press.Google Scholar logo with link to Google Scholar
Fauconnier, G. (1997). Mappings in thought and language. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Fauconnier, G., & Turner, M. (1998). Conceptual integration networks. Cognitive Science, 22(2), 133–187. Google Scholar logo with link to Google Scholar
(2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books.Google Scholar logo with link to Google Scholar
Fillmore, C. J., Kay, P., & O’Connor, M. (1988). Regularity and idiomaticity in grammatical constructions: The case of let alone. Language, 64(3), 501–538. Google Scholar logo with link to Google Scholar
Fuhrman, O., & Boroditsky, L. (2010). Cross-cultural differences in mental representations of time: Evidence from an implicit nonlinguistic task. Cognitive Science, 341, 1430–1451. Google Scholar logo with link to Google Scholar
Goldberg, A. (1995). Constructions: A Construction Grammar approach to argument structure. Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
(2006). Constructions at work. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Goldin-Meadow, S., & Alibali, M. W. (2013). Gesture’s role in speaking, learning, and creating language. Annual Review of Psychology, 1231, 448–453. Google Scholar logo with link to Google Scholar
Goldin-Meadow, S. (2003). Gesture. Encyclopedia of cognitive science, volume 21 (pp. 289–293). London: Macmillan Reference Ltd.Google Scholar logo with link to Google Scholar
Hutchins, E. (2005). Material anchors for conceptual blends. Journal of Pragmatics, 371, 1555–1577. Google Scholar logo with link to Google Scholar
Ishihara, M., Keller, P. E., Rossetti, Y. & Prinz, W. (2008). Horizontal spatial representations of time: Evidence for the STEARC effect. Cortex, 44(4), 454–461. Google Scholar logo with link to Google Scholar
Jewitt, C. (Ed.). (2009). The Routledge handbook of multimodal analysis. London: Routledge.Google Scholar logo with link to Google Scholar
Kendon, A. (1995). Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of Pragmatics, 23(3), 247–279. Google Scholar logo with link to Google Scholar
(2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Kita, S. (2009). Cross-cultural variation of speech-accompanying gesture: a review. Language and Cognitive Processes, 24(2), 145–167. Google Scholar logo with link to Google Scholar
Kövecses, Z. (2005). Metaphor in culture. Universality and variation. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (pp. 202–251). Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
(1999). Philosophy in the flesh. New York: Basic Books.Google Scholar logo with link to Google Scholar
Le Guen, O., & Pool Balam, L. I. (2012). No metaphorical timeline in gesture and cognition among Yucatec Mayas. Frontiers in Cultural Psychology, 31, Article 271. Google Scholar logo with link to Google Scholar
Leonard, T., & Cummins, F. (2011). The temporal relation between beat gestures and speech. Language and Cognitive Processes, 26(10), 1457–1471. Google Scholar logo with link to Google Scholar
Levinson, S. C. (1996). Frames of reference and Molyneux’s question: Cross-linguistic evidence. In P. Bloom, M. Peterson, L. Nadel & M. Garrett (Eds.), Language and space (pp. 109–169). Cambridge, MA: MIT Press.Google Scholar logo with link to Google Scholar
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
(2005). Gesture and thought. Chicago: University of Chicago Press. Google Scholar logo with link to Google Scholar
(2013). Gesture as a window onto mind and brain, and the relationship to linguistic creativity and ontogenesis. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body – language – communication. An international handbook on multimodality in human interaction. Volume I (pp. 28–54). Berlin & New York: Mouton de Gruyter.Google Scholar logo with link to Google Scholar
Müller, C., Cienki, A., Fricke, E., Ladewig, S. H., McNeill, D., & Bressem, J. (Eds.). (2013). Body – language – communication. An international handbook on multimodality in human interaction. Volume II. Berlin & New York: Mouton de Gruyter.Google Scholar logo with link to Google Scholar
(Eds.). (2014). Body – language – communication. An international handbook on multimodality in human interaction. Volume II. Berlin & New York: Mouton de Gruyter.Google Scholar logo with link to Google Scholar
Núñez, R., & Sweetser, E. (2006). With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 301, 401–450. Google Scholar logo with link to Google Scholar
Ouellet, M., Santiago, J., Israeli, Z., & Gabay, S. (2010). Is the future the right time? Experimental Psychology, 571, 308–314. Google Scholar logo with link to Google Scholar
Pagán Cánovas, C., Valenzuela, J., & Santiago, J. (2015). Like the machete the snake: Integration of topic and vehicle in poetry comprehension reveals meaning construction processes. Psychology of Aesthetics, Creativity, and the Arts, 9(4), 385–393. Google Scholar logo with link to Google Scholar
Pagán-Cánovas, C., & Valenzuela, J. (2017). Timelines and multimodal constructions: facing new challenges. Linguistics Vanguard, 3(1). Google Scholar logo with link to Google Scholar
Pagán-Cánovas, C., Valenzuela, J., Alcaraz-Carrión, D., Olza, I., & Ramscar, M. (2020). Quantifying the speech-gesture relation with massive multimodal datasets: Informativity in time expressions. PLoS ONE, 15(6), e0233892. Google Scholar logo with link to Google Scholar
Radden, G. (2004). The metaphor TIME AS SPACE across languages. In N. Baumgarten, C. Böttger, M. Motz & J. Probst (Eds.), Übersetzen, interkulturelle Kommunikation, Spracherwerb und Sprachvermittlung – Das Leben mit Mehreren Sprachen: Festschrift für Juliane House zum 60 Geburtstag (pp. 225–238). Bochum, Germany: Aks-Verlag.Google Scholar logo with link to Google Scholar
Richter, N. (2014). Structuring discourse: Observations on prosody and gesture in Russian TV-discourse. In C. Müller, A. J. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Teßendorf (Eds.), Body – language – communication. An international handbook on multimodality in human interaction. Volume II (pp. 1392–1400). Berlin & New York: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Santiago, J., Lupiáñez, J., Perez, E., & Funes, M. J. (2007). Time (also) flies from left to right. Psychonomic Bulletin & Review, 14(3), 512–516. Google Scholar logo with link to Google Scholar
Santiago, J., Román, A., & Ouellet, M. (2011). Flexible foundations of abstract thought: A review and a theory. Spatial Dimensions of Social Thought, 41–110. Google Scholar logo with link to Google Scholar
Scott, A. (1989). The vertical dimension and time in Mandarin. Australian Journal of Linguistics, 91, 295–331. Google Scholar logo with link to Google Scholar
Sell, A. J., & Kaschak, M. P. (2011). Processing time shifts affects the execution of motor responses. Brain & Language, 1171, 39–44. Google Scholar logo with link to Google Scholar
Sinha, C., Silva Sinha, V. da, Zinken, J., & Sampaio, W. (2011). When time is not space: The social and linguistic construction of time intervals and temporal event relations in an Amazonian culture. Language and Cognition, 3(1), 137–169. Google Scholar logo with link to Google Scholar
Steen, F., & Turner, M. (2013). Multimodal Construction Grammar. In M. Borkent, B. Dancygier, & J. Hinnell (Eds.), Language and the creative mind (pp. 255–274). Stanford: CSLI Publications.Google Scholar logo with link to Google Scholar
Stockwell, P. (1999). The inflexibility of invariance. Language and Literature, 8(2), 125–142. Google Scholar logo with link to Google Scholar
Torralbo, A., Santiago, J., & Lupiáñez, J. (2006). Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science, 30(4), 745–757. Google Scholar logo with link to Google Scholar
Turner, M. (1996). The literary mind. New York: Oxford University Press US.Google Scholar logo with link to Google Scholar
(2014). The origin of ideas: blending, creativity, and the human spark. New York: Oxford University Press US.Google Scholar logo with link to Google Scholar
Ulrich, R., Eikmeier, V., Vega, I., Ruiz Fernández, S., Alex-Ruf, S., & Maienborn, C. (2012). With the past behind and the future ahead: Back-to-front representation of past and future sentences. Memory & Cognition, 40(3), 483–495. Google Scholar logo with link to Google Scholar
Ulrich, R., & Maienborn, C. (2010). Left–right coding of past and future in language: The mental timeline during sentence processing. Cognition, 1171, 126–138. Google Scholar logo with link to Google Scholar
Valenzuela, J., Hilferty, J., & Garachana, M. (2005). On the reality of grammatical constructions. Annual Review of Cognitive Linguistics, 31, 201–215. Google Scholar logo with link to Google Scholar
Walker, E., & Cooperrider, K. (2015). The continuity of metaphor: evidence from temporal gestures. Cognitive Science, 40(2), 481–495. Google Scholar logo with link to Google Scholar
Weger, U. W., & Jay, P. (2008). Time flies like an arrow: Space–time compatibility effects suggest the use of a mental timeline. Psychonomic Bulletin and Review, 15(2), 426–430. Google Scholar logo with link to Google Scholar
Wang, L., & Chu, M. (2013). The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia, 51(13), 2847–55. Google Scholar logo with link to Google Scholar
Zhang, J., & Patel, V. L. (2006). Distributed cognition, representation, and affordance. Pragmatics & Cognition, 14(2), 333–341. Google Scholar logo with link to Google Scholar
Zima, E. (2014). Gibt es multimodale Konstruktionen? Eine Studie zu [V(motion) in circles] und [all the way from X PREP Y]. Gesprächsforschung – Online-Zeitschrift zur verbalen Interaktion, 151, 1–48.Google Scholar logo with link to Google Scholar
Cited by (19)

Cited by 19 other publications

Stolpe, Karin, Marlene Johansson Falck & Andreas Larsson
2025. Spatial constructions of time: exploring Co-speech gestures in lectures on programming. Cognitive Semiotics 18:1 ► pp. 73 ff. DOI logo
Wang, Renqiang, Heng Li & Bo Yang
2025. The physics of time. Review of Cognitive Linguistics 23:1 ► pp. 131 ff. DOI logo
Zima, Elisabeth
2025. Multimodal Construction Grammar, DOI logo
Hart, Christopher
2024. What’s the point of Donald Trump? Deictic gestures in the service of right-wing populism. Social Semiotics ► pp. 1 ff. DOI logo
Hart, Christopher
2024. Multimodal meaning making in news communication about immigration: using the NewsScape corpus to explore co-verbal images in TV news. Frontiers in Communication 9 DOI logo
Wilson, Anna, Irina Pavlova, Elinor Payne, Ilya Burenko & Peter Uhrig
2024. World futures through RT’s eyes: multimodal dataset and interdisciplinary methodology. Frontiers in Communication 9 DOI logo
Zhang, Ke, Xin Tong, Shaofeng Yang, Ying Hu, Qihan Zhang & Xuejun Bai
2024. Space-time mapping relationships in sensorimotor communication during asymmetric joint action. PeerJ 12 ► pp. e16764 ff. DOI logo
Dyrmo, Tomasz
2023. Levels of metaphor in gesture. Pragmatics & Cognition 30:2 ► pp. 330 ff. DOI logo
Dyrmo, Tomasz
2025. Extending extended conceptual metaphor theory: rethinking levels, modalities, and meaning-making. Cognitive Semiotics 18:1 ► pp. 23 ff. DOI logo
Peng, Zhibin & Muhammad Afzaal
2023. Review of Ladewig (2020): Integrating gestures: The dimension of multimodality in Cognitive Grammar. Review of Cognitive Linguistics 21:1 ► pp. 317 ff. DOI logo
Alcaraz Carrión, Daniel
2022. Physical and imaginary landmarks in English time gestures. In Time Representations in the Perspective of Human Creativity [Human Cognitive Processing, 75], ► pp. 187 ff. DOI logo
Cooperrider, Kensy, James Slotta & Rafael Núñez
2022. The ups and downs of space and time: topography in Yupno language, culture, and cognition. Language and Cognition 14:1 ► pp. 131 ff. DOI logo
Sun, Ying, Yan Zhang, Ying Fang & Wenxing Yang
2022. Representations of Diagonal Timelines in English and Mandarin Speakers. European Journal of Psychology Open 81:3 ► pp. 89 ff. DOI logo
Uhrig, Peter
2022. Hand gestures with verbs of throwing: Collostructions, style and metaphor. Yearbook of the German Cognitive Linguistics Association 10:1 ► pp. 99 ff. DOI logo
Yang, Wenxing, Jiaqi Dong, Ruidan Bi, Jian Gu & Xueqin Feng
2022. Cross-cultural differences in mental representations of diagonal time lines. Review of Cognitive Linguistics 20:2 ► pp. 357 ff. DOI logo
Alcaraz Carrión, Daniel & Javier Valenzuela
2021. Distant time, distant gesture: speech and gesture correlate to express temporal distance. Semiotica 2021:241 ► pp. 159 ff. DOI logo
Alcaraz Carrión, Daniel & Javier Valenzuela
2022. Time as space vs. time as quantity in Spanish: a co-speech gesture study. Language and Cognition 14:1 ► pp. 1 ff. DOI logo
Alcaraz Carrión, Daniel, Cristóbal Pagán Cánovas & Javier Valenzuela
2020. Enaction Through Co-speech Gesture: The Rhetorical Handing of the Mental Timeline. Zeitschrift für Anglistik und Amerikanistik 68:4 ► pp. 411 ff. DOI logo
Valenzuela, Javier & Daniel Alcaraz Carrión
2020. Temporal Expressions in English and Spanish: Influence of Typology and Metaphorical Construal. Frontiers in Psychology 11 DOI logo

This list is based on CrossRef data as of 30 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue