Embodied frames and scenes
Body-based metonymy and pragmatic inferencing in gesture
Published online: 12 January 2018
https://doi.org/10.1075/gest.16.2.03mit
https://doi.org/10.1075/gest.16.2.03mit
Abstract
This paper lays out the foundations of a frame-based account of gesture pragmatics through detailing how frames and metonymy interact not only in motivating gestural sign formation, but also in guiding crossmodal processes of pragmatic inferencing. It is argued that gestures recruiting frame structures tend to profile deeply embodied, routinized aspects of scenes (in the Fillmorian sense of the term), that is, of the motivating context of frames. Two kinds of embodied frame structures situated at different levels of abstraction, schematicity, and entrechment are proposed: (A) Basic physical action and object frames understood as directly experientially grounded frames involving physical action and interaction with the material and social world; (B) Complex, highly abstract frame structures that are more detached from the motivating contexts of experience. It is further suggested that gestures exhibiting a comparably low degree of iconicity and/or indexicality are likely to assume pragmatic rather than referential functions. Finally, potential avenues for further research into the relation of scenes, frames, and (multimodal) constructions are outlined.
Keywords: frames, scenes, metonymy, metaphor, pragmatic inferencing, motivation, indexicality, multifunctionality
Article outline
- Introduction
- Frames, scenes, and metonymy: Seizing the pragmatically minded nature of gesture
- Hands framing New York Harbor from Pier 18
- Embodied frames and scenes: Theoretical premises and some tendencies in their gestural evocation
- Basic and more complex frame structures: A first characterization
- Scenes and their relevance for gesture
-
Towards a frame-based account of gesture pragmatics
- Basic physical action and object frames
- Complex, highly abstract frame structures
- Summary of observations and proposals
- Body-based metonymy and gesture pragmatics
- Cognitive/experiential/functional DOMAINS
- FRAMES and frame metonymy
- Contiguity relations operationalized in gesture
- Crossmodal pragmatic inferencing and bodily reference points
- From reference to pragmatic inferencing in multimodal discourse
- Muted degrees of iconicity/indexicality and resulting pragmatic functions
- Conclusion and outlook
- Acknowledgements
- Notes
References
References (124)
Andrén, Mats (2010). Children’s gestures from 18 to 30 months. Lund: Centre for Languages and Literatures, Lund University.
(2014). Multimodal constructions in children: Is the headshake part of language? Gesture, 14 (2), 141–170.
Barcelona, Antonio (2003). The case of a metonymic basis of pragmatic inferencing: Evidence from jokes and funny anecdotes. In Klaus-Uwe Panther & Linda Thornburg (Eds.), Metonymy and pragmatic inferencing (pp. 81–102). Amsterdam & Philadelphia: John Benjamins.
(2009). Motivation of construction meaning and form: The roles of metonymy and inference. In Klaus-Uwe Panther, Linda L. Thornburg, & Antonio Barcelona (Eds.), Metonymy and metaphor in grammar (pp. 363–401). Amsterdam & Philadelphia: John Benjamins.
Bavelas, Janet, Nicole Chovil, Douglas Lawrie, & Allan Wade (1992). Interactive gestures. Discourse Processes, 151, 469–489.
Bavelas, Janet, Nicole Chovil, Jan L. Coates, & Lori Roe (1995). Gestures specialized for dialogue. Personality and Social Psychology Bulletin, 211, 394–405.
Benazzo, Sandra & Aliyah Morgenstern (2014). A bilingual child’s multimodal path into negation. Gesture, 14 (2), 171–202.
Benczes, Reka, Antonio Barcelona, & Francisco José Ruiz de Mendoza Ibáñez (Eds.) (2011). Defining metonymy in cognitive linguistics. Amsterdam & Philadelphia: John Benjamins.
Bressem, Jana (2012). Repetitions in gesture: Structures, functions, and cognitive aspects. Ph.D. dissertation, European University Viadrina, Frankfurt (Oder).
Bressem, Jana & Cornelia Müller (2014). The family of AWAY-gestures. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Volume 21 (pp. 1592–1604). Berlin & Boston: De Gruyter Mouton.
(2003). From cutting an object to a clear cut analysis. Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture, 3 (1), 19–46.
(2011). Elements of meaning in gesture. Amsterdam & Philadelphia: John Benjamins.
Cienki, Alan (1998). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In John-Pierre Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford: CSLI Publications.
(2013a). Cognitive Linguistics: Spoken language and gesture as expressions of conceptualization. In Cornelia Müller, Alan Cienki, Elan Fricke, Silvia H. Ladewig, David McNeill, & Sedinha Tessendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Volume 11 (pp. 182–201). Berlin & Boston: Mouton de Gruyter.
(2013b). Mimetic schemas and image schemas in cognitive linguistics and gesture studies. Review of Cognitive Linguistics, 11 (2), 417–432.
Cienki, Alan & Cornelia Müller (Eds.) (2008). Metaphor and gesture. Amsterdam & Philadelphia: John Benjamins.
(2003): Pointing and placing. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 243–268). Mahwah, N.J. & London: Lawrence Erlbaum Associates.
Coulson, Seana (2001). Semantic leaps: Frame-shifting and conceptual blending in meaning construction. Cambridge: Cambridge University Press.
Croft, William (1993). The role of domains in the interpretation of metaphor and metonymy. Cognitive Linguistics, 41, 335–370.
Croft, William & Alan A. Cruse (2004). Cognitive linguistics. Cambridge: Cambridge University Press.
Dancygier, Barbara & Eve E. Sweetser (2005). Mental spaces in grammar: Conditional constructions. Cambridge: Cambridge University Press.
Debras, Camille (2014). L’Expression multimodale du positionnement interactionnel (multimodal stance-taking). Unpublished doctoral dissertation, Université Sorbonne Nouvelle – Paris 3.
Dirven, René & Ralf Pörings (Eds.) (2002). Metaphor and metonymy in comparison and contrast. Berlin & New York: Mouton de Gruyter.
Enfield, Nick J. (2011). Elements of formulation. In Jürgen Streeck, Charles Goodwin, & Curtis LeBaron (Eds.), Embodied interaction: Language and the body in the material world (pp. 59–66). Cambridge: Cambridge University Press.
Feyaerts, Kurt (1999). Metonymic hierarchies. The conceptualization of stupidity in German idiomatic expressions. In: Klaus-Uwe Panther & Günther Radden (Eds.), Metonymy in language and thought (pp. 309–332). Amsterdam & Philadelphia: Benjamins.
Fillmore, Charles J. (1975). An alternative to checklist theories of meaning. In Cathy Cogen, Henry Thompson, Graham Thurgood, Kenneth Whistler, & James Wright (Eds.), Proceedings of the first annual meeting of the Berkeley Linguistics Society (pp. 123–131). Berkeley, CA: Berkeley Linguistics Society.
(1977). Scenes-and-frames semantics. In Antonio Zampolli (Ed.), Linguistic structures processing, Volume 41 (pp. 55–81). Amsterdam & New York: North Holland.
(1982). Frame semantics. In Linguistic Society of Korea (Ed.), Linguistics in the morning calm (pp. 111–137). Seoul: Hanshin.
Fillmore, Charles J. & Collin F. Baker (2009). A frames approach to semantic analysis. In Bernd Heine & Heiko Narrog (Eds.), The Oxford handbook of linguistic analysis (pp. 313–340). Oxford: Oxford University Press.
Fricke, Ellen (2012). Grammatik multimodal: Wie Wörter und Gesten zusammenwirken. Berlin & New York: Mouton de Gruyter.
Gibbs, Jr., Raymond W. (1994). The poetics of mind: Figurative thought, language, and understanding. Cambridge: Cambridge University Press.
(1999). Speaking and thinking with metonymy. In Günter Radden & Zoltan Kövecses (Eds.) Metonymy in language and thought (pp. 61–76). Amsterdam & Philadelphia: John Benjamins.
(2017). The embodied and discourse view of metaphor: Why these are not so different and how they can be brought closer together. In Beate Hampe (Ed.), Metaphor: Embodied cognition and discourse (pp. 319–334). Cambridge: Cambridge University Press.
Goldberg, Adele (1995). Constructions: A construction grammar approach to argument structure. Chicago: The University of Chicago Press.
Goodwin, Charles (2007). Environmentally coupled gestures. In Susan Duncan, Justine Cassell, & Elena Levy (Eds.), Gesture and the dynamic dimensions of language (pp. 195–212). Amsterdam & Philadelphia: John Benjamins.
(2011). Contextures of action. In Jürgen Streeck, Charles Goodwin, & Curtis LeBaron (Eds.), Embodied interaction: Language and body in the material world (pp. 182–193). Cambridge: Cambridge University Press.
Grady, Joe (1997). Foundations of meaning: primary metaphors and primary scenes. Doctoral dissertation, University of California at Berkeley.
Grady, Joe & Christopher Johnson (2002). Converging evidence for the notions of subscene and primary scene. In René Dirven & Ralph Pörings (Eds.), Metaphor and metonymy in comparison and contrast (pp. 533–554). Berlin: Mouton de Gruyter.
Grandhi, Sukeshini A., Gina Joue, & Irene Mittelberg (2011). Understanding naturalness and intuitiveness in gesture production: Insights for touchless gestural interfaces. Proceedings of the ACM2011 conference on Human Factors in Computing Systems (CHI), Vancouver, B.C.
Hassemer, Julius (2016). Towards a theory of Gesture Form Analysis: Imaginary forms as part of gesture conceptualisation with empirical support from motion‐capture data. Doctoral dissertation, RWTH Aachen University.
Hopper, Paul & Sandra Thompson (1980). Transitivity in grammar and discourse. Language, 561, 251–299.
Hostetter, Autumn. B., & Martha W. Alibali (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15 (3), 495–514.
Jakobson, Roman (1956/1990). Two aspects of language and two types of aphasic disturbances. In Linda R. Waugh & Monique Monville-Burston (Eds.), Roman Jakobson, On language (pp. 115–133). Cambridge, MA: Harvard University Press.
(1960/1987). Linguistics and poetics. In Krystyna Pomorska & Stephen Rudy (Eds.), Roman Jakobson – Language in literature (pp. 62–94). Cambridge, MA: Belknap of Harvard University Press.
Janzen, Terry & Barbara Shaffer (2002). Gesture as substrate in the process of ASL grammaticalization. In Richard P. Meier, Kearsy Cormier, & David Quintos-Pozos (Eds.), Modality and structure in signed and spoken language (pp. 199–223). Cambridge: Cambridge University Press.
Johnson, Mark (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press.
(2005). The philosophical significance of image schemas. In Beate Hampe (Ed.), From perception to meaning: Image schemas in cognitive linguistics (pp. 15–33). Berlin: Mouton de Gruyter.
(2007). The meaning of the body: Aesthetics of human understanding. Chicago: The University of Chicago Press.).
Kendon, Adam (1995). Gestures as illocutionary and discourse structure markers in southern Italian conversation. Journal of Pragmatics, 231, 247–279.
(2008). Some reflections on the relationship between ‘gesture’ and ‘sign’. Gesture, 8 (3), 348–366.
(2017). Pragmatic functions of gestures: Some observations on the history of their study and their nature. Gesture, 16 (2), 157–175 (
this issue
).
Labov, William (1972). The transformation of reality in narrative syntax. In William Labov, Language in the inner city (pp. 354–396). Philadelphia: The University of Pennsylvania Press.
Ladewig, Silva H. (2014). Recurrent gestures. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction, Volume 21 (pp. 1558–1575), Berlin & Boston: Mouton de Gruyter.
Lakoff, George (1987). Women, fire, and dangerous things: What categories reveal about the mind. Chicago: The University of Chicago Press.
Langacker, Ronald W. (1987). Foundations of cognitive grammar: Theoretical prerequisites, Volume 11. Stanford: Stanford University Press.
(2009). Metonymic grammar. In Klaus-Uwe Panther, Linda L. Thornburg, Antonio Barcelona (Eds.), Metonymy and metaphor in grammar (pp. 45–71). Amsterdam & Philadelphia: John Benjamins.
Littlemore, Jeanette (2015). Metonymy: Hidden shortcuts in language, thought and communication. Cambridge: Cambridge University Press.
McNeill, David (1992). Hand and mind: What gestures reveal about thought. Chicago: The University of Chicago Press.
Mittelberg, Irene (2006). Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar (Doctoral dissertation, Cornell University). Ann Arbor, MI: UMI.
(2008). Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 115–154). Amsterdam & Philadelphia: John Benjamins.
(2010a). Geometric and image-schematic patterns in gesture space. In Vyvyan Evans & Paul Chilton (Eds.), Language, cognition, and space: The state of the art and new directions (pp. 351–385). London: Equinox.
(2010b). Interne und externe Metonymie: Jakobsonsche Kontiguitäts-beziehungen in redebegleitenden Gesten. Sprache und Literatur, 41 (1), 112–143.
(2013a). The exbodied mind: Cognitive-semiotic principles as motivating forces in gesture. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Sedinha Tessendorf (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 750–779). Berlin & Boston: Mouton de Gruyter.
(2013b). Balancing acts: Image schemas and force dynamics as experiential essence in pictures by Paul Klee and their gestural enactments. In Michael Bokrent, Barbara Dancygier, & Jennifer Hinnell (Eds.), Language and the creative mind (pp. 325–346). Stanford: CSLI.
(2014). Gestures and iconicity. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 1712–1732). Berlin & Boston: Mouton de Gruyter.
(2017). Multimodal existential constructions in German: Manual actions as experiential substrate for grammatical and gestural patterns. Linguistics Vanguard, 3, s1. .
Mittelberg, Irene & Linda R. Waugh (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In Charles Forceville & Eduardo Urios-Aparisi (Eds.), Multimodal metaphor (pp. 329–356). Berlin & New York: Mouton de Gruyter.
(2014). Gestures and metonymy. In: Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (pp. 1747–1766). Berlin & Boston: Mouton de Gruyter.
Mittelberg, Irene & Gina Joue (2017). Source actions ground metaphor via metonymy: Toward a frame-based account of gestural action in multimodal discourse. In Beate Hampe (Ed.), Metaphor: Embodied cognition and discourse (pp. 119–137). Cambridge: Cambridge University Press.
Müller, Cornelia (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Spitz Verlag.
(2004). Forms and uses of the palm up open hand: A case of a gesture family? In Cornelia Müller & Roland Posner (Eds.), The semantics and pragmatics of everyday gesture: The Berlin conference (pp. 233–256). Berlin: Weidler Verlag.
(2017). How recurrent gestures mean: Conventionalized contexts-of-use and embodied motivation. Gesture, 16 (2), 277–304 (
this issue
).
Nerlich, Brigitte & David D. Clarke (2001). Ambiguities we live by: Towards a pragmatics of polysemy. Journal of Pragmatics, 33 (1), 1–20.
Núñez, Rafael E. (2008). A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 93–114). Amsterdam & Philadelphia: John Benjamins.
Panther, Klaus-Uwe & Günter Radden (Eds.) (1999). Metonymy in language and thought. Amsterdam & Philadelphia: John Benjamins.
Panther, Klaus-Uwe & Linda L. Thornburg (Eds.) (2003). Metonymy and pragmatic inferencing. Amsterdam & Philadelphia: John Benjamins.
Panther, Klaus-Uwe, Linda L. Thornburg, & Antonio Barcelona (Eds.) (2009). Metonymy and metaphor in grammar. Amsterdam & Philadelphia: John Benjamins Publishing.
Parrill, Fey (2009). Dual viewpoint in gesture. Gesture, 9 (3), 271–289.
Parrill, Fey & Eve E. Sweetser (2004). What we mean by meaning: Conceptual integration in gesture analysis and transcription. Gesture, 4 (2), 197–219.
Peirce, Charles S. (1960). Collected papers of Charles Sanders Peirce (1931–1958), Volume I: Principles of philosophy; Volume II: Elements of logic. Charles Hartshorne & Paul Weiss (Eds.). Cambridge, MA: The Belknap of Harvard University Press.
Pfau, Roland & Markus Steinbach (2004). On grammaticalization: Do sign languages follow the well-trodden paths? Proceedings of the 8th Theoretical Issues in Sign Language Research (TISLR 8) Conference, Barcelona.
Peirsman, Yves & Dirk Geeraerts (2006). Metonymy as a prototypical category. Cognitive Linguistics, 17 (3), 269–316.
Priesters, Matthias A. (2012). Functional patterns in gesture space. Correlating location and function of gestures using 3D motion capture technology. Unpublished M.A. thesis, RWTH Aachen University.
Rizzolazzi, Giacomo & Michael A. Arbib (1998). Language within our grasp. Trends in Neurosciences, 21 (5), 188–194.
Schank, Robert. C. & Robert Abelson (1977). Scripts, goals, plans, and understanding. Hillsdale, NJ: Lawrence Erlbaum.
Schoonjans, Steven (2014). Modalpartikeln als multimodale Konstruktionen. Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. Unpublished doctoral dissertation, KU Leuven.
Schoonjans, Steven, Paul Sambre, Geert Brône, & Kurt Feyaerts (2016). Vers une analyse multimodale du sens. Perspectives constructionnelles sur la gestualité co-grammaticale. Langages, 2011, 33–50.
Slobin, Dan (1985). Crosslinguistic evidence for the language-making capacity. In Dan Slobin (Ed.), A crosslinguistic study of language acquisition. Volume 2: Theoretical issues (pp. 1157–1256). Hillsdale, NJ: Lawrence Erlbaum.
Sperber, Dan & Deirdre Wilson (1986/1995). Relevance: Communication and cognition (2nd Ed.), Oxford: Blackwell.
Steen, Francis & Mark Turrier (2013). Multimodal construction grammar. In Mike Borkent, Barbara Dancygier, & Jennifer Hinnell (Eds.), Language and the creative mind (pp. 255–274), Stanford: CSLI.
Streeck, Jürgen (2009). Gesturecraft: The Manu-facture of meaning. Amsterdam & Philadelphia: John Benjamins.
(2013). Praxeology of gesture, In Cornelia Müller, Alan Cienki, Ellen Fricke, Silvia H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication. An international handbook of multimodality in human interaction, Volume 11 (pp. 674–688). Berlin & Boston: Mouton de Gruyter.
Streeck, Jürgen, Charles Goodwin, & Curtis LeBaron (Eds.) (2011). Embodied interaction: Language and the body in the material world. Cambridge: Cambridge University Press.
Sweetser, Eve E. (1990). From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure. Cambridge: Cambridge University Press.
(1998). Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes (CD-ROM), Elsevier.
(2007). Looking at space to study mental spaces: Co-speech gesture as a crucial data source in cognitive linguistics. In Monica Gonzalez-Marquez, Irene Mittelberg, Seana Coulson, & Michael Spivey (Eds.), Methods in cognitive linguistics (pp. 201–224). Amsterdam & Philadelphia: John Benjamins.
(2012). Viewpoint and perspective in language and gesture: From the ground down. In Barbara Dancygier & Eve E. Sweetser (Eds.), Viewpoint in language: A multimodal perspective (pp. 1–24). Cambridge: Cambridge University Press.
Sullivan, Karen (2013). Frames and constructions in metaphoric language. Amsterdam & Philadelphia: John Benjamins.
Tabacaru, Sabina (2014). Humorous implications and meanings: A multimodal study of sarcasm in interactional humor. Unpublished doctoral dissertation, Université Charles de Gaulle – Lille 3.
(2013). Gestures as cues to a target. Manuscript of talk presented at the ICLC 12, University of Edmonton, Alberta, June 24, 2013.
Waugh, Linda R. & Monique Monville-Burston (1990). Roman Jakobson: His life, work and influence. Introduction to: Linda R. Waugh & Monique Monville Burston (Eds.), Jakobson on language (pp. 1–45). Cambridge: Belknap of Harvard University Press.
Wehling, Elisabeth (2010). Argument is gesture war: Function, form and prosody of discourse structuring gestures in political argument. Proceedings of the 35th Annual Meeting of the Berkeley Linguistics Society (pp. 54–65). Berkeley, CA: Berkeley Linguistics Society.
(2017). Discourse management gestures. Gesture, 16 (2), 245–276 (
this issue
).
Wilcox, Phyllis P. (2004). A cognitive key: Metonymic and metaphorical mappings in ASL. Cognitive Linguistics, 15 (2), 197–222.
Wilcox, Sherman (2004). Gesture and language: Cross-linguistic and historical data from signed languages. Gesture, 41, 43–73.
Ziem, Alexander (2014). Frames of understanding in text and discourse. Amsterdam & Philadelphia: John Benjamins.
Zima, Elisabeth (2014). Gibt es multimodale Konstruktionen? Eine Studie zu [V(motion) in circles] und [all the way from X PREP Y], Gesprächsforschung, 151, 1–48.
Cited by (13)
Cited by 13 other publications
Fricke, Ellen
Dyrmo, Tomasz
Dyrmo, Tomasz
Dyrmo, Tomasz
Meissl, Katharina, Paul Sambre & Kurt Feyaerts
Schmitt, Rudolf
Gawne, Lauren, Chelsea Krajcik, Helene N. Andreassen, Andrea L. Berez-Kroeker & Barbara F. Kelly
Lopez-Ozieblo, Renia
Mittelberg, Irene
Mittelberg, Irene
Mittelberg, Irene
This list is based on CrossRef data as of 9 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.
