Cover not available

Article published In: Gesture
Vol. 17:3 (2018) ► pp.417–463

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

The multidimensionality of pointing

 | Humboldt-Universität zu Berlin, Berlin School of Mind and Brain
 | Universidade de São Paulo
Published online: 18 February 2020
DOI logo with link
https://doi.org/10.1075/gest.17018.has
Abstract
This paper proposes a novel analysis of deictic gestures which yields a taxonomy of manual pointing. ‘Gesture form analysis’ brings into relief the diversity of pointing by considering the imaginary forms necessarily involved in interpreting a gesture. It combines into a single framework insights found in the literature on how the meaning of any gesture is enabled by a series of spatial operations leading from the physical form of the articulators to the form of the target. Seven distinct spatial operations combine to define a gesture type, twenty-seven of which are illustrated with examples from open-data corpora. Most types involve not the prototypical linear vector of pointing, but the plane of an open hand. Not only deictic, but also iconic and other functions are shown to be rooted in imaginary forms and their ability to draw attention to and specify locations, directions, areas and volumes of space.
Article outline
  • Introduction
    • Gesture form analysis
    • Related literature
    • Structure of the empirical part of this paper
    • Open resources
  • Pointing with a 1D profile
    • Pointing At Object
    • Pointing At Location
    • Directing
    • Dynamic Directing
    • Drawing
    • Silhouetting
    • Preliminary conclusions
  • Pointing with a 2D profile
    • Group 1: Surface-edge marking
    • Group 2: Non-intersecting surface pointing
    • Group 3: Intersecting surface pointing
    • Group 4: Sketching with surface
    • Non-Specific Holding
  • Representing in 0D
  • Discussion
  • Acknowledgements
  • Notes
  • References
References (48)
References
Andrén, Mats (2010). Children’s gestures from 18 to 30 months. Doctoral Dissertation, Lund University.Google Scholar logo with link to Google Scholar
Arnheim, Rudolf (2004 [1969]). Visual thinking. Berkeley: University of California Press.Google Scholar logo with link to Google Scholar
Bühler, Karl (2000 [1934]). Sprachtheorie. In Ludger Hoffmann (Ed.), Sprachwissenschaft. Ein Reader (2nd Ed., pp. 51–71). Berlin: Walter de Gruyter.Google Scholar logo with link to Google Scholar
Butterworth, George (2003). Pointing is the royal road to language for babies. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 9–34). Mahwah, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Clark, Herbert H. (1996). Using language. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Clark, Herbert H., Robert Schreuder, Samuel Buttrick (1983). Common ground and the understanding of demonstrative reference. Journal of Verbal Learning and Verbal Behavior, 22 (2), 245–258. Google Scholar logo with link to Google Scholar
Cooperrider, Kensy A. (2011). Reference in action: Links between pointing and language. PhD Dissertation, University of California San Diego.Google Scholar logo with link to Google Scholar
(2014). Body-directed gestures: Pointing to the self and beyond. Journal of Pragmatics, 711, 1–16. Google Scholar logo with link to Google Scholar
Deacon, Terrence W. (1997). The symbolic species: The co-evolution of language and the brain. New York & London: W. W. Norton and Company.Google Scholar logo with link to Google Scholar
(2009). The Anatomy of Meaning. Speech, gesture, and composite utterances. Cambridge & New York: Cambridge University Press. Google Scholar logo with link to Google Scholar
Fricke, Ellen (2007). Origo, Geste und Raum. Lokaldeixis im Deutschen. Berlin/New York: Walter de Gruyter. Google Scholar logo with link to Google Scholar
Gibson, James J. (1977). The theory of affordances. In Robert Shaw (Ed.), Perceiving, acting, and knowing. Towards an ecological psychology (pp. 67–82). Hillsdale, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Goodwin, Charles (2003). Pointing as situated practice. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 217–242). Mahwah, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
(2018). Co-operative action. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Hassemer, Julius (2009). Chief pointing-eye. Multiarticulatory metaphorical gestures expressing high. MA thesis, European University Viadrina, Frankfurt (Oder). Google Scholar logo with link to Google Scholar
(2016). Towards a theory of gesture form analysis. Imaginary forms as part of gesture conceptualisation, with empirical support from motion-capture data. Doctoral Dissertation, RWTH Aachen University.Google Scholar logo with link to Google Scholar
Hassemer, Julius, Gina Joue, Klaus Willmes, & Irene Mittelberg (2011). Dimensions and mechanisms of form constitution: Towards a formal description of gestures. In Proceedings of Gesture and Speech in Interaction (GESPIN) 2011. Bielefeld.Google Scholar logo with link to Google Scholar
Hassemer, Julius, Cornelia Müller, & Irene Mittelberg (2010). Marker analysis: Geometrical dimensions of gesture form. In 4th Conference of the International Society of Gesture Studies (ISGS4). July 25–30, Frankfurt (Oder).Google Scholar logo with link to Google Scholar
(2018). Decoding gestural iconicity. Cognitive Science, 42 (8), 3034–3049. Google Scholar logo with link to Google Scholar
Haviland, John B. (2003). How to point in Zinacantán. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 139–170). Mahwah, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
James, William (1890). The principles of psychology, Vol. 21. New York: Henry Holt.Google Scholar logo with link to Google Scholar
Kendon, Adam (1988). Goffman’s approach to face-to-face Interaction. In Paul Drew & Anthony Wootton (Eds.), Erving Goffman. Exploring the interaction order (pp. 14–40). Boston: Northeastern University Press.Google Scholar logo with link to Google Scholar
(2004). Gesture: Visible action as utterance. Cambridge & New York: Cambridge University Press. Google Scholar logo with link to Google Scholar
Kendon, Adam & Laura Versante (2003). Pointing by hand in “Neapolitan”. In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 109–138). Mahwah, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Kita, Sotaro (2003). Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Lawrence Erlbaum. Google Scholar logo with link to Google Scholar
Kita, Sotaro & Hedda Lausberg (2008). Generation of co-speech gestures based on spatial imagery from the right-hemisphere: Evidence from split-brain patients. Cortex, 441, 131–39. Google Scholar logo with link to Google Scholar
Koffka, Kurt (1936). Principles of Gestalt psychology. New York: Harcourt Brace.Google Scholar logo with link to Google Scholar
Kranstedt, Alfred, Andy Lücking, Thies Pfeiffer, Hannes Rieser, & Marc Staudacher (2006). Measuring and reconstructing pointing in visual contexts. In Proceedings of the Brandial 2006 – The 10th Workshop on the Semantics and Pragmatics of Dialogue (pp. 82–89). Potsdam: Universitätsverlag.Google Scholar logo with link to Google Scholar
Lausberg, Hedda & Han Sloetjes (2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, 41 (3), 841–849. Google Scholar logo with link to Google Scholar
Liddell, Scott K. (2003). Grammar, gesture, and meaning in American Sign Language. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lücking, Andy, Kirsten Bergmann, Florian Hahn, Stefan Kopp, & Hannes Rieser (2010). The Bielefeld Speech and Gesture Alignment Corpus (SaGA). In 4th Workshop on the Representation and Processing of Sign Languages (LREC 2010): Corpora and Sign Language Technologies. University of Hamburg.Google Scholar logo with link to Google Scholar
Mandel, Mark A. (1977). Iconic devices in American Sign Language. In Lynn A. Friedman (Ed.), On the other hand: New perspectives on American Sign Language (pp. 57–107). New York: Academic Press.Google Scholar logo with link to Google Scholar
McNeill, David (1992). Hand and mind: What gestures reveal about thought. Chicago: The University of Chicago Press.Google Scholar logo with link to Google Scholar
Mittelberg, Irene & Linda R. Waugh (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In Charles Forceville & Eduardo Urios-Aparisi (Eds.), Multimodal metaphor (pp. 329–356). Berlin & New York: Mouton de Gruyter.Google Scholar logo with link to Google Scholar
Müller, Cornelia (1998). Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Arno Spitz.Google Scholar logo with link to Google Scholar
(2008). What gestures reveal about the nature of metaphor. In Alan Cienki & Cornelia Müller (Eds.), Metaphor and gesture (pp. 219–245). Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
(2014). Gestural modes of representation as techniques of depiction. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, & Jana Bressem (Eds.), Body – language – communication: An international handbook on multimodality in human interaction (Vol. 21, pp. 1687–1702). Berlin & Boston: Mouton de Gruyter.Google Scholar logo with link to Google Scholar
Pfeiffer, Thies (2010). Understanding multimodal deixis with gaze and gesture in conversational interfaces. Doctoral Dissertation, Bielefeld University.Google Scholar logo with link to Google Scholar
Salffner, Sophie (2014). Farming, food and yam: language and cultural practices among Ikaan speakers: An archive of language and cultural material from the Akaan people of Ikakumo (Ondo State and Edo State, Nigeria). London: SOAS, Endangered Languages Archive. [URL]
Sowa, Timo (2006). Understanding coverbal iconic gestures in shape descriptions. Doctoral Dissertation, University of Bielefeld.Google Scholar logo with link to Google Scholar
Talmy, Leonard (2000). Toward a cognitive semantics, Vol. 1: Concept structuring systems. Cambridge: MIT Press.Google Scholar logo with link to Google Scholar
(2018). The targeting system of language. Cambridge: MIT Press. Google Scholar logo with link to Google Scholar
Wagemans, Johan, James H. Elder, Michael Kubovy, Stephen E. Palmer, Mary A. Peterson, Manish Singh, & Joachim von der Heydt (2012). A century of Gestalt psychology in visual perception I. Perceptual grouping and figure-ground organization. Psychological Bulletin, 138 (6), 1172–1217. Google Scholar logo with link to Google Scholar
Wilkins, David P. (2003). Why pointing with the index finger is not a universal (in sociocultural and semiotic terms). In Sotaro Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 171–216). Mahwah, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Cited by (8)

Cited by eight other publications

Jehlička, Jakub & Eva Lehečková
2025. Islands of iconicity: a usage-based approach to capturing meaning-making processes in gesture. Cognitive Semiotics DOI logo
Hart, Christopher
2024. What’s the point of Donald Trump? Deictic gestures in the service of right-wing populism. Social Semiotics ► pp. 1 ff. DOI logo
Mittelberg, Irene & Jennifer Hinnell
2024. Iconicity, Schematicity, and Representation in Gesture. In The Cambridge Handbook of Gesture Studies, ► pp. 56 ff. DOI logo
Rakhilina, Ekaterina & Alan Cienki
2024. Creation and Analysis of the Multimedia Russian Corpus for Gesture Research. In The Cambridge Handbook of Gesture Studies, ► pp. 249 ff. DOI logo
Abdel-Raheem, Ahmed
2023. Do political cartoons and illustrations have their own specialized forms for warnings, threats, and the like? Speech acts in the nonverbal mode. Social Semiotics 33:1 ► pp. 64 ff. DOI logo
Winter, Bodo & Tyler Marghetis
2023. Multimodality matters in numerical communication. Frontiers in Psychology 14 DOI logo
Hassemer, Julius & Vito Evola
2021. Cognitive Linguistics and Gesture. In The Routledge Handbook of Cognitive Linguistics, ► pp. 512 ff. DOI logo
Mesh, Kate, Emiliana Cruz, Joost van de Weijer, Niclas Burenhult & Marianne Gullberg
2021. Effects of Scale on Multimodal Deixis: Evidence From Quiahije Chatino. Frontiers in Psychology 11 DOI logo

This list is based on CrossRef data as of 9 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue