The haptic nature of gesture: Rethinking gesture with new multitouch digital technologies

Sinclair, Nathalie; Freitas, Elizabeth de

doi:10.1075/gest.14.3.04sin

Article published In: Gesture
Vol. 14:3 (2014) ► pp.351–374

Get fulltext from our e-platform

Download PDF

The haptic nature of gesture

Rethinking gesture with new multitouch digital technologies

Nathalie Sinclair | Simon Fraser University

Elizabeth de Freitas | Manchester Metropolitan University

Published online: 21 January 2016

https://doi.org/10.1075/gest.14.3.04sin

This paper is about re-thinking gesture in order to reckon with its material and haptic nature, especially in the current multitouch technology environment. This re-thinking of gesture returns to the principle of indexicality found in Peirce’s material semiotics, and develops this principle through the work of Gilles Châtelet and Gilles Deleuze around hand-eye relationships. Drawing on the work of Jürgen Streek, we propose and discuss the notion of the tangible gesture, in the context of mathematical explorations of young children with a multitouch iPad environment designed to promote counting on and with the fingers.

Keywords: technology, mathematics, the senses

References (33)

Alibali, Martha & Allysa DiRusso (1999). The function of gesture in learning to count: More than keeping track. Cognitive Development, 141, 37–56.

Angel, Maria & Anna Gibbs (2013). At the time of writing: Digital media, gestures, and handwriting. Electronic Book Review. Retrieved 22.09.2013. [URL]

Bartolini-Bussi, Mariolina & Mara Boni (2003). Instruments for semiotic mediation in primary school classrooms. For the Learning of Mathematics, 23 (2), 12–19.

Châtelet, Gilles ([1993] 2000). Les enjeux du mobile. Paris: Seuil. English translation by Robert Shore & Muriel Zagha (2000): Figuring space: Philosophy, mathematics, and physics. Dordrecht: Kluwer, 2000.

(2010). L’enchantement du virtuel. Philosophie, physique, mathématique. Paris, France: Presses de l’École normale supérieure.

de Freitas, Elizabeth & Nathalie Sinclair (2014). Mathematics and the body: Material entanglements in the classroom. New York: Cambridge University Press.

Deleuze, Gilles. (2003). Francis Bacon: The logic of sensation. Translated from the French by Daniel W. Smith. London, UK: Continuum.

Gibson, James J. (1966). The senses considered as perceptual systems. Boston: Houghton Mifflin.

Goodwin, Charles (2007). Environmentally coupled gestures. In Susan D. Duncan, Justine Cassell, & Elena T. Levy (Eds.), Gesture and the dynamic dimension of language: Essays in honor of David McNeill (pp. 195–212). Amsterdam: John Benjamins.

Kendon, Adam (2000). Language and gesture: Unity or duality? In David McNeill (Ed.), Language and gesture (pp. 47–63). Cambridge, UK: Cambridge University Press.

(2004). Gesture: Visible action as utterance. Cambridge, UK: Cambridge University Press.

Kita, Sotaro (Ed.) (2003). Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Erlbaum.

Maturana, Humberto & Francisco Varela ([1984] 1992). The tree of knowledge: Biological roots of human understanding. Boston, MA: Shambhala Publications.

McNeill, David (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.

(2005). Gesture and thought. Chicago: University of Chicago Press.

Mulder, Axel (1996). Hand gestures for HCI. Technical Report, NSERC Hand Centered Studies of Human Movement project. Burnaby, BC, Canada: Simon Fraser University. Available: [URL]

Nemirovsky, Ricardo, Molly Kelton, & Bohdan Rhodehamel (2012). Gesture and imagination: On the constitution and uses of phantasms. Gesture, 12 (2), 130–165.

Noel, Marie-Pascale (2005). Finger gnosia: A predictor of numerical abilities in children? Child Neuropsychology, 51, 413–430.

Peirce, Charles Sanders ([1894] 1998). What is a sign? In Nathan Houser, et al. (Eds.), The essential Peirce, Vol. 21 (pp. 4–10). Bloomington, IN: Indiana University Press.

(1932). Elements of logic . In Charles Hartshorne & Paul Weiss (Eds.), Collected papers of Charles Sanders Peirce, Vol. 21. Cambridge: Harvard University Press.

(1994). Peirce on signs: Writings on semiotic, James Hoopes (Ed.). Chapel Hill, NC: University of North Carolina Press.

Radford, Luis (2002). The seen, the spoken and the written: A semiotic approach to the problem of objectification of mathematical knowledge. For the Learning of Mathematics, 22 (2), 14–23.

Rasmussen, Chris, Michelle Stephan, & Karen Allen (2004). Classroom mathematical practices and gesturing. Journal of Mathematical Behavior, 231, 301–323.

Reynolds, Fione J. & Robert A. Reeve (2002). Gesture in collaborative mathematical problem-solving. Journal of Mathematical Behavior, 201, 447–460.

Robutti, Ornella & Francesca Ferrara (2002). Approaching algebra through motion experiences. In Anne D. Cockburn & Elena Nardi (Eds.), Proceedings of the 26th PME International Conference , Vol. 11 (pp. 111–118).

Rossini, Nicla (2012). Reinterpreting gesture as language: Language in action. Amsterdam: IOS Press.

Roth, Wolff-Michael (2001). Gestures: Their role in teaching and learning. Review of Educational Research, 71 (3), 365–392.

Rotman, Brian (1987). Signifying nothing: A semiotics of zero. Basingstoke: Macmillan.

(2008). Becoming beside ourselves: The alphabet, ghosts, and distributed human beings. Durham: Duke University Press.

Sinclair, Nathalie & Nicholas Jackiw (2011). Touchcounts [Software application for the iPad]. Burnaby, BC: Tangible Learning Project.

Sinclair, Nathalie & David Pimm (2014). Number’s subtle touch: Expanding finger gnosis in the era of multi-touch technologies. In Nicol Cynthia et al. (Eds.), Proceedings of the Psychology of Mathematics Education Conference . Vancouver, Canada, July 15–20, 2014, Vol. 51 (pp. 209–216).

Streeck, Jürgen (2009). Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins.

Villamor, Craig, Dan Willis, & Luke Wroblewski (2010). Touch gesture reference guide. [URL]

Cited by (49)

Cited by 49 other publications

Order by:

Albuquerque, Melwina, Shizuka Dara, Priyadharshini Elangaivendan & Sanjay Chandrasekharan

2025. Designing a Sequence of Systems to Facilitate the Evolution of Embodied Learning in Teaching Rate of Change and Slope in 7th Grade. In Proceedings of the International Conference on Technology 4 Education 2024, Volume 2 [Lecture Notes in Educational Technology, ], ► pp. 230 ff.

Carotenuto, Gemma

2025. Embodied Number Sense in the Making: A Preschooler’s Interaction with the Fingu App. Digital Experiences in Mathematics Education

Elangaivendan, Priyadharshni, Setu Havanur & Sanjay Chandrasekharan

2025. Conceptual Blending as a Design and Analysis Framework for Building Embodied Learning Technologies. In Proceedings of the International Conference on Technology 4 Education 2024, Volume 1 [Lecture Notes in Educational Technology, ], ► pp. 220 ff.

Jouannet, Paula

2025. Beyond intuitive models: exploring the role of gestures in learning multiplication and division. Educational Studies in Mathematics 120:3 ► pp. 477 ff.

Gerner, Alexander Matthias, Thiemo Breyer, Niklas Grouls & Johannes F. M. Schick

2024. Diachronic Perspectives on Embodiment and Technology: Gestures and Artefacts – An Introduction. In Diachronic Perspectives on Embodiment and Technology [Philosophy of Engineering and Technology, 46], ► pp. 1 ff.

Kynigos, Chronis

2024. Embedding Mathematics in Socio-Scientific Games: The Mathematical in Grappling with Wicked Problems. Education Sciences 14:6 ► pp. 630 ff.

Miller, Marty

2024. Reflecting the seen scene: Theorizing screenshots as multistable metapictures. Convergence: The International Journal of Research into New Media Technologies

Walkington, Candace, Mitchell J. Nathan, Jonathan Hunnicutt, Julianna Washington & Monique Zhou

2024. New kinds of embodied interactions that arise in augmented reality dynamic geometry software. The Journal of Mathematical Behavior 75 ► pp. 101175 ff.

Abrahamson, Dor, Kimiko Ryokai & Justin Dimmel

2023. Learning Mathematics with Digital Resources: Reclaiming the Cognitive Role of Physical Movement. In Handbook of Digital Resources in Mathematics Education [Springer International Handbooks of Education, ], ► pp. 1 ff.

Abrahamson, Dor, Kimiko Ryokai & Justin Dimmel

2024. Learning Mathematics with Digital Resources: Reclaiming the Cognitive Role of Physical Movement. In Handbook of Digital Resources in Mathematics Education [Springer International Handbooks of Education, ], ► pp. 609 ff.

de Freitas, Elizabeth & Nathalie Sinclair

2023. Landing Sites, Cities, and Nonplaces: Collaborating Across the Conference Circuit. Qualitative Inquiry 29:1 ► pp. 7 ff.

Sinha, Ravi, Hillary Swanson, Jody Clarke-Midura, Jessica F. Shumway, Victor R. Lee & Sanjay Chandrasekharan

2023. Proceedings of the 2023 Symposium on Learning, Design and Technology, ► pp. 1 ff.

Yeung, Wing-Leung & Oi-Lam Ng

2023. Characterizing touchscreen actions in technology-enhanced embodied learning for mathematics instruction in K-12 setting – A systematic review (2010–2023). Computers & Education 205 ► pp. 104881 ff.

Robutti, Ornella, Cristina Sabena, Christina Krause, Carlotta Soldano & Ferdinando Arzarello

2022. Gestures in Mathematics Thinking and Learning. In Handbook of Cognitive Mathematics, ► pp. 685 ff.

Robutti, Ornella, Cristina Sabena, Christina Krause, Carlotta Soldano & Ferdinando Arzarello

2022. Gestures in Mathematics Thinking and Learning. In Handbook of Cognitive Mathematics, ► pp. 1 ff.

Sinclair, Nathalie, Mariam Haspekian, Ornella Robutti & Alison Clark-Wilson

2022. Revisiting Theories That Frame Research on Teaching Mathematics with Digital Technology. In The Mathematics Teacher in the Digital Era [Mathematics Education in the Digital Era, 16], ► pp. 391 ff.

Sinclair, Nathalie & Eva Jablonka

2022. Mathematics Learning: Structured Ways of Moving With. In The Palgrave Handbook of Embodiment and Learning, ► pp. 419 ff.

Tucker, Stephen I. & Teri N. Johnson

2022. Developing number sense with Fingu: a preschooler’s embodied mathematics during interactions with a multi-touch digital game. Mathematics Education Research Journal 34:2 ► pp. 393 ff.

Bairral, Marcelo

2021. Interacciones, toques en pantalla y aprendizaje de cuadriláteros. Revista Venezolana de Investigación en Educación Matemática 1:2 ► pp. 1 ff.

Billion, Lara Kristina

2021. Reconstruction of the Interpretation of Geometric Diagrams of Primary School Children Based on Actions on Various Materials -<br /> A Semiotic Perspective on Actions. International Electronic Journal of Mathematics Education 16:3 ► pp. em0650 ff.

Forte, Magali

2021. Identités sociomatérielles et création d’histoires plurilingues et numériques. The Canadian Modern Language Review 77:4 ► pp. 332 ff.

Güneş, Canan

2021. A Quantitative Shift Towards Multiplicative Thinking. Digital Experiences in Mathematics Education 7:3 ► pp. 361 ff.

Karnam, DurgaPrasad, Harshit Agrawal, Pranay Parte, Saurabh Ranjan, Priyanka Borar, Prasanna Prakash Kurup, Amose Jebin Joel, Pattamadai Sankaran Srinivasan, Uddhav Suryawanshi, Aniket Sule & Sanjay Chandrasekharan

2021. Touchy feely vectors: A compensatory design approach to support model‐based reasoning in developing country classrooms. Journal of Computer Assisted Learning 37:2 ► pp. 446 ff.

Pande, Prajakt

2021. Learning and expertise with scientific external representations: an embodied and extended cognition model. Phenomenology and the Cognitive Sciences 20:3 ► pp. 463 ff.

Shvarts, Anna, Rosa Alberto, Arthur Bakker, Michiel Doorman & Paul Drijvers

2021. Embodied instrumentation in learning mathematics as the genesis of a body-artifact functional system. Educational Studies in Mathematics 107:3 ► pp. 447 ff.

Tancredi, Sofia, Rotem Abdu, Dor Abrahamson & Ramesh Balasubramaniam

2021. Modeling nonlinear dynamics of fluency development in an embodied-design mathematics learning environment with Recurrence Quantification Analysis. International Journal of Child-Computer Interaction 29 ► pp. 100297 ff.

Tucker, Stephen I. & Patricia S. Moyer-Packenham

2021. Exploring the phenomenon of distance in children’s interactions with touchscreen digital mathematics games. International Journal of Mathematical Education in Science and Technology 52:10 ► pp. 1447 ff.

Bakos, Sandy & David Pimm

2020. Beginning to Multiply (with) Dynamic Digits: Fingers as Physical–Digital Hybrids. Digital Experiences in Mathematics Education 6:2 ► pp. 145 ff.

Janßen, Thomas, Estela Vallejo-Vargas, Angelika Bikner-Ahsbahs & David A. Reid

2020. Design and Investigation of a Touch Gesture for Dividing in a Virtual Manipulative Model for Equation-solving. Digital Experiences in Mathematics Education 6:2 ► pp. 166 ff.

Kelleher, William

2020. Digital media, the body and agency in a South African education institution from the perspective of narrative research. The Journal for Transdisciplinary Research in Southern Africa 16:1

Moon, Sung-Jae & Kyeong-Hwa Lee

2020. Deleuzian actualizations of the multiplicative concept: a study of perceptual flow and the transformation of learning assemblages. Educational Studies in Mathematics 104:2 ► pp. 221 ff.

Sinclair, Nathalie & Alf Coles

2020. Affect and Ritualisation in Early Number Work. In Theorizing and Measuring Affect in Mathematics Teaching and Learning, ► pp. 77 ff.

Abrahamson, Dor, Virginia J. Flood, Joshua A. Miele & Yue-Ting Siu

2019. Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students. ZDM 51:2 ► pp. 291 ff.

Karnam, DurgaPrasad, Harshit Agrawal, Pranay Parte, Saurabh Ranjan, Aniket Sule & Sanjay Chandrasekharan

2019. 2019 IEEE Tenth International Conference on Technology for Education (T4E), ► pp. 114 ff.

Mikulan, Petra & Nathalie Sinclair

2019. Stratigraphy as a method for studying the different modes of existence arising in the mathematical classroom. ZDM 51:2 ► pp. 239 ff.

Sinclair, Nathalie

2019. Role of the Senses in Mathematical Thinking and Learning. In Encyclopedia of Educational Philosophy and Theory, ► pp. 1 ff.

Sinclair, Nathalie

2024. Knowing as Remembering: Methodological Experiments in Embodied Experiences of Number. Digital Experiences in Mathematics Education 10:1 ► pp. 29 ff.

Sinclair, Nathalie

2025. Becoming Derivative in an AR Environment. Digital Experiences in Mathematics Education

Sinclair, Nathalie & Elizabeth de Freitas

2019. Body studies in mathematics education: diverse scales of mattering. ZDM 51:2 ► pp. 227 ff.

Streeck, Jürgen

2019. Gesture research. In Handbook of Pragmatics [Handbook of Pragmatics, ], ► pp. 3 ff.

Streeck, Jürgen

2022. Gesture research. In Handbook of Pragmatics [Handbook of Pragmatics, ], ► pp. 672 ff.

Menz, Petra & Nathalie Sinclair

2018. Diagramming and Gesturing During Mathematizing: Kinesthetic and Haptic Interactions Support Mathematical Ideation. In Signs of Signification [ICME-13 Monographs, ], ► pp. 315 ff.

Ng, Oi-Lam & Nathalie Sinclair

2018. Drawing in Space: Doing Mathematics with 3D Pens. In Uses of Technology in Primary and Secondary Mathematics Education [ICME-13 Monographs, ], ► pp. 301 ff.

Pimm, David

2018. On Number Language: A Commentary on Chapter 3. In Building the Foundation: Whole Numbers in the Primary Grades [New ICMI Study Series, ], ► pp. 71 ff.

Rahaman, Jeenath, Harshit Agrawal, Nisheeth Srivastava & Sanjay Chandrasekharan

2018. Recombinant Enaction: Manipulatives Generate New Procedures in the Imagination, by Extending and Recombining Action Spaces. Cognitive Science 42:2 ► pp. 370 ff.

Jackiw, Nicholas & Nathalie Sinclair

2017. TouchCounts and Gesture Design. In Frontiers in Pen and Touch [Human–Computer Interaction Series, ], ► pp. 51 ff.

Ng, Oi-Lam

2016. Comparing Calculus Communication across Static and Dynamic Environments Using a Multimodal Approach. Digital Experiences in Mathematics Education 2:2 ► pp. 115 ff.

Ng, Oi-Lam

2019. Examining Technology-Mediated Communication Using a Commognitive Lens: the Case of Touchscreen-Dragging in Dynamic Geometry Environments. International Journal of Science and Mathematics Education 17:6 ► pp. 1173 ff.

Sinclair, Nathalie, Maria G. Bartolini Bussi, Michael de Villiers, Keith Jones, Ulrich Kortenkamp, Allen Leung & Kay Owens

2016. Recent research on geometry education: an ICME-13 survey team report. ZDM 48:5 ► pp. 691 ff.

This list is based on CrossRef data as of 9 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.