Cover not available

In:How the Brain Got Language – Towards a New Road Map:
Edited by Michael A. Arbib
[Benjamins Current Topics 112] 2020
► pp. 289–317

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

From actions to events

Communicating through language and gesture

 | Brandeis University
Published online: 11 August 2020
DOI logo with link
https://doi.org/10.1075/bct.112.19pus
Abstract
In this paper, I argue that an important component of the language-ready brain is the ability to recognize and conceptualize events. By ‘event’, I mean any situation or activity in the world or our mental life, that we find salient enough to individuate as a thought or word. While this may sound either trivial or non-unique to humans, I hope to show that abstracting away events and their participants from the embodied flow of experience is a characteristic unique to humans. This ability is enabled, I will argue, by two critical competencies that act as scaffolds for language-ready thought in the prehuman brain. The first, as argued by Arbib (2006, 2012, 2016) and others, is a sophisticated system of gesture production and understanding in prehumans, which provided a template for schema-like sequencing and slot-filling of information units. The second involves the integration of multiple modalities of expression in the communicative act, in particular, the alignment of co-gestural speech and co-speech gesture. With such computational facilities, action-based gestures can be abstracted away from their associated objects and become full event representations. This view supports the MSH argument for the emergence of more complex linguistic expressions from initially holophrastic units. In particular, actions can be thought of as protoverbs, which through this process are abstracted to full event descriptions, i.e., verbs.
Article outline
  • 1.Introduction
  • 2.Communication with a common ground
  • 3.The structure of actions and events
  • 4.The interpretation of gesture
  • 5.Gesture sequences
  • 6.Multimodal communication: Gesture-speech ensembles
    • 6.1Co-gestural demonstratives
    • 6.2More expressive co-gestural ensembles
  • 7.Meaning in the absence of common groun
  • 8.Toward a new road map
  • Acknowledgements
  • References
References (71)
References
Aboitiz, F. (2013) “How did vocal behavior ‘take over’ the gestural communication system?.” Language and Cognition 5, no. 2–3: 167–176. Google Scholar logo with link to Google Scholar
Abner, N., Cooperrider, K., & Goldin-Meadow, S. (2015). Gesture for linguists: A handy primer. Language and linguistics compass, 9(11), 437–451. Google Scholar logo with link to Google Scholar
Arbib, M. A. (2002) The Mirror System, Imitation, and the Evolution of Language. Imitation in animals and artifacts, 229.Google Scholar logo with link to Google Scholar
ed. (2006) Action to language via the mirror neuron system. Cambridge University Press. Google Scholar logo with link to Google Scholar
, (2008). Holophrasis and the protolanguage spectrum. Interaction Studies, 9(1), pp.154–168. Google Scholar logo with link to Google Scholar
(2008). From grasp to language: Embodied concepts and the challenge of abstraction. Journal of Physiology-Paris, 102(1–3), 4–20. Google Scholar logo with link to Google Scholar
(2012). How the Brain Got Language: The Mirror System Hypothesis. New York & Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
(2013). Precis of how the brain got language: the mirror system hypothesis. Language and Cognition, 5(2–3), 107–131. Google Scholar logo with link to Google Scholar
(2016). Towards a computational comparative neuroprimatology: framing the language-ready brain. Physics of life reviews, 16, 1–54. Google Scholar logo with link to Google Scholar
Arbib, M. A., Liebal, K. and Pika, S., (2008). “Primate vocalization, gesture, and the evolution of human language.” Current anthropology 49. 6, 1053–1076. Google Scholar logo with link to Google Scholar
Arbib, M. A. & Rizzolatti, G. (1997). Neural expectations: a possible evolutionary path from manual skills to language. Communication and Cognition, 29, 393–424.Google Scholar logo with link to Google Scholar
Armstrong, D. F., Stokoe, W. C. and Wilcox, S. E. 1995. Gesture and the nature of language. Cambridge University Press. Google Scholar logo with link to Google Scholar
Armstrong, D. F., & Wilcox, S. (2007). The gestural origin of language. Oxford University Press. Google Scholar logo with link to Google Scholar
Asher, N. (1998). Common ground, corrections and coordination. Journal of Semantics.Google Scholar logo with link to Google Scholar
Asher, N. and Lascarides, A., (2003). Logics of conversation. Cambridge University Press.Google Scholar logo with link to Google Scholar
Asher, N. and A. Lascarides. (2008). Commitments, beliefs and intentions in dialogue. Proceedings of Londial, 35–42.Google Scholar logo with link to Google Scholar
Austin, J. L. (1962). How to do things with words. London: Oxford University PressGoogle Scholar logo with link to Google Scholar
Bach, K. (2003). Speech acts and pragmatics. Blackwell. Guide to the philosophy of language, 147–167.Google Scholar logo with link to Google Scholar
Bickerton, D. (1990) Language and Species, University of Chicago Press. Bickerton, Derek (2009). Adam’s Tongue. Hill and Wang. Google Scholar logo with link to Google Scholar
2009. Adam’s tongue: how humans made language, how language made humans. Macmillan.Google Scholar logo with link to Google Scholar
Bohn, M., Call, J. and Tomasello, M., (2016). Comprehension of iconic gestures by chimpanzees and human children. Journal of experimental child psychology, 142, pp.1–17. Google Scholar logo with link to Google Scholar
Cangelosi, A. (2010). Grounding language in action and perception: from cognitive agents to humanoid robots. Physics of life reviews 7(2), 139–151. Google Scholar logo with link to Google Scholar
Clark, H. H. (1996). Using language. Cambridge University Press. Google Scholar logo with link to Google Scholar
Clark, H. H., S. E. Brennan, et al. (1991). Grounding in communication. Perspectives on socially shared cognition 13(1991), 127–149. Google Scholar logo with link to Google Scholar
Clark, H. H., Schreuder, R., & Buttrick, S. (1983). Common ground at the understanding of demonstrative reference. Journal of verbal learning and verbal behavior, 22(2), 245–258. Google Scholar logo with link to Google Scholar
Cooper, R. and Ginzburg, J. (2015). “Type theory with records for natural language semantics”. Handbook of Contemporary Semantic Theory, pp.375–407. Google Scholar logo with link to Google Scholar
Corballis, Michael C., (2003) From hand to mouth: The origins of language. Princeton University Press.Google Scholar logo with link to Google Scholar
Deacon, Terrence W. (1998). The symbolic species: The co-evolution of language and the brain. WW Norton & Company.Google Scholar logo with link to Google Scholar
Diessel, H. (2013) “Where does language come from? Some reflections on the role of deictic gesture and demonstratives in the evolution of language.” Language and Cognition 5.2–3: 239–249. Google Scholar logo with link to Google Scholar
Engle, R.A. and Clark, H.H. 1995. Using composites of speech, gestures, diagrams and demonstrations in explanations of mechanical devices. In American Association for Applied Linguistics Conference, Long Beach, CA.Google Scholar logo with link to Google Scholar
Fogassi, L., Coude, G. and Ferrari, P. F., (2013). The extended features of mirror neurons and the voluntary control of vocalization in the pathway to language. Language and Cognition, 5(2–3), pp.145–155. Google Scholar logo with link to Google Scholar
Fricke, E. (2013). Towards a unified grammar of gesture and speech: A multimodal approach. Body-language-communication. An international handbook on multimodality in human interaction, 733–754.Google Scholar logo with link to Google Scholar
Gillespie-Lynch, K., Greenfield, P. M., Lyn, H., & Savage-Rumbaugh, S. (2014). Gestural and symbolic development among apes and humans: support for a multimodal theory of language evolution. Frontiers in psychology, 5. Google Scholar logo with link to Google Scholar
Gibson, J. J. (1979). The ecological approach to visual perception: classic edition. Psychology Press.Google Scholar logo with link to Google Scholar
Gilbert, M. (1992). On social facts. Princeton University Press.Google Scholar logo with link to Google Scholar
Givòn, T. (1995). Functionalism and grammar. John Benjamins Publishing. Google Scholar logo with link to Google Scholar
(1998). On the co-evolution of language, mind, and brain. Evolution of Communication, 2(1), 45–116. Google Scholar logo with link to Google Scholar
Glenberg, A. M. & Gallese, V. (2012). Action-based language: a theory of language acquisition, comprehension, and production. Cortex, 48(7), 905–22. Google Scholar logo with link to Google Scholar
Goldin-Meadow, S. and Alibali, M.W. 2013. Gesture’s role in speaking, learning, and creating language. Annual review of psychology, 64, pp. 257-283. Google Scholar logo with link to Google Scholar
Grice, H. P. (1981). Presupposition and conversational implicature. In C. Peter (ed.), Radical pragmatics. New York: Academic Press, pp. 183–198. Reprinted in Grice, H. P. (1989) Studies in the way of words. Cambridge: Harvard University Press, pp. 269–282.Google Scholar logo with link to Google Scholar
Hsiao, K. -Y., S. Tellex, S. Vosoughi, R. Kubat, and D. Roy. (2008). Object schemas for grounding language in a responsive robot. Connection Science 20(4), 253–276. Google Scholar logo with link to Google Scholar
Hauser, Marc D., Noam Chomsky, and W. Tecumseh Fitch. (2002) “The faculty of language: what is it, who has it, and how did it evolve?.” Science 298.5598: 1569–1579. Google Scholar logo with link to Google Scholar
Hockett, C. F., (1960). The origin of speech. Scientific American, 203(3), pp.88–97. Google Scholar logo with link to Google Scholar
Iverson, J. M., Capirci, O. and Caselli, M. C., (1994). From communication to language in two modalities. Cognitive development, 9(1), pp.23–43. Google Scholar logo with link to Google Scholar
Iverson, J. M., & Goldin-Meadow, S. (2005). Gesture paves the way for language development. Psychological science, 16(5), 367–371. Google Scholar logo with link to Google Scholar
Kamp, H. and Reyle, U. 1993. From discourse to logic: Introduction to model-theoretic semantics of natural language, formal logic and discourse representation. Studies in Linguistics and Philosophy. Kluwer.Google Scholar logo with link to Google Scholar
Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relationship of verbal and nonverbal communication, 207–228. The Hague: Mouton.Google Scholar logo with link to Google Scholar
(1995). Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of pragmatics, 23(3), pp. 247–279.
Lascarides, A. and M. Stone. (2006). Formal semantics for iconic gesture. In Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue (BRANDIAL), pp. 64–71.
. (2009). A formal semantic analysis of gesture. Journal of Semantics 26(4), 393–449. Google Scholar logo with link to Google Scholar
Pika, S. and Mitani, J. C., (2009). The directed scratch: evidence for a referential gesture in chimpanzees. The prehistory of language, 1, pp.166–181. Google Scholar logo with link to Google Scholar
Pustejovsky, J. (1995). The Generative Lexicon. MIT Press, Cambridge, MA.Google Scholar logo with link to Google Scholar
(2013). Where things happen: On the semantics of event localization. In Proceedings of isa-9: International workshop on semantic annotation.Google Scholar logo with link to Google Scholar
Pustejovsky, J. and N. Krishnaswamy. (2014). Generating simulations of motion events from verbal descriptions. In Lexical and Computational Semantics (*SEM 2014). ACL. Google Scholar logo with link to Google Scholar
Pustejovsky, J. and J. L. Moszkowicz. (2011). The qualitative spatial dynamics of motion in language. Spatial Cognition & Computation 11(1), 15–44. Google Scholar logo with link to Google Scholar
Rizzolatti, G., and Arbib, M. (1998). Language within our grasp. Trends in Neurosciences, 21, 188–194. Google Scholar logo with link to Google Scholar
Schlenker, P. (2015). Visible meaning: Sign language and the foundations of semantics. Manuscript, Institut Jean-Nicod and New York University.Google Scholar logo with link to Google Scholar
Searle, J. R. (1969). Speech acts: An essay in the philosophy of language (Vol. 626). Cambridge university press. Google Scholar logo with link to Google Scholar
Stalnaker, R. (2002). Common ground. Linguistics and philosophy 25(5), 701–721. Google Scholar logo with link to Google Scholar
Steels, L. (1999). The talking heads experiment: Vol. I. Words and meaning (Special preedition). Brussels: Vrije Universiteit Brussel.Google Scholar logo with link to Google Scholar
(2011). Modeling the cultural evolution of language. Physics of Life Reviews, 8(4), 339–356. Google Scholar logo with link to Google Scholar
Stout, Dietrich, & Hecht, Erin E. (2017). Evolutionary neuroscience of cumulative culture. Proceedings of the National Academy of Sciences, 114(30), 7861–7868. Google Scholar logo with link to Google Scholar
Strickland, B., Geraci, C., Chemla, E., Schlenker, P., Kelepir, M., and Pfau, R. (2015). Event representations constrain the structure of language: Sign language as a window into universally accessible linguistic biases. Proceedings of the National Academy of Sciences. doi: Google Scholar logo with link to Google Scholar
Tomasello, M. and M. Carpenter. (2007). Shared intentionality. Developmental science 10(1), 121–125. Google Scholar logo with link to Google Scholar
Vendler, Z. (1967). Linguistics and Philosophy. Ithaca: Cornell University Press.Google Scholar logo with link to Google Scholar
Volterra, V., Caselli, M. C., Capirci, O., & Pizzuto, E. (2005). Gesture and the emergence and development of language. Beyond nature-nurture: Essays in honor of Elizabeth Bates, 3–40.Google Scholar logo with link to Google Scholar
von Uexkll, J. (1957). A stroll through the worlds of animals and men: A picture book of invisible worlds, in Schiller, C. H. (ed), Instinctive Behavior: The Development of a Modern Concept. New York: International Universities Press, 5–80.Google Scholar logo with link to Google Scholar
Wang, I., M. Ben Fraj, P. Narayana, D. Patil, G. Mulay, R. Bangar, R. Beveridge, B. Draper, and J. Ruiz. (2017). Eggnog: A continuous, multimodal data set of naturally occurring gestures with ground truth labels. In Proceedings of the 12th IEEE International Conference on Automatic Face & Gesture Recognition.
Whitehead, A. N. (1919). An Enquiry concerning the Principles of Natural Knowledge, Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Ziemke, T. & Sharkey, N. E. (2001). A stroll through the worlds of robots and animals: Applying Jakob von Uexkll’s theory of meaning to adaptive robots and artificial life. Semiotica, 2001(134), 701–746. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue