Cover not available

Article published In: International Journal of Corpus Linguistics
Vol. 28:2 (2023) ► pp.172–201

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Towards a corpus-based description of speech-gesture units of meaning

The case of the circular gesture

 | Macau University of Science and Technology
 | University of Nottingham
Published online: 13 October 2022
DOI logo with link
https://doi.org/10.1075/ijcl.20174.che
Abstract
The theories and methods in corpus linguistics (CL) have had an impact on numerous areas in applied linguistics. However, the interface between CL and multimodal speech-gesture studies remains underexplored. One fundamental question is whether it is possible, and even appropriate, to apply the theories and paradigms established based on textual data to multimodal data. To explore this, we examine how CL can assist investigating lexico-grammatical patterns of speech co-occurring with a recurrent gesture (i.e. the circular gesture). Sinclair, J. (1996). The search for units of meaning. Textus, 9(1), 1000–1032.Google Scholar logo with link to Google Scholar unit of meaning model is used to describe the co-gestural speech patterns. The study draws on a subset of the Nottingham Multimodal Corpus, in which 570 instances of circular gestures and their co-occurring speech are identified and analysed. We argue that Sinclair’s unit of meaning model can be extended to include speech-gesture patterns, and that those descriptions enable a more nuanced understanding of meaning in context.
Article outline
  • 1.Introduction
  • 2.Recurrent gestures
  • 3.The unit of meaning model
  • 4.Corpus and methods
    • 4.1Data selection
    • 4.2Speech-gesture annotations and concordance
    • 4.3Cluster analysis
    • 4.4Concordance plot
    • 4.5Sampling techniques to develop a lexico-grammatical framework
    • 4.6Cross-checking and inter-rater reliability tests of speech patterns
  • 5.Analysis
    • 5.1The multimodal unit of meaning of the circular gesture: Overall distribution
    • 5.2Main category: Colligation
      • 5.2.1Clause
      • 5.2.2Verb phrase (+ ___)
      • 5.2.3Noun phrase (+ ___)
      • 5.2.4Modification (+ noun phrase)
      • 5.2.5Dysfluent speech
      • 5.2.6Prepositional phrase
      • 5.2.7Predictive (+___)
      • 5.2.8Discourse marker
      • 5.2.9Conjunction
    • 5.3Main category: Semantic preference
      • 5.3.1Deixis
      • 5.3.2Negation
      • 5.3.3Confirmation
    • 5.4Main category: Semantic prosody
      • 5.4.1Intensification
      • 5.4.2Vagueness
      • 5.4.3Monitoring mutual ground
    • 5.5Sub-category: Semantic preference (‘on-going process’)
  • 6.Towards a multimodal unit of meaning
  • Acknowledgements
  • Appendix
  • References
References (36)
References
Adolphs, S., & Carter, R. (2013). Spoken Corpus Linguistics: From Monomodal to Multimodal. Routledge. Google Scholar logo with link to Google Scholar
Adolphs, S., Brown, B., Carter, R., Crawford, P., & Sahota, O. (2004). Applying corpus linguistics in a health care context. Journal of Applied Linguistics, 1(1), 9–28. Google Scholar logo with link to Google Scholar
Anthony, L. (2019). AntConc (Version 3.5.8) [Computer Software]. [URL]
Atkins, S., & Harvey, K. (2010). How to use corpus linguistics in the study of health communication. In A. O’Keeffe & M. McCarthy (Eds.), The Routledge Handbook of Corpus Linguistics (pp. 605–619). Routledge. Google Scholar logo with link to Google Scholar
Cánovas, C. P., Valenzuela, J., Carrión, D. A., Olza, I., & Ramscar, M. (2020). Quantifying the speech-gesture relation with massive multimodal datasets: Informativity in time expressions. PLoS ONE, 15(6), 1–18. Google Scholar logo with link to Google Scholar
Chen, Y., Adolphs, S., & Knight, D. (2021). Multimodal Discourse Analysis. In J. Hardy & E. Friginal (Eds.), Routledge Handbook of Corpus Approaches to Discourse Analysis (pp. 98–115). Routledge.Google Scholar logo with link to Google Scholar
Cienki, A., & Müller, C. (2008). Metaphor, gesture, and thought. In R. W. Gibbs (Ed.), The Cambridge Handbook of Metaphor and Thought (pp. 483–501). Cambridge University Press. Google Scholar logo with link to Google Scholar
Clift, R. (2020). Stability and visibility in embodiment: The ‘Palm Up’ in interaction. Journal of Pragmatics, 1691, 190–205. Google Scholar logo with link to Google Scholar
Debras, C. (2017). The shrug. Gesture, 16(1), 1–34. Google Scholar logo with link to Google Scholar
(2021). Multimodal profiles of je (ne) sais pas in spoken French. Journal of Pragmatics, 1821, 42–62. Google Scholar logo with link to Google Scholar
ELAN (Version 6.0) [Computer software]. (2020). Nijmegen: Max Planck Institute for Psycholinguistics, The Language Archive. [URL]
Feyaerts, K., Rominger, C., Lackner, H. K., Brône, G., Jehoul, A., Oben, B., & Papousek, I. (2022). In your face? Exploring multimodal response patterns involving facial responses to verbal and gestural stance-taking expressions. Journal of Pragmatics, 1901, 6–17. Google Scholar logo with link to Google Scholar
Gullberg, M., & McCafferty, S. G. (2008). Introduction to gesture and SLA: Toward an integrated approach. Studies in Second Language Acquisition, 30(02), 133–146. Google Scholar logo with link to Google Scholar
Hinnell, J. (2018). The multimodal marking of aspect: The case of five periphrastic auxiliary constructions in North American English. Cognitive Linguistics, 29(4), 773–806. Google Scholar logo with link to Google Scholar
Hunston, S. (2002). Corpora in Applied Linguistics. Cambridge University Press. Google Scholar logo with link to Google Scholar
(2007). Semantic prosody revisited. International Journal of Corpus Linguistics, 12(2), 249–268. Google Scholar logo with link to Google Scholar
Kendon, A. (2004). Gesture: Visible Action as Utterance. Cambridge University Press. Google Scholar logo with link to Google Scholar
Kita, S. (2009). Cross-cultural variation of speech-accompanying gesture: A review. Language and Cognitive Processes, 24(2), 145–167. Google Scholar logo with link to Google Scholar
Kita, S., van Gijn, I., & van der Hulst, H. (1998). Movement phases in signs and co-speech gestures, and their transcription by human coders. In I. Wachsmuth & M. Fröhlich (Eds.), Gesture and Sign Language in Human-Computer Interaction: International Gesture Workshop Bielefeld, Germany, September 17–19, 1997 Proceedings (pp. 23–35). Springer. Google Scholar logo with link to Google Scholar
Knight, D. (2011). Multimodality and Active Listenership: A Corpus Approach. Bloomsbury.Google Scholar logo with link to Google Scholar
Ladewig, S. H. (2011). Putting the cyclic gesture on a cognitive basis. CogniTextes, 61. Google Scholar logo with link to Google Scholar
(2014). Recurrent gestures. In C. Müller, A. Cienki, F. Ellen, S. H. Ladewig, D. McNeill, & J. Bressem (Eds.), Body – Language – Communication: An International Handbook on Multimodality in Human Interaction (Volume 2) (pp. 1558–1574). De Gruyter Mouton.Google Scholar logo with link to Google Scholar
Lausberg, H., & Sloetjes, H. (2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, Instruments, & Computers, 41(3), 841–849. Google Scholar logo with link to Google Scholar
Lin, P., & Chen, Y. (2020). 5 Multimodality I: Speech, prosody and gesture. In S. Adophs & D. Knight (Eds.), The Routledge Handbook of Digital Humanities (pp. 66–84). Routledge. Google Scholar logo with link to Google Scholar
Louw, B. (1993). Irony in the text or insincerity in the writer? The diagnostic potential of semantic prosodies. In M. Baker, G. Grancis, & E. Tognini-Bonelli (Eds.), Text and Technology: In Honour of John Sinclair (pp. 157–176). John Benjamins. Google Scholar logo with link to Google Scholar
McNeill, D. (2005). Gesture and Thought. The University of Chicago Press. Google Scholar logo with link to Google Scholar
Mahlberg, M. (2013). Corpus Stylistics and Dickens’s Fiction. Routledge. Google Scholar logo with link to Google Scholar
Müller, C., Bressem, J., & Ladewig, S. H. (2013). Towards a grammar of gestures: A form-based view. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill, & S. Teßendorf (Eds.), Body-Language-Communication: An International Handbook on Multimodality in Human Interaction (Volume 1) (pp. 707–733). De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Müller, Cornelia. (2004). The Palm-Up-Open-Hand. A case of a gesture family? In C. Müller & R. Posner (Eds.), The Semantics and Pragmatics of Everyday Gestures: Proceedings of the Berlin Conference April 1998 (pp. 233–256). Weidler.Google Scholar logo with link to Google Scholar
Schegloff, E. A. (1984). On some gestures’ relation to talk. In J. M. Atkinson & J. Heritage (Eds.), Structures of Social Action (pp. 266–296). Cambridge University Press.Google Scholar logo with link to Google Scholar
Sinclair, J. (1996). The search for units of meaning. Textus, 9(1), 1000–1032.Google Scholar logo with link to Google Scholar
(2004). Trust the Text: Language, Corpus and Discourse. Routledge. Google Scholar logo with link to Google Scholar
Zima, E. (2017). On the multimodality of [all the way from X PREP Y]. Linguistics Vanguard, 3(s1), 1–12. Google Scholar logo with link to Google Scholar
Cited by (4)

Cited by four other publications

Adolphs, Svenja & Dawn Knight
2025. Corpus Linguistics and Multimodal Interaction. In Reference Module in Social Sciences, DOI logo
Fioravanti, Irene
2025. Connecting Corpus Linguistics and Psycholinguistics. In Exploration of the Intersection of Corpus Linguistics and Language Science, ► pp. 1 ff. DOI logo
Chen, Yaoyao, Svenja Adolphs & Dawn Knight
2024. Towards a speech-gesture profile of discourse markers: The case of ‘I mean’. Lingua 312 ► pp. 103836 ff. DOI logo
Wu, Suwei, Alan Cienki & Yaoyao Chen
2024. Gesture in contexts of verbal negation in Chinese. Gesture 23:1-2 ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 12 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue