Cover not available

Article published In: Gesture
Vol. 20:1 (2021) ► pp.103–134

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Discourse markers in relation to non-verbal behavior

How do speech and body language correlate?

 | University of Maribor
 | University of Maribor
 | University of Maribor
 | University of Maribor
Published online: 22 November 2021
DOI logo with link
https://doi.org/10.1075/gest.20018.mla
Abstract
The research proposed in this paper focuses on pragmatic interlinks between discourse markers and non-verbal behavior. Although non-verbal behavior is recognized to add non-redundant information and social interaction is not merely recognized as the transmission of words and sentences, the evidence regarding grammatical/linguistic interlinks between verbal and non-verbal concepts are vague and limited to restricted domains. This is even more evident when non-verbal behavior acts in the foreground but contributes to the structure and organization of the discourse. This research focuses on investigating the multimodal nature of discourse markers by observing their linguistic and paralinguistic properties in informal discourse. We perform a quantitative analysis with case studies for representative cases. The results show that discourse markers and background non-verbal behavior tend to follow a similar functionality in interaction. Therefore, by examining them together, one gains more insight into their true intent despite the high multifunctionality of both non-verbal behavior and DMs.
Article outline
  • Introduction
  • Data and methodology
    • The EVA corpus
    • The annotation process
    • Annotating discourse markers in EVA corpus
    • Annotating non-verbal behavior through NCI
  • Results
    • Co-occurrence of DMs and NCIs
    • Co-occurrence of DMs and NCIs according to different classes of DMs
    • Case analysis for individual DM classes
      • Speech formation DMs
      • Contact DMs
      • Feedback DMs
      • Dialogue structure DMs
      • Connective DMs
  • Discussion
  • Conclusion
  • Notes
  • References
References (57)
References
Allwood, J. (2014). A framework for studying human multimodal communication. In Matej Rojc & Nick Campbell (Eds.), Coverbal synchrony in human-machine interaction (pp. 17–39). Boca Raton, FL: CRC Press.Google Scholar logo with link to Google Scholar
Allwood, J., Ahlsén, E., Lund, J., & Sundqvist, J. (2005). Multimodality in own communication management. In Proceedings from the Second Nordic Conference on Multimodal Communication (pp. 1–20). Göteborg: Göteborg University.Google Scholar logo with link to Google Scholar
Arnold, L. (2012). Dialogic embodied action: Using gesture to organize sequence and participation in instructional interaction. Research on Language & Social Interaction, 45 (3), 269–296. Google Scholar logo with link to Google Scholar
Baiat, G. E., Coler, M., Pullen, M., Tienkouw, S., & Hunyadi, L. (2013, December). Multimodal analysis of “well” as a discourse marker in conversation: A pilot study. In 2013 IEEE 4th International Conference on Cognitive Infocommunications (CogInfoCom) (pp. 283–288).
CrossRef DOI logo with hyperlink to permanent DOI
. Google Scholar logo with link to Google Scholar
Birdwhistell, R. L. (2010). Kinesics and context. Philadelphia: University of Pennsylvania Press.Google Scholar logo with link to Google Scholar
Bonsignori, V. & Camiciottoli, B. C. (Eds.). (2017). Multimodality across communicative settings, discourse domains and genres. Cambridge: Cambridge Scholars Publishing.Google Scholar logo with link to Google Scholar
Cassell, J. (2001). Embodied conversational agents: representation and intelligence in user interfaces. AI magazine, 22 (4), 67.Google Scholar logo with link to Google Scholar
Chui, K., Lee, C. Y., Yeh, K., & Chao, P. C. (2018). Semantic processing of self-adaptors, emblems, and iconic gestures: An ERP study. Journal of Neurolinguistics, 471, 105–122. Google Scholar logo with link to Google Scholar
Cienki, A. & Müller, C. (Eds.). (2008). Metaphor and gesture. Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Coker, D. A. & Burgoon, J. (1987). The nature of conversational involvement and nonverbal encoding patterns. Human Communication Research, 13 (4), 463–494. Google Scholar logo with link to Google Scholar
Couper-Kuhlen, E. (2018). Finding a place for body movement in grammar. Research on Language and Social Interaction, 51 (1), 22–25. Google Scholar logo with link to Google Scholar
Crible, L. (2014). Identifying and describing discourse markers in spoken corpora: Annotation protocol v.8. Technical report. Louvain: Université catholique de Louvain.Google Scholar logo with link to Google Scholar
Crible, L. & Pascual, E. (2019). Combinations of discourse markers with repairs and repetitions in English, French and Spanish. Journal of Pragmatics, 1561, 54–67. Google Scholar logo with link to Google Scholar
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioural measurement: Theory of generalizability for scores and profiles. New York: Wiley.Google Scholar logo with link to Google Scholar
Degand, L., Cornillie, B., & Pietrandrea, P. (2013) (Eds.). Discourse markers and modal particles: Categorization and description. Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Ekman, P. & Friesen, W. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17 (2), 124–129. Google Scholar logo with link to Google Scholar
Fischer, K. (2013). Discourse markers. In Caroll Chapelle (Ed.), The encyclopedia of applied linguistics (Vol. 31, 1743–1748). Malden, MA & Oxford: Wiley & Blackwell.Google Scholar logo with link to Google Scholar
Fitzpatrick, E. (Ed.). (2007). Corpus linguistics beyond the word: Corpus research from phrase to discourse. Amsterdam & New York: Rodopi. Google Scholar logo with link to Google Scholar
Guaitella, I., Santi, S., Lagrue, B., & Cavé, C. (2009). Are eyebrow movements linked to voice variations and turn-taking in dialogue? An experimental investigation. Language and Speech, 52 (2/3), 207–222. Google Scholar logo with link to Google Scholar
Harrison, S., Adolphs, S., Dowens, M. G., Du, P., & Littlemore, J. (2018). All hands on deck. Negotiation over gesture forms in collaborative discourse. Lingua, 2071, 1–22. Google Scholar logo with link to Google Scholar
Hyland, K. (2005). Metadiscourse: Exploring interaction in writing. London & New York: Continuum.Google Scholar logo with link to Google Scholar
Keevallik, L. (2018). What does embodied interaction tell us about grammar? Research on Language and Social Interaction, 51 (1), 1–21. Google Scholar logo with link to Google Scholar
Kendon, A. (2014). Semiotic diversity in utterance production and the concept of ‘language’. Philosophical Transactions of the Royal Society, B 369 (1651), 20130293. Google Scholar logo with link to Google Scholar
Knight, D. (2011). Multimodality and active listenership: A corpus approach. London: Continuum.Google Scholar logo with link to Google Scholar
Kok, K. I. & Cienki, A. (2016). Cognitive grammar and gesture: Points of convergence, advances and challenges. Cognitive Linguistics, 27 (1), 67–100. Google Scholar logo with link to Google Scholar
Kushch, O., Igualada, A., & Prieto, P. (2018). Prominence in speech and gesture favour second language novel word learning. Language, Cognition and Neuroscience, 33 (8), 992–1004. Google Scholar logo with link to Google Scholar
Lenker, U. (2018). ‘There’s an issue there…’: Signalling functions of discourse-deictic there in the history of English. Language Sciences, 681, 94–105. Google Scholar logo with link to Google Scholar
Lester, J. C., Voerman, J. L., Towns, S. G., & Callaway, C. B. (1999). Deictic believability: Coordinated gesture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence, 13 (4/5), 383–414. Google Scholar logo with link to Google Scholar
Llanes-Coromina, J., Vilà-Giménez, I., Kushch, O., Borras-Comes, J., & Prieto, P. (2018). Beat gestures help preschoolers recall and comprehend discourse information. Journal of Experimental Child Psychology, 1721, 168–188. Google Scholar logo with link to Google Scholar
Lopes, A. C. M. (2016). Discourse markers. In L. Wetzels, S. Menuzzi, & J. Costa (Eds.), The handbook of Portuguese linguistics. New York: Wiley. Google Scholar logo with link to Google Scholar
Lopez-Ozieblo, R. & McNeill, D. (2017). Exchange on gesture-speech unity: What it is, where it came from. In R. Breckinridge Church, M. W. Alibali, & S. D. Kelly (Eds.), Why gesture? How the hands function in speaking, thinking and communicating (pp. 103–125). Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Maricchiolo, F., Gnisci, A., & Bonaiuto, M. (2012). Coding hand gestures: A reliable taxonomy and a multi-media support. In Anna Esposito et al. (Eds.), Cognitive behavioural systems (pp. 405–416). Berlin & Heidelberg: Springer. Google Scholar logo with link to Google Scholar
Maschler, Y. & Schiffrin, D. (2015). Discourse markers: Language, meaning, and context. In D. Tannen, H. E. Hamilton, & D. Schiffrin (Eds.), The handbook of discourse analysis (pp. 189–221). Hoboken, NJ: Wiley.Google Scholar logo with link to Google Scholar
McCarthy, M. (2003). Talking back: “Small” interactional response tokens in everyday conversation. Research on Language and Social Interaction, 36 (1), 33–63. Google Scholar logo with link to Google Scholar
McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22 (3), 276–282. Google Scholar logo with link to Google Scholar
McNeill, D. (2008). Gesture and thought. Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
McNeill, D., Cassell, J., & Levy, E. T. (1993). Abstract deixis. Semiotica, 95 (1/2), 5–20. Google Scholar logo with link to Google Scholar
McNeill, D., Levy, E., & Duncan, S. D. (2015). Gesture in discourse. In D. Tannen, H. E. Hamilton, & D. Schiffrin (Eds.), The handbook of discourse analysis (pp. 262–289). Hoboken, NJ: Wiley.Google Scholar logo with link to Google Scholar
Melinger, A. & Levelt, W. J. (2004). Gesture and the communicative intention of the speaker. Gesture, 4 (2), 119–141. Google Scholar logo with link to Google Scholar
Mlakar, I., Verdonik, D., Majhenič, S., & Rojc, M. (2019). Towards pragmatic understanding of conversational intent: A multimodal annotation approach to multiparty informal interaction – The EVA Corpus. In: International Conference on Statistical Language and Speech Processing. Proceedings (pp. 19–30). Cham: Springer. Google Scholar logo with link to Google Scholar
Nevile, M. (2015). The embodied turn in research on language and social interaction. Research on Language and Social Interaction, 48 (2), 121–151. Google Scholar logo with link to Google Scholar
Neville, D. O. (2015). The story in the mind: The effect of 3D gameplay on the structuring of written L2 narratives. ReCALL, 27 (1), 21–37. Google Scholar logo with link to Google Scholar
Nikolaeva, Y. (2014). Speech-accompanying gestures in Russian: functions and verbal context. In Proceedings of the Third Workshop on Vision and Language (pp. 82–86). Dublin: Dublin City University & Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Paggio, P. & Navarretta, C. (2017). The Danish NOMCO corpus: multimodal interaction in first acquaintance conversations. Language Resources and Evaluation, 51 (2), 463–494. Google Scholar logo with link to Google Scholar
Peirce, C. S. (1965). Collected papers, Vol. 51. Cambridge: Harvard University Press.Google Scholar logo with link to Google Scholar
Redeker, G. (1990). Ideational and pragmatic markers of discourse structure. Journal of Pragmatics, 14 (3), 367–381. Google Scholar logo with link to Google Scholar
Schiffrin, D. (1987). Discourse markers. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Schlenker, P. (2018). Iconic pragmatics. Natural Language & Linguistic Theory, 36 (3), 877–936. Google Scholar logo with link to Google Scholar
Schourup, L. (1999). Discourse markers. Lingua, 107 (3), 227–265. Google Scholar logo with link to Google Scholar
Trujillo, J. P., Simanova, I., Bekkering, H., & Özyürek, A. (2018). Communicative intent modulates production and comprehension of actions and gestures: A Kinect study. Cognition, 1801, 38–51. Google Scholar logo with link to Google Scholar
Verdonik, D. (2007). Upravljanje pogovora kot metadiskurzna funkcija. Metabesedilnost v uporabnem jezikoslovju. Jezik in slovstvo, 3 (4), 53–65.Google Scholar logo with link to Google Scholar
Verdonik, D., Rojc, M., & Stabej, M. (2007). Annotating discourse markers in spontaneous speech corpora on an example for the Slovenian language. Language Resources and Evaluation, 41 (2), 147–180. Google Scholar logo with link to Google Scholar
Verdonik, D., Žgank, A., & Pisanski Peterlin, A. (2008). The impact of context on discourse marker use in two conversational genres. Discourse Studies, 10 (6), 759–775. Google Scholar logo with link to Google Scholar
Vigliocco, G., Perniss, P., & Vinson, D. (2014). Language as a multimodal phenomenon: implications for language learning, processing and evolution. Philosophical Transactions of the Royal Society of London, B 369 (1651), 20130292.Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Chu, Weiwei & Yi Shan
2025. Trends and Highlights in Discourse Marker Studies From 2009 to 2022. Chinese Journal of Applied Linguistics 48:3 ► pp. 451 ff. DOI logo
Ben-Moshe, Yotam M. & Yael Maschler
2024. Hebrew clicks: From the periphery of language to the heart of grammar. Journal of Pragmatics 229 ► pp. 19 ff. DOI logo
Mlakar, Izidor, Darinka Verdonik, Simona Majhenič & Matej Rojc
2023. Understanding conversational interaction in multiparty conversations: the EVA Corpus. Language Resources and Evaluation 57:2 ► pp. 641 ff. DOI logo

This list is based on CrossRef data as of 9 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue