Cover not available

In:Multimodal Communication from a Construction Grammar Perspective
Edited by Kiki Nikiforidou and Mirjam Fried
[Constructional Approaches to Language 38] 2025
► pp. 220–250

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

Towards a functional perspective on multimodal constructions

Evidence from requesting and stance-related Tell me about it

 | University of Potsdam
Published online: 16 January 2025
DOI logo with link
https://doi.org/10.1075/cal.38.07leh
Abstract
The paper argues in favor of taking a functional perspective on constructions. To this end, a stance-related construction, Tell me about it, and its multimodal delivery in spoken language will be analyzed on a larger scale using the NewsScape Library of International Television News (Steen & Turner, 2013). The quantitative analysis shows that stance-related Tell me about it is longer in duration, accompanied by head movements and raised eyebrows, and the withdrawal of eye contact with the recipient. These findings will eventually be treated as evidence for the view that (multimodal) constructions and some unimodal, nonverbal constructions form a network of functionally related constructions.
Article outline
  • 1.Introduction
  • 2.Construction Grammar and the multimodality of interactions
  • 3.The ambiguity of Tell me about it
  • 4.Multimodal stance-taking
  • 5.Method
    • 5.1A note on the methodological approach
    • 5.2The dataset
    • 5.3Variables and values
    • 5.4Statistical analysis
  • 6.Results
  • 7.Prototypical instances of requesting and stance-related Tell me about it
  • 8.Interim conclusions on multimodal aspects of Tell me about it
  • 9.Why Tell me about it is a multimodal construction (with prototype structure)
    • 9.1Frequencies
    • 9.2High (accumulated) cue validity of features
    • 9.3Why not other features?
    • 9.4Speakers pay attention to features
  • 10.Towards a function-first perspective
  • 11.Summary and conclusions
  • Notes
  • References
References (73)
References
Arppe, A., Gilquin, G., Glynn, D., Hilpert, M., & Zeschel, A. (2010). Cognitive corpus linguistics: Five points of debate on current theory and methodology. Corpora, 5(1), 1–27. Google Scholar logo with link to Google Scholar
Auer, P. (1992). Introduction: John Gumperz’ approach to contextualization. In P. Auer, & A. Di Luzio (Eds.), The contextualization of language (pp. 39–53). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. Google Scholar logo with link to Google Scholar
Boersma, P., & Weenink, D. (2019). Praat: Doing phonetics by computer (Version 6.0. 52 [Windows]). Amsterdam.Google Scholar logo with link to Google Scholar
Bonaiuto, M., & Bonaiuto, T. (2014). Gestures and body language in Southern Europe: Italy. In M. Cornelia, C. Alan, F. Ellen, L. Silva, M. David, & B. Jana (Eds.), Body — Language — Communication, Vol. 2 (pp. 1240–1253). Berlin: De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Brône, G., Oben, B., Jehoul, A., Vranjes, J., & Feyaerts, K. (2017). Eye gaze and viewpoint in multimodal interaction management. Cognitive Linguistics, 28(3), 449–483. Google Scholar logo with link to Google Scholar
Bryant, G. A. (2010). Prosodic contrasts in ironic speech. Discourse Processes, 47(7), 545–566. Google Scholar logo with link to Google Scholar
Bryant, G. A., & Fox Tree, J. E. (2005). Is there an ironic tone of voice? Language and Speech, 48(3), 257–277. Google Scholar logo with link to Google Scholar
Caucci, G. M., & Kreuz, R. J. (2012). Social and paralinguistic cues to sarcasm. Humor, 25(1), 1–22. Google Scholar logo with link to Google Scholar
Cheang, H. S., & Pell, M. D. (2008). The sound of sarcasm. Speech communication, 50(5), 366–381. Google Scholar logo with link to Google Scholar
Cienki, A. (2017). Utterance Construction Grammar (UCxG) and the variable multimodality of constructions. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Clift, R. (2021). Embodiment in dissent: The eye roll as an interactional practice. Research on Language and Social Interaction, 54(3), 261–276. Google Scholar logo with link to Google Scholar
Colston, H. L. (2020). Eye-rolling, irony and embodiment. In A. Athanasiadou, & H. L. Colston (Eds.), The Diversity of irony (pp. 211–235). Berlin/Boston: de Gruyter Mouton. Google Scholar logo with link to Google Scholar
Couper-Kuhlen, E. (2016). A sequential approach to affect: The case of ‘disappointment’. In M. Haakana, M. Laakso, & J. Lindström (Eds.), Talk in interaction: Comparative dimensions (pp. 94–123). Helsinki: Studia Fennica Linguistica.Google Scholar logo with link to Google Scholar
Couper-Kuhlen, E., & Selting, M. (2018). Interactional linguistics: Studying language in social interaction. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
De Saussure, F. (1916). Cours de linguistique générale. Lausanne: Payot.Google Scholar logo with link to Google Scholar
(2021). Multimodal profiles of je (ne) sais pas in spoken French. Journal of Pragmatics, 182, 42–62. Google Scholar logo with link to Google Scholar
Debras, C., & Cienki, A. (2012). Some uses of head tilts and shoulder shrugs during human interaction, and their relation to stancetaking. Paper presented at the 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Conference on Social Computing.
Diessel, H. (2019). The grammar network. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Dix, C., & Groß, A. (2021). Raising both eyebrows in interaction [Paper presentation]. International Pragmatics Conference, online. [URL]
Doherty-Sneddon, G., & Phelps, F. G. (2005). Gaze aversion: A response to cognitive or social difficulty? Memory & Cognition, 33(4), 727–733. Google Scholar logo with link to Google Scholar
Ekman, P., & Friesen, W. V. (1978). The facial action coding system: A technique for The measurement of facial movement. San Francisco: Consulting Psychologists Press.Google Scholar logo with link to Google Scholar
Feyaerts, K., Brône, G., & Oben, B. (2017). Multimodality in interaction. In B. Dancygier (Ed.), The Cambridge Handbook of Cognitive Linguistics (pp. 135–156). Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Goldberg, A. E. (2006). Constructions at work. The nature of generalizations in language. New York: Oxford University Press. Google Scholar logo with link to Google Scholar
Gries, S. T., & Stefanowitsch, A. (2004). Extending collostructional analysis: A corpus-based perspective on alternations. International Journal of Corpus Linguistics, 9(1), 97–129. Google Scholar logo with link to Google Scholar
Gumperz, J. J. (1992a). Contextualization and understanding. In A. Duranti, & C. Goodwin (Eds.), Rethinking context. Language as an interactive phenomenon (pp. 229–253). Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
(1992b). Contextualization revisited. In P. Auer, & A. Di Luzio (Eds.), The contextualization of language (pp. 39–53). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Heritage, J. (2012). Epistemics in action: Action formation and territories of knowledge. Research on Language & Social Interaction, 45(1), 1–29. Google Scholar logo with link to Google Scholar
Heritage, J., & Raymond, G. (2005). The terms of agreement: Indexing epistemic authority and subordination in talk-in-interaction. Social Psychology Quarterly, 68(1), 15–38. Google Scholar logo with link to Google Scholar
Hoffmann, T. (2017). Multimodal constructs-multimodal constructions? The role of constructions in the working memory. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Hutchby, I., & Wooffitt, R. (2008). Conversation analysis. Malden, MA: Polity Press.Google Scholar logo with link to Google Scholar
Imo, W. (2007). Der Zwang zur Kategorienbildung: Probleme der Anwendung der Construction Grammar bei der Analyse gesprochener Sprache. Gesprächsforschung-Online-Zeitschrift zur verbalen Interaktion, 8, 22–45.Google Scholar logo with link to Google Scholar
Imo, W., & Lanwer, J. P. (2020). Prosodie und Konstruktionsgrammatik. Berlin: De Gruyter. Google Scholar logo with link to Google Scholar
Langacker, R. W. (2013). Essentials of Cognitive Linguistics. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
(2017). Entrenchment in Cognitive Grammar. In H.-J. Schmid (Ed.), Entrenchment and the psychology of language learning: How we reorganize and adapt linguistic knowledge (pp. 39–56). Berlin: De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Lanwer, J. P. (2017). Apposition: A multimodal construction? The multimodality of linguistic constructions in the light of usage-based theory. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Lehmann, C. (2023a). Multimodal markers of irony in televised discourse. A corpus-based approach. In L. Brown, I. Hübscher, & A. H. Jucker (Eds.), Multimodal im/politeness. Signed, spoken, written (pp. 251–274). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
(2023b). As if that wasn’t enough: English as if clauses as multimodal utterance constructions. English Language & Linguistics, 27(1), 175–202. Google Scholar logo with link to Google Scholar
(2024). The prosody of irony is diverse and sometimes construction-specific. In M. Schlechtweg (Ed.), Interfaces of phonetics (pp. 281–308). Berlin: DeGruyter. Google Scholar logo with link to Google Scholar
Lerner, G. (2003). Selecting next speaker: The context-sensitive operation of a context-free organization. Language in Society, 32(2), 177–201. Google Scholar logo with link to Google Scholar
Lüdecke, D. (2021). sjPlot: Data visualization for statistics in social science (Version R package version 2.8.10). Retrieved from [URL]
Mauchand, M., Vergis, N., & Pell, M. (2018). Ironic tones of voices. Paper presented at the 9th International Conference on Speech Prosody.
McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. Google Scholar logo with link to Google Scholar
Mondada, L. (2018). Multiple temporalities of language and body in interaction: Challenges for transcribing multimodality. Research on Language and Social Interaction, 51(1), 85–106. Google Scholar logo with link to Google Scholar
Oxford University Press. (n.d.). Tell, v. In Oxford English Dictionary. Retrieved June 29, 2022, from [URL]
Perniss, P. (2018). Why we should study multimodal language. Frontiers in Psychology, 9. Google Scholar logo with link to Google Scholar
Pomerantz, A. (1984). Agreeing and disagreeing with assessments: Some features of preferred/dispreferred turn shaped. In J. M. Atkinson, & J. Heritage (Eds.), Structures of social action (pp. 57–101). Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
R Core Team (2019). R: A language and environment for statistical computing: R foundation for statistical computing. Retrieved from [URL]
Raymond, G., & Heritage, J. (2006). The epistemics of social relations: Owning grandchildren. Language in Society, 35(5), 677–705. Google Scholar logo with link to Google Scholar
Rockwell, P. (2000). Lower, slower, louder: Vocal cues of sarcasm. Journal of Psycholinguistic Research, 29(5), 483–495. Google Scholar logo with link to Google Scholar
Rossano, F. (2013). Gaze in conversation. In J. Sidnell, & T. Stivers (Eds.), The Handbook of Conversation Analysis (pp. 308–329). Chichester: Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Rossi, G. (2012). Bilateral and unilateral requests: The use of imperatives and mi X? interrogatives in Italian. Discourse Processes, 49(5), 426–458. Google Scholar logo with link to Google Scholar
Schegloff, E. A. (1996). Turn organization: One intersection of grammar and interaction. In E. Ochs, E. Schegloff, & S. Thompson (Eds.), Interaction and grammar (pp. 52–133). Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Schoonjans, S. (2018). Modalpartikeln als multimodale Konstruktionen: Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. Berlin: De Gruyter. Google Scholar logo with link to Google Scholar
Selting, M., Auer, P., Barth-Weingarten, D., Bergmann, J. R., Bergmann, P., Birkner, K., Couper-Kuhlen, E., Deppermann, A., Gilles, P., Günthner, S., Hartung, M., Kern, F., Mertzlufft, C., Meyer, C., Morek, M., Oberzaucher, F., Peters, J., Quasthoff, U., Schütte, W., & Uhmann, S. (2009). Gesprächsanalytisches Transkriptionssystem 2 (GAT 2). Gesprächsforschung: Online-Zeitschrift zur verbalen Interaktion, 10, 353–402.Google Scholar logo with link to Google Scholar
Steen, F., & Turner, M. B. (2013). Multimodal construction grammar. In M. Borkent, B. Dancygier, & J. Hinnell (Eds.), Language and the creative mind (pp. 255–274). Stanford: CSLI Publications. Google Scholar logo with link to Google Scholar
Stefanowitsch, A. (2003). The English imperative: A construction-based approach [Unpublished manuscript].Google Scholar logo with link to Google Scholar
(2013). Collostructional analysis. In T. Hoffmann, & G. Trousdale (Eds.), The Oxford Handbook of Construction Grammar (pp. 290–306). New York: Oxford University Press. Google Scholar logo with link to Google Scholar
Tabacaru, S. (2019). A multimodal study of sarcasm in interactional humor. Berlin: de Gruyter Mouton. Google Scholar logo with link to Google Scholar
(2020). Faces of sarcasm. Exploring raised eyebrows with sarcasm in French political debates. In A. Athanasiadou, & H. L. Colston (Eds.), The diversity of irony (pp. 256–277). Berlin/Boston: de Gruyter Mouton. Google Scholar logo with link to Google Scholar
Tabacaru, S., & Lemmens, M. (2014). Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding. The European Journal of Humour Research, 2(2), 11–31. Google Scholar logo with link to Google Scholar
Taylor, J. R. (2019). Prototype effects in grammar. In E. Dąbrowska, & D. Divjak (Eds.), Cognitive Linguistics. Key topics (pp. 127–147). Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
Uhrig, P. (2018). Multimodal constructions or crossmodal collostructions? Paper presented at the International Conference on Construction Grammar(s), Paris.
(2019). Crossmodal collostructions — A usage-based approach to multimodal communication in a Construction Grammar framework. Paper presented at the 52nd Annual Meeting of the Societas Linguistica Europaea, Leipzig.
(2021). Multimodal corpus linguistics and machine learning Methods for multimodal communication research. Paper presented at the 2021 International Conference on Multimodal Communication, Changsha. [URL]
Vigliocco, G., Perniss, P., & Vinson, D. (2014). Language as a multimodal phenomenon: Implications for language learning, processing and evolution. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 369, 20130292. Google Scholar logo with link to Google Scholar
Ward, N. G. (2019). Prosodic patterns in English conversation. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., & Sloetjes, H. (2006). ELAN: A professional framework for multimodality research. Paper presented at the 5th International Conference on Language Resources and Evaluation (LREC 2006).
Wootton, A. J. (1997). Interaction and the development of mind. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Zima, E. (2017). Multimodal constructional resemblance. In F. Ruiz de Mendoza Ibáñez, A. Luzondo Oyón, & P. Pérez Sobrino (Eds.), Constructing families of constructions: Analytical perspectives and theoretical challenges (pp. 301–337). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
(submitted). Multimodale Analysen zu Fremd — und Selbstwahl in ZOOM-Videokonferenzen. In S. Kabatnik, L. Bülow, M.-L. Merten, & R. Mroczynski (Eds.), Pragmatik multimodal. Tübingen: Narr.
Cited by (2)

Cited by two other publications

Lehmann, Claudia
2025. Oh, multimodality where art thou?. Constructions and Frames DOI logo
Schoonjans, Steven & Beatrix Schönherr
2025. Frequency issues in multimodal Construction Grammar revisited. Nota Bene 2:1 ► pp. 200 ff. DOI logo

This list is based on CrossRef data as of 11 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue