Cover not available

In:Multimodal Communication from a Construction Grammar Perspective
Edited by Kiki Nikiforidou and Mirjam Fried
[Constructional Approaches to Language 38] 2025
► pp. 1–24

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

Grammar and multimodality

An introduction

ORCiD logo with link | National and Kapodistrian University of Athens
ORCiD logo with link | Charles University
Published online: 16 January 2025
DOI logo with link
https://doi.org/10.1075/cal.38.intro
Article outline
  • 1.Common ground
  • 2.Integrating non-verbal features into grammar: Questions and (some) answers
    • 2.1Which non-verbal features to integrate
    • 2.2Non-verbal correlates in the present chapters
  • 3.How to code prosodic and gestural features
  • 4.How to integrate prosodic/phonetic and gesture features into grammar
    • 4.1Sound
    • 4.2Gesture
  • 5.Conclusion and outlook
  • Acknowledgements
  • Notes
  • References
References (78)
References
Antonopoulou, E., & Nikiforidou, K. (2011). Construction Grammar and conventional discourse: A construction-based approach to discoursal incongruity. Journal of Pragmatics, 43(10), 2594–2609. Google Scholar logo with link to Google Scholar
Βjörkvall, A. (2012). Multimodality. In J. Verschueren, J.-O. Östman, J. Blommaert, & C. Bulcaen (Eds.), Handbook of Pragmatics online volume. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Boas, H. (2013). Cognitive Construction Grammar. In Th. Hoffmann, & G. Trousdale (Eds.), The Oxford Handbook of Construction Grammar (pp. 233–254). Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
Borras-Comes, J., del Mar Vanrell, M., & Prieto, P. (2014). The role of pitch range in establishing intonational contrasts. Journal of International Phonetic Association, 44(1), 1–20. Google Scholar logo with link to Google Scholar
Bressem, J., & Ladewig, S. (2011). Rethinking gesture phases: Articulatory features of gestural movement? Semiotica, 184, 53–91. Google Scholar logo with link to Google Scholar
Bressem, J., & Müller, C. (2017). The Negative-Assessment-Construction — A multimodal pattern based on a recurrent gesture? Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Cienki, A. (2017). Utterance Construction Grammar (UCxG) and the variable multimodality of constructions. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Cienki, A. J., & Müller, C. (Eds.). (2008). Metaphor and gesture. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Clark, H. (1973). Space, time, semantics, and the child. In T. E. Moore (Ed.), Cognitive development and the acquisition of language (pp. 27–63). New York: Academic Press. Google Scholar logo with link to Google Scholar
Croft, W., & Cruse, D. A. (2004). Cognitive Linguistics. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Debras, C., & Cienki, A. (2012). Some uses of head tilts and shoulder shrugs during human interaction, and their relation to stancetaking. Paper presented at the International Conference on Social Computing.
Elvira-García, W. (2019). Two constructions, one syntactic form: Perceptual prosodic differences between elliptical and independent <si + V indicative> clauses in Spanish. In K. Beijering, G. Kaltenböck, & M. S. Sansiñena (Eds.), Insubordination: Theoretical and empirical issues (pp. 240–264). Berlin: De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Elvira-García, W., Roseano, P., & Fernández-Planas, A. M. (2017). Prosody as a cue for syntactic dependency. Evidence from dependent and independent clauses with subordination marks in Spanish. Journal of Pragmatics, 109, 29–46. Google Scholar logo with link to Google Scholar
Feyaerts, K., Brône, G., & Oben, B. (2017). Multimodality in interaction. In B. Dancygier (Ed.), The Cambridge Handbook of Cognitive Linguistics (pp. 135–156). Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Fillmore, C. (1981). Pragmatics and the description of discourse. In P. Cole (Ed.), Radical pragmatics (pp. 143–166). New York: Academic Press.Google Scholar logo with link to Google Scholar
Fillmore, C. J., Kay, P., & O’Connor, M. C. (1988). Regularity and idiomaticity in grammatical constructions: The Case of Let Alone. Language, 64(3), 501–538. Google Scholar logo with link to Google Scholar
Fried, M. (2015). Construction Grammar. In A. Alexiadou, & T. Kiss (Eds.), Handbook of Syntax, vol. 4 (pp. 974–1003). Berlin: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Fried, M., & Machač, P. (2022). Intonation as a cue to epistemic stance in one type of insubordinate clauses. Folia Linguistica, 56(1), 183–214. Google Scholar logo with link to Google Scholar
Fried, M., & Östman, J-O. (2004). Construction Grammar: a thumbnail sketch. In M. Fried, & J-O. Östman (Eds.), Construction Grammar in a cross-language perspective (pp. 11–86). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
(2005). Construction Grammar and spoken language: the case of pragmatic particles. Journal of Pragmatics, 37(11), 1752–1778. Google Scholar logo with link to Google Scholar
Goldberg, A. E. (1995). Constructions: A Construction Grammar approach to argument structure. Chicago: University of Chicago Press.Google Scholar logo with link to Google Scholar
Goldberg, A. (2003). Constructions: A new theoretical approach to language. Trends in Cognitive Sciences 7(5), 219–224. Google Scholar logo with link to Google Scholar
Gras, P. (in press). Insubordination at the interaction of discourse, grammar, and prosody. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Gras, P., & Elvira-Garcia, W. (2021). The role of intonation in Construction Grammar: On prosodic constructions. Journal of Pragmatics, 180, 232–247. Google Scholar logo with link to Google Scholar
Hinnell, J., & Rice, S. (2019). The embodied marking of stance in North American English discourse: Stacked and idiomatic. Presented at the 15th International Cognitive Linguistics Conference, Kwansei Gakuin University, Nishinomiya, Japan.
Janzen, T. (2012). Lexicalization and grammaticalization. In M. Steinbach, R. Pfau, & B. Woll (Eds.), Sign language. An international handbook (pp. 816–840). Berlin: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Jehoul, A., Brône, G., & Feyaerts, K. (2017). The shrug as marker of obviousness. Corpus evidence from Dutch face-to-face conversations. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Jun, S.-A. (Ed.). (2005). Prosodic typology: The phonology of intonation and phrasing. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Kay, P., & Fillmore, C. J. (1999). Grammatical constructions and linguistic generalizations: The What’s X doing Y? construction. Language, 75, 1–33. Google Scholar logo with link to Google Scholar
Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), Nonverbal communication and language (pp. 207–227). The Hague: Mouton.Google Scholar logo with link to Google Scholar
(2004). Gesture. Visible action as utterance. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Kress, G., & Van Leeuwen, T. (2006). Reading images: The grammar of visual design. London/ New York: Routledge. Google Scholar logo with link to Google Scholar
Ladd, R. (2008). Intonational phonology. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lakoff, G. (1987). Women, fire and dangerous things. What categories reveal about the mind. Chicago: Chicago University Press. Google Scholar logo with link to Google Scholar
Lambrecht, K. (1994). Information structure and sentence form: Topic, focus, and the mental representations of discourse referents. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Lanwer, J. P. (2017). Apposition: A multimodal construction? The multimodality of linguistic constructions in the light of usage-based theory. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Levinson, S. C. (2006). On the human ‘interaction engine’. In N. J. Enfield, & S. C. Levinson (Eds.), Roots of human sociality: Culture, cognition and interaction (pp. 39–69). Oxford, UK: Berg.Google Scholar logo with link to Google Scholar
Levinson, S. C., & Holler, J. (2014). The origin of human multi-modal communication. Philological Transactions Royal Society, B, 369: 20130302. Google Scholar logo with link to Google Scholar
Liddell, S. K. (2011). Agreement disagreements. Theoretical Linguistics, 37(3–4), 161–172. Google Scholar logo with link to Google Scholar
Lindström, J., & Londen, A.-M. (2008). Constructing reasoning: The connectives för att (causal), så att (consecutive) and men att (adversative) in Swedish conversations. In J. Leino (Ed.), Constructional reorganization (pp. 105–152). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Linell, P. (2009). Grammatical constructions in dialogue. In A. Bergs, & G. Diewald (Eds.), Contexts and constructions (pp. 97–110). Amsterdam & Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Lorenz, D. (2013). Semi-modal contractions in English: The emancipating effect of frequency. Freiburg: Rombach.Google Scholar logo with link to Google Scholar
Lücking, A. (2021). Gesture. In S. Müller, A. Abeillé, R. D. Borsley, & J.-P. Koenig (Eds.), Head-Driven Phrase Structure Grammar: The handbook (pp. 1201–1250). Berlin: Language Science Press. Google Scholar logo with link to Google Scholar
Lücking, A., Ginzburg, J., & Cooper, R. (2021). Grammar in dialogue. In S. Müller, A. Abeillé, R. D. Borsley, & J.-P. Koenig (Eds.), Head-Driven Phrase Structure Grammar: The handbook (pp. 1155–1199). Berlin: Language Science Press. Google Scholar logo with link to Google Scholar
Marandin, J.-M. (2007). Contours as constructions. Constructions Online.Google Scholar logo with link to Google Scholar
McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Google Scholar logo with link to Google Scholar
(2005). Gesture and thought. Chicago: University of Chicago Press. Google Scholar logo with link to Google Scholar
Michaelis, L., & Lambrecht, K. (1996). Toward a construction-based theory of language function: The case of nominal extraposition. Language, 72, 215–247. Google Scholar logo with link to Google Scholar
Mittelberg, I. (2014). Multimodal existential constructions in German and English. Presentation at GCLA 6, Erlangen, Nurnberg.
(2017). Multimodal existential constructions in German: Manual actions of giving as experiential substrate for grammatical and gestural patterns. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Müller, C. (2004). Forms and uses of the Palm Up Open Hand: A case of a gesture family? In C. Müller, & R. Posner (Eds.), The semantics and pragmatics of everyday gesture (pp. 233–256). Berlin: Weidler Verlag.Google Scholar logo with link to Google Scholar
Niebuhr, O. (2015). Stepped intonation contours: A new field of complexity. In R. Skarnitzl, & O. Niebuhr (Eds.), Tackling the complexity of speech (pp. 39–74). Prague: Charles University Press.Google Scholar logo with link to Google Scholar
(2019). Pitch accents as multiparametric configurations of prosodic features: Evidence from pitch-accent specific micro-rhythms in German. In A. M. Nyvad (Ed.), A sound approach to language matters: In honor of Ocke-Schwen Bohn (pp. 321–351). Aarhus: Aarhus University.Google Scholar logo with link to Google Scholar
Niebuhr, O., & Ward, N. G. (2018). Challenges in the modeling of pragmatics-related prosody: Introduction to the special issue. Journal of the International Phonetic Association, 48, 1–8. Google Scholar logo with link to Google Scholar
Nikiforidou, K. (2021). Grammatical variability and the grammar of genre: Constructions, conventionality, and motivation in ‘stage directions’. Journal of Pragmatics, 173, 189–199. Google Scholar logo with link to Google Scholar
Nikiforidou, K., Marmaridou, S., & Mikros, G. K. (2014). What’s in a dialogic construction? A constructional approach to polysemy and the grammar of challenge. Cognitive Linguistics, 25(4), 655–699. Google Scholar logo with link to Google Scholar
Ogden, R. (2010). Prosodic constructions in making complaints. In D. Barth-Weingarten, E. Reber, & M. Selting (Eds.), Prosody in interaction (pp. 81–103). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Poldvere, N., & Paradis, C. (2020). ‘What and then a little robot brings it to you?’ The reactive what-x construction in spoken dialogue. English Language and Linguistics, 24(2), 307–332. Google Scholar logo with link to Google Scholar
Sag, I. (2012). Sign-Based Construction Grammar: An informal synopsis. In H. Boas, & I. Sag (Eds.), Sign-Based Construction Grammar. Stanford: CSLI Publications.Google Scholar logo with link to Google Scholar
Schoonjans, S. (2014). Modalpartikeln als multimodale Konstruktionen. Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. KU Leuven dissertation. Leuven.
(2017). Multimodal Construction Grammar issues are Construction Grammar issues. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Sommerer, L., & Van de Velde, F. (in press). Constructional networks. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Sweetser, E. (2009). What does it mean to compare language and gesture? Modalities and contrasts. In J. Guo, E. Lieven, & N. Budwig (Eds.), Crosslinguistic approaches to the psychology of language: Studies in the tradition of Dan Isaac Slobin (pp. 357–366). New York: Psychology Press. Google Scholar logo with link to Google Scholar
Tabacaru, S., & Lemmens, M. (2014). Raised eyebrows as gestural triggers in humour: The case of sarcasm and hyper-understanding. European Journal of Humour Research, 2(22),11–31. Google Scholar logo with link to Google Scholar
Uhrig, P. (2019). Crossmodal collostructions — A usage-based approach to multimodal communication in a Construction Grammar framework. Paper presented at the 52nd Annual Meeting of the Societas Linguistica Europaea, Leipzig.
Ward, N. G. (2019). Prosodic patterns in English conversation. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
(in press). Prosodic constructions. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Wells, J. (2006). English intonation: An introduction. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
(2005). Routes from gesture to language. Revista da ABRALIN — Associação Brasileira de Lingüística, 4, 11–45.Google Scholar logo with link to Google Scholar
Wilcox, S., & Martínez, R. (in press). Constructional approaches to signed language. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Ziem, A. (2017). Do we really need a Multimodal Construction Grammar? Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Zima, E. (2017). On the multimodality of [all the way from X PREP Y]. Linguistics Vanguard Vol. 3, 2016–0055. Google Scholar logo with link to Google Scholar
(in press). Construction Grammar and gesture. In M. Fried, & K. Nikiforidou (Eds.), The Cambridge Handbook of Construction Grammar. Cambridge: Cambridge University Press.
Zima, E., & Bergs, A. (2017). Multimodality and construction grammar. Linguistics Vanguard, 3(s1). Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue