Cover not available

Article published In: Language, Context and Text
Vol. 7:1 (2025) ► pp.1–41

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Multimodality writes back

Refining the contextual variable of “mode” in systemic functional linguistics

ORCiD logo with link | Bremen University, Germany
Published online: 5 September 2025
DOI logo with link
https://doi.org/10.1075/langct.00082.bat
Abstract
This paper critically engages with the treatment of “context” in SFL, particularly the contextual variable of mode, arguing that approaches to date have been hindered by focusing too narrowly on language. Although this might sound reasonable for a linguistic theory, the discussion here will suggest that overlooking the fact that language never occurs in context alone compromises the task of characterising context appropriately. By drawing on the state of the art in more recent multimodality theory, the paper works back to language from a more general semiotic framework in which occurrences of language are no longer central. This creates a broader theoretical “space” for the phenomena at issue that allows a re-positioning of some of the traditional distinctions drawn in discussions of contextual mode, arguably simplifying contextual modelling within a more internally coherent architecture.
Article outline
  • 1.Introduction: “Context” as a task of linguistics
  • 2.Context in SFL: The starting position
  • 3.Semiotic fault lines within the SFL notion of contextual mode
  • 4.Context: A “pan-semiotic” contruct?
  • 5.Conclusions
  • Acknowledgements
  • Notes
  • References
References (91)
References
Bartlett, Tom. 2017. Context in systemic functional linguistics: Towards scalar supervenience? In Tom Bartlett & Gerard O’Grady (eds.), The Routledge handbook of systemic functional linguistics, 375–390. London: Routledge. Google Scholar logo with link to Google Scholar
Bateman, John A. 2017. Triangulating transmediality: A multimodal semiotic framework relating media, modes and genres. Discourse, Context & Media 201 (Supplement C). 160–174. Google Scholar logo with link to Google Scholar
2020. The foundational role of discourse semantics beyond language. In Michele Zappavigna & Shoshana Dreyfus (eds.), Discourses of hope and reconciliation. On J. R. Martin’s contribution to systemic functional linguistics, 39–55. London & New York: Bloomsbury. Google Scholar logo with link to Google Scholar
2021a. Dimensions of materiality: Towards an external language of description for empirical multimodality research. In Jana Pflaeging, Janina Wildfeuer & John A. Bateman (eds.), Empirical multimodality research: Methods, evaluations, implications, 35–64. Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
2021b. What are digital media? Discourse, Context & Media 41Aop. (100502). Google Scholar logo with link to Google Scholar
2022a. Growing theory for practice: Empirical multimodality beyond the case study. Multimodal Communication 11(1). 63–74. Google Scholar logo with link to Google Scholar
2022b. A semiotic perspective on the ontology of documents and multimodal textuality. In Gerald Hartung, Frederik Schlupkothen & Karl-Heinrich Schmidt (eds.), Using documents: A multidisciplinary approach to document theory, 147–198. Berlin: De Gruyter. Google Scholar logo with link to Google Scholar
Bateman, John A., Janina Wildfeuer & Tuomo Hiippala. 2017. Multimodality — foundations, research and analysis. A problem-oriented introduction. Berlin: de Gruyter Mouton.Google Scholar logo with link to Google Scholar
Berry, Margaret. 2016. On describing contexts of situation. In Wendy L. Bowcher & Jennifer Yameng Liang (eds.), Society in language, language in society: Essays in honour of Ruqaiya Hasan, 184–205. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Bezemer, Jeff. 2024. What modes can and cannot do: Affordance in Gunther Kress’s theory of sign making. Text & Talk 44(4). 493–510. Google Scholar logo with link to Google Scholar
Björkvall, Anders & Anna-Malin Karlsson. 2011. The materiality of discourses and the semiotics of materials: A social perspective on the meaning potentials of written texts and furniture. Semiotica 187(1/4). 141–165. Google Scholar logo with link to Google Scholar
Bowcher, Wendy L. 2007. Field and multimodal texts. In Ruqaiya Hasan, Christian M. I. M. Matthiessen & Jonathan J. Webster (eds.), Continuing discourse on language, vol. 21, 619–646. London: Equinox.Google Scholar logo with link to Google Scholar
2013. Material action as choice in field. In Lise Fontaine, Tom Bartlett & Gerard O’Grady (eds.), Systemic functional linguistics: Exploring choice, 318–341. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
2017. Field, tenor and mode. In Tom Bartlett & Gerard O’Grady (eds.), The Routledge handbook of systemic functional linguistics, 391–403. London: Routledge.Google Scholar logo with link to Google Scholar
2021. Context of situation and the role of language. Functions of Language 28(3). 315–341. Google Scholar logo with link to Google Scholar
Bucher, Hans-Jürgen. 2011. Multimodales Verstehen oder Rezeption als Interaktion. Theoretische und empirische Grundlagen einer systematischen Analyse der Multimodalität. In Hans-Joachim Diekmannshenke, Michael Klemm & Hartmut Stöckl (eds.), Bildlinguistik. Theorien — Methoden — Fallbeispiele, 123–156. Berlin: Erich Schmidt.Google Scholar logo with link to Google Scholar
Burn, Andrew. 2014. The kineikonic mode: Towards a multimodal approach to moving-image media. In Carey Jewitt (ed.), The Routledge handbook of multimodal analysis (2nd edition), 375–385. London: Routledge.Google Scholar logo with link to Google Scholar
Burunat, Iballa, Daniel J. Levitin & Petri Toiviainen. 2024. Breaking (musical) boundaries by investigating brain dynamics of event segmentation during real-life music-listening. Proceedings of the National Academy of Sciences 121(36). e2319459121. Google Scholar logo with link to Google Scholar
Butt, David & Rebekah Wegener. 2007. The work of concepts: Context and metafunction in the systemic functional model. In Ruqaiya Hasan, Christian M. I. M. Matthiessen & Jonathan J. Webster (eds.), Continuing discourse on language, 589–618. London: Equinox.Google Scholar logo with link to Google Scholar
Butt, David G. 2001. Firth, Halliday and the development of systemic functional theory. In Sylvain Auroux, E. F. K. Koerner, Hans-Josef Niederehe & Kees Versteegh (eds.), History of the language sciences: An international handbook on the evolution of the study of language from the beginnings to the present, vol. 21 (Handbooks of Linguistics and Communication Science 18.2), 1806–1838. Berlin & New York: Walter de Gruyter.Google Scholar logo with link to Google Scholar
2004. Parameters of context: On establishing similarities and dissimilarities between social processes (Mimeo). Sydney: Centre for Language in Social Life, Macquarie University.Google Scholar logo with link to Google Scholar
2019. Firth and the origins of systemic functional linguistics: Process, pragma, and polysystem. In Geoff Thompson, Wendy L. Bowcher, Lise Fontaine & David Schöntal (eds.), The Cambridge handbook of systemic functional linguistics, 11–34. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Butt, David G., Alison Rotha Moore, Canzhong Wu & John Cartmill. 2021. The pragmatism of drawing context networks. Social hierarchy and social distance as dimensions of tenor. Functions of Language 28(3). 260–290. Google Scholar logo with link to Google Scholar
Cloran, Carmel. 1999. Context, material situation and text. In Mohsen Ghadessy (ed.), Text and context in functional linguistics, 177–217. Amsterdam & Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
Cohn, Neil. 2015. Narrative conjunction’s junction function: The interface of narrative grammar and semantics in sequential images. Journal of Pragmatics 881. 105–132. Google Scholar logo with link to Google Scholar
. 2020. Your brain on comics: A cognitive model of visual narrative comprehension. Topics in Cognitive Science 12(1). 352–386. Google Scholar logo with link to Google Scholar
Cohn, Neil & Joost Schilperoord. 2022. Reimagining language. Cognitive Science 46(7). e13174. Google Scholar logo with link to Google Scholar
Doran, Y. J., J. R. Martin & Michele Herrington. 2024. Rethinking context: realisation, instantiation, and individuation in systemic functional linguistics. Journal of World Languages. 10(1). 177–220. Google Scholar logo with link to Google Scholar
Elleström, Lars. 2014. Media transformation: The transfer of media characteristics among media. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
(ed.). 2021. Beyond media borders, volume 1: Intermedial relations among multimodal media. Basingstoke: Palgrave Macmillan.Google Scholar logo with link to Google Scholar
Firth, John R. 1957[1950]. Personality and language in society. In Papers in linguistics 1934-1951, 177–189. London: Oxford University Press. Reprinted from The Sociological Review: Journal of the Institute of Sociology 42(2): 37–52.Google Scholar logo with link to Google Scholar
Fricke, Ellen. 2013. Towards a unified grammar of gesture and speech: A multimodal approach. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill & Sedinha Tessendorf (eds.), Body — Language — Communication / Körper — Sprache — Kommunikation (Handbücher zur Sprach- und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK) 38/1), 733–754. Berlin & New York: de Gruyter Mouton. Google Scholar logo with link to Google Scholar
Gibson, James J. 1977. The theory of affordances. In Robert Shaw & John Bransford (eds.), Perceiving, acting, and knowing: Toward an ecological psychology, 62–82. Hillsdale, NJ: Erlbaum.Google Scholar logo with link to Google Scholar
Halliday, M. A. K. 1978. Language as social semiotic: The social interpretation of language and meaning. London: Edward Arnold.Google Scholar logo with link to Google Scholar
1999. The notion of ‘context’ in language education. In Mohsen Ghadessy (ed.), Text and context in functional linguistics, 1–20. Amsterdam & Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
Halliday, M. A. K. & Ruqaiya Hasan. 1985. Language, context and text: Aspects of language in a social semiotic perspective. Geelong, Victoria: Deakin University Press. (Language and Learning Series).Google Scholar logo with link to Google Scholar
Halliday, M. A. K. & Christian M. I. M. Matthiessen. 2013. Halliday’s introduction to functional grammar (4th edition). London & New York: Routledge. Google Scholar logo with link to Google Scholar
Halliday, M. A. K., Angus McIntosh & Peter Strevens. 1964. The linguistic sciences and language teaching. London: Longman.Google Scholar logo with link to Google Scholar
Halliday, M. A. K. & William S. Greaves. 2008. Intonation in the grammar of English. London: Equinox Publishing Co.Google Scholar logo with link to Google Scholar
Hasan, Ruqaiya. 1985. The structure of a text. In M. A. K. Halliday & Ruqaiya Hasan (eds.), Language, context and text: A social semiotic perspective, 52–69. Geelong, Victoria: Deakin University Press. (Language and Learning Series).Google Scholar logo with link to Google Scholar
. 1995. The conception of context in text. In Peter H. Fries & Michael Gregory (eds.), Discourse in society: Systemic functional perspectives, 183–283. Norwood, NJ: Ablex.Google Scholar logo with link to Google Scholar
. 1996. The nursery tale as a genre. In C. Cloran, D. Butt & G. Williams (eds.), Ways of saying, ways of meaning: selected papers of Ruqaiya Hasan, 51–72. London: Cassell.Google Scholar logo with link to Google Scholar
. 1999. Speaking in reference to context. In Mohsen Ghadessy (ed.), Text and context in functional linguistics, 219–319. Amsterdam & Philadelphia: Benjamins. Google Scholar logo with link to Google Scholar
. 2009. The place of context in a systemic functional model. In M. A. K. Halliday & Jonathan J. Webster (eds.), Continuum companion to systemic functional linguistics, 166–189. London: Continuum.Google Scholar logo with link to Google Scholar
. 2014. Towards a paradigmatic description of context: Systems, metafunctions, and semantics. Functional Linguistics 1(9). 1–54. Google Scholar logo with link to Google Scholar
. 2016[2001]. Wherefore context? The ontogenesis of meaning exchange. In Ruqaiya Hasan, Context in the system and process of language, volume 4 in the collected works of Ruqaiya Hasan, 95–126. Edited by Jonathan J. Webster. London: Equinox.Google Scholar logo with link to Google Scholar
. 2020. Tenor: Rethinking interactant relations. Language, Context and Text: The Social Semiotics Forum 2(2). 213–333. Google Scholar logo with link to Google Scholar
Jackendoff, Ray & Fred Lerdahl. 2006. The capacity for music: What is it, and what’s special about it? Cognition 1001. 33–72. Google Scholar logo with link to Google Scholar
Jewitt, Carey. 2014. An introduction to multimodality. In Carey Jewitt (ed.), The Routledge handbook of multimodal analysis, 15–30. London: Routledge.Google Scholar logo with link to Google Scholar
Jewitt, Carey, Jeff Bezemer & Kay O’Halloran. 2016. Introducing multimodality. London: Routledge. Google Scholar logo with link to Google Scholar
Jewitt, Carey, Gunther Kress, Jon Ogborn & Charalampos Tsatsarelis. 2000. Materiality as an aspect in learning. Zeitschrift für Erziehungswissenschaft 3(2). 267–284. Google Scholar logo with link to Google Scholar
Krämer, Sybille. 1998. Form als Vollzug oder: Was gewinnen wir mit Niklas Luhmanns Unterscheidung von Medium und Form? Rechtshistorisches Journal 171. 558–573.Google Scholar logo with link to Google Scholar
Kress, Gunther. 2003. Literacy in the new media age. London: Routledge.Google Scholar logo with link to Google Scholar
. 2023[1999]. A response to ‘A theory of educational linguistics’. Language, Context and Text: The Social Semiotics Forum 2(2). 403–412. Republished version of paper presented in 1999.Google Scholar logo with link to Google Scholar
Kress, Gunther, Carey Jewitt, Jon Ogborn & Charalampos Tsatsarelis. 2001. Multimodal teaching and learning. The rhetorics of the science classroom. London & New York: Continuum.Google Scholar logo with link to Google Scholar
Kress, Gunther & Theo van Leeuwen. 2001. Multimodal discourse: The modes and media of contemporary communication. London: Arnold.Google Scholar logo with link to Google Scholar
. 2020[1996, 2006]. Reading images: The grammar of visual design (3rd edition). London & New York: Routledge. Google Scholar logo with link to Google Scholar
Ledin, Per & David Machin. 2019. Doing critical discourse studies with multimodality: From metafunctions to materiality. Critical Discourse Studies 16(5). 497–513. Google Scholar logo with link to Google Scholar
Luhmann, Niklas. 1984. Soziale systeme. Grundriß einer allgemeinen theorie. Frankfurt: Suhrkamp.Google Scholar logo with link to Google Scholar
Lukin, Annabelle, Alison Moore, Maria Herke, Rebekah Wegener & Canzhong Wu. 2008. Halliday’s model of register revisited and explored. Linguistics and the Human Sciences 4(2). 187–213. Google Scholar logo with link to Google Scholar
Malinowski, Branislaw. 1923. The problem of meaning in primitive languages. In Charles K. Ogden & Ivor A. Richards, The meaning of meaning, 451–510. New York: Harcourt, Brace, and Co., Inc.Google Scholar logo with link to Google Scholar
1999. Modelling context: The crooked path of progress in contextual linguistics (Sydney SFL). In M. Ghadessy (ed.), Text and context in functional linguistics (CILT Series IV), 25–61. Amsterdam: Benjamins. Google Scholar logo with link to Google Scholar
2011. Multimodal semiotics: Theoretical challenges. In Shooshi Dreyfus, Sue Hood & Maree Stenglin (eds.), Semiotic margins: Reclaiming meaning, 243–270. London: Continuum.Google Scholar logo with link to Google Scholar
2019[2017]. Revisiting Field: Specialized knowledge in secondary school science and humanities discourse. In J. R. Martin (ed.), Accessing academic discourse, 114–147. London & New York: Routledge. Originally appeared in Onomázein Número especial LSF y TCL sobre educación y conocimiento (2017): 111–148.Google Scholar logo with link to Google Scholar
Martin, J. R. & Erika Matruglio. 2019[2013]. Revisiting Mode: Context in/dependency in ancient history classroom discourse. In J. R. Martin (ed.), Accessing academic discourse, 89–113. London & New York: Routledge. Originally appeared in Guowen Huang, Delu Zhang & Xinzhang Yang (eds.), Studies in functional linguistics and discourse analysis, volume 51, 2–95. Beijing: Higher Education Press.Google Scholar logo with link to Google Scholar
Martin, J. R. & David Rose. 2008. Genre relations: Mapping culture. London and New York: Equinox.Google Scholar logo with link to Google Scholar
Matthiessen, Christian M. I. M. 1993. Register in the round, or diversity in a unified theory of register. In M. Ghadessy (ed.), Register analysis: Theory and practice, 221–292. London: Pinter.Google Scholar logo with link to Google Scholar
2009. Multisemiosis and context-based register typology: Registerial variation in the complementarity of semiotic systems. In Eija Ventola & Arsenio Jesús Moya Guijarro (eds.), The world told and the world shown: Multisemiotic issues, 11–38. Basingstoke: Palgrave Macmillan.Google Scholar logo with link to Google Scholar
2015. Register in the round: Registerial cartography. Functional linguistics 2(9). 1–48. Google Scholar logo with link to Google Scholar
Matthiessen, Christian M. I. M., Kazuhiro Teruya & Marvin Lam. 2010. Key terms in systemic functional linguistics. London: Continuum.Google Scholar logo with link to Google Scholar
Miller, Daniel. 2005. Materiality: An introduction. In Daniel Miller (ed.), Materiality, 1–50. Durham, NC: Duke University Press.Google Scholar logo with link to Google Scholar
Mondada, Lorenza. 2014. Interactional space and the study of embodied talk-in-interaction. In Peter Auer, Martin Hilpert, Anja Stukenbrock & Benedikt Szmrecsanyi (eds.), Space in language and linguistics. Geographical, interactional, and cognitive perspectives, 247–275. Berlin: de Gruyter.Google Scholar logo with link to Google Scholar
Moore, Alison Rotha. 2020. Progress and tensions in modelling register as a semantic configuration. Language, Context and Text: The Social Semiotics Forum 2(1). 22–57. Google Scholar logo with link to Google Scholar
Muntigl, Peter. 2004. Modelling multiple semiotic systems: The case of gesture and speech. In Eija Ventola, Cassily Charles & Martin Kaltenbacher (eds.), Perspectives on multimodality, 31–50. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Ngo, Thu, Susan Hood, J. R. Martin, Claire Painter, Bradley A. Smith & Michelle Zappavigna (eds.). 2021. Modelling paralanguage using systemic functional semiotics: Theory and application. London: Bloomsbury Academic.Google Scholar logo with link to Google Scholar
O’Halloran, Kay L. 2005. Mathematical discourse: Language, symbolism and visual images. London & New York: Continuum.Google Scholar logo with link to Google Scholar
Patel-Grosz, Pritty, Patrick Georg Grosz, Tejaswinee Kelkar & Alexander Refsum Jensenius. 2022. Steps towards a Semantics of Dance. Journal of Semantics 39(4). 693–748. Google Scholar logo with link to Google Scholar
Pfeiffer, K. Ludwig. 1994. The materiality of communication. (Translated by William Whobrey). In Hans Ulrich Gumbrecht & K. Ludwig Pfeiffer (eds.), Materialities of communication, 1–14. Stanford University Press.Google Scholar logo with link to Google Scholar
Ravelli, Louise. 2019. Multimodality and the register of disciplinary History. Challenges for new texts and old theories. Language, Context and Text: The Social Semiotics Forum 1(2). 341–365.Google Scholar logo with link to Google Scholar
Ravelli, Louise, Theo van Leeuwen, Markus A. Höllerer & Dennis Jancsary (eds.). 2023. Organizational semiotics: Multimodal perspectives on organization studies. New York & London: Routledge. Google Scholar logo with link to Google Scholar
Schlenker, Phillippe. 2017. Outline of Music Semantics. Music perception 351. 3–37. Google Scholar logo with link to Google Scholar
Stöckl, Hartmut. 2004. In between modes: Language and image in printed media. In Eija Ventola, Cassily Charles & Martin Kaltenbacher (eds.), Perspectives on multimodality, 9–30. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
. 2016. Multimodalität — Semiotische und textlinguistische Grundlagen. In Nina-Maria Klug & Hartmut Stöckl (eds.), Handbuch sprache im multimodalen kontext (Handbücher Sprachwissen / Linguistics Handbooks (HSW) 7), 3–35. Berlin: de Gruyter Mouton. Google Scholar logo with link to Google Scholar
Streeck, Jürgen, Charles Goodwin & Curtis LeBaron. 2011. Embodied interaction in the material world: An introduction. In Jürgen Streeck, Charles Goodwin & Curtis LeBaron (eds.), Embodied interaction: Language and body in the material world, 1–28. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
van Leeuwen, Theo. 1999. Speech, music, sound. London: Macmillan. Google Scholar logo with link to Google Scholar
. 2005. Introducing social semiotics. London: Routledge.Google Scholar logo with link to Google Scholar
. 2022. Multimodality and identity. London & New York: Routledge.Google Scholar logo with link to Google Scholar
Wolf, George. 1989. Malinowski’s ‘context of situation’. Language & Communication 9(4). 259–267. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue