Cover not available

In:Media as Procedures of Communication
Edited by Martin Luginbühl and Jan Georg Schneider
[Pragmatics & Beyond New Series 348] 2024
► pp. 16–39

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

Chapter 2
Mediality vs. materiality

A multimodal perspective on the notion of media as procedures

 | University of Groningen
Published online: 21 November 2024
DOI logo with link
https://doi.org/10.1075/pbns.348.02wil
Abstract
This contribution adds a multimodal perspective to the notions of medium and mediality in the context of media linguistics and critically reflects upon their conceptualisations as ‘technical infrastructures’ and/or ‘socially constituted semiotic procedures’. By critically assessing the different understandings of medium in media linguistics and multimodality research, it will point out advantages and disadvantages of each concept. In a second step, the definitions will be applied to practical examples of current media artifacts, i.e., a graphic novel and a digital learning environment. The application will demonstrate the need for a clear distinction between various levels of description for both semiotic modes as well as media.
Article outline
  • 1.Introduction
  • 2.Two understandings of ‘medium’
    • 2.1Media as procedures of sign processing
    • 2.2Media as sites for the deployment of semiotic modes
  • 3.Media and modes at work
    • 3.1Sign processing in comics
    • 3.2Sign processing in digital learning environments
  • 4.The mutual influence of medium and modes
  • Notes
  • References
References (39)
References
Baldrauf-Quilliatre, Heike, and Isabel Colón de Carvajal. 2019. “Encouragement in Videogame Interactions.” Social Interaction. Video-Based Studies of Human Sociality 2(2). Google Scholar logo with link to Google Scholar
Bateman, John A. 2013. “Multimodality and Film.” In The Encyclopedia of Applied Linguistics, edited by Carol A. Chapelle. London: Blackwell Publishing. Google Scholar logo with link to Google Scholar
2016. “Methodological and Theoretical Issues for the Empirical Investigation of Multimodality.” In Handbuch Sprache im multimodalen Kontext, edited by Nina-Maria Klug, and Hartmut Stöckl, 36–74. Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
Bateman, John A. 2017. “Triangulating Transmediality: A Multimodal Semiotic Framework Relating Media, Modes and Genres.” Discourse, Context & Media 20: 160–174. Google Scholar logo with link to Google Scholar
Bateman, John A. 2020. “When Holism Comes Back to Bite You: Comments on “Medial Shaping from the Outset” by Martin Luginbühl, and Jan Georg Schneider.” Journal für Medienlinguistik/Journal for Media Linguistics – Open Peer Review. [URL]
2021. “What are Digital Media?” Discourse, Context & Media 41: 100502. Google Scholar logo with link to Google Scholar
Bateman, John A., and Janina Wildfeuer. 2014a. “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics 74: 180–218. Google Scholar logo with link to Google Scholar
. 2014b. “Defining Units of Analysis for the Systematic Analysis of Comics: A Discourse-Based Approach.” Studies in Comics 5(2): 371–401. Google Scholar logo with link to Google Scholar
Bateman, John A., Janina Wildfeuer, and Tuomo Hiippala. 2017. Multimodality. Foundations, Research, Analysis. A Problem-Oriented Discussion. Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
Bender, Michael, Ruth M. Mell, and Janina Wildfeuer. 2022. “Zur Spezifik digitaler Medien als Diskursraum: Materialität, Daten, Affordanzen.” In Diskurse — digital. Theorien, Methoden, Fallstudien, edited by Eva Gredel, 27–45. Berlin, Boston: de Gruyter. Google Scholar logo with link to Google Scholar
Berlanga-Fernandez, Inmaculada, and Everardo Reyes. 2022. “The Digital Approach to Semiotics: A Systematic Review.” Text & Talk, ahead of print. Google Scholar logo with link to Google Scholar
Brock, Alexander, and Peter Schildhauer (eds). 2017. Communication Forms and Communicative Practices. New Perspectives on Communication Forms, Affordances and What Users Make of Them. Bern, Berlin: Lang.Google Scholar logo with link to Google Scholar
Cohn, Neil. 2013. The Visual Language of Comics. Introduction to the Structure and Cognition of Sequential Images. New York: Bloomsbury.Google Scholar logo with link to Google Scholar
Cohn, Neil, and Stephen Maher. 2015. “The Notion of the Motion. The Neurocognition of Motion Lines in Visual Narratives.” Brain Research 160: 73–84. Google Scholar logo with link to Google Scholar
Elleström, Lars. 2010. Media Borders. Multimodality and Intermediality. London: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Frobenius, Maximiliane (2014). “Audience Design in Monologues: How Vloggers Involve Their Viewers.” Journal of Pragmatics 72: 59–72. Google Scholar logo with link to Google Scholar
Holly, Werner. 2016. “Nachrichtenfilm als multimodale audiovisuelle Texte.” In Handbuch Sprache im multimodalen Kontext, edited by Nina-Maria Klug, and Hartmut Stöckl, 392–409. Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
Horton, Donald, and Wohl, Richard R. (1956). “Mass Communication and Para-Social Interaction: Observations on Intimacy at a Distance.” Psychiatry 19: 215–229. Google Scholar logo with link to Google Scholar
Klug, Nina-Maria, and Hartmut Stöckl. 2016. Handbuch Sprache im multimodalen Kontext. Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
Kress, Gunther, and Theo van Leeuwen. 2001. Multimodal Discourse. The Modes and Media of Contemporary Communication. London: Arnold.Google Scholar logo with link to Google Scholar
Luginbühl, Martin, and Jan Georg Schneider. 2020. “Medial Shaping from the Outset. On the Mediality of the Second Presidential Debate, 2016.” Journal für Medienlinguistik 3(1): 57–93. Google Scholar logo with link to Google Scholar
Mawil (2014). Kinderland. Berlin: Reprodukt.Google Scholar logo with link to Google Scholar
McCloud, Scott. 1994. Understanding Comics: The Invisible Art. New York: HarperPerennial.Google Scholar logo with link to Google Scholar
Nake, Frieder. 1994. “Human-Computer Interaction Viewed as Pseudo-Communication.” Knowledge-Based Systems 14: 441–447. Google Scholar logo with link to Google Scholar
O’Halloran, Kay L. 2004. “Visual Semiosis in Film.” In Multimodal Discourse Analysis: Systemic Functional Perspectives, edited by Kay L. O’Halloran, 109–130. London: Continuum.Google Scholar logo with link to Google Scholar
Packard, Stephan, Andreas Rauscher, Veronique Sina, Jan-Noel Thon, Lukas Wilde, and Janina Wildfeuer. 2019. Comicanalyse. Eine Einführung. Stuttgart: J.B. Metzler. Google Scholar logo with link to Google Scholar
Ryan, Marie-Laure. 2004. Narrative across Media: The Languages of Storytelling. Nebraska: University of Nebraska Press.Google Scholar logo with link to Google Scholar
Schmitz, Ulrich. 2018. “Media Linguistic Landscapes. Alle Linguistik sollte Medienlinguistik sein.” Journal Für Medienlinguistik / Journal for Media Linguistics 1(1): 1–34. Google Scholar logo with link to Google Scholar
. 2022. “Kann und soll Linguistik angesichts multimodaler Kommunikation eine eigene Disziplin sein?” Osnabrücker Beiträge zur Sprachtheorie 100: 11–28. Google Scholar logo with link to Google Scholar
Schneider, Jan Georg. 2008. Spielräume der Medialität. Linguistische Gegenstandskonstitution aus medientheoretischer und pragmatischer Perspektive. Berlin, New York: de Gruyter.Google Scholar logo with link to Google Scholar
. 2017. “Medien als Verfahren der Zeichenprozessierung. Grundsätzliche Überlegungen zum Medienbegriff und ihre Relevanz für die Gesprächsforschung.” Gesprächsforschung — Online-Zeitschrift zur verbalen Interaktion 18: 34–55. [URL]Google Scholar logo with link to Google Scholar
Unsworth, Len. 2007. “Image/Text Relations and Intersemiosis: Towards Multimodal Text Description for Multiliteracies Education.” In Proceedings of the 33rd International Systemic Functional Congress (33rd ISFC), edited by Leila Barbara, and Tony B. Sardinha, 1165–1205. [URL]
Wildfeuer, Janina. 2012. “Intersemiosis in Film: Towards a New Organisation of Semiotic Resources in Multimodal Filmic Text.” Multimodal Communication 1(3): 233–304. Google Scholar logo with link to Google Scholar
. 2018. “It’s All about Logics?! Analyzing the Rhetorical Structure of Multimodal Filmic Text.” Semiotica 220: 95–121. Google Scholar logo with link to Google Scholar
. 2019. “The Inferential Semantics of Comics: Panels and their Meanings.” Poetics Today 40(2): 215–234. Google Scholar logo with link to Google Scholar
. 2021. “Discourse Semantics and Textual Logic: Methodological Considerations for Multimodal Analysis.” In OBST — Osnabrücker Beiträge zur Sprachtheorie 99. Special Issue on Linguistic and Multimodality, edited by Arne Krause, and Ulrich Schmitz, 87–114.Google Scholar logo with link to Google Scholar
Wildfeuer, Janina, and John A. Bateman. 2018. “Theoretische und methodologische Perspektiven des Multimodalitätskonzepts aus linguistischer Sicht.” IMAGE — Zeitschrift Für Interdisziplinäre Bildwissenschaft 28: 5–46.Google Scholar logo with link to Google Scholar
Wildfeuer, Janina, Jana Pflaeging, John A. Bateman, Ognyan Seizov, and Chiao-I Tseng. 2019. “Multimodality: Disciplinary Thoughts and the Challenge of Diversity. Introduction.” In Multimodality: Disciplinary Thoughts and the Challenge of Diversity, edited by Janina Wildfeuer, Jana Pflaeging, John A. Bateman, Ognyan Seizov, and Chiao-I Tseng, 3–38. Berlin: de Gruyter. Google Scholar logo with link to Google Scholar
Winkler, Hartmut. 2008. Basiswissen Medien. Frankfurt a.M.: FischerGoogle Scholar logo with link to Google Scholar
Cited by (1)

Cited by one other publication

Thomson, T. J., Daniel Pfurtscheller, Katharina Lobinger, Nataliia Laba & Katharina Christ
2025. Visual literacies in transition: multimodal co-production with generative AI. Journal of Visual Literacy ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 29 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue