Tracing relevance beyond codes and across modes: A multimodal pragmatic analysis of children’s rights advocacy campaign posters

Turath Awad Al Tamimi and Thulfiqar H. Altahmazi

Drawing on Relevance Theory, the paper sketches out a framework that accounts for inference-making in creative multimodal texts, taking advocacy campaign posters as its case study. The analysis shows that in each poster semiotic resources are employed to create a micro-narrative exemplifying actors affected by a sociopolitical problem, whose function is to create assumptions against which a higher-order intention is recognized. The text-internal relevance within the micro-narrative is optimized by combining verbal and visual elements to communicate multimodal explicatures and implicatures. The visual elements are employed to invoke non-propositional effects that activate perceptual mechanisms to maximize emotional attachment with the issue advocated for. These non-propositional effects communicated by visual connotation carriers are essential, rather than extra, elements, contributing to the understanding of the propositional meaning communicated at the text-external level. The analysis shows that an inferential approach to multimodality is indispensable to account for (non)propositional content across different modes.

Publication history
Table of contents

Technology has swept our world making it easier to produce and disseminate verbal, visual and aural content on various platforms. This makes communication in the twenty-first century increasingly multimodal. Such a tendency necessitates an inferential approach, rather than an approach based on the code model as in traditional multimodal analysis, wherein “codes and the tacit rules and constraints that underlie the production and interpretation of meaning within each code” are identified (Chandler 2017, 185; also see Kress 2010, 34–36). In pragmatics, text comprehension is often conceptualized as “a context building process” (Maillat 2013), through which contextual assumptions are made salient to help the addressees draw inferences about the text producer’s intended meanings. In multimodal texts, the context building process is rather more complex, as it requires inferring contextual assumptions based on the contents communicated across the different modes. Although such an inferential process is a pragmatic process par excellence, multimodality has tended to fall outside the interest of mainstream pragmatic scholarship. This is because pragmatics, broadly defined as the study of content-sensitive meaning, has traditionally been concerned with the study of verbal communication (Dicerto 2018, 37). Fortunately, the field has recently witnessed an increase in the number of studies applying pragmatic theories to account for the meaning communicated multimodally (e.g. Forceville 2020; Forceville and Clark 2014).

Full-text access is restricted to subscribers. Log in to obtain additional credentials. For subscription information see Subscription & Price. Direct PDF access to this article can be purchased through our e-platform.

References

Ariel, Mira
2010Defining Pragmatics. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Arnheim, Rudolph
1969Art and Visual Perception: A Psychology of the Creative Eye. Berkeley: University of California Press.Google Scholar logo with link to Google Scholar
Barthes, Roland
1977Rhetoric of the Image. In Image, Music, Text, transl. by Stephen Heath, 32–51. London: Macmillan.Google Scholar logo with link to Google Scholar
Bateman, John
2014Text and Image: A Critical Introduction to the Visual/Verbal Divide. New York: Routledge. Google Scholar logo with link to Google Scholar
Chandler, Daniel
2017Semiotics: The Basics. 3rd edition. New York: Routledge.Google Scholar logo with link to Google Scholar
Clark, Billy
2013Relevance Theory. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
2022Pragmatics the Basics. New York: Routledge.Google Scholar logo with link to Google Scholar
Cornevin, Vanessa, and Charles Forceville
2017 “From Metaphor to Allegory: The Japanese Manga Afuganisu-tan.” Metaphor and the Social World 7 (2): 235–251. Google Scholar logo with link to Google Scholar
Crisp, Peter
2005 “Allegory and Symbol — A Fundamental Opposition?Language and Literature 14: 323–338. Google Scholar logo with link to Google Scholar
Dicerto, Sara
2018Multimodal Pragmatics and Translation: A New Model for Source Text Analysis. London: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Falkum, Ingrid Lossius
2019 “Metaphor and Metonymy in Acquisition: A Relevance- Theoretic Perspective.” In Relevance, Pragmatics and Interpretation: Essays in Honour of Deirdre Wilson, ed. by Kate Scott, Billy Clark, and Robyn Carston, 205–217. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Fauconnier, Gilles, and Mark Turner
2002The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books.Google Scholar logo with link to Google Scholar
Finnegan, A. Cara
2001 “The Naturalistic Enthymeme and Visual Argument: Photographic Representation in the Skull Controversy.” Argumentation and Advocacy 37: 133–149. Google Scholar logo with link to Google Scholar
Forceville, Charles
2020Visual and Multimodal Communication: Applying the Relevance Principle. New York: Oxford University Press. Google Scholar logo with link to Google Scholar
Forceville, Charles, and Billy Clark
2014 “Can Pictures Have Explicatures?Linguagem em (Dis)curso 14: 451–472. Google Scholar logo with link to Google Scholar
Garcés-Conejos Blitvich, Pilar
2022 “Moral Emotions, Good Moral Panics, Social Regulation, and Online Public Shaming.” Language & Communication 84: 61–75. Google Scholar logo with link to Google Scholar
Grice, H. Paul
1975 “Logic and Conversation.” In Syntax and Semantics (Vol. 3): Speech Acts, ed. by Peter Cole, and Jerry L. Morgan, 41–58. New York: Academic Press. Google Scholar logo with link to Google Scholar
Jewitt, Carey
2013 “Multimodal Methods for Researching Digital Technologies.” In The SAGE Handbook of Digital Technology Research, eds. by Sara Price, Carey Jewitt, and Barry Brown, 240–260. London: SAGE. Google Scholar logo with link to Google Scholar
Kjeldsen, Jens
2015 “Where Is Visual Argument?” In Reflections on Theoretical Issues in Argumentation Theory, ed. by Frans H. van Eemeren, and Bart Garssen, 107–117. Cham: Springer. Google Scholar logo with link to Google Scholar
Kress, Gunther
2010Multimodality: A Social Semiotic Approach to Contemporary Communication. Oxon: Routledge.Google Scholar logo with link to Google Scholar
Lakoff, George, and Mark Johnson
1980Metaphors We Live By. Chicago: The University of Chicago Press.Google Scholar logo with link to Google Scholar
Leech, Geoffrey Neil
1985 “Stylistics.” In Discourse and Literature, ed. by Teun A. Van Dijk, 39–57. John Benjamins. Google Scholar logo with link to Google Scholar
Lindblom, Kenneth
2009 “Cooperative Principle.” In Concise Encyclopedia of Pragmatics, 2nd edition, ed. by Mey Jacob, 151–158. Oxford: Elsevier.Google Scholar logo with link to Google Scholar
Machin, David
2016Introduction to Multimodal Analysis. London: Bloomsbury.Google Scholar logo with link to Google Scholar
Maillat, Didier
2013 “Constraining Context Selection: On the Pragmatic Inevitability of Manipulation.” Journal of Pragmatics 59: 190–199. Google Scholar logo with link to Google Scholar
Markham, Annette, and Elizabeth Buchanan
2012 “Ethical Decision-Making and Internet Research: Recommendations from the AoIR Ethics Working Committee (Version 2.0).” https://​aoir​.org​/reports​/ethics2​.pdf
Marlow, Mikaela
2017 “Public Discourse and Intergroup Communication.” In Oxford Research Encyclopedia of Communication, ed. by Jon Nussbaum. Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
Martinec, Radan, and Andrew Salway
2005 “A System for Image-Text Relations in New (and Old) Media.” Visual Communication 4 (3): 337–371. Google Scholar logo with link to Google Scholar
McQuarrie, Edward F., and Barbara J. Phillips
2005 “Indirect Persuasion in Advertising: How Consumers Process Metaphors Presented in Pictures and Words.” Journal of Advertising 34 (2): 7–20. Google Scholar logo with link to Google Scholar
Moeschler, Jacques
2017 “Formal and Natural Languages: What Logic Tell Us About Natural Language.” In The Routledge Handbook of Pragmatics, ed. by Anne Barron, Yueguo Gu, and Gerard Steen, 241–256, Oxon: Routledge. Google Scholar logo with link to Google Scholar
O’Halloran, Kay L.
2011 “Multimodal Discourse Analysis.” In The Continuum Companion to Discourse Analysis, ed. by Ken Hyland, and Brian Paltridge, 120–137. London: Continuum.Google Scholar logo with link to Google Scholar
O’Halloran, Kay L., Sabine Tan, and Marissa K. L. E.
2013 “Multimodal Pragmatics.” In Pragmatics of Discourse, eds. by Klaus P. Schneider, and Anne Barron, 239–268. Berlin: De Gruyter Mouton.Google Scholar logo with link to Google Scholar
Pilkington, Adrian
2000Poetic Effects: A Relevance Theory Perspective. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Searle, John R.
1979Expression and Meaning: Studies in the Theory of Speech Acts. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Sperber, Dan and Deirdre Wilson
1987 “Presumptions of Relevance.” Behavioral and Brain Sciences 10 (4): 736–753. Google Scholar logo with link to Google Scholar
Sperber, Dan, and Deirdre Wilson
1995Relevance: Communication and Cognition. 2nd edition. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
2006 “Relevance Theory.” In The Handbook of Pragmatics, ed. by Laurence R. Horn, and Gregory Ward, 241–256. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
2008 “A Deflationary Account of Metaphors.” In The Cambridge Handbook of Metaphor and Thought, ed. by Raymond W. Gibbs, 84–105. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Terkourafi, Marina
2021 “Inference and Implicature.” In Cambridge Handbook of Sociopragmatics, ed. by Michael Haugh, Daniel Kadar, and Marina Terkourafi, 30–47. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Unger, Christoph
2019 “Allegory in Relation to Metaphor and Irony.” In Relevance, Pragmatics and Interpretation: Essays in Honour of Deirdre Wilson, ed. by Kate Scott, Billy Clark, and Robyn Carston, 240–252. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Wilson, Deirdre
2013 “Irony Comprehension: A Developmental Perspective.” Journal of Pragmatics 59: 40–56. Google Scholar logo with link to Google Scholar
2016 “Relevance Theory.” In The Oxford Handbook of Pragmatics, ed. by Yang Huang, 1–23. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
2017 “Irony, Hyperbole, Jokes and Banter”. In Formal Models in the Study of Language: Applications in Interdisciplinary Contexts, ed. by Joanna Blochowiak, Cristina Grisot, Stephanie Durrleman, and Christopher Laenzlinger, 201–219. Cham: Springer International Publishing. Google Scholar logo with link to Google Scholar
2018 “Relevance Theory and Literary Interpretation.” In Reading Beyond the Code: Literature and Relevance Theory, ed. by Terence Cave, and Deirdre Wilson, 185–204. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
2022 “Communication, Comprehension and Interpretation”. In Dynamism in Metaphor and Beyond (Festschrift for Ray Gibbs), ed. by Herbert Colston, Teenie Matlock, and Gerard Steen, 143–155. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Wilson, Deirdre, and Dan Sperber
2007 “On Verbal Irony.” In Irony in Language and Thought: A Cognitive Science Reader, ed. by Raymond Gobbs Jr., and Herbert Colston, 35–56. New York: Taylor and Francis.Google Scholar logo with link to Google Scholar
2012 “A Deflationary Account of Metaphor.” In Meaning and Relevance, ed. by Deirdre Wilson, and Dan Sperber, 97–122. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Wilson, Deirdre, and Robyn Carston
2019 “Pragmatics and the Challenge of ‘Non-Propositional’ Effects.” Journal of Pragmatics 145: 31–38. Google Scholar logo with link to Google Scholar
 
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue