Cover not available

In:Exploring Discourse Strategies in Social and Cognitive Interaction: Multimodal and cross-linguistic perspectives
Edited by Manuela Romano and M. Dolores Porto
[Pragmatics & Beyond New Series 262] 2016
► pp. 111–135

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

The construction of meaning in multimodal discourse

A digital story as a case study

 | Technical University of Madrid
Published online: 31 March 2016
DOI logo with link
https://doi.org/10.1075/pbns.262.05mol
This chapter’s main goal is to shed light onto the most characteristic meaning making processes in digital storytelling by thoroughly analyzing a single digital narrative from a functional-cognitive perspective. This mixed approach for a case study allows the analysts to unveil the macro and micro discourse strategies and the cognitive processes developed by both the narrator and the audience to produce and interpret the multiple meanings conveyed by the different modes in a digital story. We believe findings presented here can be of interest for discourse analysts, cognitivists and researchers working in multimodality.
References (43)
References
Alexander, Brian and Alan Levine. 2008. “Web 2.0 Storytelling: Emergence of a New Genre”. EDUCAUSE Review 43 (6): 40–48.Google Scholar logo with link to Google Scholar
Alonso Belmonte, Isabel, Silvia Molina Plaza and M. Dolores Porto Requejo. 2013. “Multimodal digital storytelling: Integrating information, emotion and social cognition”. In The Annual Review of Cognitive Linguistics, Special issue “Multimodality and Cognitive Linguistics”, 11 (2): 369–385. Amsterdam/Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Baldry, Anthony P. and Paul J. Thibault. 2006. Multimodal Transcription and Text Analysis. London and New York: Equinox.Google Scholar logo with link to Google Scholar
Barthes, Roland. 1977. Image, Music, Text. London: Fontana.Google Scholar logo with link to Google Scholar
Bateman, John A. 2008. Multimodality and Genre. A Foundation for the Systematic Analysis of Multimodal Documents. Basingstoke: Palgrave MacMillan.Google Scholar logo with link to Google Scholar
. 2014. Text and Image: A Critical Introduction to the Visual/Verbal Divide. London and New York: Routledge.Google Scholar logo with link to Google Scholar
Bateman, John A. and Janina Wildfeuer. 2014. “A Multimodal Discourse Theory of Visual Narrative.” Journal of Pragmatics 74: 180–208. Google Scholar logo with link to Google Scholar
Burgess, Jean. 2006. “Hearing Ordinary Voices”. Continuum: Journal of Media & Culture 20 (2): 201–214. Google Scholar logo with link to Google Scholar
Cohn, Neil. 2013. Visual Narrative Structure. Cognitive Science 37 (3): 413–452. Google Scholar logo with link to Google Scholar
Dancygier, Barbara. 2008. “The text and the story: levels of blending in fictional narratives.” In Mental Spaces in Discourse and Interaction, ed. by Todd Oakley and Anders Hougaard, 51–78. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
. 2012. The Language of Stories: A Cognitive Approach. Cambridge: 
Cambridge University Press.Google Scholar logo with link to Google Scholar
Fauconnier, Giles and Mark Turner. 2002. The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books.Google Scholar logo with link to Google Scholar
Forceville, Charles and Eduardo Urios-Aparisi (eds). 2009. Multimodal Metaphor. Berlin: 
Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Gleason, Christina. 2012. “5 Brands Taking Digital Storytelling to the Next Level”. Ignite Social Media at [URL] (last accessed, January 2015).
Handler Miller, Carolyn. 2014 [2004]. Digital Storytelling: A Creator’s Guide to Interactive Entertainment. New York and London: Focal Press.Google Scholar logo with link to Google Scholar
Hühn, Peter, Wolf Schmid and Jörg Schönert (eds). 2009. Point of View, Perspective, and Focalization: Modeling Mediation in Narrative. Berlin: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Jewitt, Carey. (ed). 2009. The Routledge Handbook of Multimodal Analysis. London: Routledge.Google Scholar logo with link to Google Scholar
Kövecses, Zoltan. 2002. Metaphor: A Practical Introduction. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Kress, Gunther. 2010. Multimodality. A Social Semiotic Approach to Contemporary Communication. London: Routledge.Google Scholar logo with link to Google Scholar
Kress, Gunther and Theo J. van Leeuwen. 2001. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.Google Scholar logo with link to Google Scholar
. 2006 [1996]. Reading Images: The Grammar of Visual Design. London: Routledge.Google Scholar logo with link to Google Scholar
Lakoff, George and Mark Turner. 1989. More than Cool Reason: A Field Guide to Poetic Metaphor. Chicago: University of Chicago Press. Google Scholar logo with link to Google Scholar
Lambert, Joe. 2013. Digital Storytelling: Capturing Lives, Creating Community. London and New York: Routledge.Google Scholar logo with link to Google Scholar
Lundby, Knut. (ed). 2008. Digital Storytelling, Mediatized Stories: Self-Representations in New Media. New York: Peter Lang Publishing.Google Scholar logo with link to Google Scholar
McLellan, Hilary. 2007. “Digital Storytelling in Higher Education.” Journal of Computing in Higher Education 19 (1): 65–79. Google Scholar logo with link to Google Scholar
McNeill, David. 2005. Gesture and Thought. Chicago: Chicago University Press. Google Scholar logo with link to Google Scholar
Oakley, Todd and Anders Hougaard (eds). 2008. Mental Spaces in Discourse and Interaction. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
O’Halloran, Kay L. (ed). 2004. Multimodal Discourse Analysis: Systemic Functional Perspectives. London: Bloomsbury Academic.Google Scholar logo with link to Google Scholar
. 2008. “Systemic Functional Multimodal Discourse Analysis (SFMDA): Constructing Ideational Meaning Using Language and Visual Imagery.” Visual Communication 7 (4): 443–475. Google Scholar logo with link to Google Scholar
O’Halloran, Kay L. and Bradley A. Smith. 2011. “Multimodal Text Analysis.” In The Encyclopedia of Applied Linguistics, ed. by Carol A. Chapelle. Oxford, UK: Wiley-Blackwell.Google Scholar logo with link to Google Scholar
Porto Requejo, M. Dolores. 2016. “Music in Multimodal Narratives: The Role of the Soundtrack in Digital Stories”. In Audionarratology: Interfaces of Sound and Narrative, ed. by Jarmila Mildorf and Till Kinzel, 29–46. Berlin: De Gruyter.
Porto, M. Dolores and Manuela Romano. 2010. “Conceptual Integration in Natural Oral Narratives”. Actes des Journées D’étude “Narratology and the New Social Dimension of Narrative. [URL] (last accessed January 2015).Google Scholar logo with link to Google Scholar
Porto Requejo, M. Dolores and Isabel Alonso Belmonte. 2014. “From Local to Global: Visual Strategies of Glocalisation in Digital Storytelling.“ Language and Communication 39: 14–23. Google Scholar logo with link to Google Scholar
Roland, Craig. 2006. Digital Stories in the Classroom. School Art 105 (7): 26.Google Scholar logo with link to Google Scholar
Rossiter, Marsha and Penny A. Garcia. 2010. “Digital Storytelling: A New Player on the Narrative Field.” New Directions for Adult and Continuing Education 126: 37–48. Google Scholar logo with link to Google Scholar
Tara La Rose, Janice. 2012. “Digital Media Stories through Multimodal Analysis: A Case Study of Erahoneybee’s Song about a Child Welfare Agency.” Journal of Technology in Human Services 30 (3–4): 299–311. Google Scholar logo with link to Google Scholar
Tseng, Chiao. 2013. Cohesion in film: Tracking film elements. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Unsworth, Len and Chris Cleirigh. 2009. “Multimodality and Reading: The Construction of Meaning through Image-Text Interaction.” In The Routledge Handbook of Multimodal Analysis, ed. by Carey Jewitt, 151–163. London: Routledge.Google Scholar logo with link to Google Scholar
Velasco, Olga I. 2001. “Metaphor, Metonymy, and Image-Schemas: an Analysis of Conceptual Interaction Patterns.” Journal of English Studies 3: 47–63.Google Scholar logo with link to Google Scholar
Zacks, Jeffrey M. and Barbara Tversky. 2001. “Event Structure in Perception and Conception.” Psychological Bulletin 127 (1): 3–21 Google Scholar logo with link to Google Scholar
Zwaan, Rolf A. 1999. “Five Dimensions of Narrative Comprehension: The Event-Indexing Model.” In Narrative Comprehension, Causality, and Coherence: Essays in Honor of Tom Trabasso, ed. by Susan R. Goldman, Arthur C. Graesser and Paul van den Broek, 93–110. Mahwah, NJ, US: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Zwaan, Rolf A., Mark C. Langston and Arthur C. Graesser. 1995. “The Construction of Situation Models in Narrative Comprehension: An Event-Indexing Model.” Psychological Science 6 (5): 292–297. Google Scholar logo with link to Google Scholar
Zwaan, Rolf A. and Gabriel A. Radvansky. 1998. “Situation Models in Language Comprehension and Memory.” Psychological Bulletin 123 (2): 162–185. Google Scholar logo with link to Google Scholar
Cited by (2)

Cited by two other publications

Dong, Huahua & Junxi Yao
2025. Technological Empowerment and Meaning Co-Construction: The Reinforced Persuasion Mechanism of Multimodal Synergy in Smart Product Launch Events. Journal of Theoretical and Applied Electronic Commerce Research 20:4 ► pp. 304 ff. DOI logo
Krysanova, Tetiana
2024. Unveiling the eco-storytelling: multimodal layers of meaning in Greenpeace’s environmental videos. Cognition, Communication, Discourse :28 ► pp. 33 ff. DOI logo

This list is based on CrossRef data as of 28 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue