In:Multimodality and Cognitive Linguistics
Edited by María Jesús Pinar Sanz
[Benjamins Current Topics 78] 2015
► pp. 147–164
Multimodal digital storytelling
Integrating information, emotion and social cognition
Published online: 15 October 2015
https://doi.org/10.1075/bct.78.10alo
https://doi.org/10.1075/bct.78.10alo
Digital stories are a very recent multimedia practice by which ordinary people construct short narratives on personal affairs combining voice, images and sometimes music. This paper contributes to the description of this new emergent genre from both a multimodal and a cognitive point of view, by exploring how diverse semiotic channels in digital storytelling provide different kinds of information (factual, emotional, cultural, etc.) which are finally integrated to construct the global meaning of the narrative. For this purpose, we combine Kress and Van Leeuwen’s (1996) scholarly work related to multimodal representation, with the use of some notions of the Mental Spaces and Conceptual Integration theory (Dancygier, 2008; Fauconnier & Turner, 2002). The results of this study are of interest to those concerned with the representational and communicational modes of semiotic resources in storytelling.
References (20)
Cameron, L. (1999). Operationalising metaphor for applied linguistic research. In L. Cameron & G. Low (Eds.), Researching and applying metaphor (pp. 3–28). Cambridge: Cambridge University Press.
Dancygier, B. (2008). The text and the story: levels of blending in fictional narratives. In T. Oakley & A. Hougaard (Eds.), Mental spaces in discourse and interaction (pp. 51–78). Amsterdam: John Benjamins.
Emmott, C. (1997). Narrative comprehension: A discourse perspective. Oxford: Oxford University Press.
Fauconnier G., & Turner, M. (2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books.
Halliday, M.A.K., & Matthiessen, C.M. (2004). An introduction to functional grammar. Third revised edition. London: Edward Arnold.
Herman, D. (2003). Stories as a tool for thinking. In D. Herman (Ed.), Narrative theory and the cognitive sciences (pp. 163–192). Stanford: CSLI Publications.
. (2004). Toward a transmedial narratology. In M.L. Ryan (Ed.), Narrative across media: The languages of storytelling (pp. 47–75). Lincoln: University of Nebraska Press.
. (2009). Cognitive approaches to narrative analysis. In G. Brône & J. Vandaele (Eds.), Cognitive poetics: Goals, gains and gaps (pp. 79–118). Berlin: Mouton de Gruyter.
Kress, G., & van Leeuwen, T. (1996) (2nd edition 2006). Reading images: The grammar of visual design. London: Routledge.
Labov, W., & Waletzky, J. (1967). Narrative analysis: Oral versions of personal experience. In J. Helm (Ed.), Essays on the verbal and visual arts (pp. 12–44). Seattle, WA: University of Washington Press.
Lakoff, G., & Turner, M. (1989). More than cool reason: A field guide to poetic metaphor. Chicago: University of Chicago Press.
O’Halloran, K., & Lim Fei, V. (2009). Sequential visual discourse frames. In E. Ventola & A.J. Moya Guijarro (Eds.), The world told and the world shown: Multisemiotic issues (pp. 139–156). London: Palgrave Macmillan.
Oakley, T., & Coulson, S. (2008). Connecting the dots: Mental spaces and metaphoric language in discourse. In T. Oakley & A. Hougaard (Eds.), Mental spaces in discourse and interaction (pp. 27–50). Amsterdam: John Benjamins.
Porto, D., & Romano, M. (2010). Conceptual integration in natural oral narratives. Actes des journées d’étude Narratology and the new social dimension of narrative (01–02 Février 2010). At [URL] (last accessed 8th March 2013).
Ryan, M.L. (2005). On the theoretical foundations of transmedial narratology. In J.C. Meister (Ed.), Narratology beyond literary criticism: Mediality, disciplinarity (pp. 1–23). Berlin/New York: Walter de Gruyter.
Semino, E. (2009). Text worlds. In G. Brône & J. Vandaele (Eds.), Cognitive poetics: Goals, gains and gaps (pp. 33–37). Berlin: Mouton de Gruyter.
Cited by (2)
Cited by two other publications
Rodríguez Cuadrado, Ruth & Paloma Úbeda Mansilla
This list is based on CrossRef data as of 11 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.