Cover not available

Article published In: Target
Vol. 36:2 (2024) ► pp.184–214

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

The multimodal translation workshop as a method of creative inquiry

Acousmatic sound, affective perception, and experiential literacy

ORCiD logo with link | University of Edinburgh
ORCiD logo with link | King’s College London
Published online: 1 March 2024
DOI logo with link
https://doi.org/10.1075/target.22051.cam
Abstract
This article investigates the role of affective perception in the development of translation and experiential literacy through the medium of a multimodal translation workshop held with twelve arts practitioners, academics, and translators. Both the rationale for the workshop format and the interpretation of workshop outputs draw on a transdisciplinary framework spanning theories of intermediality and multimodality, the study of acousmatic sound, acoustic atmospheres, and corporeal music/sound reception. Adopting a phenomenographic approach, we discuss the role of the body and the senses in communication and how the sensory exercises developed for our workshop can provide access to the prenoetic nature of perception from both a cognitive and affective standpoint. Recognizing the narrative quality of participants’ comments, a deductive approach was taken to analyze their translations and reflections through the lens of narrative modes of acousmatic music. The article concludes with pedagogical implications on the basis of participants’ reflections. Our findings support the use of a multimodal online translation workshop as both a research method to investigate meaning-making and a pedagogical resource to develop experiential literacy for both practitioners and developing translators.
Article outline
  • 1.Introduction: The creative translator and the multisensorial experience of the text
  • 2.Modalities, affordances, affect, and the body
    • 2.1Language and embodied perception
    • 2.2Towards an affective phenomenology of (intersemiotic) translation
    • 2.3Sound, body, and language
    • 2.4Acousmatic listening and dépaysement
  • 3.Workshop and data generation
    • 3.1Researcher positionality
    • 3.2Research approach
  • 4.Analysis and discussion
    • 4.1Cognitive and affective analysis of listening and drawing exercises
    • 4.2Narrative analysis
      • 4.2.1Spatial narrative
      • 4.2.2Mimetic narrative
      • 4.2.3Material narrative
      • 4.2.4Embodied narrative
  • 5.Conclusion and pedagogical implications
  • Acknowledgements
  • Notes
  • References
References (69)
References
Alves, Fabio, and Arnt Lykke Jakobsen, eds. 2021. The Routledge Handbook of Translation and Cognition. Abingdon: Routledge.Google Scholar logo with link to Google Scholar
Andean, James. 2016. “Narrative Modes in Acousmatic Music.” In Sound and Narrative, edited by James Andean, special issue of Organised Sound 21 (3): 192–203. Google Scholar logo with link to Google Scholar
Andrews, Molly, Corinne Squire, and Maria Tamboukou, eds. 2013. Doing Narrative Research. 2nd ed. London: SAGE. Google Scholar logo with link to Google Scholar
Aryani, Arash, Erin S. Isbilen, and Morten H. Christiansen. 2020. “Affective Arousal Links Sound to Meaning.” Psychological Science 31 (8): 978–986. Google Scholar logo with link to Google Scholar
Atkinson, Dwight. 2019. “Beyond the Brain: Intercorporeality and Co-Operative Action for SLA Studies.” The Modern Language Journal 103 (4): 724–738. Google Scholar logo with link to Google Scholar
Bailenson, Jeremy N. 2021. “Nonverbal Overload: A Theoretical Argument for the Causes of Zoom Fatigue.” Technology, Mind, and Behavior 2 (1). Google Scholar logo with link to Google Scholar
Berger, Christopher C., and H. Henrik Ehrsson. 2017. “The Content of Imagined Sounds Changes Visual Motion Perception in the Cross-Bounce Illusion.” Scientific Reports 7 (1): 40123. Google Scholar logo with link to Google Scholar
Böhme, Gernot. 2000. “Acoustic Atmospheres.” Soundscape: The Journal of Acoustic Ecology 1 (1): 14–18.Google Scholar logo with link to Google Scholar
Bower, Matthew, and Shaun Gallagher. 2016. “Bodily Affects as Prenoetic Elements in Enactive Perception.” Phenomenology and Mind (4): 108–131.Google Scholar logo with link to Google Scholar
Campbell, Madeleine. 2017. “Entre l’audible et l’inaudible: Intersemiotic Translation of Mohammed Dib’s Poetry.” In Language – Literature – the Arts: A Cognitive-Semiotic Interface, edited by Elżbieta Chrzanowska-Kluczewska and Olga Voroboya, 167–182. Frankfurt: Peter Lang.Google Scholar logo with link to Google Scholar
Campbell, Madeleine, Birthe Jørgensen, and Bethan. R. Parkes, eds. 2013. Haجar and the Anجel: A Response to John Runciman’s Painting ‘Hagar and the Angel’ (c1766). Collaborative Installation at The Hunterian, Glasgow, UK. [URL]
Campbell, Madeleine, and Ricarda Vidal, eds. 2019. Translating across Sensory and Linguistic Borders: Intersemiotic Journeys between Media. Cham: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Creswell, John W., and J. David Creswell. 2018. Research Design: Qualitative, Quantitative & Mixed Methods Approaches. Los Angeles: SAGE.Google Scholar logo with link to Google Scholar
Damasio, Antonio. R. 1994. Descartes’ Error: Emotion, Reason, and the Human Brain. New York: Avon.Google Scholar logo with link to Google Scholar
1999. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. New York: Harcourt Brace.Google Scholar logo with link to Google Scholar
Dib, Mohammed. (1996) 2007. “L’Aube Ismaël (louange) [Dawn Ismaël (Hymn of praise)].” In Oeuvres Complètes: I Poésies [Complete works: I Poems], edited by Habib Tengour. Paris: Éditions de la Différence.Google Scholar logo with link to Google Scholar
Elleström, Lars. 2010. “The Modalities of Media: A Model for Understanding Intermedial Relations.” In Media Borders, Multimodality and Intermediality, edited by Lars Elleström, 11–48. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
. 2016. “Visual Iconicity in Poetry: Replacing the Notion of ‘Visual Poetry’.” Orbis Litterarum 71 (6): 437–472. Google Scholar logo with link to Google Scholar
Englund, Axel. 2010. “Intermedial Topography and Metaphorical Interaction.” In Media Borders, Intermediality and Multimodality, edited by Lars Elleström, 69–80. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Erickson, Frederick. 2011. “Reflections on Practice, Teaching/Learning, Video, and Theorizing.” In Theories of Learning and Studies of Instructional Practice, edited by Timothy Koschmann, 385–402. New York: Routledge. Google Scholar logo with link to Google Scholar
ffrench, Patrick. 2015. “Barthes and the Voice: The Acousmatic and Beyond.” L’Esprit créateur 55 (4): 56–69. Google Scholar logo with link to Google Scholar
Gibson, James J. 1979. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.Google Scholar logo with link to Google Scholar
Greener, Ian. 2011. Designing Social Research: A Guide for the Bewildered. London: SAGE. Google Scholar logo with link to Google Scholar
Hajar, Anas. 2021. “Theoretical Foundations of Phenomenography: A Critical Review.” Higher Education Research and Development 40 (7): 1421–1436. Google Scholar logo with link to Google Scholar
Halliday, M. A. K. 1978. Language as Social Semiotic: The Social Interpretation of Language and Meaning. London: Hodder Arnold.Google Scholar logo with link to Google Scholar
Hokkanen, Sari, and Kaisa Koskinen. 2016. “Affect as a Hinge: The Translator’s Experiencing Self as a Sociocognitive Interface.” Translation Spaces 5 (1): 78–96.Google Scholar logo with link to Google Scholar
Hutchins, Edwin. 1995. Cognition in the Wild. Cambridge: MIT Press.Google Scholar logo with link to Google Scholar
Katschthaler, Karl. 2022. Zwischen Atmosphäre und Narration: Zum Verhältnis von Musik, Sprache und Literatur im 20. und 21. Jahrhundert [Between atmosphere and narration: The relationship of music, language and literature in the 20th and 21st century]. Bielefeld: Transcript Verlag.Google Scholar logo with link to Google Scholar
Kellough, B. R. 2013. “Sound Installation.” In Haجar and the Anجel: A Response to John Runciman’s Painting ‘Hagar and the Angel’ (c1766), edited by Madeleine Campbell, Birthe Jørgensen, and Bethan. R. Parkes. Collaborative Installation at The Hunterian, Glasgow, UK. [URL]
Kohler Riessman, Catherine. 2008. Narrative Methods for the Human Sciences. Thousand Oaks: SAGE.Google Scholar logo with link to Google Scholar
Koskinen, Kaisa. 2012. “Domestication, Foreignization and the Modulation of Affect.” In Domestication and Foreignization in Translation Studies, edited by Hannu Kemppainen, Marja Jänis, and Alexandra Belikova, 13–32. Berlin: Frank & Timme.Google Scholar logo with link to Google Scholar
. 2015. “Training Translators for a Superdiverse World: Translators’ Intercultural Competence and Translation as Affective Work.” Russian Journal of Linguistics 19 (4): 175–184.Google Scholar logo with link to Google Scholar
Kress, Gunther R. 2003. Literacy in the New Media Age. London: Routledge.Google Scholar logo with link to Google Scholar
2010. Multimodality: A Social Semiotic Approach to Contemporary Communication. Abingdon: Routledge.Google Scholar logo with link to Google Scholar
Kress, Gunther R., and Theo van Leeuwen. 1996. Reading Images: The Grammar of Visual Design. Abingdon: Routledge.Google Scholar logo with link to Google Scholar
Kress, Gunther R., and Theo Van Leeuwen. 2021. Reading Images: The Grammar of Visual Design. 3rd ed. Abingdon: Routledge.Google Scholar logo with link to Google Scholar
Leavy, Patricia. 2017. Research Design: Quantitative, Qualitative, Mixed Methods, Arts-Based, and Community-Based Participatory Research Approaches. New York: The Guilford PressGoogle Scholar logo with link to Google Scholar
Lebrecht, Sophie, Moshe Bar, Lisa Feldman Barrett, and Michael J. Tarr. 2012. “Micro-Valences: Perceiving Affective Valence in Everyday Objects.” Frontiers in Psychology 31: 107. Google Scholar logo with link to Google Scholar
Lehr, Caroline. 2021. “Translation, Emotion and Cognition.” In Alves and Jakobsen (2021, 294–309). Google Scholar logo with link to Google Scholar
Leibowitz, Brenda. 2009. “The Professional Development of Academics as Teachers.” In Researching Higher Education: International Perspectives on Theory, Policy and Practice, edited by Jennifer M. Case and Jeroen Huisman, 153–170. New York: Routledge.Google Scholar logo with link to Google Scholar
Marton, Ference. 1986. “Phenomenography: A Research Approach to Investigating Different Understandings of Reality.” Journal of Thought 21 (3): 28–49.Google Scholar logo with link to Google Scholar
Marton, Ference, and Shirley Booth. 1997. Learning and Awareness. Mahwah: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Martin, J. R., and White, R. 2005. The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Massey, Doreen. 2005. For Space. London: Sage.Google Scholar logo with link to Google Scholar
Meschonnic, Henri. 2007. Éthique et politique du traduire [The ethics and politics of translation]. Paris: Verdier.Google Scholar logo with link to Google Scholar
Meyer, Christian, Jürgen Streeck, and J. Scott Jordan, eds. 2017. Intercorporeality: Emerging Socialities in Interaction. Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
Muñoz Martín, Ricardo. 2016. “Processes of What Models? On the Cognitive Indivisibility of Translation Acts and Events.” In Cognitive Space: Exploring the Situational Interface, edited by Birgitta Englund Dimitrova and Maureen Ehrensberger-Dow, special issue of Translation Spaces 5 (1): 145–161.Google Scholar logo with link to Google Scholar
Muñoz Martín, Ricardo, and Celia Martín de León. 2021. “Translation and Cognition.” In Alves and Jakobsen (2021, 52–68). Google Scholar logo with link to Google Scholar
Noë, Alva. 2004. Action in Perception. Cambridge, MA: MIT Press.Google Scholar logo with link to Google Scholar
O’Regan, J. Kevin, and Alve Noë. 2001a. “A Sensorimotor Account of Vision and Visual Consciousness.” Behavioral and Brain Sciences 24 (5): 939–973. Google Scholar logo with link to Google Scholar
. 2001b. “What Is It Like to See: A Sensorimotor Theory of Perceptual Experience.” Synthese 129 (1): 79–103. Google Scholar logo with link to Google Scholar
Painter, Clare, J. R. Martin, and Len Unsworth. 2013. Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox Publishing.Google Scholar logo with link to Google Scholar
Petrides, K. V. 2009. Technical Manual for the Trait Emotional Intelligence Questionnaires (TEIQue). London: London Psychometric Laboratory.Google Scholar logo with link to Google Scholar
Pherali, Tejendra J. 2011. Phenomenography as a Research Strategy: Researching Environmental Conceptions. Saarbrücken: Lambert Academic Publishing.Google Scholar logo with link to Google Scholar
Raguet, Christine. 2016. “Alienability and Creativity: The Role of Sounds and Sensations in Translation.” In Translation and Creativity – La traduction comme création, edited by Martine Hennard Dutheil de la Rochère and Irene Weber-Henking, 135–160. Lausanne: Cahiers du CTL.Google Scholar logo with link to Google Scholar
Risku, Hanna, and Regina Rogl. 2021. “Translation and Situated, Embodied, Distributed and Extended Cognition.” In Alves and Jakobsen (2021, 478–499). Google Scholar logo with link to Google Scholar
Rowell, Lonnie L., Margaret M. Riel, and Elena Yu. Polush. 2017. “Defining Action Research: On Dialogic Spaces for Constructing Shared Meanings.” In The Palgrave International Handbook of Action Research, edited by Lonnie L. Rowell, Catherine D. Bruce, Joseph M. Shosh, and Margaret M. Riel, 85–101. New York: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Rudenko, Svetlana, and Maria José de Córdoba Serrano. 2017. “Musical-Space Synaesthesia: Visualisation of Musical Texture.” Multisensory Research 30 (3–5): 279–285. Google Scholar logo with link to Google Scholar
Schaeffer, Pierre. 1966. Traité des objets musicaux [A treatise on musical objects]. Paris: Le Seuil.Google Scholar logo with link to Google Scholar
Scott, Clive. 2010. “Intermediality and Synesthesia: Literary Translation as Centrifugal Practice.” Art in Translation 2 (2): 153–169. Google Scholar logo with link to Google Scholar
Sheets-Johnstone, Maxine. 2011. “Embodied Minds or Mindful Bodies? A Question of Fundamental, Inherently Inter-related Aspects of Animation.” Subjectivity 4 (4): 451–466. Google Scholar logo with link to Google Scholar
Silverstein, Michael. 2015. “How Language Communities Intersect: Is ‘Superdiversity’ an Incremental or Transformative Condition?” Language and Communication 441: 7–18. Google Scholar logo with link to Google Scholar
Stuart, Susan A. J. 2017. “Feeling Our Way: Enkinesthetic Enquiry and Immanent Intercorporeality.” In Intercorporeality: Emerging Socialities in Interaction, edited by Christian Meyer, Jürgen Streeck, and J. Scott Jordan, 51–72. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Van Lier, Leo. 2008. “Ecological-Semiotic Perspectives on Educational Linguistics.” In The Handbook of Educational Linguistics, edited by Bernard Spolsky and Francis M. Hult, 596–605. Malden, MA: Blackwell. Google Scholar logo with link to Google Scholar
Vatakis, Argiro, and Charles Spence. 2007. “Crossmodal Binding: Evaluating the ‘Unity Assumption’ Using Audiovisual Speech Stimuli.” Perception & Psychophysics 69 (5): 744–756. Google Scholar logo with link to Google Scholar
Venuti, Lawrence. 1995. The Translator’s Invisibility: A History of Translation. Abingdon: Routledge.Google Scholar logo with link to Google Scholar
Ward, Jamie. 2008. The Frog Who Croaked Blue: Synesthesia and the Mixing of the Senses. Hove: Routledge.Google Scholar logo with link to Google Scholar
Williams, Ralph Vaughan. 1925. “The Lark Ascending: Romance for Violin and Orchestra.” Score. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Cited by (1)

Cited by one other publication

Wen, Ting
2024. Transforming Intermediality and Transregional Communication. Scientific Journals Monograghs DOI logo

This list is based on CrossRef data as of 4 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue