Cover not available

Article published In: Revista Española de Lingüística Aplicada/Spanish Journal of Applied Linguistics
Vol. 32:2 (2019) ► pp.515–542

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

The VIW project

Multimodal corpus linguistics for audio description analysis

 | Universitat Autònoma de Barcelona
Published online: 5 November 2019
DOI logo with link
https://doi.org/10.1075/resla.17001.mat
Abstract
Following an overview of corpus linguistics in audiovisual translation, and more specifically in audio description, this article presents the VIW (Visuals Into Words) project and its resulting corpus. It describes the compilation and annotation processes, highlighting the main challenges found. The article also presents the web application that has been developed, explaining in detail various data visualisation and search possibilities.
Resumen
Después de una panorámica general sobre la lingüística de corpus en traducción audiovisual, y más específicamente en el ámbito de la audiodescripción, el artículo presenta el proyecto VIW y el corpus que se ha desarrollado. Se describen los procesos de compilación y anotación del corpus, destacando los principales retos que se han encontrado. El artículo también presenta la aplicación web desarrollada durante el proyecto, que permite varias visualizaciones de los datos así como múltiples posibilidades de búsquedas.
Article outline
  • 1.Corpus studies and audiovisual translation
    • 1.1Audio descriptions and corpus studies
    • 1.2From textual analysis to multimodal analysis: Challenges and limitations
  • 2.The VIW project
    • 2.1Corpus description: The short film
    • 2.2Corpus descriptions: The audio descriptions
  • 3.Corpus processing: Segmentation and annotation
    • 3.1Linguistic tiers
    • 3.2Filmic tiers
  • 4.Corpus exploitation: Web app and visualisations
    • 4.1Visualisations at corpus and subcorpus level
    • 4.2Visualisations for each audio description file
  • 5.A first quantitative sample analysis
  • 6.Conclusions
  • Acknowledgements
  • Note
  • References
References (42)
References
Adolphs, S., & Carter, R. (2013). Spoken Corpus Linguistics. From Monomodal to Multimodal. London and New York: Routledge. Google Scholar logo with link to Google Scholar
Arma, V. (2011). The language of filmic audio descriptions: a corpus-based analysis of adjectives. PhD dissertation. Naples: Università degli Studi di Napoli Federico II.Google Scholar logo with link to Google Scholar
Baldry, A., & O’Halloran, K. (2010). The annotation of multimodal corpus of university websites: an illustration of multimodal corpus linguistics. In T. Harris, & M. Moreno Jaén (Eds.), Corpus linguistics in language teaching (pp. 177–2010). Bern: Peter Lang.Google Scholar logo with link to Google Scholar
Baños, R. (2014). Orality markers in Spanish native and dubbed sitcoms: Pretended spontaneity and prefabricated orality. Meta: journal des traducteurs/Meta: Translators’ Journal, 59(2), 406–435. Google Scholar logo with link to Google Scholar
Baños, R., Bruti, S., & Zanotti, S. (2013). Corpus linguistics and AVT: in search of an integrated approach. Perspectives. Studies in Translatology, 21(4), 483–490. Google Scholar logo with link to Google Scholar
Baños-Piñero, R., & Chaume, F. (2009). Prefabricated orality: a challenge in audiovisual translation research. In M. Giorgio Marrano, G. Nadiani, & C. Rundle (Eds.), Intralinea Special Issue: The Translation of Dialects in Multimedia. Available online: [URL] (last access 20 September 2016)
Bednarek, M. (2015). Corpus-assisted multimodal discourse analysis of television and film narratives. In P. Baker, & T. McEnery (Eds.), Corpora and discourse studies (pp. 63–87). Basingstoke/New York: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Bonsignori, V., Bruti, S., & Masi, S. (2011). Formulae across languages: English greetings, leave-takings and good wishes in dubbed Italian. In A. Serban, A. Matamala, & J.-M. Lavaur (Eds.), Audiovisual translation in close-up. Practical and theoretical approaches (pp. 23–44). Bern: Peter Lang.Google Scholar logo with link to Google Scholar
(2012). Exploring greetings and leave-takings in original and dubbed Language. In A. Remael, P. Orero, & M. Carroll (Eds.), Audiovisual translation and media accessibility at the crossroads – Media for All 3 (pp. 357–379). Amsterdam, New York: Rodopi. Google Scholar logo with link to Google Scholar
Bordwell, D. & Thompson, K. (2008). Film art: An introduction. New York: McGraw Hill.Google Scholar logo with link to Google Scholar
Braun, S. (2013). Audio description research: State of the art and beyond. Translation Studies in the New Millennium, 61, 14–30.Google Scholar logo with link to Google Scholar
Bywood, L., Volk, M., Fishel, M., & Georgakopoulou, P. (2013). Parallel subtitle corpora and their applications in machine translation and translatology. Perspectives. Studies in Translatology, 21(4), 595–610. Google Scholar logo with link to Google Scholar
Chafe, W. (Ed.). (1980). The Pear Stories. Norwood: Ablex.Google Scholar logo with link to Google Scholar
Forchini, P. (2012). Movie language revisited: evidence from multi-dimensional analysis and corpora. Bern: Peter Lang. Google Scholar logo with link to Google Scholar
Freddi, M., & Pavesi, M. (Eds.). (2009). Analysing audiovisual dialogue: linguistic and translational insights. Bologna: Clueb.Google Scholar logo with link to Google Scholar
Freddi, M. (2009). The phraseology of contemporary filmic speech: Formulaic language and translation. In M. Freddi, & M. Pavesi (Eds.), Analysing audiovisual dialogue: Linguistic and translational insights (pp. 101–123). Bologna: Clueb.Google Scholar logo with link to Google Scholar
Heiss, C., & Soffritti, M. (2008). Forlixt 1 – The Forlì Corpus of Screen Translation. Exploring microstructures. In D. Chiaro, C. Heiss, & C. Bucaria (Eds.), Between text and image. Updating research in screen translation (pp. 51–62). Amsterdam and Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Jiménez Hurtado, C., & Seibel, C. (2012). Multisemiotic and multimodal corpus analysis of audio descriptions. In A. Remael, P. Orero, & M. Carroll (Eds.), Audiovisual translation and media accessibility at the crossroads (pp. 409–425). Amsterdam: Rodopi. Google Scholar logo with link to Google Scholar
Knight, D. (2011). The future of multimodal corpora. Revista Brasileira de Linguistica Aplicada, 11(2), 491–415. Google Scholar logo with link to Google Scholar
Maszerowska, A., Matamala, A., & Orero, P. (Eds.). (2014). Audio Description. New perspectives illustrated. Amsterdam: Benjamins.Google Scholar logo with link to Google Scholar
Matamala, A. (2005). Les interjeccions en un corpus audiovisual. Descripció i representació lexicogràfica [Interjections in an audiovisual corpus. Description and lexicographical representation]. Barcelona: Publicacions de l’IULA.Google Scholar logo with link to Google Scholar
(2009). Interjections in original and dubbed sitcoms: a comparison. Meta: journal des traducteurs/Meta: Translators’ Journal, 54(3), 485–502. Google Scholar logo with link to Google Scholar
(2018). One short film, different audio descriptions. Analysing the language of audio descriptions created by students and professionals. Onomázein, 411, 185–207. Google Scholar logo with link to Google Scholar
Matamala, A., & Orero, P. (2016). Researching audio description. New approaches. London: Palgrave. Google Scholar logo with link to Google Scholar
Mattson, J. (2009). The subtitling of discourse particles. A corpus-based study of well, you know, I mean, and like, and their Swedish translations in ten American films. Unpublished PhD thesis. Gothemburg, Weden: University of Gothemburg.Google Scholar logo with link to Google Scholar
Mazur, I., & Kruger, J.-L. (2012). Pear Stories and audio description: language, perception and cognition across cultures. Perspectives. Studies in Translatology, 20(1), 1–3. Google Scholar logo with link to Google Scholar
Mouka, E., Saridakis, I., & Fotopoulou, A. (2012). Racism goes to the movies: a corpus-driven study of cross-linguistic racist discourse annotation and translation analysis. In C. Fantinuoli, & F. Zanettin (Eds.), New directions in corpus-based translation (pp. 31–61). Berlin: Language Science Press.Google Scholar logo with link to Google Scholar
Pavesi, M. (2009). Dubbing English into Italian: a closer look at the translation of spoken language. In J. Díaz-Cintas (Ed.), New trends in audiovisual translation (pp. 197–209). Bristol: Multilingual Matters. Google Scholar logo with link to Google Scholar
Reviers, N., Remael, A., & Daelemans, W. (2015). The language of Audio Description in Dutch: Results of a corpus study. In A. Jankowska, & A. Szarkowska (Eds.), New Points of View on Audiovisual Translation and Accessibility (pp. 167–189). Bern: Peter Lang.Google Scholar logo with link to Google Scholar
Rica, J. P. (2014). La traducción de marcadores discursives (DM) inglés-español en los subtítulos de películas: un estudio de corpus. Jostrans. Journal of Specialised Translation, 211, 177–199.Google Scholar logo with link to Google Scholar
Rohrbach, A., Rohrbach, M., Tandon, N., & Schiele, B. (2015). A data set for movie description. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), 3202–3212.Google Scholar logo with link to Google Scholar
Romero-Fresco, P. (2009). Naturalness in the Spanish dubbing language: a case of not-so-close friends. Meta: journal des traducteurs/Meta: Translators’ Journal, 54(1), 49–72. Google Scholar logo with link to Google Scholar
(2013). Accessible filmmaking: Joining the dots between audiovisual translation, accessibility and filmmaking. Jostrans. Journal of Specialised Translation, 201, 201–223.Google Scholar logo with link to Google Scholar
Salway, A. (2007). A corpus-based analysis of audio description. In J. Díaz-Cintas, P. Orero, & A. Remael (Eds.), Media for all. Subtitling for the deaf, audio description, and sign language (pp. 151–174). Amsterdam: Rodopi. Google Scholar logo with link to Google Scholar
Sloetjes, H. & Wittenburg, P. (2008). Annotation by category – ELAN and ISO DCR. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008).Google Scholar logo with link to Google Scholar
Sotelo Dios, P., & Gómez Guinovart, X. (2012). A multimedia parallel corpus of English-Galician film subtitling. In A. Simões, R. Queirós, & D. da Cruz (Eds.), Proceedings of the 1st symposium on languages, applications and technologies, SLATE 2012 (pp. 255–266). Schloss-Dagstuhl: OASIcs.Google Scholar logo with link to Google Scholar
Tiedemann, J. (2007). Building a multilingual parallel subtitle corpus. In P. Dirix, I. Schuurman, V. Vandeghinste, & F. Van Eynde (Eds.), Proceedings of the 17th meeting of computational linguistics in the Netherlands (pp. 147–162). Utrecht: Utrecht University.Google Scholar logo with link to Google Scholar
Tirkkonen-Condit, S., & Mäkisalo, J. (2007). Cohesion in subtitles: a corpus-based study. Across languages and cultures, 8(2), 221–230. Google Scholar logo with link to Google Scholar
Valentini, C. (2008). Forlixt 1 – The Forlì Corpus of Screen Translation. Exploring macrostructures. In D. Chiaro, C. Heiss, & C. Bucaria (Eds.), Between text and image. Updating research in screen translation (pp. 37–50). Amsterdam and Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
(2013). Phrasal verbs in Italian dubbed dialogues: a multimedia corpus-based study. Perspectives. Studies in translatology, 21(4), 543–562. Google Scholar logo with link to Google Scholar
Cited by (4)

Cited by four other publications

Lincker, Elise, Camille Guinaudeau & Shin’ichi Satoh
2025. AD2AT: Audio Description to Alternative Text, a Dataset of Alternative Text from Movies. In MultiMedia Modeling [Lecture Notes in Computer Science, 15520], ► pp. 58 ff. DOI logo
Fernández Alonso, Inés, María J. Machuca & Anna Matamala
2024. Voces neutras y alteración tonal. Círculo de Lingüística Aplicada a la Comunicación 97 ► pp. 195 ff. DOI logo
Álvarez Lamas, Soledad & Mayte Victoria Gorrostorrazo Babuglia
2024. Características lingüísticas y textuales de la audiodescripción fílmica en Uruguay. Íkala, Revista de Lenguaje y Cultura 29:3 DOI logo
柏, 薇
2023. Research on Audiovisual Translation and Reception at Home and Abroad Based on CiteSpace. Modern Linguistics 11:04 ► pp. 1373 ff. DOI logo

This list is based on CrossRef data as of 30 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue