Cover not available

Article published In: Audiovisual translation and media accessibility in language learning contexts: Multiple modes, innovative practices and further explorations
Edited by Marina Manfredi, Catia Nannoni and Rosa Pugliese
[Translation and Translanguaging in Multilingual Contexts 9:2] 2023
► pp. 261–282

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Reverse subtitles in foreign language learning

Noticing and memory

 | University of Warsaw
Published online: 14 July 2023
DOI logo with link
https://doi.org/10.1075/ttmc.00111.rag
Abstract
This paper reports on an eye-tracking study investigating the processing and mnemonic retention of reverse subtitles (foreign-language subtitles presented alongside native-language audio) in learners of Italian as a Foreign language (IFL). 26 English native speakers with a CEFR B2+ Italian level watched an English clip with Italian subtitles in two translation conditions, formal similarity (literal transfer) and formal discrepancy (non-literal transfer). Immediately after watching, they answered recognition and recall questions. This study examines memory, attention allocation and the concept of noticing, which was investigated through triangulation of eye tracking, verbatim recognition and explicit reports. Data analysis methods include generalised mixed-effect modelling. Results revealed that reverse subtitles have acquisitional potential for advanced IFL learners, noticing can be probed experimentally, and formal (dis)similarity appears to have some psychological reality in the mind of the learner, being able to affect both recognition and recall. Evidence of novel word learning as well as deepening of existing knowledge emerged from the analyses, supporting the view that reversed subtitles could be more fruitfully exploited in FLL contexts. The paper presents details of the data analyses, discusses them in relation to Second Language Acquisition (SLA) and psycholinguistic concepts, and draws some recommendations based on the findings.
Article outline
  • 1.Introduction
  • 2.Literature review and theoretical framework
    • 2.1Attention and noticing
    • 2.2Working memory
  • 3.Aims and method
    • 3.1Research questions
    • 3.2Translation condition
    • 3.3Participants and experimental stimuli
    • 3.4Experimental tools
      • 3.4.1Recognition post-test
      • 3.4.2Working memory post-test
      • 3.4.3Questionnaire
    • 3.5Data collection and analysis
      • 3.5.1Hardware and software
      • 3.5.2Variables
      • 3.5.3Types of analysis
  • 4.Results
    • 4.1Working memory
    • 4.2Mixed-effect modelling
    • 4.3Free recall
  • 5.Discussion
    • 5.1Working memory
    • 5.2Noticing
  • 6.Conclusion
  • Notes
  • References
  • Filmography
References (63)
References
Ávila-Cabrera, José Javier. 2021. “Reverse Subtitling in the ESP Class to Improve Written Skills in English: A Case Study.” Journal of Audiovisual Translation 4 (1): 27–49. Google Scholar logo with link to Google Scholar
Baddeley, Alan. 1986. Working Memory. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
. 2003. “Working Memory and Language: An Overview.” Journal of Communication Disorders 36 (3): 189–208. Google Scholar logo with link to Google Scholar
Baddeley, Alan, and Graham Hitch. 1974. “Working Memory.” In The Psychology of Learning and Motivation: Advances in Research and Theory, ed. by Gordon A. Bower, 47–89. New York: Academic Press.Google Scholar logo with link to Google Scholar
Baltova, Iva. 1999. The Effect of Subtitled and Staged Video Input on the Learning and Retention of Content and Vocabulary in a Second Language. PhD diss. University of Toronto.
Bates, Douglas. 2010. Lme4: Mixed-Effects Modeling with R. New York: Springer.Google Scholar logo with link to Google Scholar
Bird, Stephen A., and John N. Williams. 2002. “The Effect of Bimodal Input on Implicit and Explicit Memory: An Investigation into the Benefits of Within-language Subtitling.” Applied Psycholinguistics 23 (04). Google Scholar logo with link to Google Scholar
Bolker, Benjamin M., Mollie E. Brooks, Connie J. Clark, Shane W. Geange, John R. Poulsen, Michael H. Stevens, and Jada-Simone S. White. 2009. “Generalized Linear Mixed Models: A Practical Guide for Ecology and Evolution.” Trends in Ecology and Evolution 24 (3): 127–135. Google Scholar logo with link to Google Scholar
BOS (Bristol Online Survey). 2016. The Online Survey Tool Designed for Academic Research, Education and Public Sector Organisations. [URL]
Čepon, Slavica. 2011. “Interlingual Subtitling as a Mode of Facilitating Incidental Foreign Language Acquisition.” English for Specific Purposes World 33 (11): 1–37.Google Scholar logo with link to Google Scholar
. 2013. “Effective Use of the Media: Video in the Foreign Language Classroom.” Medijska Istrazivanja 19 (1): 83–104.Google Scholar logo with link to Google Scholar
Clifton, Charles, Adrian Staub, and Keith Rayner. 2007. “Eye Movements in Reading Words and Sentences.” In Eye Movements: A Window on Mind and Brain, ed. by Roger P. G. Van Gompel, Martin H. Fischer, Wayne S. Murray, and Robin L. Hill, 341–371. Oxford: Elsevier. Google Scholar logo with link to Google Scholar
Conklin, Kathy, and Ana Pellicer-Sánchez. 2016. “Using Eye-tracking in Applied Linguistics and Second Language Research.” Second Language Research 32 (3): 453–467. Google Scholar logo with link to Google Scholar
Cowan, Nelson, Emily M. Elliott, J. Scott Saults, Candice C. Morey, Sam Mattox, Anna Hismjatullina, and Andrew R. A. Conway. 2005. “On the Capacity of Attention: Its Estimation and its Role in Working Memory and Cognitive Aptitudes.” Cognitive Psychology 51 (1): 42–100. Google Scholar logo with link to Google Scholar
d’Ydewalle, Géry, and Wim De Bruycker. 2007. “Eye Movements of Children and Adults while Reading Television Subtitles.” European Psychologist 12 (3): 196–205. Google Scholar logo with link to Google Scholar
Danan, Martine. 1992. “Reversed Subtitling and Dual Coding Theory: New Directions for Foreign Language Instruction.” Language Learning 421: 497–527. Google Scholar logo with link to Google Scholar
Daneman, Meredyth, and Patricia A. Carpenter. 1980. “Individual Differences in Working Memory and Reading.” Journal of Verbal Learning and Verbal Behaviour 19 (4): 450–466. Google Scholar logo with link to Google Scholar
Daneman, Meredyth, and Philip M. Merikle. 1996. “Working Memory and Language Comprehension: A Meta-analysis.” Psychonomic Bulletin & Review 3 (4): 422–433. Google Scholar logo with link to Google Scholar
Dixon, Peter. 2008. “Models of Accuracy in Repeated-measures Designs.” Journal of Memory and Language 59 (4): 447–456. Google Scholar logo with link to Google Scholar
Doughty, Catherine. 2001. “Cognitive Underpinnings of Focus on Form.” In Cognition and Second Language Instruction, ed. by Peter Robinson, 206–257. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Farmer, Thomas A., Jennifer B. Misyak, and Morten H. Christiansen. 2012. “Individual Differences in Sentence Processing.” In The Cambridge Handbook of Psycholinguistics, ed. by Michael Spivey, Ken McRae, and Marc Joanisse, 353–364. New York: Cambridge University Press. Google Scholar logo with link to Google Scholar
Gass, Susan M., Paula Winke, Daniel R. Isbell, and Jieun Irene Ahn. 2019. “How Captions Help People Learn Languages: A Working-memory, Eye-tracking Study.” Language Learning & Technology 231: 84–104.Google Scholar logo with link to Google Scholar
Gathercole, Susan E. 2007. “Working Memory and Language”. In The Oxford Handbook of Psycholinguistics, ed. by M. Gareth Gaskell, 757–769. Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
Ghia, Elisa. 2007. “A Case Study on the Role of Interlingual Subtitles on the Acquisition of L2 Syntax: Initial Results.” In Voices on Translation. Linguistic, Multimedia, and Cognitive Perspectives, ed. by Annalisa Baicchi, special issue of RILA. Rassegna Italiana di Linguistica Applicata 49 (1–2): 167–177.Google Scholar logo with link to Google Scholar
. 2012a. Subtitling Matters: New Perspectives on Subtitling and Foreign Language Learning. Bern: Peter Lang.Google Scholar logo with link to Google Scholar
. 2012b. “The Impact of Translation Strategies on Subtitle Reading.” In Eye Tracking in Audiovisual Translation, ed. by Elisa Perego, 157–182. Rome: Aracne.Google Scholar logo with link to Google Scholar
Godfroid, Aline, Alex Housen, and Frank Boers. 2010. “A Procedure for Testing the Noticing Hypothesis in the Context of Vocabulary Acquisition.” In Inside the Learner’s Mind: Cognitive Processing and Second Language Acquisition, ed. by Martin Pütz, and Laura Sicola, 169–197. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Godfroid, Aline, and Bronson Hui. 2020. “Five Common Pitfalls in Eye-tracking Research.” Second Language Research 36 (3): 277–305. Google Scholar logo with link to Google Scholar
Gottlieb, Henrik. 1992. “Subtitling: A New University Discipline.” In Teaching Translation and Interpreting: Training Talent and Experience. Papers from the First Language International Conference, Elsinore, Denmark, 1991, ed. by Cay Dollerup, and Anne Loddegaard, Amsterdam: John Benjamins 161–170.
Harrington, Michael, and Mark Sawyer. 1992. “L2 Working Memory Capacity and L2 Reading Skill.” Studies in Second Language Acquisition 141: 25–38. Google Scholar logo with link to Google Scholar
Holobow, Naomi E., Wallace E. Lambert, and Laura Sayegh. 1984. “Pairing Script and Dialogue: Combinations that Show Promise for Second or Foreign Language Learning.” Language Learning 341: 59–74. Google Scholar logo with link to Google Scholar
Lambert, Wallace E., Inga Boehler, and Nelly Sidoti. 1981. “Choosing the Languages of Subtitles and Spoken Dialogues for Media Presentations: Implications for Second Language Education.” Applied Psycholinguistics 2 (02). Google Scholar logo with link to Google Scholar
Lambert, Wallace E., and Naomi E. Holobow. 1984. “Combinations of Printed Script and Spoken Dialogue that Show Promise for Students of a Foreign Language.” Canadian Journal of Behavioural Science 161: 1–11. Google Scholar logo with link to Google Scholar
Lavaur, Jean-Marc, and Dominique Bairstow. 2011. “Languages on the Screen: Is Film Comprehension Related to the Viewers’ Fluency Level and to the Language in the Subtitles?” International Journal of Psychology 46 (6): 455–62. Google Scholar logo with link to Google Scholar
Lee, Minjin, and Andrea Révész. 2020. “Promoting Grammatical Development through Captions and Textual Enhancement in Multimodal Input-based Tasks.” Studies in Second Language Acquisition 42 (3): 625–651. Google Scholar logo with link to Google Scholar
Leow, Ronald P. 2013. “Attention, Noticing, and Awareness in Second Language Acquisition.” In The Encyclopedia of Applied Linguistics, ed. by C. A. Chapelle, 269–275. Chichester, West Sussex: Wiley-Blackwell Publishing.Google Scholar logo with link to Google Scholar
Leow, Ronald P., and Melissa A. Bowles. 2005. “Attention and Awareness in SLA.” In Mind and Context in Adult Second Language Acquisition: Methods, Theory, and Practice, ed. by Cristina Sanz, 179–196. Washington, DC: Georgetown University Press.Google Scholar logo with link to Google Scholar
Nida, Eugene. 1964. Toward a Science of Translating, with Special Reference Involved in Bible Translating. Leiden: A.J. Brill.Google Scholar logo with link to Google Scholar
Orero, Pilar, Stephen Doherty, Jan-Louis Kruger, Anna Matamala, Jan Pedersen, Elisa Perego, Pablo Romero-Fresco, Sara Rovira-Esteva, Olga Soler-Vilageliu, and Agnieszka Szarkowska. 2018. “Conducting Experimental Research in Audiovisual Translation (AVT): A Position Paper.” The Journal of Specialised Translation (JoSTrans) 301: 105–126.Google Scholar logo with link to Google Scholar
Puimège, Eva, Maribel Montero Perez, and Elke Peters. 2021. “Promoting L2 Acquisition of Multiword Units through Textually Enhanced Audiovisual Input: An Eye-tracking Study.” Second Language Research: 1–22. Google Scholar logo with link to Google Scholar
R Development Core Team. 2017. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. Accessed August 8, 2022: [URL]
Ragni, Valentina. 2017. More than Meets the Eye. A Reception Study on the Effects of Translation on Noticing and Memorisation of L2 Reverse Subtitles. PhD Diss. University of Leeds.
. 2020. “More than Meets the Eye: An Eye-tracking Study of the Effects of Translation on the Processing and Memorisation of Reversed Subtitles.” Journal of Specialised Translation 331: 99–128.Google Scholar logo with link to Google Scholar
Robinson, Peter. 1995. “Attention, Memory, and the ‘Noticing’ Hypothesis.” Language Learning 45 (2): 283–331. Google Scholar logo with link to Google Scholar
. 1996. “Learning Simple and Complex Second Language Rules under Implicit, Incidental, Rule-search, and Instructed Conditions.” Studies in Second Language Acquisition 18 (1): 27–67. Google Scholar logo with link to Google Scholar
Schmidt, Richard. 1990. “The Role of Consciousness in Second Language Learning.” Applied Linguistics 111: 129–158. Google Scholar logo with link to Google Scholar
. 2001. “Attention.” In Cognition and Second Language Instruction, ed. by Peter Robinson, 3–32. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Smith, Brian. 2012. “Eye Tracking as a Measure of Noticing: A Study of Explicit Recasts in SCMC.” Language Learning & Technology 16 (3): 53–81.Google Scholar logo with link to Google Scholar
Swain, Merrill. 1985. “Communicative Competence: Some Roles of Comprehensible Input & Comprehensible Output in its Development.” In Input in Second Language Acquisition, ed. by Susan M. Gass, and Carolyn Madden, 235–256. New York: Newbury House.Google Scholar logo with link to Google Scholar
Szmalec, Arnaud, Marc Brysbaert, and Wouter Duyck. 2013. “Working Memory and (Second) Language Processing.” In Memory, Language, and Bilingualism: Theoretical and Applied Approaches, ed. by Jeanette Altarriba, and Ludmila Isurin, 74–94. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Talaván, Noa, Ana Ibáñez Moreno, and Elena Bárcena. 2017. “Exploring Collaborative Reverse Subtitling for the Enhancement of Written Production Activities in English as a Second Language.” ReCALL 11: 1–20. Google Scholar logo with link to Google Scholar
Tomlin, Russell S., and Victor Villa. 1994. “Attention in Cognitive Science and Second Language Acquisition.” Studies in Second Language Acquisition 16 (2): 183–203. Google Scholar logo with link to Google Scholar
Truscott, John. 1998. “Noticing in Second Language Acquisition: A Critical Review.” Second Language Research 14 (2): 103–135. Google Scholar logo with link to Google Scholar
Van den Noort, Maurits, Peggy M. P. C. Bosch, Marco Haverkort, and Kenneth Hugdahl. 2008. “A Standard Computerized Version of the Reading Span Test in Different Languages.” European Journal of Psychological Assessment 241: 35–42. Google Scholar logo with link to Google Scholar
Van Lommel, Sven, Annouschka Laenen, and Géry d’Ydewalle. 2006. “Foreign-Grammar Acquisition While Watching Subtitled Television Programmes.” British Journal of Educational Psychology 76 (2): 243–258. Google Scholar logo with link to Google Scholar
Vanderplank, Robert. 1998. “Subtitles, Silent Film to Teletext.” In Concise Encyclopaedia of Pragmatics, ed. by Jacob L. Mey, 961–964. Oxford: Elsevier.Google Scholar logo with link to Google Scholar
Vulchanova, Mila, Lisa M. G. Aurstad, Ingrid E. N. Kvitnes, and Hendrik Eshuis. 2015. “As Naturalistic as it Gets: Subtitles in the English Classroom in Norway.” Frontiers in Psychology 51: 1–10. Google Scholar logo with link to Google Scholar
Wang, Andi, and Ana Pellicer-Sánchez. 2022. “Incidental Vocabulary Learning from Bilingual Subtitled Viewing: An Eye-tracking Study.” Language Learning 72 (3): 765–805. Google Scholar logo with link to Google Scholar
Winke, Paula, Susan Gass, and Tetyana Sydorenko. 2013. “Factors Influencing the Use of Captions by Foreign Language Learners: An Eye-tracking Study.” The Modern Language Journal 97 (1): 254–275. Google Scholar logo with link to Google Scholar
Wood Bowden, Harriet, Cristina Sanz, and Catherine Stafford. 2005. “Individual Differences: Age, Sex, Working Memory, and Prior Knowledge.” In Mind and Context in Adult Second Language Acquisition: Methods, Theory and Practice, ed. by Cristina Sanz, 105–140. Washington, DC: Georgetown University Press.Google Scholar logo with link to Google Scholar
Zarei, Abbas Ali. 2009. “The Effect of Bimodal, Standard, and Reversed Subtitling on L2 Vocabulary Recognition and Recall.” Pazhuhesh-e Zabanha-ye Khareji 491: 65–85.Google Scholar logo with link to Google Scholar
Filmography
Wachowski, Andy, and Lana Wachowski. 1999. The Matrix.Google Scholar logo with link to Google Scholar
Cited by (1)

Cited by one other publication

Gesa, Ferran & Anastasia Pattemore
2025. Introduction. In Foreign Language Learning from Audiovisual Input [Educational Linguistics, 66], ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 21 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue