Cover not available

Article published In: Journal of Second Language Pronunciation
Vol. 10:3 (2024) ► pp.309–338

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Exploring the potential of textually‑enhanced captioned video to direct learners’ attention to challenging sound contrasts

ORCiD logo with link | University of Murcia
 | University of Barcelona
Published online: 14 January 2025
DOI logo with link
https://doi.org/10.1075/jslp.24043.fou
Abstract
This study investigates the potential of textually-enhanced captioned video to direct EFL learners’ attention to a difficult L2 vowel contrast (English /æ/-/ʌ/) while watching a 30-minute episode of Ted Lasso. Spanish EFL learners (N = 89) were randomly assigned to five different viewing conditions: unenhanced captions (1); enhanced captions with /æ/ and /ʌ/ in two different colours with the target words either in phonetic symbols (2) or orthography (3); or with /æ/ and /ʌ/ in the same colour, either in phonetic symbols (4) or orthography (5). The participants’ eye movements were recorded with a Tobii TX-1200 eye-tracker. The textual enhancement implemented was effective in directing learners’ attention to the target words and attention was generally maintained during the episode. The enhanced conditions promoted higher fixation rates and durations than the unenhanced one. Additionally, the participants’ answers to a post-viewing questionnaire revealed that they considered these types of enhancement useful to help them spot instances of the target sounds and that the captions were not overwhelming.
Article outline
  • 1.Introduction
  • 2.Facilitating pronunciation training with technology
  • 3.Captioned video and pronunciation learning
  • 4.The present study
    • 4.1Study design and procedure
    • 4.2Participants
    • 4.3Target stimuli
    • 4.4Eye-gaze measures
    • 4.5Data analyses
  • 5.Results
    • 5.1Effects of textual enhancement conditions on viewing behavior
    • 5.2Learners’ attention to target words over time
    • 5.3Video viewing habits and perception of textual enhancement
  • 6.Discussion and conclusions
  • Acknowledgements
  • Note
  • References
References (60)
References
Baranowska, K. (2020). Learning most with least effort: subtitles and cognitive load. ELT Journal, 74(2), 105–115. Google Scholar logo with link to Google Scholar
Barriuso, T. A., & Hayes-Harb, R. (2018). High variability phonetic training as a bridge from research to practice. The CATESOL Journal, 30(1), 177–194. Google Scholar logo with link to Google Scholar
Bashori, M., van Hout, R., Strik, H., & Cucchiarini, C. (2024). I Can Speak: improving English pronunciation through automatic speech recognition-based language learning systems. Innovation in Language Learning and Teaching, 18(5), 443–461. Google Scholar logo with link to Google Scholar
Bird, S. A., & Williams, J. N. (2002). The effect of bimodal input on implicit and explicit memory: An investigation into the benefits of within-language subtitling. Applied Psycholinguistics, 23(4), 509–533. Google Scholar logo with link to Google Scholar
Bradlow, A. R., Pisoni, D. B., Akahane-Yamada, R., & Tohkura, Y. (1997). Training Japanese listeners to identify English /r/ and /l/: IV. Some effects of perceptual learning on speech production. Journal of the Acoustical Society of America, 101(4), 2299–2310. Google Scholar logo with link to Google Scholar
Brooks, M. E., Kristensen, K., van Benthem, K. J., Magnusson, A., Berg, C. W., Nielsen, A., Skaug, H. J., Maechler, M., & Bolker, B. M. (2017). GlmmTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modelling. The R Journal, 9(2), 378–400. Google Scholar logo with link to Google Scholar
Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41(4), 977–990. Google Scholar logo with link to Google Scholar
Carlet, A., & Cebrian, J. (2019). Assessing the effect of perceptual training on L2 vowel identification, generalization and long-term effects. In A. M. Nyvad, M. Hejná, A. Højen, A. B. Jespersen & M. Hjortshøj Sørensen (Eds.), A sound approach to language matters: In honor of Ocke-Schwen Bohn (pp. 91–119). Aarhus University.Google Scholar logo with link to Google Scholar
Cebrian, J. (2019). Perceptual assimilation of British English vowels to Spanish monophthongs and diphthongs. Journal of the Acoustical Society of America, 1451, EL52–EL58. .Google Scholar logo with link to Google Scholar
Chun, D. M., Jiang, Y., Meyr, J., & Yang, R. (2015). Acquisition of L2 Mandarin Chinese tones with learner-created tone visualizations. Journal of Second Language Pronunciation, 1(1), 86–114. Google Scholar logo with link to Google Scholar
Cintrón-Valentín, M. C., & García-Amaya, L. (2021). Investigating textual enhancement and captions in L2 grammar and vocabulary: An experimental study. Studies in Second Language Acquisition, 43(5), 1068–1093. Google Scholar logo with link to Google Scholar
Conklin, K., Pellicer-Sánchez, A., & Carrol, G. (2018). Eye-tracking: A guide for applied linguistics research. Cambridge University Press. Google Scholar logo with link to Google Scholar
Darcy, I., & Holliday, J. (2019). Teaching an old work new tricks: phonological updates in the L2 mental lexicon. In J. Levis, C. Nagle & E. Todey (Eds.), Proceedings of the 10thPronunciation in Second Language Learning and Teaching Conference (pp. 10–26). Iowa State University.Google Scholar logo with link to Google Scholar
d’Ydewalle, G., & De Bruycker, W. (2007). Eye movements of children and adults while reading television subtitles. European Psychologist, 12(3), 196–205. Google Scholar logo with link to Google Scholar
Flege, J. E., & Bohn, O. -S. (2021). The revised speech learning model (SLM-r). In R. Wayland (Ed.), Second Language Speech Learning: Theoretical and Empirical Progress (pp. 3–83). Cambridge University Press. Google Scholar logo with link to Google Scholar
Fouz-González, J. (forthcoming). Technology-assisted pronunciation training: Bridging research and pedagogy. University of Toronto Press.
Fouz-González, J. & Mompean, J. A. (2021). Exploring the potential of phonetic symbols and keywords as labels for perceptual training. Studies in Second Language Acquisition, 43(2), 297–328. Google Scholar logo with link to Google Scholar
Galimberti, V., Mora, J. C., & Gilabert, R. (2023). Audio-synchronized textual enhancement in foreign language pronunciation learning from videos. System, 1161, 103078. Google Scholar logo with link to Google Scholar
Guion, S. G., & Pederson, E. (2007). Investigating the role of attention in phonetic learning. In O. -S. Bohn & M. J. Munro (Eds.), Language experience in second language speech learning (pp. 57–77). John Benjamins. Google Scholar logo with link to Google Scholar
Hardison, D. M. (2004). Generalization of computer-assisted prosody training: Quantitative and qualitative findings. Language Learning & Technology, 8(1), 34–52. Google Scholar logo with link to Google Scholar
Hirata, Y. (2004). Computer-assisted pronunciation training for native English speakers learning Japanese pitch and duration contrasts. Computer Assisted Language Learning, 17(3–4), 357–376. Google Scholar logo with link to Google Scholar
Kartushina, N., Hervais-Adelman, A., Frauenfelder, U. H., & Golestani, N. (2015). The effect of phonetic production training with visual feedback on the perception and production of foreign speech sounds. Journal of the Acoustical Society of America, 138(2), 817–832. Google Scholar logo with link to Google Scholar
Kruger, J. -L., Hefer, E., & Matthew, G. (2014). Attention distribution and cognitive load in a subtitled academic lecture: L1 vs. L2. Journal of Eye Movement Research, 7(5), 1–15. Google Scholar logo with link to Google Scholar
Kuhl, P. K., Conboy, B. T., Coffey-Corina, S., Padden, D., Rivera-Gaxiola, M., & Nelson, T. (2008). Phonetic learning as a pathway to language: new data and native language magnet theory expanded (NLM-e). Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 363(1493), 979–1000. Google Scholar logo with link to Google Scholar
Lambacher, S., Martens, W., Kakehi, K., Marasinghe, C., & Molholt, G. (2005). The effects of identification training on the identification and production of American English vowels by native speakers of Japanese. Applied Psycholinguistics, 26(2), 227–247. Google Scholar logo with link to Google Scholar
Lee, M., & Révész, A. (2020). Promoting grammatical development through captions and textual enhancement in multimodal input-based tasks. Studies in Second Language Acquisition, 42(3), 625–651. Google Scholar logo with link to Google Scholar
Mathias, B., & von Kriegstein, K. (2023). Enriched learning: Behavior, brain, and computation. Trends in Cognitive Sciences, 27(1), 81–97. Google Scholar logo with link to Google Scholar
Meara, P., & Miralpeix, I. (2015). V_YesNo Lognostics Vocabulary Test. [URL]
Mitterer, H., & McQueen, J. M. (2009). Foreign subtitles help but native-language subtitles harm foreign speech perception. PLoS ONE, 4(11), e7785. Google Scholar logo with link to Google Scholar
Mohsen, M. A., & Mahdi, H. S. (2021). Partial versus full captioning mode to improve L2 vocabulary acquisition in a mobile-assisted language learning setting: Words pronunciation domain. Journal of Computing in Higher Education, 331, 524–543. Google Scholar logo with link to Google Scholar
Mompean, J. A. & Fouz-González, J. (2021). Phonetic symbols in contemporary pronunciation instruction. RELC Journal, 52(1), 155–168. Google Scholar logo with link to Google Scholar
Mompean, J. A. & Lintunen, P. (2015). Phonetic notation in foreign language teaching and learning: Potential advantages and learners’ views. Research in Language, 13(3), 292–314. Google Scholar logo with link to Google Scholar
Montero Perez, M., Peters, E., & Desmet, P. (2015). Enhancing vocabulary learning through captioned Video: An eye‐tracking study. The Modern Language Journal, 99(2), 308–328. Google Scholar logo with link to Google Scholar
(2018). Vocabulary learning through viewing video: the effect of two enhancement techniques. Computer assisted language learning, 31(1–2), 1–26. Google Scholar logo with link to Google Scholar
Mora, J. C., & Fouz-González, J. (2024). Contrastive input enhancement in captioned video for L2 pronunciation learning. In C. Muñoz & I. Miralpeix (Eds.), Audiovisual Input and Second Language Learning (pp. 154–178). John Benjamins. Google Scholar logo with link to Google Scholar
Mora, J. C. & Mora-Plaza, I. (2019). Contributions of cognitive attention control to L2 speech learning. In A. M. Nyvad, M. Hejná, A. Højen, A. B. Jespersen & M. Hjortshøj Sørensen (Eds.), A sound approach to language matters: In honor of Ocke-Schwen Bohn (pp. 477–499). Aarhus University.Google Scholar logo with link to Google Scholar
Motohashi-Saigo, M. & Hardison, D. M. (2009). Acquisition of L2 Japanese geminates: Training with waveform displays. Language Learning & Technology, 13(2), 29–47. Google Scholar logo with link to Google Scholar
Offerman, H. M., & Olson, D. J. (2016). Visual feedback and second language segmental production: The generalizability of pronunciation gains. System, 591, 45–60. Google Scholar logo with link to Google Scholar
Olson, D. (2014). The benefits of visual feedback on segmental production in L2 classrooms. Language Learning & Technology, 18(3), 173–192. Google Scholar logo with link to Google Scholar
Pattemore, A., & Muñoz, C. (2022). Captions and learnability factors in learning grammar from audio-visual input. JALT CALL Journal, 18(1), 83–109. Google Scholar logo with link to Google Scholar
Pederson, E., & Guion-Anderson, S. (2010). Orienting attention during phonetic training facilitates learning. The Journal of the Acoustical Society of America, 127(2), EL54–EL59. Google Scholar logo with link to Google Scholar
Peterson, R. A. (2021). Finding optimal normalizing transformations via bestNormalize. The R Journal, 13(1), 310–329. Google Scholar logo with link to Google Scholar
Peterson, R. A., & Cavanough, J. E. (2020). Ordered quantile normalization: a semiparametric transformation built for the cross-validation era. Journal of Applied Statistics, 47, (13–15), 2312–2327. Google Scholar logo with link to Google Scholar
Popova, M., & Miralpeix, I. (2024). Maximizing L2 learning from captioned TV viewing. In C. Muñoz & I. Miralpeix (Eds.) Audiovisual input and second language learning (pp. 100–125). John Benjamins. Google Scholar logo with link to Google Scholar
Rehman, I. (2021). Real-time formant extraction for second language vowel production training. [Unpublished doctoral dissertation]. Iowa State University.Google Scholar logo with link to Google Scholar
Roon, K. D., Kang, J., & Whalen, D. H. (2020). Effects of ultrasound familiarization on production and perception of nonnative contrasts. Phonetica, 77(5), 350–393. Google Scholar logo with link to Google Scholar
Sharwood-Smith, M. (1993). Input enhancement in instructed SLA: Theoretical bases. Studies in Second Language Acquisition, 15 (2), 165–179. Google Scholar logo with link to Google Scholar
Scheffler, P., & Baranowska, K. (2023). Learning pronunciation through television series. Language Learning & Technology, 27(1), 1–16. [URL]
Suemitsu, A., Dang, J., Ito, T., & Tiede, M. (2015). A real-time articulatory visual feedback approach with target presentation for second language pronunciation learning. The Journal of the Acoustical Society of America, 138(4), EL382–EL387. Google Scholar logo with link to Google Scholar
Thomson, R. I. (2011). Computer assisted pronunciation training: Targeting second language vowel perception improves pronunciation. CALICO Journal, 28(3), 744–765. Google Scholar logo with link to Google Scholar
(2012). Improving L2 listeners’ perception of English vowels: A computer-mediated approach. Language Learning, 62(4), 1231–1258. Google Scholar logo with link to Google Scholar
VanPatten, B. (1990). Attending to form and content in the input: An experiment in consciousness. Studies in Second Language Acquisition, 12(3), 287–301. Google Scholar logo with link to Google Scholar
Wang, S., Li, J., & Liang, Q. (2024). Visual reinforcement through digital zoom technology in FL pronunciation instruction. Language Learning & Technology, 28(1), 1–26. [URL]
Winke, P., Gass, S., & Sydorenko, T. (2013). Factors influencing the use of captions by foreign language learners: An eye‐tracking study. The Modern Language Journal, 97(1), 254–275. Google Scholar logo with link to Google Scholar
Wisniewska, N., & Mora, J. (2020). Can captioned video benefit second language pronunciation? Studies in Second Language Acquisition, 42(3), 599–624. Google Scholar logo with link to Google Scholar
Zhu, J., Zhang, X., & Li, J. (2024). Using AR filters in L2 pronunciation training: practice, perfection, and willingness to share, Computer Assisted Language Learning, 371(5–6), 1364–1396. Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Azpilicueta-Martínez, Raúl & Nora Ocáriz-Tejada
2025. The impact of audio-synchronised keyword captions on young CLIL learners’ foreign language pronunciation. System 133 ► pp. 103730 ff. DOI logo
Mora, Joan C.
2025. Phonological acquisition in instructed SLA: L2 pronunciation training in and beyond the classroom and individual differences. Language Teaching ► pp. 1 ff. DOI logo
Phillips, Sadi & Isabelle Darcy
2025. Speech Perception in a Second Language. In The Handbook of Second Language Listening, ► pp. 87 ff. DOI logo

This list is based on CrossRef data as of 21 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue