Cover not available

Article published In: Translation and Interpreting at the Interface of Cognition and Emotion
Edited by Ana María Rojo López and Purificación Meseguer Cutillas
[Translation, Cognition & Behavior 7:1] 2024
► pp. 90–115

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB

Measuring psychological immersion through cardiovascular response measures in subtitled films

ORCiD logo with link | Universidad Católica de Temuco
ORCiD logo with link | University of Antwerp
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Open Access publication of this article was funded through a Transformative Agreement with University of Antwerp.

Published online: 10 October 2024
DOI logo with link
https://doi.org/10.1075/tcb.00090.gon
Abstract
This paper presents a conceptual replication of Kruger, Jan-Louis, Stephen Doherty, and Ronny Ibrahim. 2017a. “Electroencephalographic Beta Coherence as an Objective Measure of Psychological Immersion in Film.” Rivista Internazionale di Tecnica della Traduzione 191, 99–111. Google Scholar logo with link to Google Scholar study, investigating the utility of heart rate (HR) and heart rate variability (HRV) indicators as objective measurements of psychological immersion in subtitled audiovisual products. The study employed a quasi-experimental design in ten participants viewing video clips with and without subtitles, while their physiological responses were monitored. Results suggest lower arousal levels in both conditions compared to baseline, with statistically significant differences observed in mean HR, mean RR, and RMSSD for the subtitles condition. While most inferential tests did not yield significant outcomes, descriptive trends indicate decreased arousal relative to the baseline. They also show increased arousal throughout the clip for most indicators and opposing trends for Mean HR, RMSSD and LF/HF between the two conditions. Methodological suggestions include using multiple data points for HR and HRV analysis, exploring ultra short-term measurements, selecting self-contained clips for viewing, and considering post-task baselines for a more accurate resting state representation. Despite limitations due to the small sample size, this study underscores the potential of HR and HRV as measures of immersion in translated audiovisual products, emphasising the need for larger sample sizes in future research to enhance statistical power and generalizability.
Article outline
  • 1.Introduction
  • 2.Psychological immersion
  • 3.Materials and methods
    • 3.1Participants
    • 3.2Procedures
    • 3.3Measures
    • 3.4Analysis
  • 4.Results and discussion
    • 4.1Self-reports on immersion
    • 4.2Descriptive results on HR and HRV
    • 4.3HR and HRV intervals
    • 4.4Scenes as a continuum
    • 4.5English level of the participants
  • 5.Discussion and suggestions
    • 5.1Data points in audiovisual products
    • 5.2HR and HRV ultra short-term measurements
    • 5.3Materials
    • 5.4Immersion and arousal
    • 5.5Baseline
  • 6.Conclusions
  • Notes
  • References
References (56)
References
Agrawal, Sarvesh, Adèle Simon, Søren Bech, Klaus Bærentsen, and Søren Forchhammer. 2020. “Defining Immersion: Literature Review and Implications for Research on Audiovisual Experiences.” Journal of the Audio Engineering Society 68 (6): 404–417. Google Scholar logo with link to Google Scholar
Baek, Hyun J., Chul-Ho Cho, Jaegon Cho, and Jong-Min Woo. 2015. “Reliability of Ultra-Short-Term Analysis as a Surrogate of Standard 5-Min Analysis of Heart Rate Variability.” Telemed J E Health 211: 404–414. Google Scholar logo with link to Google Scholar
Baños, Rosa, Cristina Botella, Mariano Alcañiz, Víctor Liaño, Belén Guerrer, and Beatriz Rey. 2004. “Immersion and Emotion: Their Impact on the Sense of Presence.” Cyberpsychol. Behav. 71: 734–741. Google Scholar logo with link to Google Scholar
Castaldo, Rossana, Paolo Melillo, Umberto Bracale, Marco Caserta, Maria Triassi, and Leandro Pecchia. 2015. “Acute Mental Stress Assessment via Short Term HRV Analysis in Healthy Adults: A Systematic Review with Meta-Analysis.” Biomedical Signal Processing and Control 181: 370–377. Google Scholar logo with link to Google Scholar
Chaume, Frederic. 2018. “An Overview of Audiovisual Translation: Four Methodological Turns in a Mature Discipline.” Journal of Audiovisual Translation 1 (1): 40–63. Google Scholar logo with link to Google Scholar
Díaz Cintas, Jorge, and Agnieszka Szarkowska. 2020. “Introduction: Experimental Research in Audiovisual Translation — Cognition, Reception, Production.” The Journal of Specialised Translation (33): 3–16.Google Scholar logo with link to Google Scholar
Di Lascio, Elena, Shkurta Gashi, and Silvia Santini. 2018. “Unobtrusive Assessment of Students’ Emotional Engagement during Lectures Using Electrodermal Activity Sensors.” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2 (3): 1–21. Google Scholar logo with link to Google Scholar
Fairclough, Stephen, Katie Ewing, and Jenna Roberts. 2009. “Measuring Task Engagement as an Input to Physiological Computing.” Proceedings — 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII, 1–9. Google Scholar logo with link to Google Scholar
Flis, Gabriela, Adam Sikorski, and Agnieszka Szarkowska. 2020. “Does the Dubbing Effect Apply to Voice-Over? A Conceptual Replication Study on Visual Attention and Immersion.” Journal of Specialised Translation (33): 41–69.Google Scholar logo with link to Google Scholar
Fryer, Louise. 2013. Putting it into Words: The Impact of Visual Impairment on Perception, Experience and Presence. PhD thesis, Goldsmiths College University of London.
Fryer, Louise and Jonathan Freeman. 2012. “Cinematic Language and the Description of Film: Keeping AD Users in the Frame.” Perspectives: Studies in Translatology 21 (3): 412–426. Google Scholar logo with link to Google Scholar
Gambier, Yves, and Elena di Giovanni, eds. 2018. Reception Studies and Audiovisual Translation. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Greco, Gian Maria, Anna Jankowska, and Agnieszka Szarkowska. 2022. “Addressing Methodological Issues in Experimental Research in Audiovisual Translation.” Translation Spaces 11 (1): 1–11. Google Scholar logo with link to Google Scholar
Green, Melanie, and Timothy Brock. 2013. Transport narrative questionnaire. Measurement Instrument Database for the Social Science. Retrieved from [URL]
Iturregui-Gallardo, Gonzalo, and Anna Matamala. 2021. “Audio Subtitling: Dubbing and Voice-Over Effects and Their Impact on User Experience.” Perspectives 29 (1): 64–83. Google Scholar logo with link to Google Scholar
Jamovi project. (2023). Jamovi (Version 2.3) [Computer Software]. [URL]
Jankowska, Anna, Joanna Pilarczyk, Kinga Wołoszyn, and Michał Kuniecki. 2022. “Enough is Enough: How Much Intonation is Needed in the Vocal Delivery of Audio Description?” Perspectives 31 (4): 705–723. Google Scholar logo with link to Google Scholar
Kim, Taeyong, and Frank Biocca. 1997. “Telepresence via Television: Two Dimensions of Telepresence May Have Different Connections to Memory and Persuasion.” Journal of Computer-Mediated Communication 3 (2). Google Scholar logo with link to Google Scholar
Kim, Hye-Geum, Eun-Jin Cheon, Dai-Seg Bai, Young Lee, and Bon-Hoon Koo. 2018. “Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature.” Psychiatry Investigation 15 (3): 235–245. Google Scholar logo with link to Google Scholar
Kruger, Haidee, and Jan-Louis. Kruger. 2017. “Cognition and Reception.” In The Handbook of Translation and Cognition, edited by Fabio Alves, and Arnt Jakobsen, 71–89. New Jersey: John Wiley & Sons. Google Scholar logo with link to Google Scholar
Kruger, Jan-Louis, María Soto-Sanfiel, Stephen Doherty, and Ronny Ibrahim. 2016. “Towards a Cognitive Audiovisual Translatology: Subtitles and Embodied Cognition”. In Reembedding Translation Process Research, edited by Ricardo Muñoz, 171–194. London: John Benjamins. Google Scholar logo with link to Google Scholar
Kruger, Jan-Louis, Stephen Doherty, and Ronny Ibrahim. 2017a. “Electroencephalographic Beta Coherence as an Objective Measure of Psychological Immersion in Film.” Rivista Internazionale di Tecnica della Traduzione 191, 99–111. Google Scholar logo with link to Google Scholar
Kruger, Jan-Louis, Stephen Doherty, and María Soto-Sanfiel. 2017b. “Original Language Subtitles: Their Effects on the Native and Foreign Viewer.” Comunicar 501, 23–32. Google Scholar logo with link to Google Scholar
Laborde, Sylvain, Emma Mosley, and Julian Thayer. 2017. “Heart Rate Variability and Cardiac Vagal Tone in Psychophysiological Research — Recommendations for Experiment Planning, Data Analysis, and Data Reporting.” In Frontiers in Psychology 81: 213. Google Scholar logo with link to Google Scholar
Lemhöfer, Kristin, and Mirjam Broersma. 2012. “Introducing LexTALE: A Quick and Valid Lexical Test for Advanced Learners of English.” Behavior Research Methods 44 (2), 325–343. Google Scholar logo with link to Google Scholar
Lemma, Anna, Bethany Kok, and Tania Singer. 2015. “Is Meditation Always Relaxing? Investigating Heart Rate, Heart Rate Variability, Experienced Effort and Likeability during Training of Three Types of Meditation.” International Journal of Psychophysiology 97 (1): 38–45. Google Scholar logo with link to Google Scholar
Lipponen, Jukka, and Mika Tarvainen. 2019. “A Robust Algorithm for Heart Rate Variability Time Series Artefact Correction Using Novel Beat Classification.” Journal of Medical Engineering & Technology 43(3): 173–181. Google Scholar logo with link to Google Scholar
Lohani, Monika, Brennan Payne, and David Strayer. 2019. “A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving.” Frontiers in Human Neuroscience. 131: 57. Google Scholar logo with link to Google Scholar
Loudon, Gareth, and Gina Deininger. 2016. “The Physiological Response During Divergent Thinking.” Journal of Behavioral and Brain Science 6 (1): 28–37. Google Scholar logo with link to Google Scholar
McCarthy, Cameron, Nikhilesh Pradhan, Calum Redpath, and Andy Adler. 2016. “Validation of the Empatica E4 wristband.” 2016 IEEEEMBS International Student Conference (ISC), Ottawa, ON, Canada, 1–4. Google Scholar logo with link to Google Scholar
Moses, Ziev, Linda Luecken, and Eason, James. 2007. “Measuring Task-Related Changes in Heart Rate Variability.” 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society: 644–647. Lyon, France: IEEE. Google Scholar logo with link to Google Scholar
Munoz M. Loreto, Arie van Roon, Harriëtte Riese, Chris Thio, Emma Oostenbroek, Iris Westrik, Eco de Geus, Ron Gansevoort, Joop Lefrandt, Ilja Nolte, and Harold Sneider. 2015. “Validity of (Ultra-) Short Recordings for Heart Rate Variability Measurements.” PLOS ONE 10 (9). Google Scholar logo with link to Google Scholar
Nussinovitch, Udi, Keren Elishkevitz, Keren Katz, Moshe Nussinovitch, Shlomo Segev, Benjamin Volovitz, and Naomi Nussinovitch. 2011. “Reliability of Ultra-Short ECG Indices for Heart Rate Variability.” Annals of Noninvasive Electrocardiology 16 (2):117–22. Google Scholar logo with link to Google Scholar
Olalla-Soler, Christian, Nicoletta Spinolo, and Ricardo Muñoz. 2023. “Under Pressure? A Study of Heart Rate and Heart-Rate Variability Using the SmarTerp CAI tool.” HERMES — Journal of Language and Communication in Business (63): 119–142. Google Scholar logo with link to Google Scholar
Orero, Pilar, Stephen Doherty, Jan-Louis Kruger, Anna Matamala, Jan Pedersen, Elisa Perego, Pablo Romero Fresco, Sara Rovira-Esteva, Olga Soler, and Agnieszka Szarkowska. 2018. “Conducting Experimental Research in Audiovisual Translation (AVT): A Position Paper.” Journal of specialised translation 301: 105–126. [URL]
Price, Mathew, Natasha Mehta, Erin Tone, and Page Anderson. 2011. “Does Engagement with Exposure Yield Better Outcomes? Components of Presence as a Predictor of Treatment Response for Virtual Reality Exposure Therapy for Social Phobia.” Journal of Anxiety Disorder. 251: 763–770. Google Scholar logo with link to Google Scholar
Raffi, Francesca. 2020. “The Impact of Italian Dubbing on Viewers’ Immersive Experience: An Audience Reception Study.” Online Journal of Communication and Media Technologies 10 (3). Google Scholar logo with link to Google Scholar
Ramos Caro, Marina. 2015. “The Emotional Experience of Films: Does Audio Description Make a Difference?” Translator 21 (1): 68–94. Google Scholar logo with link to Google Scholar
. 2016. “Testing Audio Narration: The Emotional Impact of Language in Audio Description.” Perspectives 24 (4): 606–634. Google Scholar logo with link to Google Scholar
Ramos Caro, Marina, Laura Espín López, and Ana Rojo López. 2021. “The Psychophysiological Impact of Audio Described Porn.” Onomázein Revista de Lingüística Filología y Traducción (8): 104–127. Google Scholar logo with link to Google Scholar
Rojo Lopez, Ana, Marina Ramos Caro, and Laura Espin Lopez. 2021. “Audio Described vs. Audiovisual Porn: Cortisol, Heart Rate and Engagement in Visually Impaired vs. Sighted Participants.” Frontiers in Psychology 121: 661452. Google Scholar logo with link to Google Scholar
Rojo López, Ana, and Paweł Korpal. 2020. “Through Your Skin to Your Heart and Brain: A Critical Evaluation of Physiological Methods in Cognitive Translation and Interpreting Studies.” Linguistica Antverpiensia, New Series — Themes in Translation Studies 191: 191–217. Google Scholar logo with link to Google Scholar
Rojo López, Ana, Paula Cifuentes-Férez, and Laura Espín López. 2021. “The Influence of Time Pressure on Translation Trainees’ Performance: Testing the Relationship Between Self-Esteem, Salivary Cortisol and Subjective Stress Response.” PLOS ONE 16 (9): e0257727. Google Scholar logo with link to Google Scholar
Romero Fresco, Pablo. 2020. “The Dubbing Effect: An Eye-Tracking Study on how Viewers Make Dubbing Work.” The Journal of Specialised Translation 33: 17–40.Google Scholar logo with link to Google Scholar
Salahuddin, Lizawati, Jaegeol Cho, Myeong G. Jeong, and Desok Kim. 2007. “Ultra Short-Term Analysis of Heart Rate Variability for Monitoring Mental Stress in Mobile Settings.” 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society: 4656–4659. Google Scholar logo with link to Google Scholar
Schuurmans, Angela, Peter de Looff, Karin S. Nijhof, Catarina Rosada, Ron H. J. Scholte, Arne Popma, and Roy Otten. 2020. “Validity of the Empatica E4 wristband to Measure Heart Rate Variability (HRV)Pparameters: A Comparison to Electrocardiography (ECG).” Journal of Medical Systems 44 (11): 190. Google Scholar logo with link to Google Scholar
Shaffer, Fred, and J. P. Ginsberg. 2017. “An Overview of Heart Rate Variability Metrics and Norms.” Frontiers in Public Health 51: 258. Google Scholar logo with link to Google Scholar
Shaffer, Fredric, Steven Shearman, and Zachary Meehan. 2016. “The Promise Of Ultra-Short-Term (UST) Heart Rate Variability Measurements”. Biofeedback 441: 229–33. Google Scholar logo with link to Google Scholar
Spinolo, Nicoletta, Christian Olalla-Soler, and Ricardo Muñoz. 2022. “Finding a Way Into an Interpreter’s Heart: Methodological Considerations on Heart-Rate Variability Building on an Exploratory Study.” Interpreters Newsletter 271, 63–87. Google Scholar logo with link to Google Scholar
Szarkowska, Agnieszka, and Julianna Boczkowska. 2020. “Colour Coding Subtitles in Multilingual Films — A Reception Study.” Perspectives 30 (3): 520–536. Google Scholar logo with link to Google Scholar
Tal-Or, Nurit, and Jonathan Cohen. 2010. “Understanding Audience Involvement: Conceptualizing and Manipulating Identification and Transportation.” Poetics 381: 402–418. Google Scholar logo with link to Google Scholar
Task Force of The European Society of Cardiology and The North American Society of Pacing and Electrophysiology (1996). “Heart Rate Variability.” Eur Heart J 17/281.Google Scholar logo with link to Google Scholar
Veltman, J. A. and Anthony Gaillard. 1993. “Indices of Mental Workload in a Complex Task Environment.” Neuropsychobiology 28(1–2): 72–75. Google Scholar logo with link to Google Scholar
Walczak, Agnieszka, and Louis Fryer. 2017. “Creative Description: The Impact of Audio Description Style on Presence in Visually Impaired Audiences.” British Journal of Visual Impairment 35 (1): 6–17. Google Scholar logo with link to Google Scholar
. 2017. “Vocal Delivery of Audio Description by Genre: Measuring Users’ Presence.” Perspectives 261: 69–83. Google Scholar logo with link to Google Scholar
Wissmath, Bartholomäus, David Weibel, and Rudolf Groner. 2009. “Dubbing or Subtitling?: Effects on Spatial Presence, Transportation, Flow, and Enjoyment.” Journal of Media Psychology 21 (3): 114–125.
Cited by (2)

Cited by two other publications

Baghery Moghaddam, Fatemeh & Ghasem Modarresi
2025. Exploring the role of reading emotions, emotion regulation, and translation involvement in translation performance. FORUM. Revue internationale d’interprétation et de traduction / International Journal of Interpretation and Translation 23:2 ► pp. 161 ff. DOI logo
Jalalian Daghigh, Ali, Yuezeng Niu & Pei Soo Ang
2025. A multimethod inquiry into translators’ emotions in dealing with socio-politically sensitive news texts. Perspectives 33:4 ► pp. 858 ff. DOI logo

This list is based on CrossRef data as of 5 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue