Cover not available

Article published In: Interpreting
Vol. 27:1 (2025) ► pp.1–27

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Fundamental frequency as an acoustic mirror of interpreters’ cognitive states

 | Zhejiang Gongshang University
 | Ghent University
Published online: 8 August 2024
DOI logo with link
https://doi.org/10.1075/intp.00107.sha
Abstract
The present study proposes the acoustic parameter of fundamental frequency (F0) as an alternative method for gauging interpreters’ cognitive load and fatigue during simultaneous interpreting (SI). The data collected from an English-Chinese SI experiment reported in Shao, Z. & Chai, M. (2021). The effect of cognitive load on simultaneous interpreting performance: An empirical study at the local level. Perspectives 29 (5), 778–794. Google Scholar logo with link to Google Scholar were re-analysed for the purposes of this study. Cognitive load was measured as the number of undelivered information chunks within the ear-voice span (EVS), whereas fatigue was assumed to increase with time. F0 variables were correlated with cognitive load within the EVS of selected sentences and with fatigue in one-minute segments distributed over the entire interpreting performance. The results show that cognitive constructs are associated in various ways with F0-related variables. Higher F0 peaks and a wider F0 range appear to be associated with higher cognitive load in a majority of interpreters. A higher mean F0 and, to a lesser extent, higher F0 peaks are associated with fatigue. The findings suggest that F0 could be used as a promising indicator of cognitive load and fatigue in SI.
Article outline
  • 1.Introduction
  • 2.Fundamental frequency and its associated cognitive variables
  • 3.Cognitive load and fatigue in simultaneous interpreting
    • 3.1Cognitive load
    • 3.2Fatigue
    • 3.3The measurement gap
  • 4.Data and methods
    • 4.1Data
    • 4.2Areas of interest
    • 4.3Statistical analysis
  • 5.Results
    • 5.1Fatigue and F0
    • 5.2Cognitive load and F0
  • 6.Discussion
  • 7.Conclusion
  • Note
  • References
References (94)
References
Aaronson, L. S., Teel, C. S., Cassmeyer, V., Neuberger, G. B., Pallikkathayil, L., Pierce, J., Press, A. N., Williams, P. D. & Wingate, A. (1999). Defining and measuring fatigue. Image – The Journal of Nursing Scholarship 31 (1), 45–50. Google Scholar logo with link to Google Scholar
Ackerman, P. L. & Kanfer, R. (2009). Test length and cognitive fatigue: An empirical examination of effects on performance and test-taker reactions. Journal of Experimental Psychology. Applied 15 (2), 163–181. Google Scholar logo with link to Google Scholar
AIIC (2023). Schools directory. [URL] (accessed 30 June 2023).
Ayres, P., Lee, J. Y., Paas, F. & Van Merriënboer, J. (2021). The validity of physiological measures to identify differences in intrinsic cognitive load. Frontiers in Psychology 121, 702538. Google Scholar logo with link to Google Scholar
Behrens, M., Gube, M., Chaabene, H., Prieske, O., Zenon, A., Broscheid, K. C., Schega, L., Husmann, F. & Weippert, M. (2023). Fatigue and human performance: An updated framework. Sports Medicine (Auckland, N.Z.) 53 (1), 7–31. Google Scholar logo with link to Google Scholar
Benson, P. (1995). Analysis of the acoustic correlates of stress from an operational aviation emergency. In I. Trancoso & R. Moore (Eds.), Proceedings of the ESCA-NATO Tutorial and Research Workshop on Speech under Stress. Lisbon: INESC, 61–64.Google Scholar logo with link to Google Scholar
Bjørheim, F., Pavlou, D. G. & Siriwardane, S. C. (2022). Nonlinear fatigue life prediction model based on the theory of the S–N fatigue damage envelope. Fatigue & Fracture of Engineering Material & Structures 45 (5), 1480–1493. Google Scholar logo with link to Google Scholar
Boersma, P. & Weenink, D. (2023). Praat: Doing phonetics by computer, version 6.3.09. [URL] (accessed 2 March 2023).
Boos, M., Kobi, M., Elmer, S. & Jäncke, L. (2022). The influence of experience on cognitive load during simultaneous interpretation. Brain and Language 2341, 105185. Google Scholar logo with link to Google Scholar
Bortfeld, H., Leon, S. D., Bloom, J. E., Schober, M. F. & Brennan, S. E. (2001). Disfluency rates in conversation: Effects of age, relationship, topic, role, and gender. Language and Speech 44 (2), 123–147. Google Scholar logo with link to Google Scholar
Boyer, S., Paubel, P. V., Ruiz, R., El Yagoubi, R. & Daurat, A. (2018). Human voice as a measure of mental load level. Journal of Speech, Language, and Hearing Research 61 (11), 2722–2734. Google Scholar logo with link to Google Scholar
Cañas, J. J., Muñoz-de-Escalona, E. & Morales-Guaman, J. F. (2021). Fundamental frequency as an alternative method for assessing mental fatigue of distance learning teachers. In L. Longo & M. C. Leva (Eds.), Human mental workload: Models and applications. H-WORKLOAD 2021. Cham: Springer, 45–59. Google Scholar logo with link to Google Scholar
Carroll, T., Nix, J., Hunter, E., Emerich, K., Titze, I. & Abaza, M. (2006). Objective measurement of vocal fatigue in classical singers: A vocal dosimetry pilot study. Otolaryngology-Head and Neck Surgery 135 (4), 595–602. Google Scholar logo with link to Google Scholar
Chen, S. (2017). The construct of cognitive load in interpreting and its measurement. Perspectives 25 (4), 640–657. Google Scholar logo with link to Google Scholar
Christodoulides, G. (2016). Effects of cognitive load on speech production and perception. PhD dissertation, Université Catholique de Louvain.
Cowan, N. (2000). The magic number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Science 241, 87–114. Google Scholar logo with link to Google Scholar
De Jong, T. (2010). Cognitive load theory, educational research, and instructional design: Some food for thought. Instructional Science 38 (2), 105–134. Google Scholar logo with link to Google Scholar
Evgrafova, K., Evdokimova, V., Skrelin, P. & Chukaeva, T. (2016). Vocal fatigue in voice professionals: Collecting data and acoustic analysis. ExLing 2016: Proceedings of 7th Tutorial and Research Workshop on Experimental Linguistics, 27 June – 2 July 2016, Saint Petersburg, Russia, 59–62.Google Scholar logo with link to Google Scholar
Gawron, V. J. (2016). Overview of self-reported measures of fatigue. The International Journal of Aviation Psychology 26 (3/4), 120–131. Google Scholar logo with link to Google Scholar
Gieshoff, A. C. & Albl-Mikasa, M. (2022). Interpreting accuracy revisited: A refined approach to interpreting performance analysis, Perspectives 32 (2), 210–228. Google Scholar logo with link to Google Scholar
Gile, D. (1999). Testing the effort models’ tightrope hypothesis in simultaneous interpreting – a contribution. Hermes 231, 153–172.Google Scholar logo with link to Google Scholar
(2021). The effort models of interpreting as a didactic construct. In R. Muñoz Martín, S. Sun & D. Li (Eds.), Advances in cognitive translation studies. Singapore: Springer, 139–160. Google Scholar logo with link to Google Scholar
Grassmann, M., Vlemincx, E., von Leupoldt, A., Mittelstädt, J. M. & Van den Bergh, O. (2016). Respiratory changes in response to cognitive load: A systematic review. Neural Plasticity 2016, 8146809. Google Scholar logo with link to Google Scholar
Griffin, G. R. & Williams, C. E. (1987). The effects of different levels of task complexity on three vocal measures. Aviation, Space, and Environmental Medicine 581, 1165–1170.Google Scholar logo with link to Google Scholar
Habay, J., Proost, M., De Wachter, J., Díaz-García, J., De Pauw, K., Meeusen, R., Van Cutsem, J. & Roelands, B. (2021). Mental fatigue-associated decrease in table tennis performance: Is there an electrophysiological signature? International Journal of Environmental Research and Public Health 18 (24), 12906. Google Scholar logo with link to Google Scholar
Hart, S. G. & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload, New York: North-Holland, 139–183. Google Scholar logo with link to Google Scholar
Hebbar, P. A., Bhattacharya, K., Prabhakar, G., Pashilkar, A. A. & Biswas, P. (2021). Correlation between physiological and performance-based metrics to estimate pilots’ cognitive workload. Frontiers in Psychology 121, 555446. Google Scholar logo with link to Google Scholar
Hirst, D. J. & De Looze, C. (2021). Measuring speech: Fundamental frequency and pitch. In R. Knight & J. Setter (Eds.), The Cambridge handbook of phonetics. Cambridge: Cambridge University Press, 336–361. Google Scholar logo with link to Google Scholar
Hoza, J. (2010). Team interpreting as collaboration and interdependence. Alexandria, VA: RID Press.Google Scholar logo with link to Google Scholar
Hubscher-Davidson, S. (2020). Ethical stress in the translation and interpreting profession. In K. Koskinen & N. K. Pokorn (Eds.), The Routledge handbook of translation and ethics. Abingdon: Routledge, 415–430. Google Scholar logo with link to Google Scholar
Huttunen, K., Keränen, H., Väyrynen, E., Pääkkönen, R. & Leino, T. (2011). Effect of cognitive load on speech prosody in aviation: Evidence from military simulator flights. Applied Ergonomics 42 (2), 348–357. Google Scholar logo with link to Google Scholar
Hyönä, J., Tommola, J. & Alaja, A.-M. (1995). Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Quarterly Journal of Experimental Psychology Section A 48 (3), 598–612. Google Scholar logo with link to Google Scholar
Jacquet, T., Lepers, R., Poulin-Charronnat, B., Bard, P., Pfister, P. & Pageaux, B. (2021). Mental fatigue induced by prolonged motor imagery increases perception of effort and the activity of motor areas. Neuropsychologia 1501, 107701. Google Scholar logo with link to Google Scholar
Johannes, B., Wittels, P., Enne, R., Eisinger, R., Castro, C. A., Thomas, J. L., Adler, A. B. & Gerzer, R. (2007). Non-linear function model of voice pitch dependency on physical and mental load. European Journal of Applied Physiology 1011, 267–276. Google Scholar logo with link to Google Scholar
Kauffman, B. Y., Garey, L., Nordan, A., Jardin, C., Mayorga, N. A., Robles, Z. & Zvolensky, M. J. (2019). The development and initial validation of the fatigue sensitivity questionnaire. Cognitive Behaviour Therapy 48 (5), 419–429. Google Scholar logo with link to Google Scholar
Kitch, J. A. & Oates, J. (1994). The perceptual features of vocal fatigue as self-reported by a group of actors and singers. Journal of Voice 8 (3), 207–214. Google Scholar logo with link to Google Scholar
Korpal, P. (2017). Linguistic and psychological indicators of stress in simultaneous interpreting. Poznań: Wydawnictwo Naukowe UAM.Google Scholar logo with link to Google Scholar
Krejtz, K., Duchowski, A. T., Niedzielska, A., Biele, C. & Krejtz, I. (2018). Eye tracking cognitive load using pupil diameter and microsaccades with fixed gaze. PloS ONE 13 (9), e0203629. Google Scholar logo with link to Google Scholar
Kurz, I. (2001). Conference interpreting: Quality in the ears of the user. Meta 46 (2), 394–409. Google Scholar logo with link to Google Scholar
(2003). Physiological stress during simultaneous interpreting: A comparison of experts and novices. The Interpreters’ Newsletter 121, 51–67.Google Scholar logo with link to Google Scholar
Lambooij, M. T. M., IJsselsteijn, W. A., Fortuin, M. F. & Heynderickx, I. E. J. (2009). Visual discomfort and visual fatigue in stereoscopic displays: A review. Journal of Imaging Science and Technology 53 (3), 1–14. Google Scholar logo with link to Google Scholar
Laukkanen, A.-M., Ilomäki, I., Leppänen, K. & Vilkman, E. (2008). Acoustic measures and self-reports of vocal fatigue by female teachers. Journal of Voice 22 (3), 283–289. Google Scholar logo with link to Google Scholar
Lewis, M. (1997). Pedagogical implications of the lexical approach. In J. Coady & T. Huckin (Eds.), Second language vocabulary acquisition: A rationale for pedagogy. Cambridge: Cambridge University Press, 255–270.Google Scholar logo with link to Google Scholar
Liang, J., Fang, Y., Lv, Q. & Liu, H. (2017). Dependency distance differences across interpreting types: Implications for cognitive demand. Frontiers in Psychology 81, 2132. Google Scholar logo with link to Google Scholar
Lim, J., Wu, W., Wang, J., Detre, J., Dinges, D. & Rao, H. (2010). Imaging brain fatigue from sustained mental workload: An ASL perfusion study of the time-on-task effect. Neuroimage 491, 3426–3425. Google Scholar logo with link to Google Scholar
Lively, S. E., Pisoni, D. B., Van Summers, W. & Bernacki, R. H. (1993). Effects of cognitive workload on speech production: Acoustic analyses and perceptual consequences. Journal of the Acoustical Society of America 93 (5), 2962–2973. Google Scholar logo with link to Google Scholar
Mehta, R. K. & Parasuraman, R. (2013). Neuroergonomics: A review of applications to physical and cognitive work. Frontiers in Human Neuroscience 71, 889. Google Scholar logo with link to Google Scholar
Melnicuk, V., Thompson, S., Jennings, P. & Birrell, S. (2021). Effect of cognitive load on drivers’ state and task performance during automated driving: Introducing a novel method for determining stabilisation time following take-over of control. Accident Analysis & Prevention 1511, 105967. Google Scholar logo with link to Google Scholar
Mizuno, A. (2017). Simultaneous interpreting and cognitive constraints. Journal of College of Literature, Aoyama Gakuin University 581, 1–28.Google Scholar logo with link to Google Scholar
Morris, T. L. & Miller, J. C. (1996). Electrooculographic and performance indices of fatigue during simulated flight. Biological Psychology 42 (3), 343–360. Google Scholar logo with link to Google Scholar
Moser-Mercer, B. (2003). Remote interpreting: Assessment of human factors and performance parameters. [URL] (accessed 21 June 2021)
Nattinger, J. R. & DeCarrico, J. S. (1992). Lexical phrases and language teaching. New York: Oxford University Press.Google Scholar logo with link to Google Scholar
Nilsson, E. J., Bärgman, J., Ljung, Aust M., Matthews, G. & Svanberg, B. (2022). Let complexity bring clarity: A multidimensional assessment of cognitive load using physiological measures. Frontiers in Neuroergonomics 31, 787295. Google Scholar logo with link to Google Scholar
O’Halpin, R. (2010). The perception and production of stress and intonation by children with cochlear implants. PhD dissertation, University of London.
Ots, N. (2021). Cognitive constraints on advance planning of sentence intonation. PLoS ONE 16 (11), e0259343. Google Scholar logo with link to Google Scholar
Paas, F., Renkl, A. & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist 381, 1–4. Google Scholar logo with link to Google Scholar
Pageaux, B. & Lepers, R. (2018). The effects of mental fatigue on sport-related performance. Progress in Brain Research 2401, 291–315. Google Scholar logo with link to Google Scholar
Petsche, H., Etlinger, S. C. & Filz, O. (1993). Brain electrical mechanisms of bilingual speech management: An initial investigation. Electroencephalography and Clinical Neurophysiology 86 (6), 385–394. Google Scholar logo with link to Google Scholar
Pilcher, J. J., Ginter, D. R. & Sadowsky, B. (1997). Sleep quality versus sleep quantity: Relationships between sleep and measures of health, well-being and sleepiness in college students. Journal of Psychosomatic Research 42 (6), 583–596. Google Scholar logo with link to Google Scholar
Pöchhacker, F. (2001). Quality assessment in conference and community interpreting. Meta 461, 410–425. Google Scholar logo with link to Google Scholar
Pöchhacker, F. & Zwischenberger, C. (2010). Survey on quality and role: Conference interpreters’ expectations and self-perception. [URL] (accessed 17 June 2024).
Pym, A. (2008). On omission in simultaneous interpreting: Risk analysis of a hidden effort. In G. Hansen, A. Chesterman & H. Gerzymisch-Arbogast (Eds.), Efforts and models in interpreting and translation research: A tribute to Daniel Gile. Amsterdam: John Benjamins, 83–105.Google Scholar logo with link to Google Scholar
Remacle, A., Garnier, M., Gerber, S., David, C. & Petillon, C. (2018). Vocal change patterns during a teaching day: Inter- and intra-subject variability. Journal of Voice 32 (1), 57–63. Google Scholar logo with link to Google Scholar
Russell, A., Oates, J. & Greenwood, K. M. (1998). Prevalence of voice problems in teachers. Journal of Voice 12 (4), 467–479. Google Scholar logo with link to Google Scholar
Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., Martín Antón, L. J., González-Díez, R. & Carbonero Martín, M. Á. (2023). Using eye tracking technology to analyse cognitive load in multichannel activities in university students. International Journal of Human-Computer Interaction 40 (12), 3263–3281. Google Scholar logo with link to Google Scholar
Schnotz, W. & Kürschner, C. (2007). A reconsideration of cognitive load theory. Educational Psychology Review 191, 469–508. Google Scholar logo with link to Google Scholar
Shao, Z. & Chai, M. (2021). The effect of cognitive load on simultaneous interpreting performance: An empirical study at the local level. Perspectives 29 (5), 778–794. Google Scholar logo with link to Google Scholar
Smets, E. M., Garssen, B., Bonke, B. & De Haes, J. C. (1995). The Multidimensional Fatigue Inventory (MFI) psychometric qualities of an instrument to assess fatigue. Journal of Psychosomatic Research 39 (3), 315–325. Google Scholar logo with link to Google Scholar
Souchet, A. D., Philippe, S., Lourdeaux, D. & Leroy, L. (2022). Measuring visual fatigue and cognitive load via eye tracking while learning with virtual reality head-mounted displays: A review. International Journal of Human-Computer Interaction 38 (9), 801–824. Google Scholar logo with link to Google Scholar
Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction 4 (4), 295–312. Google Scholar logo with link to Google Scholar
Sweller, J., van Merrienboer, J. J. G. & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review 10 (3), 251–296. Google Scholar logo with link to Google Scholar
Sweller, J., van Merriënboer, J. J. G. & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review 31 (2), 261–292. Google Scholar logo with link to Google Scholar
Tanaka, M., Ishii, A. & Watanabe, Y. (2014). Neural effects of mental fatigue caused by continuous attention load: A magnetoencephalography study. Brain Research 15611, 60–66. Google Scholar logo with link to Google Scholar
Tolkmitt, F. J. & Scherer, K. R. (1986). Effect of experimentally induced stress on vocal parameters. Journal of Experimental Psychology: Human Perception and Performance 12 (3), 302–313.Google Scholar logo with link to Google Scholar
Tommola, J. & Niemi, P. (1986). Mental load in simultaneous interpreting: An on-line pilot study. In L. S. Evensen (Ed.), Nordic research in text linguistics and discourse analysis. Trondheim: Tapir. 171–184.Google Scholar logo with link to Google Scholar
Van Puyvelde, M., Neyt, X., McGlone, F. & Pattyn, N. (2018). Voice stress analysis: A new framework for voice and effort in human performance. Frontiers in Psychology 91, 1994. Google Scholar logo with link to Google Scholar
Vogel, A. P. (2010). Acoustic analysis of the effects of sustained wakefulness on speech. Journal of the Acoustical Society of America 128 (6), 3747–3756. Google Scholar logo with link to Google Scholar
Wang, M. (2013). Correlation between error types and local cognitive load in simultaneous interpreting – taking English-Chinese language pair in Cognitive Load Model as an example. MA Thesis, Shanghai International Studies University.
Wang, Y., Zhai, G., Chen, S., Min, X., Gao, Z. & Song, X. (2019). Assessment of eye fatigue caused by head-mounted displays using eye-tracking. BioMed Engineering OnLine 181, 111. Google Scholar logo with link to Google Scholar
Welham, N. V. & Maclagan, M. A. (2003). Vocal fatigue: Current knowledge and future directions. Journal of Voice 17 (1), 21–30. Google Scholar logo with link to Google Scholar
Xu, R., Zhang, C., He, F., Zhao, X., Qi, H., Zhou, P., Zhang, L. & Ming, D. (2018). How physical activities affect mental fatigue based on EEG energy, connectivity, and complexity. Frontiers in Neurology 91, 915. Google Scholar logo with link to Google Scholar
Yamada, Y. & Kobayashi, M. (2018). Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults. Artificial Intelligence in Medicine 911, 39–48. Google Scholar logo with link to Google Scholar
Yap, T. F., Epps, J., Ambikairajah, E. & Choi, E. H. C. (2011). Formant frequencies under cognitive load: Effects and classification. EURASIP Journal on Advances in Signal Processing 2011, 219253. Google Scholar logo with link to Google Scholar
Yoo, G., Kim, H. & Hong, S. (2023). Prediction of cognitive load from electroencephalography signals using long short-term memory network. Bioengineering 10 (3), 361. Google Scholar logo with link to Google Scholar
Zheng, R. Z. & Greenberg, K. (2017). The boundary of different approaches in cognitive load measurement strengths and limitations. In R. Z. Zheng & K. Grenberg (Eds.), Cognitive load measurement and application. New York: Routledge, 45–56.Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue