Cover not available

Article published In: Translation, Cognition & Behavior
Vol. 7:2 (2024) ► pp.265–290

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Effects of raters’ nativeness and interpreting expertise on the assessment of speech fluency and comprehensibility of interpreter trainees

 | Nahavand Higher Education Complex, Bu-Ali Sina University, Iran
 | Leiden University | University of Pannonia
 | Arak University
Published online: 13 March 2025
DOI logo with link
https://doi.org/10.1075/tcb.24006.yen
Abstract
Speech fluency and comprehensibility have been examined more thoroughly in L1 and L2 research than in interpreting studies. We studied the fluency and comprehensibility of 25 interpreter trainees, whose spontaneous English speech samples were rated on 9-point scales by 6 American and 6 Persian judges, who were either interpreting experts or lays. Samples were rated once as originally recorded, and a month later with the silent pauses edited out. Native English judges proved less lenient (−.21) in their fluency ratings than the Persians; no nativeness effect was found for comprehensibility. Editing out pauses increased the fluency ratings (+.71) across all four rater groups but comprehensibility was not significantly affected (+.10). Both American and Persian experts rated the edited versions as more fluent (+1.80) than lays (+1.03). Experts may judge fluency more analytically, which will be helpful to them when providing feedback.
Article outline
  • 1.Introduction
  • 2.Background
    • 2.1Fluency and comprehensibility in interpreting
    • 2.2Pause phenomena in speech
  • 3.Current study
  • 4.Method
    • 4.1Materials
    • 4.2Raters
    • 4.3Procedure
  • 5.Results
    • 5.1Preliminaries
    • 5.2Main analysis
  • 6.Discussion
  • 7.Conclusion
  • Acknowledgment
  • References
References (68)
References
Arons, Barry. 1993. “Speech Skimmer: Interactively Skimming Recorded Speech.” Paper presented at the 6th Annual ACM Symposium on User Interface Software and Technology, San Jose, CA. [URL]
Barranco-Droege, Rafael. 2015. “Too fast to be true? Exploring time compression in simultaneous interpreting.” Speech Communication, 751, 84–96. Google Scholar logo with link to Google Scholar
Bakti, Mária. 2009. “Speech disfluencies in simultaneous interpretation.” In Selected Papers of the CETRA Research Seminar in Translation Studies 2008. Edited by Dries De Crom, 1–18. Elkins Park, PA: CETRA. [URL]Google Scholar logo with link to Google Scholar
Beattie, Geoffrey W., and Brian Butterworth. 1979. “Contextual Probability and Word Frequency as Determinants of Pauses and Errors in Spontaneous Speech.” Language and Speech 22 (3): 201–211. Google Scholar logo with link to Google Scholar
Boersma, Paul, and David Weenink. 2012. Praat: Doing Phonetics by Computer [Computer Program]. [URL] (Version 3.5.18)Google Scholar logo with link to Google Scholar
Bosker, Hans Rutger, Hugo Quené, Ted Sanders, and Nivja H. de Jong. 2014. ““Native ‘um’s Elicit Prediction of Low-frequency Referents, but Non-native ‘um’s do not.””. Journal of Memory and Language 751: 104–116. Google Scholar logo with link to Google Scholar
Bosker, Hans Rutger, Anne France Pinget, Hugo Quené, Ted Sanders, & and Nivja H. de Jong. 2014. “What Makes Speech Sound Fluent? The Contributions of Pauses, Speed and Repairs.” Language Testing 30 (2): 159–175. Google Scholar logo with link to Google Scholar
Bosker, Hans Rutger, Jade Tjiong, Hugo Quené, Ted Sanders, and Nivja H. de Jong. 2015. “Both Native and Nonnative Disfluencies Trigger Listeners’ Attention.” Poster presented at the International Congress of Phonetic Sciences Satellite Meeting: Disfluency in Spontaneous Speech, Edinburgh.
Campione, Estelle, and Jean Véronis. 2002. “A Large-Scale Multilingual Study of Silent Pause Duration.” In Proceedings of Speech Prosody 2002. Edited by Bernard Bel and Isabelle Marlien, 199–202. Google Scholar logo with link to Google Scholar
Chiaro, Delia, and Giuseppe Nocella. 2004. “Interpreters’ Perception of Linguistic and Non-linguistic Factors Affecting Quality: A Survey through the World Wide Web.” Meta 49 (2): 278–293. Google Scholar logo with link to Google Scholar
Collados Ais, Ángela. 1998. La evaluación de la calidad en interpretación simultánea. La importancia de la comunicación no verbal [The Evaluation of Quality in Simultaneous Interpretation. The Importance of Non-verbal Communication]. Granada: Comares.Google Scholar logo with link to Google Scholar
. 2002. “Quality assessment in simultaneous interpreting: The importance of nonverbal communication.” In The Interpreting Studies Reader. Edited by Franz Pöchhacker, and Miriam Shlesinger, 327–336. London: Routledge.Google Scholar logo with link to Google Scholar
Derwing, Tracy, and Murray J. Munro. 1997. “Accent, Intelligibility, and Comprehensibility: Evidence from Four L1s.” Studies in Second Language Acquisition 19 (1): 1–16. Google Scholar logo with link to Google Scholar
Derwing, Tracy, Marian J. Rossiter, Murray Munro, J., and Ron I. Thomson. 2004. “L2 Fluency: Judgments on Different Tasks.” Language Learning 54 (4): 655–679. Google Scholar logo with link to Google Scholar
Derwing, Tracy, Murray Munro, J., and Ron. I. Thomson, eds. 2022. The Routledge Handbook of Second Language Acquisition and Speaking. New York: Routledge. Google Scholar logo with link to Google Scholar
Ginther, April, Slobodanka Dimova, and Rui Yang. 2010. “Conceptual and Empirical Relationships between Temporal Measures of Fluency and Oral English Proficiency with Implications for Automated Scoring.” Language Testing 27 (3): 379–399. Google Scholar logo with link to Google Scholar
Guillot, Marie-Noelle. 1999. Fluency and its Teaching. Clevedon: Multilingual Matters.Google Scholar logo with link to Google Scholar
Gooskens, Charlotte, and Renée van Bezooijen. 2014. “The Effect of Pause Insertion on the Intelligibility of Danish among Swedes.” In Above and Beyond the Segments: Experimental Linguistics and Phonetics. Edited by Johanneke Caspers, Yiya Chen, Willemijn Heeren, Jos Pacilly, Niels O. Schiller, and Ellen van Zanten, 96–108. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Hahn, Laura D. 2004. “Primary Stress and Intelligibility: Research to Motivate the Teaching of Suprasegmentals.” TESOL Quarterly 38 (2): 201–223. Google Scholar logo with link to Google Scholar
Holmes, Virginia. 1988. “Hesitations and Sentence Planning.” Language and Cognitive Processes 3 (4): 323–361. Google Scholar logo with link to Google Scholar
de Jong, Nivja H., Margarita P. Steinel, Arjen F. Florijn, Rob Schoonen, and Jan H. Hulstijn. 2012. “Facets of Speaking Proficiency.” Studies in Second Language Acquisition 34 (1): 5–35. Google Scholar logo with link to Google Scholar
de Jong, Nivja H. 2016. “Fluency in Second Language Assessment.” In Handbook of Second Language Assessment. Edited by Dina Tsagari, and Jayanti Banerjee, 203–218. Berlin: Mouton de Gruyter. Google Scholar logo with link to Google Scholar
Fraenkel, Jack. R., Wallen, Norman E., and Hyun, Helen. 2006. How to Design and Evaluate Research in Education. Boston. MA: McGraw Hill.Google Scholar logo with link to Google Scholar
Goldman-Eisler, Frieda. 1968. “Psycholinguistics Experiments in Spontaneous Speech.” London: Academic Press.Google Scholar logo with link to Google Scholar
Gósy, Mária. 2007. “Disfluencies and self-monitoring.” Govor 24 (2): 91–110.Google Scholar logo with link to Google Scholar
Kahng, Jimin. 2014. “Exploring Utterance and Cognitive Fluency of L1 and L2 English Speakers: Temporal Measures and Stimulated Recall.” Language Learning 64 (4): 809–854. Google Scholar logo with link to Google Scholar
Kalina, Sylvia. 2005. “Quality Assurance for Interpreting Processes.” Meta 50 (2): 769–784. Google Scholar logo with link to Google Scholar
. 2015. “Measure for Measure — Comparing Speeches with their Interpreted Versions.” In Interpreting Quality: A Look around and ahead. Edited by Cornelia Zwischenberger, and Martina Behr, 15–34. Berlin: Frank & Timme.Google Scholar logo with link to Google Scholar
Kang, Okim, Donald Rubin, and Lucy Pickering. 2010. “Suprasegmental Measures of Accentedness and Judgments of Language Learner Proficiency in Oral English.” The Modern Language Journal 94 (4): 554–566. Google Scholar logo with link to Google Scholar
Kennedy, Sara, and Pavel Trofimovich. 2008. “Intelligibility, Comprehensibility, and Accentedness of L2 Speech: The role of Listener Experience and Semantic Context.” Canadian Modern Language Review 64 (3): 459–490. Google Scholar logo with link to Google Scholar
Kormos, Judit. 2000. “The Timing of Self-repairs in Second Language Speech Production.” Studies in Second Language Acquisition 22 (2): 145–169. Google Scholar logo with link to Google Scholar
Kormos, Judit, and Mariann Dénes. 2004. “Exploring Measures and Perceptions of Fluency in the Speech of Second Language Learners.” System 32 (2): 145–164. Google Scholar logo with link to Google Scholar
Kurz, Ingrid. 1993. “Conference interpretation: Expectations of Different User Groups.” The Interpreters’ Newsletter 51: 3–16. [URL]
. 2008. “The Impact of Non-native English on Students’ Interpreting Performance.” In Efforts and Models in Interpreting and Translation Research. Edited by Gyde Hansen, Andrew Chesterman, and Heidrun Gerzymisch-Arbogast, 179–792. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Levelt, Willem J. M. 1983. “Monitoring and Self-repair in Speech.” Cognition 14 (1): 41–104. Google Scholar logo with link to Google Scholar
Lennon, Paul. 1990. “Investigating Fluency in EFL: A Quantitative Approach.” Language Learning 40 (3): 387–417. Google Scholar logo with link to Google Scholar
Liu, HYPERLINK “[URL] Minhua, Chia-Chien Chan, and Shao-Chuan Wu. 2008. “Interpretation Evaluation Practices: Comparison of Eleven Schools in Taiwan, China, Britain, and the USA.” Compilation and Translation Review 1 (1): 1–42. [URL]Google Scholar logo with link to Google Scholar
MacGregor, Lucy J., Martin Corley, and David I. Donaldson. 2010. “Listening to the Sound of Silence: Disfluent Silent Pauses in Speech have Consequences for Listeners.” Neuropsychologia 48 (14): 3982–3992. Google Scholar logo with link to Google Scholar
Maclay, Howard, and Charles E. Osgood. 1959. “Hesitation Phenomena in Spontaneous English Speech.” Word 15 (1): 19–44. Google Scholar logo with link to Google Scholar
Mead, Peter. 2005. “Methodological Issues in the Study of Interpreters’ Fluency.” The Interpreters’ Newsletter 131: 39–63. [URL]
Munro, Murray J., and Tracy Derwing. 1995. “Foreign Accent, Comprehensibility, and Intelligibility in the Speech of Second Language Learners.” Language Learning 45 (1): 73–97. Google Scholar logo with link to Google Scholar
Munro, Murray J., Tracy Derwing, and Susan L. Morton. 2006. “The Mutual Intelligibility of L2 Speech.” Studies in Second Language Acquisition 28 (1): 113–131. Google Scholar logo with link to Google Scholar
Pawley, Andrew, and Frances H. Syder. 2000. “The One Clause at a Time Hypothesis.” In Perspectives on fluency. Edited by Heidi Riggenbach, 163–191. Ann Arbor, MI: University of Michigan Press.Google Scholar logo with link to Google Scholar
Pöchhacker, Franz. 1994. “Quality Assurance in Simultaneous Interpreting.” In Teaching Translation and Interpreting: Insights, Aims, Visions. Selected papers from the Second ‘Language International’ Conference, Elsinore, Denmark, 4–6 June 1993. Edited by Cay Dollerup, and Annette Lindegaard, 233–242. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
. 2012. “Interpreting Quality: Global Professional Standards?” In Interpreting in the Age of Globalization: Proceedings of the 8th National Conference and International Forum on Interpreting. Edited by Wen Ren, 305–318. Beijing: Foreign Language Teaching and Research Press.Google Scholar logo with link to Google Scholar
Pöchhacker, Franz, and Cornelia Zwischenberger. 2010. “Survey on Quality and Role: Conference Interpreters’ Expectations and Self-perceptions.” Communicate! – A Webzine for Conference Interpreters and the Conference Industry 531: 1–11. [URL]Google Scholar logo with link to Google Scholar
Pradas Macías, Macarena. 2003. “Repercusión del Intraparámetro Pausas Silenciosas en la Fluidez: Influencia en las Expectativas y en la Evaluación de la Calidad en Interpretación Simultánea [Impact of Silent Intraparameter Pauses on Fluency: Influence on Expectations and Quality Assessment in Simultaneous Interpretation].” PhD dissertation, University of Granada.
. 2006. “Probing Quality Criteria in Simultaneous Interpreting.” Interpreting 8 (1): 25–43. Google Scholar logo with link to Google Scholar
Rennert, Sylvi. 2010. “The Impact of Fluency on the Subjective Assessment of Interpreting Quality.” The Interpreters’ Newsletter 151: 101–115. Accessed [URL]
Rossiter, Marian J. 2009. “Perceptions of L2 Fluency by Native and Non-native Speakers of English.” Canadian Modern Language Review 65 (3): 395–412. Google Scholar logo with link to Google Scholar
Scharpff, Peter J. and Heuven, Vincent. J. van. 1988. “Effects of pause insertion on the intelligibility of low-quality speech”. In Proceedings of the 7th FASE/Speech-88 Symposium. Edited by William A. Ainsworth, and John N. Holmes, 261–268. Edinburgh: The Institute of Acoustics. [URL]
Segalowitz, Norman. 2016. “Second Language Fluency and its Underlying Cognitive and Social Determinants.” International Review of Applied Linguistics 54 (2): 79–95. Google Scholar logo with link to Google Scholar
Suzuki, Yuichi. 2020. “Optimizing Fluency Training for Speaking Skills Transfer: Comparing the Effects of Blocked and Interleaved Task Repetition.” Language Learning 71 (2): 285–325. Google Scholar logo with link to Google Scholar
Tavakoli, Parvaneh. 2011. “Pausing Patterns: Differences between L2 Learners and Native Speakers.” ELT Journal 65 (1): 71–79. Google Scholar logo with link to Google Scholar
van Heuven, Vincent J., and Peter J. Scharpff. 1991. “Acceptability of several speech pausing strategies in low quality speech synthesis: interaction with intelligibility”. In Proceedings of the 12th International Congress of Phonetic Sciences, Aix-en-Provence, 458–461. [URL]Google Scholar logo with link to Google Scholar
Wood, David. 2010. Formulaic Language and Second Language Speech Fluency: Background, Evidence and Classroom Applications. London: ContinuumGoogle Scholar logo with link to Google Scholar
Yenkimaleki, Mahmood. 2021. “Prosody Training Benefits in Perception vs. Production Skills in Simultaneous Interpreting: An Experimental Study”. Dutch Journal of Applied Linguistics. 101, 1–15. Google Scholar logo with link to Google Scholar
Yenkimaleki, Mahmood, and Vincent J. van Heuven. 2018. “The Effect of Teaching Prosody Awareness on Interpreting Performance: An Experimental Study of Consecutive Interpreting from English into Farsi.” Perspectives: Studies in Translatology 26 (1): 84–99. Google Scholar logo with link to Google Scholar
. 2019. “The Relative Contribution of Computer Assisted Prosody Training vs. Instructor Based Prosody Teaching in Developing Speaking Skills by Interpreter Irainees: An Experimental Study.” Speech Communication, 1071: 48–57. Google Scholar logo with link to Google Scholar
. 2022. “Comparing the Nativeness vs. Intelligibility Approach in Prosody Instruction for Developing Speaking Skills by Interpreter Trainees: An Experimental Study.” Speech Communication 1371: 92–102. Google Scholar logo with link to Google Scholar
. 2023. “Effect of Pedagogic Intervention in Enhancing Speech Fluency by EFL Students: A Longitudinal Study.” Language Teaching Research 0 (0): 1–24. Google Scholar logo with link to Google Scholar
. 2024a. “Developing Interpreter Trainees’ Speech Comprehensibility: Does Nativeness of the Instructor Matter?” Teaching English as a Second or Foreign Language 27 (4):1–29. Google Scholar logo with link to Google Scholar
. 2024b. “Perception of ‘Broad’ and ‘Narrow’ Fluency in the EFL Performance of Student Interpreters.” Journal of Applied Linguistics and Professional Practice 18 (2): 246–263. Google Scholar logo with link to Google Scholar
Yenkimaleki, Mahmood, Vincent J. van Heuven, and Hossein Moradimokhles. 2023. “The Effect of Prosody Instruction in Developing Listening Comprehension Skills by Interpreter Trainees: Does Methodology Matter?” Computer Assisted Language Learning 361 (5–6): 968–1004. Google Scholar logo with link to Google Scholar
Yu, Wenting, and Vincent J. van Heuven. 2013. “Effects of Immediate Repetition at Different Stages of Consecutive Interpreting Training. An Experimental Study.” In Linguistics in the Netherlands 2013. Edited by Suzanne Aalberse, and Anita Auer, 201–213. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
. 2017. “Predicting Judged Fluency of Consecutive Interpreting from Acoustic Measures: Potential for Automatic Assessment and Pedagogy.” Interpreting: International Journal of Research and Practice in Interpreting 19 (1): 47–68. Google Scholar logo with link to Google Scholar
. 2021. “Quantitative Correlates as Predictors of Judged Fluency in Consecutive Interpreting: Implications for Automatic Assessment and Pedagogy.” In Testing and Assessment of Interpreting. Edited by Jing Chen, and Chao Han, 117–142. Berlin: Springer Nature. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue