Cover not available

Article published In: Interpreting
Vol. 22:2 (2020) ► pp.211–237

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting

 | Southwest University
 | Southwest University
 | Ningbo University
 | Southwest University
Published online: 20 March 2020
DOI logo with link
https://doi.org/10.1075/intp.00040.han
Abstract
Fluency is an important, yet insufficiently understood, construct in interpreting studies. This article reports on an empirical study which explored the relationship between utterance fluency measures and raters’ perceived fluency ratings of English/Chinese consecutive interpreting. It also examined whether such relationship was consistent across interpreting directions and rater types. The results partially supported the categorization of utterance fluency into breakdown, speed and repair fluency. It was also found that mean length of unfilled pauses, phonation time ratio, mean length of run and speech rate had fairly strong correlations with perceived fluency ratings in both interpreting directions and across rater types. Among a number of competing regression models that were built to predict raters’ fluency ratings, a parsimonious model, using mean length of unfilled pauses and mean length of run as predictors, accounted for about 60% of the variance of fluency ratings in both directions and across rater types. These results are expected to help create, rewrite and modify rubrics and scalar descriptors of fluency scales in rater-mediated interpretation assessment and to inform automated scoring of fluency in interpreting.
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Fluency as a multifaceted and multidimensional construct
    • 2.2Fluency research in second language learning
    • 2.3Fluency research in interpreting studies
  • 3.Research questions
  • 4.Method
    • 4.1Participants
    • 4.2Interpretation recordings
    • 4.3Rater-generated fluency ratings
    • 4.4Transcription, annotation and analysis
    • 4.5Calculation of utterance fluency measures
    • 4.6Data analysis
  • 5.Results
  • 6.Discussion
  • 7.Conclusion
  • Acknowledgements
  • References
References (45)
References
Altman, J. (1994). Error analysis in the teaching of simultaneous interpretation: A pilot study. In S. Lambert & B. Moser-Mercer (Eds.), Bridging the gap: Empirical research in simultaneous interpretation. Amsterdam: John Benjamins, 25–38. Google Scholar logo with link to Google Scholar
Barik, H. C. (1975). Simultaneous interpretation: Temporal and quantitative data. Language and Speech 16 (3), 237–270. Google Scholar logo with link to Google Scholar
Bosker, H. R., Pinget, A-F., Quené, H., Sanders, T. & De Jong, N. H. (2012). What makes speech sound fluent? The contributions of pauses, speed and repair. Language Testing 30 (2), 159–175. Google Scholar logo with link to Google Scholar
Bosker, H. R., Quené, H., Sanders, T. & De Jong, N. H. (2014). The perception of fluency in native and nonnative speech. Language Learning 64 (3), 579–614. Google Scholar logo with link to Google Scholar
Cecot, M. (2001). Pauses in simultaneous interpretation: A contrastive analysis of professional interpreters’ performances. The Interpreters’ Newsletter 111, 63–85.Google Scholar logo with link to Google Scholar
Chiaro, D. & Nocella, G. (2004). Interpreters’ perception of linguistic and non-linguistic factors affecting quality: A survey through the World Wide Web. Meta 49 (2), 278–293. Google Scholar logo with link to Google Scholar
Derwing, T. M., Munro, M. J., Thompson, R. I. & Rossiter, M. J. (2009). The relationship between L1 fluency and L2 fluency development. Studies in Second Language Acquisition 31 (4), 533–557. Google Scholar logo with link to Google Scholar
Fu, R-B. (2013). 口译方向性对译语非流利产出的影響 [Directional effects on disfluencies in consecutive interpreting]. Modern Foreign Languages 36 (2), 198–205.Google Scholar logo with link to Google Scholar
Fulcher, G. (1996). Does thick description lead to smart tests? A data-based approach to rating scale construction. Language Testing 13 (2), 208–238. Google Scholar logo with link to Google Scholar
Fulcher, G., Davidson, F. & Kemp, J. (2011). Effective rating scale development for speaking tests: Performance decision trees. Language Testing 28 (1), 5–29. Google Scholar logo with link to Google Scholar
Gerver, D. (1975). A psychological approach to simultaneous interpretation. Meta 20 (2), 119–128. Google Scholar logo with link to Google Scholar
Ginther, A., Dimova, S. & Yang, R. (2010). Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring. Language Testing 27 (3), 379–399. Google Scholar logo with link to Google Scholar
Goldman-Eisler, F. (1972). Segmentation of input in simultaneous translation. Journal of Psycholinguistic Research 1 (2), 127–140. Google Scholar logo with link to Google Scholar
Han, C. (2015). (Para)linguistic correlates of perceived fluency in English-to-Chinese simultaneous interpretation. International Journal of Comparative Literature and Translation Studies 3 (4), 32–37.Google Scholar logo with link to Google Scholar
(2017). Using analytic rating scales to assess English–Chinese bi-directional interpreting: A longitudinal Rasch analysis of scale utility and rater behaviour. Linguistica Antverpiensia, New Series: Themes in Translation Studies 161, 196–215.Google Scholar logo with link to Google Scholar
(2018a). A longitudinal quantitative investigation into the concurrent validity of self and peer assessment applied to English-Chinese bi-directional interpretation in an undergraduate interpreting course. Studies in Educational Evaluation 581, 187–196. Google Scholar logo with link to Google Scholar
(2018b). Latent trait modelling of rater accuracy in formative peer assessment of English-Chinese consecutive interpreting. Assessment and Evaluation in Higher Education 43 (6), 979–994. Google Scholar logo with link to Google Scholar
(2019). A generalizability theory study of optimal measurement design for a summative assessment of English/Chinese consecutive interpreting. Language Testing 36 (3), 419–438. Google Scholar logo with link to Google Scholar
Interagency Language Roundtable. (2015). ILR skill level descriptions for interpretation performance. [URL] (accessed 1 July 2015).
Jin, T. & Mak, B. (2012). Distinguishing features in scoring L2 Chinese speaking performance: How do they work? Language Testing 30 (1), 23–47. Google Scholar logo with link to Google Scholar
Kormos, J. & Dénes, M. (2004). Exploring measures and perceptions of fluency in the speech of second language learners. System 32 (2), 145–164. Google Scholar logo with link to Google Scholar
Lee, J. (2008). Rating scales for interpreting performance assessment. The Interpreter and Translator Trainer 2 (2), 165–184. Google Scholar logo with link to Google Scholar
(2017). University students’ experience of ‘scale-referenced’ peer assessment for a consecutive interpreting examination. Assessment & Evaluation in Higher Education 421 (71), 1015-1029. Google Scholar logo with link to Google Scholar
Lennon, P. (1990). Investigating fluency in EFL: A quantitative approach. Language Learning 40 (3), 387–417. Google Scholar logo with link to Google Scholar
Liu, M. (2013). Design and analysis of Taiwan’s interpretation certification examination. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting. Frankfurt: Peter Lang, 163–178.Google Scholar logo with link to Google Scholar
Mead, P. (2000). Control of pauses by trainee interpreters in their A and B languages. The Interpreters’ Newsletter 101, 89–102.Google Scholar logo with link to Google Scholar
(2005). Methodological issues in the study of interpreters’ fluency. The Interpreters’ Newsletter 131, 39–63.Google Scholar logo with link to Google Scholar
Piccaluga, M., Nespoulous, J-L. & Harmegnies, B. (2005). Disfluencies as a window on cognitive processing: An analysis of silent pauses in simultaneous interpreting. [URL] (accessed 10 May 2016).
Préfontaine, Y., Kormos, J. & Johnson, D. E. (2015). How do utterance measures predict raters’ perceptions of fluency in French as a second language? Language Testing 33 (1), 53–73. Google Scholar logo with link to Google Scholar
Rossiter, M. J. (2009). Perceptions of L2 fluency by native and non-native speakers of English. The Canadian Modern Language Review 65 (3), 395–412. Google Scholar logo with link to Google Scholar
Segalowitz, N. (2010). Cognitive bases of second language fluency. New York: Routledge. Google Scholar logo with link to Google Scholar
Skehan, P. & Foster, P. (1999). The influence of task structure and processing conditions on narrative retelling. Language Learning 49 (1), 93–120.Google Scholar logo with link to Google Scholar
Tabachnick, B. G. & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education.Google Scholar logo with link to Google Scholar
Tavakoli, P. & Skehan, P. (2005). Strategic planning, task structure and performance testing. In R. Ellis (Ed.), Planning and task performance in a second language. Amsterdam: John Benjamins, 239–273. Google Scholar logo with link to Google Scholar
The Institute of Linguists Educational Trust. (2010). Diploma in public service interpreting: handbook for candidates. [URL] (accessed 5 May 2016).
Tissi, B. (2000). Silent pauses and disfluencies in simultaneous interpretation: A descriptive analysis. The Interpreters’ Newsletter 101, 103–127.Google Scholar logo with link to Google Scholar
Towell, R., Hawkins, R. & Bazergui, N. (1996). The development of fluency in advanced learners of French. Applied Linguistics 17 (1), 84–119. Google Scholar logo with link to Google Scholar
Wang, J., Napier, J., Goswell, D. & Carmichael, A. (2015). The design and application of rubrics to assess signed language interpreting performance. The Interpreter and Translator Trainer 9 (1), 83–103. Google Scholar logo with link to Google Scholar
Yang, L-Y. (2015). 中国口译学习者汉英交替传译流利度的探索性研究 [An exploratory study of fluency in English output of Chinese consecutive interpreting learners]. Journal of Zhejiang International Studies University (1), 60–68.Google Scholar logo with link to Google Scholar
Zwischenberger, C. & Pöchhacker, F. (2010). Survey on quality and role: Conference interpreters’ expectations and self-perceptions. Communicate! [URL] (accessed 20 May 2016).
Cited by (21)

Cited by 21 other publications

Han, Chao, Xiaolei Lu & Shirong Chen
2025. Modeling rater judgments of interpreting quality: Ordinal logistic regression using neural-based evaluation metrics, acoustic fluency measures, and computational linguistic indices. Research Methods in Applied Linguistics 4:1 ► pp. 100194 ff. DOI logo
Han, Chao & Yueqing Wang
2025. Conducting replication in translation and interpreting studies. Target. International Journal of Translation Studies 37:3 ► pp. 444 ff. DOI logo
Chmiel, Agnieszka, Przemysław Janikowski, Danijel Koržinek, Agnieszka Lijewska, Marta Kajzer-Wietrzny, Dariusz Jakubowski & Koen Plevoets
2024. Lexical frequency modulates current cognitive load, but triggers no spillover effect in interpreting. Perspectives 32:5 ► pp. 905 ff. DOI logo
Chmiel, Agnieszka, Marta Kajzer-Wietrzny, Danijel Koržinek, Dariusz Jakubowski & Przemysław Janikowski
2024. Syntax, stress and cognitive load, or on syntactic processing in simultaneous interpreting. Translation, Cognition & Behavior 7:1 ► pp. 22 ff. DOI logo
Gu, Chonglong & Binhua Wang
2024. From “Within” to “Beyond” in interpreting studies. Babel. Revue internationale de la traduction / International Journal of Translation / Revista Internacional de Traducción 70:6 ► pp. 783 ff. DOI logo
Guo, Wei, Xun Guo, Junkang Huang & Sha Tian
2024. Modeling listeners’ perceptions of quality in consecutive interpreting: a case study of a technology interpreting event. Humanities and Social Sciences Communications 11:1 DOI logo
Wang, Xiaoman & Binhua Wang
2024. Identifying fluency parameters for a machine-learning-based automated interpreting assessment system. Perspectives 32:2 ► pp. 278 ff. DOI logo
Wang, Xiaoman & Binhua Wang
2025. Advancing automatic assessment of target-language quality in interpreter training with large language models: insights from explainable AI. The Interpreter and Translator Trainer 19:3-4 ► pp. 465 ff. DOI logo
Zhou, Jinhua & Yanping Dong
2024. Effects of note-taking on the accuracy and fluency of consecutive interpreters' immediate free recall of source texts: A three-stage developmental study. Acta Psychologica 248 ► pp. 104359 ff. DOI logo
Han, Chao & Liuyan Yang
2023. Relating utterance fluency to perceived fluency of interpreting. Translation and Interpreting Studies 18:3 ► pp. 421 ff. DOI logo
Lu, Xiaolei & Chao Han
2023. Automatic assessment of spoken-language interpreting based on machine-translation evaluation metrics. Interpreting. International Journal of Research and Practice in Interpreting 25:1 ► pp. 109 ff. DOI logo
Wang, Xiaoman & Lu Yuan
2023. Machine-learning based automatic assessment of communication in interpreting. Frontiers in Communication 8 DOI logo
Khau, Anh Hoang & Van Thuy Mai Huynh
2022. An investigation into oral fluency perceived by teachers and students—in a Vietnamese context of English education. Language Testing in Asia 12:1 DOI logo
Li, Xinguang, Xiaoning Li, Shuai Chen, Shanxian Ma & Fenfang Xie
2022. Neural-based automatic scoring model for Chinese-English interpretation with a multi-indicator assessment. Connection Science 34:1 ► pp. 1638 ff. DOI logo
Chen, Jing & Chao Han
2021. Testing and Assessment of Interpreting in China: An Overview. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 1 ff. DOI logo
Han, Chao
2021. Analytic rubric scoring versus comparative judgment: a comparison of two approaches to assessing spoken-language interpreting. Meta 66:2 ► pp. 337 ff. DOI logo
Han, Chao
2022. Interpreting testing and assessment: A state-of-the-art review. Language Testing 39:1 ► pp. 30 ff. DOI logo
Han, Chao & Kerui An
2021. Using unfilled pauses to measure (dis)fluency in English-Chinese consecutive interpreting: in search of an optimal pause threshold(s). Perspectives 29:6 ► pp. 917 ff. DOI logo
Han, Chao & Xiaolei Lu
2021. Interpreting quality assessment re-imagined: The synergy between human and machine scoring. Interpreting and Society 1:1 ► pp. 70 ff. DOI logo
Wu, Zhiwei
2021. Chasing the Unicorn? The Feasibility of Automatic Assessment of Interpreting Fluency. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 143 ff. DOI logo
Yu, Wenting & Vincent J. van Heuven
2021. Quantitative Correlates as Predictors of Judged Fluency in Consecutive Interpreting: Implications for Automatic Assessment and Pedagogy. In Testing and Assessment of Interpreting [New Frontiers in Translation Studies, ], ► pp. 117 ff. DOI logo

This list is based on CrossRef data as of 12 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue