Cover not available

Article published In: Translation and Interpreting Studies
Vol. 18:3 (2023) ► pp.421–447

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Relating utterance fluency to perceived fluency of interpreting

A partial replication and a mini meta-analysis

ORCiD logo with link | Xiamen University
ORCiD logo with link | Xiamen University
Published online: 15 November 2022
DOI logo with link
https://doi.org/10.1075/tis.20091.han
Abstract
The prospect of automated scoring for interpreting fluency has prompted investigations into the predictability of human raters’ perceived fluency based on acoustically measured utterance fluency. Recently, Han, Chao, Sijia Chen, Rongbo Fu, and Qing Fan. 2020. “Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting.” Interpreting 22(2): 211–237. Google Scholar logo with link to Google Scholar correlated ten utterance fluency measures with raters’ perceived fluency ratings. To verify previous correlational patterns, the present study partially replicated Han, Chao, Sijia Chen, Rongbo Fu, and Qing Fan. 2020. “Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting.” Interpreting 22(2): 211–237. Google Scholar logo with link to Google Scholar. Our analysis shows that most of the correlations observed in Han, Chao, Sijia Chen, Rongbo Fu, and Qing Fan. 2020. “Modeling the relationship between utterance fluency and raters’ perceived fluency of consecutive interpreting.” Interpreting 22(2): 211–237. Google Scholar logo with link to Google Scholar were successfully replicated. To produce overall interim estimates of the true relationships, we conducted a mini meta-analysis of correlation coefficients reported in six relevant studies, informed by the “continuously cumulating meta-analysis” approach (Braver, Sanford L., Felix J. Thoemmes, and Robert Rosenthal. 2014. “Continuously cumulating meta-analysis and replicability.” Perspectives on Psychological Science 9(3): 333–342. Google Scholar logo with link to Google Scholar). We found that phonation time ratio, mean length of run, and speech rate had relatively strong correlations with perceived fluency. We discuss these findings in light of automated fluency assessment and the need for replication and meta-analysis in translation and interpreting studies.
Article outline
  • Introduction
  • Literature review
    • Relationship between utterance fluency and perceived fluency of interpreting
    • The target study for replication: Han et al. (2020)
    • Replication research
    • Meta-analysis
  • Methods
    • The replication study
    • Data source
    • Calculation of utterance fluency measures
    • Measurement of perceived fluency
    • Data analysis
    • Mini meta-analysis
  • Results
    • The replication results
    • Replication analysis
    • Mini meta-analysis results
  • Discussion
  • Conclusion
  • Notes
  • References
References (43)
References
Allen, Mike and Raymond Preiss. 1993. “Replication and meta-analysis: A necessary connection.” Journal of Social Behavior & Personality 8(6): 9–20.Google Scholar logo with link to Google Scholar
Asendorpf, Jens B., et al. 2013. “Recommendations for increasing replicability in psychology.” European Journal of Personality 27(2): 108–119. Google Scholar logo with link to Google Scholar
Borenstein, Michael, Larry V. Hedges, Julian Higgins, and Hannah R. Rothstein. 2009. Introduction to meta-analysis. Chichester: Wiley. Google Scholar logo with link to Google Scholar
Brandt, Mark J. et al. 2014. “The Replication Recipe: What makes for a convincing replication?” Journal of Experimental Social Psychology 501: 217–224. Google Scholar logo with link to Google Scholar
Braver, Sanford L., Felix J. Thoemmes, and Robert Rosenthal. 2014. “Continuously cumulating meta-analysis and replicability.” Perspectives on Psychological Science 9(3): 333–342. Google Scholar logo with link to Google Scholar
Cecot, Michela. 2001. “Pauses in simultaneous interpretation: A contrastive analysis of professional interpreters’ performances.” The Interpreters’ Newsletter 111: 63–85.Google Scholar logo with link to Google Scholar
Collard, Camille and Bart Defrancq. 2020. Disfluencies in simultaneous interpreting a corpus-based study with special reference to sex. In New empirical perspectives on translation and interpreting, ed. by Lore Vandevoorde, Joke Daems and Bart Defrancq, 264–299. New York: Routledge. Google Scholar logo with link to Google Scholar
Corey, David M., William P. Dunlap, and Michael J. Burke. 1998. “Averaging correlations: Expected values and bias in combined Pearson rs and Fisher’s z transformations.” Journal of General Psychology 125(3): 245–262. Google Scholar logo with link to Google Scholar
Crandall, Christian S. and Jeffrey W. Sherman. 2016. “On the scientific superiority of conceptual replications for scientific progress.” Journal of Experimental Social Psychology 661: 93–99. Google Scholar logo with link to Google Scholar
Easley, Richard W., Charles S. Madden, and Mark G. Dunn. 2000. “Conducting marketing science: The role of replication in the research process.” Journal of Business Research 48(1): 83–92. Google Scholar logo with link to Google Scholar
Gile, Daniel. 2009. “Interpreting studies: a critical view from within.” MonTI 11: 135–155. Google Scholar logo with link to Google Scholar
Goh, Jin X., Judith A. Hall, and Robert Rosenthal. 2016. “Mini meta-analysis of your own studies: Some arguments on why and a primer on how.” Social and Personality Psychology Compass 10(10): 535–549. Google Scholar logo with link to Google Scholar
Han, Chao and Ke-rui An. 2021. “Using unfilled pauses to measure (dis)fluency in English-Chinese consecutive interpreting: in search of an optimal pause threshold(s).” Perspectives 29(6): 917–933. Google Scholar logo with link to Google Scholar
Han, Chao. 2015. “(Para)linguistic correlates of perceived fluency in English-to-Chinese simultaneous interpretation.” International Journal of Comparative Literature and Translation Studies 3(4): 32–37. Google Scholar logo with link to Google Scholar
Hoeppner, Sven. 2019. “A note on replication analysis.” International Review of Law and Economics 591: 98–102. Google Scholar logo with link to Google Scholar
Hubbard, Raymond and J. Scott Armstrong. 1994. “Replications and extensions in marketing: Rarely published but quite contrary.” International Journal of Research in Marketing 11(3): 233–248. Google Scholar logo with link to Google Scholar
Lee, Jieun. 2008. “Rating scales for interpreting performance assessment.” The Interpreter and Translator Trainer 2(2): 165–184. Google Scholar logo with link to Google Scholar
Mead, Peter. 2000. “Control of pauses by trainee interpreters in their A and B languages.” The Interpreters’ Newsletter 101: 89–102.Google Scholar logo with link to Google Scholar
. 2005. “Methodological issues in the study of interpreters’ fluency.” The Interpreters’ Newsletter 131: 39–63.Google Scholar logo with link to Google Scholar
Mellinger, Christopher D. and Thomas A. Hanson. 2017. Quantitative Research Methods in Translation and Interpreting Studies. New York: Routledge. Google Scholar logo with link to Google Scholar
. 2019. “Meta-analyses of simultaneous interpreting and working memory.” Interpreting 21(2): 165–195. Google Scholar logo with link to Google Scholar
. 2020. “Meta-analysis and replication in interpreting studies.” Interpreting 22(1): 140–149. Google Scholar logo with link to Google Scholar
Park, Jung Hwan, Olesya Venger, Doo Yeon Park, and Leonard N. Reid. 2015. “Replication in advertising research, 1980–2012: A longitudinal analysis of leading advertising journals.” Journal of Current Issues and Research in Advertising 36(2): 115–135. Google Scholar logo with link to Google Scholar
Pöchhacker, Franz. 2011. “Replication in research on quality in conference interpreting.” T&I Review 11: 35–58.Google Scholar logo with link to Google Scholar
Pöchhacker, Franz and Cornelia Zwischenberger. 2010. “Survey on quality and role: Conference interpreters’ expectations and self-perceptions.” aiic.org [URL] Last accessed 20 January 2022.
Porte, Graeme and Keith Richards. 2012. “Replication in second language writing research.” Journal of Second Language Writing 21 (3): 284–293. Google Scholar logo with link to Google Scholar
Porte, Graeme. 2013. “Who needs replication?” Computer assisted language instruction Consortium Journal 30 (1): 10–15. [URL].
Rennert, Sylvi. 2010. “The impact of fluency on the subjective assessment of interpreting quality.” The Interpreters’ Newsletter 151: 101–115.Google Scholar logo with link to Google Scholar
Rosenthal, Robert and Ralph L. Rosnow. 1984. “Applying Hamlet’s question to the ethical conduct of research: A conceptual addendum.” American Psychologist 39(5): 561–563. Google Scholar logo with link to Google Scholar
Saldanha, Gabriela and Sharon O’Brien. 2013. Research methodologies in translation studies. London: Routledge. Google Scholar logo with link to Google Scholar
Segalowitz, Norman. 2010. Cognitive bases of second language fluency. New York: Routledge. Google Scholar logo with link to Google Scholar
Song, Shuxian. 2020. Fluency in simultaneous interpreting of trainee interpreters: The perspectives of cognitive, utterance and perceived fluency. Unpublished PhD dissertation. The Hong Kong Polytechnic University. [URL]
Stanley, David J. and Jeffrey R. Spence. 2014. “Expectations for replications: Are yours realistic?” Perspectives on Psychological Science 9(3): 305–318. Google Scholar logo with link to Google Scholar
Tavakoli, Parveneh and Peter Skehan. 2005. Strategic planning, task structure and performance testing. In Planning and task performance in a second language, ed. by Rod Ellis, 239–273. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Valentine, Jeffrey C. et al. 2011. “Replication in prevention science.” Prevention Science 12, 103 (2011). Google Scholar logo with link to Google Scholar
Van Aert, Bobbie C. M. and Marcel A. L. M. van Assen. 2018. “Examining reproducibility in psychology: A hybrid method for combing a statistically significant original study and a replication.” Behavior Research Methods 501: 1515–1539. Google Scholar logo with link to Google Scholar
Wang, Binhua and Li Tao. 2015. “An empirical study of pauses in Chinese-English simultaneous interpreting.” Perspectives 23(1): 124–142. Google Scholar logo with link to Google Scholar
Wen, Hao and Yanping Dong. 2019. “How does interpreting experience enhance working memory and short-term memory: A meta-analysis.” Journal of Cognitive Psychology 31(8): 769–784. Google Scholar logo with link to Google Scholar
Wright, Ben D. and John M. Linacre. 1994. “Reasonable mean-square fit values.” Rasch Measurement Transactions 8(3): 370. [URL]
Yang, Liu-yan. 2015. “An exploratory study of fluency in English output of Chinese consecutive interpreting learners.” Journal of Zhejiang International Studies University 11, 60–68.Google Scholar logo with link to Google Scholar
Cited by (6)

Cited by six other publications

Han, Chao, Xiaolei Lu & Shirong Chen
2025. Modeling rater judgments of interpreting quality: Ordinal logistic regression using neural-based evaluation metrics, acoustic fluency measures, and computational linguistic indices. Research Methods in Applied Linguistics 4:1 ► pp. 100194 ff. DOI logo
Han, Chao, Xiaolei Lu & Qin Fan
2025. Taming generative AI for interpreter education: using large language models in classroom-based assessment of English-Chinese consecutive interpreting. The Interpreter and Translator Trainer 19:3-4 ► pp. 444 ff. DOI logo
Han, Chao, Xiaolei Lu, Weiwei Wang & Shirong Chen
2025. Applying n-gram-based evaluation metrics to assess human interpreting. Interpreting. International Journal of Research and Practice in Interpreting DOI logo
Han, Chao & Yueqing Wang
2025. Conducting replication in translation and interpreting studies. Target. International Journal of Translation Studies 37:3 ► pp. 444 ff. DOI logo
Jiang, Zhaokun & Ziyin Zhang
2025. From black box to transparency: Enhancing automated interpreting assessment with explainable AI in college classrooms. Research Methods in Applied Linguistics 4:3 ► pp. 100237 ff. DOI logo
Han, Chao
2023. Interrogating the predictive validity of aptitude testing for interpreting: a systematic methodological review. The Interpreter and Translator Trainer 17:1 ► pp. 7 ff. DOI logo

This list is based on CrossRef data as of 6 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue