Cover not available

Article published In: Interpreting
Vol. 18:2 (2016) ► pp.225–252

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Test validation in interpreter certification performance testing

An argument-based approach

 | Southwest University (China)
 | Macquarie University
Published online: 1 November 2016
DOI logo with link
https://doi.org/10.1075/intp.18.2.04han
Over the past decade, interpreter certification performance testing has gained momentum. Certification tests often involve high stakes, since they can play an important role in regulating access to professional practice and serve to provide a measure of professional competence for end users. The decision to award certification is based on inferences from candidates’ test scores about their knowledge, skills and abilities, as well as their interpreting performance in a given target domain. To justify the appropriateness of score-based inferences and actions, test developers need to provide evidence that the test is valid and reliable through a process of test validation. However, there is little evidence that test qualities are systematically evaluated in interpreter certification testing. In an attempt to address this problem, this paper proposes a theoretical argument-based validation framework for interpreter certification performance tests so as to guide testers in carrying out systematic validation research. Before presenting the framework, validity theory is reviewed, and an examination of the argument-based approach to validation is provided. A validity argument for interpreter tests is then proposed, with hypothesized validity evidence. Examples of evidence are drawn from relevant empirical work, where available. Gaps in the available evidence are highlighted and suggestions for research are made.
References (100)
Al-Khanji, R., El-Shiyab, S. & Hussein, R. (2000). On the use of compensatory strategies in simultaneous interpretation. Meta 45 (3), 548–557. Google Scholar logo with link to Google Scholar
ALTA Language Services. (2007). Study of California’s court interpreter certification and registration testing. [URL] (accessed 10 June 2015).Google Scholar logo with link to Google Scholar
American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1966). Standards for educational and psychological tests and manuals. Washington, DC: Author.Google Scholar logo with link to Google Scholar
Anastasi, A. (1986). Evolving concepts of test validation. Annual Review of Psychology 371, 1–16. Google Scholar logo with link to Google Scholar
. (2007). Assessing medical interpreters. The Translator 13 (1), 63–82. Google Scholar logo with link to Google Scholar
. (2009). Using a rubric to assess translation ability: Defining the construct. In C. Angelelli & H.E. Jacobson (Eds.), Testing and assessment in translation and interpreting studies. Amsterdam: John Benjamins, 13–47. Google Scholar logo with link to Google Scholar
Arocha, I.S. & Joyce, L. (2013). Patient safety, professionalization, and reimbursement as primary drivers for National Medical Interpreter Certification in the United States. Translation & Interpreting 5 (1), 127–142.Google Scholar logo with link to Google Scholar
Bachman, L.F. & Palmer, A.S. (1996). Language testing in practice. Oxford, UK: Oxford University Press.Google Scholar logo with link to Google Scholar
. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford, UK: Oxford University Press.Google Scholar logo with link to Google Scholar
Bachman, L.F. (1990). Fundamental considerations in language testing. Oxford, UK: Oxford University Press.Google Scholar logo with link to Google Scholar
. (2003). Constructing an assessment use argument and supporting claims about test taker-assessment task interactions in evidence-centered assessment design. Measurement: Interdisciplinary Research and Perspectives 1 (1), 63–65. Google Scholar logo with link to Google Scholar
. (2005). Building and supporting a case for test use. Language Assessment Quarterly 2 (1), 1–34. Google Scholar logo with link to Google Scholar
Bachman, L.F., Davidson, F. & Milanovic, M. (1996). The use of test method characteristics in the content analysis and design of EFL proficiency tests. Language Testing 13 (2), 125–150. Google Scholar logo with link to Google Scholar
Bond, T.G. & Fox, C.M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). London: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Brennan, R.L. (1998). Misconceptions at the intersection of measurement theory and practice. Educational Measurement: Issues and Practice 17 (1), 5–9. Google Scholar logo with link to Google Scholar
. (2010). Generalizability theory and classical test theory. Applied Measurement in Education 24 (1), 1–21. Google Scholar logo with link to Google Scholar
Briggs, D.C. (2004). Comment: Making an argument for design validity before interpretive validity. Measurement: Interdisciplinary Research and Perspectives 2 (3), 171–174. Google Scholar logo with link to Google Scholar
Campbell, S. & Hale, S. (2003). Translation and interpreting assessment in the context of educational measurement. In G. Anderman & M. Rogers (Eds.), Translation today: Trends and perspectives. Clevedon: Multilingual Matters, 205–224. Google Scholar logo with link to Google Scholar
Cardinet, J., Johnson, S. & Pini, G. (2010). Applying generalizability theory using EduG. New York: Routledge.Google Scholar logo with link to Google Scholar
Certification Commission for Healthcare Interpreters. (2010). Job task analysis study and results. [URL] (accessed 22 May 2015).Google Scholar logo with link to Google Scholar
Chain News Service. (2013). 全国翻译资格考试报考人数近5万 入围中大型考试. [CATTI becomes one of large and medium-scale tests in China, with nearly 50,0000 registered candidates]. [URL] (accessed 10 June 2015).
Chapelle, C.A. (2008). The TOEFL validity argument. In C. Chapelle, M. Enright, & J. Jamieson (Eds.), Building a validity argument for the Test of English as a Foreign Language. London: Routledge, 319–352.Google Scholar logo with link to Google Scholar
. (2012). Validity argument for language assessment: The framework is simple… Language Testing 29 (1), 19–27. Google Scholar logo with link to Google Scholar
Chapelle, C.A., Enright, M.E. & Jamieson, J. (2010). Does an argument-based approach to validity make a difference? Educational Measurement: Issues and Practice 29 (1), 3–13. Google Scholar logo with link to Google Scholar
Chen, J. (2009). Authenticity in accreditation tests for interpreters in China. The Interpreter and Translator Trainer 3 (2), 257–273. Google Scholar logo with link to Google Scholar
. (2011). Language assessment: Its development and future – An interview with Lyle F. Bachman. Language Assessment Quarterly 8 (3), 277–290. Google Scholar logo with link to Google Scholar
Christoffels, I.K., De Groot, A.M.B. & Waldorp, L.J. (2003). Basic skills in a complex task: A graphical model relating memory and lexical retrieval to simultaneous interpreting. Bilingualism: Language and Cognition 6 (3), 201–211. Google Scholar logo with link to Google Scholar
Cronbach, L.J. & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin 52 (4), 281–302. Google Scholar logo with link to Google Scholar
Cronbach, L.J. (1971). Test validation. In R.L. Thorndike (Ed.) Educational measurement (2nd ed.). Washington, DC: American Council on Education, 443–507.Google Scholar logo with link to Google Scholar
Crooks, T.J., Kane, M.T. & Cohen, A.S. (1996). Threats to the valid use of assessments. Assessment in Education: Principles, Policy & Practice 3 (3), 265–286. Google Scholar logo with link to Google Scholar
Darò, V., Lambert, S. & Fabbro, F. (1996). Conscious monitoring of attention during simultaneous interpretation. Interpreting 1 (1), 101–124. Google Scholar logo with link to Google Scholar
Dawrant, A., & Jiang, H. (2001). Conference interpreting in Mainland China. Communicate! [URL] (accessed 10 June 2015).Google Scholar logo with link to Google Scholar
De Groot, A.M.B. (2000). A complex-skill approach to translation and interpreting. In S. Tirkkonen-Condit & R. Jääskeläinen (Eds.), Tapping and mapping the processes of translation and interpreting. Amsterdam: John Benjamins, 53–68. Google Scholar logo with link to Google Scholar
Ebel, R. (1961). Must all tests be valid? American Psychologist 16 (10), 640–647. Google Scholar logo with link to Google Scholar
Eckes, T. (2011). Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments. Frankfurt: Peter Lang. Google Scholar logo with link to Google Scholar
Embretson, S.E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin 93 (1), 179–197. Google Scholar logo with link to Google Scholar
Feuerle, L. (2013). Testing interpreters: Developing, administering, and scoring court interpreter certification exams. Translation & Interpreting 5 (1), 80–93. Google Scholar logo with link to Google Scholar
Fulcher, G., Davidson, F. & Kemp, J. (2011). Effective rating scale development for speaking tests: Performance decision trees. Language Testing 28 (1), 5–29. Google Scholar logo with link to Google Scholar
Giambruno, S. (2013). EU member states country profiles: The current state of affairs in Europe. [URL] (accessed 10 June 2015).Google Scholar logo with link to Google Scholar
Green, R. (2013). Statistical analysis for language testers. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Guion, R. (1977). Content validity: The source of my discontent. Applied Psychological measurement 1 (1), 1–10. Google Scholar logo with link to Google Scholar
Guion, R.M. (1980). On trinitarian doctrines of validity. Professional Psychology 11 (3), 385–398. Google Scholar logo with link to Google Scholar
Gulliksen, H. (1950). Intrinsic validity. American Psychologist 5 (10), 51–517. Google Scholar logo with link to Google Scholar
Hale, S., Garcia, I., Hlavac, J., Kim, M., Lai, M., Turner, B. & Slatyer, H. (2012). Development of a conceptual overview for a new model for NAATI standards, testing and assessment. Sydney, Australia. [URL] (accessed 22 May 2015).Google Scholar logo with link to Google Scholar
Hlavac, J. (2013). A cross-national overview of translator and interpreter certification procedures. Translation & Interpreting 51, 32–65. Google Scholar logo with link to Google Scholar
Jacobs, E.A., Lauderdale, D.S., Meltzer, D., Shorey, J.M., Levinson, W. & Thisted, R.A. (2001). Impact of interpreter services on delivery of health care to limited-English-proficient patients. Journal of General Internal Medicine 16 (7), 468–474. Google Scholar logo with link to Google Scholar
Kane, M.T. (1990). An argument-based approach to validation. Iowa City, Iowa: American College Testing Program.Google Scholar logo with link to Google Scholar
. (1992). An argument-based approach to validity. Psychological Bulletin 112 (3), 527–535. Google Scholar logo with link to Google Scholar
. (1994).Validating interpretive arguments for licensure and certification examinations. Evaluation and the Health Professions 17 (2), 133–159. Google Scholar logo with link to Google Scholar
. (2001). Current concerns in validity theory. Journal of Educational Measurement 38 (4), 319–342. Google Scholar logo with link to Google Scholar
. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research and Perspectives 2 (3), 135–170. Google Scholar logo with link to Google Scholar
. (2006). Validation. In R.L. Brennan (Ed.), Educational measurement (4th ed.). Westport, CT: American Council on Education/Praeger, 17–64.Google Scholar logo with link to Google Scholar
Kane, M.T., Crooks, T. & Cohen, A. (1999). Validating measures of performance. Educational Measurement: Issues and Practice 18 (2), 5–17. Google Scholar logo with link to Google Scholar
Kunnan, A.J. (2010). Test fairness and Toulmin’s argument structure. Language Testing 27 (2), 183–189. Google Scholar logo with link to Google Scholar
Linn, R.L. (1989). Educational measurement (3rd ed.). New York: American Council on Education and Macmillan.Google Scholar logo with link to Google Scholar
Lissitz, R.W. & Samuelsen, K. (2007). A suggested change in terminology and emphasis regarding validity and education. Educational Researcher 36 (8), 437–448. Google Scholar logo with link to Google Scholar
Liu, M. (2013). Design and analysis of Taiwan’s interpretation certification examination. In D. Tsagari & R. van Deemter (Eds.), Assessment issues in language translation and interpreting. Frankfurt: Peter Lang, 163–178.Google Scholar logo with link to Google Scholar
. (2015a). Assessment. In F. Pöchhacker (Ed.), Routledge encyclopedia of interpreting studies. New York: Routledge, 20–22.Google Scholar logo with link to Google Scholar
. (2015b). Certification. In F. Pöchhacker (Ed.), Routledge encyclopedia of interpreting studies. New York: Routledge, 45–46.Google Scholar logo with link to Google Scholar
Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports 3 (3), 635–694. Google Scholar logo with link to Google Scholar
MacCorquodale, K. & Meehl, P.E. (1948). On a distinction between hypothetical constructs and intervening variables. Psychological Review 551, 97–105. Google Scholar logo with link to Google Scholar
McNamara, T.F. (1996). Measuring second language performance. London: Longman.Google Scholar logo with link to Google Scholar
Mehrens, W.A. (1992). Using performance assessment for accountability purposes. Educational Measurement: Issues and Practice 11 (1), 3–9. Google Scholar logo with link to Google Scholar
Messick, S. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist 30 (10), 955–966. Google Scholar logo with link to Google Scholar
. (1981). Evidence and ethics in the evaluation of tests. Educational Researcher 10 (9), 9–20. Google Scholar logo with link to Google Scholar
. (1988). The once and future issues of validity. Assessing the meaning and consequences of measurement. In H. Wainer & H. Braun (Eds.), Test validity. Hillsdale, NJ: Lawrence Erlbaum Associates, 33–45.Google Scholar logo with link to Google Scholar
. (1989). Validity. In R.L. Linn (Ed.), Educational measurement (3rd ed.). New York: American Council on Education and Macmillan, 13–103.Google Scholar logo with link to Google Scholar
. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher 23 (2), 13–23. Google Scholar logo with link to Google Scholar
Mislevy, R.J. (2007). Validity by design. Educational Researcher 36 (8), 463–469. Google Scholar logo with link to Google Scholar
Mislevy, R.J., Almond, R.G. & Lukas, J. (2004). A brief introduction to evidence-centered design (CSE Technical Report 632). [URL] (accessed 10 June 2015). Google Scholar logo with link to Google Scholar
Mislevy, R.J., Steinberg, L.S. & Almond, R.G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives 1 (1), 3–66. Google Scholar logo with link to Google Scholar
Moser, B. (1978). Simultaneous interpretation: A hypothetical model and its practical application. In D. Gerver & H.W. Sinaiko (Eds.), Language, interpretation and communication. New York/London: Plenum Press, 353–368. Google Scholar logo with link to Google Scholar
Office of China Accreditation Tests for Translators and Interpreters. (2005). 二级口译英语同声传译类考试大纲. 外文出版社 [Syllabus of CATTI Level-two Simultaneous Interpreting Test]. Beijing: Foreign Languages Press.Google Scholar logo with link to Google Scholar
PSI Services LLC. (2010). Development and validation of oral and written examinations for medical interpreter certification: Technical report. Burbank, California, USA. [URL] (accessed 22 May 2015).Google Scholar logo with link to Google Scholar
. (2013). Development and validation of oral examinations for Medical Interpreter Certification: Mandarin, Russian, Cantonese, Korean, and Vietnamese forms. [URL] (accessed 22 May 2015).Google Scholar logo with link to Google Scholar
Ra, S. & Napier, J. (2013). Community interpreting: Asian language interpreters’ perspectives. Translation & Interpreting 5 (2), 45–61. Google Scholar logo with link to Google Scholar
Roat, C.E. (2006). Certification of health care interpreters in the United States: A primer, a status report and considerations for national certification. Los Angeles, CA. [URL] (accessed 22 May 2015).Google Scholar logo with link to Google Scholar
Russell, D. & Malcolm, K. (2009). Assessing ASL–English interpreters: The Canadian model of national certification. In C.V. Angelelli & H.E. Jacobson (Eds.), Testing and assessment in translation and interpreting studies: A call for dialogue between research and practice. Amsterdam: John Benjamins, 331–376. Google Scholar logo with link to Google Scholar
Schumacker, R. & Lomax, R.G. (2010). A beginner’s guide to structural equation modeling (3rd ed.). New York & London: Routledge.Google Scholar logo with link to Google Scholar
Shavelson, R.J., Baxter, G.P. & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement 30 (3), 215–232. Google Scholar logo with link to Google Scholar
Shepard, L.A. (1993). Evaluating test validity. Review of Research in Education 191, 405–450.Google Scholar logo with link to Google Scholar
Skinner, B.F. (1945). The operational analysis of psychological terms. Psychological Review 52 (5), 270–277. Google Scholar logo with link to Google Scholar
Slatyer, H., Elder, C., Hargreaves, M. & Luo, K. (2008). An investigation into rater reliability, rater behavior and comparability of test tasks. Sydney: Access Macquarie.Google Scholar logo with link to Google Scholar
Thurstone, L.L. (1932). The reliability and validity of tests. Ann Arbor, MI: Edwards Brothers.Google Scholar logo with link to Google Scholar
Toulmin, S.E. (1958). The uses of argument. Cambridge, UK: Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2003). The uses of argument (2nd ed.). Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Turner, B., Lai, M. & Huang, N. (2010). Error deduction and descriptors – a comparison of two methods of translation test assessment. Translation & Interpreting 2 (1), 11–23.Google Scholar logo with link to Google Scholar
Vermeiren, H., Gucht, J.V. & De Bontridder, L. (2009). Standards as critical success factors in assessments: Certifying social interpreters in Flanders, Belgium. In C.V. Angelelli & H.E. Jacobson (Eds.), Testing and assessment in translation and interpreting studies: A call for dialogue between research and practice. Amsterdam: John Benjamins, 291–330. Google Scholar logo with link to Google Scholar
Xie, Q. & Andrews, S. (2012). Do test design and uses influence test preparation? Testing a model of washback with Structural Equation Modeling. Language Testing 30 (1), 49–70. Google Scholar logo with link to Google Scholar
Yu, D.R. (2005). T&I labor market in China. Sydney, Australia. [URL] (accessed 22 May 2015).Google Scholar logo with link to Google Scholar
Cited by (11)

Cited by 11 other publications

Han, Chao, Mengting Jiang & Qionglu Chen
2025. Rubricizing the assessment practice: A systematic review and meta-analysis of rubrics in rater-mediated assessment of language interpreting. Language Testing DOI logo
Guo, Wei, Xun Guo, Junkang Huang & Sha Tian
2024. Modeling listeners’ perceptions of quality in consecutive interpreting: a case study of a technology interpreting event. Humanities and Social Sciences Communications 11:1 DOI logo
Song, Shuxian & Dechao Li
2023. Aptitude for interpreting: the predictive value of cognitive fluency. The Interpreter and Translator Trainer 17:1 ► pp. 155 ff. DOI logo
Zhang, Yifan & Vahid Aryadoust
2022. A Systematic Review of the Validity of Questionnaires in Second Language Research. Education Sciences 12:10 ► pp. 723 ff. DOI logo
Han, Chao & Xiaolei Lu
2021. Interpreting quality assessment re-imagined: The synergy between human and machine scoring. Interpreting and Society 1:1 ► pp. 70 ff. DOI logo
Han, Chao
2018. Using rating scales to assess interpretation. Interpreting. International Journal of Research and Practice in Interpreting 20:1 ► pp. 63 ff. DOI logo
Han, Chao
2019. Conceptualizing and Operationalizing a Formative Assessment Model for English-Chinese Consecutive Interpreting. In Quality Assurance and Assessment Practices in Translation and Interpreting [Advances in Linguistics and Communication Studies, ], ► pp. 89 ff. DOI logo
Han, Chao
2019. A generalizability theory study of optimal measurement design for a summative assessment of English/Chinese consecutive interpreting. Language Testing 36:3 ► pp. 419 ff. DOI logo
Han, Chao
2022. Interpreting testing and assessment: A state-of-the-art review. Language Testing 39:1 ► pp. 30 ff. DOI logo
Shang, Xiaoqi
2017. Conference interpreting: a trainer’s guide. Perspectives 25:4 ► pp. 682 ff. DOI logo
[no author supplied]
2017.  Teaching Dialogue Interpreting [Benjamins Translation Library, 138], DOI logo

This list is based on CrossRef data as of 12 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue