Cover not available

Article published In: Pragmatics
Vol. 26:1 (2016) ► pp.69–91

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF

Reconsidering the development of the discourse completion test in interlanguage pragmatics

Open Access logo
Available under the Creative Commons Attribution-NonCommercial (CC BY-NC) 4.0 license.
Published online: 1 March 2016
DOI logo with link
https://doi.org/10.1075/prag.26.1.04lab
A survey of the field of Interlanguage Pragmatics (ILP) shows that the Discourse Completion Test (DCT)2, also referred to as a ‘discourse completion task’ or a ‘production questionnaire’, has been the most frequently used instrument to evaluate second/foreign language learners’ ability to perform speech acts in a target language, despite the harsh criticism leveled against its low construct validity and its failure to represent the features of authentic discourse. Interestingly, focusing on the statement of objectives of a number of ILP studies using DCTs, one can notice that such studies rarely refer to the DCT as a language test. In addition, an overview of the DCT design process as described in several ILP studies shows that ever since its adaptation for the study of pragmatic ability (Blum-Kulka, 1982), there has been a tendency to use or adapt one of the existing DCT versions used in previous studies based on the argument of comparability of results. While a number of ILP researchers tried to improve the design of the DCT by the inclusion of rejoinders or by enhancing the prompt material (e.g. Billmyer and Varghese, 2000), few attempts have been made to reconsider the DCT development process. McNamara and Roever (2006: 253) urge for the need for “more research on testing of sociopragmatic knowledge and design of discourse completion tests for testing purposes.”The present paper starts with an overview of the literature about DCTs with special reference to the cognitive validity of the instrument and to previous studies dealing with DCT structure and content. Then, with reference to research in the fields of language testing and psychometrics, it shows that, whether used for research or instructional purposes, the DCT shares several qualities with language tests. As such, it is argued that the DCT should be treated as a language test and not as a questionnaire and should, thus, undergo a rigorous developmental process. Based on recent models of language test construction, the paper concludes with an overview of the stages of DCT development.
References (63)
Bachman, L.F., and A.S. Palmer (1996) Language testing in practice. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Bachman, L.F. (2004) Statistical analyses for language assessment. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Bardovi-Harlig, K., and B.S. Hartford (1993) Learning the rules of academic talk: A longitudinal study of pragmatic change. Studies in Second Language Acquisition 15.3: 279-304. Google Scholar logo with link to Google Scholar
Bardovi‐Harlig, K. (2013) Developing L2 pragmatics. Language Learning 63.1: 68-86. Google Scholar logo with link to Google Scholar
Bax, S. (2013). Readers’ cognitive processes during IELTS reading tests: Evidence from eye tracking. ELT Research Papers 13-06.Google Scholar logo with link to Google Scholar
Beebe, L.M., and M.C. Cummings. (1985) Speech act performance: A function of the data collection procedure? Paper presented at the TESOL convention , New York.
Bergman, M.L., and G. Kasper (1993) Perception and performance in native and non-native apology. In G. Kasper, and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press, pp. 82-107.  BoPGoogle Scholar logo with link to Google Scholar
Bialystok, E. (1993) Symbolic representation and attentional control. In G. Kaspe,r and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press, pp. 43-57.  BoPGoogle Scholar logo with link to Google Scholar
Billmyer, K., and M. Varghese (2000) Investigating instrument-based pragmatic variability: Effects of enhancing discourse completion tests. Applied Linguistics 21.4: 517-552.  BoPGoogle Scholar logo with link to Google Scholar
Blum-Kulka, S. (1982) Learning to say what you mean in a second language: A study of the speech act performance of learners of Hebrew as a second language. Applied Linguistics 3.1: 29-59.  BoPGoogle Scholar logo with link to Google Scholar
Blum-Kulka, S., J. House, and G. Kasper (eds.) (1989) Cross-cultural pragmatics: Requests and apologies. Norwood, NJ: Albex.  BoPGoogle Scholar logo with link to Google Scholar
Boxer, D. (2002) Discourse issues in cross-cultural pragmatics. Annual Review of Applied Linguistics 221: 150-167.  BoPGoogle Scholar logo with link to Google Scholar
Brown, J.D. (2008) Raters, functions, item types and the dependability of L2 pragmatics tests. In E.A. Soler, and A.M. Flor (eds.), Investigating pragmatics in foreign language learning, teaching and testing (Vol. 301). Multilingual Matters. 224-48. Google Scholar logo with link to Google Scholar
Callies, M. (2013) Advancing the research agenda of Interlanguage Pragmatics: The role of learner corpora. In Yearbook of Corpus Linguistics and Pragmatics 2013. New York: Springer, pp. 9-36. Google Scholar logo with link to Google Scholar
Cohen, A.D. (1996) Developing the ability to perform speech acts. Studies in Second Language Acquisition 18.2: 253-269. Google Scholar logo with link to Google Scholar
. (1996) Speech acts. In N. Hornberger, and S. McKay (eds.), Sociolinguistics and language teaching. Cambridge: Cambridge University Press, pp. 382-419.  BoPGoogle Scholar logo with link to Google Scholar
. (2004) Assessing speech acts in a second language. In D. Boxer, and A.D. Cohen (eds.), Studying speaking to inform second language learning (Vol. 8). Multilingual Matters 302-327.Google Scholar logo with link to Google Scholar
. (2008) Teaching and assessing L2 pragmatics: What can we expect from learners? Language Teaching 41.2: 213-235. Google Scholar logo with link to Google Scholar
Cohen, A.D., and E. Olshtain (1994) Researching the production of second-language speech acts. Research methodology in second-language acquisition 143-156.Google Scholar logo with link to Google Scholar
Dornyei, Z. (2003) Questionnaires in second Language Research: Construction, administration, and processing. Mahwah, NJ: Lawrence Erlbraum.Google Scholar logo with link to Google Scholar
Douglas, D. (2000) Assessing languages for specific purposes. Cambridge, UK: Cambridge University Press.Google Scholar logo with link to Google Scholar
Ellis, R. (2004) The definition and measurement of L2 explicit knowledge. Language Learning 541 : 227–275. Google Scholar logo with link to Google Scholar
Faerch. C., and G. Kasper (1984) Pragmatic knowledge: Rules and procedures. Applied Linguistics 5.3: 214-225.  BoPGoogle Scholar logo with link to Google Scholar
Foster, P., and P. Skehan (1996). The influence of planning and task type on second language performance. Studies in Second language acquisition 18.3: 299-323.  BoPGoogle Scholar logo with link to Google Scholar
Golato, A. (2003) Studying compliment responses: A comparison of DCTs and recordings of naturally occurring talk. Applied linguistics 24.1: 90-121.  BoPGoogle Scholar logo with link to Google Scholar
Grabowski, K.C. (2007) Reconsidering the measurement of pragmatic knowledge using a reciprocal written task format. Columbia University Working Papers in TESOL & Applied Linguistics 7.1: 1-48.Google Scholar logo with link to Google Scholar
. (2008) Investigating the construct validity of a performance test designed to measure grammatical and pragmatic knowledge. Spaan Fellow Working Papers in Foreign Language Assessment 61: 131-179.Google Scholar logo with link to Google Scholar
Hinkel, E. (1997) Appropriateness of advice: DCT and multiple choice data. Applied linguistics 18.1: 1-26.  BoPGoogle Scholar logo with link to Google Scholar
Hendriks, B. (2008) Dutch English requests: A study of request performance by Dutch learners of English. In M. Pütz, and J. Neff-van Aertselaer (eds.), Developing contrastive pragmatics: Interlanguage and cross-cultural perspectives (Vol. 31). Berlin: Walter de Gruyter, pp. 335-354.Google Scholar logo with link to Google Scholar
Hudson, T., E. Detmer, and J.D. Brown (1995) Developing prototypic measures of crosscultural pragmatics (Tech. Rep. No 7). Honolulu: University of Hawai’i, Second Language Teaching and Curriculum Center.Google Scholar logo with link to Google Scholar
Hymes, D. (1972) On communicative competence. Sociolinguistics 269-293.Google Scholar logo with link to Google Scholar
Johnston, B., G. Kasper,and S. Ross (1998) Effect of rejoinders in production questionnaires. Applied Linguistics 19.2: 157-182. Google Scholar logo with link to Google Scholar
Kasper, G. (1997) The role of pragmatics in language teacher education. In K. Bardovi-Harlig, and B. Hartford (eds.), Beyond methods. Components of second language teacher education. New York: McGraw Hill, pp. 113-141.Google Scholar logo with link to Google Scholar
. (2010) Interlanguage pragmatics. In M. Fried, J.O. Östman, and J. Verschueren (eds.), Variation and change: Pragmatic perspectives (Vol. 61). Amsterdam: John Benjamins Publishing Company, pp. 141-154.  BoPGoogle Scholar logo with link to Google Scholar
Kasper, G., and M. Dahl (1991) Research methods in interlanguage pragmatics. Studies in Second Language Acquisition 131: 215-247.  BoPGoogle Scholar logo with link to Google Scholar
Kasper, G., and S. Blum-Kulka (eds.) (1993) Interlanguage pragmatics. Oxford: Oxford University Press.  BoPGoogle Scholar logo with link to Google Scholar
Kasper, G., and S. Ross (2013) Assessing second language pragmatics: An overview and introductions. In S. Ross, and G. Kasper (eds.), Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan, pp. 1-40. Google Scholar logo with link to Google Scholar
Korsko, P. (2004) The narrative shape of two-party complaints in Portuguese: A discourse analytic study. Unpublished doctoral dissertation, Teachers College, Columbia University, New York City.Google Scholar logo with link to Google Scholar
Leech, G. (1983) Principles of pragmatics. London: Longman.  BoPGoogle Scholar logo with link to Google Scholar
Lin, M.F. (2014) An interlanguage pragmatic study on Chinese EFL learners’ refusal: Perception and performance. Journal of Language Teaching and Research 5.3: 642-653. Google Scholar logo with link to Google Scholar
Martínez-Flor, A., and E. Usó-Juan (2006) Learners’ use of request modifiers across two University ESP disciplines. Ibérica 121: 23-41.Google Scholar logo with link to Google Scholar
McNamara, T.F., and C. Roever (2006) Language testing: The social dimension. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
Olshtain, E., and L. Weinbach (1993) Interlanguage features of the speech act of complaining. In G. Kasper, and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press.  BoPGoogle Scholar logo with link to Google Scholar
Rintell, E., and C.L. Mitchell (1989) Studying requests and apologies: An inquiry into methods. In S. Blum-Kulka., J. House, and G. Kasper (eds.), Cross-cultural pragmatics: Requests and apologies. Norwood, NJ: Albex, pp. 248-72.Google Scholar logo with link to Google Scholar
Roever, C. (2004) Difficulty and practicality in tests of interlanguage pragmatics. In D. Boxer, and A.D. Cohen. (eds.), Studying speaking to inform second language learning (Vol. 8). Multilingual Matters 283-301.Google Scholar logo with link to Google Scholar
. (2006) Validation of a web-based test of ESL pragmalinguistics. Language Testing 23.2: 229-256. Google Scholar logo with link to Google Scholar
. (2007) DIF in the assessment of second language pragmatics. Language Assessment Quarterly 4.2: 165-189. Google Scholar logo with link to Google Scholar
. (2008) Rater, item and candidate effects in discourse completion tests: A FACETS approach. In E.A. Soler, and A.M. Flor (eds.), Investigating pragmatics in foreign language learning, teaching and testing (Vol. 30). Multilingual Matters 249-266. Google Scholar logo with link to Google Scholar
. (2010) Effects of cultural background in a test of ESL pragmalinguistics: A DIF approach. Pragmatics and language learning 121.Google Scholar logo with link to Google Scholar
. (2011) Testing of second language pragmatics: Past and future. Language Testing 28.4: 463-481. Google Scholar logo with link to Google Scholar
Rose, K.R. (1992) Speech acts and questionnaires: The effect of hearer response. Journal of pragmatics 17.1: 49-62. Google Scholar logo with link to Google Scholar
. (1994) On the validity of discourse completion tests in non-Western contexts. Applied Linguistics 15.1: 1-14. Google Scholar logo with link to Google Scholar
Rose, K.R., and R. Ono (1995) Eliciting speech act data in Japanese: The effect of questionnaire type. Language learning 45.2: 191-223. Google Scholar logo with link to Google Scholar
Ross, S., and G. Kasper (eds.) (2013) Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Rylander, J., P. Clark, and R. Derrah (2013) A video-based method of assessing pragmatic awareness. In S. Ross, and G. Kasper (eds.), Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan, pp. 65-97. Google Scholar logo with link to Google Scholar
Salehi, R. (2014) A comparative analysis of apology strategy: Iranian EFL learners and native English speakers. Procedia-Social and Behavioral Sciences 981: 1658-1665. Google Scholar logo with link to Google Scholar
Samavarchi, L., and H. Allami (2012) Giving condolences by Persian EFL learners: A contrastive sociopragmatic study. International Journal of English Linguistics 2.1: 71-78. Google Scholar logo with link to Google Scholar
Sasaki, M. (1998) Investigating EFL students’ production of speech acts: A comparison of production questionnaires and role plays. Journal of Pragmatics 30.4: 457-484.  BoPGoogle Scholar logo with link to Google Scholar
Schmidt, R. (1993) Consciousness, learning, and interlanguage pragmatics. In G. Kasper, and S. Blum-Kulka (eds.), Interlanguage pragmatics. Oxford: Oxford University Press, pp. 21-42.  BoPGoogle Scholar logo with link to Google Scholar
Thomas, J. (1983) Cross-cultural pragmatic failure. Applied Linguistics 41: 91-112. Google Scholar logo with link to Google Scholar
Walters, F.S. (2013) Interfaces between a discourse completion test and a conversation analysis-informed test of L2 pragmatic competence. In S. Ross, and G. Kasper (eds.), Assessing Second Language Pragmatics. Basingstoke, UK: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Wolfson, N. (1989) Perspectives: Sociolinguistics and TESOL. New York: Newsbury House.  BoPGoogle Scholar logo with link to Google Scholar
Cited by (18)

Cited by 18 other publications

Mcdermid, Campbell & Jacob Lipps
2025. Conversational Regulators in American Sign Language Students. Turkish Journal of Special Education Research and Practice 6:2 ► pp. 1 ff. DOI logo
Afzaal, Muhammad, Xiao Shanshan & Dina Abdel Salam El-Dakhs
2024. A comprehensive bibliometric analysis of speech acts in international journals (2013–2023). Cogent Arts & Humanities 11:1 DOI logo
Hayashi, Yuko, Yusuke Kondo & Yutaka Ishii
2024. Automated speech scoring of dialogue response by Japanese learners of English as a foreign language. Innovation in Language Learning and Teaching 18:1 ► pp. 32 ff. DOI logo
El-Dakhs, Dina Abdel Salam, Jawaher Nasser Alhaqbani & Laila Mardini
2023. How do Saudi EFL learners realize the speech act of request?. Cogent Arts & Humanities 10:2 DOI logo
Loranc, Barbara, Shannon M. Hilliker & Chesla Ann Lenkaitis
2023. Virtual exchange and L2 pragmatic development: analysing learners' formulation of compliments in L2. The Language Learning Journal 51:6 ► pp. 706 ff. DOI logo
Tai, Hsuan-Yu, Chin-Ting Liu & Yuan-Shan Chen
2023. Validation of a multiple-choice implicature test: insights from Chinese EFL learners’ cognitive processes. International Review of Applied Linguistics in Language Teaching 61:4 ► pp. 1481 ff. DOI logo
Rieger, Caroline L.
2022. “I want a real apology”. Pragmatics. Quarterly Publication of the International Pragmatics Association (IPrA) ► pp. 553 ff. DOI logo
Conti, Sergio & Carmen Lepadat
2021. Situation-bound utterances in Chinese as a foreign language: the effectiveness of the identification task. Chinese as a Second Language Research 10:1 ► pp. 1 ff. DOI logo
Sperlich, Darcy, Jaiho Leem & Eui-Jeen Ahn
2021. Explicit and implicit dimensions of sociopragmatic and pragmalinguistic competence . International Review of Applied Linguistics in Language Teaching 59:3 ► pp. 369 ff. DOI logo
Ahmadian, Mohammad Javad
2020. Explicit and implicit instruction of refusal strategies: Does working memory capacity play a role?. Language Teaching Research 24:2 ► pp. 163 ff. DOI logo
Baider, Fabienne H., Georgeta Cislaru & Chantal Claudel
2020. Researching Politeness: From the ‘Classical’ Approach to Discourse Analysis … and Back. Corpus Pragmatics 4:3 ► pp. 259 ff. DOI logo
Hashimoto, Brett J. & Kyra Nelson
2020. Using a corpus in creating and evaluating a DCT. Applied Pragmatics 2:1 ► pp. 80 ff. DOI logo
Lappalainen, Hanna
2019. Imaginary customers and public figures. In It’s not all about you [Topics in Address Research, 1], ► pp. 99 ff. DOI logo
Athimni, Moez
2018. Investigating Assessment Literacy in Tunisia: The Case of EFL University Writing Teachers. In Assessing EFL Writing in the 21st Century Arab World, ► pp. 163 ff. DOI logo
El-Dakhs, Dina Abdel Salam
2018. Investigating the apology strategies of Saudi learners of English. Pragmatics and Society 9:4 ► pp. 598 ff. DOI logo
Verschueren, Jef
2016. Contrastive pragmatics. In Handbook of Pragmatics, DOI logo
Verschueren, Jef
2022. Contrastive pragmatics. In Handbook of Pragmatics [Handbook of Pragmatics, ], ► pp. 349 ff. DOI logo
[no author supplied]
2021. Nominal and Pronominal Address in Jamaica and Trinidad [Topics in Address Research, 3], DOI logo

This list is based on CrossRef data as of 30 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue