Cover not available

Article published In: Applied Pragmatics
Vol. 7:2 (2025) ► pp.192–224

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Learner gains from pragmatics instruction across contexts

A mixed-effects analysis

ORCiD logo with link | Kyoto University
ORCiD logo with link | University of Maryland
Published online: 4 November 2025
DOI logo with link
https://doi.org/10.1075/ap.22005.ryl
Abstract
This study reports findings from a mixed-effects analysis on item- and person-levels for five intact classes of learners following a semester-long pragmatics instruction. Data represent multiple-choice responses on an instrument assessing pragmatic awareness from learners at three levels of formal Japanese education (lower- to post-secondary). Instructional interventions differed in the degree of explicitness for both pragmalinguistics and sociopragmatics. Ten speech acts represented pragmalinguistic forms (apologies, complaints, compliments, farewells, greetings, introductions, invitations, offers, requests, and suggestions), with the number of speech acts taught operationalized as part of the explicitness. Twelve relationship status categories represented the sociopragmatic feature (acquaintances, best friends, boss/employee, coworkers, customer/service person, family, friends, girlfriend/boyfriend, wife/husband, professional relationship, neighbors, and strangers). Item-level results reveal a significant effect regarding learner accuracy on items measuring sociopragmatic content relative to pragmalinguistic content. Person-level results reveal that increased explicitness accounted for differences between intervention groups. However, the highest-performing group did not receive the most explicit instruction. The discussion includes an argument for greater use of video-based content in pragmatic instruction and assessment practices and the value of mixed-effects models when analyzing longitudinal classroom research across multiple sites.
Article outline
  • 1.Introduction
  • 2.Background research
    • 2.1Interlanguage pragmatics and models of communicative competence
    • 2.2ILP research
    • 2.3Measuring learning outcomes
    • 2.4Construct and measurement
    • 2.5Noticing hypothesis and explicit instruction
    • 2.6Results from intervention studies
    • 2.7Research questions
  • 3.Method
    • 3.1Context and participants
    • 3.2Intervention conditions
    • 3.3Instructional materials framework
    • 3.4Data collection instrument
      • 3.4.1Pragmatic awareness assessment
      • 3.4.2Instrument reliability measures
    • 3.5Data analysis
  • 4.Results
    • 4.1Item-level effects
    • 4.2Group-level effects
  • 5.Discussion
    • 5.1Item-level analysis: Sociopragmatics and pragmalinguistics
    • 5.2Person-level analysis: Instructional group differences
  • 6.Limitations and conclusion
  • Acknowledgements
  • References
References (89)
References
Alcón-Soler, E. (2005). Does instruction work for learning pragmatics in the EFL context? System, 331, 417–435. Google Scholar logo with link to Google Scholar
Al-Gahtani, S., & Roever, C. (2015). The development of requests by L2 learners of modern standard Arabic: A longitudinal and cross-sectional study. Foreign Language Annals, 48(4), 570–583. Google Scholar logo with link to Google Scholar
Austin, J. (1962). How to do things with words. Clarendon Press.Google Scholar logo with link to Google Scholar
Bachman, L. (1990). Fundamental considerations in language testing. Oxford University Press.Google Scholar logo with link to Google Scholar
Bachman, L., & Palmer, A. (1996). Language testing in practice. Oxford University Press.Google Scholar logo with link to Google Scholar
(2010). Language assessment in practice. Oxford University Press.Google Scholar logo with link to Google Scholar
Badjadi, N. (2016). A meta-analysis of the effects of instructional tasks on L2 pragmatics comprehension and production. In S. F. Tang & L. Logonnathan (Eds.), Assessment for learning within and beyond the classroom (pp. 241–268). Springer. Google Scholar logo with link to Google Scholar
Bardovi-Harlig, K., & Shin, S. Y. (2014). Expanding traditional testing measures with tasks from L2 pragmatics research. Iranian Journal of Language Testing, 4(1), 26–49.Google Scholar logo with link to Google Scholar
Barron, A. (2012). Interlanguage pragmatics: From use to acquisition to second language pedagogy. Language Teaching, 45(1). 44–63. Google Scholar logo with link to Google Scholar
Blum-Kulka, S., & Olshtain, E. (1984). Requests and apologies: A cross-cultural study of speech act realization patterns (CCSARP). Applied Linguistics, 5(3), 196–213. Google Scholar logo with link to Google Scholar
Brown, P., & Levinson, S. (1987). Politeness: Some universals in language use. Cambridge University Press. Google Scholar logo with link to Google Scholar
Bygate, M. (2009). Teaching and testing speaking. In M. Long & C. Doughty (Eds.), The handbook of language teaching (pp. 412–462). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Byon, A. S. (2004). Sociopragmatic analysis of Korean requests: Pedagogical settings. Journal of Pragmatics, 361, 1673–1704. Google Scholar logo with link to Google Scholar
Canale, M. (1983). On some dimensions of language proficiency. In J. Oller, Jr. (Ed.), Issues in language testing research (pp. 333–342). Newbury House.Google Scholar logo with link to Google Scholar
Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. Google Scholar logo with link to Google Scholar
Celce-Murcia, M., Dörnyei, Z., & Thurrell, S. (1995). Communicative competence: A pedagogically motivated model with content specifications. Issues in Applied Linguistics, 6(2), 5–35. Google Scholar logo with link to Google Scholar
Chapelle, C. (1994). Are C-tests valid measures for L2 vocabulary research? Second Language Research, 101, 157–187. Google Scholar logo with link to Google Scholar
(1999). Construct definition and validity inquiry in SLA research. In L. F. Bachman & A. D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 32–70). Cambridge University Press. Google Scholar logo with link to Google Scholar
Cobb, T. (n.d.). Range for texts (Version 3) [Computer software]. Retrieved July 15, 2021, from [URL]
Cohen, A. D., Salazar-Kleiner, S., & Robinson, P. (n.d.). Pragmatics and speech acts. Pragmatics and Speech Acts Bibliography. Retrieved April 2, 2023, from [URL]
Crystal, D. (1997). The Cambridge encyclopedia of language (2nd ed.). Cambridge University Press.Google Scholar logo with link to Google Scholar
Cunnings, I. (2012). An overview of mixed-effects statistical models for second language researchers. Second Language Research, 28(3), 369–382. Google Scholar logo with link to Google Scholar
DeKeyser, R. (2003). Implicit and explicit learning. In C. Doughty & M. Long (Eds.), Handbook of second language acquisition (pp. 313–348). Blackwell. Google Scholar logo with link to Google Scholar
(2007). Introduction: Situating the concept of practice. In R. M. DeKeyser (Ed.), Practice in second language: Perspectives from applied linguistics and cognitive psychology (pp. 1–18). Cambridge University Press. Google Scholar logo with link to Google Scholar
DeKeyser, R., & Larson-Hall, J. (2005). What does the critical period really mean? In J. F. Kroll & A. M. D. De Groot (Eds.), Handbook of bilingualism: Psycholinguistic approaches (pp. 88–108). Oxford University Press.Google Scholar logo with link to Google Scholar
Fernández Guerra, A. & Martínez Flor, A. (2003). Requests in films and in EFL textbooks: A comparison. Elia: Estudios de lingüística inglesa aplicada, 41, 17–34.Google Scholar logo with link to Google Scholar
Fisher, W. P., Jr. (2007). Rasch measurement and the R statistics environment. Transactions of the Rasch Measurement SIG American Educational Research Association, 21(1), 1095.Google Scholar logo with link to Google Scholar
Flowerdew, J. (1990). Problems of speech act theory from an applied perspective. Language Learning, 40(1), 79–105. Google Scholar logo with link to Google Scholar
(2009). Corpora in language teaching. In M. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 327–350). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Frater, P. (2022, May 24). Subscription video continues to gain in Japan — Study. Variety. Retrieved from [URL]
Geis, M. L. (1995). Speech acts and conversational interaction. Cambridge University Press. Google Scholar logo with link to Google Scholar
Golato, A. (2003). Studying compliment responses: A comparison of DCTs and naturally occurring talk. Applied Linguistics, 24(1), 90–121. Google Scholar logo with link to Google Scholar
Grabe, W. (2009). Teaching and testing reading. In M. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 441–462). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Grabowski, K. (2013). Investigating the construct validity of a role-play test designed to measure grammatical and pragmatic knowledge at multiple proficiency levels. In S. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 149–171). Palgrave MacMillan. Google Scholar logo with link to Google Scholar
Grant, L., & Starks, D. (2001). Screening appropriate teaching materials: Closings from textbooks and television soap operas. International Review of Applied Linguistics in Language Teaching, 39(1), 39–50. Google Scholar logo with link to Google Scholar
Henrichsen, L. E. (1984). Sandhi-variation: A filter of input for learners of ESL. Language Learning, 341, 103–126. Google Scholar logo with link to Google Scholar
Houck, N. R., & Fujimori, J. (2010). Teacher, you should lose some weight: Advice giving in English. In D. H. Tatsuki & N. R. Houck (Eds.), Pragmatics: Teaching speech acts (pp. 89–104). TESOL.Google Scholar logo with link to Google Scholar
Hudson, T., Detmer, E., & Brown, J. D. (1992). A framework for testing cross-cultural pragmatics. University of Hawaiʻi at Mānoa.Google Scholar logo with link to Google Scholar
(1995). Developing prototypic measures of cross-cultural pragmatics (Technical Report #7). University of Hawaiʻi, Second Language Teaching and Curriculum Center.Google Scholar logo with link to Google Scholar
Hymes, D. H. (1972). On communicative competence. In J. B. Pride & J. Holmes (Eds.), Sociolinguistics. Penguin.Google Scholar logo with link to Google Scholar
Ito, Y. (2006). The comprehension of English reduced forms by second language learners and its effect on input-intake process. In J. D. Brown & K. Kondo-Brown (Eds.), Perspectives on teaching connected speech to second language speakers (pp. 67–81). University of Hawaiʻi, National Foreign Language Resource Center.Google Scholar logo with link to Google Scholar
Jeon, E. H., & Kaya, T. (2006). Effects of L2 instruction on interlanguage pragmatic development: A meta-analysis. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 165–211). John Benjamins. Google Scholar logo with link to Google Scholar
(2007). Pragmatics in second language learning: Current developments. Foreign Languages in China, 111(6), 17–25.Google Scholar logo with link to Google Scholar
(2009). Locating cognition in second language interaction and learning: Inside the skull or in public view. International Review of Applied Linguistics in Language Teaching, 471. 11–36. Google Scholar logo with link to Google Scholar
Kasper, G., & Dahl, M. (1991). Research methods in interlanguage pragmatics. Studies in Second Language Acquisition, 131, 215–247. Google Scholar logo with link to Google Scholar
Kasper, G., & Rose, K. R. (2002). Pragmatic development in a second language. Blackwell.Google Scholar logo with link to Google Scholar
Knapp, A. (2011). Using English as a lingua franca for (mis-)managing conflict in an international university context: An example from a course in engineering. Journal of Pragmatics, 431, 978–990. Google Scholar logo with link to Google Scholar
Larsen-Freeman, D. (2009). Teaching and testing grammar. In M. Long & C. Doughty (Eds.), Handbook of Language Teaching (pp. 518–542). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Leech, G. (1983). The principles of pragmatics. Longman.Google Scholar logo with link to Google Scholar
Li, S. (2019). Cognitive approaches in L2 pragmatics research. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 113–127). Routledge. Google Scholar logo with link to Google Scholar
Linacre, J. M. (2016). Winsteps® (Version 3.92.0) [Computer software]. Winsteps.com. Retrieved January 1, 2016, from [URL]
Linck, J. A., & Cunnings, I. (2015). The utility and application of mixed-effects models in second language research. Language Learning, 65(S1), 185–207. Google Scholar logo with link to Google Scholar
McNamara, T., & Roever, C. (2006). Language testing: The social dimension. Blackwell. [URL]
Messick, S. (1996). Validity and washback in language testing. ETS Research Report Series, 1996(1), i–18. Google Scholar logo with link to Google Scholar
Miura, J. (2022, September 5). Government sets goal of 100,000 students going abroad by 2027. The Asahi Shimbun. Retrieved Feb 1, 2023, from [URL]
Nation, P., & Chung, M. (2009). Teaching and testing vocabulary. In M. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 543–559). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Nguyen, T. T. M. (2019). Data collection methods in L2 pragmatics research: An overview. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 195–211). Routledge. Google Scholar logo with link to Google Scholar
Nguyen, T. T. M., Do, T. T. H., Nguyen, A. T., & Pham, T. T. T. (2015). Teaching email requests in the academic context: A focus on the role of corrective feedback. Language Awareness, 24(2), 169–195. Google Scholar logo with link to Google Scholar
Norris, J., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. Google Scholar logo with link to Google Scholar
(2003). Defining and measuring SLA. In C. J. Doughty & M. H. Long (Eds.), The handbook of second language acquisition (pp. 716–761). Blackwell. Google Scholar logo with link to Google Scholar
Plonsky, L., & Gass, S. (2011). Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning, 61(2), 325–366. Google Scholar logo with link to Google Scholar
Plonsky, L., & Zhuang, J. (2019). A meta-analysis of second language pragmatics instruction. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 287–307). Routledge.Google Scholar logo with link to Google Scholar
Polio, C., & Williams, J. (2009). Teaching and testing writing. In M. Long & C. Doughty (Eds.), Handbook of language teaching, (pp. 486–517). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
R Core Team. (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing. [URL]
Roever, C. (2011). Testing of second language pragmatics: Past and future. Language Testing, 28(4), 463–481. Google Scholar logo with link to Google Scholar
(2021). Teaching and testing second language pragmatics and interaction: A practical guide (1st ed.). Routledge. Google Scholar logo with link to Google Scholar
Rose, K. (1997). Film in interlanguage pragmatics research. Perspectives, 9(1), 111–144.Google Scholar logo with link to Google Scholar
(2005). The effects of instruction in second language pragmatics. System, 33(3), 385–399. Google Scholar logo with link to Google Scholar
Rylander, J., Clark, P., & Derrah, R. (2013). A video-based method of assessing pragmatic awareness. In S. J. Ross & G. Kasper (Eds.), Assessing second language pragmatics (pp. 65–97). Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Rylander, J., Collins, B., Derrah, R., Ferguson, P., & D’Andrea, J. (2014). Speech acts: Piloting a pragmatics course in the Japanese context. In N. Sonda & A. Krause (Eds.), JALT2013 conference proceedings. Japan Association for Language Teaching.Google Scholar logo with link to Google Scholar
Savignon, S. J. (1983). Communicative competence: Theory and classroom practice. Addison-Wesley.Google Scholar logo with link to Google Scholar
Schegloff, E. (2007). Sequence organization in interaction: A primer in conversation analysis. Cambridge University Press. Google Scholar logo with link to Google Scholar
Schmidt, R. (1993). Consciousness, learning and interlanguage pragmatics. In G. Kasper & S. Blum-Kulka (Eds.), Interlanguage pragmatics (pp. 21–42). Oxford University Press. Google Scholar logo with link to Google Scholar
(2010). Attention, awareness, and individual differences in language learning. In W. M. Chan, S. Chi, K. N. Cin, J. Istanto, M. Nagami, J. W. Sew, T. Suthiwan, & I. Walker (Eds.), Proceedings of CLaSIC 2010 (pp. 721–737). National University of Singapore, Centre for Language Studies.Google Scholar logo with link to Google Scholar
Searle, J. R. (1969). Speech acts. Cambridge University Press. Google Scholar logo with link to Google Scholar
Taguchi, N. (2019). Second language acquisition and pragmatics: An overview. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 1–14). Routledge.Google Scholar logo with link to Google Scholar
Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford University Press.Google Scholar logo with link to Google Scholar
Takahashi, S. (2010). Assessing learnability in second language pragmatics. In A. Trosborg (Ed.), Handbook of pragmatics (Vol. 71, pp. 391–421). Mouton de Gruyter.Google Scholar logo with link to Google Scholar
Takimoto, M. (2006). The effects of explicit feedback on the development of pragmatic proficiency. Language Teaching Research, 101, 393–417. Google Scholar logo with link to Google Scholar
(2009). The effects of input-based tasks on the development of learners’ pragmatic proficiency. Applied Linguistics, 30(1), 1–25. Google Scholar logo with link to Google Scholar
(2012). Assessing the effects of identical task repetition and task-type repetition on learners’ recognition and production of second language request downgraders. Intercultural Pragmatics, 9(1), 71–96. Google Scholar logo with link to Google Scholar
Thomas, J. (1983). Cross-cultural pragmatic failure. Applied Linguistics, 4(2), 91–112. Google Scholar logo with link to Google Scholar
Vandergrift, L., & Goh, C. (2009). Teaching and testing listening comprehension. In M. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 395–411). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Wall, D. (2012). Washback. In G. Flucher & F. Davidson (Eds.), The Routledge handbook of language testing (1st ed., pp. 79–92). Routledge.Google Scholar logo with link to Google Scholar
Walters, A. (2009). Advances in materials design. In M. Long & C. Doughty (Eds.), Handbook of language teaching (pp. 311–326). Wiley-Blackwell. Google Scholar logo with link to Google Scholar
Webb, S., & Rodgers, M. (2009). Vocabulary demands of television programs. Language Learning, 59(2), 335–366. Google Scholar logo with link to Google Scholar
Woodfield, H. (2008). Problematising discourse completion tasks: Voices from verbal report. Evaluation & Research in Education, 21(1), 43–69. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue