Cover not available

Article In: Applied Pragmatics: Online-First Articles

A methodological synthesis of research on the effects of instruction in L2 pragmatics

ORCiD logo with link | University of Otago
ORCiD logo with link | University of Languages and International Studies | Vietnam National University Hanoi
This content is being prepared for publication; it may be subject to changes.
Abstract
This study synthesizes research methodologies employed in instructed second language pragmatics. We analyzed 99 instructional effects studies published since 2010 to present. Each study was coded for sampling features, research designs, use of statistics, and reporting practices. Results highlighted both methodological strengths and limitations. Notably, we observed growing attention to younger learners and exploration of innovative instructional techniques. Many studies also use delayed post-tests to assess long-term effects and report rater reliability to enhance interpretation. Additionally, there is a trend toward including instructional materials and sample instruments to support replication efforts. However, limitations persist, including small, non-randomized samples, absence of control groups, complex treatment designs, methodological bias toward explicit knowledge in outcome measurement, as well as inadequate reporting of various aspects of data and methodologies. Based on these findings, we put forward recommendations to enhance research design and reporting practices in future L2 pragmatics studies.
Article outline
  • 1.Introduction
  • 2.Instructed L2 pragmatics
  • 3.Prior methodological syntheses of instructed L2 pragmatics research
  • 4.Research questions
  • 5.Methods
    • 5.1Study identification
    • 5.2Coding
  • 6.Results
    • 6.1Research Question 1: Sample features in instructed L2 pragmatics research
    • 6.2Research Question 2: General features of study design in instructed L2 pragmatics research
      • 6.2.1Sample size planning
      • 6.2.2Randomization and multisite sampling
      • 6.2.3Control groups
      • 6.2.4Equivalence of groups before treatment
      • 6.2.5Delayed post-testing
    • 6.3Research Question 3: Design features specific to instructed L2 pragmatics research
      • 6.3.1Study types
      • 6.3.2Research questions and target features
      • 6.3.3Treatment designs
      • 6.3.4Treatment length and frequency
      • 6.3.5Teachers
      • 6.3.6Outcome measures
    • 6.4Research Question 4: Reporting practices in instructed L2 pragmatics research
      • 6.4.1Descriptive statistics
      • 6.4.2Inferential statistics
      • 6.4.3Statistical tests and assumption checks
      • 6.4.4Reliability estimates
  • 7.Discussion
    • 7.1Participants and research contexts
    • 7.2Sample sizes
    • 7.3Randomization, control groups, and delayed post-testing
    • 7.4Study types and research agenda
    • 7.5Learning targets
    • 7.6Treatment designs
    • 7.7Teachers
    • 7.8Outcome measures
    • 7.9Reporting practices
  • 8.Conclusion
  • Notes
  • References
References (46)
References
Alemi, M., & Haeri, N. (2020). Robot-assisted instruction of L2 pragmatics: Effects on young EFL learners’ speech act performance. Language Learning and Technology, 24(2), 86–103. [URL].
Almalki, Z., & Jones, C. (2022). ‘Why did you do that?’ The effects of instruction on recognition and production of informal second party complaints. The Language Learning Journal, 50(4), 443–459. Google Scholar logo with link to Google Scholar
Bardovi-Harlig, K. (2015a). Operationalising conversation in studies of instructional effect in L2 pragmatics. System, 481, 21–34. Google Scholar logo with link to Google Scholar
(2015b). Designing instructional effect studies for L2 pragmatics: A guide for teachers and researchers. In S. Gesuato, F. Bianchi, & W. Cheng (Eds.), Teaching, learning and investigating pragmatics (pp. 135–164). Cambridge Scholars Publishing.Google Scholar logo with link to Google Scholar
(2018). Matching modality in L2 pragmatics research design. System, 751, 13–22. Google Scholar logo with link to Google Scholar
(2023). Pragmatics as a pedagogical target. In C. Chapelle (Ed.), Encyclopedia of Applied Linguistics (2nd ed., pp. 1–7). Wiley Blackwell. Google Scholar logo with link to Google Scholar
Byrnes, H. (2013). Notes from the editor. Modern Language Journal, 971, 825–827. [URL]
Cohen, A. (2016). The teaching of pragmatics by native and non-native language teachers: What they know and what they report doing. Studies in Second Language Learning and Teaching, 6(4), 561–585. Google Scholar logo with link to Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd ed.). Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Fritz, E., Dormer, R., Sumi, S., & Kudo, T. (2022). The acquisition of formulaic sequences in EFL email writing. English for Specific Purposes, 651, 15–29. Google Scholar logo with link to Google Scholar
Goo, J., & Mackey, A. (2013). The case against the case against recasts. Studies in Second Language Acquisition, 351, 127–165. Google Scholar logo with link to Google Scholar
Hernández, T. A. (2011). Re-examining the role of explicit instruction and input flood on the acquisition of Spanish discourse markers. Language Teaching Research, 15(2), 159–182. Google Scholar logo with link to Google Scholar
Hudson, T., & Llosa, L. (2015). Design issues and inference in experimental L2 research. Language Learning, 65(S1), 76–96. Google Scholar logo with link to Google Scholar
Jeon, E. H., & Kaya, T. (2006). Effects of L2 instruction on interlanguage pragmatic development: A meta-analysis. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 165–211). John Benjamins. Google Scholar logo with link to Google Scholar
Larson-Hall, J. (2005). A guide to doing statistics in second language research using SPSS. Routledge.Google Scholar logo with link to Google Scholar
Li, M., & DeKeyser, R. (2019). Distribution of practice effects in the acquisition and retention of L2 Mandarin tonal word production. The Modern Language Journal, 1031, 607–628. Google Scholar logo with link to Google Scholar
Li, S. (2012). The effects of input-based practice on pragmatic development of requests in L2 Chinese. Language Learning, 62(2), 403–438. Google Scholar logo with link to Google Scholar
Lindstromberg, S. (2016). Inferential statistics in language teaching research: A review and ways forward. Language Teaching Research, 20(6), 741–768. Google Scholar logo with link to Google Scholar
Liu, Q., & Brown, D. (2015). Methodological synthesis of research on the effectiveness of corrective feedback in L2 writing. Journal of Second Language Writing, 301, 66–81. Google Scholar logo with link to Google Scholar
Loewen, S., & Hui, B. (2021). Small samples in instructed second language acquisition research. The Modern Language Journal, 105(1), 187–193. [URL].
Loewen, S., & Plonsky, L. (2016). An A–Z of applied linguistics research methods. Palgrave. Google Scholar logo with link to Google Scholar
Loewen, S., & Sato, M. (Eds.). (2017). The Routledge handbook of instructed second language acquisition. Routledge. Google Scholar logo with link to Google Scholar
Norouzian, R. (2020). Sample size planning in quantitative L2 research: A pragmatic approach. Studies in Second Language Acquisition, 42(4), 849–870. Google Scholar logo with link to Google Scholar
Norouzian, R., & Plonsky, L. (2018). Eta- and partial eta-squared in L2 research: A cautionary review and guide to more appropriate language usage. Second Language Research, 34(2), 257–271. Google Scholar logo with link to Google Scholar
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. Google Scholar logo with link to Google Scholar
Phakiti, A. (2015). Experimental research methods in language learning. Bloomsbury.Google Scholar logo with link to Google Scholar
Plonsky, L. (2013). Study quality in SLA: An assessment of designs, analyses, and reporting practices in quantitative L2 research. Studies in Second Language Acquisition, 35(4), 655–687. Google Scholar logo with link to Google Scholar
(2014). Study quality in quantitative L2 research (1990–2010): A methodological synthesis and call for reform. Modern Language Journal, 98(1), 450–470. Google Scholar logo with link to Google Scholar
Plonsky, L., & Kim, Y. (2016). Task-based learner production: A substantive and methodological review. Annual Review of Applied Linguistics, 361, 73–97. Google Scholar logo with link to Google Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is ‘big’? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. Google Scholar logo with link to Google Scholar
Plonsky, L., & Zhuang, J. (2019). A meta-analysis of L2 pragmatics instruction. In N. Taguchi (Ed.), The Routledge handbook of second language acquisition and pragmatics (pp. 287–307). Routledge. Google Scholar logo with link to Google Scholar
Ren, W., Li, S., & Lu, X. (2023). A meta-analysis of the effectiveness of second language pragmatics instruction. Applied Linguistics, 44(6), 1010–1029. Google Scholar logo with link to Google Scholar
Rose, K. R. (2005). On the effects of instruction in second language pragmatics. System, 33(3), 385–399. Google Scholar logo with link to Google Scholar
Rubin, M. (2021). When to adjust alpha during multiple testing: A consideration of disjunction, conjunction, and individual testing. Synthese, 1991, 10969–11000. Google Scholar logo with link to Google Scholar
Schmidt, R. (1993). Consciousness, learning, and interlanguage pragmatics. In G. Kasper & S. Blum-Kulka (Eds.), Interlanguage pragmatics (pp. 21–42). Oxford University Press. Google Scholar logo with link to Google Scholar
Shively, R., Acevedo, J., Cano, R., & Etxeberria-Ortego, I. (2022). Teaching humorous irony to L2 and heritage speakers of Spanish. Language Teaching Research, 26(2), 279–302. Google Scholar logo with link to Google Scholar
Sok, S., Kang, E., & Han, Z. (2019). Thirty-five years of ISLA on form-focused instruction: A methodological synthesis. Language Teaching Research, 23(4), 403–427. Google Scholar logo with link to Google Scholar
Taguchi, N. (2008). Cognition, language contact, and development of pragmatic comprehension in a study-abroad context. Language Learning, 581, 33–71. Google Scholar logo with link to Google Scholar
(2015). Instructed pragmatics at a glance: Where instructional studies were, are, and should be going. Language Teaching, 48(1), 1–50. Google Scholar logo with link to Google Scholar
Taguchi, N., & Dixon, D. (2023). Game-informed approach to teaching request-making in English: A comparison of feedback mechanics. CALICO Journal, 40(2), 153–177. Google Scholar logo with link to Google Scholar
Taguchi, N., Dixon, D., Qin, Y., & Chen, Y. (2025). Pedagogic tasks in digital games: Effects of feedback conditions and individual characteristics on learning request-making. Language Teaching Research, 29(5), 2177–2203. Google Scholar logo with link to Google Scholar
Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford University PressGoogle Scholar logo with link to Google Scholar
Taguchi, N., & Youn, S. J. (2022). Research methods in instructed pragmatics: Assessment of learning outcomes. In L. Gurzynski-Weiss & Y. Kim (Eds.), Instructed second language acquisition research methods (pp. 149–180). John Benjamins. Google Scholar logo with link to Google Scholar
Takimoto, M. (2012). Metapragmatic discussion in interlanguage pragmatics. Journal of Pragmatics, 441, 1240–1253. Google Scholar logo with link to Google Scholar
Tang, X. (2019). The effects of task modality on L2 Chinese learners’ pragmatic development: Computer-mediated written chat vs. face-to-face oral chat. System, 801, 48–59. Google Scholar logo with link to Google Scholar
Vitta, J., Nicklin, C., & McLean, S. (2022). Effect size–driven sample-size planning, randomization, and multisite use in L2 instructed vocabulary acquisition experimental samples. Studies in Second Language Acquisition, 44(5), 1424–1448. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue