Cover not available

Article published In: Journal of Second Language Studies
Vol. 1:2 (2018) ► pp.231–253

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Meta-analysis in second language acquisition research

A critical appraisal

 | Curtin University
Published online: 27 August 2018
DOI logo with link
https://doi.org/10.1075/jsls.00002.ell
Abstract
Meta-analysis has become increasingly popular in second language acquisition research (SLA) and has provided valuable summative information about a number of key areas. There are, however, dangers. This article examines a number of key issues that need to be considered in conducting a meta-analysis – inclusiveness, the heterogeneity of language learners, the definition of the independent and dependent variables, the need to consider alternative explanations of observed effects, the importance of examining the quality of the studies included in the analysis, and the apples and oranges problem. These issues are illustrated in a discussion of number of SLA meta-analyses (e.g. Norris, J., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. Google Scholar logo with link to Google Scholar; Plonsky, L. (2011). The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 611, 993–1038. Google Scholar logo with link to Google Scholar; Qureshi, M. (2016). A meta-analysis: Age and second language grammar acquisition. System, 601, 147–160. Google Scholar logo with link to Google Scholar; Spada, N., & Tomita, Y. (2010). Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 601, 268–308. Google Scholar logo with link to Google Scholar). The article concludes by suggesting a number of factors that need to be considered in deciding whether to conduct a meta-analysis and when carrying one out. I argue the need for systematic reviews but suggest that these can often best present their findings in narrative form rather than statistically. I also suggest that a preliminary narrative account of the findings of s systematic review can indicate whether a meta-analysis is appropriate.
Article outline
  • Introduction
    • Issue 1.Inclusiveness
    • Issue 2.Heterogeneity of language learners
    • Issue 3.Defining the independent and dependent variables
    • Issue 4.Explaining an observed effect
    • Issue 5.Quality of studies
    • Issue 6.Apples and oranges
    • Issue 7.Effects of grouping
  • Is there another way?
  • 3.Conclusion
  • Notes
  • References
References (49)
References
Berns, M. (1990). ‘Second’ and ‘foreign’ in second language acquisition? Foreign language learning: A sociolinguistic perspectives. In B. VanPatten & J. Lee (Eds), Second language acquisition – foreign language learning (pp. 3–11). Clevedon: Multilingual Matters.Google Scholar logo with link to Google Scholar
Bialystok, E., & Hakuta, K. (1999). Confounded age: Linguistic and cognitive factors in age differences for second language acquisition. In D. Birdsong (Ed.), Second language acquisition and the critical period hypothesis. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Birdsong, D. (2006). Age and second language acquisition and processing: A selective overview. In M. Gullberg & P. Indefrey (Eds.), The cognitive neuroscience of second language acquisition (pp. 9–49). Malden, MA: Blackwell.Google Scholar logo with link to Google Scholar
Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. (2006). Comprehensive meta-analysis (Version 2.2.027) [Computer software]. Englewood, NJ: Biostat.Google Scholar logo with link to Google Scholar
(2009). Introduction to meta-analysis. Hoboken, NJ: John Wiley and Sons. Google Scholar logo with link to Google Scholar
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Cooper, H. (2010). Research synthesis and meta-analysis. Los Angeles, CA: Sage.Google Scholar logo with link to Google Scholar
DeKeyser, R. (2003). Implicit and explicit learning. In C. Doughty & M. Long (Eds.), The handbook of second language acquisition (pp. 313–348). Malden, MA: Blackwell. Google Scholar logo with link to Google Scholar
Ellis, N. (2006). Meta-analysis, human cognition, and language learning. In J. Norris & L. Ortega (Eds.), Synthesizing research on language teaching and learning (pp. 301–322). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Ellis, R. (1991). Grammar teaching – practice or consciousness-raising. In R. Ellis (Ed.), Second language acquisition and second language pedagogy (pp. 232–241). Clevedon: Multilingual Matters.Google Scholar logo with link to Google Scholar
(1997). SLA research and language teaching. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
(2007). The differential effects of corrective feedback on two grammatical structures. In A. Mackey (Ed.), Conversational interaction and second language acquisition: A series of empirical studies (pp. 339–360). Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
(2012). Second language research and language pedagogy. Malden, MA: Wiley Blackwell.Google Scholar logo with link to Google Scholar
(2015). Understanding second language acquisition (2nd ed.). Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
(2015b). Introduction: Complementarity in research syntheses. Applied Linguistics, 36(3), 258–259. Google Scholar logo with link to Google Scholar
Eysenck, H. (1984). Meta-analysis: An abuse of research integration. Journal of Special Education, 151, 9–11.Google Scholar logo with link to Google Scholar
(1994). Systematic reviews: Meta-analysis and its problems. British Medication Journal, 3091, 789–792. Google Scholar logo with link to Google Scholar
Glass, G. (1976). Primary, secondary and meta-analysis of Research. Educational Researcher, 51, 3–8. Google Scholar logo with link to Google Scholar
Glass, S., & Smith, M. (1979). Meta-analysis on class size and achievement. Educational Evolution and Policy Analysis, 11, 2–16. Google Scholar logo with link to Google Scholar
Goo, J., Granena, G., Novella, M., & Yilmaz, M. (2009). Implicit and explicit instruction in L2 learning: Norris and Ortega (2000) revisited. Paper presented at the Second Language Research Forum. East Lansing, MI.
Jeon, E., & Kaya, T. (2006). Effects of L2 instruction on interlanguage pragmatic development: A meta-analysis. In J. Norris & L. Ortega (Eds.), Synthesizing research on language teaching and learning (pp. 165–212). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Johnson, J., & Newport, E. (1989). Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cognitive Psychology, 211, 60–99. Google Scholar logo with link to Google Scholar
Khan, S., Kunz, R., Kleijnen, M., & Antes, G. (2003). Five steps to conducing a systematic review. Journal of the Royal Society of Medicine, 961, 118–121. Google Scholar logo with link to Google Scholar
Li, S. (2010). The effectiveness of corrective feedback in SLA; A meta-analysis. Language Learning, 601, 309–365. Google Scholar logo with link to Google Scholar
Li, S., Shitani, N., & Ellis, R. (2013). Doing meta-analysis in SLA: Practices, choices and standards. Contemporary Foreign Language Studies, 3841, 1–17.Google Scholar logo with link to Google Scholar
Lipsey, M., & Wilson, D. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.Google Scholar logo with link to Google Scholar
Long, M. (1988). Maturational constraints on language development. University of Hawai’i Working Papers in ESL, 7(1), 1–53.Google Scholar logo with link to Google Scholar
(1991). Focus on form: A design feature in language teaching methodology. In K. de Bot, R. Ginsberg, & C. Kramsch (Eds.), Foreign language research in cross-cultural perspective (pp. 39–52). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Lyster, R., & Saito, K. (2010). Oral feedback in SLA: A meta-analysis. Studies in Second Language Acquisition, 321, 265–302. Google Scholar logo with link to Google Scholar
Mackey, A., & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research (pp. 407–449). In. A. Mackey (Eds.), Conversational interaction in second language acquisition: A collection of empirical studies (pp. 407–452). Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Norris, J., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 501, 417–528. Google Scholar logo with link to Google Scholar
(2006). The value and practice of research synthesis for language learning and teaching. In J. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching (pp. 1–50). Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Oswald, L., & Plonsky, L. (2010). Meta-analysis in second language research: Choices and challenges. Annual Review of Applied Linguistics, 301, 85–110. Google Scholar logo with link to Google Scholar
Plonsky, L. (2011). The effectiveness of second language strategy instruction: A meta-analysis. Language Learning, 611, 993–1038. Google Scholar logo with link to Google Scholar
Personal website downloaded on 31/7/17 from <[URL]>
Plonsky, L., & Oswald, F. (2012). How to do a meta-analysis. In A. Mackey & S. Gass (Eds.), Research methods in second language acquisition: A practical introduction (pp. 275–295). Malden, MA: Blackwell. Google Scholar logo with link to Google Scholar
Qureshi, M. (2016). A meta-analysis: Age and second language grammar acquisition. System, 601, 147–160. Google Scholar logo with link to Google Scholar
Ross, S. (1998). Self-assessment in second language testing: A meta-analysis and analysis of experiential factors. Language Testing, 151, 1–20. Google Scholar logo with link to Google Scholar
Russo, M. (2007). How to review a meta-analysis. Gastroenterology and Hepatology, 3(8), 637–642.Google Scholar logo with link to Google Scholar
Shintani, N., Li, S., & Ellis, R. (2013). Comprehension-based versus production-based grammar instruction: A meta-analysis of comparative studies. Language Learning, 63(2), 296–329. Google Scholar logo with link to Google Scholar
Shintani, N. (2015). The effectiveness of processing instruction and production-based instruction on L2 grammar acquisition: A meta-analysis. Applied Linguistics, 36(3), 306–325. Google Scholar logo with link to Google Scholar
Smith, M., & Glass, G. (1977). Meta-analysis of psychotherapy outcome studies. American Psychology, 321, 752–60. Google Scholar logo with link to Google Scholar
Smith, M., Glass, G., & Miller, T. (1980). The benefits of psychotherapy. Baltimore, MD: Johns Hopkins Press.Google Scholar logo with link to Google Scholar
Spada, N., & Lightbown, P. (2008). Form-focused instruction: Isolated or integrated? TESOL Quarterly, 421, 181–207. Google Scholar logo with link to Google Scholar
Spada, N., & Tomita, Y. (2010). Interactions between type of instruction and type of language feature: A meta-analysis. Language Learning, 601, 268–308. Google Scholar logo with link to Google Scholar
The International Research Foundation for English language Education. Meta-analyses in language research: A selected bibliography. Downloaded from <[URL]> (31 July, 2017).
Truscott, J. (2007). The effect of error correction on learners’ ability to write accurately. Journal of Second Language Writing, 161, 255–272. Google Scholar logo with link to Google Scholar
VanPatten, B. (1996). Input processing and grammar instruction in second language acquisition. Norwood, NJ: Ablex.Google Scholar logo with link to Google Scholar
Walker, E., Hernandez, A., & Katten, M. (2008). Meta-analysis: Its strength and limitations. Cleveland. Clinic Journal of Medicine, 751, 431–439.Google Scholar logo with link to Google Scholar
Cited by (5)

Cited by five other publications

Anani Sarab, Mohammad Reza & Mohammad Amini Farsani
2024. Second-order synthesis of meta-analytic studies in applied linguistics (1998–2021). Quality & Quantity 58:2 ► pp. 1517 ff. DOI logo
Yamashita, Taichi
2024. Effectiveness and inclusiveness of locally adapted human-delivered synchronous written corrective feedback for English referential articles. Computer Assisted Language Learning 37:5-6 ► pp. 1074 ff. DOI logo
Yapp, Deborah June, Rick de Graaff & Huub van den Bergh
2021. Improving second language reading comprehension through reading strategies. Journal of Second Language Studies 4:1 ► pp. 154 ff. DOI logo
Bardovi-Harlig, Kathleen & Llorenç Comajoan-Colomé
2020. THE ASPECT HYPOTHESIS AND THE ACQUISITION OF L2 PAST MORPHOLOGY IN THE LAST 20 YEARS. Studies in Second Language Acquisition 42:5 ► pp. 1137 ff. DOI logo
[no author supplied]
2025. Second Language Cognitive Task Complexity [Task-Based Language Teaching, 18], DOI logo

This list is based on CrossRef data as of 13 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue