Cover not available

In:Understanding L2 Proficiency: Theoretical and meta-analytic investigations
Edited by Eun Hee Jeon and Yo In'nami
[Bilingual Processing and Acquisition 13] 2022
► pp. 109–158

Get fulltext
PDF logo with download linkDownload Chapter PDF

Chapter 5
L2 writing and its internal correlates

A meta-analysis

ORCiD logo with link | Gifu City Women’s College
ORCiD logo with link | Teikyo University of Science
Open Access logo
Available under the Creative Commons Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Published online: 4 August 2022
DOI logo with link
https://doi.org/10.1075/bpa.13.05koj
Abstract
This study examined the overall average correlation between second/foreign language (L2) writing performance and frequently investigated features of writing (i.e., writing-internal correlates). The correlates of L2 writing performance included objective measures of text features (syntactic complexity, lexical complexity, accuracy, fluency, and cohesion) as well as subjective measures (content, argument, organization, cohesion, coherence, grammar, vocabulary, language use, mechanics, and accuracy). A series of moderator analyses were also carried out for each type of objective measure to examine the effects of participants’ age, L2 proficiency, learning context, first language (L1) and L2 distance, task type, writing scoring method, and some characteristics of objective measures. By doing so, the present study aimed to identify key correlates of L2 writing performance and compare their relative importance. To this end, a total of 103 retrieved studies contributed 1,045 effect sizes based on 15,537 independent participants. The results for objective measures demonstrated that fluency had the strongest mean correlation with L2 writing performance (r = .570), followed by accuracy (r = .477), lexical complexity (r = .295), syntactic complexity (r = .271), and cohesion (r = .198). All subjective measure components had strong mean correlations with L2 writing performance (r = .668 to .927), but content and language use features had the strongest effects and cohesion and coherence features showed the least effects. Participants’ age, learning context, L1–L2 distance, writing scoring method, and some measurement characteristics were found to be significant moderators for certain components. The findings of this study have implications for L2 instruction suggesting that fluency and language accuracy of L2 writing should be promoted across the various developmental stages of L2 learners in various conditions, whereas lexical and syntactic competence should be more focused upon when instructing child/adolescent or low/intermediate L2 writers.
Article outline
  • 1.Introduction
  • 2.Background to the meta-analysis
    • 2.1Review of objective measures
      • 2.1.1Syntactic complexity
      • 2.1.2Lexical complexity
      • 2.1.3Accuracy
      • 2.1.4Fluency
      • 2.1.5Cohesion
    • 2.2Review of subjective measures
    • 2.3Review of moderator variables
      • 2.3.1Age
      • 2.3.2L2 proficiency
      • 2.3.3Learning context
      • 2.3.4L1–L2 distance
      • 2.3.5Task type
      • 2.3.6Writing scoring method
      • 2.3.7Measurement types of complexity and fluency
  • 3.Research questions for the meta-analysis
  • 4.Method
    • 4.1Literature search and inclusion criteria
    • 4.2Acceptable measures of L2 writing performance and internal correlates
    • 4.3Coding the primary studies
    • 4.4Research synthesis
  • 5.Results
  • 6.Discussion
    • 6.1Objective measures
      • 6.1.1Syntactic complexity
      • 6.1.2Lexical complexity
      • 6.1.3Accuracy
      • 6.1.4Fluency
      • 6.1.5Cohesion
    • 6.2Subjective measures
  • 7.Conclusion
  • References
  • Appendix
References (89)
References
Bacha, N. (2001). Writing evaluation: What can analytic versus holistic essay scoring tell us? System, 29 (3), 371–383. Google Scholar logo with link to Google Scholar
Barkaoui, K. (2010a). Do ESL essay raters’ evaluation criteria change with experience? A mixed-methods, cross-sectional study. TESOL Quarterly, 44 (1), 31–57. Google Scholar logo with link to Google Scholar
(2010b). Explaining ESL essay holistic scores: A multilevel modeling approach. Language Testing, 27 (4), 515–535. Google Scholar logo with link to Google Scholar
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45 (1), 5–35. Google Scholar logo with link to Google Scholar
Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37 (5), 639–668. Google Scholar logo with link to Google Scholar
Bouwer, R., Béguin, A., Sanders, T., & Van den Bergh, H. (2015). Effect of genre on the generalizability of writing scores. Language Testing, 32 (1), 83–100. Google Scholar logo with link to Google Scholar
Brodkey, D., & Young, R. (1981). Composition correctness scores. TESOL Quarterly, 15 (2), 159–167. Google Scholar logo with link to Google Scholar
Bulté, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26 , 42–65. Google Scholar logo with link to Google Scholar
Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (Vol. 2). Sage.Google Scholar logo with link to Google Scholar
Crossley, S. A., Kyle, K., Allen, L. K., Guo, L., & McNamara, D. S. (2014). Linguistic microfeatures to predict L2 writing proficiency: A case study in automated writing evaluation. The Journal of Writing Assessment, 7 (1), 1–34.Google Scholar logo with link to Google Scholar
Crossley, S. A., Kyle, K., & McNamara, D. S. (2016a). The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing, 32 , 1–16. Google Scholar logo with link to Google Scholar
(2016b). The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods, 48 (4), 1227–1237. Google Scholar logo with link to Google Scholar
Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35 (2), 115–135. Google Scholar logo with link to Google Scholar
Crossley, S. A., Salsbury, T., & McNamara, D. S. (2012). Predicting the proficiency level of language learners using lexical indices. Language Testing, 29 (2), 243–263. Google Scholar logo with link to Google Scholar
(2015). Assessing lexical proficiency using analytic ratings: A case for collocation accuracy. Applied Linguistics, 36 (5), 570–590. Google Scholar logo with link to Google Scholar
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H. (2012). Facets of speaking proficiency. Studies in Second Language Acquisition, 34 (1), 5–34. Google Scholar logo with link to Google Scholar
Engber, C. A. (1992). A study of lexis and the relationship to quality in written texts of second language learners of English (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (303986307)
(1995). The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing, 4 (2), 139–155. Google Scholar logo with link to Google Scholar
Evola, J., Mamer, E., & Lentz, B. (1980). Discrete-point versus global scoring for cohesive devices. In J. W. Oiler & K. Perkins (Eds.), Research and language testing (pp. 177–181). Newbury House.Google Scholar logo with link to Google Scholar
Flahive, D. E., & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. In J. W. Oiler & K. Perkins (Eds.), Research and language testing (pp. 171–176). Newbury House.Google Scholar logo with link to Google Scholar
Grabe, W., & Kaplan, R. B. (1996). Theory and practice of writing: An applied linguistic perspective. Longman.Google Scholar logo with link to Google Scholar
Guo, L., Crossley, S. A., & McNamara, D. S. (2013). Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, 18 (3), 218–238. Google Scholar logo with link to Google Scholar
Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Longman.Google Scholar logo with link to Google Scholar
Hinkel, E. (2002). Second language writers’ text: Linguistic and rhetorical features. Routledge. Google Scholar logo with link to Google Scholar
Housen, A., Kuiken, F., & Vedder, I. (2012). Complexity, accuracy and fluency: Definitions, measurement and research. In A. Housen, F. Kuiken, & F. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 1–19). John Benjamins. Google Scholar logo with link to Google Scholar
Hunt, K. W. (1965). Grammatical structures written at three grade levels (Vol. 3). National Council of Teachers of English.Google Scholar logo with link to Google Scholar
Jacobs, H. L., Zinkgraf, S. A., Wormuth, D. R., Hartfiel, V. F., & Hughey, J. B. (1981). Testing ESL composition: A practical approach. Newbury House.Google Scholar logo with link to Google Scholar
Jak, S. (2015). Meta-analytic structural equation modelling. Springer. Google Scholar logo with link to Google Scholar
Jarvis, S. (2002). Short texts, best-fitting curves and new measures of lexical diversity. Language Testing, 19 (1), 57–84. Google Scholar logo with link to Google Scholar
(2013). Capturing the diversity in lexical diversity. Language Learning, 63 (S1), 87–106. Google Scholar logo with link to Google Scholar
Jeon, E.-H., & Yamashita, Y. (2014). L2 reading comprehension and its correlates: A meta-analysis. Language Learning, 64 (1), 160–212. Google Scholar logo with link to Google Scholar
Kaczmarek, C. M. (1980). Scoring and rating essay tasks. In J. W. Oiler & K. Perkins (Eds.), Research and language testing (pp. 151–159). Newbury House.Google Scholar logo with link to Google Scholar
Kim, M., & Crossley, S. A. (2018). Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing, 37 , 39–56. Google Scholar logo with link to Google Scholar
Kim, M., Crossley, S. A. & Kyle, K. (2018). Lexical sophistication as a multidimensional phenomenon: Relations to second language lexical proficiency, development, and writing quality. The Modern Language Journal, 102 (1), 120–141. Google Scholar logo with link to Google Scholar
Kim, Y. A., & Na, Y. H. (2009). Cohesive devices and quality of argumentative writing produced by Korean EFL learners. Studies in English Education, 14 (2), 1–29.Google Scholar logo with link to Google Scholar
Koda, K. (2005). Insights into second language reading: A cross-linguistic approach. Cambridge University Press. Google Scholar logo with link to Google Scholar
Koda, K., & Yamashita, J. (2019). Reading to learn in a foreign language: An integrated approach to foreign language instruction and assessment. Routledge.Google Scholar logo with link to Google Scholar
Kojima, M. (2020). A study synthesis on the relationship between second language writing performance and text features: Focusing on text-based measures and study features. Learner Corpus Studies in Asia and the World, 5 , 1–24. Google Scholar logo with link to Google Scholar
Kojima, M., & Kaneta, T. (2020). Raitingu hyoka to gengoteki shihyo no kankei: Meta bunseki ni yoru kenkyu seika no togo [The relationship between writing performance and linguistic indices: A meta-analysis]. In Y. Ishii & Y. Kondo (Eds.), Eigo Kyoiku ni okeru jido saiten: Genjo to kadai [Automated scoring in English language education: Its current situation and issues] (pp. 33–72). Hituzi Shobo.Google Scholar logo with link to Google Scholar
Kroll, B. (1982). Levels of error in ESL composition (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (303230742)
(1990). What does time buy? ESL student performance on home versus class compositions. In B. Kroll (Ed.), Second language writing: Research insights for the classroom (pp. 140–154). Cambridge University Press. Google Scholar logo with link to Google Scholar
Kyle, K., & Crossley, S. (2016). The relationship between lexical sophistication and independent and source-based writing. Journal of Second Language Writing, 34 , 12–24. Google Scholar logo with link to Google Scholar
(2017). Assessing syntactic sophistication in L2 writing: A usage-based approach. Language Testing, 34 (4), 513–535. Google Scholar logo with link to Google Scholar
Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal, 102 (2), 333–349. Google Scholar logo with link to Google Scholar
Larsen-Freeman, D. (2009). Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30 (4), 579–589. Google Scholar logo with link to Google Scholar
Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16 (3), 307–322. Google Scholar logo with link to Google Scholar
Lee, Y.-W., Gentile, C., & Kantor, R. (2008). Analytical scoring of TOEFL CBT essays: Scores by humans and e-rater, TOEFL Research Report, 81 , ETS RR-08-01. Educational Testing Service. Google Scholar logo with link to Google Scholar
(2010). Toward automated multi-trait scoring of essays: Investigating links among holistic, analytic, and text feature scores. Applied Linguistics, 31 (3), 391–417. Google Scholar logo with link to Google Scholar
Leki, I., Cumming, A., & Silva, T. (2008). A synthesis of research on second language writing in English. Routledge.Google Scholar logo with link to Google Scholar
Linnarud, M. (1986). Lexis in composition: A performance analysis of Swedish learners’ written English. CWK Gleerup.Google Scholar logo with link to Google Scholar
Liu, M., & Braine, G. (2005). Cohesive features in argumentative writing produced by Chinese undergraduates. System, 33 (4), 623–636. Google Scholar logo with link to Google Scholar
Lu, X., & Ai, H. (2015). Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing, 29 , 16–27. Google Scholar logo with link to Google Scholar
Luk, Z. P. S., & Shirai, Y. (2009). Is the acquisition order of grammatical morphemes impervious to L1 knowledge? Evidence from the acquisition of plural -s, articles, and possessive’s. Language Learning, 59 (4), 721–754. Google Scholar logo with link to Google Scholar
Malvern, D., & Richards, B. (2002). Investigating accommodation in language proficiency interviews using a new measure of lexical diversity. Language Testing, 19 (1), 85–104. Google Scholar logo with link to Google Scholar
Martínez, A. C. L. (2018). Analysis of syntactic complexity in secondary education EFL writers at different proficiency levels. Assessing Writing, 35 , 1–11. Google Scholar logo with link to Google Scholar
McCutchen, D., & Perfetti, C. A. (1982). Coherence and connectedness in the development of discourse production. Text-Interdisciplinary Journal for the Study of Discourse, 2 (1–3), 113–140. Google Scholar logo with link to Google Scholar
Meara, P., & Bell, H. (2001). P_Lex: A simple and effective way of describing the lexical characteristics of short L2 texts. Prospect, 16 (3), 5–19.Google Scholar logo with link to Google Scholar
Murakami, A., & Alexopoulou, T. (2016). L1 influence on the acquisition order of English grammatical morphemes: A learner corpus study. Studies in Second Language Acquisition, 38 (3), 365–401. Google Scholar logo with link to Google Scholar
Norris, J. M., & Ortega, L. (2000). Effectiveness of L2 instruction: A research synthesis and quantitative meta-analysis. Language Learning, 50 (3), 417–528. Google Scholar logo with link to Google Scholar
(2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30 (4), 555–578. Google Scholar logo with link to Google Scholar
Oh, E., Lee, C. M., & Moon, Y. I. (2015). The contributions of planning, L2 linguistic knowledge and individual differences to L2 writing. The Journal of Asia TEFL, 12 (2), 45–85.Google Scholar logo with link to Google Scholar
Oh, S. (2006). Investigating the relationship between fluency measures and second language writing placement test decisions (Unpublished master’s thesis). University of Hawai’i. Retrieved from [URL]
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24 (4), 492–518. Google Scholar logo with link to Google Scholar
(2015). Syntactic complexity in L2 writing: Progress and expansion. Journal of Second Language Writing, 29 , 82–94. Google Scholar logo with link to Google Scholar
Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30 (4), 590–601. Google Scholar logo with link to Google Scholar
Park, S. K. (2013). Lexical analysis of Korean university students’ narrative and argumentative essays. English Teaching (영어교육), 68 (3), 131–157.Google Scholar logo with link to Google Scholar
Perkins, K. (1980). Using objective methods of attained writing proficiency to discriminate among holistic evaluations. TESOL Quarterly, 14 (1), 61–69. Google Scholar logo with link to Google Scholar
(1983). On the use of composition scoring techniques, objective measures, and objective tests to evaluate ESL writing ability. TESOL Quarterly, 17 (4), 651–671. Google Scholar logo with link to Google Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64 (4), 878–912. Google Scholar logo with link to Google Scholar
Read, J. (2000). Assessing vocabulary. Cambridge University Press. Google Scholar logo with link to Google Scholar
Roca de Larios, J., Manchón, R., Murphy, L., & Marín, J. (2008). The foreign language writer’s strategic behaviour in the allocation of time to writing processes. Journal of Second Language Writing, 17 (1), 30–47. Google Scholar logo with link to Google Scholar
Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86 , 638–641. Google Scholar logo with link to Google Scholar
Sasaki, M. (2009). Changes in English as a foreign language students’ writing over 3.5 years: A sociocognitive account. In R. M. Manchón (Ed.), Writing in foreign language contexts: Learning, teaching, and research (pp. 49–76). Multilingual Matters. Google Scholar logo with link to Google Scholar
Schoonen, R., Van Gelderen, A., De Glopper, K., Hulstijn, J., Simis, A., Snellings, P., & Stevenson, M. (2003). First language and second language writing: The role of linguistic knowledge, speed of processing, and metacognitive knowledge. Language Learning, 53 (1), 165–202. Google Scholar logo with link to Google Scholar
Schoonen, R., Van Gelderen, A., Stoel, R. D., Hulstijn, J., & De Glopper, K. (2011). Modeling the development of L1 and EFL writing proficiency of secondary school students. Language Learning, 61 (1), 31–79. Google Scholar logo with link to Google Scholar
Skehan, P. (1998). A cognitive approach to language learning. Oxford University Press.Google Scholar logo with link to Google Scholar
(2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30 (4), 510–532. Google Scholar logo with link to Google Scholar
Stoller, F. L. (2004). Content-based instruction: Perspectives on curriculum planning. Annual Review of Applied Linguistics, 24 , 261–283. Google Scholar logo with link to Google Scholar
Thewissen, J. (2013). Capturing L2 accuracy developmental patterns: Insights from an error-tagged EFL learner corpus. The Modern Language Journal, 97 (S1), 77–101. Google Scholar logo with link to Google Scholar
Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21 (3), 239–263. Google Scholar logo with link to Google Scholar
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36 (3), 1–48. Google Scholar logo with link to Google Scholar
Weigle, S. C. (2002). Assessing Writing. Cambridge University Press. Google Scholar logo with link to Google Scholar
White, E. M. (1984). Holisticism. College Composition and Communication, 35 (4), 400–409. Google Scholar logo with link to Google Scholar
Wolfe-Quintero, Y., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy, & complexity. University of Hawai’i Press.Google Scholar logo with link to Google Scholar
Yang, W., Lu, X., & Weigle, S. C. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28 , 53–67. Google Scholar logo with link to Google Scholar
Yde, P., & Spoelders, M. (1985). Text cohesion: An exploratory study with beginning writers. Applied Psycholinguistics, 6 (4), 407–415. Google Scholar logo with link to Google Scholar
Yu, G. (2010). Lexical diversity in writing and speaking task performances. Applied Linguistics, 31 (2), 236–259. Google Scholar logo with link to Google Scholar
Yun, Y. (2005). Factors explaining EFL learners’ performance in a timed essay writing test: A structural equation modeling approach (Doctoral dissertation). Retrieved from ProQuest Dissertations & Theses Global. (3199191)
Cited by (4)

Cited by four other publications

Naismith, Ben & Alan Juffs
2025. The impact of collocational proficiency features on expert ratings of L2 English learners’ writing. Studies in Second Language Acquisition 47:1 ► pp. 336 ff. DOI logo
Yamashita, Taichi
2025. Examining English language learners’ longitudinal development of syntactic complexity across five CEFR levels with a robust measurement design: A mixed-methods approach. Applied Linguistics DOI logo
Mavrou, Irini & Javier Chao
2023. What Does Linguistic Distance Predict When It Comes to L2 Writing of Adult Immigrant Learners of Spanish?. Written Communication 40:3 ► pp. 943 ff. DOI logo
Monteiro, Kátia, Scott Crossley, Robert-Mihai Botarleanu & Mihai Dascălu
2023. L2 and L1 semantic context indices as automated measures of lexical sophistication. Language Testing 40:3 ► pp. 576 ff. DOI logo

This list is based on CrossRef data as of 3 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue