Cover not available

Article published In: Chinese as a Second Language (漢語教學研究—美國中文教師學會學報)
Vol. 56:2 (2021) ► pp.118–145

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

The effects of linguistic measures in the analysis of L2 Chinese descriptive writing

 | Arizona State University
Published online: 4 February 2022
DOI logo with link
https://doi.org/10.1075/csl.21012.lia
Abstract
The current study investigates whether the CAF (complexity, accuracy, and fluency) analysis framework, frequently used in L2 studies of English and other languages, may effectively analyze L2 essays in Chinese, a language typologically different from English. CAF measures were examined for both their discriminative and predictive effects for L2 Chinese writing quality. The results revealed that most of the measures captured linguistic differences between lower-score and higher-score essays. Similar to findings in other L2s, more advanced L2 Chinese learners achieved syntactic complexification increasingly through phrasal features in their essays, in contrast to lower-level learners. The predictive effects of the measures for writing quality differed between lower-score and higher-score essays. T-unit length, a syntactic measure successfully analyzing Indo-European languages, did not serve as an effective predictor for L2 Chinese textual quality. The current study provides implications for both L2 writing research and L2 writing curriculum.
摘要
本文考察了由复杂性 (complexity)、准确性 (accuracy)、流利性 (fluency) 所构成的CAF分析框架对于汉语二语写作的分析效果。CAF框架在英语及其他印欧语言的二语研究中应用较广，而对于汉语这一与印欧语言类型距离较大的语言，其具体分析效果尚不清楚。本研究考察了CAF指标两方面的特征：对于低分、高分汉语二语作文不同语言表现的区分能力；对于汉语二语作文人工评分的预测效果。研究结果显示，所采用的一系列CAF分析指标能够有效甄别低分、高分作文中的不同语言特征。与其他二语研究的结论相似，相较于低阶汉语学习者，高级水平的汉语学习者能更多地通过短语手段来实现句法方面的复杂性。对于能有效预测人工评分的分析指标，结果显示低分、高分组作文的预测指标有所不同。而印欧语言二语研究中所证实有效的评分预测指标 – T单位，未能有效预测当前的汉语二语作文评分。本研究对于二语写作研究及写作教学均有一定启示。
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1L2 writing constructs and measures
    • 2.2L2 writing performance in CAF measures
      • 2.2.1L2 written fluency, accuracy, and lexical complexity
      • 2.2.2L2 written syntactic complexity
      • 2.2.3Relationships among CAF sub-dimensions
    • 2.3Predictive effects of CAF measures for human judgments of L2 writing quality
  • 3.The present study
    • 3.1Participants and dataset
    • 3.2Measures in the current study
    • 3.3Analysis
  • 4.Results
    • 4.1Research question 1: CAF analysis of lower-score and higher-score essays
    • 4.2Research question 2: Predictive power of the CAF measures for human ratings of writing quality
  • 5.Discussion
    • 5.1Lower-score and higher-score writing performance according to CAF measures
    • 5.2Predictive effects of the CAF measures for human ratings of writing quality
    • 5.3Discrepancy between discriminative indicators and predictors of ratings
  • 6.Conclusion
  • Note
  • References
References (61)
References
ACTFL (2012). ACTFL proficiency guidelines. Retrieved from [URL]
An, F. (2015). Butong shuiping CSL xuexizhe zuowen liuchangxing, jufa fuzadu he zhunquexing fenxi – Yixiang jiyu T-danwei celiangfa de yanjiu 不同水平CSL学习者作文流畅性、句法复杂度和准确性分析—一项基于T单位测量法的研究 [Analysis of fluency, grammatical complexity, and accuracy of CSL writing: A study based on T-unit analysis]. Language Teaching and Linguistics Studies 语言教学与研究, 3 1, 11–20.Google Scholar logo with link to Google Scholar
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45 1, 5–35. Google Scholar logo with link to Google Scholar
Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37 1, 639–668. Google Scholar logo with link to Google Scholar
Bulté, B., & Housen, A. (2012). Defining and operationalising L2 complexity. In A. Housen, F. Kuiken & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 21–46). Amsterdam, The Netherlands: John Benjamins. Google Scholar logo with link to Google Scholar
(2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26 1, 42–65. Google Scholar logo with link to Google Scholar
Casal, J. E., & Lee, J. (2019). Syntactic complexity and writing quality in assessed first-year L2 writing. Journal of Second Language Writing, 44 1, 51–62. Google Scholar logo with link to Google Scholar
Chu, C. (1998). A discourse grammar of Mandarin Chinese. New York, NY: P. Lang.Google Scholar logo with link to Google Scholar
Crossley, S., & McNamara, D. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35 1, 115–135. Google Scholar logo with link to Google Scholar
(2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26 1, 66–79. Google Scholar logo with link to Google Scholar
Engber, C. (1995). The relationship of lexical proficiency to the quality of ESL compositions. Journal of Second Language Writing, 4 1, 139–155. Google Scholar logo with link to Google Scholar
Ferris, D. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28 1, 414–420. Google Scholar logo with link to Google Scholar
Foster, P., Tonkyn, A., & Wigglesworth, G. (2000). Measuring spoken language: A unit for all reasons. Applied Linguistics, 21 1, 354–375. Google Scholar logo with link to Google Scholar
Garner, J., Crossley, S., & Kyle, K. (2019). N-gram measures and L2 writing proficiency. System, 80 1, 176–187. Google Scholar logo with link to Google Scholar
Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9 1, 123–145. Google Scholar logo with link to Google Scholar
Guiraud, P. (1960). Problémes et méthodes de la statistique linguistique [Problems and methods of statistical linguistics]. Dordrecht, The Netherlands: Reidel.Google Scholar logo with link to Google Scholar
Halliday, M., & Matthiessen, C. (1999). Construing experience through meaning: A language-based approach to cognition. London, UK: Cassell.Google Scholar logo with link to Google Scholar
Hartshorn, K., Evans, N., Merrill, P., Sudweeks, R., Strong-Krause, D., & Anderson, N. (2010). Effects of dynamic corrective feedback on ESL writing accuracy. TESOL Quarterly, 44 1, 84–109. Google Scholar logo with link to Google Scholar
Hu, R. (2021). Jiyu dapei de jufa fuzadu zhibiao jiqi yu hanyu eryu xiezuo zhiliang guanxi yanjiu 基于搭配的句法复杂度指标及其与汉语二语写作质量关系研究 [On the relationship between collocation-based syntactic complexity and Chinese second language writing]. Applied Linguistics 语言文字应用, 1 1, 132–144. Google Scholar logo with link to Google Scholar
Huang, B., & Liao, X. (2007). Xiandai Hanyu 现代汉语 [Modern Chinese Language]. Beijing, China: Higher Education Press.Google Scholar logo with link to Google Scholar
Hunt, K. (1966). Recent measures in syntactic development. Elementary English, 43 1, 732–739.Google Scholar logo with link to Google Scholar
Ishikawa, S. (1995). Objective measurement of low-proficiency EFL narrative writing. Journal of Second Language Writing, 4 1, 51–69. Google Scholar logo with link to Google Scholar
Jarvis, S. (2002). Short texts, best-fitting curves and new measures of lexical diversity. Language Testing, 19 1, 57–84. Google Scholar logo with link to Google Scholar
Jiang, W. (2013). Measurements of development in L2 written production: The case of L2 Chinese. Applied Linguistics, 34 1, 1–24. Google Scholar logo with link to Google Scholar
Johnson, M. (2017). Cognitive task complexity and L2 written syntactic complexity, accuracy, lexical complexity, and fluency: A research synthesis and meta-analysis. Journal of Second Language Writing, 37 1, 13–38. Google Scholar logo with link to Google Scholar
Johnson, M., Mercado, L., Acevedo, A. (2012). The effect of planning sub-processes on L2 writing fluency, grammatical complexity, and lexical complexity. Journal of Second Language Writing, 21 1, 264–282. Google Scholar logo with link to Google Scholar
Kormos, J. (2011). Task complexity and linguistic and discourse features of narrative writing performance. Journal of Second Language Writing, 20 1, 148–161. Google Scholar logo with link to Google Scholar
Kuiken, F., & Vedder, I. (2019). Syntactic complexity across proficiency and languages: L2 and L1 writing in Dutch, Italian and Spanish. International Journal of Applied Linguistics, 29 1, 192–210. Google Scholar logo with link to Google Scholar
Kyle, K., & Crossley, S. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. Modern Language Journal, 102 1, 333–349. Google Scholar logo with link to Google Scholar
Larsen-Freeman, D. (1978). An ESL index of development. TESOL Quarterly, 12 1, 439–448. Google Scholar logo with link to Google Scholar
(1983). Assessing global second language proficiency. In H. W. Seliger & M. H. Long (Eds.), Classroom oriented research in second language acquisition (pp. 287–304). Rowley, MA: Newbury House.Google Scholar logo with link to Google Scholar
(2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics, 27 1, 590–619. Google Scholar logo with link to Google Scholar
(2015). Saying what we mean: Making a case for ‘language acquisition’ to become ‘language development’. Language Teaching, 48 1, 491–505. Google Scholar logo with link to Google Scholar
Lesonen, S., Steinkrauss, R., Suni, M., & Verspoor, M. (2021). Dynamic usage-based principles in the development of L2 Finnish evaluative constructions. Applied Linguistics, 42 1, 442–472. Google Scholar logo with link to Google Scholar
Lowie, W., & Verspoor, M. (2015). Variability and variation in second language acquisition orders: A dynamic reevaluation. Language Learning, 65 1, 63–88. Google Scholar logo with link to Google Scholar
Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45 1, 36–62. Google Scholar logo with link to Google Scholar
Mazgutova, D., & Kormos, J. (2015). Syntactic and lexical development in an intensive English for academic purposes programme. Journal of Second Language Writing, 29 1, 3–15. Google Scholar logo with link to Google Scholar
Miller, K. (2000). Academic writers on-line: Investigating pausing in the production of text. Language Teaching Research, 4 1, 123–148. Google Scholar logo with link to Google Scholar
Mostafa, T., & Crossley, S. (2020). Verb argument construction complexity indices and L2 writing quality: Effects of writing tasks and prompts. Journal of Second Language Writing, 49 1, 1–13. Google Scholar logo with link to Google Scholar
Norris, J., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30 1, 555–578. Google Scholar logo with link to Google Scholar
Parkinson, J., & Musgrave, J. (2014). Development of noun phrase complexity in the writing of English for academic purposes students. Journal of English for Academic Purposes, 14 1, 48–59. Google Scholar logo with link to Google Scholar
Polio, C. (1997). Measures of linguistic accuracy in second language writing research. Language Learning, 47 1, 101–143. Google Scholar logo with link to Google Scholar
Polio, C., Fleck, C., & Leder, N. (1998). “If I only had more time:” ESL learners’ changes in linguistic accuracy on essay revisions. Journal of Second Language Writing, 7 1, 43–68. Google Scholar logo with link to Google Scholar
Polio, C., & Shea, M. (2014). An investigation into current measures of linguistic accuracy in second language writing research. Journal of Second Language Writing, 26 1, 10–27. Google Scholar logo with link to Google Scholar
Read, J. (2000). Assessing vocabulary. Oxford, UK: Oxford University Press. Google Scholar logo with link to Google Scholar
Reid, J. (1990). Responding to different topic types: A quantitative analysis from a contrastive rhetoric perspective. In B. Kroll (Ed.), Second language writing: Research insights for the classroom (pp. 191–210). Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Révész, A., Kourtali, N., & Mazgutova, D. (2017). Effects of task complexity on L2 writing behaviors and linguistic complexity. Language Learning, 67 1, 208–241. Google Scholar logo with link to Google Scholar
Skehan, P. (2009). Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied Linguistics, 30 1, 510–532. Google Scholar logo with link to Google Scholar
Spoelman, M., & Verspoor, M. (2010). Dynamic patterns in development of accuracy and complexity: A longitudinal case study in the acquisition of Finnish. Applied Linguistics, 31 1, 532–553. Google Scholar logo with link to Google Scholar
Ure, J. (1971). Lexical density: A computational technique and some findings. In M. Coultard (Ed.), Talking about text (pp. 27–48). Birmingham, UK: English Language Research, University of Birmingham.Google Scholar logo with link to Google Scholar
van Hout, R., & Vermeer, A. (2007). Comparing measures of lexical richness. In H. Daller, J. Milton & J. Treffers-Daller (Eds.), Modelling and assessing vocabulary knowledge (pp. 93–115). Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Verspoor, M., Lowie, W., & van Dijk, M. (2008). Variability in second language development from a dynamic systems perspective. Modern Language Journal, 92 1, 214–231. Google Scholar logo with link to Google Scholar
Verspoor, M., Schmid, M., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21 1, 239–263. Google Scholar logo with link to Google Scholar
Vyatkina, N. (2012). The development of second language writing complexity in groups and individuals: A longitudinal learner corpus study. Modern Language Journal, 96 1, 576–598. Google Scholar logo with link to Google Scholar
Wang, Y. (2017). Hanyu eryuzhe cihui fengfuxing yu xiezuo chengji de xianguanxing 汉语二语者词汇丰富性与写作成绩的相关性 [The correlation between lexical richness and writing score of CSL learner]. Applied Linguistics 语言文字应用, 2 1, 93–101. Cite to nonCR doi: Google Scholar logo with link to Google Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. Honolulu, HI: Second Language Teaching & Curriculum Center, University of Hawaii at Manoa.Google Scholar logo with link to Google Scholar
Wu, J. (2016). Yingyu muyuzhe hanyu xiezuo zhong de cihui fengfuxing fazhan yanjiu 英语母语者汉语写作中的词汇丰富性发展研究 [Research on lexical richness development in CSL writing by English native speakers]. Chinese Teaching in the World 世界汉语教学, 30 1, 129–142. Cite to nonCR doi: Google Scholar logo with link to Google Scholar
Yang, W., Lu, X., & Weigle, S. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28 1, 53–67. Google Scholar logo with link to Google Scholar
Yoon, H., & Polio, C. (2017). The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51 1, 275–301. Google Scholar logo with link to Google Scholar
Yu, Q. (2021). An organic syntactic complexity measure for the Chinese language: The TC-unit. Applied Linguistics, 42 1, 60–92. Google Scholar logo with link to Google Scholar
Zenker, F., & Kyle, K. (2021). Investigating minimum text lengths for lexical diversity indices. Assessing Writing, 47 1, 1–15. Google Scholar logo with link to Google Scholar
Cited by (2)

Cited by two other publications

Fonteyne, Margot, Maribel Montero Perez, Joke Daems & Lieve Macken
2025. Selecting Instruments for Investigating L2 Swedish Writing Data of L1 Dutch Learners Across Proficiencies. Nordand 20:1 ► pp. 1 ff. DOI logo
Chen, Heng
2023. A lexical network approach to second language development. Humanities and Social Sciences Communications 10:1 DOI logo

This list is based on CrossRef data as of 8 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue