Cover not available

In:Research Methods in Complex Dynamic Systems Theory Approaches to Second Language Development
Edited by Wander Lowie, Rosmawati and Vanessa De Wilde
[Research Methods in Applied Linguistics 14] 2025
► pp. 167–190

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

Chapter 9
Variability analysis using CAF measures

 | University of Groningen
 | University of Groningen
 | University of Groningen
Published online: 11 September 2025
DOI logo with link
https://doi.org/10.1075/rmal.14.09wu
Abstract
This chapter examines potential challenges for longitudinal L2 studies using quantitative CAF (complexity, accuracy, and fluency) measures. For a period of 13 months, we followed the CAF development of two L2 English students, collecting multiple samples at each monthly data point for several CAF measures with different levels of reliability. We then plotted multiple developmental trajectories by randomly selecting one single result from each time point. Comparing the resulting trajectories for high-, medium-, and low-reliability CAF measures, noticeably different developmental paths emerged for low-reliability measures for the same measure and participant. These distinct trajectories confirm that low-reliability measures are inadequate for capturing meaningful variability (Wu et al., 2023; Wu, 2025) when measured only once at each data point.
Article outline
  • Introduction
  • Methodological review of variability analyses using CAF measures
  • The reliability issue of CAF measures
  • Variability shown by CAF measures with different levels of reliability
  • The empirical study
    • Methods
      • Participants
      • Ethical considerations in longitudinal dense data collection
      • Data collection procedures
      • Data analysis and visualization
    • Results
      • Variability of high-, medium-, and low-reliability CAF measures
      • Developmental trajectories drawn from single-sample assessments
  • Affordances and limitations of using CAF measures in variability analysis
  • Implications for Future CDST-Inspired research
  • Conclusion
  • References
References (55)
References
Boersma, P., & Weenink, D. (2023). Praat: Doing phonetics by computer (Version 6.3.08). [URL]
Brennan, R. L. (2001). Generalizability Theory. Springer. Google Scholar logo with link to Google Scholar
Cambridge ESOL. (2002). Cambridge IELTS 3 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2005). Cambridge IELTS 4 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2006). Cambridge IELTS 5 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2007). Cambridge IELTS 6 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2009). Cambridge IELTS 7 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2011). Cambridge IELTS 8 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
Cambridge Press & Assessment. (2013). Cambridge English IELTS 9 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2015). Cambridge English IELTS 10 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2016). Cambridge IELTS 11 academic with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2017). Cambridge IELTS 12 Academic with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2018). Cambridge IELTS 13 Academic with Answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2019). Cambridge IELTS 14 Academic with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2020). Cambridge IELTS 15 Academic with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2021). Cambridge IELTS academic 16 with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
. (2022). Cambridge IELTS. 17 Academic with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
Chan, H., Verspoor, M., & Vahtrick, L. (2015). Dynamic development in speaking versus writing in identical twins. Language Learning, 65(2), 298–325. Google Scholar logo with link to Google Scholar
Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34(2), 213. Google Scholar logo with link to Google Scholar
Cullen, P., French, A., & Jakeman, V. (2014). The Official Cambridge Guide to IELTS. Student’s book with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
De Jong, N. H., Pacilly, J., & Heeren, W. (2021). PRAAT scripts to measure speed fluency and breakdown fluency in speech automatically. Assessment in Education: Principles, Policy & Practice, 28(4), 456–476. Google Scholar logo with link to Google Scholar
Fogal, G. G. (2020). Investigating variability in L2 development: Extending a complexity theory perspective on L2 writing studies and authorial voice. Applied Linguistics, 41(4), 575–600. Google Scholar logo with link to Google Scholar
Fogal, G. G., & Verspoor, M. H. (Eds.). (2020). Complex Dynamic Systems Theory and L2 writing development. John Benjamins. Google Scholar logo with link to Google Scholar
Guillemin, M., & Gillam, L. (2004). Ethics, reflexivity, and “ethically important moments” in research. Qualitative Inquiry: QI, 10(2), 261–280. Google Scholar logo with link to Google Scholar
Guiraud, P. (1954). Les caractères statistiques du vocabulaire: Essai de méthodologie. Presses universitaires de France.Google Scholar logo with link to Google Scholar
Hashemi, L., & Thomas, B. (2011). Cambridge IELTS trainer: Six practice tests with answers. Cambridge University Press.Google Scholar logo with link to Google Scholar
Huang, T., Steinkrauss, R., & Verspoor, M. (2021). Variability as predictor in L2 writing proficiency. Journal of Second Language Writing, 52(4), 100787. Google Scholar logo with link to Google Scholar
IELTS Speaking band scores. (n.d.). IDP IELTS. Retrieved on 29 August, 2023 from [URL]
Larsen-Freeman, D. (2006). The emergence of complexity, fluency, and accuracy in the oral and written production of five Chinese learners of English. Applied Linguistics, 27(4), 590–619.
Lowie, W. (2014). Variability in L2 phonology: A plea for a dynamic, process-based methodology. Dag van de Fonetiek 2014. [URL]
Lowie, W. M., & Verspoor, M. H. (2019). Individual differences and the ergodicity problem. Language Learning, 69(1), 184–206. Google Scholar logo with link to Google Scholar
Lowie, W., van Dijk, M., Chan, H., & Verspoor, M. (2017). Finding the key to successful L2 learning in groups and individuals. Studies in Second Language Learning and Teaching, 7(1), 127–148. Google Scholar logo with link to Google Scholar
MacWhinney, B. (2000). The CHILDES project: tools for analyzing talk. NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Meara, P., & Bell, H. (2001). P_Lex: A simple and effective way of describing the lexical characteristics of short L2 texts. Prospect, 13(3), 5–19.Google Scholar logo with link to Google Scholar
Meara, P., & Miralpeix, I. (2016). 3. P_Lex v3.00. In P. Meara & I. Miralpeix (Eds.), Tools for researching vocabulary (pp. 44–65). Multilingual Matters. Google Scholar logo with link to Google Scholar
Michel, M. (2017). Complexity, accuracy, and fluency in L2 production. In S. Loewen & M. Sato (Eds.), The Routledge handbook of instructed second language acquisition (pp. 50–68). Routledge. Google Scholar logo with link to Google Scholar
Paquot, M. (2019). The phraseological dimension in interlanguage complexity research. Second Language Research, 35(1), 121–145. Google Scholar logo with link to Google Scholar
Pavlovia. (n.d.). Retrieved on 12 April, 2024 from [URL]
Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., Kastman, E., & Lindeløv, J. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51(1), 195–203. Google Scholar logo with link to Google Scholar
Penris, W., & Verspoor, M. (2017). Academic writing development: A complex, dynamic Process. In S. Pfenninger & J. Navracsics (Eds.), Future research directions for Applied Linguistics (pp. 215–242). Multilingual Matters. Google Scholar logo with link to Google Scholar
Plat, R., Lowie, W., & de Bot, K. (2017). Word naming in the L1 and L2: A dynamic perspective on automatization and the degree of semantic involvement in naming. Frontiers in Psychology, 8, 2256. Google Scholar logo with link to Google Scholar
Plotly Technologies. (2015). Collaborative data science. Montréal, QC. [URL]
RStudio Team. (2015). RStudio: Integrated development environment for R. RStudio. [URL]
Spoelman, M., & Verspoor, M. (2010). Dynamic patterns in development of accuracy and complexity: A longitudinal case study in the acquisition of Finnish. Applied Linguistics, 31(4), 532–553. Google Scholar logo with link to Google Scholar
Van Dijk, M., Lowie, W., Smit, N., Verspoor, M., & van Geert, P. (2024). Complex Dynamic Systems Theory as a foundation for process-oriented research on second language development. Second Language Research. Google Scholar logo with link to Google Scholar
Van Dijk, M., Verspoor, M., & Lowie, W. (2011). Variability and DST. In M. H. Verspoor, K. de Bot, & W. Lowie (Eds.), A dynamic approach to second language development: Methods and techniques (pp. 55–84). John Benjamins. Google Scholar logo with link to Google Scholar
Van Geert, P., & van Dijk, M. (2002). Focus on variability: New tools to study intra-individual variability in developmental data. Infant Behavior & Development, 25(4), 340–374. Google Scholar logo with link to Google Scholar
Verspoor, M., & Lowie, W. (2020). Complex Dynamic Systems Theory (CDST). In N. Tracy-Ventura & M. Paquot (Eds.), The Routledge handbook of second language acquisition and corpora (pp. 189–200). Routledge. Google Scholar logo with link to Google Scholar
Verspoor, M., Lowie, W., & de Bot, K. (2021). Variability as normal as apple pie. Linguistics Vanguard, 7(s2). Google Scholar logo with link to Google Scholar
Verspoor, M., Lowie, W., & Van Dijk, M. (2008). Variability in second language development from a dynamic systems perspective. Modern Language Journal, 92(2), 214–231. Google Scholar logo with link to Google Scholar
Waninge, F. (2014). 14. Motivation, emotion and cognition: Attractor states in the classroom. In Z. Dörnyei, P. D. MacIntyre, & A. Henry (Eds.), Motivational dynamics in language learning (pp. 195–213). Multilingual Matters. Google Scholar logo with link to Google Scholar
Wickham, H., François, R., Henry, L., Müller, K., & Vaughan, D. (2023). dplyr: A grammar of data manipulation. [URL]
Wu, M. Y., Steinkrauss, R., & Lowie, W. (2023). The reliability of single task assessment in longitudinal L2 writing research. Journal of Second Language Writing, 59, 100950. Google Scholar logo with link to Google Scholar
Wu, Y. M. (2025). The reliability of quantitative CAF measures in the assessment of L2 English speaking skills. In Mind the metrics: CAF measure reliability and the implications for L2 studies (Ch. 3, Doctoral dissertation). University of Groningen.
Yu, H., Lowie, W. & Peng, H. (2024). Understanding salient trajectories and emerging profiles in the development of Chinese learners’ motivation: A growth mixture modeling approach. International Review of Applied Linguistics in Language Teaching, 62(2), 223–248. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue