Cover not available

In:Instructed Second Language Acquisition Research Methods
Edited by Laura Gurzynski-Weiss and YouJin Kim
[Research Methods in Applied Linguistics 3] 2022
► pp. 329–352

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
EPUB logo with download linkDownload Book EPUB
authorization required

Chapter 14
Speaking

Complexity, accuracy, fluency, and functional adequacy (CAFFA)

 | University of Amsterdam
 | University of Amsterdam
Published online: 8 December 2022
DOI logo with link
https://doi.org/10.1075/rmal.3.14kui
Abstract
This chapter focuses on the assessment of oral performance in a second language (L2), viewed from the perspective of task-based language assessment and instructed second language acquisition. The notion of L2 proficiency, as presented in the Common European Framework of References (Council of Europe, 2001), rests on two pillars that have to be taken into account when measuring L2 speaking: (1) the linguistic dimension, referring to the complexity, accuracy, and fluency (CAF) of the speaker’s utterances; and (2) the communicative dimension, concerning the appropriateness and effectiveness of the message to be conveyed, labeled functional adequacy (FA). Whereas the majority of studies on the assessment of learner performance have focused on CAF, little attention has been devoted to the functional aspects of L2 performance. The rationale underlying the present chapter is that assessing L2 speaking is impossible without considering both CAF and FA (henceforth, CAFFA) and the mutual relationship between the two constructs. It is argued that L2 speaking should not only be assessed by measures along the CAF-triad but also in terms of FA. The chapter discusses the theoretical underpinnings and applicability of a rating scale for FA (Kuiken & Vedder, 2017, 2018).
Article outline
  • 1.What is CAFFA and why is it important?
    • 1.1Levelt’s model of speech production
  • 2.What we know and what we need to know about CAFFA in ISLA
    • 2.1CAFFA: Definition of constructs and operationalizations
    • 2.2CAFFA: Studies on the assessment of oral performance
  • 3.Data elicitation and interpretation
    • 3.1Guidelines for conducting CAFFA research
    • 3.2The WISP project
    • 3.3Development of a rating scale for FA
  • 4.Advice to future CAFFA researchers
    • 4.1Type of research
    • 4.2Target and source languages
    • 4.3Measures
    • 4.4Task type
    • 4.5Rater training
  • 5.Troubleshooting CAFFA research in ISLA
    • 5.1L1 and L2 speakers in one and the same study
    • 5.2Rater variables
    • 5.3Scale adaptations
  • 6.Conclusions
  • 7.Further reading and additional resources
    • 7.1Books and journal articles
    • 7.2Journals, professional organizations, and websites
  • References
References (58)
References
Brezina, V., & Pallotti, G. (2019). Morphological complexity in written L2 texts. Second Language Research, 35(1), 99–120. Google Scholar logo with link to Google Scholar
Bulté, B., & Housen, A. (2012). Defining and operationalising L2 complexity. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency: Complexity, accuracy and fluency in SLA (pp. 21–46). John Benjamins. Google Scholar logo with link to Google Scholar
Council of Europe (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press.Google Scholar logo with link to Google Scholar
De Bot, K. (1992). A bilingual production model: Levelt’s ‘speaking’ model adapted. Applied Linguistics 13(1), 1–24. Google Scholar logo with link to Google Scholar
De Clercq, B., & Housen, A. (2019). The development of morphological complexity: A cross-linguistic study of L2 French and English. Second Language Research, 35(1), 71–98. Google Scholar logo with link to Google Scholar
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H. (2012). The effect of task complexity on functional adequacy, fluency and lexical diversity in speaking performances of native and non-native speakers. In A. Housen, F. Kuiken, & I. Vedder (Eds.), Dimensions of L2 performance and proficiency. Complexity, accuracy and fluency in SLA (pp. 121–142). John Benjamins. Google Scholar logo with link to Google Scholar
Duijm, K., Schoonen, R., & Hulstijn, J. H. (2018). Professional and non-professional raters’ responsiveness to fluency and accuracy in L2 speech: An experimental approach. Language Testing, 35(4), 501–527. Google Scholar logo with link to Google Scholar
Ekiert, M., Lampropoulou, S., Révész, A., & Torgersen, E. (2018). The effects of task type and L2 proficiency on discourse appropriacy in oral task performance. In N. Taguchi & Y-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 247–264). John Benjamins. Google Scholar logo with link to Google Scholar
Foster, P., & Skehan, P. (1996). The influence of planning and task type on second language performance. Studies in Second Language Acquisition, 18(3), 299–323. Google Scholar logo with link to Google Scholar
Foster, P., & Wiggleworth, G. (2016). Capturing accuracy in second language performance: The case for a weighted clause ratio. Annual Review of Applied Linguistics, 36, 98–116. Google Scholar logo with link to Google Scholar
Gilabert, R., Barón Pares, J., & Levkina, M. (2011). Manipulating task complexity across task types and modes. In P. Robinson (Ed.), Second language task complexity: Researching the Cognition Hypotheses of language learning and performance (pp. 105–138). John Benjamins. Google Scholar logo with link to Google Scholar
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36(2), 193–202. Google Scholar logo with link to Google Scholar
Grice, H. P. (1975). Logic and conversation. In P. Cole, & J. L. Morgan (Eds.), Speech acts (pp. 41–58). Academic Press.Google Scholar logo with link to Google Scholar
Herraiz Martínez, A. (2018). Functional adequacy: The influence of English-medium instruction, English proficiency, and previous language learning experiences [Unpublished doctoral dissertation]. Universitat Jaume I, Castellón de la Plana.
Herraiz Martínez, A., & Alcón Soler, E. (2019). Pragmatic outcomes in the English-medium instruction context. Applied Pragmatics, 1(1), 68–91, Google Scholar logo with link to Google Scholar
Hulstijn, J. H., Schoonen, R., De Jong, N. H., Steinel, M. P., & Florijn, A. F. (2012). Linguistic competence of learners of Dutch as a second language at the B1 and B2 levels of speaking proficiency of the Common European Framework of Reference for Languages (CEFR). Language Testing, 29(2), 203–221. Google Scholar logo with link to Google Scholar
Izumi, S. (2003). Comprehension and production processes in second language learning: In search of the psycholinguistic rationale of the Output Hypothesis. Applied Linguistics 24(2), 168–197. Google Scholar logo with link to Google Scholar
Knoch, U. (2009). Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26(2), 275–304. Google Scholar logo with link to Google Scholar
Kormos, J. (2006). Speech production and second language acquisition. Psychology Press.Google Scholar logo with link to Google Scholar
Kuiken, F., & Vedder, I. (2014). Rating written performance: What do raters do and why? Language Testing, 31(3), 329–348. Google Scholar logo with link to Google Scholar
(2017). Functional adequacy in L2 writing: Towards a new rating scale. Language Testing, 34(3), 321–336. Google Scholar logo with link to Google Scholar
(2018). Assessing functional adequacy of L2 performance in a task-based approach. In N. Taguchi, & Y.-J. Kim (Eds.), Task-based approaches to teaching and assessing pragmatics (pp. 265–285). John Benjamins. Google Scholar logo with link to Google Scholar
Kuiken, F., Vedder, I., & Gilabert, R. (2010). Communicative adequacy and linguistic complexity in L2 writing. In I. Bartning, M. Martin, & I. Vedder (Eds.), Communicative proficiency and linguistic development: Intersections between SLA and language testing research (pp. 81–100). European Second Language Association.Google Scholar logo with link to Google Scholar
Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication [Doctoral dissertation, Georgia State University]. ScholarWorks. [URL]
Lahmann, C., Steinkrauss, R., & Schmid, M. S. (2019). Measuring linguistic complexity in long-term L2 speakers of English and L1 attriters of German. International Journal of Applied Linguistics, 29(2), 173–191. Google Scholar logo with link to Google Scholar
Lambert, C., & Nakamura, S. (2019). Proficiency-related variation in syntactic complexity: A study of English L1 and L2 oral descriptive discourse. International Journal of Applied Linguistics, 29(2), 248–264. Google Scholar logo with link to Google Scholar
Larsen-Freeman, D. (2009). Adjusting expectations: The study of complexity, accuracy, and fluency in second language acquisition. Applied Linguistics, 30(4), 579–589. Google Scholar logo with link to Google Scholar
Levelt, W. J. M. (1989). Speaking. From intention to articulation. The MIT Press.Google Scholar logo with link to Google Scholar
Long, M. (2015). Second language acquisition and task-based language teaching. Wiley Blackwell.Google Scholar logo with link to Google Scholar
Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4), 474–496. Google Scholar logo with link to Google Scholar
McNamara, T., & Roever, C. (2007). Testing: The social dimension. Blackwell. Google Scholar logo with link to Google Scholar
Michel, M. (2017). Complexity, accuracy, and fluency in L2 production. In S. Loewen & M. Sato (Eds.), The Routledge handbook of instructed second language acquisition (pp. 50–68). Routledge. Google Scholar logo with link to Google Scholar
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. Google Scholar logo with link to Google Scholar
Nuzzo, E., & Bove, G. (2020). Assessing functional adequacy across tasks: A comparison of learners’ and speakers’ written texts. E-JournALL, 7(2), 9–27. Google Scholar logo with link to Google Scholar
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24(4), 492–518. Google Scholar logo with link to Google Scholar
Pallotti, G. (2009). CAF: Defining, refining and differentiating constructs. Applied Linguistics, 30(4), 590–601. Google Scholar logo with link to Google Scholar
(2015). A simple view of linguistic complexity. Second Language Research, 31(1), 117–134. Google Scholar logo with link to Google Scholar
Paquot, M. (2019). The phraseological dimension in interlanguage complexity research. Second Language Research, 35(1), 121–145. Google Scholar logo with link to Google Scholar
Pill, J., & Smart, C. (2021). Rating: Behavior and training. In P. Winke & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 135–144). Routledge. Google Scholar logo with link to Google Scholar
Révész, A., Ekiert, M., & Torgersen, E. (2016). The effects of complexity, accuracy and fluency on communicative adequacy in oral task performance. Applied Linguistics, 37(6), 828–848. Google Scholar logo with link to Google Scholar
Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15(1), 18–39. Google Scholar logo with link to Google Scholar
Robinson, P. (2001). Task complexity, cognitive resources and syllabus design: A triadic framework for examining task influences on SLA. In P. Robinson (Ed.), Cognition and second language instruction (pp. 185–316). Cambridge University Press. Google Scholar logo with link to Google Scholar
Skehan, P. (1996). A framework for the implementation of task-based instruction. Applied Linguistics 17(1), 38–62. Google Scholar logo with link to Google Scholar
Timpe, V. (2013). Assessing intercultural communicative competence: The dependence of receptive sociopragmatic competence and discourse competence on learning opportunities and input. Peter Lang.Google Scholar logo with link to Google Scholar
Timpe-Laughlin, V. (2018). Pragmatics in task-based language assessment. Opportunities and challenges. In N. Taguchi, & Y.-J. Kim (Eds.), Task-based approaches to assessing pragmatics (pp. 288–304). John Benjamins. Google Scholar logo with link to Google Scholar
Upshur, J. A., & Turner, C. E. (1995). Constructing rating scales for second language tests. ELT Journal, 49(1), 3–12. Google Scholar logo with link to Google Scholar
Vasylets, O., Gilabert, R., & Manchón, R. M. (2019). Differential contribution of oral and written modes to lexical, syntactic and propositional complexity in L2 performance in instructed contexts. Instructed Second Language Acquisition, 3(2), 206–227. Google Scholar logo with link to Google Scholar
Vercellotti, M. A. (2019). Finding variation: Assessing the development of syntactic complexity in ESL speech. International Journal of Applied Linguistics, 29(2), 233–247. Google Scholar logo with link to Google Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy, and complexity. University of Hawai’i Press.Google Scholar logo with link to Google Scholar
Yuan, F., & Ellis, R. (2003). The effects of pre-task planning and on-line planning on fluency, complexity and accuracy in L2 monologic production. Applied Linguistics 24(1), 1–27. Google Scholar logo with link to Google Scholar
Zhang, Y., & Elder, C. (2014). Investigating native and non-native English-speaking teacher raters’ judgements of oral proficiency in the College English Test-Spoken English Test (CET-SET). Assessment in Education: Principles, Policy & Practice, 21(3), 306–325. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue