Cover not available

Article published In: International Journal of Learner Corpus Research
Vol. 7:2 (2021) ► pp.259–274

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required
Materials & Methods Report

fsca

French syntactic complexity analyzer

 | Université catholique de Louvain | Vrije Universiteit Brussel
Published online: 11 October 2021
DOI logo with link
https://doi.org/10.1075/ijlcr.20018.van
Abstract
This article reports on an open-source R package for the extraction of syntactic units from dependency-parsed French texts. To evaluate the reliability of the package, syntactic units were extracted from a corpus of L2 French and were compared to units extracted manually from the same corpus. The f-score of the extracted units ranged from 0.53–0.97. Although units were not always identical between the two methods, manual and automatically-derived syntactic complexity measures were strongly and significantly correlated (ρ = 0.62–0.97, p < 0.001), suggesting that this package may be a suitable replacement for manual annotation in some cases where manual annotation is not possible but that care should be used in interpreting the measures based on these units.
Article outline
  • 1.Introduction
  • 2.Methodology
    • 2.1Manual annotation
    • 2.2Automatic extraction of syntactic units
  • 3.Results
    • 3.1Precision and recall of automatically identified units
    • 3.2Correlation between manual and automatic methods
    • 3.3Sources of error
  • 4.Discussion and conclusion
  • Disclosures
  • Acknowledgements
  • Notes
  • References
References (34)
References
Abeillé, A., & Barrier, N. (2004). Enriching a French treebank. In Proceedings of the Fourth International Conference on Language Resources and Evaluations (LREC ’04), 2233–2236.Google Scholar logo with link to Google Scholar
Benevento, C., & Storch, N. (2011). Investigating writing development in secondary school learners of French. Assessing Writing, 16(2), 97–110. Google Scholar logo with link to Google Scholar
Bernardini, P., & Granfeldt, J. (2019). On cross-linguistic variation and measures of linguistic complexity in learner texts: Italian, French and English. International Journal of Applied Linguistics, 29(2), 211–232. Google Scholar logo with link to Google Scholar
Brown, J. D. (2014). Classical theory reliability. In A. J. Kunnen (Ed.), The companion to language assessment (pp. 1165–1181). Oxford: Wiley-Blackwell.Google Scholar logo with link to Google Scholar
Candito, M., Nivre, J., Denis, P., & Anguiano, E. H. (2010). Benchmarking of statistical dependency parsers for French. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010: Poster Volume), 108–116.Google Scholar logo with link to Google Scholar
Council of Europe. (2001). The common european framework of reference for languages: Learning, teaching, assessment. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Csardi, G., & Nepusz, T. (2006). The igraph software package for complex network research. InterJournal (Complex Systems), 1695. [URL]
De Clercq, B., & Housen, A. (2017). A cross-linguistic perspective on syntactic complexity in L2 development: Syntactic elaboration and diversity. The Modern Language Journal, 101(2), 315–334. Google Scholar logo with link to Google Scholar
Demol, A., & Hadermann, P. (2008). An exploratory study of discourse organisation in French L1, Dutch L1, French L2 and Dutch L2 written narratives. In G. Gilquin, S. Papp, & M. B. Díez-Bedmar (Eds.), Linking up contrastive and learner corpus research (pp. 255–282). Amsterdam: Brill. Google Scholar logo with link to Google Scholar
Denis, P., & Sagot, B. (2012). Coupling an annotated corpus and a lexicon for state-of-the-art POS tagging. Language Resources and Evaluation, 461, 721–736. Google Scholar logo with link to Google Scholar
Garretson, G. (2011). Dexter coder. Retrieved from [URL]
Henry, L., & Wickham, H. (2020). purrr: Functional programming tools. Retrieved from [URL]
Honnibal, M., & Montani, I. (2017). spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing.Google Scholar logo with link to Google Scholar
Klein, D., & Manning, C. (2003). Fast exact inference with a factored model for natural language parsing. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems 151 (pp. 3–10). Cambridge, MA: The MIT Press.Google Scholar logo with link to Google Scholar
Kuiken, F., & Vedder, I. (2008). Cognitive task complexity and written output in Italian and French as a foreign language. Journal of Second Language Writing, 17(1), 48–60. Google Scholar logo with link to Google Scholar
Kyle, K. (2021). (Ed.) Natural language processing for learner corpus research [Special issue]. International Journal of Learner Corpus Research 7(1). Google Scholar logo with link to Google Scholar
Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal, 102(2), 333–349. Google Scholar logo with link to Google Scholar
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. Google Scholar logo with link to Google Scholar
Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4), 474–496. Google Scholar logo with link to Google Scholar
(2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62.Google Scholar logo with link to Google Scholar
Nivre, J., Hall, J., & Nilsson, J. (2006). MaltParser: A data-driven parser-generator for dependency parsing. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006), 2216–2219.Google Scholar logo with link to Google Scholar
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30(4), 555–578. Google Scholar logo with link to Google Scholar
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college level L2 writing. Applied Linguistics, 24(4), 492–518. Google Scholar logo with link to Google Scholar
Plonsky, L., & Derrick, D. J. (2016). A meta-analysis of reliability coefficients in second language research. The Modern Language Journal, 100(2), 538–553. Google Scholar logo with link to Google Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. Google Scholar logo with link to Google Scholar
R Core Team. (2019). R: A language and environment for statistical computing. Retrieved from [URL]
RStudio Team. (2018). RStudio: Integrated Development Environment for R. Retrieved from [URL]
Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. The Public Opinion Quarterly, 19(3), 321–325. Google Scholar logo with link to Google Scholar
Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301–317. Google Scholar logo with link to Google Scholar
Vanderbauwhede, G. (2012). Le déterminant démonstratif en français et en néerlandais à travers les corpus: Théorie, description, acquisition (Unpublished doctoral dissertation). Katholieke Universiteit Leuven, Leuven, Belgium; Université Paris Ouest Nanterre La Défense, Paris, France.
Vandeweerd, N., Housen, A., & Paquot, M. (2021). Applying phraseological complexity measures to L2 French: A partial replication study. International Journal of Learner Corpus Research, 7(2), 197–229. Google Scholar logo with link to Google Scholar
Way, D. P., Joiner, E. G., & Seaman, M. A. (2000). Writing in the secondary foreign language classroom: The effects of prompts and tasks on novice learners of French. The Modern Language Journal, 84(2), 171–184. Google Scholar logo with link to Google Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. Honolulu, HI: Second Language Teaching & Curriculum Center.Google Scholar logo with link to Google Scholar
Cited by (4)

Cited by four other publications

Alzahrani, Alaa & Adel Alfaifi
2025. Arabic syntactic analyzer (ARSA): an automated tool for the analysis of Arabic written texts. Reading and Writing DOI logo
Loignon, Guillaume
2021. ILSA: an automated language complexity analysis tool for French. Mesure et évaluation en éducation 44:spécial ► pp. 61 ff. DOI logo
Loignon, Guillaume
2021. ALSI : un nouvel outil d’analyse automatisée de la complexité linguistique pour le français québécois. Mesure et évaluation en éducation 44:3 ► pp. 29 ff. DOI logo
Vandeweerd, Nathan, Alex Housen & Magali Paquot
2021. Applying phraseological complexity measures to L2 French. International Journal of Learner Corpus Research 7:2 ► pp. 197 ff. DOI logo

This list is based on CrossRef data as of 12 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue