Cover not available

Article published In: Journal of Immersion and Content-Based Language Education
Vol. 9:1 (2021) ► pp.4–30

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Are CLIL texts too complicated?

A computational analysis of their linguistic characteristics

 | University of the Basque Country (UPV/EHU)
 | University of the Basque Country (UPV/EHU)
ORCiD logo with link | University of the Basque Country (UPV/EHU)
Published online: 27 October 2020
DOI logo with link
https://doi.org/10.1075/jicb.19022.agu
Abstract
This article presents a comparative study of the linguistic characteristics of some materials used to teach English as a foreign language, and Geography and History through English in a Content and Language Integrated Learning (CLIL) experience in the Basque Country with students aged 11–13. This paper analyzes and compares the contents of current textbooks using Coh-Metrix and AzterTest, which calculate stylistic and linguistic metrics regarding lexical and grammatical complexity, readability and coherence. Finally, the study suggests that there are significant differences mainly in vocabulary level, narrativity and cohesion, it identifies the potential difficulties of CLIL texts and offers advice on how to overcome them. Raising awareness of the complexity of some texts used in CLIL can provide a starting point for pedagogical adaptations and contribute to optimizing learning.
Laburpena
Artikulu honetan, Euskal Autonomia Erkidegoan 11–13 urteko ikasleek Ingelesa eta Geografia eta Historia ingelesez ikasteko erabiltzen dituzten materialen arteko konparazioa egiten dugu. Zehazki, edukien eta hizkuntzen ikaskuntza integratua (CLIL) erabiltzen diren testuen hizkuntza-ezaugarriak aztertzen ditugu Coh-Metrix eta AzterTest tresnak erabilita. Tresna horiek neurri linguistikoak eta estilistikoak kalkutatzen dituzte hainbat mailatan: hiztegia, sintaxia, sematika eta koheretzia. Neurri horietan oinarrituta ondorioztatzen dugu alde nabarmenak daudela batez ere hiztegiaren, narratibotasunaren eta kohesioaren mailetan. Ikerketa honetan, CLILeko testuen zailtasun potentzialak identifikatzen ditugu eta horiek gainditzeko aholkularitza eskaintzen dugu. Izan ere, CLILen erabiltzen diren testu batzuen konplexutasuna nabarmentzea abiapuntua izan daiteke egokitzapen pedagogikoetarako eta ikaskuntza hobetzen laguntzeko.
Article outline
  • 1.Introduction
  • 2.Method
    • 2.1Participants
    • 2.2Characteristics of genres in History
    • 2.3Tools
  • 3.Results
    • 3.1Analysis of principal components in CLIL texts and texts for native English speakers
    • 3.2Analysis of principal components in ESL and CLIL texts
    • 3.3Differences between ESL and CLIL texts
      • 3.3.1Descriptive metrics
      • 3.3.2Lexical diversity
      • 3.3.3Word information
      • 3.3.4Polysemy and hypernymy
      • 3.3.5Morphological information
      • 3.3.6Syntactic complexity
      • 3.3.7Referential cohesion
      • 3.3.8Connectives (logical cohesion)
      • 3.3.9Causal/Intentional cohesion
  • 4.Discussion
  • 5.Conclusion
  • Notes
  • References
References (53)
References
Anderson, A., Garrod, S. C., & Sanford, A. J. (1983). The accessibility of pronominal antecedents as a function of episode shifts in narrative text. Quarterly Journal of Experimental Psychology, 35A1, 427–440. Google Scholar logo with link to Google Scholar
Alva-Manchego, F., Scarton, C., & Specia, L. (2020). Data-driven sentence simplification: Survey and benchmark. Computational Linguistics, 1–87. Google Scholar logo with link to Google Scholar
Beacco, J. C. & Byram, M. (2007). From linguistic diversity to plurilingual education: Guide for the development of language education policies in Europe. Council of Europe.Google Scholar logo with link to Google Scholar
Bengoetxea, K., Gonzalez-Dios, I., & Aguirregoitia, A. (2020). AzterTest: Open source linguistic and stylistic analysis tool. Procesamiento del Lenguaje Natural, 641, 1–8. Google Scholar logo with link to Google Scholar
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? Tesol Quarterly, 45(1), 5–35. Google Scholar logo with link to Google Scholar
Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman grammar spoken and written English. Longman.Google Scholar logo with link to Google Scholar
Boroş, T., Dumitrescu, S. D., & Burtica, R. (2018). NLP-Cube: End-to-end raw text processing with neural networks. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies (pp. 171–179). Association for Computational Linguistics. Google Scholar logo with link to Google Scholar
Cenoz, J. (2009). Towards multilingual education: Basque educational research from an international perspective. Multilingual Matters. Google Scholar logo with link to Google Scholar
(2019). Translanguaging pedagogies and English as a lingua franca. Language Teaching, 52(1), 71–85. Google Scholar logo with link to Google Scholar
Coyle, D. (2010). Foreword. In D. Lasagabaster & Y. Ruiz de Zarobe (Eds.) CLIL in Spain: Implementation, results and teacher training, pp. vii–viii. Cambridge Scholars Publishing. Google Scholar logo with link to Google Scholar
Crossley, S. A., Allen, D. B., & McNamara, D. S. (2011). Text readability and intuitive simplification: A comparison of readability formulas. Reading in a foreign language, 23(1), 84–101.
CrossRef DOI logo with hyperlink to permanent DOI
Crossley, S. A., Greenfield, J., & McNamara, D. S. (2008). Assessing text readability using cognitively based indices. Tesol Quarterly, 42(3), 475–493. Google Scholar logo with link to Google Scholar
Crossley, S. A., Louwerse, M. M., McCarthy, P. M., & McNamara, D. S. (2007). A linguistic analysis of simplified and authentic texts. The Modern Language Journal, 91(1), 15–30. Google Scholar logo with link to Google Scholar
Cummins, J. (2008). Teaching for transfer: Challenging the two solitudes assumption in bilingual education. Encyclopedia of language and education, 51, 65–75. Google Scholar logo with link to Google Scholar
(2009). Multilingualism in the English-language classroom: Pedagogical considerations. TESOL quarterly, 43(2), 317–321. Google Scholar logo with link to Google Scholar
Der, G., & Everitt, B. S. (2008). A handbook of statistical analyses using SAS. Chapman and Hall CRC.. Google Scholar logo with link to Google Scholar
Eusko Jaurlaritza (2010). Proceso de experimentación del marco de educación trilingüe. Documento marco, 2010–2011. Departamento de educación. [URL]
(2018). Matrikula: Bilakaera irudiak. [URL]
Franceschini, R. (2013). “History of multilingualism.” In the Encyclopedia of Applied Linguistics, edited by C. A. Chapelle. 1–9. Blackwell Publishing.Google Scholar logo with link to Google Scholar
Gierlinger, E. M. (2017). Teaching CLIL? Journal of Immersion and Content-Based Language Education, 5(2), 187–213. Google Scholar logo with link to Google Scholar
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational researcher, 40(5), 223–234. Google Scholar logo with link to Google Scholar
Graesser, A. C., McNamara, D. S., Cai, Z., Conley, M., Li, H., & Pennebaker, J. (2014). Coh-Metrix measures text characteristics at multiple levels of language and discourse. The Elementary School Journal, 115(2), 210–229. Google Scholar logo with link to Google Scholar
Gonzalez-Dios, I., Bengoetxea, K., & Aguirregoitia, A. (2020). LagunTest: A NLP based application to enhance reading comprehension. In Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding Difficulties (READI), pp. 63–69. European Language Resources Association.Google Scholar logo with link to Google Scholar
Heras, A., & Lasagabaster, D. (2015). The impact of CLIL on affective factors and vocabulary learning. Language Teaching Research, 19(1), 70–88. Google Scholar logo with link to Google Scholar
Huguet, À., Lasagabaster, D., & Vila, I. (2008). Bilingual education in Spain: Present realities and future challenges. Encyclopedia of language and education, 1672–1682. Google Scholar logo with link to Google Scholar
Just, M. A., & Carpenter, P. A. (1987). The psychology of reading and language comprehension. Allyn & Bacon, Inc.Google Scholar logo with link to Google Scholar
Kintsch, W. (1998). Comprehension: A paradigm for cognition. Cambridge University Press.Google Scholar logo with link to Google Scholar
Kintsch, W., & Van Dijk, T. A. (1978). Toward a model of text comprehension and production. Psychological review, 85(5), 363–394. Google Scholar logo with link to Google Scholar
Llinares, A., & Morton, T. (Eds.). (2017). Applied linguistics perspectives on CLIL. John Benjamins Publishing Company. Google Scholar logo with link to Google Scholar
Llinares, A., Morton, T., & Whittaker, R. (2012). The roles of language in CLIL. Cambridge University Press.Google Scholar logo with link to Google Scholar
Llinares, A. & Whittaker, R.. (2010). Writing and speaking in the history class. In C. Dalton-Puffer & T. Nikula (Eds.) Language use and language learning in CLIL classrooms, (pp.125–124). John Benjamin Publishing. Google Scholar logo with link to Google Scholar
Lorenzo, F. (2008). Instructional discourse in bilingual settings. An empirical study of linguistic adjustments in content and language integrated learning. Language Learning Journal, 36(1), 21–33. Google Scholar logo with link to Google Scholar
(2013). Genre-based curricula: multilingual academic literacy in content and language integrated learning. International Journal of Bilingual Education and Bilingualism, 16(3), 375–388. Google Scholar logo with link to Google Scholar
Lorenzo, F., Granados, A. & Ávila, I. (2019). The development of cognitive academic language proficiency in multilingual education: Evidence of a longitudinal study on the language of history. Journal of English for Academic Purposes, 411, 100767. Google Scholar logo with link to Google Scholar
López, L. E., Sichra, I. (2008). Intercultural Bilingual Education Among Indigenous Peoples in Latin America. In Hornberger, N. H. (eds). Encyclopedia of Language and Education. Springer. Google Scholar logo with link to Google Scholar
Lyster, R., Ruiz de Zarobe, Y. (2018). Introduction: instructional practices and teacher development in CLIL and immersion school settings. International Journal of Bilingual Education and Bilingualism, 21(3), 273–274. Google Scholar logo with link to Google Scholar
McCarthy, P. M., & Jarvis, S. (2007). vocd: A theoretical and empirical evaluation. Language Testing, 24(4), 459–488. Google Scholar logo with link to Google Scholar
McCarthy, P. M. (2005). An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD) [Doctoral dissertation]. The University of Memphis.Google Scholar logo with link to Google Scholar
McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge University Press. Google Scholar logo with link to Google Scholar
Mercer, N., Wegerif, R., & Major, L. (2019). The Routledge international handbook of research on dialogic education. Routledge. Google Scholar logo with link to Google Scholar
Meyer, O. (2010). Towards quality CLIL: Successful planning and teaching strategies. PULSO. Revista de Educación, 331, 11–29.Google Scholar logo with link to Google Scholar
Mikolov, T., Grave, É., Bojanowski, P., Puhrsch, C., & Joulin, A. (2018). Advances in pre-training distributed word representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pp 52–55. European Language Resources Association.Google Scholar logo with link to Google Scholar
Nikula, T., & Moore, P. (2019). Exploring translanguaging in CLIL. International Journal of Bilingual Education and Bilingualism, 22(2), 237–249. Google Scholar logo with link to Google Scholar
O’Reilly, T., & McNamara, D. S. (2007). The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high-stakes” measures of high school students’ science achievement. American Educational Research Journal, 44(1), 161–196. Google Scholar logo with link to Google Scholar
Otwinowska, A., & Foryś, M. (2017). They learn the CLIL way, but do they like it? Affectivity and cognition in upper-primary CLIL classes. International Journal of Bilingual Education and Bilingualism, 20(5), 457–480. Google Scholar logo with link to Google Scholar
Qi, P., Dozat, T., Zhang, Y., & Manning, C. 2018. Universal Dependency Parsing from Scratch. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pp. 160–170. Google Scholar logo with link to Google Scholar
Ruiz de Zarobe, Y. (2013). CLIL implementation: From policy-makers to individual initiatives. International Journal of Bilingual Education and Bilingualism, 161, 231–243. Google Scholar logo with link to Google Scholar
Saggion, H. (2017). Automatic text simplification. Synthesis Lectures on Human Language Technologies, 10(1), 1–137. Google Scholar logo with link to Google Scholar
Schleppegrell, M. J. (2004). The language of schooling: A functional linguistics perspective. Lawrence Erlbaum Associates. Google Scholar logo with link to Google Scholar
Snow, C. (2002). Reading for understanding: Toward an R&D program in reading comprehension. Rand.Google Scholar logo with link to Google Scholar
Speer, R., J. Chin, A. Lin, S. Jewett, & Nathan, L. (2018). Luminosoinsight/wordfreq:v2.2. Google Scholar logo with link to Google Scholar
Whittaker, R., Llinares, A., & McCabe, A. (2011). Written discourse development in CLIL at secondary school. Language Teaching Research, 15(3), 343–362. Google Scholar logo with link to Google Scholar
Zeno, S., Ivens, S. H., Millard, R. T., & Duvvuri, R. (1995). The educator’s word frequency guide. Touchstone Applied Science Associates.Google Scholar logo with link to Google Scholar
Cited by (6)

Cited by six other publications

Vasilachi, Serghei & Livia Dombrovschi
2025. Multilingvism și Interculturalitate în Contextul Globalizări, Ediția 4, ► pp. 117 ff. DOI logo
Wang, Qingru & Wang Ping
2024. 30 Years of development of English teaching materials: A bibliometric analysis. International Journal of Applied Linguistics 34:1 ► pp. 383 ff. DOI logo
Alas, Ene, Aleksandra Ljalikova & Merle Jung
2023. CLIL teacher beliefs as they emerge working in tandem. Journal of Immersion and Content-Based Language Education 11:2 ► pp. 229 ff. DOI logo
Milla, Ruth & Amaia Aguirregoitia Martínez
2023. CLIL Teachers' Beliefs and Practices. In Handbook of Research on Training Teachers for Bilingual Education in Primary Schools [Advances in Early Childhood and K-12 Education, ], ► pp. 129 ff. DOI logo
Cenoz, Jasone & Durk Gorter
2022. Pedagogical Translanguaging, DOI logo
Cenoz, Jasone & Durk Gorter
2025. The potential of pedagogical translanguaging in English language and in English-medium content classes. Linguistics and Education 86 ► pp. 101399 ff. DOI logo

This list is based on CrossRef data as of 13 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue