Cover not available

Article published In: ITL - International Journal of Applied Linguistics
Vol. 174:1 (2023) ► pp.135–165

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

3K-LEx-MC

The creation and validation of a multiple-choice vocabulary placement test for Spanish language learners

ORCiD logo with link | University of TorontoMississauga
ORCiD logo with link | The Human Resources Research Organization (HumRRO)
ORCiD logo with link | University of CaliforniaDavis
Published online: 25 October 2022
DOI logo with link
https://doi.org/10.1075/itl.22008.rob
Abstract
This study presents the development and validation of a 132-item Spanish-English bilingual multiple-choice vocabulary test based on the 3,000 most frequent lemmas that distinguishes between North American university students who satisfy the Foreign Language requirement and those who need to complete coursework. 819 students were assigned to one of the two 144-item forms of the preliminary test, which had 72 shared anchor items and other 72 form-specific items. Factor analysis was used to evaluate dimensionality and the Rasch model was used to select the items that best differentiated between these two student populations. This final form was administered to 213 students. Results showed high levels of unidimensionality, and the final form provided a Rasch reliability coefficient of 0.97.
Article outline
  • High frequency vocabulary in Spanish
  • Spanish vocabulary tests
  • The Rasch model
  • Methods
    • Participants
    • Instruments
    • Procedure
  • Results
    • Calibration and item bank development
    • Unidimensionality
    • Item level fit statistics
    • Construction of final form
    • Participants
    • Explanatory IRT
  • Conclusions
  • Notes
  • References
References (65)
References
Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research, and Evaluation, 18(6), 1–13. Google Scholar logo with link to Google Scholar
Beglar, D. (2010). A Rasch-based validation of the vocabulary size test. Language Testing, 27(1), 101–118. Google Scholar logo with link to Google Scholar
Berendt, E., Kurosaki, E., Maeda, A., Matsui, K., & Ochi, N. (2006). English loan words of Japanese elementary school children and their mental lexicon. Asian Englishes, 8(2), 26–45. Google Scholar logo with link to Google Scholar
Biemiller, A. (2003). Vocabulary: Needed if more children are to read well. Reading Psychology, 24(3–4), 323–335. Google Scholar logo with link to Google Scholar
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge. Google Scholar logo with link to Google Scholar
Bürkner, P. C. (2017). brms: An R package for bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. Google Scholar logo with link to Google Scholar
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddel, A. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76(1), 1–32. Google Scholar logo with link to Google Scholar
Casado, M. A., & Dereshiwsky, M. I. (2001). Foreign language anxiety of university students. College Student Journal, 35(4), 539–552.Google Scholar logo with link to Google Scholar
Chalmers, R. P. (2012). mirt: A Multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. Google Scholar logo with link to Google Scholar
Davies, M. (2005). Vocabulary range and text coverage: Insights from the forthcoming Routledge frequency dictionary of Spanish. In D. Eddington (Ed.), Selected proceedings of the 7th Hispanic Linguistics Symposium (pp. 106–115). Somerville, MA: Cascadilla Proceedings Project.Google Scholar logo with link to Google Scholar
Davies, M., & Hayward Davies, K. (2017). A frequency dictionary of Spanish: Core vocabulary for learners (2nd ed.). Routledge. Google Scholar logo with link to Google Scholar
de Ayala, R. J. (2009). The theory and practice of item response theory. The Guilford Press.Google Scholar logo with link to Google Scholar
De Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. Springer. Google Scholar logo with link to Google Scholar
Desjardins, C. D., & Bulut, O. (2018). Handbook of educational measurement and psychometrics using R. CRC Press. Google Scholar logo with link to Google Scholar
Dorans, N. J., Moses, T. P., & Eignor, D. R. (2010). Principles and practices of test score equating. ETS Research Report Series, 2010(2), i–41. Google Scholar logo with link to Google Scholar
Frantzen, D., & Magnan, S. S. (2005). Anxiety and the true beginner – false beginner dynamic in beginning French and Spanish classes. Foreign Language Annals, 38(2), 171–186. Google Scholar logo with link to Google Scholar
Floyd, F. J., & Widaman, K. F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment, 7(3), 286–299. Google Scholar logo with link to Google Scholar
Gairns, R., & Redman, S. (1986). Working with words. A guide to teaching and learning vocabulary. Cambridge University Press.Google Scholar logo with link to Google Scholar
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian data analysis (3rd ed.). CRC Press. Google Scholar logo with link to Google Scholar
Gyllstad, H., Vilkaitė, L., & Schmitt, N. (2015). Assessing vocabulary size through multiple-choice formats: Issues with guessing and sampling rates. International Journal of Applied Linguistics, 166(2), 278–306. Google Scholar logo with link to Google Scholar
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. SAGE Publications, Inc.Google Scholar logo with link to Google Scholar
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. Google Scholar logo with link to Google Scholar
Horwitz, E. (1990). Attending to the affective domain in the foreign language classroom. In S. Magnan (Ed.), Shifting the instructional focus to the learner (pp. 15–33). Northeast Conference on the Teaching of Foreign Languages.
Hu, M., & Nation, I. S. P. (2000). Vocabulary density and reading comprehension. Reading in a Foreign Language, 23(1), 4031–430.Google Scholar logo with link to Google Scholar
Institute for Test Research and Test Development. (2013). Assessing evidence of validity of the ACTFL reading proficiency test (RPT).Google Scholar logo with link to Google Scholar
Izura, C., Cuetos, F., & Brysbaert, M. (2014). Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica, 35(1), 49–66.Google Scholar logo with link to Google Scholar
Jones, R., & Tschirner, E. (2006). A frequency dictionary of German: Core vocabulary for learners. Routledge.Google Scholar logo with link to Google Scholar
Kaplan, R. M., & Saccuzzo, D. P. (2009). Psychological testing principles, applications, and issues (7th ed.). Wadsworth.Google Scholar logo with link to Google Scholar
Kruschke, J. K. (2015). Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan (2nd ed.). Elsevier.Google Scholar logo with link to Google Scholar
Laufer, B., & Aviad-Levitzky, A. (2017). What type of vocabulary knowledge predicts reading comprehension: Word meaning recall or word meaning recognition? The Modern Language Journal, 101(4), 729–741. [URL]Google Scholar logo with link to Google Scholar
Laufer, B., & McLean, S. (2016). Loanwords and vocabulary size test scores: A case of different estimates for different L1 learners. Language Assessment Quarterly, 13(3), 202–217. Google Scholar logo with link to Google Scholar
Liu, N., & Nation, I. S. P. (1985). Factors affecting guessing vocabulary in context. RELC Journal, 16(1), 33–42. Google Scholar logo with link to Google Scholar
Looney, D., & Lusin, N. (2019). Enrollments in languages other than English in United States institutions of higher education, summer 2016 and fall 2016: Final report. Modern Language Association of America. [URL]
McElreath, R. (2020). Statistical rethinking (2nd ed.). CRC Press. Google Scholar logo with link to Google Scholar
McLean, S., Kramer, B., & Beglar, D. (2015a). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741–760. Google Scholar logo with link to Google Scholar
McLean, S., Kramer, B., & Stewart, J. (2015b). An empirical examination of the effect of guessing on vocabulary size test scores. Vocabulary Learning and Instruction, 4(1), 26–35. Google Scholar logo with link to Google Scholar
McLean, S., Stewart, J., & Batty, A. O. (2020). Predicting L2 reading proficiency with modalities of vocabulary knowledge: A bootstrapping approach. Language Testing, 37(3), 389–411. Google Scholar logo with link to Google Scholar
McNamara, T., & Knoch, U. (2012). The Rasch wars: The emergence of Rasch measurement in language testing. Language Testing, 29(4), 555–576. Google Scholar logo with link to Google Scholar
Meara, P. (2010). EFL vocabulary test (2nd ed.). Centre for Applied Language Studies.Google Scholar logo with link to Google Scholar
Meara, P., & Milton, J. (2003). The Swansea levels test. Express.Google Scholar logo with link to Google Scholar
Milton, J. (2009). Measuring second language vocabulary acquisition. Multilingual Matters. Google Scholar logo with link to Google Scholar
Nation, I. S. P. (2006). How large a vocabulary is needed for reading and listening? Canadian Modern Language Review, 63(1), 591–82. Google Scholar logo with link to Google Scholar
Nation, I. S. P., & Beglar, D. (2007). A vocabulary size test. The Language Teacher, 31(7), 96–13.Google Scholar logo with link to Google Scholar
Nation, I. S. P., & Webb, S. A. (2011). Researching and analyzing vocabulary. Cengage.Google Scholar logo with link to Google Scholar
Nguyen, L. T. C., & Nation, I. S. P. (2011). A bilingual vocabulary size test of English for Vietnamese learners. RELC Journal, 42(1), 86–99. Google Scholar logo with link to Google Scholar
O’Connor, B. P. (2000). SPSS and BAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behavior Research Methods, Instruments, & Computers, 32(3), 396–402. Google Scholar logo with link to Google Scholar
Pellicer-Sánchez, A., & Schmitt, N. (2012). Scoring Yes–No vocabulary tests: Reaction time vs. nonword approaches. Language Testing, 29(4), 489–509. Google Scholar logo with link to Google Scholar
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. [URL]
Revelle, W. (2020). Psych: Procedures for psychological, psychometric, and personality research. Northwestern University. [URL]
Robles-García, P. (2020). 3K-LEx: Desarrollo y validación de una prueba de amplitud Léxica en español. Journal of Spanish Language Teaching, 7(1), 64–76. Google Scholar logo with link to Google Scholar
(2022). Receptive vocabulary knowledge in L2 learners of Spanish: The role of high-frequency words. Foreign Language Annals, 1–22. Google Scholar logo with link to Google Scholar
Schmith, R. (2002). Qualtrics [survey software]. Utah, U.S. available at [URL]
Schmitt, N., Cobb, T., Horst, M., & Schmitt, D. (2017). How much vocabulary is needed to use English? Replication of van Zeeland & Schmitt (2012), Nation (2006) and Cobb (2007). Language Teaching, 50(2), 212–226. Google Scholar logo with link to Google Scholar
Schmitt, N., Jiang, X., & Grabe, W. (2011). The percentage of words known in a text and reading comprehension. Modern Language Journal, 95(1), 26–43. Google Scholar logo with link to Google Scholar
Schmitt, N. & Schmitt, D. (2014). A reassessment of frequency and vocabulary size in L2 vocabulary teaching. Language Teaching, 47(4), 484–503. Google Scholar logo with link to Google Scholar
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the vocabulary levels test. Language Testing, 18(1), 55–88. Google Scholar logo with link to Google Scholar
Stoeckel, T. (2018). High-frequency and academic English vocabulary growth among first-year students at UNP. Journal of International Studies and Regional Development, 91, 15–30.Google Scholar logo with link to Google Scholar
Stoeckel, T., McLean, S., & Nation, I. S. P. (2021). Limitations of size and levels tests of written receptive vocabulary knowledge. Studies in Second Language Acquisition, 43(1), 181–203. Google Scholar logo with link to Google Scholar
Van Zeeland, H., & Schmitt, N. (2012). Lexical coverage in L1 and L2 listening comprehension: The same or different from reading comprehension? Applied Linguistics, 34(4), 457–479. Google Scholar logo with link to Google Scholar
Webb, S. A., & Chang, A. C.-S. (2012). Second language vocabulary growth. RELC Journal, 43(1), 113–126. Google Scholar logo with link to Google Scholar
Webb, S., Sasao, Y., & Ballance, O. (2017). The updated vocabulary levels test: Developing and validating two new forms of the VLT. International Journal of Applied Linguistics, 168(1), 34–70. Google Scholar logo with link to Google Scholar
Wilson, M., De Boeck, P., & Carstensen, C. H. (2008). Explanatory item response models: A brief introduction. In M. Wilson & P. De Boeck (Eds.), Explanatory Item Response Models (2nd ed.). Springer.Google Scholar logo with link to Google Scholar
Zhang, S., & Zhang, X. (2020). The relationship between vocabulary knowledge and L2 reading/listening comprehension: A meta-analysis. Language Teaching Research, 26(4), 696–725. Google Scholar logo with link to Google Scholar
Zhang, X. (2013). The I don’t know option in the vocabulary size test. TESOL Quarterly, 47(4), 790–811. Google Scholar logo with link to Google Scholar
Zipf, G. (1935). The psychobiology of language: An introduction to dynamic philology. MIT Press.Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Robles-García, Pablo, Ji-young Shin, Jeffrey Stewart & Stuart McLean
2026. Do modality and frequency matter in predicting reading proficiency from vocabulary knowledge? A comparison of meaning recognition and meaning recall. Journal of English for Academic Purposes 81 ► pp. 101667 ff. DOI logo
Robles-García, Pablo, Stuart McLean, Jeffrey Stewart, Ji-young Shin & Claudia Helena Sánchez-Gutiérrez
2024. The Development and Initial Validation of O-WSVLT, a Meaning-Recall Online L2 Spanish Vocabulary Levels Test. Language Assessment Quarterly 21:2 ► pp. 181 ff. DOI logo
Robles-García, Pablo, Jeffrey Stewart, Christopher Nicklin, Joseph P. Vitta, Stuart McLean & Brandon Kramer
2023. ‘The wisdom of crowds’: When teacher judgments outperform word-frequency as a predictor of students’ vocabulary knowledge. Language Teaching Research DOI logo

This list is based on CrossRef data as of 30 march 2026. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue