Cover not available

Article published In: ITL - International Journal of Applied Linguistics
Vol. 176:1 (2025) ► pp.1–43

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB

Development and initial validation of a yes/no vocabulary test for North Sámi

Drawing on item response theory and signal detection theory

ORCiD logo with link | Lund University
 | University of Konstanz | UiT The Arctic University of Norway | Cluster of Excellence “The Politics of Inequality”
 | University of Konstanz | Cluster of Excellence “The Politics of Inequality”
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

Open Access publication of this article was funded through a Transformative Agreement with Lund University.

Published online: 5 August 2024
DOI logo with link
https://doi.org/10.1075/itl.23005.gyl
Abstract
This paper accounts for the development and initial validation of a yes/no vocabulary test of North Sámi called North Sámi Vocabulary Test (NSVT). North Sámi (NS) is an Indigenous language spoken in northern Scandinavia. Being an endangered language, NS is in need of revitalisation efforts. One contribution is the provision of proficiency assessment tools. We administered a 75-item NSVT version (50 real verbs and 25 pseudoverbs) to users of North Sámi in Sweden and Norway (N = 289). Evidence of high item- and person-based reliability and separation indices were observed, as well as support for a number of validity facets. High correlations existed between NSVT scores and self-reported and rated spoken proficiency. The NSVT provides quick and reliable assessment of vocabulary knowledge and an indication of overall language proficiency in North Sámi.
Article outline
  • Introduction
  • Background
    • Testing vocabulary knowledge
    • Psychometric theories and techniques
    • Aims and research questions
  • Developing the NSVT
    • Item construction and piloting
    • Item selection
    • Main administration of the NSVT in Sweden and Norway
  • Rasch and signal detection theory analyses
    • Summary statistics from the Rasch model and variable map
    • A SDT-based score calculation
    • Probing the relation between NSVT scores and spoken ability in North Sámi
  • Discussion
    • Reliability and validity
    • Recommendation for future implementations of the NSVT and future directions
    • Challenges
  • Conclusions
  • Notes
  • References
References (75)
References
Aikio-Puoskari, U. (2018). Revitalization of Sámi languages in three nordic countries: Finland, Norway, and Sweden. In L. Hinton, L. Huss, & G. Roche (Eds.), The Routledge Handbook of Language Revitalization (pp. 355–363). Routledge. Google Scholar logo with link to Google Scholar
Albury, N. J. (2016). Holding them at arm’s length: A critical review of Norway’s policy on Sámi language maintenance. Journal of Home Language Research, 11, 1–16. Google Scholar logo with link to Google Scholar
Alderson, C. (2005). Diagnosing foreign language proficiency. Continuum.Google Scholar logo with link to Google Scholar
Amenta, S., Badan, L., & Brysbaert, M. (2021). LexITA: A quick and reliable assessment tool for Italian L2 receptive vocabulary size. Applied Linguistics, 42(2), 292–314. Google Scholar logo with link to Google Scholar
Aryadoust, V., Ng, L. Y., & Sayama, H. (2021). A comprehensive review of Rasch measurement in language assessment: Recommendations and guidelines for research. Language Testing, 38(1), 6–40. Google Scholar logo with link to Google Scholar
Beeckmans, R., Eyckmans, J., Janssens, V., Dufranne, M., & Van de Velde, H. (2001). Examining the yes/no vocabulary test: Some methodological issues in theory and practice. Language Testing, 18(3), 235–274. Google Scholar logo with link to Google Scholar
Beglar, D. (2010). A Rasch-based validation of the Vocabulary Size Test. Language Testing, 27(1), 101–118. Google Scholar logo with link to Google Scholar
Belančić, K. (2020). Language policy and Sámi education in Sweden: ideological and implementational spaces for Sámi language use [Unpublished PhD thesis]. Umeå University.
Belančić, K., & Lindgren, E. (2017). Discourses of functional bilingualism in the Sámi curriculum in Sweden. International Journal of Bilingual Education and Bilingualism, 23(5), 601–616. Google Scholar logo with link to Google Scholar
Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Brysbaert, M. (2013). Lextale_FR a fast, free, and efficient test to measure language proficiency in French. Psychologica Belgica, 53(1), 23–37. Google Scholar logo with link to Google Scholar
Duncan, P. W., Bode, R. K., Lai, S. M., & Perera, S. (2003). Rasch analysis of a new stroke-specific outcome scale: The stroke impact scale. Archives of Physical Medicine and Rehabilitation, 841, 950–963. Google Scholar logo with link to Google Scholar
Eberhard, D. M., Simons, G. F., & Fennig, C. D. (2022). Ethnologue: Languages of the World. 25th Edition. SIL International. Available at: [URL]
Eyckmans, J. (2004). Measuring receptive vocabulary size. LOT.Google Scholar logo with link to Google Scholar
Ferin, M., Gyllstad, H., Venagli, I., Zordan, A., & Kupisch, T. (2023). LexVEN: A quick vocabulary test for proficiency in Venetan. Isogloss, 9(1), 1–31. Google Scholar logo with link to Google Scholar
Giellatekno – Saami Language Technology, UiT The Arctic University of Norway and The Divvun group at UiT The Arctic University of Norway (2015). SIKOR North Saami free corpus, Common Language Resources and Technology Infrastructure Norway (CLARINO) Bergen Repository. [URL]
Green, R. (2013). Statistical analyses for language testers. Routledge. Google Scholar logo with link to Google Scholar
Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.Google Scholar logo with link to Google Scholar
Grenoble, L. A., & Osipov, B. (2023). The dynamics of bilingualism in language shift ecologies. Linguistic Approaches to Bilingualism, 13(1), 1–39. Google Scholar logo with link to Google Scholar
Gyllstad, H. (2007). Testing English collocations: Developing receptive tests for use with advanced Swedish learners [Unpublished PhD Thesis]. Lund University.
(2013). Looking at L2 vocabulary knowledge dimensions from an assessment perspective: Challenges and potential solutions. In C. Bardel, C. Lindqvist, & B. Laufer (Eds.), L2 vocabulary acquisition, knowledge and use. New perspectives on assessment and corpus analysis (pp. 11–28). Eurosla Monograph Series 2. European Association of Second Language Acquisition.Google Scholar logo with link to Google Scholar
(2020). Measuring knowledge of multiword items. In S. Webb (Ed.), The Routledge Handbook of Vocabulary Studies (pp. 387–405). Routledge.Google Scholar logo with link to Google Scholar
Gyllstad, H., Vilkaitė, L., & Schmitt, N. (2015). Assessing vocabulary size through multiple-choice formats: Issues with guessing and sampling rates. ITL International Journal of Applied Linguistics, 166(2), 278–306. Google Scholar logo with link to Google Scholar
Gyllstad, H., McLean, S., & Stewart, J. (2021). Using confidence intervals to determine adequate item sample sizes for vocabulary tests: An essential but overlooked practice. Language Testing, 38(4), 558–579. Google Scholar logo with link to Google Scholar
Harrington, M., & Carey, M. (2009). The on-line Yes/No test as a placement tool. System, 37(4), 614–626. Google Scholar logo with link to Google Scholar
Hashimoto, B. J. (2021). Is frequency enough?: The frequency model in vocabulary size testing. Language Assessment Quarterly, 18(2), 171–187. Google Scholar logo with link to Google Scholar
Henning, G. (1987). A guide to language testing. Heinle and Heinle.Google Scholar logo with link to Google Scholar
Hirvonen, V. (2008). ‘Out on the fells, I feel like a Sámi’: Is There Linguistic and Cultural Equality in the Sámi School? In N. Hornberger (Ed.), Can schools save indigenous languages? (pp. 15–41). Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Huibregtse, I., Admiraal, W., & Meara, P. (2002). Scores on a yes–no vocabulary test: Correction for guessing and response style. Language Testing, 19(3), 227–245. Google Scholar logo with link to Google Scholar
Izura, C., Cuetos, F., & Brysbaert, M. (2014). Lextale-Esp: A test to rapidly and efficiently assess the Spanish vocabulary size. Psicológica, 35(1), 49–66.Google Scholar logo with link to Google Scholar
Knoch, U., & McNamara, T. (2015). Rasch analysis. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 275–304). Routledge. Google Scholar logo with link to Google Scholar
Kupisch, T., Arona, S., Besler, A., Cruschina, S., Ferin, M., Gyllstad, H., & Venagli, I. (2023). LexSIC: A quick vocabulary test for proficiency in Sicilian. Isogloss, 9(1), 1–24. Google Scholar logo with link to Google Scholar
Leiner, D. J. (2014). SoSci Survey (Version 2.4.00-i). [Computer software].Google Scholar logo with link to Google Scholar
Lemhöfer, K., & Broersma, M. (2012). Introducing LexTALE: A quick and valid lexical test for advanced learners of English. Behavior Research Methods, 44(2), 325–343. Google Scholar logo with link to Google Scholar
Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean. Rasch Measurement Transactions, 16(2), 878.Google Scholar logo with link to Google Scholar
(2018, 3 Sep.). Detecting multidimensionality in Rasch data using Winsteps Table 231 [Video]. YouTube. [URL]
(2021a). Winsteps® Rasch measurement computer program (Version 5.1.6). [URL]
(2021b). A user’s guide to Winsteps® Ministep Rasch-model computer programs (Program manual 5.1.6). [URL]
Lloyd-Smith, A., Bergman, F., Sapir, Y., Yasar, R., & Kupisch, T. (2023a). Why language matters. Inequality perceptions among the in Sweden and Norway. Policy Paper 11, Policy Papers of the Cluster “The Politics of Inequality”.Google Scholar logo with link to Google Scholar
Lloyd-Smith, A., Bergman, F., Hund, L., & Kupisch, T. (2023b). Can policies improve language vitality? The Sámi languages in Sweden and Norway. Frontiers in Psychology, 141. Google Scholar logo with link to Google Scholar
McIvor, O. (2020). Indigenous Language Revitalization and Applied Linguistics: Parallel Histories, Shared Futures? Annual Review of Applied Linguistics, 401, 78–96. Google Scholar logo with link to Google Scholar
McIvor, O., & Jacobs, P. (2016). NETOLNEW Language Learning Assessment Tool (v3.1). Retrieved from: [URL]
McLean, S., Kramer, B., & Beglar, D. (2015). The creation and validation of a listening vocabulary levels test. Language Teaching Research, 19(6), 741–760. Google Scholar logo with link to Google Scholar
Meara, P. (1996). The dimensions of lexical competence. In G. Brown, K. Malmkjær, & J. Williams (Eds.), Performance and competence in second language acquisition (pp. 33–55). Cambridge University Press.Google Scholar logo with link to Google Scholar
Meara, P. M., & Buxton, B. (1987). An alternative to multiple choice vocabulary tests. Language Testing, 4(2), 142–154. Google Scholar logo with link to Google Scholar
Meara, P., & Miralpeix, I. (2017). Tools for researching vocabulary. Multilingual Matters.Google Scholar logo with link to Google Scholar
Mercator European Research Centre on Multilingualism and Language Learning (2020). SÁMI. The Sámi language in education in Sweden.Google Scholar logo with link to Google Scholar
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–104). American Council on education and Macmillan.Google Scholar logo with link to Google Scholar
(1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. Google Scholar logo with link to Google Scholar
Milton, J., Wade, J., & Hopkins, N. (2010). Aural word recognition and oral competence in English as a foreign language. In R. Chacón-Beltrán, C. Abello-Contesse, & M. del Mar Torreblanca-López (Eds.), Insights into non-native vocabulary teaching and learning (pp. 83–98). Multilingual Matters. Google Scholar logo with link to Google Scholar
Min, S., & Aryadoust, V. (2021). A systematic review of item response theory in language assessment: Implications for the dimensionality of language ability. Studies in Educational Evaluation, 681, 100963. Google Scholar logo with link to Google Scholar
Minde, H. (2003). Assimilation of the Sami: Implementation and consequences. Acta Borealia, 201, 121–146. Google Scholar logo with link to Google Scholar
Mochida, K., & Harrington, M. (2006). The Yes/No test as a measure of receptive vocabulary knowledge. Language Testing, 23(1), 73–98. Google Scholar logo with link to Google Scholar
Nakata, T., Tamura, Y., & Aubrey, S. (2020). Examining the validity of the LexTALE test for Japanese college students. The journal of Asia TEFL, 17(2), 335–348.Google Scholar logo with link to Google Scholar
Nicklin, C., & Vitta, J. P. (2022). Assessing Rasch measurement estimation methods across R packages with yes/no vocabulary test data. Language Testing, 39(4), 513–540. Google Scholar logo with link to Google Scholar
Nation, P., & Beglar, D. (2007). A vocabulary size test. The Language Teacher, 31(7), 9–13.Google Scholar logo with link to Google Scholar
Ockey, G. (2017). Item response theory. In G. Fulcher, & F. Davidson (Eds.), The Routledge Handbook of Language Testing (pp. 336–349). Routledge.Google Scholar logo with link to Google Scholar
Outakoski, H. (2015). Multilingual Literacy among Young Learners of North Sámi: Contexts, Complexity and Writing in Sápmi. Department of language studies, Umeå University.Google Scholar logo with link to Google Scholar
Pellicer-Sánchez, A., & Schmitt, N. (2012). Scoring Yes–No vocabulary tests: Reaction time vs. nonword approaches. Language Testing, 29(4), 489–509. Google Scholar logo with link to Google Scholar
Plonsky, L. (2023). Sampling and generalizability in Lx research: A second-order synthesis. Languages, 8(1), 75. Google Scholar logo with link to Google Scholar
Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. Google Scholar logo with link to Google Scholar
Rasmussen, T., & Nolan, J. S. (2011). Reclaiming Sámi languages: Indigenous language emancipation from East to West. International Journal of the Sociology of Language, 2091, 35–55. Google Scholar logo with link to Google Scholar
Salminen, T. (2007). Europe and North Asia. In C. Moseley (Ed.), Encyclopedia of the World’s Endangered Languages (pp. 211–281). Routledge.Google Scholar logo with link to Google Scholar
Salmela, R., Lehtonen, M., Garusi, S., & Bertram, R. (2021). Lexize: A test to quickly assess vocabulary knowledge in Finnish. Scandinavian Journal of Psychology, 62(6), 806–819. Google Scholar logo with link to Google Scholar
Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55–88. Google Scholar logo with link to Google Scholar
Solstad, K. J., Balto, A. M. V., Nygaard, V., Jodesfsen, E., & Solstad, M. (2012). Samisk Sprakundersokelne (7).Google Scholar logo with link to Google Scholar
Staehr, L. S. (2008). Vocabulary size and the skills of listening, reading and writing. Language Learning Journal, 36(2), 139–152. Google Scholar logo with link to Google Scholar
Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection theory measures. Behavior Research Methods, Instruments and Computers, 31(1), 137–149. Google Scholar logo with link to Google Scholar
Stubbe, R. (2012). Do pseudoword false alarm rates and overestimation rates in yes–no vocabulary tests change with Japanese university students’ ability levels? Language Testing, 29(4), 471–488. Google Scholar logo with link to Google Scholar
Trosterud, T. (2008). Language assimilation during the modernisation process: Experiences from Norway and north-west Russia. Acta Borealia, 25(2), 93–112. Google Scholar logo with link to Google Scholar
Uchihara, T., & Clenton, J. (2020). Investigating the role of vocabulary size in second language speaking ability. Language Teaching Research, 24(4), 540–556. Google Scholar logo with link to Google Scholar
Vangsnes, Ø. (2019, December 13–16). A prognosis for Sámi in Norway: School as key to revitalization [Conference presentation]. Foundation for Endangered Languages XXIII, University of Sydney, Australia. [URL]
Yanagisawa, A., & Webb, S. (2020). Measuring depth of vocabulary knowledge. In S. Webb (Ed.), The Routledge Handbook of Vocabulary Studies (pp. 371–386). Routledge.Google Scholar logo with link to Google Scholar
Zhang, X., Liu, J., & Ai, H. (2020). Pseudowords and guessing in the Yes/No format vocabulary Test. Language Testing, 37(1), 6–30. Google Scholar logo with link to Google Scholar
Zimmerman, J., Broder, P. K., Shaughnessy, J. J., & Underwood, B. J. (1977). A recognition test of vocabulary using signal-detection measures and some correlates of word and non-word recognition. Intelligence, 1(1), 5–13. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue