In:Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 141–155
Chapter 8Designing and executing a score concordance study to establish
score mapping to proficiency levels
Published online: 29 June 2023
https://doi.org/10.1075/illa.1.08li
https://doi.org/10.1075/illa.1.08li
Abstract
The TOEFL ProPlacer® test is a multistage adaptive
test (MST) designed to rapidly distinguish among learners at different
proficiency levels and to place them into English instruction courses
that best suit their learning needs. The project reported in this
chapter aimed to establish the mapping of the TOEFL ProPlacer scores to
the CEFR levels through comparison with the scores of another test in
the TOEFL® Family, the TOEFL ITP® test, whose
scores had already been linked to the CEFR levels. 1,013 test takers
were recruited to complete both tests in a counterbalanced design. Using
the equipercentile method, we examined the CEFR cutoff scores on the
TOEFL ITP score scale and identified the corresponding cutoff scores on
the TOEFL ProPlacer score scale. The proposed cutoff scores were then
evaluated using TOEFL ITP and TOEFL ProPlacer operational test data to
check for reasonableness and consistency. The current study demonstrates
a useful application of establishing CEFR mapping of the scores of a
language test though the CEFR mapping of the scores of another language test. In some contexts, such an application may be more
feasible than conducting a standard setting study with a panel of
experts, and therefore useful in that it can facilitate the
interpretation of test scores through their association to the CEFR
levels. Limitations of the study and suggestions for future research are
also discussed.
Article outline
- Introduction
- Method
- Instruments
- Data collection
- Data analysis
- Results
- Descriptive statistics
- Conclusion
Notes References
References (15)
Angoff, W. H. (1971). Scales,
norms and equivalent
scores. In R. L. Thorndike (Ed.), Educational
measurement (2nd
ed., pp. 508–600). American Council on Education.
Council of
Europe. (2001). Common
European Framework of Reference for Languages: Learning,
teaching,
assessment. Cambridge University Press.
. (2009). Relating
language examinations to the Common European Framework of
Reference for Languages: Learning, teaching, assessment
(CEFR). A Manual. Council of Europe. Retrieved on 8
February 2023 from [URL]
Dorans, N. J. (1999). Correspondences
between ACT and SAT I
scores (College Board Report
99–1). College Entrance Examination Board.
ETS. (2010). Comparing
TOEFL® and IELTS™ total
scores. Retrieved on 8
February 2023 from [URL]
. (2017). TOEFL
ITP test taker
handbook. Retrieved on 8
February 2023 from [URL]
Holland, P. W. (2007). A
framework and history for score
linking. In N. J. Dorans, M. Pommerich, & P. W. Holland (Eds.), Linking
and aligning scores and
scales (pp. 5–30). Springer.
Holland, P. W., & Dorans, N. J. (2006). Linking
and
equating. In R. L. Brennan (Ed.), Educational
measurement (4th
ed., pp. 187–220). Praeger.
Papageorgiou, S. (2016). Aligning language
assessments to standards and
frameworks. In D. Tsagari & J. V. Banerjee (Eds.), Handbook
of second language
assessment (pp. 327–340). De Gruyter Mouton.
Papageorgiou, S., Tannenbaum, R. J., Bridgeman, B., & Cho, Y. (2015). The
association between TOEFL iBT® test scores and
the Common European Framework of Reference (CEFR)
levels (Research Memorandum No.
RM–15–06). ETS. Retrieved
on 8 February 2023 from [URL]
Papageorgiou, S., Wu, S., Hsieh, C.-N., Tannenbaum, R. J., & Cheng, M. M. (2019). Mapping
the TOEFL iBT®
test scores to China’s Standards of English Language
Ability: Implications for score interpretation and
use (Research
Report No.
TOEFL-RR–89). Educational Testing Service.
Peng, C., Liu, J. & Cai, H. (2022). Aligning
China’s standards of English language ability with the
Common European Framework of Reference for
Languages. Asia-Pacific
Education
Researcher 31, 667–677.