In:Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 35–60
Chapter 3Assessment design issues in developing vertical scales for
language tests
Published online: 29 June 2023
https://doi.org/10.1075/illa.1.03pap
https://doi.org/10.1075/illa.1.03pap
Abstract
To support decisions about readiness to take a test in the
TOEFL® Family of Assessments, a multi-year project was
launched in 2017 to explore the possibility of expressing scores across
multiple tests on a single, consistently interpreted scale by applying
vertical linking procedures. This chapter focuses on the assessment
design aspects of the vertical linking project. It describes the
selection of vertical linking items for listening and reading and the
development of test forms so that test taker responses could be
collected. The chapter also discusses examples of content analysis of
the vertical linking items flagged during the statistical analysis
described in Chapter 4. The
chapter concludes with implications for suites of language proficiency
tests regarding the selection of vertical linking items.
Article outline
- Introduction
- Overview of the vertical linking design for the TOEFL family of assessments
- Selection of listening and reading task types
- Selection of vertical linking items for the test forms
- TOEFL Primary test forms with TOEFL Junior vertical linking items
- TOEFL Junior test forms with TOEFL Primary vertical linking items
- TOEFL Junior test forms with TOEFL ITP vertical linking items
- TOEFL ITP test forms with TOEFL Junior vertical linking items
- TOEFL ITP test forms with TOEFL iBT vertical linking items
- TOEFL iBT test forms with TOEFL ITP vertical linking items
- Content analysis after data collection
- Conclusion
Acknowledgments Notes References
References (19)
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, teaching, assessment. Cambridge University Press.
Cho, Y., Ginsburgh, M., Moulder, B., Morgan, R., Xi, X., & Hauck, M. (2016). Designing
the TOEFL Primary Tests (ETS
Research Memorandum
RM–16–02). ETS. Retrieved
on 7 February 2023 from [URL]
ETS (2017). Test
taker handbook for the TOEFL ITP®
tests. Retrieved on 7 February 2023
from [URL]
(2018). Handbook
for the TOEFL Junior®
tests. Retrieved on 7 February 2023
from [URL]
(2019). Handbook
for the TOEFL Primary®
tests. Retrieved on 7 February 2023
from [URL]
Gu, L., Li, Y., Monfils, L., & Papageorgiou, S. (this
volume). Statistical
methodology for developing vertical scales for language
tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Gu., L., Wang, L., Cho, Y. (2020). Impact
of printed stems on listening performance of TOEFL ITP test
takers (Unpublished technical
report). ETS.
Kolen, M. J., & Brennan, R. L. (2014). Test
equating, scaling, and linking: Methods and
practices (3rd
ed.). Springer.
Li, F. (2020). Mode
comparability study for the TOEFL Primary
tests (Unpublished technical
report). ETS.
Monfils, L, & Manna, V. F. (this
volume). Considerations in
developing vertical scales for language
tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Papageorgiou, S., & Baron, P. A. (2017). Using
the Common European Framework of Reference for young
learners’ English language proficiency
assessments. In M. K. Wolf & Y. G. Butler (Eds.), English
language proficiency assessments for young
learners (pp. 136–152). Routledge.
Papageorgiou, S., Davis, L., Norris, J. M., Garcia Gomez, P., Manna, V. F., & Monfils, L. (2021). Design
framework for the TOEFL® Essentials™
test
2021 (Research
Memorandum No.
RM–21–03). ETS. Retrieved
on 7 February 2023 from [URL]
Papageorgiou, S., Morgan, R., & Becker, V. (2015). Enhancing
the interpretability of the overall results of an
international test of English language
proficiency. International
Journal of
Testing, 15(4), 310–336.
Papageorgiou, S., Tannenbaum, R. J., Bridgeman, B., & Cho, Y. (2015). The
association between TOEFL iBT® test scores and
the Common European Framework of Reference (CEFR)
levels (Research Memorandum No.
RM–15–06). ETS. Retrieved
on 7 February 2023 from [URL]
Papageorgiou, S., Wu, S., Hsieh, C.-N., Tannenbaum, R. J., & Cheng, M. M. (2019). Mapping
the TOEFL iBT®
test scores to China’s Standards of English Language
Ability: Implications for score interpretation and
use (Research
Report No.
TOEFL-RR–89). ETS.
Powers, D., Schedl, M., & Papageorgiou, S. (2017). Facilitating
the interpretation of English language proficiency scores:
Combining scale anchoring and test score mapping
methodologies. Language
Testing, 34(2), 175–195.
So, Y., Wolf, M. K., Hauck, M. C., Mollaun, P., Rybinski, P., Tumposky, D., & Wang, L. (2015). TOEFL
Junior® Design
Framework (TOEFL Junior®
Research Report TOEFL
JR–02). ETS.
Tannenbaum, R. J., & Baron, P. A. (2011). Mapping
TOEFL® ITP scores onto the Common European
Framework of Reference. (ETS
Research Memorandum
RM–11–33). ETS. Retrieved
on 7 February 2023 from [URL]
Wang, L., & Papageorgiou, S. (this
volume). Scale anchoring
methodology for developing revised performance level
descriptors for the TOEFL iBT
test. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Cited by (1)
Cited by one other publication
This list is based on CrossRef data as of 12 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.