In:Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 120–140
Chapter 7Mapping TOEFL® Essentials™ speaking and writing scores
to the CEFR levels
Published online: 29 June 2023
https://doi.org/10.1075/illa.1.07dav
https://doi.org/10.1075/illa.1.07dav
Abstract
This chapter describes a study to map scores from a new
English language proficiency assessment, the TOEFL®
Essentials™ test, to the Common European Framework of Reference (CEFR).
The project started with a comparison of content between the TOEFL
Essentials test and the CEFR descriptors, which established a claim that
scores were relevant to language ability as described in the CEFR. A
standard setting study, using the Performance Profile method, was then
conducted to establish cut scores for each CEFR level. Additional
information to support the reasonableness of these cut scores is also
reported.
Article outline
- Introduction
- Speaking and Writing sections of the TOEFL Essentials test
- Methodology
- Construct congruence with the CEFR
- Materials used for the standard setting meeting
- Panelist selection and preparation
- Standard setting procedure
- Online context
- Results
- Construct congruence
- Raw cut scores for CEFR levels
- Conversion of raw cut scores to band scores
- Discussion
- Validity of study results
- Advantages and disadvantages of working remotely
- Limitations and future research
- Conclusion
References
References (18)
Cizek, G. J., & Bunch, M. B. (2007). Standard
setting: A guide to establishing performance standards on
tests. Sage.
Council of
Europe. (n.d.). CEFR
Descriptors (searchable). Retrieved
on 8 February 2023 from [URL]
. (2001). The
Common European Framework of Reference for languages:
Learning, teaching,
assessment. Cambridge University Press. Retrieved
on 8 February 2023 from [URL]
. (2009). Relating
language examinations to the Common European Framework of
Reference for Languages: Learning, teaching, assessment. A
manual. Council of Europe. Retrieved on 8
February 2023 from [URL]
. (2020). Common
European Framework of Reference for Languages: Learning,
teaching, assessment. Companion volume with new
descriptors. Retrieved on 8
February 2023 from [URL]
ETS. (2021a). TOEFL
Essentials Test: Speaking Scoring
Guide. Retrieved on 8 February 2023
from [URL]
. (2021b). TOEFL
Essentials Test: Writing Scoring
Guide. Retrieved on 8 February 2023
from [URL]
. (2021c). Setting
Score Requirements. Retrieved on 8
February 2023 from [URL]
Fleckenstein, J., Keller, S., Kruger, M., Tannenbaum, R. J., & Koller, O. (2020). Linking
TOEFL iBT® writing rubrics to CEFR levels: Cut
scores and validity evidence from a standard setting
study. Assessing
Writing, 43, 1–15.
Geisinger, K. F., & McCormick, C. A. (2010). Adopting
cut scores: Post-standard-setting panel considerations for
decision makers. Educational
Measurement: Issues and
Practice, 29(1), 38–44.
Gu, L., Li, Y., Monfils, L., & Papageorgiou, S. (this
volume). Statistical
methodology for developing vertical scales for language
tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Livingston, S. A. (2014). Equating
test scores (without
IRT). ETS. Retrieved
on 8 February 2023 from [URL]
Papageorgiou, S. (2010). Investigating
the decision-making process of standard setting
participants. Language
Testing, 27(2), 261–82.
Papageorgiou, S., Davis, L., Norris, J. M., Garcia Gomez, P., Manna, V. F., & Monfils, L. (2021). Design
framework for the TOEFL® Essentials™
test
2021 (Research
Memorandum No.
RM–21–03). ETS. Retrieved
on 8 February 2023 from [URL]
Papageorgiou, S., Ginsburgh, M., & Garcia Gomez, P. (this
volume). Assessment design
issues in developing vertical scales for language
tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Papageorgiou, S., Wu, S., Hsieh, C.-N., Tannenbaum, R. J., & Cheng, M. (2019). Mapping
the TOEFL iBT test scores to China’s Standards of English
Language Ability: Implications for score interpretation and
use (Research Report No.
TOEFL-RR–89). ETS.
Schmidgall, J. (this
volume). Exploring the
benefits and challenges of web-based approaches to standard
setting. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.