In:Meaningful Language Test Scores: Research to enhance score interpretation
Edited by Spiros Papageorgiou and Venessa F. Manna
[Innovations in Language Learning and Assessment 1] 2023
► pp. 158–167
Chapter 9Literacy, transparency, and (mis)interpretations in communicating
with testing stakeholders
Published online: 29 June 2023
https://doi.org/10.1075/illa.1.09bak
https://doi.org/10.1075/illa.1.09bak
Abstract
In this chapter, I reflect upon the common goal of the
contributors of this volume – as evident in its title – to enhance score
meaning for test users. I first discuss this goal related to concepts
such as language assessment literacy and transparency. Then I focus on
the score interpretation process itself. The contributions of this
volume arguably represent the first half of this process: the provision
of research outcomes and other information designed to make scores more
meaningful. The second half of the process begins with communication of
these research outcomes to the intended audience. I suggest a program of
research for this second half of the interpretation process to examine
whether this research information is taken up and understood as
intended.
Article outline
- Introduction
- Transparency as a motivation to provide enhanced information to score users
- Critical moments of (mis)interpretation
- Possible misinterpretations related to the common vertical scale
- Possible misinterpretations related to scale anchoring
- Possible misinterpretations in score mapping
- Research program
- Phase 1: Identifying misconceptions
- Phase 2: Remediating misconceptions (and evaluating success)
- Conclusion
References
References (15)
Chi, M. T. H. (2008). Three
types of conceptual change: Belief revision, mental model
transformation, and categorical
shift. In S. Vosniadou (Ed.), Handbook
of research on conceptual
change (pp. 61–82). Lawrence Erlbaum Associates.
(2013). Two
kinds and four sub-types of misconceived knowledge, ways to
change it and the learning
outcomes. In S. Vosniadou (Ed.), International
handbook of research on conceptual
change (pp. 49–70). Routledge.
Davis, L., Garcia Gomez, P., Li, S., & Manna, V. F. (this
volume). Mapping TOEFL
Essentials speaking and writing scores to the CEFR
levels. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Gu, L., Li, Y., Monfils, L., & Papageorgiou, S. (this
volume). Statistical
methodology for developing vertical scales for language
tests. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Jankowski, N., & Provezis, S. (2014). Neoliberal
ideologies, governmentality and the academy: An examination
of accountability through assessment and
transparency. Educational
Philosophy and
Theory, 46(5), 475–487.
Jönsson, A. (2014). Rubrics
as a way of providing transparency in
assessment. Assessment and
Evaluation in Higher
Education, 39(7), 840–852.
Jönsson, A., & Prins, F. (2019). Editorial:
Transparency in assessment: Exploring the influence of
explicit assessment
criteria. Frontiers in
Education, 3, 1–3.
Kolen (2011). Issues
associated with vertical scales for PARCC
Assessments. Retrieved on 8
February 2023 from [URL]
Li, T., & Papageorgiou, S. (this
volume). Designing and
executing a score concordance study to establish score
mapping to proficiency
levels. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Papageorgiou, S., & Manna, V. F. (this
volume). Overview of
research to enhance score
interpretation. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.
Wang, L., & Papageorgiou, S. (this
volume). Scale anchoring
methodology for developing revised performance level
descriptors for the TOEFL iBT
test. In S. Papageorgiou & V. F. Manna (Eds.), Meaningful
language test scores: Research to enhance score
interpretation. John Benjamins.