Cover not available

Article published In: Journal of Second Language Studies
Vol. 6:1 (2023) ► pp.61–94

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Developing and evaluating a contextualized interactional competence rating scale based on a metaphorical conceptualization

A pragmatic mixed-method approach

 | Iowa State University of Science and Technology, United States
Published online: 10 January 2023
DOI logo with link
https://doi.org/10.1075/jsls.22003.nei
Abstract
Rating scales developed to measure interactional competence (IC) are mainly data-driven which can incur unaffordable costs for assessment practitioners with limited resources, such as universities using this test for placement purposes. A cheaper alternative is to use models proposed in the literature to develop such rating scales. This mixed-methods study evaluates a contextualized analytic IC rating scale developed based on Galaczi, E., & Taylor, L. (2018). Interactional Competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly, 15(3), 219–236. Google Scholar logo with link to Google Scholar metaphorical conceptualization of IC. A many-facet Rasch analysis indicated this rating scale could separate examinees into three levels of ability, which was lower than the expected four levels. To explore this finding, raters were interviewed about their experiences of using this rating scale. The findings show the potential and limitations of developing contextualized IC ratings scales based on literature and possible avenues for future research. They also suggest a direct interaction between assessment and instructional practices of IC with implications for language teaching.
Article outline
  • 1.Introduction
  • 2.Literature review
  • 3.Material and methods
    • 3.1Participants
    • 3.2Materials
      • 3.2.1Videos
      • 3.2.2Rating scale
    • 3.3Procedures
  • 4.Results
    • 4.1Quantitative analysis
    • 4.2Qualitative analysis
  • 5.Discussion
  • 6.Conclusion
  • References
References (50)
References
Bachman, L. F. (2004). Fundamental considerations in language testing (7. impr). Oxford Univ. Press.Google Scholar logo with link to Google Scholar
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice. Oxford University Press.Google Scholar logo with link to Google Scholar
Bonk, W. J., & Ockey, G. J. (2003). A many-facet Rasch analysis of the second language group oral discussion task. Language Testing, 20(1), 89–110. Google Scholar logo with link to Google Scholar
Borger, L. (2019). Assessing interactional skills in a paired speaking test. Apples – Journal of Applied Language Studies, 13(1), 151–174. Google Scholar logo with link to Google Scholar
Canale, M. (1983). From communicative competence to communicative langauge pedagogy. In J. C. Richards & R. W. Schmidt (Eds.), Language and communication (pp. 2–28). Longman.Google Scholar logo with link to Google Scholar
Canale, M., & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1(1), 1–47. Google Scholar logo with link to Google Scholar
Creswell, J. W. (2010). Mapping th edeveloping landscape of mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp. 45–68). SAGE Publications. Google Scholar logo with link to Google Scholar
Dörnyei, Z. (2007). Research methods in applied linguistics: Quantitative, qualitative, and mixed methodologies. Oxford University Press.Google Scholar logo with link to Google Scholar
Ducasse, A. M., & Brown, A. (2009). Assessing paired orals: Raters’ orientation to interaction. Language Testing, 26(3), 423–443. Google Scholar logo with link to Google Scholar
Eckes, T. (2015). Introduction to Many-Facet Rasch Measurement: Analyzing and evaluating rater-mediated assessments. Peter Lang GmbH, Internationaler Verlag der Wissenschaften.Google Scholar logo with link to Google Scholar
Ezzy, D. (2002). Qualitative analysis: Practice and innovation. Routledge.Google Scholar logo with link to Google Scholar
Fulcher, G. (2010). Practical language testing. Hodder Education.Google Scholar logo with link to Google Scholar
Galaczi, E. D. (2014). Interactional Competence across Proficiency Levels: How do Learners Manage Interaction in Paired Speaking Tests? Applied Linguistics, 35(5), 553–574. Google Scholar logo with link to Google Scholar
Galaczi, E., & Taylor, L. (2018). Interactional Competence: Conceptualisations, Operationalisations, and Outstanding Questions. Language Assessment Quarterly, 15(3), 219–236. Google Scholar logo with link to Google Scholar
Galaczi, E., & Taylor, L. B. (2021). Measuring interactional competence. In P. Winke & T. Brunfaut (Eds.), The Routledge handbook of second language acquisition and language testing (pp. 338–348). Routledge.Google Scholar logo with link to Google Scholar
Hughes, R. (2007). Testing the visible: Literate biases in oral language testing. Journal of Applied Linguistics and Professional Practice, 1(3), 295–309. Google Scholar logo with link to Google Scholar
Hymes, D. (1972). On communicative competence. In J. B. Pride & J. Holmes (Eds.), Sociolinguistics: Selected readings (pp. 53–73). Penguin.Google Scholar logo with link to Google Scholar
Kim, H. (2018). What constitutes professional communication in aviation: Is language proficiency enough for testing purposes? Language Testing, 35(3), 403–426. Google Scholar logo with link to Google Scholar
Kramsch, C. (1986). From Language Proficiency to Interactional Competence. The Modern Language Journal, 70(4), 366–372. Google Scholar logo with link to Google Scholar
Lam, D. M. K. (2018). What counts as “responding”? Contingency on previous speaker contribution as a feature of interactional competence. Language Testing, 35(3), 377–401. Google Scholar logo with link to Google Scholar
(2021). Don’t Turn a Deaf Ear: A Case for Assessing Interactive Listening. Applied Linguistics, 42(4), 740–764. Google Scholar logo with link to Google Scholar
Lantolf, J. P. (Ed.). (2000). Sociocultural theory and second language learning. Oxford Univ. Press.Google Scholar logo with link to Google Scholar
Lazaraton, A. (1996). Interlocutor support in oral proficiency interviews: The case of CASE. Language Testing, 13(2), 151–172. Google Scholar logo with link to Google Scholar
Linacre, J. M. (1998). FACETS 3.17. Computer Program. MESA Press.Google Scholar logo with link to Google Scholar
(2021). Winsteps® Rasch measurement computer program (5.1.1) [Computer software]. [URL]
Mackey, A., & Gass, S. M. (2010). Second language research: Methodology and design (Repr). Routledge.Google Scholar logo with link to Google Scholar
May, L. (2011). Interactional Competence in a Paired Speaking Test: Features Salient to Raters. Language Assessment Quarterly, 8(2), 127–145. Google Scholar logo with link to Google Scholar
May, L., Nakatsuhara, F., Lam, D., & Galaczi, E. (2020). Developing tools for learning oriented assessment of interactional competence: Bridging theory and practice. Language Testing, 37(2), 165–188. Google Scholar logo with link to Google Scholar
McNamara, T. F., & Roever, C. (2006). Language testing: The social dimension. Blackwell Pub.Google Scholar logo with link to Google Scholar
Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and implementation (Fourth edition). Jossey-Bass.Google Scholar logo with link to Google Scholar
Nakatsuhara, F., May, L., Lam, D., & Galaczi, E. (2018). Learning oriented feedback in the development and assessment of interactional competence. Reserach Notes, 701, 1–68.Google Scholar logo with link to Google Scholar
Ockey, G. J. (2012). Item response theory. In G. Fulcher & F. Davidson (Eds.), The Routledge handbook of language testing (pp. 336–349).Google Scholar logo with link to Google Scholar
Ockey, G. J., & Chukharev-Hudilainen, E. (2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. Google Scholar logo with link to Google Scholar
Ockey, G. J., Koyama, D., Setoguchi, E., & Sun, A. (2015). The extent to which TOEFL iBT speaking scores are associated with performance on oral language tasks and oral ability components for Japanese university students. Language Testing, 32(1), 39–62. Google Scholar logo with link to Google Scholar
O’Sullivan, B. (2000). Exploring gender and oral proficiency interview performance. System, 28(3), 373–386. Google Scholar logo with link to Google Scholar
O’Sullivan, B., Weir, C. J., & Saville, N. (2002). Using observation checklists to validate speaking-test tasks. Language Testing, 19(1), 33–56. Google Scholar logo with link to Google Scholar
Pekarek Doehler, S., & Pochon-Berger, E. (2011). Developing “methods” for interaction: A cross-sectional study of disagreement sequences in French L2. In J. K. Hall, J. Hellermann, & S. P. Doehler (Eds.), L2 interactional competence and development (pp. 206–243). Multilingual Matters. Google Scholar logo with link to Google Scholar
(2015). The development of L2 interactional competence: Evidence from turn-taking organization, sequence organization, repair organization, an preference organization. In T. Cadierno & S. W. Eskildsen (Eds.), Usage-based perspectives on second language learning (pp. 233–268). De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Plough, I., Banerjee, J., & Iwashita, N. (2018). Interactional competence: Genie out of the bottle. Language Testing, 35(3), 427–445. Google Scholar logo with link to Google Scholar
Roever, C., & Ikeda, N. (2021). What scores from monologic speaking tests can(not) tell us about interactional competence. Language Testing, 026553222110033. Google Scholar logo with link to Google Scholar
Roever, C., & Kasper, G. (2018). Speaking in turns and sequences: Interactional competence as a target construct in testing speaking. Language Testing, 35(3), 331–355. Google Scholar logo with link to Google Scholar
Ross, S. (2018). Listener response as a facet of interactional competence. Language Testing, 35(3), 357–375. Google Scholar logo with link to Google Scholar
Sandlund, E., & Greer, T. (2020). How do raters understand rubrics for assessing L2 interactional engagement? A comparative study of CA-and non-CA-formulated performance descriptors. Papers in Language Testing and Assessment: An International Journal of the Association for Language Testing and Assessment of Australia and New Zealand, 9(1), 128–163.Google Scholar logo with link to Google Scholar
Sandlund, E., Sundqvist, P., & Nyroos, L. (2016). Testing L2 Talk: A Review of Empirical Studies on Second-Language Oral Proficiency Testing: Testing L2 Talk. Language and Linguistics Compass, 10(1), 14–29. Google Scholar logo with link to Google Scholar
Vo, S. (2021). Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study. TESOL Journal, 12(2). Google Scholar logo with link to Google Scholar
Wright, B. D. (1996). Localy dependency, correlations and prinicpal components. Rasch Measurement Transactions, 101, 509–511.Google Scholar logo with link to Google Scholar
Wu, S., & Ma, Z. (2020). How is Chinese reading affected by under-specification and over-specification? Evidence from self-paced reading experiments. Journal of Pragmatics, 1551, 213–233. Google Scholar logo with link to Google Scholar
Youn, S. J. (2015). Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225. Google Scholar logo with link to Google Scholar
Young, R. (2008). Language and interaction: An advanced resource book. Routledge.Google Scholar logo with link to Google Scholar
Cited by (2)

Cited by two other publications

Syquia, John & Paul Leeming
2024. Assessing interactional competence through group discussion: A mixed methods validation. Research Methods in Applied Linguistics 3:3 ► pp. 100144 ff. DOI logo
Vo, Sonca
2024. Exploring the construct of interactional competence in different types of oral communication assessment. Interaction Studies. Social Behaviour and Communication in Biological and Artificial Systems 25:1 ► pp. 1 ff. DOI logo

This list is based on CrossRef data as of 13 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue