Cover not available

Article published In: Interaction Studies
Vol. 25:1 (2024) ► pp.1–35

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Exploring the construct of interactional competence in different types of oral communication assessment

ORCiD logo with link | University of Foreign Language Studies–The University of Danang | The University of UtahSalt Lake City
Published online: 7 June 2024
DOI logo with link
https://doi.org/10.1075/is.00022.vo
Abstract
Research on interaction in speaking assessment suggests that both verbal and nonverbal interaction are integral parts of the construct of interactional competence (Galaczi, E. D., & Taylor, L. (2018). IC: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15(3), 219–236. Google Scholar logo with link to Google Scholar; Plough, I., Banerjee, J., & Iwashita, N. (2018). Interactional competence: Genie out of the bottle. Language Testing, 35(3), 427–445. Google Scholar logo with link to Google Scholar; Young, R. (2011). Interactional competence in language learning, teaching, and testing. In E. Hinkel (Ed.), Handbook of Research in Second Language Teaching and Learning (pp. 426–443). London, UK & New York, NY: Routledge.Google Scholar logo with link to Google Scholar). However, little has been done to investigate which features significantly contribute to interactional competence scores. This study, therefore, examined which interaction features that raters noticed in individual scripted interview and paired discussion tasks to gain an insight into the interactional competence construct, providing validity evidence for an inclusion of interactional competence in speaking assessment. Sixty-eight student performances were rated based on interaction rating scales. Exploratory factor analysis revealed four factors: nonverbal communication, topic management, interactional management, and interactive listening. Logistic regressions showed that while raters attended to more topic management features in the individual scripted interview task, they noticed more interactional management features in the paired discussion task. Simple regressions showed that nonverbal communication and topic management features predicted interactional competence scores in the individual scripted interview task, whereas nonverbal communication, topic management, interactional management, and interactive listening features were predictors of scores in the paired discussion task. The findings suggest that both nonverbal and verbal interaction features are important in the interactional competence construct with the paired task providing test-takers with more opportunities to demonstrate their interactional ability.
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Raters and test-takers’ perceptions toward IC
    • 2.2IC in different types of oral communication assessment
    • 2.3Methodology used in IC research
  • 3.Methods
    • 3.1Participants
    • 3.2Materials
      • 3.2.1Test-takers’ video-recorded performances
      • 3.2.2Interactional competence rating scale (ICS)
      • 3.2.3Interaction ability scale (IAS)
    • 3.3Procedures
      • 3.3.1Rater training
      • 3.3.2Rating
      • 3.3.3Data analysis
        • 3.3.3.1Data screening for RQ1
        • 3.3.3.2Data screening for RQ2
          • 3.3.3.2.1Data screening for the individual scripted interview task
          • 3.3.3.2.2Data screening for the paired discussion task
  • 4.Results
    • 4.1The extent to which the individual scripted interview task shared interaction features with the paired discussion task
      • 4.1.1Exploratory factor analysis
      • 4.1.2Logistic regression
        • 4.1.2.1Topic management (TM)
        • 4.1.2.2Nonverbal communication (NC)
        • 4.1.2.3Interactional management (IM)
        • 4.1.2.4Interactive listening (IL)
    • 4.2The extent to which the nonverbal and the verbal interaction features contribute to variances in the ICS scores in the two tasks
      • 4.2.1Individual scripted interview task
      • 4.2.2Paired discussion task
  • 5.Discussion
    • 5.1The extent to which the individual scripted interview task shared interaction features with the paired discussion task
    • 5.2The extent to which the interaction features contributed to variance in the ICS scores across the two tasks
  • 6.Limitations
  • 7.Conclusion
  • Acknowledgements
  • References
References (43)
References
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 431, 561–573. Google Scholar logo with link to Google Scholar
Brooks, L. (2009). Interacting in pairs in a test of oral proficiency: Co-constructing a better performance. Language Testing, 26(3), 342–366. Google Scholar logo with link to Google Scholar
Chapelle, C. A., Enright, M. K., & Jamieson, J. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign Language. London, English: Routledge.Google Scholar logo with link to Google Scholar
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing. Cambridge University Press.Google Scholar logo with link to Google Scholar
De Winter, J. C., Dodou, D., & Wieringa, P. A. (2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44(2), 147–181. Google Scholar logo with link to Google Scholar
Ducasse, A. M. (2014). Interaction in paired oral proficiency assessment in Spanish. Frankfurt/Main, Germany: Peter Lang.Google Scholar logo with link to Google Scholar
Ducasse, A. M., & Brown, A. (2009). Assessing paired orals: Raters’ orientation to interaction. Language Testing, 261, 423–443. Google Scholar logo with link to Google Scholar
Eckes, T. (2015). Introduction to Many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments. Peter Lang Pub. Inc.Google Scholar logo with link to Google Scholar
EPT Oral Communication Test. Retrieved on February 18, 2020 from [URL]
Fabrigar, L. R. (2012). Exploratory factor analysis: Understanding statistics. Oxford.Google Scholar logo with link to Google Scholar
Galaczi, E. D. (2014). IC across proficiency levels: How do learners manage interaction in paired speaking tests? Applied Linguistics, 35(5), 553–574. Google Scholar logo with link to Google Scholar
Galaczi, E. D., & Taylor, L. (2018). IC: Conceptualisations, operationalisations, and outstanding questions. Language Assessment Quarterly, 15(3), 219–236. Google Scholar logo with link to Google Scholar
He, L., & Dai, Y. (2006). A corpus-based investigation into the validity of the CET–SET group discussion. Language Testing, 231, 370–401. Google Scholar logo with link to Google Scholar
Heinze, G., & Schemper, M. (2002). A solution to the problem of separation in logistic regression. Statistics in Medicine, 21(16), 2409–2419. Google Scholar logo with link to Google Scholar
Kline, R. (2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford.Google Scholar logo with link to Google Scholar
Kramsch, C. (1986). From language proficiency to IC. Modern Language Journal, 70(4), 366–372. Google Scholar logo with link to Google Scholar
Lam, D. M. K. (2018). What counts as ‘responding’? Contingency on previous speaker contribution as a feature of IC. Language Testing, 35(3), 377–401. Google Scholar logo with link to Google Scholar
Leech, N. L., Barrett, K. C., & Morgan, G. A. (2011). IBM SPSS for intermediate statistics: Use and interpretation (4th ed.). New York, NY: Taylor & Francis.Google Scholar logo with link to Google Scholar
Linacre, J. M. (2014). Facets (3.71.4) [Computer software]. [URL]. [URL]
(1994). Many-facet Rasch measurement (2nd ed.). MESA Press.Google Scholar logo with link to Google Scholar
Matsumoto, D. (2006). Culture and nonverbal behavior. In V. L. Manusov & M. L. Patterson (Eds.), The SAGE handbook of nonverbal communication (pp. 219–235). Sage Publications, Inc.. Google Scholar logo with link to Google Scholar
May, L. (2011). Interactional competence in a paired speaking test: Features salient to raters. Language Assessment Quarterly, 8(2),127–145. Google Scholar logo with link to Google Scholar
McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 921, 350–371. Google Scholar logo with link to Google Scholar
Ockey, G. J. (2014). The potential of the L2 group oral to elicit discourse with a mutual contingency pattern and afford equal speaking rights in an ESP context. English for Specific Purposes, 351, 17–29. Google Scholar logo with link to Google Scholar
(2013). Exploratory factor analysis and structural equation modeling. In A. J. Kunnan (Ed.), The companion to language assessment. Vol. III: Evaluation, Methodolody, and Interdisciplinary Themes (pp. 1224–1244, Part 101, Chapter 73). Malden, MA: John Wiley & Sons. Google Scholar logo with link to Google Scholar
Ockey, G. J., & Chukharev-Hudilainen, E. (2021). Human versus computer partner in the paired oral discussion test. Applied Linguistics, 42(5), 924–944. Google Scholar logo with link to Google Scholar
Ockey, G. J., & Li, Z. (2015). New and not so new methods for assessing oral communication. Language Value, 71, 1–21. Google Scholar logo with link to Google Scholar
Ockey, G. J., & Wagner, E. (2018). Assessing L2 listening: Moving towards authenticity, vol. 50, John Benjamins, Philadelphia, PA. Google Scholar logo with link to Google Scholar
O’Sullivan, B., Weir, C. J., & Saville, N. (2002). Using observation checklists to validate speaking test tasks. Language Testing, 191, 33–56. Google Scholar logo with link to Google Scholar
Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction. Belmont, Ca.: Wadsworth.Google Scholar logo with link to Google Scholar
Plough, I., Banerjee, J., & Iwashita, N. (2018). Interactional competence: Genie out of the bottle. Language Testing, 35(3), 427–445. Google Scholar logo with link to Google Scholar
Roever, C., & Kasper, G. (2018). Speaking in turns and sequences: IC as a target construct in testing speaking. Language Testing, 35(2), 331–355. Google Scholar logo with link to Google Scholar
Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. 4th Edition, Allyn and Bacon, Boston.Google Scholar logo with link to Google Scholar
Van Lier, L. (1989). Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 231, 489–508. Google Scholar logo with link to Google Scholar
Van Moere, A. (2007). Group oral test: How does task affect candidate performance and test score? (Unpublished Ph.D. Dissertation). The University of Lancaster.
Vo, S. (2021). Evaluating interactional competence in interview and paired discussion tasks: A rater cognition study. TESOL Journal, 12(2), 1–18. Google Scholar logo with link to Google Scholar
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370. [URL]
Youn, S. J. (2020). Interactional features of L2 pragmatic interaction in role-play speaking assessment. TESOL Quarterly, 54(1), 201–233. Google Scholar logo with link to Google Scholar
(2015). Validity argument for assessing L2 pragmatics in interaction using mixed methods. Language Testing, 32(2), 199–225. Google Scholar logo with link to Google Scholar
Young, R. (2011). Interactional competence in language learning, teaching, and testing. In E. Hinkel (Ed.), Handbook of Research in Second Language Teaching and Learning (pp. 426–443). London, UK & New York, NY: Routledge.Google Scholar logo with link to Google Scholar
Yuki, M., Maddux, W. W., & Masuda, T. (2007). Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology, 43(2), 303–311. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue