In:Instructed Second Language Acquisition Research Methods
Edited by Laura Gurzynski-Weiss and YouJin Kim
[Research Methods in Applied Linguistics 3] 2022
► pp. 257–280
Chapter 11Listening
Exploring the underlying processes
Published online: 8 December 2022
https://doi.org/10.1075/rmal.3.11suv
https://doi.org/10.1075/rmal.3.11suv
Abstract
Out of the four language skills, listening is generally deemed to be the least understood and most under-researched (Aryadoust, Kumaran et al., 2020), partly due to its complex and ephemeral nature. Given that neither the process nor the product of instructed L2 listening can be directly observed (Brown & Abeywickrama, 2019), understanding L2 listening development and learners’ performance on listening tasks poses a number of challenges for language educators and researchers alike. Traditionally, listening has been measured indirectly by analyzing its products, such as responses to comprehension questions, whereas research on the processes underlying L2 listening has been lacking (Vandergrift, 2010). Aiming to encourage the use of process-oriented approaches to investigating L2 listening, this chapter summarizes key research trends that are of particular relevance for ISLA and discusses four main methods that can be utilized for data collection and analysis in process-oriented L2 listening studies. The chapter concludes with step-by-step guidelines for implementing eye tracking into L2 listening studies and some additional recommendations for researchers interested in embarking on this type of research.
Keywords: listening, technology, eye tracking, process-oriented approaches, methods
Article outline
- 1.What is listening and why is it important in ISLA?
- 2.What we know and what we need to know about listening research in ISLA
- 3.Data elicitation and interpretation options
- 3.1Survey research
- 3.2Verbal report methods
- 3.3Behavioral and neuroimaging methods
- 4.Step-by-step guidelines for implementing eye tracking in an L2 listening study
- Step 1.Select an appropriate eye-tracker
- Step 2.Design the experiment
- Step 3.Conduct a pilot study
- Step 4.Collect the data
- Step 5.Analyze the data and interpret the results
- 5.Advice to future listening researchers
- 6.Troubleshooting ISLA listening research
- 7.Conclusions
- 8.Further reading and additional resources
- 8.1Suggested books and book chapters
- 8.2Suggested journals and professional organizations
References
References (82)
Aryadoust, V. (2020). Dynamics of item reading and answer changing in two hearings in a computerized while-listening performance test: An eye-tracking study. Computer Assisted Language Learning, 33(5–6), 510–537.
Aryadoust, V., Kumaran, Y. I., & Ferdinand, S. (2020). Linguistics. In D. L. Worthington & G. D. Bodie (Eds.), The handbook of listening (pp. 139–161). John Wiley & Sons.
Aryadoust, V., Ng, L. Y., Foo, S., & Esposito, G. (2020). A neurocognitive investigation of test methods and gender effects in listening assessment. Computer Assisted Language Learning, 1–21.
Batty, A. O. (2015). A comparison of video- and audio-mediated listening tests with many-facet Rasch modeling and differential distractor functioning. Language Testing, 32(1), 3–20.
(2021). An eye-tracking study of attention to visual cues in L2 listening tests. Language Testing, 38(4), 511–535.
Becker, S. R. (2021). Metacognitive instruction in L2 French: An analysis of listening performance and automaticity. Foreign Language Annals, 54(1), 9–26.
Becker, S. R., & Sturm, J. L. (2017). Effects of audiovisual media on L2 listening comprehension: A preliminary study in French. CALICO Journal, 34(2), 147–177.
Bejar, I., Douglas, D., Jamieson, J., Nissan, S., & Turner, J. (2000). TOEFL 2000 listening framework: A working paper (Report No. RM-00-07). Educational Testing Service. [URL]
Berne, J. E. (2004). Listening comprehension strategies: A review of the literature. Foreign Language Annals, 37(4), 521–533.
Bloomfield, A., Wayland, S., Rhoades, E., Blodgett, A., Linck, J., & Ross, S. (2010). What makes listening difficult? Factors affecting second language listening comprehension (Technical Report No. E.3.1 TTO 81434). University of Maryland, Center for Advanced Study of Language. [URL].
(2019). Verbal reports in instructed SLA research: Opportunities, challenges, and limitations. In R. P. Leow (Ed.), The Routledge handbook of second language research in classroom learning (pp. 31–43). Routledge.
Brown, H. D., & Abeywickrama, P. (2019). Language assessment: Principles and classroom practices (3rd ed.). Pearson.
Brown, S. (2017). L2 listening. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning: Volume III (pp. 201–213). Routledge.
Brunfaut, T., & Rukthong, A. (2018). Integrative testing of listening. In J. I. Liontas (Ed.), The TESOL encyclopedia of English language teaching (pp. 1–7). John Wiley & Sons.
Çakmak, F., & Erçetin, G. (2018). Effects of gloss type on text recall and incidental vocabulary learning in mobile-assisted L2 listening. ReCALL, 30(1), 24–47.
Conklin, K., & Pellicer-Sánchez, A. (2016). Using eye-tracking in applied linguistics and second language research. Second Language Research, 32(3), 453–467.
Cross, J., & Vandergrift, L. (2015). Guidelines for designing and conducting L2 listening studies. ELT Journal, 69(1), 86–89.
(2018). Metacognitive listening strategies. In The TESOL Encyclopedia of English language teaching. Wiley.
Cubilo, J., & Winke, P. (2013). Redefining the L2 listening construct within an integrated writing task: Considering the impacts of visual-cue interpretation and note-taking. Language Assessment Quarterly, 10(4), 371–397.
Dahl, T. I., & Ludvigsen, S. (2014). How I see what you’re saying: The role of gestures in native and foreign language listening comprehension. Modern Language Journal, 98(3), 813–833.
Emerick, M. R. (2019). Explicit teaching and authenticity in L2 listening instruction: University language teachers’ beliefs. System, 80, 107–119.
Flowerdew, J., & Miller, L. (2010). Listening in a second language. In A. D. Wolvin (Ed.), Listening and human communication in the 21st century (pp. 158–177). Wiley-Blackwell.
Fung, D., & Macaro, E. (2021). Exploring the relationship between linguistic knowledge and strategy use in listening comprehension. Language Teaching Research, 25(5), 540–564.
Gass, S. M., & Mackey, A. (2016). Stimulated recall methodology in applied linguistics and L2 research (2nd ed.). Routledge.
Gilmore, A. (2007). Authentic materials and authenticity in foreign language learning. Language Teaching, 40(2), 97–118.
(2011). “I prefer not text:” Developing Japanese learners’ communicative competence with authentic materials. Language Learning, 61(3), 786–819.
Godfroid, A. (2019b). Investigating instructed second language acquisition using L2 learners’ eye-tracking data. In R. P. Leow (Ed.), The Routledge handbook of second language research in classroom learning (pp. 44–57). Routledge.
Goh, C. C. M., & Hu, G. (2014). Exploring the relationship between metacognitive awareness and listening performance with questionnaire data. Language Awareness, 23(3), 255–274.
Goh, C. C. M., & Taib, Y. (2006). Metacognitive instruction in listening for young learners. ELT Journal, 60(3), 222–232.
Graham, S. (2017). Research into practice: Listening strategies in an instructed classroom setting. Language Teaching, 50(1), 107–119.
Graham, S., Santos, D., & Vanderplank, R. (2010). Strategy clusters and sources of knowledge in French L2 listening comprehension. Innovation in Language Learning and Teaching, 4(1), 1–20.
(2011). Exploring the relationship between listening development and strategy use. Language Teaching Research, 15(4), 435–456.
Hayati, A., & Mohmedi, F. (2011). The effect of films with and without subtitles on listening comprehension of EFL learners. British Journal of Educational Technology, 42(1), 181–192.
Herron, C., Morris, M., Secules, T., & Curtis, L. (1995). A comparison study of the effects of video-based versus text-based instruction in the foreign language classroom. The French Review, 68(5), 775–795. [URL]
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Van de Weijer, J. (2011). Eye tracking: A comprehensive guide to methods and measures. Oxford University Press.
Holzknecht, F., Eberharter, K., Kremmel, B., Zehentner, M., McCray, G., Konrad, E., & Spöttl, C. (2017). Looking into listening: Using eye-tracking to establish the cognitive validity of the Aptis Listening Test. ARAGs Research Reports Online (Report AR-G/2017/3). [URL]
Huang, A. (2020). The dialogical nature of language use in interactive listening: Revisiting meaning in context. Language Awareness, 29(1), 21–40.
Huang, H. T. D., Hung, S. T. A., & Plakans, L. (2018). Topical knowledge in L2 speaking assessment: Comparing independent and integrated speaking test tasks. Language Testing, 35(1), 27–49.
Inceçay, V., & Koçoğlu, Z. (2017). Investigating the effects of multimedia input modality on L2 listening skills of Turkish EFL learners. Education and Information Technologies, 22(3), 901–916.
International Listening Association (ILA). (1995, April). A ILA definition of listening. The Listening Post, 53, 1, 4–5.
Just, M. A., & Carpenter, P. A. (1980). A theory of reading: From eye fixations to comprehension. Psychological Review, 87(4), 329–354.
Leow, R. P. (2019). From SLA > ISLA > ILL: A curricular/pedagogical perspective. In R. P. Leow (Ed.), The Routledge handbook of second language research in classroom learning (pp. 483–491). Routledge.
Littlewood, W. (2011). Communicative language teaching: An expanding concept for a changing world. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning: Volume II (pp. 541–557). Routledge.
Loewen, S., & Sato, M. (2017). Instructed second language acquisition (ISLA): An overview. In S. Loewen & M. Sato (Eds.), The Routledge handbook of instructed second language acquisition (pp. 1–12). Routledge.
Londe, Z. C. (2009). The effects of video media in English as a second language listening comprehension tests. Issues in Applied Linguistics, 17(1), 41–50. [URL].
Macaro, E., Graham, S., & Vanderplank, R. (2007). A review of listening strategies: Focus on sources of knowledge and on success. In A. D. Cohen & E. Macaro (Eds.), Language learner strategies: Thirty years of research and practice (pp. 165–185). Oxford University Press.
Mayer, R. E. (2005). Introduction to multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 1–16). Cambridge University Press.
Mohsen, M. A. (2016). The use of help options in multimedia listening environments to aid language learning: A review. British Journal of Educational Technology, 47(6), 1232–1242.
Montero Perez, M., Van Den Noortgate, W., & Desmet, P. (2013). Captioned video for L2 listening and vocabulary learning: A meta-analysis. System, 41(3), 720–739.
Nunan, D. (2002). Listening in language learning. In J. C. Richards & W. A. Renandya (Eds.), Methodology in language teaching: An anthology of current practice (pp. 238–241). Cambridge University Press.
Ockey, G. J. (2007). Construct implications of including still image or video in computer-based listening tests. Language Testing, 24(4), 517–537.
Pinner, R. (2014). The authenticity continuum: Towards a definition incorporating international voices. English Today, 30(4), 22–27.
Pusey, K., & Lenz, K. (2014). Investigating the interaction of visual input, working memory, and listening comprehension. Language Education in Asia, 5(1), 66–80.
Révész, A., & Brunfaut, T. (2013). Text characteristics of task input and difficulty in second language listening comprehension. Studies in Second Language Acquisition, 35(1), 31–65.
Rubin, J. (1994). A review of second language listening comprehension research. The Modern Language Journal, 78(2), 199–221.
Rukthong, A. (2021). MC listening questions vs. integrated listening-to-summarize tasks: What listening abilities do they assess? System, 97, 1–12.
Rukthong, A., & Brunfaut, T. (2020). Is anybody listening? The nature of second language listening in integrated listening-to-summarize tasks. Language Testing, 37(1), 31–53.
Seo, K. (2002). Research note: The effect of visuals on listening comprehension: A study of Japanese learners’ listening strategies. International Journal of Listening, 16(1), 57–81.
Suvorov, R. (2009). Context visuals in L2 listening tests: The effects of photographs and video vs. audio-only format. In C. A. Chapelle, H. G. Jun & I. Katz (Eds.), Developing and evaluating language learning materials (pp. 53–68). Iowa State University.
(2015). The use of eye tracking in research on video-based second language (L2) listening assessment: A comparison of context videos and content videos. Language Testing, 32(4), 463–483.
(2018). Investigating test-taking strategies during the completion of computer-delivered items from Michigan English Test (MET): Evidence from eye tracking and cued retrospective reporting. Cambridge Michigan Language Assessment (CaMLA) Working Papers 2018–02. Retrieved on 2 June 2022 from [URL]
(2019). Multimedia in the teaching and assessment of listening. In M. A. Peters & R. Heraud (Eds.), Encyclopedia of Educational Innovation (pp. 1–6). Springer.
Suvorov, R., & Camp, A. (2017). An exploratory study of test-takers’ cognitive states using electroencephalography (EEG) and retrospective interviews in the context of L2 testing. In J. Colpaert, A. Aerts, R. Kern, & M. Kaiser (Eds.), Proceedings of the XVIIIth International CALL Research Conference: CALL in context (pp. 698–704). University of Antwerp. Retrieved on 2 June 2022 from [URL]
Suvorov, R., & He, S. (2022). Visuals in the assessment and testing of second language listening: A methodological synthesis. International Journal of Listening, 36(2), 80–99.
van Compernolle, R. A., & McGregor, J. (2016). Introducing authenticity, language and interaction in second language contexts. In R. A. van Compernolle & J. McGregor (Eds.), Authenticity, language and interaction in second language contexts (pp. 1–9). Multilingual Matters.
Vandergrift, L. (2007). Recent developments in second and foreign language listening comprehension research. Language Teaching, 40(3), 191–210.
(2010). Researching listening. In B. Paltridge & A. Phakiti (Eds.), Continuum companion to research methods in applied linguistics (pp. 160–173). Continuum.
Vandergrift, L., & Goh, C. M. (2012). Teaching and learning second language listening: Metacognition in action. Routledge.
Vandergrift, L., Goh, C. C. M., Mareschal, C. J., & Tafaghodtari, M. H. (2006). The metacognitive awareness listening questionnaire (MALQ): Development and validation. Language Learning, 56(3), 431–462.
Vandergrift, L., & Tafaghodtari, M. H. (2010). Teaching L2 learners how to listen does make a difference: An empirical study. Language Learning, 60(2), 470–497.
Wagner, E. (2010). The effect of the use of video texts on ESL listening test-taker performance. Language Testing, 27(4), 493–513.
(2015). Survey research. In B. Paltridge & A. Phakiti (Eds.), Research methods in applied linguistics: A practical resource (pp. 83–99). Bloomsbury.
Wagner, E., Liao, Y.-F., & Wagner, S. (2021). Authenticated spoken texts for L2 listening tests. Language Assessment Quarterly, 18(3), 205–227.
Weyers, J. R. (1999). The effect of authentic video on communicative competence. Modern Language Journal, 83(3), 339–349.
Winke, P., & Lim, H. (2014). The effects of testwiseness and test-taking anxiety on L2 listening test performance: A visual (eye-tracking) and attentional investigation. IELTS Research Reports Online Series, 3, 1–30. Retrieved on 2 June 2022 from [URL]