Article published In: Concentric
Vol. 46:1 (2020) ► pp.1–20
Hearing faces and seeing accents?
The factors behind the multisensory speech perception of native Mandarin speakers
Published online: 1 May 2020
https://doi.org/10.1075/consl.00011.yu
https://doi.org/10.1075/consl.00011.yu
Abstract
In this paper, the McGurk effect displayed by native Mandarin Speakers is examined in the light of reaction time
(RT) and response types. Two within-subject factors, FACE and ACCENT, and one between-subject factor, English Proficiency, were
incorporated in the experiment. The results showed that FACE and ACCENT, but not English Proficiency, had effects on the participants’ RT and response types.
When a foreign ACCENT was dubbed onto a familiar FACE, the RT was the longest, and the McGurk effect was most likely to be found. Other kinds of McGurk stimuli composition did not receive different RT but induced different response types. When the FACE was
foreign, regardless of the ACCENT, the participants tended to respond with perceptive illusion. The author concluded that the
expectations of the perceiver influenced the use of multisensory integration and thus the longer RT and the appearance of the McGurk effect.
Article outline
- 1.The McGurk effect (multisensory integration)
- 1.1MSI and the McGurk effect of the second language speech perception
- 1.2Research purposes and questions
- 2.Methodology
- 2.1Participants
- 2.2Materials
- 2.2.1Raw materials preparation
- 2.2.2Accent analysis
- 2.2.3Materials normalization
- 2.3Procedure
- 2.4Data analysis
- 2.4.1Reaction time (RT)
- 2.4.2Response type
- 3.Results
- 3.1Reaction time
- 3.2Response types
- 4.Discussion
- 5.Limitations
- 6.Conclusion
- Acknowledgements
- Notes
References
References (19)
Beauchamp, Michael S. 2017. Beauchamp: McGurkStimuli. Retrieved February 4, 2019, from [URL]
Bovo, Roberto, Andrea Ciorba, Silvano Prosser, and Alessandro Martini. 2009. The McGurk phenomenon in Italian listeners. Acta Otorhinolaryngologica Italica 29.4:203–208.
Burnham, Denis, and Susanna Lau. 1998. The effect of tonal information on auditory reliance in the McGurk effect. Paper presented at the AVSP’98 International Conference on Auditory-Visual Speech Processing, University of New South Wales, Sydney.
Chen, Yuchun, and Valérie Hazan. 2007. Developmental factor in auditory-visual speech perception-the McGurk effect in Mandarin-Chinese and English Speakers. Paper presented at the AVSP2007 International Conference on Auditory-Visual Speech Processing, Kasteel Groenendaal, Hilvarenbeek, Netherlands.
Fuster-Duran, Angela. 1996. Perception of conflicting audio-visual speech: An examination across Spanish and German. Speechreading by Humans and Machines: Models, Systems, and Applications, ed. by David G. Stork and Marcus E. Hennecke, 135–143. Berlin: Springer.
Hirata, Yukari, and Spencer D. Kelly. 2010. Effects of lips and hands on auditory learning of second-language speech sounds. Journal of Speech, Language, and Hearing Research 53.2:298–310.
Kotz, Sonja A., and Kerrie Elston-Güttler. 2004. The role of proficiency on processing categorical and associative information in the L2 as revealed by reaction times and event-related brain potentials. Journal of Neurolinguistics 17.2–3:215–235.
Magnotti, John F., Debshila Basu Mallick, Guo Feng, Bin Zhou, Wen Zhou, and Michael S. Beauchamp. 2015. Similar frequency of the McGurk effect in large samples of native Mandarin Chinese and American English speakers. Experimental Brain Research 233.9:2581–2586.
Munro, Murray J., and Tracey M. Derwing. 1995. Processing time, accent, and comprehensibility in the perception of native and foreign-accented speech. Language and Speech 38.3:289–306.
Nath, Audrey R., and Michael S. Beauchamp. 2012. A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. NeuroImage 59.1:781–787.
Navarra, Jordi, and Salvador Soto-Faraco. 2007. Hearing lips in a second language: Visual articulatory information enables the perception of second language sounds. Psychological Research 71.1:4–12.
Marian, Viorica. 2009. Audio-visual integration during bilingual language processing. The Bilingual Mental Lexicon: Interdisciplinary Approaches, ed. by Aneta Pavlenko, 52–78. Bristol, Buffalo & Toronto: Multilingual Matters.
Sams, Mikko, Petri Manninen, Veikko Surakka, Pia Helin, and Riitta Kättö. 1998. McGurk effect in Finnish syllables, isolated words, and words in sentences: Effects of word meaning and sentence context. Speech Communication 26.1–2: 75–87.
Sekiyama, Kaoru. 1997. Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects. Perception & Psychophysics 59.1:73–80.
Sekiyama, Kaoru, and Yoh’ichi Tohkura. 1991. McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. The Journal of the Acoustical Society of America 90.4:1797–1805.
Sueyoshi, Ayano, and Debra M. Hardison. 2005. The role of gestures and facial cues in second language listening comprehension. Language Learning 55.4:661–699.
Tona, Risa, Yasushi Naito, Saburo Moroto, Rinko Yamamoto, Keizo Fujiwara, Hiroshi Yamazaki, Shogo Shinohara, and Masahiro Kikuchi. 2015. Audio-visual integration during speech perception in prelingually deafened Japanese children revealed by the McGurk effect. International Journal of Pediatric Otorhinolaryngology 79.12:2072–2078.