Article published In: Language, Interaction and Acquisition
Vol. 14:2 (2023) ► pp.218–246
Lending a hand to speech
Gestures help fluency and increase pitch in second language speakers
Published online: 22 March 2024
https://doi.org/10.1075/lia.22023.cav
https://doi.org/10.1075/lia.22023.cav
Abstract
This study examines the effect of gestures on second language (L2) pitch measures and fluency in storytelling
tasks. It is well documented that L2 learners commonly use gestures in conversations and storytelling. Research has shown that
gestures benefit L2 word learning and recall, but it is still unclear to what extent gestures affect speech fluency and pitch
range. We compared a series of speech fluency and pitch measures across two storytelling tasks: task repetition and explicit
instruction on the use of gestures. By analysing the use of gestures in these tasks, we provide insights on how gestures can help
L2 fluency and intonation.
Résumé
Cette étude examine l’effet des gestes sur le registre tonal et la fluidité en L2 dans des tâches
narratives. Il est bien connu que les apprenants d’une L2 utilisent couramment des gestes dans les conversations et les récits. La
recherche a montré que les gestes favorisent l’apprentissage et la mémorisation des mots en L2, mais on ne sait toujours pas dans
quelle mesure ils affectent la fluidité et la hauteur tonale en L2. Nous avons comparé la fluidité de la parole et le registre
tonal dans deux types différents de narration : la répétition et l’instruction explicite sur l’utilisation des gestes. En
examinant l’impact des gestes sur ces deux tâches, nous donnons un aperçu de la façon dont les gestes peuvent contribuer à
améliorer la fluidité et le registre tonal en aider la L
Mots-clés : fluidité de la parole, registre tonal, gestes manuels, acquisition L2
Article outline
- 1.Introduction
- 1.1Online planning, gestures, and prosody in L2
- 1.1.1L2 speech planning
- 1.1.2Gestures and L2
- 1.1.3Pitch range in L2
- 1.2Task repetition and explicit instruction in L2
- 1.3Motivation for this study and research questions
- 1.1Online planning, gestures, and prosody in L2
- 2.Method
- 2.1Participants
- 2.2Materials
- 2.3Procedure
- 2.3.1First trial
- 2.3.2Second trial
- 2.4Acoustic analyses
- 2.4.1Speech transcription and annotation
- 2.4.2Gesture annotation
- 2.4.3Gesture typology annotation
- 2.4.4Gesture rate
- 2.5Statistical analysis
- 3.Results
- 3.1Vocal parameters
- 3.2Fluency and prosody measures
- 3.3Gesture rate
- 4.Discussion
- Notes
References
References (124)
Ahmadian, M. J., & Tavakoli, M. (2011). The
effects of simultaneous use of careful online planning and task repetition on accuracy, fluency, and complexity of EFL
learners’ oral production. Language Teaching
Research, 151, 35–59.
Alibali, M. W., Kita, S., & Young, A. J. (2000). Gesture
and the process of speech production: We think, therefore we gesture. Language and Cognitive
Processes, 15(6), 593–613.
Alibali, M. W., Heath, D. C., & Myers, H. J. (2001). Effects
of visibility between speaker and listener on gesture production: Some gestures are meant to be
seen. Journal of Memory and
Language, 441, 169–188.
Andreeva, B., & Dimitrova, S. (2022). Prosodic
characteristics of Bulgarian-accented German: Prosodic units and prenuclear pitch
accents. Languages, 71, 1–18.
Andreeva, B., Demenko, G., Wolska, M., Möbius, B., Zimmerer, F., Jügler, J., & Trouvain, J. (2014). Comparison
of pitch range and pitch variation in Slavic and German
languages. In N. Campbell, D. Gibbon, & D. Hirst (Eds.), Proceedings
of the 7th international conference on speech
prosody (pp. 776–780). Retrieved
from [URL]
Arevart, S., & Nation, P. (1993). Adjusting
fluency and grammar through repetition. In J. Oller (Ed.), Methods
that work: Ideas for language
teachers (pp. 298–308). Heinle & Heinle.
Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random
effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and
Language, 681, 255–278.
Blomgren, M., Chen, Y., Ng, M. L., & Gilbert, H. R. (1998). Acoustic,
aerodynamic, physiologic, and perceptual properties of modal and vocal fry registers. Journal
of Acoustic Society of
America, 103(5), 2649–2658.
Boersma, P., & Weenink, D. (2018). Praat:
Doing phonetics by computer [Computer program]. Version
6.0.37. Retrieved on March 14
2018 from [URL]
Bosker, H., Pinget, A., Quené, H., Sanders, T., & de Jong, N. (2013). What
makes speech sound fluent? The contributions of pauses, speed and repairs. Language
Testing, 301, 159–175.
Bruin, J. (2014). R
library: Contrast coding systems for categorical variables. Retrieved
from [URL]
Busà, M. G. (2007). New
perspectives in teaching pronunciation. In A. Baldry, M. Pavesi, C. Taylor Torsello, & C. Taylor (Eds.), From
DIDACTAS to eCoLingua: An ongoing research project on translation and corpus
linguistics (pp. 171–188). EUT.
Busà, M. G., & Urbani, M. (2011). A
cross linguistic analysis of pitch range in English L1 and
L2. In W. S. Lee & E. Zee (Eds.), Proceedings
of 17th International Congress of Phonetic
Sciences (pp. 380–383). Retrieved
from [URL]
Butterworth, B. (1975). Hesitation
and semantic planning in speech. Journal of Psycholinguistic
Research, 41, 75–87.
Bygate, M. (2001). Effects
of task repetition on the structure and control of oral
language. In M. Bygate, P. Skehan, & M. Swain (Eds.), Researching
pedagogic
tasks (pp. 23–48). Harlow Longman.
Calbris, G. (2008). From left to right…: Coverbal gestures and their symbolic use of space. In A. Cienki & C. Müller (Eds.), Metaphor and gesture (pp. 27–53). John Benjamins.
Cavicchio, F., & Poesio, M. (2009). Multimodal Corpora Annotation: Validation Methods to Assess Coding Scheme Reliability. In M. Kipp, J.-C. Martin, P. Paggio, and D. Heylen (Eds.), Multimodal Corpora LNAI 5509, Berlin: Springer-Verlag, 109–121.
Chu, M., Meyer, A., Foulkes, L., & Kita, S. (2014). Individual
differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and
empathy. Journal of Experimental Psychology:
General, 1431, 694–709.
Cravotta, A., Prieto, P., & Busà, M. G. (2021). Exploring
the effects of encouraging the use of gestures on speech. Speech
Communication, 1351, 25–36.
de Bot, K. (1992). A
bilingual production model: Levelt’s speaking model adapted. Applied
Linguistics, 131, 1–24.
De Jong, N. H., Steinel, M. P., Florijn, A. F., Schoonen, R., & Hulstijn, J. H. (2012). Linguistic
skills and speaking fluency in a L2. Applied
Psycholinguistics, 331, 1–24.
Doughty, C. J. (2005). Instructed
SLA: Constraints, compensation, and enhancement. In C. J. Doughty, & M. H. Long (Eds.), The
handbook of second language
acquisition (pp. 1–39). John Wiley & Sons.
Dörnyei, Z., & Kormos, J. (1998). Problem-solving
mechanisms in L2 communication: a psycholinguistic perspective. Studies in Second Language
Acquisition, 20(3), 349–385.
Ellis, R. (2005). Measuring
implicit and explicit knowledge of a second language: a psychometric Study. Studies in Second
Language
Acquisition, 271, 141–172.
Esteve-Gibert, N., & Prieto, P. (2013). Prosodic
structure shapes the temporal realisation of intonation and manual gesture movements. Journal
of Speech, Language and Hearing
Research, 56(3), 850–864.
Ferrero, F., Magno Caldognetto, E., & Cosi, P. (1996). Sui
piani formantici acustici e uditivi delle vocali di uomo, donna e
bambino. In A. Perretti & P. Simonetti (Eds.), Atti
del XXIV Convegno Nazionale
dell’A.I.A. (pp. 169–178). Arti Grafiche Padovane.
Gao, M. (2021). Comparison
of pitch variation and pitch range in L1 and L2 Mandarin Chinese. Poster presented at
the 1st International Conference on Tone and Intonation
(TAI). Retrieved from [URL].
Gillespie, M., James, A. N., Federmeier, K. D., & Watson, D. G. (2014). Verbal
working memory predicts co-speech gesture: Evidence from individual
differences. Cognition, 1321, 174–180.
Gluhareva, D., & Prieto, P. (2017). Training
with rhythmic beat gestures favors L2 pronunciation in discourse-demanding situations. Language
Teaching
Research, 21(5), 609–631.
Graziano, M., Nicoladis, E., & Marentette, P. (2020). How
referential gestures align with speech: Evidence from monolingual and bilingual
speakers. Language
Learning, 701, 266–304.
Goodwin, M., & Goodwin, C. (1986). Gesture and coparticipation in the activity of searching for a word. Semiotica, 62(1–2), 51–76.
Gósy, M., Gyarmathy, D., & Beke, A. (2017). Phonetic
analysis of filled pauses based on a Hungarian-English learner corpus. International Journal of
Learner Corpus
Research, 3(2), 149–174.
Götz, S. (2013). Fluency
in native and non-native English speech. John Benjamins.
Gullberg, M. (1998). Gesture
as a communication strategy in second language discourse: A study of learners of French and
Swedish. Lund University Press.
Hagoort, P., & Indefrey, P. (2014). The
neurobiology of language beyond single words. Annual Review of
Neuroscience, 371, 347–362.
Herment, S., &, Tortel, A. (2022). The
intonation contour of non-finality revisited Implications for EFL
teaching. In A. Kirkova-Naskova, A. Henderson, & J. Fouz-González (Eds.), English
pronunciation instruction: Research-based
insights (pp. 176–195). John Benjamins.
Hincks, R. (2005). Measuring liveliness in presentation speech. In Proceedings of Interspeech
2005 (pp. 765–768). Retrieved
from [URL].
Hirata, Y., Kelly, S., Huang, J., & Manansala, M. (2014). Effect
of hand gestures on auditory learning of second-language vowel length contrasts. Journal of
Speech Language and Hearing
Research, 571, 2090–2101.
Hoetjes, M., & van Maastricht, L. (2020). Using
gesture to facilitate L2 phoneme acquisition: The importance of gesture and phoneme
complexity. Frontiers in
Psychology, 111, 575032.
Hoetjes, M., Krahmer, E., & Swerts, M. (2014). Does our speech change when we cannot gesture? Speech Communication, 571. 257–267.
Holler, J., & Stevens, R. (2007). The
effect of common ground on how speakers use gesture and speech to represent size
information. Journal of Language and Social
Psychology, 26(1), 4–27.
Housen, A., & Pierrard, M. (2006). Investigating
instructed second language acquisition. In A. Housen, & M. Pierrard (Eds.), Investigations
in instructed second language
acquisition (pp. 12–27). Mouton de Gruyter.
Indefrey, P. (2011). The
spatial and temporal signatures of word production components: A critical update. Frontiers in
Psychology, 2(255), 1–16.
Kearney, E., & Guenther, F. H. (2019). Articulating:
The neural mechanisms of speech production. Language, Cognition and
Neuroscience, 34(9), 1214–1229.
Keating, P., & Kuo, G. (2012). Comparison
of speaking fundamental frequency in English and Mandarin. The Journal of the Acoustical
Society of
America, 132(2), 1050–1060.
Kelly, S. D., McDevitt, T., & Esch, M. (2009). Brief
training with co-speech gesture lends a hand to word learning in a foreign language. Language
and Cognitive
Processes, 24(2), 313–334.
Kita, S. (2000). Hito wa naze jesuchaa o suru noka [why do people
gesture?]. Ninchikagaku [Cognitive
Studies], 7(1), 9–21.
Kita, S., & Özyürek, A. (2003). What
does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface
representation of spatial thinking and speaking. Journal of Memory and
Language, 48(1), 16–32.
(2011). Speech
production and the cognition hypothesis. In P. Robinson (Ed.), L2
task complexity: Researching the cognition hypothesis of language learning and
performance (pp. 39–60). John Benjamins.
Krauss, R. M. (1998). Why
do we gesture when we speak? Current Directions in Psychological
Science, 71, 54–60.
Krauss, R. M., Chen, Y., & Gottesman, R. F. (2000). Lexical
gestures and lexical access: A process model. In D. McNeill (Ed.), Language
and
gesture (pp. 261–283). Cambridge University Press.
Kushch, O., & Prieto, P. (2016). The
effects of pitch accentuation and beat gestures on information recall in contrastive
discourse. In J. Barnes, A. Brugos, S. Shattuck-Hufnagel, & N. Veilleux. Proceedings
of the Speech
Prosody (pp. 922–925). International Speech Communication Association.
Kushch, O., Igualada, A., & Prieto, P. (2018). Prominence
in speech and gesture favour second language novel word learning. Language, Cognition and
Neuroscience, 33(8), 992–1004.
Lazaraton, A. (2004). Gesture
and speech in the vocabulary explanations of one ESL teacher: A microanalytic inquiry. Language
Learning, 54(1), 79–117.
(1999). Producing
spoken language: A blueprint of the speaker. In C. Brown & P. Hagoort (Eds.), Neurocognition
of
language (pp. 83–122). Oxford University Press.
Lichtman, K. (2021). What
about fluency? Implicit vs. explicit training affects artificial mini-language
production. Applied
Linguistics, 42(4), 668–691.
Lin, Y. L. (2021). Gestures
as scaffolding for L2 narrative recall: The role of gesture type, task complexity, and working
memory. Language Teaching Research.
Liu, D., & Reed, M. (2021). Exploring
the complexity of the L2 intonation system: An acoustic and eye-tracking study. Frontiers in
Communication, 61, 627316.
Loehr, D. P. (2012). Temporal,
structural, and pragmatic synchrony between intonation and gesture. Laboratory
Phonology, 3(1), 71–89.
Lynch, T., & Maclean, J. (2000). Exploring
the benefits of task repetition and recycling for classroom language learning. Language
Teaching
Research, 41, 221–250.
Ma, S., & Jin, G. (2022). The
relationship between different types of co-speech gestures and L2 speech performance. Frontiers
in Psychology, 131, 941114.
Macedonia, M. (2014). Bringing
back the body into the mind: Gestures enhance word learning in foreign language. Frontiers in
Psychology, 51, 1467.
Macedonia, M., Müller, K., & Friederici, A. D. (2011). The
impact of iconic gestures on foreign language word learning and its neural substrate. Human
Brain
Mapping, 321, 982–998.
Macedonia, M., & von Kriegstein, K. (2012). Gestures
enhance foreign language
learning. Biolinguistics, 61, 393–416.
McCafferty, S. G. (2002). Gesture
and creating zones of proximal development for second language learning. The Modern Language
Journal, 86(2), 192–203.
Mennen, I. (2007). Phonological
and phonetic influences in non-native intonation. In J. Trouvain, & U. Gut (Eds.), Non-native
prosody. Phonetic description and teaching
practice (pp. 53–76). Mouton de Gruyter.
Mennen, I., Schaeffler, F., & Docherty, G. (2008). A
methodological study into the linguistic dimensions of pitch range differences between German and
English. In P. L. Barbosa, S. Madureira, & C. Reis. Proceedings
of the IV Conference on Speech
Prosody (pp. 527–530). ISCA Archive [URL]
Mennen, I., Schaeffler, F., Watt, N., & Miller, N. (2008). An
autosegmental-metrical investigation of intonation in people with Parkinson’s
Disease. Asia-Pacific Journal of Speech Language and
Hearing, 11(4), 205–219.
Mennen, I., Schaeffler, F., Docherty, G. (2012). Cross-language
differences in fundamental frequency range: A comparison of English and German. The Journal of
the Acoustical Society of
America, 131(3), 2249–2260.
Mittelberg, I., & Waugh, L. R. (2009). Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In C. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor (pp. 329–356). Mouton de Gruyter.
Morett, L. M., & Chang, L. Y. (2014). Emphasising
sound and meaning: Pitch gestures enhance Mandarin lexical tone acquisition. Language,
Cognition and
Neuroscience, 98(3), 773–794.
Morett, L. M., & Fraundorf, S. H. (2019). Listeners
consider alternative speaker productions in discourse comprehension and memory: Evidence from beat gesture and pitch
accenting. Memory and
Cognition, 471, 1515–1530.
Mori, J., & Hayashi, M. (2006). The
achievement of intersubjectivity through embodied completions: A study of interactions between first and second language
speakers. Applied
Linguistics, 271, 195–219.
Morris, D., Collett, P., Marsh, P., & O’Shaughnessy, M. (1980). Gestures: Their origins and distribution. New York.
Morsella, E., & Krauss, R. M. (2004). The
role of gestures in spatial working memory and speech. American Journal of
Psychology, 1171, 411–424.
Nergis, A. (2021). Can
explicit instruction of formulaic sequences enhance L2 oral
fluency? Lingua, 2551, 1–21.
Nguyen, A. T., & Dao, D. M. (2018). The
acquisition of question intonation by Vietnamese learners of English. Asian Journal of Second
Foreign Language
Education, 3(3), 1–13.
Núñez, R. (2008). A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor. In A. Cienki & C. Müller (Eds.), Metaphor and gesture (pp. 93–114). John Benjamins.
Pang, F., & Skehan, P. (2014). Self-reported
planning behaviour and L2 performance in narrative
retelling. In P. Skehan (Ed.), Processing
perspectives on task
performance (pp. 95–128). John Benjamins.
Peters, J., Frank, M., & Rohloff, M. (2020). Pitch
range variation in High German (L1) and Low German
(L2). In Proceedings of the 10th International Conference on Speech
Prosody (pp. 650–654).
Pika, S., Nicoladis, E., & Marentette, P. F. (2006). A
cross-cultural study on the use of gestures: Evidence for cross-linguistic
transfer? Bilingualism: Language and
Cognition, 91, 319–327.
Pine, K. J., Bird, H., & Kirk, E. (2007). The
effects of prohibiting gestures on children’s lexical retrieval ability. Developmental
Science, 101, 747–754.
Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., & R
Core Team. (2020). nlme: Linear and Nonlinear Mixed Effects
Models. R package version 3.1–149. Retrieved from [URL]
Poesio, M., & Artstein, R. (2008). Introduction
to the special issue on ambiguity and semantic judgments. Research on Language and
Computation, 61, 241–245.
Quinn-Allen, L. Q. (1995). The
effects of emblematic gestures on the development and access of mental representations of French
expressions. The Modern Language
Journal, 791, 521–529.
Rauscher, F. H., Krauss, R. M. & Chen, Y. (1996). Gesture,
speech, and lexical access: The role of lexical movements in speech production. Psychological
Science, 7(4), 226–231.
Robinson, P. (1996). Learning
simple and complex second language rules under implicit, incidental, rule-search, and instructed
conditions. Studies in Second Language
Acquisition, 18(1), 27–67.
Rohrer, D., & Pashler, H. (2007). Increasing
retention without increasing study time. Current Directions in Psychological
Science, 16(4), 183–186.
Rousse-Malpat, A., Koote, L., Steinkrauss, R., & Verspoor, M. (2021). Parlez-vous
francais? Effects of structure-based versus dynamic-usage-based approaches on oral
proficiency. Language Teaching Research.
Santiago, F., & Delais-Roussarie, E. (2014). What
explains the distribution and form of non-final rising contours in French as an
L2? In Concordia Working Papers in Applied Linguistics Vol.
5 (pp. 561–579). Retrieved
from [URL]
Schmidt, R. (2000). Attention. In P. Robinson (Ed.), Cognition
and second language
instruction (pp. 3–32). Cambridge University Press.
Schwab, S., & Goldman, J. P. (2016). Do
speakers show different F when they speak in different languages? The case of English, French and
German. In Proceedings of Speech Prosody
2016. Retrieved from [URL].
(2016). Second
language fluency and its underlying cognitive and social determinants. International Review of
Applied Linguistics in Language
Teaching, 54(2), 79–95.
Seyfeddinipur, M. (2006). Disfluency:
Interrupting speech and gesture. Doctoral thesis. Radboud University.
Shattuck-Hufnagel, S., Yasinnik, Y., Veilleux, N., & Renwick, M. (2007). A
method for studying the time alignment of gestures and prosody in American English: ‘Hits’ and pitch accents in
academic-lecture-style speech. In A. Esposito, M. Bratanic, E. Keller & M. Marinaro (Eds.), Fundamentals
of verbal and nonverbal communication and the biometric
issue (pp. 34–44). IOS Press.
(2009). Modelling
second language performance: Integrating complexity, accuracy, fluency, and lexis. Applied
Linguistics, 30(4), 510–532.
(2014). Processing
perspectives on task performance. John Benjamins.
Skehan, P., & Foster, P. (2001). Cognition
and tasks. In P. Robinson (Ed.), Cognition
and second language
instruction (pp. 183–205). Cambridge University Press.
Sloetjes, H., & Wittenburg, P. (2008). Annotation
by category – ELAN and ISO DCR. In N. Calzolari, K. Choukri, M. Bente, J. Mariani, J. Odijk, S. Piperidis, & D. Tapias (Eds.), Proceedings
of the 6th International Conference on Language Resources and Evaluation. Retrieved
from [URL]
Smotrova, T., & Lantolf, J. P. (2013). The
function of gesture in lexically focused L2 instructional conversations. Modern Language
Journal, 971, 397–416.
Sueyoshi, A., & Hardison, D. M. (2005). The
role of gestures and facial cues in second language listening comprehension. Language
Learning, 55(4), 661–699.
Taguchi, N. (2008). Building
language blocks in L2 Japanese: Chunk learning and the development of complexity and fluency in spoken
production. Foreign Language
Annals, 41(1), 132–156.
Tavakoli, P., & Skehan, P. (2005). Strategic
planning, task structure, and performance testing. In R. Ellis (Ed.), Planning
and task performance in a second language. John Benjamins.
Tellier, M. (2008). The
effect of gestures on second language memorisation by young
children. Gesture, 8(2), 219–235.
Trouvain, J., Zimmerer, F., Möbius, B., Gósy, M., & Bonneau, A. (2017). Segmental,
prosodic and fluency features in phonetic learner corpora: Introduction to the special
issue. International Journal of Learner Corpus
Research, 3(2), 105–117.
Ullakanoja, R. (2007). Comparison
of pitch range in Finnish (L1) and Russian (L2). In J. Trouvain, & W. J. Barry (Eds), Proceedings
of the 16th International congress of phonetic
sciences (pp. 1701–1704). Retrieved
from [URL]
Volin, J., Poesova, K., & Weingartova, L. (2015). Speech
melody properties in English, Czech and Czech English: Reference and interference. Research in
Language, 13(1), 107–123.
Wang, L., & Chu, M. (2013). The
role of beat gesture and pitch accent in semantic processing: An ERP
study. Neuropsychologia, 51(13), 2847–2855.
Wang, Z. (2014). Online
time pressure manipulations: L2 speaking performance under five types of planning and repetition
conditions. In P. Skehan (Ed.), Task-based
language teaching: Processing perspectives on task
performance (p. 27–61). John Benjamins.
Yasinnik, Y., Renwick, M., Shattuck-Hufnagel, S. (2004). The
timing of speech-accompanying gestures with respect to
gestures. In Proceedings of from sound to
sense (pp. 97–102). Retrieved
from [URL]
Zimmerer, F., Jügler, J., Andreeva, B., Möbius, B., & Trouvain, J. (2014). Too
cautious to vary more? A comparison of pitch variation in native and non-native productions of French and German
speakers. In N. Campbell, D. Gibbon, & D. Hirst (Eds.), Proceedings
of the 7th international conference on speech
prosody (pp. 1037–1041). Retrieved
from [URL].
