Cover not available

In:Linguistic Foundations of Narration in Spoken and Sign Languages:
Edited by Annika Hübl and Markus Steinbach
[Linguistik Aktuell/Linguistics Today 247] 2018
► pp. 15–40

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload Book PDF
authorization required

A new technique for analyzing narrative prosodic effects in sign languages using motion capture technology

 | Purdue University
 | Purdue University
Published online: 25 May 2018
DOI logo with link
https://doi.org/10.1075/la.247.02wil
The present paper addresses prosody at a sentence level analysis within short narratives, developing a novel method based on a combination of data. Our first objective, then, is to establish the validity of our new method by confirming the findings of previous reports on Phrase Final Lengthening. Our second objective is to further extend the validity of the new method by confirming a prior claim regarding the role of increased velocity for signs that are stressed in sentences. Finally, we report new results in the form of patterns across signs within sentences using our new method.
Article outline
  • 1.Introduction
    • 1.1Perceptual studies of prosody and fluency
    • 1.2Production studies of fluency and prosody
    • 1.3Prior kinematic work
      • 1.3.1Prior motion capture work showing sign lowering
      • 1.3.2Prior motion capture work showing verb class differences
      • 1.3.3Prior motion capture work showing phrase final lengthening
      • 1.3.4Prior motion capture work showing stress marking
  • 2.The present study
    • 2.1Procedure
      • 2.1.1Motion capture method for current report
      • 2.1.2Coding for non-kinematic linguistic and perceptual variables
      • 2.1.3Coding for kinematic variables
    • 2.2Analyses
    • 2.3Results
      • 2.3.1Kinematic analysis by sentence position
      • 2.3.2Kinematic analysis of stress
      • 2.3.3More general test of relationships among measured variables and weight
    • 2.4Summary
  • 3.Conclusion
  • Acknowledgements
  • Notes
  • References
References (56)
References
Allen, G. D., Wilbur, R. B. & Schick, B. 1991. Aspects of rhythm in American Sign Language. Sign Language Studies 72: 297–320.Google Scholar logo with link to Google Scholar
Boyes Braem, P. 1999. Rhythmic temporal patterns in the signing of deaf early and later learners of Swiss German Sign Language. Language & Speech 42: 177–208.Google Scholar logo with link to Google Scholar
Brentari, D.. 1998. A Prosodic Model of Sign Language Phonology. Cambridge MA: The MIT Press.Google Scholar logo with link to Google Scholar
Brentari, D. & Crossley, L. 2002. Prosody on the hands and face: Evidence from American Sign Language. Sign Language & Linguistics 5: 105–130.Google Scholar logo with link to Google Scholar
Brentari, D., Nadolske, M. & Wolford, G. 2012. Can experience with co-speech gesture influence the prosody of a sign language? Sign language prosodic cues in bimodal bilinguals. Bilingualism: Language and Cognition 15(2): 402–412.Google Scholar logo with link to Google Scholar
Brentari, D., González, C., Seidl, A. & Wilbur, R. B. 2011. Sensitivity to visual prosodic cues in signers and nonsigners. Language & Speech 54: 49–72.Google Scholar logo with link to Google Scholar
González, C. 2011. Perception of Prosody in American Sign Language. PhD dissertation, Purdue University.Google Scholar logo with link to Google Scholar
Green, K.. 1984. Sign boundaries in American Sign Language. Sign Language Studies 42: 65–91.Google Scholar logo with link to Google Scholar
Grosjean, F. & Lane, H. 1977. Pauses and syntax in American Sign Language. Cognition 5: 101–117.Google Scholar logo with link to Google Scholar
Grosvald, M. A. 2009. Long-distance Coarticulation: A Production and Perception Study of English and American Sign Language. PhD dissertation, University of California at Davis.Google Scholar logo with link to Google Scholar
Kantor, R. 1978. Identifying native and second-language signers. In Sign Language Research, R. B. Wilbur (ed), Special issue of Communication and Cognition 11(1): 39–55.Google Scholar logo with link to Google Scholar
Kröger, B. J., Birkholz, P., Kannampuzha, J., Kaufmann, E. & Mittelberg, I. 2011. Movements and holds in fluent sentence production of American Sign Language: The action-based approach. Cognitive Computation 3(3): 449–465.Google Scholar logo with link to Google Scholar
Liddell, S. K. 1977. An Investigation into the Syntactic Structure of American Sign Language. PhD dissertation, University of California, San Diego.Google Scholar logo with link to Google Scholar
Lupton, L. K.. 1998. Fluency in American Sign Language. Journal of Deaf Studies and Deaf Education 3(4): 320–328.Google Scholar logo with link to Google Scholar
Lupton, L. K. & Zelaznik, H. 1990. Motor learning in sign language students. Sign Language Studies 67: 153–174.Google Scholar logo with link to Google Scholar
Weast, T. 2008. Questions in American Sign Language: A Quantitative Analysis of Raised and Lowered Eyebrows. Phd dissertation, University of Texas, Arlington, U.S.A.Google Scholar logo with link to Google Scholar
Malaia, E.. 2014. It still isn’t over: Event boundaries in language and perception. Language and Linguistics Compass 8(3): 89–98.Google Scholar logo with link to Google Scholar
. 2017. Current and future methodologies for quantitative analysis of information transfer in sign language and gesture data. Behavioral and Brain Sciences 40.Google Scholar logo with link to Google Scholar
Malaia, E., Borneman, J. D. & Wilbur, R. B. 2016. Assessment of information content in visual signal: analysis of optical flow fractal complexity. Visual Cognition 24(3): 246–251.Google Scholar logo with link to Google Scholar
2017. Information transfer capacity of articulators in American Sign Language. Language and Speech. doi:.Google Scholar logo with link to Google Scholar
Malaia, E., Ranaweera, R., Wilbur R. B. & Talavage, T. M. 2012. Neural representation of event structure in American Sign Language: fMRI comparison of cortical activations in deaf signers and hearing non-signers. Neuroimage 59: 4094–4101.Google Scholar logo with link to Google Scholar
Malaia, E. & Wilbur, R. B. 2012a. Kinematic signatures of telic and atelic events in ASL predicates. Language & Speech 55(3): 407–421.Google Scholar logo with link to Google Scholar
2012b. Telicity expression in the visual modality. In Telicity, Change, and State: A Cross-categorial View of Event Structure, V. Demonte & L. McNally (eds), 122–136. Oxford: OUP.Google Scholar logo with link to Google Scholar
Malaia, E., Wilbur, R. B. & Milković, M. 2013. Kinematic parameters of signed verbs at the morpho-phonology interface. Journal of Speech, Language and Hearing Research 56: 1677–1688.Google Scholar logo with link to Google Scholar
Malaia, E., Wilbur, R. B. & Weber-Fox, C. 2013. Event end-point primes the undergoer argument: Neurobiological bases of event structure processing. In Studies in the Composition and Decomposition of Event Predicates, B. Arsenijevic, B. Gehrke & R. Marin (eds), 231–248. Dordrecht: Springer.Google Scholar logo with link to Google Scholar
Mauk, C. E. 2003. Undershoot in Two Modalities: Evidence from Fast Speech and Fast Signing. PhD dissertation, University of Texas at Austin.Google Scholar logo with link to Google Scholar
Mauk, C. E. & Tyrone, M. E. 2008. Sign lowering as phonetic reduction in American Sign Language. In Proceedings of the 2008 International Seminar on Speech Production, R. Sock, S. Fuchs & Y. Laprie (eds), 185–188. <[URL]>Google Scholar logo with link to Google Scholar
2012. Location in ASL: Insights from phonetic variation. Sign Language & Linguistics 15(1): 128–146.Google Scholar logo with link to Google Scholar
McDonald, J., Wolfe, R., Wilbur, R. B., Moncrief, R., Malaia, E., Fujimoto, S., Baowidan, S. & Stec, J. 2016. A new tool to facilitate prosodic analysis of motion capture data and a data-driven technique for the improvement of avatar motion. In Proceedings of the Seventh Workshop on the Representation and Processing of Sign Languages: Corpus Mining (LREC-2016), 153–158, Portorož, Slovenia.Google Scholar logo with link to Google Scholar
Milković, M. 2011. Verb Classes in Croatian Sign Language (HZJ): Syntactic and Semantic Properties PhD dissertation, University of Zagreb, Croatia.Google Scholar logo with link to Google Scholar
Nespor, M. & Sandler, W. 1999. Prosody in Israeli Sign Language. Language and Speech 42: 143–176.Google Scholar logo with link to Google Scholar
Ormel, E. & Crasborn, O. 2012. Prosodic correlates of sentences in signed languages: A literature review and suggestions for new types of studies. Sign Language Studies 12(2): 279–315.Google Scholar logo with link to Google Scholar
Pfeiffer, T. 2013. Documentation of gestures with motion capture. In Body – Language – Communication, C. Müller, A. Cienki, E. Fricke, H. S. Ladewig, D. McNeill & S. Tessendorf (eds), 857–868. Berlin: De Gruyter Mouton.Google Scholar logo with link to Google Scholar
Pfeiffer, T., Hofmann, F., Hahn, F., Rieser, H. & Röpke, I. 2013. Gesture semantics reconstruction based on motion capturing and complex event processing: A circular shape example. In Proceedings of the SIGDIAL 2013 Conference, Maxine Eskenazi, Michael Strube, Barbara Di Eugenio & Jason D. Williams (eds), 270–279. Metz: ACL. <[URL]>
Russell, K., Wilkinson, E. & Janzen, T. 2011. ASL sign lowering as undershoot: A corpus study. Laboratory Phonology 2(2): 403–422.Google Scholar logo with link to Google Scholar
Sandler, W. 2012. Visual prosody. In Sign language, R. Pfau, M. Steinbach & B. Woll (eds), 55–76. Berlin: De Gruyter Mouton.Google Scholar logo with link to Google Scholar
Sandler, W. & Lillo-Martin, D. 2006. Sign Language and Linguistic Universals. Cambridge: CUP.Google Scholar logo with link to Google Scholar
Tang, G., Brentari, D., González, C., & Sze, F. 2010. Crosslinguistic variation in prosodic cues. In Sign languages, D. Brentari (ed), 519–542. Cambridge: CUP.Google Scholar logo with link to Google Scholar
Tyrone, M. E. 2007. Simultaneity in atypical signers – Implications for the structure of signed language. In Simultaneity in Signed Language: Form and Function [Current Issues in Linguistic Theory 281], M. Vermeerbergen, L. Leeson & O. Crasborn (eds), 317–335. Amsterdam: John Benjamins.Google Scholar logo with link to Google Scholar
Tyrone, M. E., Nam, H., Saltzman, E., Mathur, G. & Goldstein, L. 2010. Prosody and movement in American Sign Language: A task-dynamics approach. In Speech Prosody 2010 Conference Proceedings. <[URL]>
Tyrone, M. E., Atkinson, J. R., Marshall, J. & Woll, B. 2009. The effects of cerebellar ataxia on sign language production: A case study. Neurocase 15(5): 419–426.Google Scholar logo with link to Google Scholar
Tyrone, M. E. & Mauk, C. E. 2010. Sign lowering and phonetic reduction in American Sign Language. Journal of Phonetics 38(2): 317–328.Google Scholar logo with link to Google Scholar
Malaia, E., Wilbur, R. B. 2012. Kinematic signatures of telic and atelic events in ASL predicates. Language and Speech 55(3): 407–421. http://dx.doi.org/10.1177/0023830911422201Google Scholar logo with link to Google Scholar
Wilbur, R. B.. 1990. An experimental investigation of stressed sign production. International Journal of Sign Linguistics 1(1): 41–59.Google Scholar logo with link to Google Scholar
. 1994. Eyeblinks and ASL phrase structure. Sign Language Studies 84: 221–240.Google Scholar logo with link to Google Scholar
1997. A prosodic/pragmatic explanation for word order variation in ASL with typological implications. In Lexical and Syntactic Constructions and the Construction of Meaning, Vol. 1 [Current Issues in Linguistic Theory 150], M. Verspoor, K. D. Lee & E. Sweetser (eds) 89–104. Amsterdam: John Benjamins.Google Scholar logo with link to Google Scholar
1999. Stress in ASL: Empirical evidence and linguistic issues. Language & Speech 42: 229–250.Google Scholar logo with link to Google Scholar
2000. Phonological and prosodic layering of non-manuals in American Sign Language. In The Signs of Language Revisited: Festschrift for Ursula Bellugi and Edward Klima, H. Lane & K. Emmorey (eds), 213–241. Hillsdale NJ: Lawrence Erlbaum Associates.Google Scholar logo with link to Google Scholar
2009. Effects of varying rate of signing on ASL manual signs and non-manual markers. Language & Speech 52(2–3): 245–285.Google Scholar logo with link to Google Scholar
2010. The semantics-phonology interface. In Sign languages, D. Brentari (ed), 357–382. Cambridge: CUP.Google Scholar logo with link to Google Scholar
2011b. Sign syllables. In The Blackwell Companion to Phonology, M. van Oostendorp, C. J. Ewen, E. Hume & K. Rice (eds), 1309–1334. Oxford: Blackwell.Google Scholar logo with link to Google Scholar
Wilbur, R. B. & Patschke, C. 1998. Body leans and marking contrast in ASL. Journal of Pragmatics 30: 275–303.Google Scholar logo with link to Google Scholar
Wilbur, R. B. & Malaia, E. 2008. Contributions of sign language research to gesture understanding: What can multimodal computational systems learn from sign language research. International Journal of Semantic Computing 2(1): 5–19.Google Scholar logo with link to Google Scholar
Wilbur, R. B. & Schick, B. S. 1987. The effects of linguistic stress on ASL signs. Language & Speech 30: 301–323.Google Scholar logo with link to Google Scholar
Wilbur, R. B. 2003. Representations of telicity in ASL. Chicago Linguistics Society 39, 354–368.Google Scholar logo with link to Google Scholar
Cited by (5)

Cited by five other publications

Krebs, Julia, Ronnie B. Wilbur, Dietmar Roehm & Evie A. Malaia
2025. The interaction of syntax, non-manuals, and prosodic cues as potential topic markers in Austrian Sign Language. Sign Language & Linguistics 28:1 ► pp. 1 ff. DOI logo
Haug, Tobias, Nivja H. de Jong, Franz Holzknecht, Katja Tissi, Sandra Sidler-Miserez, Alessia Battisti, Regula Perrollaz, Sarah Ebling, Sabine Reinhard & Sarah Caminada
2024. Development and validation of a fluency rating scale for Swiss German Sign Language. Frontiers in Education 9 DOI logo
Napoli, Donna Jo & Rachel Sutton-Spence
2024. Torso articulation in sign languages. Sign Language & Linguistics 27:1 ► pp. 1 ff. DOI logo
Catteau, Fanny
2022. Translating sign language poetry into spoken French. Sign Language & Linguistics 25:1 ► pp. 99 ff. DOI logo
Krebs, Julia, Evie Malaia, Ronnie B. Wilbur & Dietmar Roehm
2020. Interaction between topic marking and subject preference strategy in sign language processing. Language, Cognition and Neuroscience 35:4 ► pp. 466 ff. DOI logo

This list is based on CrossRef data as of 22 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue