In:The Documentarist Turn: From observable linguistic behaviour to typological generalizations
Edited by Sonja Riesberg, Uta Reinöhl and Birgit Hellwig
[Studies in Language Companion Series 240] 2026
► pp. 182–209
Chapter 8Taking language use seriously
On the importance of multimodality and interaction in documentary practices
This content is being prepared for publication; it may be subject to changes.
Abstract
The aim of language documentation is to provide a comprehensive record of the observable linguistic
behaviour and practices of a language community to be used now and in the future. The endeavour goes beyond the mere
description of the linguistic system. To serve as a multipurpose record, the documentary record must capture language
use in its fullest form — as a fundamentally multimodal and interactional phenomenon. This means that video recording
is essential to the collection of data in language documentation efforts, and that the description and analysis of
data must ultimately extend to and integrate multimodal and interactive components of language use. In this chapter,
we outline the implications for language documentation from such a conception of language.
Keywords: language use, interaction, multimodality, gesture, sign language, language documentation
Article outline
- 1.Introduction
- 2.Aim of language documentation
- 3.Some observations on the history of documentation
- 4.Language use, multimodality and joint action
- 5.What we stand to gain
- 5.1Manual gestures
- 5.2Sign languages
- 6.Implications for language documentation
- 6.1Implications for video recording
- 6.2Implications for transcription and analysis
- 6.3Implications for training
- 7.Conclusion
Acknowledgements References
References (122)
Adone, Marie Carla D., and Elaine Maypilama. 2014. A
Grammar Sketch of Yolngu Sign
Language. München: Lincom Europe.
Albert, Saul, and Jan P. de Ruiter. 2018. “Repair:
The Interface between Interaction and Cognition.” Topics in Cognitive
Science 10 (2): 279–313.
Bangerter, Adrian, and Herbert H. Clark. 2003. “Navigating
Joint Projects with Dialogue.” Cognitive
Science 27 (2): 195–225.
Bauer, Anastasia. 2014. The
Use of Signing Space in a Shared Sign Language of Australia. Berlin, Boston: De Gruyter Mouton.
Bavelas, Janet Beavin. 2022. Face-to-Face
Dialogue: Theory, Research, and
Applications. Oxford: Oxford University Press.
Bolden, Galina B. 2011. “On the
Organization of Repair in Multiperson Conversation: The Case of ‘Other’-Selection in Other-Initiated Repair
Sequences.” Research on Language and Social
Interaction 44 (3): 237–262.
Braithwaite, Ben. 2019. “Sign
Language Endangerment and Linguistic
Diversity.” Language 95 (1): e161–e187.
Bressem, Jana, Nicole Stein, and Claudia Wegener. 2017. “Multimodal
Language Use in Savosavo: Refusing, Excluding and Negating with Speech and
Gesture.” Pragmatics 27 (2): 173–206.
Burenhult, Niclas. 2020. “Domain-driven
Documentation: The Case of Landscape.” In Approaches
to Interdisciplinary Research in Language Documentation, Language Documentation and
Conservation, ed. by Susan Penfield, 9–22. Honolulu: University of Hawai’i Press. [URL]
Clark, Herbert H., and Jean E. Fox Tree. 2002. “Using
‘uh’ and ‘um’ in spontaneous
speaking.” Cognition 84 (1): 73–111.
Clark, Herbert H., and Richard J. Gerrig. 1990. “Quotations
as
Demonstrations.” Language 66 (4): 764–805.
Coppola, Marie. 2021. “Sociolinguistic
Sketch: Nicaraguan Sign Language and Homesign Systems in
Nicaragua.” In Emerging Sign Languages of the
Americas, ed. by Olivier Le Guen, Josefina Safar, and Marie Coppola, 439–450. Berlin, Boston: De Gruyter Mouton.
Defina, Rebecca. 2016. “Do
Serial Verb Constructions Describe a Single Event? A Study of Co-speech Gestures in
Avatime.” Language 92 (4): 890–910.
Dimmendaal, Gerrit J. 2010. “Language
Description and ‘The New Paradigm’: What Linguists May Learn from
Ethnocinematographers.” Language Documentation and
Conservation 4: 152–158. [URL]
Dingemanse, Mark. 2013. “Ideophones
and Gesture in Everyday
Speech.” Gesture 13 (2): 143–165.
Dingemanse, Mark, and Nick J. Enfield. 2015. “Other-initiated
Repair Across Languages: Towards a Typology of Conversational Structures.” Open
Linguistics 1 (1): 98–118.
Dingemanse, Mark, Seán G. Roberts, Julija Baranova, Joe Blythe, Paul Drew, Simeon Floyd, Rosa S. Gisladottir, Kobin H. Kendrick, Stephen C. Levinson, Elizabeth Manrique, Giovanni Rossi, and Nick J. Enfield. 2015. “Universal
Principles in the Repair of Communication Problems.” PLOS
ONE 10 (9): e0136100.
Duerr, Michael. 2013. “Die Erforschung der Sprachen Kamčatkas [Research into the Kamčatkas languages].” In Reisen an den Rand des Russischen Reiches: Die wissenschaftliche Erschließung der
nordpazifischen Küstengebiete im 18. und 19. Jahrhundert [Journeys to the edge of the Russian empire: The scientific exploration of the North Pacific coastal regions
in the 18th and 19th centuries], ed. by Erich Kasten, 287–296. Fürstenberg/Havel: Kulturstiftung Sibirien.
. 2005. “History
of Linguistic Anthropology.” In Linguistic
Anthropology: Encyclopedia of Life Support Systems (EOLSS), ed.
by Anita Sujodzic, 263–278. Paris: EOLSS Publishers.
Eberhard, David M., Gary F. Simons, and Charles D. Fennig (eds). 2025. Ethnologue:
Languages of the World, 28th Edition. Dallas, Texas: SIL International. Online
version: [URL]
Emmorey, Karen, Helsa B. Borinstein, Robin Thompson, and Tamar H. Gollan. 2008. “Bimodal
Bilingualism.” Bilingualism: Language and
Cognition 11 (1): 43–61.
Ennever, Thomas Blake. 2024. Topics in a
Grammar of Space of Kukatja. PhD
Thesis, Monash University.
Evans, Nicholas. 2008. “Review
of Essentials of Language Documentation. 2006.” Language Documentation and
Conservation 2 (2): 340–350. [URL]
Floyd, Simeon. 2016. “Modally
Hybrid Grammar? Celestial Pointing for Time-of-day Reference in
Nheengatú.” Language 92 (1): 31–64.
Gawne, Lauren, and Kensy Cooperrider. 2024. “Emblems:
Meaning at the Interface of Language and Gesture.” Glossa: A Journal of General
Linguistics 9 (1).
Gippert, Jost, Nikolaus P. Himmelmann, and Ulrike Mosel. 2006. Essentials
of Language Documentation. Berlin, New York: De Gruyter Mouton.
González Pérez, Manuel David. Submitted. “Widening
the View: Immersive 360° Filming in Language Documentation.” Language
Documentation and Description.
González Pérez, Manuel David, and Agnes Conrad. 2023. Language,
Environment and Social Cognition in Southeastern Yunnan: A Holistic Documentation of Multimodal Communication
amongst the Phola. Endangered Languages Archive. [URL]
Goodwin, Charles. 1986. “Gestures
as a Resource for the Organization of Mutual
Orientation.” Semiotica 62 (1–2): 29–49.
. 2014. “The
Intelligibility of Gesture Within a Framework of Co-operative
Action.” In From Gesture in Conversation to Visible
Action as Utterance: Essays in Honor of Adam Kendon, ed. by Mandana Seyfeddinipur, and Marianne Gullberg, 199–216. Amsterdam: John Benjamins.
Goodwin, Marjorie Harness, and Charles Goodwin. 1986. “Gesture
and Coparticipation in the Activity of Searching for a
Word.” Semiotica 62 (1–2): 51–75.
Green, Jennifer. 2021. “Mothers,
Mountain Devils, and Pointing to Eternity: The ‘Horns’ Handshape in Australian Indigenous Sign
Languages.” Sign Language
Studies 22 (1): 5–41.
Gumperz, John J., and Dell Hymes (eds). 1964. “The
Ethnography of Communication.” Special
Issue, American
Anthropologist 66 (6.2). [URL]
Hale, Kenneth, Michael Krauss, Lucille J. Watahomigie, Akira Y. Yamamoto, Colette Craig, LaVerne Masayesva Jeanne, and Nora C. England. 1992. “Endangered
Languages.” Language 68 (1): 1–42.
Hammarström, Harald, Robert Forkel, Martin Haspelmath, and Sebastian Bank. 2025. Glottolog
5.2. Leipzig: Max Planck Institute for Evolutionary Anthropology.
Harrison, Simon. 2024. “On
Grammar-Gesture Relations: Gestures Associated with
Negation.” In The Cambridge Handbook of Gesture
Studies, ed. by Alan Cienki, 446–474. Cambridge: Cambridge University Press.
Haviland, John B. 1993. “Anchoring,
Iconicity, and Orientation in Guugu Yimithirr Pointing Gestures.” Journal of
Linguistic
Anthropology 3 (1): 3–45.
Henley, Paul. 2020. Beyond
Observation: A History of Authorship in Ethnographic
Film. Manchester: Manchester University Press.
Himmelmann, Nikolaus P. 1998. “Documentary
and Descriptive
Linguistics.” Linguistics 36 (1): 161–195.
2006. “Language
Documentation: What Is It and What Is It Good
For?” In Essentials of Language
Documentation, ed. by Jost Gippert, Nikolaus P. Himmelmann, and Ulrike Mosel, 1–30. Berlin, New York: De Gruyter Mouton.
2011. On
Linguistic Uses of Language
Documentations. In Selected Papers from the
International Conference on Language Documentation and Tradition (7–9 November
2008), ed. by Carol Everhard, Elizabeth Mela-Athanasopoulou, and Youli Theodosiadou, 37–51. Thessaloniki: Aristotle University of Thessaloniki, University Studio Press.
2012. “Linguistic
Data Types and the Interface between Language Documentation and
Description.” Language Documentation and
Conservation 6: 187–207. [URL]
2018. “Meeting the
Transcription Challenge.” In Reflections on Language
Documentation 20 Years after Himmelmann 1998, ed. by Bradley McDonnell, Andrea L. Berez-Kroeker, and Gary Holton, 33–40. Honolulu: University of Hawai’i Press. [URL]
Hinton, Leanne, Leena Huss, and Gerald Roche (eds.). 2018. The
Routledge Handbook of Language Revitalization. New York: Routledge.
Hou, Lynn Y.-S. 2016. “Making
Hands”: Family Sign Languages in the San Juan Quiahije Community. PhD
Thesis, The University of Texas at Austin. [URL]
Hou, Lynn, and Connie de Vos. 2022. “Classifications
and Typologies: Labeling Sign Languages and Signing Communities.” Journal of
Sociolinguistics 26 (1): 118–125.
Humphries, Tom. 1977. Communicating
Across Cultures (Deaf-Hearing) and Language Learning. PhD
Thesis, Union Institute and University. ProQuest (DP10817).
Hymes, Dell. 1962. “The
Ethnography of Speaking.” In Anthropology and Human
Behavior, ed. by Thomas Gladwin, and William C. Sturtevant, 13–53. Washington, D.C.: Anthropology Society of Washington.
. 1964. “Toward
Ethnographies of Communication.” American
Anthropologist 66 (6): 1–34. [URL]
Inbar, Anna, and Leon Shor. 2025. “The
Depiction of Negativity: The Case of the Brushing Hands Gesture Used by Hebrew
Speakers.” In Perspectives on Negation: Views from
across the Language Sciences, ed. by Frances Blanchette, and Cynthia Lukyanenko, 21–56. Berlin, Boston: De Gruyter Mouton.
Jefferson, Gail. 2004. “Glossary
of Transcript Symbols with an
Introduction.” In Conversation Analysis: Studies from
the First Generation, ed. by Gene H. Lerner, 13–31. Amsterdam: John Benjamins.
Kegl, Judy, Ann Senghas, and Marie Coppola. 1999. “Creation
through Contact: Sign Language Emergence and Sign Language Change in
Nicaragua.” In Language Creation and Language Change:
Creolization, Diachrony, and Development, ed. by Michel DeGraff, 179–237. Cambridge, MA: MIT Press.
Kendon, Adam. 1967. “Some
Functions of Gaze-direction in Social Interaction.” Acta
Psychologica 26: 22–63.
. 1979. “Some
Emerging Features of Face-to-Face Interaction Studies.” Sign Language
Studies 22: 7–22.
. 1983. “The
Study of Gesture: Some Remarks on its
History.” In Semiotics
1981, ed. by John N. Deely, and Margot D. Lenhart, 153–164. Boston, MA: Springer.
. 1984. “Knowledge
of Sign Language in an Australian Aboriginal Community.” Journal of
Anthropological
Research 40 (4): 556–576.
. 1987. “Speaking
and Signing Simultaneously in Warlpiri Sign Language
Users.” Multilingua 6 (1), 25–68.
. 1988. Sign
Languages of Aboriginal Australia: Cultural, Semiotic, and Communicative
Perspectives. Cambridge: Cambridge University Press.
. 1995. “Gestures
as Illocutionary and Discourse Structure Markers in Southern Italian
Conversations.” Journal of
Pragmatics 23 (3): 247–279.
. 2002. “Some
Uses of the Head
Shake.” Gesture 2 (2): 147–182.
Kita, Sotaro, and Karen Emmorey. 2023. “Gesture
Links Language and Cognition for Spoken and Signed Languages.” Nature Reviews
Psychology 2: 407–420.
Lambrecht, Linda, Barbara Earth, and James Woodward. 2013. “History
and Documentation of Hawai’i Sign Language: First
Report.” Presentation, 3rd International
Conference on Language Documentation and Conservation, University of
Hawai’i, Honolulu, 28 February–7 March
2013. [URL]
LaPolla, Randy J. 2020. “Forward to the
Past: Modernizing Linguistic Typology by Returning to its Roots.” Asian
Languages and
Linguistics 1 (1): 147–167.
Le Guen, Olivier, Josefina Safar, and Marie Coppola (eds). 2020. Emerging
Sign Languages of the Americas. Berlin, Boston: De Gruyter Mouton.
Levinson, Stephen C. 2003. Space in Language and
Cognition: Explorations in Cognitive
Diversity. Cambridge: Cambridge University Press.
Levinson, Stephen C., Sarah Cutfield, Michael J. Dunn, Nick J. Enfield, and Sérgio Meira (eds). 2018. Demonstratives
in Cross-Linguistic
Perspective. Cambridge: Cambridge University Press.
Linell, Per. 2004. The
Written Language Bias in Linguistics: Its Nature, Origins and
Transformations. London: Routledge.
Margutti, Piera, Liisa Tainio, Paul Drew, and Véronique Traverso. 2018. “Invitations
and Responses across Different Languages: Observations on the Feasibility and Relevance of a Cross-linguistic
Comparative Perspective on the Study of Actions.” Journal of
Pragmatics 125: 52–61.
McNeill, David. 1992. Hand
and Mind: What Gestures Reveal About
Thought. Chicago: The University of Chicago Press.
Meakins, Felicity, Jennifer Green, and Myfany Turpin. 2018. Understanding
Linguistic
Fieldwork. London: Routledge.
Mesh, Kate. 2021. “It’s
as Far as the Arm Can Raise: Pointing Height Marks Target Distance among the San Juan Quiahije
Chatino.” Lingua 259: 103099.
Mondada, Lorenza. 2008. “Using
Video for a Sequential and Multimodal Analysis of Social Interaction: Videotaping Institutional Telephone
Calls.” Forum: Qualitative Social
Research 9 (3): 39.
Ochs, Elinor. 1988. Culture
and Language Development: Language Acquisition and Language Socialization in a Samoan
Village. Cambridge: Cambridge University Press.
Ong, Walter J. 1982. Orality and Literacy:
The Technologizing of the Word. London, New York: Routledge.
Pakendorf, Brigitte, and Françoise Rose (eds). 2025. Fillers:
Hesitatives and
Placeholders. Berlin: Language Science Press.
Parry, Ruth. 2010. “Video-based
Conversation Analysis.” In The SAGE Handbook of
Qualitative Methods in Health Research, ed. by Ivy Bourgeault, Robert Dingwall, and Raymond De Vries, 373–396. London: Sage.
Perniss, Pamela, Roland Pfau, and Markus Steinbach (eds). 2007. Visible
Variation: Comparative Studies on Sign Language Structure. Berlin, New York: De Gruyter Mouton.
Rarrick, Samantha. 2025. “Bridging
Signed Language Documentation and Spoken Language Documentation.” Language
Documentation and
Conservation 19: 24–39. [URL]
Sacks, Harvey. 1992. Lectures
on Conversation, Vol. 1 and 2, ed.
by Gail Jefferson. Oxford: Blackwell Publishing.
Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson. 1974. “A
Simplest Systematics for the Organization of Turn-taking for
Conversations.” Language 50 (4.1): 696–735.
Schegloff, Emanuel A. 1982. “Discourse as an
Interactional Achievement: Some Uses of ‘uh huh’ and Other Things that Come between
Sentences.” In Analyzing Discourse: Text and
Talk, ed. by Deborah Tannen, 71–93. Washington, D.C.: Georgetown University Press.
1998. “Body
Torque.” Social
Research 65 (3): 535–596. [URL]
2009. “A Practice for
(Re-)Exiting a Sequence: And/But/So + Uh(m) +
Silence.” In Language in Life, and a Life in
Language: Jacob Mey — A Festschrift, ed. by Bruce Fraser, and Ken Turner, 365–374. Bingly: Emerald.
Schegloff, Emanuel A., Gail Jefferson, and Harvey Sacks. 1977. “The
Preference for Self-Correction in the Organization of Repair in
Conversation.” Language 53 (2): 361–382.
Schultze-Berndt, Eva. 2015. “Language
Documentation.” In Syntax: Theory and Analysis: An
International Handbook, Vol. 3, ed.
by Tibor Kiss, and Artemis Alexiadou, 2063–2094. Berlin, Boston: De Gruyter Mouton.
Seyfeddinipur, Mandana. 2004. “Meta-discursive
Gestures from Iran: Some Uses of the ‘Pistol
Hand’.” In The Semantics and Pragmatics of Everyday
Gestures, ed. by Cornelia Müller, and Roland Posner, 205–216. Berlin: Weidler.
. 2012. “Reasons
for Documenting Gestures and Suggestions for How to Go About
It.” In The Oxford Handbook of Linguistic
Fieldwork, ed. by Nick Thieberger, 147–165. Oxford: Oxford University Press.
Seyfeddinipur, Mandana, Felix Ameka, Lissant Bolton, Jonathan Blumtritt, Brian Carpenter, Hilaria Cruz, Sebastian Drude, Patience L. Epps, Vera Ferreira, Ana Vilacy Galucio, Birgit Hellwig, Oliver Hinte, Gary Holton, Dagmar Jung, Irmgarda Kasinskaite Buddeberg, Manfred Krifka, Susan Kung, Miyuki Monroig, Ayu’nwi Ngwabe Neba, Sebastian Nordhoff, Brigitte Pakendorf, Kilu von Prince, Felix Rau, Keren Rice, Michael Riessler, Vera Szoelloesi-Brenig, Nick Thieberger, Paul Trilsbeek, Hein van der Voort, and Tony Woodbury. 2019. “Public
Access to Research Data in Language Documentation: Challenges and Possible
Strategies.” Language Documentation and
Conservation 13: 545–563. [URL]
Seyfeddinipur, Mandana, and Felix Rau. 2020. “Keeping
it Real: Video Data in Language Documentation and Language Archiving.” Language
Documentation and
Conservation 14: 503–519. [URL]
Sherzer, Joel. 1973. “Verbal
and Nonverbal Deixis: The Pointed Lip Gesture among the San Blas
Cuna.” Language in
Society 2 (1): 117–131.
Singleton, Jenny L., Amber J. Martin, and Gary Morgan. 2015. “Ethics,
Deaf-friendly Research, and Good Practice When Studying Sign
Languages.” In Research Methods in Sign Language
Studies: A Practical Guide, ed. by Eleni Orfanidou, Bencie Woll, and Gary Morgan, 5–20. Oxford: Wiley-Blackwell?.
Streeck, Jürgen. 2009. Gesturecraft:
The Manu-facture of
Meaning. Amsterdam: John Benjamins.
Swerts, Marc. 1998. “Filled
pauses as markers of discourse structure.” Journal of
Pragmatics 30 (4): 485–496.
van Loon, Ester, Roland Pfau, and Markus Steinbach. 2014. “The
Grammaticalization of Gestures in Sign
Languages.” In Body — Language — Communication: An
International Handbook on Multimodality in Human
Interaction, Vol. 2, ed.
by Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill, and Jana Bressem, 2133–2149. Berlin, Boston: De Gruyter Mouton.
Vigliocco, Gabriella, Pamela Perniss, and David Vinson. 2014. “Language
as a Multimodal Phenomenon: Implications for Language Learning, Processing and
Evolution.” Philosophical Transactions of the Royal Society
B 369 (1651): 20130292.
Wegener, Claudia, and Jana Bressem. 2019. “Sharing
the Load — The Interplay of Verbal and Gestural Negation in
Savosavo.” Presentation, LingCologne 2019,
University of Cologne, Cologne, 6–7 June
2019.
Wilkins, David. 2003. “Why
Pointing with the Index Finger is Not a Universal (in Sociocultural and Semiotic
Terms).” In Pointing: Where Language, Culture and
Cognition Meet, ed. by Sotaro Kita, 171–215. Mahwah, NJ: Laurence Erlbaum.
Woodbury, Anthony C. 2003. “Defining
Documentary Linguistics.” In Language Documentation
and Description, Vol. 1, ed.
by Peter K. Austin, 35–51. London: SOAS.
2011. “Language
Documentation.” In The Cambridge Handbook of
Endangered Languages, ed. by Peter K. Austin, and Julia Sallabank, 159–186. Cambridge: Cambridge University Press.
Woodward, James. 2014. Documentation
of Hawaii Sign Language: Building the Foundation for Documentation, Conservation, and Revitalization of
Endangered Pacific Island Sign Languages. Endangered Languages Archive. [URL]
Wulff, Mareike. 2023. Documenting
Lhokpu Language in Bhutan and the Endangered Yarn Production
Practices. Endangered Languages Archive. [URL]
Zeshan, Ulrike (ed). 2006. Interrogative
and Negative Constructions in Sign
Language. Nijmegen: Ishara Press.
. 2013. “Sign
Languages.” In The World Atlas of Language Structures
Online, v2020.4, ed. by Matthew S. Dryer, and Martin Haspelmath. Zenodo. (available online at [URL]).
Zeshan, Ulrike, and Pamela Perniss (eds). 2008. Possessive
and Existential Constructions in Sign
Languages. Nijmegen: Ishara Press.
