Cover not available

Article published In: Language, Context and Text
Vol. 4:2 (2022) ► pp.335–374

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Emotion in speech

A systemic functional semiotic approach to the vocalisation of affect

ORCiD logo with link | University of Wollongong, Australia | Universidad Nacional de La Pampa
ORCiD logo with link | University of Wollongong, Australia
ORCiD logo with link | University of Wollongong, Australia
Published online: 29 September 2022
DOI logo with link
https://doi.org/10.1075/langct.21012.ari
Abstract
This paper explores how emotion is conveyed in spoken language based on a sample of three stories for children read aloud by a trained storyteller. It draws on both Martin & White’s appraisal framework (Martin, James R. & Peter R. R. White. 2005. The language of evaluation. Appraisal in English. Houndmills, UK: Palgrave Macmillan. Google Scholar logo with link to Google Scholar) and a systematic account of vocal features. Interpreting and profiling emotion is a challenge both for researchers interested in spoken language and for English-as-an-additional-language teachers, when working to improve students’ spoken expression of emotion. Underpinning this challenge is the absence of a unified theoretical stance and a consequent lack of shared analytical tools for describing how speakers use semiotic resources to convey emotion in their speech. A systemic functional semiotic approach can make visible the co-patterning of wordings and vocalisations that express emotion.
Article outline
  • 1.Introduction
  • 2.The study
    • 2.1Methodology
      • 2.1.1Data collection and data preparation
    • 2.2Theoretical framework for the analysis
      • 2.2.1Emotion in speech: SFS perspective
      • 2.2.2Analysis of evaluative language
      • 2.2.3Analysis of vocal features
        • 2.2.3.1Suprasegmental features
        • 2.2.3.2Paralinguistic features: Definition and analytical technique
  • 3.Results and discussion
    • 3.1Preliminary vocal profiles of inscribed affect
      • 3.1.1Vocal profiles for inscribed affect and vocal emotion profiles outside the SFS tradition
      • 3.1.2Prediction value of profiles and teaching implications
      • 3.1.3Association between affect categories and paralinguistic features
      • 3.1.4Additional understandings gained from associating affect categories and vocal profiles
      • 3.1.5Distinguishing affect types: The example of ‘unhappy’
  • 4.Conclusions and pedagogical implications
  • Acknowledgements
  • Notes
  • References
References (60)
References
Ariztimuño, Lilian I. 2017. The expression of emotion in fairy tales: A multimodal approach to improve EFL students’ oral renderings. Phonetics Teaching and Learning Conference (PTLC2017) Proceedings, 11–15.Google Scholar logo with link to Google Scholar
Arnfield, Simon, Peter Roach, Jane Setter, Peter Greasley & Dave Horton. 1995. Emotional stress and speech tempo variation. ESCA-NATO Tutorial and research workshop on speech under stress, 13–15.Google Scholar logo with link to Google Scholar
Bednarek, Monika. 2008. Emotion talk in corpora. Basingstoke/New York: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Boersma, Paul & David Weenink. 2021. Praat: Doing phonetics by computer [Computer program]. Version 6.0. 381. [URL]
Bombelli, Griselda, Lidia R. Soler & María S. Waasaf. 2013. The language of Evaluation: Paralinguistic features as a phonological domain for Appraisal. DELTA Documentação de Estudos em Lingüística Teórica e Aplicada 29 (2). 267–280. Google Scholar logo with link to Google Scholar
Caldwell, David. 2014. The interpersonal voice: Applying appraisal to the rap and sung voice. Social Semiotics 24 (1). 40–55. Google Scholar logo with link to Google Scholar
Cruttenden, Alan. 2008. Gimson’s pronunciation of English (7th edition). London: Hodder Education.Google Scholar logo with link to Google Scholar
Crystal, David & Randolph Quirk. 1964. Systems of prosodic and paralinguistic features in English. The Hague: De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Douglas-Cowie, Ellen, Nick Campbell, Roddy Cowie & Peter Roach. 2003. Emotional speech: Towards a new generation of databases. Speech Communication 40 (1). 33–60. Google Scholar logo with link to Google Scholar
Eggins, Suzanne & Diana Slade. 2005. Analysing casual conversation. London: Equinox.Google Scholar logo with link to Google Scholar
Ekman, Paul. 1999. Basic emotions. In Tim Dalgleish & Mick Power (eds.), Handbook of cognition and emotion. Chichester: John Wiley & Sons. Google Scholar logo with link to Google Scholar
Germani, Miriam P. & Lucía I. Rivas. 2017. A genre approach to prosody: Teaching intonation from a discourse perspective. Phonetics Teaching and Learning Conference (PTLC2017) Proceedings, 49–53.Google Scholar logo with link to Google Scholar
Halliday, Michael A. K. 1970. A course in spoken English: Intonation. London: OUP.Google Scholar logo with link to Google Scholar
1978. Language as social semiotic: The social interpretation of language and meaning. London: Hodder Education.Google Scholar logo with link to Google Scholar
2003. On language and linguistics, volume 3 in the collected works of M.A.K Halliday. Edited by Jonathan J. Webster. London: Continuum.Google Scholar logo with link to Google Scholar
Halliday, Michael A. K. & William S. Greaves. 2008. Intonation in the grammar of English. Equinox.Google Scholar logo with link to Google Scholar
Halliday, Michael A. K. & Ruqaiya Hasan. 1985. Language, context, and text: Aspects of language in a social-semiotic perspective. Waurn Ponds, Victoria: Deakin University Press.Google Scholar logo with link to Google Scholar
Halliday, Michael A. K. & Christian M. I. M. Matthiessen. 2014. Halliday’s introduction to functional grammar. London: Routledge. Google Scholar logo with link to Google Scholar
Hasan, Ruqaiya. 1984. The nursery tale as a genre. Nottingham Linguistic Circular, 71–102.Google Scholar logo with link to Google Scholar
. 1985. Linguistics, language, and verbal art. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Humphrey, Sally & Shoshana Dreyfus. 2012. Exploring the interpretive genre in applied linguistics. Indonesian Journal of SFL 1 (2). 156–174.Google Scholar logo with link to Google Scholar
Jewitt, Carey, Jeff Bezemer & Kay O’Halloran. 2016. Introducing multimodality. London/New York: Routledge. Google Scholar logo with link to Google Scholar
Johnstone, Tom, Van Reekum, Carien M. & Klaus R. Scherer. 2001. Vocal expression correlates of appraisal processes. In Klaus R. Scherer, Angela Schorr & Tom Johnstone (eds.), Appraisal processes in emotion: Theory, methods, research, 271–284. New York: Oxford University Press.Google Scholar logo with link to Google Scholar
Juslin, Patrik N. & Klaus R. Scherer. 2005. Vocal expression of affect. In Jinni A. Harrigan, Robert Rosenthal & Klaus R. Scherer (eds.), The new handbook of methods in nonverbal behavior research. Oxford: Oxford University Press. Google Scholar logo with link to Google Scholar
Kress, Gunther R. 2010. Multimodality: A social semiotic approach to contemporary communication. New York: Routledge.Google Scholar logo with link to Google Scholar
Laukka, Petri, Hillary Anger Elfenbein, Nutankumar S. Thingujam, Thomas Rockstuhl, Frederick. K. Iraki, Wanda Chui & Jean Althoff. 2016. The expression and recognition of emotions in the voice across five nations: A lens model analysis based on acoustic features. Journal of Personality and Social Psychology 111 (5). 686–705. Google Scholar logo with link to Google Scholar
Lorette, Parnelle & Jean-Marc Dewaele. 2019. The relationship between bi/multilingualism, nativeness, proficiency and multimodal emotion recognition ability. International Journal of Bilingualism 23 (6). 1502–1516. Google Scholar logo with link to Google Scholar
Macken-Horarik, Mary. 2003. Appraisal and the special instructiveness of narrative. Text & Talk 23 (2). 285–312. Google Scholar logo with link to Google Scholar
Macken-Horarik, Mary & Anne Isaac. 2014. Appraising appraisal. In Geoff Thompson & Laura Alba-Juez (eds.), Evaluation in context. Amsterdam/Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Mackenzie, J. Lachlan & Laura Alba-Juez. 2019. Emotion in discourse. Amsterdam/Philadelphia: John Benjamins. Google Scholar logo with link to Google Scholar
Martin, James. R. 2017. The discourse semantics of attitudinal relations: Continuing the study of lexis. Russian Journal of Linguistics 21 (1). 22–47. Google Scholar logo with link to Google Scholar
Martin, James R. & David Rose. 2008. Genre relations: Mapping culture. London/Oakville: Equinox.Google Scholar logo with link to Google Scholar
Martin, James R. & Peter R. R. White. 2005. The language of evaluation. Appraisal in English. Houndmills, UK: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Martin, James R. & Michele Zappavigna. 2019. Embodied meaning: A systemic functional perspective on paralanguage. Functional Linguistics 6 (1). 1–33. Google Scholar logo with link to Google Scholar
Matthiessen, Christian M. I. M. 2009. Multisemiosis and context-based register typology. In Eija Ventola & Arsenio J. M. Guijarro (eds.), The world told and the world shown: Multisemiotic issues. 11–38. London: Palgrave Macmillan.Google Scholar logo with link to Google Scholar
Mills, Kathy A. & Len Unsworth. 2016. Multimodal Literacy: Oxford Research Encyclopedia of Education. Google Scholar logo with link to Google Scholar
. 2017. iPad animations: Powerful multimodal practices for adolescent literacy and emotional language. Journal of Adolescent & Adult Literacy 61 (6). 609–620. Google Scholar logo with link to Google Scholar
Noad, Betty & Georgina Barton. 2018. Emotion resonance and divergence: A semiotic analysis of music and sound in “The Lost Thing” an animated short film and “Elizabeth” a film trailer. Social Semiotics 30 (2). 206–224. Google Scholar logo with link to Google Scholar
O’Grady, Gerard. 2010. A grammar of spoken English: The intonation of increments. London: Continuum.Google Scholar logo with link to Google Scholar
Painter, Clare, James. R. Martin & Len Unsworth. 2013. Reading visual narratives: Image analysis of children’s picture books. Sheffield: Equinox.Google Scholar logo with link to Google Scholar
Pavlenko, Aneta. 2007. Emotions and multilingualism. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Rintell, Ellen. M. 1984. But how did you feel about that? The learner’s perception of emotion in speech. Applied Linguistics 5 (3). 255–264. Google Scholar logo with link to Google Scholar
Roach, Peter. 2000. Techniques for the phonetic description of emotional speech. In ISCA Tutorial and Research Workshop (ITRW) on Speech and Emotion. 53–59.Google Scholar logo with link to Google Scholar
. 2009. English phonetics and phonology: A practical course. Cambridge: Cambridge University Press.Google Scholar logo with link to Google Scholar
Roach, Peter, Richard Stibbard, Jane Osborne, Simon Arnfield & Jane Setter. 1998. Transcription of prosodic and paralinguistic features of emotional speech. Journal of the International Phonetic Association 28 (1–2). 83–94. Google Scholar logo with link to Google Scholar
Royce, Terry D. 2007. Multimodal communicative competence in second language contexts. In Terry D. Royce & Wendy L. Bowcher (eds.), New directions in the analysis of multimodal discourse. 361–390. Mahwah, NJ: Lawrence Erlbaum.Google Scholar logo with link to Google Scholar
Scherer, Klaus R. 2003. Vocal communication of emotion: A review of research paradigms. Speech Communication 40 (1–2). 227–256. Google Scholar logo with link to Google Scholar
2019. Acoustic patterning of emotion vocalizations. In Sascha Fruhholz & Pascal Belin (eds.), The Oxford handbook of voice perception. 61–91. Oxford: OUP.Google Scholar logo with link to Google Scholar
Scherer, Klaus R. & Heiner Ellgring. 2007. Multimodal expression of emotion: Affect programs or componential appraisal patterns? Emotion 7 (1). 158–171. Google Scholar logo with link to Google Scholar
Schuller, Björn, Anton Batliner, Stefan Steidl & Dino Seppi. 2011. Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge. Speech Communication 53 (9–10). 1062–1087. Google Scholar logo with link to Google Scholar
Smith, Bradley A. 2012. Speech and writing: Intonation within multimodal studies. In Kay. L. O’Halloran & Bradley A. Smith (eds.), Multimodal studies: Exploring issues and domains. 291–313. London: Routledge.Google Scholar logo with link to Google Scholar
Smith, Bradley A. & William S. Greaves. 2015. Intonation. In Jonathan J. Webster (ed.), The Bloomsbury companion to MAK Halliday. 291–313. London: Bloomsbury.Google Scholar logo with link to Google Scholar
Storynory. 2013. King Thrushbeard, The elves and the shoemaker, Rumpelstiltskin. Available at [URL]. (17 July, 2013.)
Styler, Will. 2015. Using Praat for linguistic research. University of Colorado at Boulder Phonetics Laboratory. [URL]
Tench, Paul. 1996. The intonation systems of English. London: Cassell.Google Scholar logo with link to Google Scholar
. 2014. A systemic functional model of the intonation of clauses in English. Readings in English Phonetics and Phonology (1). 261–296.Google Scholar logo with link to Google Scholar
Unsworth, Len & Kathy A. Mills. 2020. English language teaching of attitude and emotion in digital multimodal composition. Journal of Second Language Writing (47). 100712. Google Scholar logo with link to Google Scholar
van Leeuwen, Theo. 1999. Speech, music, sound. Basingstoke: Macmillan. Google Scholar logo with link to Google Scholar
Zappavigna, Michele, Chris Cléirigh, Paul Dwyer & James R. Martin. 2010. Visualising appraisal prosody. In Ahmar Mahboob & Naomi K. Knight (eds.), Appliable Linguistics: Texts, Contexts, and Meanings, 150–167. London & New York: Continuum.Google Scholar logo with link to Google Scholar
Cited by (4)

Cited by four other publications

Ariztimuño, Lilián I.
2025. How do we communicate emotions in spoken language? Modelling affectual vocal qualities in storytelling. Journal of World Languages DOI logo
Logi, Lorenzo
2025. War(n/m)ing the room: the role of vocal sound semiotics in nuancing comedian-audience relations. Journal of World Languages DOI logo
Santana-Monagas, Elisa, Samuel Falcon & Jaime Leon
2025. Emotions in teacher engaging messages: Their interplay with student outcomes. Teaching and Teacher Education 166 ► pp. 105195 ff. DOI logo
Fryer, Daniel Lees
2024. Feeling, thinking, acting. Language, Context and Text. The Social Semiotics Forum 6:1 ► pp. 122 ff. DOI logo

This list is based on CrossRef data as of 25 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue