Cover not available

Article published In: Revista Española de Lingüística Aplicada/Spanish Journal of Applied Linguistics
Vol. 37:2 (2024) ► pp.486–517

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

A protocol for the annotation of evaluative stance and metaphor across four discourse genres

ORCiD logo with link | Universidad Autónoma de Madrid
ORCiD logo with link | Universidad de La Rioja
ORCiD logo with link | Universidad de Valladolid
ORCiD logo with link | Universidad Complutense de Madrid
ORCiD logo with link | Universidad Complutense de Madrid
ORCiD logo with link | Universidad Complutense de Madrid
ORCiD logo with link | Universidad de Cantabria
Published online: 12 March 2024
DOI logo with link
https://doi.org/10.1075/resla.21058.hid
Abstract
The present article contributes to research on evaluation by addressing two complementary objectives: first, we present a protocol for the identification and annotation of evaluation in English discourse and, second, we show the results of the implementation of the protocol in the annotation of evaluation in a sample of a corpus of four genres. We first describe the protocol by discussing the theoretical and methodological grounding of the annotation scheme, the criteria, the categories, the steps for the implementation of the protocol and an illustrative example of the application of the protocol to a short extract. We subsequently provide the preliminary results of a pilot study with the frequency of evaluative expressions across the four genres. Results show that while adjectives and non-metaphoric evaluative expressions are overall more frequent, there are differences regarding the preference for positive or negative value and regarding the frequency of function.
Resumen
Este artículo contribuye a la investigación sobre la evaluación al abordar dos objetivos complementarios: en primer lugar, presentamos un protocolo para la identificación y anotación de la evaluación en el discurso en lengua inglesa; y, en segundo lugar, mostramos los resultados obtenidos al implementar dicho protocolo en una muestra de un corpus de cuatro géneros discursivos. Comenzamos con la descripción del protocolo y la fundamentación teórica y metodológica del mismo, así como los criterios, los pasos para la implementación del protocolo y un ejemplo ilustrativo de su aplicación a un texto breve. A continuación, presentamos los resultados preliminares de un estudio piloto sobre la frecuencia de las expresiones evaluativas en cuatro géneros discursivos. Los resultados muestran que los adjetivos y las expresiones evaluativas no metafóricas son más frecuentes, pero hay diferencias en los diferentes géneros en la frecuencia de la evaluación positiva o negativa y la frecuencia de las funciones.
Article outline
  • 1.Introduction
  • 2.A protocol for the identification and annotation of evaluation in discourse
    • 2.1Evaluation and stance: Theoretical grounding and methodological issues
    • 2.2Motivations for the design of the protocol
    • 2.3Criteria for identifying evaluative stance
    • 2.4Categories
      • 2.4.1Part of speech
      • 2.4.2Function
      • 2.4.3Metaphoricity
      • 2.4.4Value
    • 2.5Annotation procedure
    • 2.6Stages in the development and implementation of the protocol
  • 3.Implementing the protocol: An example
  • 4.Variation of evaluation across genres: A pilot study
    • 4.1Data
    • 4.2Method
    • 4.3Results and discussion
    • 4.4Examples from the different genres: Preliminary results from case studies
      • 4.4.1Newspaper opinion discourse
      • 4.4.2Political discourse
      • 4.4.3Press popularization articles
      • 4.4.4Fora
  • 5.Conclusions
  • Notes
  • References
References (35)
References
Alba-Juez, L. & G. Thompson. (2014). The many faces and phases of evaluation. In G. Thompson & L. Alba-Juez (Eds.) Evaluation in context (pp. 3–24). John Benjamins. Google Scholar logo with link to Google Scholar
Bednarek, M. (2009). Language patterns and ATTITUDE. Functions of Language, 16(2), 165–192. Google Scholar logo with link to Google Scholar
Bednarek, M., & Caple, H. (2014). Why do news values matter? Towards a new methodological framework for analysing news discourse in Critical Discourse Analysis and beyond. Discourse & Society, 25(2), 135–158. Google Scholar logo with link to Google Scholar
Biber, D., Johansson, S., Leech, G., Conrad, S., & Finegan, E. (1999). Longman grammar of spoken and written English. Longman Pearson.Google Scholar logo with link to Google Scholar
Deignan, A. (2010). The evaluative properties of metaphors. In G. Low, Z. Todd, A. Deignan, & L. Cameron (Eds.) Researching and applying metaphor in the real world (pp. 357–374). John Benjamins. Google Scholar logo with link to Google Scholar
Du Bois, J. W. (2007). The stance triangle. In R. Englebretson (Ed.) Stancetaking in discourse (pp. 139–182). John Benjamins. Google Scholar logo with link to Google Scholar
Englebretson, R. (Ed.). (2007). Stancetaking in discourse. John Benjamins. Google Scholar logo with link to Google Scholar
Fuoli, M. (2018). A stepwise method for annotating Appraisal. Functions of Language, 25(2), 229–258. Google Scholar logo with link to Google Scholar
Fuoli, M., & Hommerberg, C. (2015). Optimising transparency, reliability and replicability: Annotation principles and inter-coder agreement in the quantification of evaluative expressions. Corpora, 10(3), 315–349. Google Scholar logo with link to Google Scholar
Fuoli, M., Littlemore, J., & Turner, S. (2021). Sunken ships and screaming banshees: Metaphor and evaluation in film reviews. English Language and Linguistics, 26(1), 1–29. Google Scholar logo with link to Google Scholar
Hidalgo-Downing, L. (2016). Grammar and evaluation. In C. Chapelle (Ed.) The Applied Linguistics Encyclopedia. Online publication. [Accessed 2 October 2021].Google Scholar logo with link to Google Scholar
Hidalgo-Downing, L., & Pérez-Sobrino, P. (2022). Developing an annotation protocol for evaluative stance and metaphor in discourse: Theoretical and methodological considerations. Text and Talk. Advanced online publication. Google Scholar logo with link to Google Scholar
(2023). ‘Pushing Britain off the precipice’: A CDA approach to negative evaluative stance in opinion articles on Brexit. In J. Marín-Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (Eds.) Stance, inter/subjectivity and identity in discourse (pp. 201–226). Peter Lang.Google Scholar logo with link to Google Scholar
Hidalgo Tenorio, E., & Benítez Castro, M. Á. (2020). The language of evaluation in the narratives by the Magdalene laundries survivors: The discourse of female victimhood. Applied Linguistics, 42(2), 315–341. Google Scholar logo with link to Google Scholar
Hunston, S. (2007). Using a corpus to investigate stance quantitatively and qualitatively. In R. Englebretson (Ed.) Stancetaking in discourse (pp. 27–48). John Benjamins. Google Scholar logo with link to Google Scholar
(2011). Corpus Approaches to Evaluation. Phraseology and Evaluative Language. Routledge.Google Scholar logo with link to Google Scholar
Hunston, S., & Su, H. (2019). Patterns, constructions, and local grammar: A case study of evaluation. Applied Linguistics, 40(4), 567–593. Google Scholar logo with link to Google Scholar
Hunston, S., & Thompson, G. (2000). Evaluation in text. Oxford University Press. Google Scholar logo with link to Google Scholar
Jaki, S., De Smedt, T., Gwóźdź, M., Panchal, R., Rossa, A., & De Pauw, G. (2019). Online hatred of women in the Incels. me forum: Linguistic analysis and automatic detection. Journal of Language Aggression and Conflict, 7(2), 240–268. Google Scholar logo with link to Google Scholar
Lorenzo-Dus, N., & Nouri, L. (2020). The discourse of the US alt-right online–a case study of the Traditionalist Worker Party blog. Critical Discourse Studies, 18(4), 1–19.Google Scholar logo with link to Google Scholar
Maíz-Arévalo, C., & Sánchez-Moya, A. (2023). “Histrionic, appalling, a major turkey”: The expression of evaluative stance in the discourse of online forums. In J. Marín Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (Eds.) Stance, inter/subjectivity and identity in discourse (pp. 249–269). Peter Lang.Google Scholar logo with link to Google Scholar
Martin, J. R., & White, P. R. R. (2005). The language of evaluation. Appraisal in English. Macmillan. Google Scholar logo with link to Google Scholar
Núñez-Perucha, B., & Filardo-Llamas, L. (2023). From “roaring lion” to “chlorinated chicken”: Evaluative stance and ideological positioning in a corpus of British political discourse. In J. Marín-Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (Eds.) Stance, inter/subjectivity and identity in discourse (pp. 227–248). Peter Lang.Google Scholar logo with link to Google Scholar
Pragglejazz Group (2007). A method for identifying metaphorically used words in discourse. Metaphor and Symbol, 22(1), 1–39. Google Scholar logo with link to Google Scholar
Prażmo, E. (2020). Foids are worse than animals. A cognitive linguistics analysis of dehumanizing metaphors in online discourse. Topics in Linguistics, 21(2), 16–27. Google Scholar logo with link to Google Scholar
Read, J., & Carroll, J. (2012). Annotating expressions of Appraisal in English. Language Resources and Evaluation, 46(3), 421–447. Google Scholar logo with link to Google Scholar
Simaki, V., Paradis, C., & Kerren, A. (2018). Evaluating stance-annotated sentences from the Brexit Blog Corpus: A quantitative linguistic analysis. ICAME journal, 42(1), 133–166. Google Scholar logo with link to Google Scholar
Simaki, V., Paradis, C., Skeppstedt, M., Sahlgren, M., Kucher, K., & Kerren, A. (2020). Annotating speaker stance in discourse: The Brexit Blog Corpus. Corpus Linguistics and Linguistic Theory, 16(2), 215–248.Google Scholar logo with link to Google Scholar
Steen, G., Dorst, Aletta G., Herrmann, J. B., Kaal, A. A., & Krennmayr, T. (2010). Metaphor in usage. Cognitive Linguistics, 21(4), 765–796. Google Scholar logo with link to Google Scholar
Taboada, M., & Carretero, M. (2010). Contrastive analyses of evaluation in text: Key issues in the design of an annotation system for attitude applicable to consumer reviews in English and Spanish. Linguistics and the Human Sciences, 6(1–3), 275–295. Google Scholar logo with link to Google Scholar
Thompson, G. (2014). Affect and emotion, target-value mismatches, and Russian dolls: Refining the appraisal model. In L. Alba-Juez, & G. Thompson (Eds.) Evaluation in context (pp. 47–66). John Benjamins. Google Scholar logo with link to Google Scholar
Van Dijk, T. A. (1995). Power and the news media. In D. L. Paletz (Ed.) Political communication and action (pp. 9–36). Hampton Press.Google Scholar logo with link to Google Scholar
van Leeuwen, T. (2008). Discourse and Practice: New Tools for Critical Discourse Analysis. Oxford University Press. Google Scholar logo with link to Google Scholar
Williams Camus, J. T. (2023). Evaluative stance in science popularisations in the English press. In J. Marín-Arrese, L. Hidalgo-Downing, & J. R. Zamorano Mansilla (eds.) Stance, inter/subjectivity and identity in discourse (pp. 271–293). Peter Lang.Google Scholar logo with link to Google Scholar
Wodak, R. & M. Meyer (Eds.) (2015). Methods of Critical Discourse Studies. Sage.Google Scholar logo with link to Google Scholar
Cited by (6)

Cited by six other publications

Domínguez Romero, Elena & Jelena Bobkina
2025. Stance de-centring in Intercultural Virtual Societal Challenge (IVSC) e-modules for anti-racist foreign language education. Innovations in Education and Teaching International ► pp. 1 ff. DOI logo
Romero, Elena Domínguez & Jelena Bobkina
2024. Towards a stance de-centring model for the foreign language classroom: Training 21st-century agents of change. Thinking Skills and Creativity 54 ► pp. 101603 ff. DOI logo
Filardo-Llamas, Laura, Barbara De Cock, Philippe Hambye & Nadezda Shchinova
2025. “I am not populist”. Pragmatics and Society 16:5 ► pp. 653 ff. DOI logo
Foley, Jennifer
2025. Lost in a sea of highlight reels: The use of social media and mental health metaphors in online health blogs. Research in Corpus Linguistics 13:1 ► pp. 26 ff. DOI logo
Radoslava Trnavac & Katie J. Patterson
2025. The interplay of conceptual metaphors and evaluation in press reports on the AUKUS agreement. Russian Journal of Linguistics 29:3 ► pp. 560 ff. DOI logo
[no author supplied]
2025. Evaluative Discourse Metaphor in Online Communities [Discourse Approaches to Politics, Society and Culture, 106], DOI logo

This list is based on CrossRef data as of 30 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue