Cover not available

Article published In: Functions of Language
Vol. 22:3 (2015) ► pp.297–331

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
authorization required

Adjectives, communities, and taxonomies of evaluative meaning

 | University of Tsukuba
 | University of Birmingham
Published online: 10 December 2015
DOI logo with link
https://doi.org/10.1075/fol.22.3.01mil
From a corpus consisting of free text comments submitted to the website RateMyProfessors.com, adjectives describing people have been identified and clustered using Principal Components Analysis. This follows the Meaning Extraction method used by Chung & Pennebaker (2007). The outcome is a set of seven factors, each of which is interpretable as representing a dimension of meaning along which individual professors have been evaluated. These dimensions are in turn discussed using Martin & White’s (2005) parameters of Judgement and Appreciation, and Coffin’s (2002) concept of evaluative voices. It is argued that contributors to RateMyProfessors.com have available three distinct voices: ‘novice intellectual’, ‘consumer’, and ‘subordinate’. The paper demonstrates how a bottom-up, statistical technique may be used to provide the initial data for identifying evaluative parameters. It raises the possibility that such parameters may be specific to individual discourse communities. It therefore offers a complementary and problematizing alternative to top-down, researcher-driven taxonomies of evaluative meaning.
References (66)
Argamon, Shlomo, Moshe Koppel, James W. Pennebaker & Jonathan Schler. 2007. Mining the blogosphere: Age, gender and the varieties of self-expression. First Monday 121. Available at: <[URL]>. Date accessed: 12 August 2014. Google Scholar logo with link to Google Scholar
Bednarek, Monika. 2006. Evaluation in media discourse. London: Continuum.Google Scholar logo with link to Google Scholar
. 2008b. Emotion talk across corpora. London: Palgrave. Google Scholar logo with link to Google Scholar
Biber, Douglas. 1988. Variation across speech and writing. Cambridge: CUP. Google Scholar logo with link to Google Scholar
. 1993. The multi-dimensional approach to linguistic analyses of genre variation: An overview of methodology and findings. Computers and the Humanities 261. 331–345.Google Scholar logo with link to Google Scholar
. 2006. Stance in spoken and written university registers. Journal of English for Academic Purposes 51. 97–116. Google Scholar logo with link to Google Scholar
Biber, Douglas & Susan Conrad. 2009. Register, genre, and style. Cambridge: CUP. Google Scholar logo with link to Google Scholar
Biber, Douglas & Edward Finegan. 1988. Adverbial stance types in English. Discourse Processes 111. 1–34. Google Scholar logo with link to Google Scholar
Brown, Michael J., Michelle Baillie & Shawndel Fraser. 2009. Rating RateMyProfessors.com: A comparison of online and official student evaluations of teaching. College Teaching 57(2). 89–92. Google Scholar logo with link to Google Scholar
Charles, Maggie. 2006. The construction of stance in reporting clauses: A cross-disciplinary study of theses. Applied Linguistics 271. 492–518. Google Scholar logo with link to Google Scholar
. 2007. Argument or evidence? Disciplinary variation in the use of the noun-that pattern in stance construction. English for Specific Purposes 261. 203–218. Google Scholar logo with link to Google Scholar
Chen, Yining & Leon B. Hoshower. 2003. Student evaluation of teaching effectiveness: An assessment of student perception and motivation. Assessment and Evaluation in Higher Education 281. 71–88. Google Scholar logo with link to Google Scholar
Chung, Cindy K. & James W. Pennebaker. 2008. Revealing dimensions of thinking in open-ended self-descriptions: An automated meaning extraction method for natural language. Journal of Research in Personality 42(1). 96–132. Google Scholar logo with link to Google Scholar
Coffin, Caroline. 2002. The voices of history: Theorising the interpersonal semantics of historical discourses. Text 22(4). 503–528. Google Scholar logo with link to Google Scholar
Coffin, Caroline & Kieran O’Halloran. 2006. The role of appraisal and corpora in detecting covert evaluation. Functions of Language 131. 77–110. Google Scholar logo with link to Google Scholar
Conrad, Susan & Douglas Biber. 2000. Adverbial marking of stance in speech and writing. In Susan Hunston & Geoff Thompson (eds.), 56–73.Google Scholar logo with link to Google Scholar
Crawley, Michael J. 2007. The R book. New Jersey, NJ: Wiley. Google Scholar logo with link to Google Scholar
Davison, Elizabeth & Jammie Price. 2009. How do we rate? An evaluation of online student evaluations. Assessment & Evaluation in Higher Education 34(1). 51–65. Google Scholar logo with link to Google Scholar
Drasovean, Anda & Caroline Tagg. Submitted. Evaluation and its community-building role.
Dunbar, George. 2012. Adaptive Resonance Theory as a model of polysemy and vagueness in the cognitive lexicon. Cognitive Linguistics 23(3). 507–537. Google Scholar logo with link to Google Scholar
Eggins, Suzanne & Diane Slade. 1997. Analysing casual conversation. London: Cassell.Google Scholar logo with link to Google Scholar
Felton, James, Peter T. Koper, John Mitchell & Michael Stinson. 2008. Attractiveness, easiness and other issues: Student evaluations of professors on Ratemyprofessors.com. Assessment & Evaluation in Higher Education 33(1). 45–61. Google Scholar logo with link to Google Scholar
Felton, James, John Mitchell & Michael Stinson. 2004. Web-based student evaluations of professors : The relations between perceived quality, easiness and sexiness. Assessment & Evaluation in Higher Education 29(1). 91–108. Google Scholar logo with link to Google Scholar
Francis, Gill. 1995. Corpus-driven grammar and its relevance to the learning of English in a cross-cultural situation. Unpublished manuscript.
Fuoli, Matteo. 2012. Assessing social responsibility: A quantitative analysis of Appraisal in BP’s and IKEA’s social reports. Discourse and Communication 6(1). 55–81. Google Scholar logo with link to Google Scholar
Garside, Roger & Nicholas Smith. 1997. A hybrid-grammatical tagger: CLAWS 4. In Roger Garside, Geoffrey Leech & Anthony McEnery (eds.), Corpus annotation: Linguistic information from computer text corpora, 102–121. Harlow: Longman.Google Scholar logo with link to Google Scholar
Gee, James Paul. 2005. Semiotic social spaces and affinity spaces: From The Age of Mythology to today’s schools. In David Barton & Karin Tusting (eds.), Beyond communities of practice: Language power and social context, 214–232. Cambridge: CUP. Google Scholar logo with link to Google Scholar
Halliday, M.A.K. 1978. Language as social semiotic: The social interpretation of language and meaning. London: Arnold.Google Scholar logo with link to Google Scholar
. 1994. An introduction to functional grammar, 2nd edition. London: Arnold.Google Scholar logo with link to Google Scholar
Hardy, Jack A. & Ute Römer. 2013. Revealing disciplinary variation in student writing: A multi-dimensional analysis of the Michigan Corpus of Upper-level Student papers (MICUSP). Corpora 81. 183–207. Google Scholar logo with link to Google Scholar
Hiltunen, Turo. 2010. Grammar and disciplinary culture: A corpus-based study. Helsinki: University of Helsinki.Google Scholar logo with link to Google Scholar
Hunston, Susan. 1993. Evaluation and ideology in scientific writing. In M. Ghadessy (ed.), Register analysis: Theory and practice, 57–74. London: Pinter.Google Scholar logo with link to Google Scholar
. 2011. Corpus approaches to evaluation: Phraseology and evaluative meaning. London: Routledge.Google Scholar logo with link to Google Scholar
Hunston, Susan & Geoff Thompson (eds.). 2000. Evaluation in text: Authorial stance and the construction of discourse. Oxford: OUP.Google Scholar logo with link to Google Scholar
Hyland, Ken. 2005. Metadiscourse. London: Continuum.Google Scholar logo with link to Google Scholar
. 2009. Academic discourse. London: Continuum.Google Scholar logo with link to Google Scholar
. 2012. Disciplinary identities: Individuality and community in academic discourse. Cambridge: CUP.Google Scholar logo with link to Google Scholar
Iedema, Rick, Susan Feez & Peter R.R. White. 1994. Media literacy. Sydney: Disadvantaged Schools Program, NSW Department of School Education.Google Scholar logo with link to Google Scholar
Knight, Naomi. 2008. ‘Still cool…and american too!’: An SFL analysis of deferred bonds in internet messaging humour. In Nina Nørgaard (ed.), Systemic Functional Linguistics in use, Odense Working Papers in Language and Communication, 481–502. Retrieved 3rd January 2014 from [URL].Google Scholar logo with link to Google Scholar
Kramer, Adam D.I. & Cindy K. Chung. 2011. Dimensions of self-expression in Facebook status updates. Proceedings of the 2008 International Conference on Weblogs and Social Media (ICWSM2011), Palo Alto, CA, July 2011. 169–176.
Kristiansen, Gitte & Dirk Geeraerts. 2013. Contexts and usage in Cognitive Sociolinguistics. Journal of Pragmatics 521. 1–4. Google Scholar logo with link to Google Scholar
Liu Bing. 2010. Sentiment analysis and subjectivity. In Nitin Indurkhya & Fred J. Damerau (eds.), Handbook of Natural Language Processing, 2nd edn, 627–665. Boca Raton, FL: Chapman & Hall / CRC.Google Scholar logo with link to Google Scholar
Liu Dilin. 2013. Salience and construal in the use of synonymy: A study of two sets of near-synonymous nouns. Cognitive Linguistics 24(1). 67–113. Google Scholar logo with link to Google Scholar
Marsh, Herbert W. 2007. Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases and usefulness. In Raymond P. Perry & John C. Smart (eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective, 319–383. Berlin: Springer. Google Scholar logo with link to Google Scholar
Martin, James R. 2000. Beyond exchange: Appraisal systems in English. In Susan Hunston & Geoff Thompson (eds.), 142–175.Google Scholar logo with link to Google Scholar
Martin, James R. & Peter R.R. White. 2005. The language of evaluation: Appraisal in English. London: Palgrave. Google Scholar logo with link to Google Scholar
Norman, Geoffrey R. & David L. Streiner. 1994. Biostatistics: The bare essentials. St Louis: Mosby.Google Scholar logo with link to Google Scholar
Polanyi, Livia. 1985. Telling the American story: From the structure of linguistic texts to the grammar of a culture. Norwood, NJ: Ablex.Google Scholar logo with link to Google Scholar
Ramírez-Esparza, Nairan, Cindy K. Chung, Ewa Kacewicz & James W. Pennebaker. 2008. The psychology of word use in depression forums in English and in Spanish. Testing two text analytic approaches. Proceedings of the 2008 International Conference on Weblogs and Social Media (ICWSM2008) , Menlo Park, CA, April 2008. 102–108.
RateMyProfessors.com. 2013. About us. Retrieved August 6, 2013, from [URL]
Revelle, William. 2013. Psych. Procedures for personality and psychological research, Northwestern University, Evanston, Illinois, USA, [URL]. Version = 1.3.2Google Scholar logo with link to Google Scholar
Sanders, Shane, Bhavneet Walia, Joel Potter & Kenneth W. Linna. 2011. Do more online instructional ratings lead to better prediction of instructor quality? Practical Assessment, Research & Evaluation 16(2). 1–6.Google Scholar logo with link to Google Scholar
Schiffrin, Deborah. 1984. How a story says what it means and does. Text 41. 313–346. Google Scholar logo with link to Google Scholar
Schönefeld, Doris. 2013. ‘It is…quite common for theoretical predictions to go untested (BNC_CMH)’. A register-specific analysis of the English go un-V-ed construction. Journal of Pragmatics 521: 17–33. Google Scholar logo with link to Google Scholar
Shevlin, Mark, Philip Banyard, Mark Davies & Mark Griffiths. 2000. The validity of student evaluation of teaching in higher education: Love me, love my lectures? Assessment and Evaluation in Higher Education 251. 397–405. Google Scholar logo with link to Google Scholar
Stenglin, Maree K. 2004. Packaging curiosities: Towards a grammar of three-dimensional space. Sydney: University of Sydney PhD thesis.Google Scholar logo with link to Google Scholar
Su, Hang. In preparation. Adjectives of judgement in biographical discourse: A corpus study. Birmingham: University of Birmingham PhD thesis.
Teubert, Wolfgang. 2010. Meaning, discourse and society. Cambridge: CUP. Google Scholar logo with link to Google Scholar
Thompson, Geoff & Susan Hunston. 2000. Evaluation: An introduction. In Susan Hunston & Geoff Thompson (eds.), 1–27.Google Scholar logo with link to Google Scholar
Tuggy, David. 1993. Ambiguity, polysemy and vagueness. Cognitive Linguistics 41. 273–290. Google Scholar logo with link to Google Scholar
Turney, Peter D. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , Philadelphia, July 2002. 417–424.
Wolf, Markus, Cindy K. Chung & Hans Kordy. 2010. Inpatient treatment to online aftercare: E-mailing themes as a function of therapeutic outcomes. Psychotherapy research: Journal of the Society for Psychotherapy Research 20(1). 71–85. Google Scholar logo with link to Google Scholar
Zappavigna, Michelle. 2011. Ambient affiliation: A linguistic perspective on Twitter. New Media & Society 13(5). 788–806. Google Scholar logo with link to Google Scholar
Cited by (20)

Cited by 20 other publications

Fetterman, Adam K., Nicholas D. Evans, Eriksen P. Ravey, Perla R. Henderson, Bao Han L. Tran & Ryan L. Boyd
2025. The Topics of Nostalgic Recall: The Benefits of Nostalgia Depend on the Topics That One Recalls. Social Psychological and Personality Science 16:4 ► pp. 444 ff. DOI logo
He, Juan
2025. Positioning and Footing in Chinese Social Media News Comments: The Case of the Mercedes Car Quality Dispute. WORD 71:1 ► pp. 20 ff. DOI logo
Cheng, Shi
2024. A review of interpersonal metafunction studies in systemic functional linguistics (2012–2022). Journal of World Languages 10:3 ► pp. 623 ff. DOI logo
Etaywe, Awni
2024. Discursive pragmatics of justification in terrorist threat texts: Victim-blaming, denying, discrediting, legitimating, manipulating, and retaliation. Discourse & Society 35:6 ► pp. 725 ff. DOI logo
Etaywe, Awni & Michele Zappavigna
2022. Identity, ideology and threatening communication. Journal of Language Aggression and Conflict 10:2 ► pp. 315 ff. DOI logo
Etaywe, Awni & Michele Zappavigna
2024. The role of social affiliation in incitement: A social semiotic approach to far-right terrorists’ incitement to violence. Language in Society 53:4 ► pp. 623 ff. DOI logo
Ho, Mei-ching
2022. A corpus-pragmatics approach to evaluation in professor reviews. Concentric. Studies in Linguistics 48:1 ► pp. 114 ff. DOI logo
Foxman, Maxwell, David M Markowitz & Donna Z Davis
2021. Defining empathy: Interconnected discourses of virtual reality’s prosocial impact. New Media & Society 23:8 ► pp. 2167 ff. DOI logo
Markowitz, David M.
2021. The Meaning Extraction Method: An Approach to Evaluate Content Patterns From Large-Scale Language Data. Frontiers in Communication 6 DOI logo
Li, Jing, Lei Lei & Le Cheng
2020. Mapping Evaluation, Appraisal and Stance in Discourse (2000–2015): A Bibliometric Analysis. Glottotheory 10:1-2 ► pp. 31 ff. DOI logo
Markowitz, David M. & Paul Slovic
2020. Social, psychological, and demographic characteristics of dehumanization toward immigrants. Proceedings of the National Academy of Sciences 117:17 ► pp. 9260 ff. DOI logo
Goddard, Cliff, Maite Taboada & Radoslava Trnavac
2019. The semantics of evaluational adjectives. Functions of Language 26:3 ► pp. 308 ff. DOI logo
Ikizer, Elif G., Nairán Ramírez-Esparza & Ryan L. Boyd
2019. #sendeanlat (#tellyourstory): Text Analyses of Tweets About Sexual Assault Experiences. Sexuality Research and Social Policy 16:4 ► pp. 463 ff. DOI logo
Perić, Marija & Nikolina Miletić
2019. Pejorativa in deutschen und kroatischen Leserkommentaren auf Terroranschläge im Internet. Jezikoslovni zapiski 25:1 DOI logo
Su, Hang & Susan Hunston
2019. Adjective complementation patterns and judgement: Aligning lexical-grammatical and discourse-semantic approaches in appraisal research. Text & Talk 39:3 ► pp. 415 ff. DOI logo
Su, Hang & Susan Hunston
2019. Language patterns and attitude revisited. Functions of Language 26:3 ► pp. 343 ff. DOI logo
Blackburn, Kate G., Gamze Yilmaz & Ryan L. Boyd
2018. Food for thought: Exploring how people think and talk about food online. Appetite 123 ► pp. 390 ff. DOI logo
Huntley, Selene J., Michaela Mahlberg, Viola Wiegand, Yves van Gennip, Hui Yang, Rachel S. Dean & Marnie L. Brennan
2018. Analysing the opinions of UK veterinarians on practice-based research using corpus linguistic and mathematical methods. Preventive Veterinary Medicine 150 ► pp. 60 ff. DOI logo
Su, Hang
2016. How products are evaluated? Evaluation in customer review texts. Language Resources and Evaluation 50:3 ► pp. 475 ff. DOI logo
[no author supplied]
2021. The Sociopragmatics of Stance [Pragmatics & Beyond New Series, 329], DOI logo

This list is based on CrossRef data as of 9 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue