Cover not available

Article published In: Concentric
Vol. 48:1 (2022) ► pp.114–146

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

A corpus-pragmatics approach to evaluation in professor reviews

 | University of Taipei
Published online: 2 May 2022
DOI logo with link
https://doi.org/10.1075/consl.21010.ho
Abstract
This study takes a corpus pragmatics approach to investigate the use of evaluative language in professor reviews, focusing on how review writers express evaluation through recurrent four-word sequences and the pragmatic functions of these sequences in positive and negative reviews on the website, RateMyProfessors​.com. Based on an analysis of a 2.9-million-word corpus of free text comments, the findings indicate that positive reviews used more 4-grams, and more varied types, than negative ones. The 4-word sequences were found to carry out four pragmatic functions: attitudinal evaluation, reader engagement, referential expression, and discourse organization. While a similar distribution of the main functional categories was observed among the top 100 4-grams in both review types, with evaluative clusters being most predominant, distinctive intra-genre variations were found in the ways review writers employed different functional sub-categories. For example, positive reviews relied heavily on hedged suggestion 4-grams to engage readers, whereas negative reviews used directive 4-grams for the same purpose. These findings suggest the important role of multi-word sequences in the understanding of evaluative resources in professor reviews of different valence types.
Article outline
  • 1.Introduction
  • 2.Evaluation across genres
  • 3.Theoretical framework
  • 4.Methodology
    • 4.1Corpora
    • 4.2Analysis
  • 5.Results
    • 5.1High-frequency four-grams in positive and negative reviews
    • 5.2Pragmatic functions of high-frequency 4-grams
      • 5.2.1Attitudinal evaluation 4-grams
      • 5.2.2Reader engagement 4-grams
      • 5.2.3Referential 4-grams
      • 5.2.4Discourse organizing 4-grams
  • 6.Discussion and conclusion
  • Acknowledgements
  • Notes
  • References
References (58)
References
Biber, Douglas, Stig Johansson, Geoffrey Leech, Susan Conrad, and Edward Finegan (eds.) 1999. Longman of Grammar of Spoken and Written English. Essex, UK: Pearson Education.Google Scholar logo with link to Google Scholar
Biber, Douglas, Susan Conrad, and Viviana Cortes. 2004. If you look at…: Lexical bundles in university teaching and textbooks. Applied Linguistics 25.3:371–405. Google Scholar logo with link to Google Scholar
Biber, Douglas, and Federica Barbieri. 2007. Lexical bundles in university spoken and written registers. English for Specific Purposes 26.3:263–286. Google Scholar logo with link to Google Scholar
Blackwell, Sarah E. 2010. Evaluation as a pragmatic act in Spanish film narratives. Journal of Pragmatics 42.11:2945–2963. Google Scholar logo with link to Google Scholar
Bychkovska, Tetyana, and Joseph J. Lee. 2017. At the same time: Lexical bundles in L1 and L2 university student argumentative writing. Journal of English for Academic Purposes 301:38–52. Google Scholar logo with link to Google Scholar
Cenni, Irene, and Patrick Goethals. 2017. Negative hotel reviews on TripAdvisor: A cross-linguistic analysis. Discourse, Context & Media 161:22–30. Google Scholar logo with link to Google Scholar
Clancy, Brian, and Anne O’Keeffe. 2015. Pragmatics. The Cambridge handbook of English corpus linguistics, ed. by Douglas Biber and Randi Reppen, 235–251. Cambridge, UK: Cambridge University Press. Google Scholar logo with link to Google Scholar
Cortes, Viviana. 2004. Lexical bundles in published and student disciplinary writing: Examples from history and biology. English for Specific Purposes 23.4:397–423. Google Scholar logo with link to Google Scholar
Cortes, Viviana, and Jack A. Hardy. 2013. Analyzing the semantic prosody and semantic preference of lexical bundles. Critical and Corpus-based Approaches to Intercultural Rhetoric, ed. by Diane Belcher and Gayle Nelson, 180–201. University of Michigan Press.Google Scholar logo with link to Google Scholar
Cunningham, Kelly J., and Stephanie Link. 2021. Video and text feedback on ESL writing: Understanding attitude and negotiating relationships. Journal of Second Language Writing 521:100797. Google Scholar logo with link to Google Scholar
Dam-Jensen, Helle, and Karen Korning Zethsen. 2007. Pragmatic patterns and the lexical system – A reassessment of evaluation in language. Journal of Pragmatics 39.9:1608–1623. Google Scholar logo with link to Google Scholar
Fortanet, Inmaculada. 2008. Evaluative language in peer review referee reports. Journal of English for Academic Purposes 7.1:27–37. Google Scholar logo with link to Google Scholar
Grabowski, Łukasz. 2015. Keywords and lexical bundles within English pharmaceutical discourse: A corpus-driven description. English for Specific Purposes 381:23–33. Google Scholar logo with link to Google Scholar
Gray, Bethany, and Douglas Biber. 2013. Lexical frames in academic prose and conversation. International Journal of Corpus Linguistics 18.1:109–136. Google Scholar logo with link to Google Scholar
Haugh, Michael. 2018. Corpus-based metapragmatics. Methods in Pragmatics, ed. by Andreas H. Jucker, Klaus P. Schneider and Wolfram Bublitz, 619–644. Boston: De Gruyter Mouton. Google Scholar logo with link to Google Scholar
Hewings, Martin. 2004. An ‘important contribution’ or ‘tiresome reading’? A study of evaluation in peer reviews of journal article submissions. Journal of Applied Linguistics 1.3:247–274. Google Scholar logo with link to Google Scholar
Hrežo, Vladislav. 2020. Exploring Phraseology in EU Legal Discourse. Language, Culture, Politics. International Journal 1.5:29–52.Google Scholar logo with link to Google Scholar
Hunston, Susan. 2011. Corpus Approaches to Evaluation: Phraseology and Evaluative Language. New York: Routledge.Google Scholar logo with link to Google Scholar
Hyland, Ken. 2008. As can be seen: Lexical bundles and disciplinary variation. English for Specific Purposes 27.1:4–21. Google Scholar logo with link to Google Scholar
Hyland, Ken, and Polly Tse. 2009. ‘The leading journal in its field’: Evaluation in journal descriptions. Discourse Studies 11.6:703–720. Google Scholar logo with link to Google Scholar
Hyland, Ken. 2012. Bundles in academic discourse. Annual Review of Applied Linguistics 321: 150–169. Google Scholar logo with link to Google Scholar
Hyon, Sunny. 2011. Evaluation in tenure and promotion letters: Constructing faculty as communicators, stars, and workers. Applied Linguistics 32.4:389–407. Google Scholar logo with link to Google Scholar
Itakura, Hiroko. 2013. Hedging praise in English and Japanese book reviews. Journal of Pragmatics 45.1:131–148. Google Scholar logo with link to Google Scholar
Itakura, Hiroko, and Amy B. Tsui. 2011. Evaluation in academic discourse: Managing criticism in Japanese and English book reviews. Journal of Pragmatics 43.5:1366–1379. Google Scholar logo with link to Google Scholar
Liu, Chen-Yu, and Hao-Jan Howard Chen. 2020. Analyzing the functions of lexical bundles in undergraduate academic lectures for pedagogical use. English for Specific Purposes 581:122–137. Google Scholar logo with link to Google Scholar
Luzón, María José. 2012. “Your argument is wrong”: A contribution to the study of evaluation in academic weblogs. Text & Talk 32.2:145–165. Google Scholar logo with link to Google Scholar
Martin, James R., and Peter White (eds.) 2005. The Language of Evaluation: Appraisal in English. New York: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Millar, Neil, and Susan Hunston. 2015. Adjectives, communities, and taxonomies of evaluative meaning. Functions of Language 22.3:297–331. Google Scholar logo with link to Google Scholar
Moreno, Ana I., and Lorena Suárez. 2008. A study of critical attitude across English and Spanish academic book reviews. Journal of English for Academic Purposes 7.1:15–26. Google Scholar logo with link to Google Scholar
Pan, Fan, Randi Reppen, and Douglas Biber. 2016. Comparing patterns of L1 versus L2 English academic professionals: Lexical bundles in Telecommunications research journals. Journal of English for Academic Purposes 211:60–71. Google Scholar logo with link to Google Scholar
Pomerantz, Anita. 1986. Extreme case formulations: A way of legitimizing claims. Human Studies 9.2:219–229. Google Scholar logo with link to Google Scholar
Reber, Jeffrey S., Robert D. Ridge, and Samuel D. Downs. 2017. Perceptual and behavioral effects of expectations formed by exposure to positive or negative Ratemyprofessors.com evaluations. Cogent Psychology 4.1:1–16. Google Scholar logo with link to Google Scholar
Römer, Ute. 2005. “This seems somewhat counterintuitive, though…”: Negative evaluation in linguistic book reviews by male and female authors. Strategies in Academic Discourse, ed. by Elena Tognini-Bonelli and Gabriella Del Lungo Camiciotti, 97–115. Philadelphia, PA: John Benjamins. Google Scholar logo with link to Google Scholar
Rühlemann, Christoph, and Karin Aijmer. 2015. Introduction. Corpus pragmatics: Laying the foundations. Corpus Pragmatics: A handbook, ed. by Karin Aijmer and Christoph Rühlemann, 1–26. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Scherr, Sebastian, Philipp Müller, and Victoria Fast. 2013. Single comments or average ratings: Which elements of RateMyProfessors. com™ shape university students’ judgments and course choice intentions? Educational Assessment, Evaluation and Accountability 25.2:131–141. Google Scholar logo with link to Google Scholar
Scott, Mike. 2017. WordSmith Tools Version 7. Stroud: Lexical Analysis Software.Google Scholar logo with link to Google Scholar
Simpson-Vlach, Rita, and Nick C. Ellis. 2010. An academic formulas list: New methods in phraseology research. Applied Linguistics 31.4:487–512. Google Scholar logo with link to Google Scholar
Siyanova-Chanturia, Anna, and Taha Omidian. 2019. Key issues in researching multiword items. The Routledge Handbook of Vocabulary Studies, ed. by Stuart Webb, 529–544. New York: Routledge. Google Scholar logo with link to Google Scholar
Skalicky, Stephen. 2013. Was this analysis helpful? A genre analysis of the Amazon.com discourse community and its “most helpful” product reviews. Discourse, Context & Media 2.2:84–93. Google Scholar logo with link to Google Scholar
Song, Chengfang. 2015. Lexical realization of attitudinal meaning and its coding in discourse analysis. Text & Talk 35.3:383–405.Google Scholar logo with link to Google Scholar
Stubbs, Michael. 2001. Words and Phrases: Corpus Studies of Lexical Semantics. Oxford: Blackwell publishers.Google Scholar logo with link to Google Scholar
Su, Hang. 2016. How products are evaluated? Evaluation in customer review texts. Language Resources and Evaluation 50.3:475–495. Google Scholar logo with link to Google Scholar
Subtirelu, Nicholas. 2018. Exploring the intersection of gender and race in evaluations of mathematics instructors on ratemyprofessors.com. Studies in Corpus-based Sociolinguistics, ed. by Eric Friginal, 219–235. New York: Routledge.Google Scholar logo with link to Google Scholar
Taboada, Maite. 2011. Stages in an online review genre. Text & Talk 31.2:247–269. Google Scholar logo with link to Google Scholar
Taboada, Maite, Marta Carretero, and Jennifer Hinnell. 2014. Loving and hating the movies in English, German and Spanish. Languages in Contrast 14.1:127–161. Google Scholar logo with link to Google Scholar
Taboada, Maite. 2016. Sentiment analysis: An overview from linguistics. Annual Review of Linguistics 21:325–347. Google Scholar logo with link to Google Scholar
Thominet, Luke. 2016. 10/10 would review again: Variation in the player game review genre. Technical Communication Quarterly 25.3:167–177. Google Scholar logo with link to Google Scholar
Thompson, Geoffrey, and Susan Hunston. 2000. Evaluation: An introduction. Evaluation in text: Authorial Stance and the Construction of Discourse, ed. by Susan Hunston and Geoffrey Thompson, 1–27. New York: Oxford University Press.Google Scholar logo with link to Google Scholar
Vásquez, Camilla. 2011. Complaints online: The case of TripAdvisor. Journal of Pragmatics 43.6:1707–1717. Google Scholar logo with link to Google Scholar
. 2014. The Discourse of Online Consumer Reviews. New York: Bloomsbury Publishing.Google Scholar logo with link to Google Scholar
White, Peter. 2000. Dialogue and inter-subjectivity: Reinterpreting the semantics of modality and hedging. Dialogue Analysis VII: Working with Dialogue, ed. by Malcolm Coulthard, Janet Cotterill and Frances Rock, 67–80. Tubingen: Neimeyer. Google Scholar logo with link to Google Scholar
. 2020. The Appraisal Website: 5. Engagement and Dialogic Positioning – An Outline of Engagement. Retrieved October 8, 2021, from [URL] [URL]
Wright, Heidi R. 2019. Lexical bundles in stand-alone literature reviews: Sections, frequencies, and functions. English for Specific Purposes 541:1–14. Google Scholar logo with link to Google Scholar
Yin, Xiaoyi, and Shuangling Li. 2021. Lexical bundles as an intradisciplinary and interdisciplinary mark: A corpus-based study of research articles from business, biology, and applied linguistics. Applied Corpus Linguistics 1.1:100006. Google Scholar logo with link to Google Scholar
Cited by (1)

Cited by one other publication

Liu, Xueying & Haoran Zhu
2025. The diachronic change in linguistic positivity in the academic book reviewing of language studies: a text-mining analysis. Scientometrics 130:1 ► pp. 133 ff. DOI logo

This list is based on CrossRef data as of 8 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue