Cover not available

Article published In: Pragmatics and Society
Vol. 13:2 (2022) ► pp.193–223

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB
authorization required

Clean room, uncomfortable bed

A corpus analysis of evaluation devices in hotel reviews

 | HSE University | University of Vigo
Published online: 23 June 2022
DOI logo with link
https://doi.org/10.1075/ps.19046.smi
Abstract
Online customer reviews, being an essential factor that determines success or failure in business, in particular in the tourism industry, demand close attention since the investigation of this type of discourse might bear some implications both for language specialists and for hotel managers. This paper is a quantitative and qualitative study of evaluation devices occurring in the corpus of hotel reviews from Booking​.com. The analysis is based on methods associated with corpus linguistics and Systemic Functional Linguistics (Halliday, M. A. K. 1994. An Introduction to Functional Grammar. London: Edward Arnold.Google Scholar logo with link to Google Scholar; Eggins, Suzanne. 2004. An Introduction to Systemic Functional Linguistics. London & New York: Continuum.Google Scholar logo with link to Google Scholar), specifically, the Appraisal framework (Martin, James R., and Peter R. R. White. 2005. The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar), which is applied here to a new type of discourse, that of hotel ratings. The study focuses on the categories of Attitude, Graduation and Engagement suggested by Martin and White and investigates inscribed and invoked evaluation. In order to adapt the Attitude dimensions to the corpus under study, new subcategories were created as a contribution to evaluation theory. The practical result of the study is a list of evaluation devices employed in hotel reviews; the list may be useful for further corpus analyses and for designing systems for automated analysis of customer evaluation. Since online customer reviews are commonly used by hotels as a benchmark of their guests’ satisfaction and a valuable information source of features and services that need to be improved, the present study is also likely to make an important contribution to the industry.
Article outline
  • 1.Introduction
  • 2.Literature review
    • 2.1Evaluation
    • 2.2Corpus studies of online reviews
  • 3.Data and methods
  • 4.Application of the appraisal framework to hotel reviews
    • 4.1Attitude
      • 4.1.1Appreciation
      • 4.1.2Judgement
      • 4.1.3Affect
    • 4.2Graduation and engagement
  • 5.Results
    • 5.1Attitude
    • 5.2Graduation
    • 5.3Engagement
    • 5.4Targets of evaluation
    • 5.5Invoked evaluation
  • 6.Discussion and conclusion
  • Acknowledgements
  • Notes
  • References
References (69)
References
Amin, Muslim, Zatilaziya Yahya, Wan F. A. Ismayatim, Siti Z. Nasharuddin, and Emma Kassim. 2013. “Service quality dimension and customer satisfaction: An empirical study in the Malaysian hotel industry.” Services Marketing Quarterly 34 (2): 115–125. Google Scholar logo with link to Google Scholar
Antony, Laurence. 2014. AntConc (Version 3.4.4) [Computer Software]. Tokyo: Waseda University. Available from [URL]
Banerjee, Snehasish, & Alton Y. K. Chua. 2016. “In search of patterns among travellers’ hotel ratings in TripAdvisor”. Tourism Management 531: 125–131. Google Scholar logo with link to Google Scholar
Banerjee, Snehasish, and Alton Y. K. Chua. 2017. “Theorizing the textual differences between authentic and fictitious reviews: Validation across positive, negative and moderate polarities.” Internet Research 27 (2): 321–337. Google Scholar logo with link to Google Scholar
Ben-Aaron, Diana. 2005. “Given and news: Evaluation in newspaper stories about national anniversaries.” Text – Interdisciplinary Journal for the Study of Discourse 25 (5): 691–718. Google Scholar logo with link to Google Scholar
Biber, Douglas, and Edward Finegan. 1988. “Adverbial stance types in English.” Discourse Processes 11 (1): 1–34. Google Scholar logo with link to Google Scholar
. 1989. “Styles of stance in English: lexical and grammatical marking of evidentiality and affect.” Text 9 (1): 93–124.Google Scholar logo with link to Google Scholar
Biber, Douglas. 2006. “Stance in spoken and written university registers.” Journal of English for Academic Purposes 5 (2): 97–116. Google Scholar logo with link to Google Scholar
Booking.com URL: [URL] (Accessed April 8, 2018).
Boucher, Jerry, and Charles E. Osgood. 1969. “The Pollyanna hypothesis.” Journal of Verbal Learning and Verbal Behavior 8 (1): 1–8. Google Scholar logo with link to Google Scholar
Brookes, Gavin, and Paul Baker. 2017. “What does patient feedback reveal about the NHS? A mixed methods study of comments posted to the NHS Choices online service.” BMJ Open 7 (4). Google Scholar logo with link to Google Scholar
Buhalis, Dimitrios, and Rob Law. 2008. “Progress in information technology and tourism management: 20 years on and 10 years after the Internet – The state of eTourism research.” Tourism Management 29 (4): 609–623. Google Scholar logo with link to Google Scholar
Camiciottoli, Belinda C. 2013. Rhetoric in Financial Discourse: A Linguistic Analysis of ICT-mediated Disclosure Genres. Amsterdam & New York: Rodopi. Google Scholar logo with link to Google Scholar
Cedefop’s European skills and jobs survey. 2014. URL: [URL] (Accessed February 11, 2019).
Cenni, Irene, and Patrick Goethals. 2017. “Negative hotel reviews on TripAdvisor: A cross-linguistic analysis.” Discourse, Context & Media 161: 22–30. Google Scholar logo with link to Google Scholar
Chipkin, Harvey. 2012, July 31. “Consumer trends 2012: review websites.” Travel Weekly. URL: [URL] Accessed: April 4, 2019.
Coffin, Caroline, and Kieran O’Halloran. 2006. “The role of appraisal and corpora in detecting covert evaluation.” Functions of Language 13 (1): 77–110. Google Scholar logo with link to Google Scholar
Conrad, Susan, and Douglas Biber. 2000. “Adverbial marking of stance in speech and writing.” In Evaluation in Text: Authorial Stance and the Construction of Discourse, ed. by Susan Hunston and Geoff Thompson, 56–73. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Cook, Lory S., David E. Bowen, Richard B. Chase, Sriram Dasu, Doug M. Stewart, and David A. Tansik. 2002. “Human resource issues in service design.” Journal of Operations Management 20 (2): 159–174. Google Scholar logo with link to Google Scholar
Eggins, Suzanne. 2004. An Introduction to Systemic Functional Linguistics. London & New York: Continuum.Google Scholar logo with link to Google Scholar
Halliday, M. A. K. 1994. An Introduction to Functional Grammar. London: Edward Arnold.Google Scholar logo with link to Google Scholar
Hao, Jing, and Sally Humphrey. 2012. “The role of ‘coupling’ in biological experimental reports.” Linguistics and the Human Sciences 5 (2): 169–194. Google Scholar logo with link to Google Scholar
Ho-Dac, Nga N., Stephen J. Carson, and William L. Moore. 2013. “The effects of positive and negative online customer reviews: do brand strength and category maturity matter?” Journal of Marketing 77 (6): 37–53. Google Scholar logo with link to Google Scholar
Hoffman, Lily M., and Jiri Musil. 2009. “Prague, tourism and the post-industrial city.” A Great Cities Institute Working Papers. Chicago: University of Illinois.Google Scholar logo with link to Google Scholar
Hood, Susan. 2012. “Voice and stance as APPRAISAL: Persuading and positioning in research writing across intellectual fields.” In Stance and Voice in Written Academic Genres, ed. by Ken Hyland and Carmen S. Guinda. London: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Hunston, Susan. 2011. Corpus Approaches to Evaluation: Phraseology and Evaluative Language. New York & London: Routledge.Google Scholar logo with link to Google Scholar
Hyland, Ken. 1998. “Exploring corporate rhetoric: Metadiscourse in the CEO’s letter.” Journal of Business Communication 35 (2): 224–244. Google Scholar logo with link to Google Scholar
. 2005. Metadiscourse: Exploring Interaction in Writing. London and New York: Continuum.Google Scholar logo with link to Google Scholar
Jackendoff, Ray. 1997. “Twistin’ the night away.” Language 73 (3): 534–559. Google Scholar logo with link to Google Scholar
Jeong, Miyoung, and Myunghee M. Jeon. 2008. “Customer reviews of hotel experiences through consumer generated media (CGM).” Journal of Hospitality & Leisure Marketing 17 (1–2): 121–138. Google Scholar logo with link to Google Scholar
Kachniewska, Magdalena. 2002. Zarządzanie Jakością Usług Turystycznych [Quality Management in the Tourist Industry]. Warszawa: Difin.Google Scholar logo with link to Google Scholar
Kaltenbacher, Martin. 2006. “Culture related linguistic differences in tourist websites: the emotive and the factual – a corpus analysis within the framework of APPRAISAL.” In System and Corpus: Exploring Connections, ed. by Geoff Thompson and Susan Hunston, 269–92. London: Equinox.Google Scholar logo with link to Google Scholar
Keuc, Natasha. 2014. “Factors influencing the decision on the measurement of customer satisfaction.” In Economic and Social Development: Book of Proceedings, ed. by Ivica Filipovic, Marina Klacmer Calopa, and Fran Galetic, 552–558. Varazdin: Varazdin Development and Entrepreneurship Agency.Google Scholar logo with link to Google Scholar
Li, Huiying, Qiang Ye, and Rob Law. 2013. “Determinants of Customer Satisfaction in the Hotel Industry: An Application of Online Review Analysis.” Asia Pacific Journal of Tourism Research 18 (7): 784–802. Google Scholar logo with link to Google Scholar
Lin, Ingrid Y. 2004. “Evaluating a servicescape: the effect of cognition and emotion.” International Journal of Hospitality Management 23 (2): 163–178. Google Scholar logo with link to Google Scholar
Martin, James R. 2000. “Beyond exchange: Appraisal systems in English.” In Evaluation in Text. Authorial Stance and the Construction of Discourse, ed. by Susan Hunston and Geoff Thompson, 142–175. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Martin, James R., and David Rose. 2003. Working with Discourse. Meaning beyond the Clause. London: Continuum.Google Scholar logo with link to Google Scholar
Martin, James R., and Peter R. R. White. 2005. The Language of Evaluation: Appraisal in English. Basingstoke: Palgrave Macmillan. Google Scholar logo with link to Google Scholar
Martin-Fuentes, Eva, and Juan P. Mellinas. 2018. “Hotels that most rely on Booking. com–online travel agencies (OTAs) and hotel distribution channels.” Tourism Review. Google Scholar logo with link to Google Scholar
Mel’cuk, Igor. 1996. “Lexical functions: A tool for the description of lexical relations in a lexicon.” In Lexical Junctions in Lexicography and Natural Language Processing, ed. by Leo L. Wanner. 87-102. Amsterdam & Philadelphia: John Benjamins.Google Scholar logo with link to Google Scholar
Mellinas, Juan P. 2019. “Dependency of Spanish urban hotels on Booking. com.” Tourism Analysis 24 (1): 3–12. Google Scholar logo with link to Google Scholar
Mohsin, Asad, and Tim Lockyer. 2010. “Customer perceptions of service quality in luxury hotels in New Delhi, India: an exploratory study.” International Journal of Contemporary Hospitality Management 22 (2): 160–173. Google Scholar logo with link to Google Scholar
Oteíza, Teresa. 2017. “The appraisal framework and discourse analysis”. In The Routledge Handbook of Systemic Functional Linguistics, ed. by Tom Bartlett and Gerard O’Grady, 481–496. New York: Routledge.Google Scholar logo with link to Google Scholar
Oxford English Dictionary (OED) URL: [URL] Accessed July 18, 2021.
Ozimek, Irena, Julita Szlachciuk, Olena Kulykovets and Natalia Przeździecka-Czyżewska. 2017. “Factors Influencing the Choice and Quality Assessment of Hotel Facilities in Poland”. Folia Oeconomica Stetinensia 17 (1): 170–181. Google Scholar logo with link to Google Scholar
Pang, Bo, and Lillian Lee. 2008. “Opinion Mining and Sentiment Analysis.” Foundations and Trends in Information Retrieval 2 (1–2): 1–135. Google Scholar logo with link to Google Scholar
Pawley, Andrew, and Frances H. Syder. 1983. “Two puzzles for linguistic theory: nativelike selection and nativelike fluency.” In Language and Communication 7(1), ed. by Jack C. Richards and Richard W. Schmidt, 191–226. London: Longman.Google Scholar logo with link to Google Scholar
Pierini, Patrizia. 2009. “Adjectives in tourism English on the web: a corpus-based study.” CÍRCULO de Lingüística Aplicada a la Comunicación 401: 93–116.Google Scholar logo with link to Google Scholar
Reuland, Ruud, Janet Choudry, and Ans Fagel. 1985. “Research in the field of hospitality.” International Journal of Hospitality Management 4 (4): 141–146. Google Scholar logo with link to Google Scholar
Ryu, Kisang, and Soocheong S. Jang. 2007. “The effect of environmental perceptions on behavioral intentions through emotions: The case of upscale restaurants.” Journal of Hospitality & Tourism Research 31 (1): 56–72. Google Scholar logo with link to Google Scholar
Sammons, Gail, Pat Moreo, Lori F. Benson, and Fred Demicco. 1999. “Analysis of female business travelers’ selection of lodging accommodations.” Journal of Travel & Tourism Marketing 8 (1): 65–83. Google Scholar logo with link to Google Scholar
Silver, Marc. 2003. “The stance of stance: a critical look at ways stance is expressed and modeled in academic discourse.” Journal of English for Academic Purposes 2 (4): 359–374. Google Scholar logo with link to Google Scholar
Sinclair, John. 1991. Corpus, Concordance and Collocation. Oxford: Oxford University Press.Google Scholar logo with link to Google Scholar
Smith, Trevor, and Andrew J. Spencer. 2011. “Predictors of value for money in Jamaican all-inclusive hotels.” International Journal of Humanities and Social Science 1 (4): 93–102.Google Scholar logo with link to Google Scholar
Sparks, Beverley. 2001. “Managing service failure through recovery.” In Service quality management in hospitality, tourism and leisure, ed. by Jay Kandampully, Connie Mok, and Beverley Sparks, 193–219. New York & London: Routledge.Google Scholar logo with link to Google Scholar
Su, Hang. 2016. “How products are evaluated? Evaluation in customer review texts.” Language Resources and Evaluation 50 (3): 475–495. Google Scholar logo with link to Google Scholar
Thompson, Geoff. 1996. Introducing Functional Grammar. London: Edward Arnold.Google Scholar logo with link to Google Scholar
. 2014. “AFFECT and emotion, target-value mismatches, and Russian dolls: Refining the APPRAISAL model”. In Evaluation in context, ed. by Geoff Thompson and Laura Alba-Juez, 47–66. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Thompson, Geoff, and Laura Alba-Juez. 2014. Evaluation in Context. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Tian, Youfei. 2013. “Engagement in online hotel reviews: A comparative study.” Discourse, Context & Media 2 (4): 184–191. Google Scholar logo with link to Google Scholar
Torres, Edwin N., and Sheryl Kline. 2013. “From customer satisfaction to customer delight.” International Journal of Contemporary Hospitality Management 25 (5): 642–659. Google Scholar logo with link to Google Scholar
Tuan, Nham P., and Nguyen T. H. Linh. 2014. “Impact of service quality performance on customer satisfaction: A case study of Vietnam’s five star hotel.” ABAC Journal 34 (3): 53–70.Google Scholar logo with link to Google Scholar
Vásquez, Camilla. 2011. “Complaints online: The case of TripAdvisor.” Journal of Pragmatics 43 (6): 1707–1717. Google Scholar logo with link to Google Scholar
Wang, Hongway, Pei Yin, Lijuan Zheng, and James N. K. Liu. 2014. “Sentiment classification of online reviews: using sentence-based language model.” Journal of Experimental & Theoretical Artificial Intelligence 26 (1): 13–31. Google Scholar logo with link to Google Scholar
White, Peter R. R. 2002. “Appraisal – the language of evaluation and stance.” In The Handbook of Pragmatics, ed. by Jef Verschueren, Jan-Ola Östman, Jan Blommaert, and Chris Bulcaen, 1–27. Amsterdam: John Benjamins.Google Scholar logo with link to Google Scholar
2003. “Beyond modality and hedging: a dialogic view of the language of intersubjective stance.” Appraisal: Special Edition of Text 23 (2): 259–84. Google Scholar logo with link to Google Scholar
2011. “Appraisal.” In Discursive Pragmatics, ed. by Jan Zienkowski, Jan-Ola Östman, and Jef Verschueren, 14–36. Amsterdam: John Benjamins. Google Scholar logo with link to Google Scholar
Yang, Yang, Zhenxing Mao, and Jingyin Tang. 2018. “Understanding guest satisfaction with urban hotel location.” Journal of Travel Research 57 (2): 243–259. Google Scholar logo with link to Google Scholar
Cited by (3)

Cited by three other publications

Cheng, Shi
2024. A review of interpersonal metafunction studies in systemic functional linguistics (2012–2022). Journal of World Languages 10:3 ► pp. 623 ff. DOI logo
Contu, Giulia, Cinzia Dessí, Carla Massidda & Marco Ortu
2024. Online reviews explain differences in coastal and inland tourists’ satisfaction. Scientific Reports 14:1 DOI logo
Yasin, Dhiyaul Hafidh Fatah, Raudlotul Firdaus Fatah Yasin, Muhammad Husni Hasbullah & Ku Mohd Syarbaini Ku Yaacob
2024. An Analytical Inquiry into Ethical Dilemmas in Event Planning: Insights from Muslim Organizers. In Islamic Finance [Contributions to Management Science, ], ► pp. 477 ff. DOI logo

This list is based on CrossRef data as of 30 november 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.

Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue