Cover not available

Article published In: Dialogic Dimensions of the Digital
Edited by Răzvan Săftoiu
[Language and Dialogue 16:1] 2026
► pp. 8–29

Get fulltext from our e-platform
JBE logo with link
PDF logo with download linkDownload PDF
EPUB logo with download linkDownload EPUB

Emoji in dialogue with images

A multimodal social semiotic approach

 | Free University of Bolzano-Bozen
Open Access logo
Available under the Creative Commons Attribution (CC BY) 4.0 license.

For any use beyond this license, please contact the publisher at rights@benjamins.nl.

This article was made Open Access under a CC BY 4.0 license through payment of an APC by or on behalf of the author.

Published online: 29 September 2025
DOI logo with link
https://doi.org/10.1075/ld.00211.laz
Abstract
This paper explores the visual nature of emoji and their potential to co-create meaning when in dialogue with other visual semiotic resources, such as images, paintings, videos, and photographs. Drawing on a multimodal social semiotic framework and studies that emphasise the independent meaning-making capabilities of visual communication, it challenges the traditional view that emoji are merely paralinguistic elements dependent on verbal co-texts. The study analyses a corpus of art museum social media posts, highlighting instances of intermodal convergence between the shared images and emoji used in the captions. This analysis reveals formal and semantic dialogic relations, showcasing how emoji can function as independent semiotic resources, capable of establishing meaningful interactions with other visual elements.
Article outline
  • 1.Introduction
  • 2.Arguing for a pictorial account of emoji
  • 3.Extending the notion of dialogue to visual modes
  • 4.Materials and methods
  • 5.Discussion
    • 5.1Convergence of compositional meanings between image and emoji
    • 5.2Convergence of representational meanings between image and emoji
  • 6.Conclusions
  • Note
  • References
References (31)
References
Abercrombie, David. 1968. “Paralanguage.” British Journal of Disorders of Communication 3(1): 55–59. Google Scholar logo with link to Google Scholar
Bakhtin, Mikhail M. 1986. Speech Genres and Other Late Essays. Austin: University of Texas Press.Google Scholar logo with link to Google Scholar
Camillini, Gianluca, Marcello Barison, and Roberto Gigliotti. 2022. “Images in Dialogue: How They Talk and What They Say.” Img Journal 71: 56–73.Google Scholar logo with link to Google Scholar
Częstochowska, Justyna, Kristina Gligorić, Maxime Peyrard, Yann Mentha, Michał Bień, Andrea Grütter, Anita Auer, Aris Xanthos, and Robert West. 2022. “On the Context-free Ambiguity of Emoji.” Proceedings of the International AAAI Conference on Web and Social Media 161: 1388–1392. Google Scholar logo with link to Google Scholar
Danesi, Marcel. 2016. The Semiotics of Emoji. London: Bloomsbury Publishing.Google Scholar logo with link to Google Scholar
. 2018. “Emojis: New Language or Technology-based Trend?” ETC: A Review of General Semantics 75(3/4): 317–328.Google Scholar logo with link to Google Scholar
Dürscheid, Christa, Dimitrios Meletis, and Yannis Haralambous. 2019. “Emojis: A grapholinguistic approach.” Grapholinguistics and Its Applications 11: 167–183. Google Scholar logo with link to Google Scholar
Fulford, Jason and Tamara Shopsin. 2014. This Equals That. New York: Aperture.Google Scholar logo with link to Google Scholar
Halliday, Michael Alexander Kirkwood. 1978. Language as Social Semiotic. London: Hodder Arnold.Google Scholar logo with link to Google Scholar
Holquist, Michael. 2003. Dialogism: Bakhtin and His World. London: Routledge. Google Scholar logo with link to Google Scholar
Jovanovic, Danica and Theo Van Leeuwen. 2018. “Multimodal dialogue on social media.” Social Semiotics 28(5): 683–699. Google Scholar logo with link to Google Scholar
Kessels, Erik. 2014. In Almost Every Picture 13 — Attack Of The Giant Fingers. Amsterdam: Kesselskramer.Google Scholar logo with link to Google Scholar
Kress, Gunther and Theo Van Leeuwen. 2006. Reading Images. The Grammar of Visual Design. Abingdon: Routledge.Google Scholar logo with link to Google Scholar
Kress, Gunther. 2009. Multimodality: A Social Semiotic Approach to Contemporary Communication. London: Routledge. Google Scholar logo with link to Google Scholar
Lazzeretti, Cecilia. 2023. “Exploring the Use of Emoji in Museum Social Network Sites.” Discourse, Context & Media 531, 100697. Google Scholar logo with link to Google Scholar
Lee, SooJin. 2018. “Emoji at MoMA: Considering the ‘original emoji’ as art.” First Monday 23(9). Google Scholar logo with link to Google Scholar
Maier, Emar. 2023. “Emojis as Pictures.” Ergo: An Open Access Journal of Philosophy 10(12): 317–355.Google Scholar logo with link to Google Scholar
Martin, James R. 2011. “Multimodal semiotics: Theoretical challenges.” In Semiotic margins: Meaning in multimodalities, ed. by Shoshana Dreyfus, Maree Stenglin, and Susan Hood, 243–270. London: Bloomsbury Publishing.Google Scholar logo with link to Google Scholar
Moya-Guijarro, Arsenio Jesús. 2024. “A Multimodal Analysis of Character-character Interaction in LGTB Picture Books and its Educational Implications.” Linguistics and Education 821: 101312. Google Scholar logo with link to Google Scholar
Painter, Clare, Jim Martin, and Len Unsworth. 2013. Reading Visual Narratives: Image Analysis of Children’s Picture Books. Sheffield: Equinox Publishing Ltd.Google Scholar logo with link to Google Scholar
Ravelli, Louise J. 2006. “Genre and the Museum Exhibition.” Linguistics and the Human Sciences 2(2): 299–317. Google Scholar logo with link to Google Scholar
Sampietro, Agnese. 2016. “Exploring the Punctuating Effect of Emoji in Spanish Whatsapp Chats.” Lenguas Modernas 471: 91–113.Google Scholar logo with link to Google Scholar
Seargeant, Philip. 2019. The Emoji Revolution: How Technology is Shaping the Future of Communication. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Sindoni, Maria Grazia. 2011. Systemic-functional grammar and multimodal studies: An introduction with text analysis. Pavia: Ibis.Google Scholar logo with link to Google Scholar
. 2019. “Mode-switching in Video-mediated Interaction: Integrating Linguistic Phenomena into Multimodal Transcription Tasks.” Linguistics and Education 621: 1–11.Google Scholar logo with link to Google Scholar
Thomson, Samantha, Emily Kluftinger, and Jocelyn Wentland. 2018. “Are You Fluent in Sexual Emoji? 😉: Exploring the Use of Emoji in Romantic and Sexual Contexts.” The Canadian Journal of Human Sexuality 27(3): 226–234. Google Scholar logo with link to Google Scholar
Weigand, Edda. 2010. “Language as Dialogue.” Intercultural Pragmatics 7(3): 505–515. Google Scholar logo with link to Google Scholar
Weissman, Benjamin. 2019. “Peaches and Eggplants or… Something Else? The Role of Context in Emoji Interpretations.” Proceedings of the Linguistic Society of America 4(29): 1–6. Google Scholar logo with link to Google Scholar
Wicke, Philipp and Marianna Bolognesi. 2020. “Emoji-based Semantic Representations for Abstract and Concrete Concepts.” Cognitive Processing 21(4): 615–635. Google Scholar logo with link to Google Scholar
Zappavigna, Michele and Lorenzo Logi. 2024. Emoji and Social Media Paralanguage. Cambridge: Cambridge University Press. Google Scholar logo with link to Google Scholar
Mobile Menu Logo with link to supplementary files background Layer 1 prag Twitter_Logo_Blue