In:Mathematical Modelling in Linguistics and Text Analysis: Theory and applications
Edited by Adam Pawłowski, Sheila Embleton, Jan Mačutek and Aris Xanthos
[Current Issues in Linguistic Theory 370] 2025
► pp. 207–216
Corpus-driven analysis using Convolutional Neural Networks with Multi-Head Attention
Published online: 13 October 2025
https://doi.org/10.1075/cilt.370.17van
https://doi.org/10.1075/cilt.370.17van
Abstract
This paper addresses challenges associated with the interpretability of deep learning classification
models, particularly relevant for researchers in the humanities. A proposed methodological framework integrates corpus-driven
approaches and interpretable deep learning architectures, resulting in the development of the Multi-channel Convolutional
Transformer (MCT). This model effectively balances performance and interpretability, as demonstrated through a case study in
political science examining discursive conditions surrounding immigration as an electoral issue in 21st-century French
politics. The MCT emerges as a potent tool for text analysis, offering practical advantages for researchers in various
domains.
Keywords: corpus, deep learning, convolution, self-attention, political science, humanities
Article outline
- 1.Introduction
- 2.Model
- 2.1Pretraining with Convolutional Neural Network
- 2.2Classification based on Multi-Head Attention
- 2.3Multi-channels approach
- 3.Political discourse case study
- 3.1Contextual framework and objectives
- 3.2Methodological insights into political discourse analysis
- 3.3Implications and applications beyond Political Science
- 4.Conclusion
Note References
References (16)
Apoorv, Nandan. 2020. Text
classification with transformer. [URL]
Barats, Christine. 1999. Immigration:
carrefour de la suspicion (discours présidentiels et
juridiques). In Mots. Les langages du
politique 60(1). pages 43–58.
Bojanowski, Piotr, Edouard Grave, Armand Joulin & Tomas Mikolov. 2017. Enriching
word vectors with subword information. Transactions of the Association for
Computational
Linguistics 5. pages 135–146. Cambridge, MA.
Devlin, Jacob, Ming-Wei Chang, Kenton Lee & Kristina Toutanova. 2019. BERT:
Pre-training of deep bidirectional transformers for language
understanding. In Jill Burstein, Christy Doran & Thamar Solorio (eds.), Proceedings
of the 2019 conference of the North American chapter of the Association for Computational Linguistics: Human language,
technologies. Vol. 1, pages 4171–4186. Minneapolis, Minnesota.
Feng, Yue & Yan Cheng. 2021. Short
text sentiment analysis based on multi-channel CNN with multi-head attention
mechanism. IEEE
Access vol. 9, pages 19854–19863.
Kim, Yoon. 2014. Convolutional
neural networks for sentence classification. In Alessandro Moschitti, Bo Pang & Walter Daelemans (eds.), In
Proceedings of the
(EMNLP), pages 1746–1751. Doha, Qatar.
Li, Xuhong et al. 2022. Interpretable
deep learning: interpretation, interpretability, trustworthiness, and beyond. Knowledge
and Information
Systems 64. pages 3197–3234.
Mayaffre, Damon & Laurent Vanni. 2021. L’intelligence
artificielle des textes: Des algorithmes à
l’interprétation. Paris: Champion.
Noh, Hyeonwoo, Seunghoon Hong & Bohyung Han. 2015. Learning
Deconvolution Network for Semantic Segmentation, Proceedings of the 2015 IEEE
International Conference on Computer Vision
(ICCV), 2015, pages 1520–1528, Santiago, Chile.
Noiriel, Gérard. 2007. Immigration,
antisémitisme et racisme en France (XIXe-XXe siècle): Discours publics, humiliations
privées. Paris: Fayard.
Pennington, Jeffrey, Richard Socher & Christopher Manning. 2014. GloVe:
Global vectors for word representation. In Proceedings of the
2014 Conference on Empirical Methods in Natural Language Processing
(EMNLP), pages 1532–1543, Doha, Qatar.
Ribeiro, Marco Tulio, Sameer Singh & Carlos Guestrin. 2016. Why
should I trust you?: Explaining the predictions of any
classifier. In KDD’16: Proceedings of the 22nd ACM SIGKDD
conference on knowledge discovery and data
mining, pages 1135–1144, New York, USA.
Scott, M. Lundberg & Su-In Lee. 2017. A
unified approach to interpreting model
predictions. In Proceedings of the 31st International
Conference on Neural Information Processing Systems (NIPS’17). Curran Associates
Inc., pages 4768–4777, New York, USA.
