Text to Ideology or Text to Party Status?

Hirst, Graeme; Riabinin, Yaroslav; Graham, Jory; Boizot-Roche, Magali; Morris, Colin

doi:10.1075/dapsac.55.05hir

In:From Text to Political Positions: Text analysis across disciplines
Edited by Bertie Kaal, Isa Maks and Annemarie van Elfrinkhof
[Discourse Approaches to Politics, Society and Culture 55] 2014
► pp. 93–116

Get fulltext from our e-platform

Download Book PDF

Text to Ideology or Text to Party Status?

Graeme Hirst | Department of Computer Science, University of Toronto

Yaroslav Riabinin | Department of Computer Science, University of Toronto

Jory Graham | Department of Computer Science, University of Toronto

Magali Boizot-Roche | Department of Computer Science, University of Toronto

Colin Morris | Department of Computer Science, University of Toronto

Published online: 7 May 2014

https://doi.org/10.1075/dapsac.55.05hir

Recent papers have used support-vector machines with word features to classify political texts by ideology. Our own work on this topic led us to hypothesize that such classifiers are sensitive not to expressions of ideology but rather to expressions of attack and defense, opposition and government. We test this hypothesis by training on one set of parliamentary speeches and testing on another in which party roles have been interchanged, and we find that the performance of the classifier completely disintegrates. Moreover, some features that are indicative of each party ‘swap sides’ with the change of government. Our results suggest that the language of attack and defense, of government and opposition, will dominate and confound any sensitivity to ideology in these kinds of classifiers.

References (17)

References

DEFT (2009). Actes de l’atelier de clôture de la cinquième édition du DÉfi Fouille de Textes, Paris, 22 June 2009.

Diermeier, D., J.-F. Godbout, B. Yu and S. Kaufmann. 2007. Language and ideology in Congress. Annual Meeting of the Midwest Political Science Association.

Forest, D., A. van Hoeydonck, D. Létourneau and M. Bélanger. 2009. Impacts de la variation du nombre de traits discriminants sur la catégorisation des documents. Actes de l’atelier de clôture de la cinquième édition du DÉfi Fouille de Textes, Paris, 22 June 2009, pp. 75–88.

Germann, U. 2001. Aligned Hansards of the 36th Parliament of Canada. Available at [URL]

Greene, S. 2007. Spin: Lexical semantics, transitivity, and the identification of implicit sentiment. PhD thesis, University of Maryland, College Park.

Grouin, C., B. Arnulphy, J.-B. Berthelin, S. El Ayari, A. Garcia-Fernandez, A. Grappy, M. Hurault-Plantet, P. Paroubek, I. Robba and P. Zweigenbaum. 2009. Présentation de l’édition du D’Efi Fouille de Textes (DEFT’09). Actes de l’atelier de clôture de la cinquième édition du DÉfi Fouille de Textes, Paris, 22 June 2009, pp. 35–50.

Hix, S., A.G. Noury and G. Roland. 2007. Democratic Politics in the European Parliament. Cambridge: Cambridge University Press.

Jiang, M. and S. Argamon. 2008. Political leaning categorization by exploring subjectivities in political blogs. Proceedings, 4th International Conference on Data Mining (DMIN 2008), pp. 647–653.

Lin, W., T. Wilson, J. Wiebe and A. Hauptmann. 2006. Which side are you on? Identifying perspectives at the document and sentence levels. Proceedings of the 10th Conference on Natural Language Learning (CoNLL-X), pp. 109–116.

Marx, M. and A. Schuth. 2010. DutchParl: The parliamentary documents in Dutch. Proceedings of the 7th Conference on Language Resources and Evaluation (LREC 2010), Malta.

Mullen, T. and R. Malouf. 2006. A preliminary investigation into sentiment analysis of informal political discourse. Proceedings of the AAAI Symposium on Computational Approaches to Analysing Weblogs (AAAI-CAAW), pp. 159–162.

Pennebaker, J.W. 2008. The meaning of words: Obama versus McCain. Weblog, 12 October 2008. Available at [URL]

Pennebaker, J.W., C.K. Chung, M. Ireland, A. Gonzales and R.J. Booth. 2007a. The Development and Psychometric Properties of LIWC2007. Available at [URL]

. 2007b. Linguistic inquiry and word count (LIWC2007). Available at [URL]

Riabinin, Y. 2009. Computational identification of ideology in text: A study of Canadian parliamentary debates. MSc paper, Department of Computer Science, University of Toronto, January 2009. Available at [URL]

Thomas, M., B. Pang and L. Lee. 2006. Get out the vote: Determining support or opposition from congressional floor-debate transcripts. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pp. 327–335.

Yu, B., S. Kaufmann and D. Diermeier. 2008. Classifying party affiliation from political speech. Journal of Information Technology in Politics 5(1), pp. 33–48.

Cited by (10)

Cited by ten other publications

Order by:

Laurer, Moritz, Wouter van Atteveldt, Andreu Casas & Kasper Welbers

2025. On Measurement Validity and Language Models: Increasing Validity and Decreasing Bias with Instructions. Communication Methods and Measures 19:1 ► pp. 46 ff.

Mennig, Philipp

2025. Who cares about agriculture? Analyzing German parliamentary debates on agriculture and food with structural topic modeling. Food Policy 130 ► pp. 102788 ff.

Gireesan, K.

2024. Democracy—Application of Artificial Intelligence. In Democracy, Leadership and Governance – Application of Artificial Intelligence, ► pp. 351 ff.

Baden, Christian, Christian Pipal, Martijn Schoonvelde & Mariken A. C. G van der Velden

2022. Three Gaps in Computational Text Analysis Methods for Social Sciences: A Research Agenda. Communication Methods and Measures 16:1 ► pp. 1 ff.

Cochrane, Christopher, Ludovic Rheault, Jean-François Godbout, Tanya Whyte, Michael W.-C. Wong & Sophie Borwein

2022. The Automatic Analysis of Emotion in Political Speech Based on Transcripts. Political Communication 39:1 ► pp. 98 ff.

Zuba, Krzysztof

2021. From the poster boy of Europeanization to the sick man of Europe: thirty years (1990–2019) of Poland’s European policy. European Politics and Society 22:3 ► pp. 394 ff.

Baden, Christian, Neta Kligler-Vilenchik & Moran Yarchi

2020. Hybrid Content Analysis: Toward a Strategy for the Theory-driven, Computer-assisted Classification of Large Text Corpora. Communication Methods and Measures 14:3 ► pp. 165 ff.

Rheault, Ludovic & Christopher Cochrane

2020. Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora. Political Analysis 28:1 ► pp. 112 ff.

Dehghani, Mostafa, Hosein Azarbonyad, Jaap Kamps & Maarten Marx

2016. Two-Way Parsimonious Classification Models for Evolving Hierarchies. In Experimental IR Meets Multilinguality, Multimodality, and Interaction [Lecture Notes in Computer Science, 9822], ► pp. 69 ff.

Dallmann, Alexander, Florian Lemmerich, Daniel Zoller & Andreas Hotho

2015. Proceedings of the 26th ACM Conference on Hypertext & Social Media - HT '15, ► pp. 133 ff.

This list is based on CrossRef data as of 8 december 2025. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.