Contrastive corpus annotation in the CONTRANOT project

In this paper we outline a number of issues and problems which arise during the process of contrastive human-coded corpus annotation of certain semantic and discourse categories within the framework of the CONTRANOT project, aimed at the creation and validation of contrastive functional descriptions through corpus analysis and annotation. Human-coded corpus annotation is a preliminary step for the training of computer algorithms which allow the automation of the annotation of large corpora, but it can also serve as a mechanism for testing aspects of linguistic theories empirically, such as theory formation and theory-redefinition, as well as enriching theories with quantitative information. The work reported in this paper focuses on the annotation of the category of Thematisation, on the one hand, and on Modality, on the other, to illustrate the challenges researchers have to face when confronted with the task of developing well-designed and reliable annotation procedures for complex linguistic phenomena in a contrastive manner. We describe the annotation tasks and procedures developed so far, which include the design of annotation schemas on the basis of available linguistic theories and the testing of their reliability through agreement studies. We also evaluate and discuss the results of the annotations on the basis of their relevance for the theoretical characterisation of the investigated phenomena. We expect that our work will have an impact in the area of contrastive textual analysis, and that it will pave the way for the development of automated annotation systems for computational applications.