Specific-purpose language performance tests

Oral performance tests designed to assess candidates’ ability to function in a particular professional role typically involve role plays which simulate aspects of professional interaction between a candidate, in his/her professional role, and an interlocutor, in the role of client or consumer of the professional services. The validity of such tests is dependent both on the incorporation of appropriate content and on the extent to which the interaction between candidate and interlocutor simulates authentic professional interaction. Materials for such role plays are commonly produced according to specifications developed by language testers following consultation with professional informants, but there has been little investigation of how well tasks reflect their purpose (i.e. the aims/intentions of the professional experts). Test interlocutors, generally language teachers, normally undergo some form of training in the test administration procedure, but there has been little analysis of how well their behaviour reflects client behaviour.

This paper reports on an investigation into the role which expert professional informants can play in providing post hoc content validation of tasks used in this kind of occupation-specific test, focusing on both the content of the tasks and the behaviour of the interlocutors. The study draws on data from the oral interaction component of an occupation-specific test, the Occupational English Test (McNamara 1996), a 4-skills test for overseas-trained health professionals (with attention in this study to nurses). The study involved a two-day workshop where representatives of the nursing profession, in conjunction with the researchers, analysed both written test materials and audiotaped test data, contributing to an analysis of the realism of the tasks and the features of interlocutor behaviour.