Reading aloud and the delay of feedback: Explanations for the effectiveness of reader protocols

An important objective of research in information design is the validation of methods that are available for the evaluation of public information. In this article three evaluation methods using verbal self-reports are compared: think aloud reader protocols, the plus-minus method and Focus (a software tool designed to collect reader responses on documents). The results of this comparison indicate that reader protocols produce better feedback on comprehension problems than the other two methods. In a series of experiments two aspects of reader protocols are further investigated in order to explain these results. The first factor is delay of feedback, which seems to result in fewer problem detections. The second factor is reading aloud, which may result in more problem detections. These experiments indicate that delay is the key factor. Finally an eye tracking study was done in order to analyze the effect on the reading process of reading aloud and giving comments. The results indicate that reading aloud causes a delay in the reading process and the task of giving comments increases the number of times readers look back in the text while reading.