Chapter 6
Module 2Critical reading
6.1Response to titles of art works
Read the article by Leder et al. and use the terms from Chapter 6 to evaluate the claims of the researchers. Write a review of the article using the following terms: (1) Conceptual validity (look at the dependent as well as the independent variables); (2) external validity; (3) internal validity.
Before we start our critical reading of the article by Leder et al., we would first like to point out some aspects of this paper. We would like you to note the way the authors structured the article. This format is typically used in research reports of a scientific kind. It therefore can be used as a model for your own reports (but also read Chapter 12 for the guidelines writers should keep in mind). Please run through the article with us:
Page 178. Top of the page: just like the authors of this article, always explicitly mention what the aims of your research are. Also, look at page 180 where the authors briefly summarize their aims.
Page 180. Always give good reasons for the choices you made, for instance, in your method or procedure. This way you inform the readers in two ways: namely that there were choices to be made and that yours were made for good reasons. Here the authors argue that the paintings were shown just once, because otherwise you may expect mere exposure effects. Also, here is a good place to anticipate responses by your critics. Why would you choose 10 seconds of exposure? Well, the authors here justify their choice by referring to previous research.
Page 182. It is a good idea to follow the authors in their systematic report of the effects they tested.
Page 186. In case you do more than one study, make clear, just like Leder et al. do, what the second adds to the first, the third to the second, etc.
Page 192. It is very important that you assess the validity of your findings yourself (before your critics do). Often this is the starting point for a discussion, or for other researchers to work on. Here, for instance, the researchers suggest that physiological measures could be a useful addition to their own findings.
Note also how the researchers conclude their article – consider doing the same in your own writing: what is it that you want to tell the world? What is the gist of your report? What is it that you want your readers to take home, or to remember?
Now let us look at the aspects of validity mentioned in the assignment. Please keep in mind that the points of criticism discussed below are suggestions rather than the final word on the matter. It may well be that you have thought of other or even better objections against the study by Leder et al.
First of all, let us look at construct validity. We will discuss below several aspects of the conceptual design: both on the side of the dependent and the independent variables. In empirical research construct validity concerns the way researchers ‘translated’ concepts into measurable phenomena. Central to their enterprise is the ‘aesthetic experience’. What did you notice about the way they operationalized this concept?
Aesthetic experienceOne of the things that you might have observed is that the researchers put a lot of emphasis on cognitive aspects of aesthetic experience. It seems that we might as well be talking about responses to mathematical formulae. This is not the strongest objection that you could think of, but consider that the approach the researchers chose is one that may be typical for people in academia, possibly as the result of a socialization processes in which they learned how to deal with art objects. In general art education trains us in reflection on formal aspects of the objects (whether it is a poem or a painting). Look at the model now (Figure 1). Where do we place intuitive or emotional responses to artworks? It seems these are of little interest to the authors. What about the shivers that a music lover feels running down his or her spine during a concert? What of the irrational associations a viewer has gazing at some artwork? Such effects probably have nothing to do with “cognitive mastering”.
Cognitive processingThis brings us to the next question: How do the researchers measure cognitive processing? They use exposure time to manipulate the extent of cognitive processing. But how reliable is such an operationalization? How do we know what participants are actually doing during, for instance, a 10-seconds exposure time?
Because we do not know, the researchers run into some problems at this point. It seems they made a mistake in their thinking about their manipulation. Just imagine the following: say that the processing of the painting without the title Phase 1 takes 1 second, and Phase 2 (cognitive mastering) 4 seconds. The sum of both phases would be five seconds. Now when we add a descriptive title that neatly fits the painting (“Wavy Lines” for a painting with wavy lines) hardly any extra cognitive processing is required. This means that the required processing of the painting remains 5 seconds. However, many viewers may come to the conclusion that the title does not help their understanding of the painting at all and reflect much longer on how to interpret it. So would this not take longer than 5 seconds? As you can see, you can argue in two directions about the researchers’ manipulation. The first suggests a processing time of 5 seconds; the second longer than 5 seconds. Again: we cannot tell what the participants are doing during exposure time.
A similar problem occurs when we look at the possibilities of an elaborative title combined with a painting. One could reason that adding extra information (that is, the elaborative title) needs to be interpreted in combination with the painting; this may take a little longer than 5 seconds. However, it may also be that the title facilitates viewers’ interpretation of the painting to the extent that they need less than 5 seconds. We can argue that adding a title may either lead to a longer or a shorter processing time. In doing research, we try to avoid such lack of conceptual clarity.
Descriptive versus elaborative titlesOne problem that may have occurred to you is that of the titles. How can the researchers speak of titles in such a general way? Does it seem legitimate to you to classify titles into just two categories? A title like “Composition IV” for a work by Kandinsky may cause viewers to lose interest in the combination of title and image almost immediately. It does not seem to be informative at all, and interchangeable for any other title. On the other hand, the title of Kandinsky’s “Three Spots” (hardly a descriptive title as you will realize if you know the painting) may invite viewers to look for all three spots. By the way, both paintings are can easily be located on the internet via “Google images”: see whether you agree with our point!
Abstract versus representational artworksA similar problem occurs when we look at another classification the researchers use. How do they conceptualize the categories of abstract and representational artworks? They use a method very common in this type of study. They ask a number of independent researchers (six ‘art novices’) to determine which painting belongs to which of these two categories. However, why did they have art novices do this work? Would it not have been better to have experts (instead of novices) look at their stimulus materials?
ComprehensionIn their assessment of ‘comprehension’ the researchers rely on self-reported understanding. Probably they do this for practical reasons: how else could you assess comprehension? By testing participants’ correct understanding of a painting? Does such an understanding exist? Also, it would take too much time to test participants this way. They would need to answer open-ended questions, or one could interview them. But then the researchers would need to analyze and interpret their data to see to which extent the opinions of the participants correspond with a ‘correct’ interpretation. The problem you may have with the practical solution they chose is that the comprehension of one participant may not be comparable to that of another. I know that my understanding of some cubist sculpture is not equal to that of the director of a museum of modern art. It must be clear by now that Leder et al. measured “perceived understanding” instead of comprehension, and that these two are rather different concepts.
Art knowledgeIn their operationalization of ‘art knowledge’ the researchers asked participants to name the nationality of certain artists. Isn’t this a little trivial? And does this really help to assess what researchers wanted to know? What they are investigating here is one detailed aspect of encyclopedic knowledge of art which may have absolutely no bearing on “art knowledge” at all. Does it really matter that you know that Beethoven was born of Flemish parents (hence “van” Beethoven, not “von” Beethoven) in Germany? Such things seem to be trivialities that do not really capture a person’s knowledge about art.
3. External validityInformation that accompanies paintings in museums not only includes titles, but also the name of the artist. This aspect is not included in the design of the study, even though one could reason that such information could affect appreciation.
Why do you think the researchers reported that participants received 10 Euro for their participation in the study? Probably because this is a factor that influences external validity and therefore it is appropriate that they mention it in their report.
We have no information about the students. What did they study? This is more than just a problem for external validity: it makes it hard for us to estimate the validity of the study.
Comparing the setting of the experiment with your own experiences in museums, you may see that there is little resemblance. Therefore, in terms of ecological validity these experiments do not score well. However, it is not the researchers’ primary aim to end up formulating important recommendations for museums, let alone artists. Their interest is primarily a psychological one.
4. Internal validityOn page 181 the researchers explain why they chose unfamiliar paintings. Their argument seems reasonable: familiarity may influence the results in a way that has nothing to do with the central question of the studies. However, the researchers do not report that they actually checked whether participants knew the paintings. And, Jackson Pollock (page 195) is not exactly an unknown painter; especially his style is easy to recognize.
Participants see quite a number of paintings (48), one after the other. Why would that be? The idea is that the results may not depend on idiosyncratic aspects of the paintings but that they can be generalized (external validity). But due to this repeated measures design, it may well be that after, say, 20 paintings participants got tired of the procedure, even those who liked the paintings. This again may affect internal validity, though the article shows no indication that the researchers tested for this.
Ages of the participants ranged from 19-45. This is something that the researchers should have avoided. In Chapter 6 we discussed matters of control: One way to obtain control is to have homogeneous groups. With such a wide range in age, it will be harder to claim that the groups are homogeneous.
In their procedure they have groups of students look at the paintings. Would this not cause all sorts of group dynamics that could easily distort the results? They are positioned in a semi circle at 1.2 meter distance from a computer screen. Wouldn’t they be able to see each others’ responses? And might this not affect their own?
5. Statistical validityAs you will find out soon enough, it is often hard to evaluate the statistical validity of research reports like the present one. Most readers simply lack the background to see what is wrong. Here are just one simple point. Page 180: one of the things you will learn in Chapter 12 is that “Means” should always be accompanied by “Standard Deviations”; on page 180 you see that the authors failed to report the standard deviation for “age”.