An exploratory, qualitative experiment sheds light on the depictive theory of mental imagery. The study analyzes the very operations subjects undertake when solving visuospatial tasks. Preliminary results indicate that subjects do not make use of stable mental images: instead, they continuously assemble and reassemble different perspectives through the guidance of heuristics and prototypes. These observations allow a reinterpretation of mental imagery. We want to forward the hypotheses that a) the assembly process itself is of much higher importance than usually acknowledged; b) that an assembled perspective (or figure) is defined by one's orientation towards certain operations; and c), that heuristics and prototypes are instantiated by a heterarchical organization of mental operations.
What characteristics does the cognitive representation of a shape have? How do humans
reason about and with shapes? Besides from subjective reports of the appearance and
the usage of mental images, the existence of a faculty of imagery – however it may be
conceptualized – has been proven to be functionally involved in cognitive processes.
Evidence is for example presented from studies showing that humans can mentally
synthesize and subsequently recognize figures. In the paradigmatic experiment of Finke et
al. [
Imagine the letter . Put a small circle at the bottom of it. Add a horizontal line halfway up. Now rotate the figure 180 degrees. ([1, p. 62])
Two opposed conceptualizations of mental imagery are established: the depiction
theory and the description theory. These have been forwarded by Kosslyn [
In the experiments of [
According to Tversky “[t]here are two basic tenets of the approach, one regarding representations and the other regarding operations on representations: that mental images resemble percepts, and that mental transformations on images resemble observable changes in things in the world, as in mental rotation, or perceptual processes performed on things in the world, as in mental scanning” [4, p. 211]. In a nutshell, the depictive account says that in solving visuospatial problems, one can distinguish two types of operations: (a) image-construction operations, and (b) subsequent inspection of the image, analogous to visual perception.
So far the theory. There is something in disorder, though. Apart from transformations of the image, Kosslyn highlights that images fade – ”they are transient and begin to decay as soon as they are activated” [2, p. 50]. A frequent reconstruction is necessary, which eventually brings about transformations of the imagined objects or a new perspective. The fading of images stands in opposition to their asserted analogue nature to external images (or scenes). This seems to be a flaw in what otherwise could be a pretty useful imagery faculty. We suggest that shifting the research focus away from a supposed presence of images, and towards the process of reconstructing these will be of great value.
In an exploratory protocol study, six subjects had to solve visuospatial tasks like the
above with a slightly higher level of complexity. Interviews lasted for about one hour,
and were guided according to an interview protocol devised by Petitmengin [
Imagine a square. Imagine its vertical lines being separated each into three parts of equal length. Now connect the resulting division marks by two horizontal lines, such that the square is divided into three identical rectangles. Now add the two diagonals to the square. How many triangles does the pattern contain? (Adapted from Wiener [6, p. 82f].)
Even though the elements of the figure are easy to imagine, and the whole figure is not more complicated than the stick-figure, it is very difficult to find all the triangles in this task (there are much more then one initially might think). Interestingly, constructing the figure seemed feasible. A typical report of the resulting image reads like this: Well . . . I have to . . . juggle a bit to keep the components in focus. Everything is there. But I successively imagine how the horizontal lines join the square, and how the diagonals cross these lines, that’s all a bit waggly.
But when asked to count the triangles, the picture changes. A typical report is: I lose the whole image. It isn’t as clear as before any more. (Can you still examine it?) Yes, but I have to build it up anew. And in considering a detail of the image I . . . somehow have to retrace the constituents, for example putting in the diagonal again.
This change of the subject’s representation of the figure suggests that the task – in this case, constructing the image versus finding triangles – guides the process. In the latter case, for example, subjects rely on the given elements of the figure and assemble subsets of these in order to construct triangles. That is, instead of looking at the image and recognizing triangles, subjects actively try to construct triangles from the given material. The following transcript illustrates such an attempt; the subject has the idea that there could be another type of triangle and actively tries to assemble it:
There is another idea - to search for another line with which the diagonal can form a triangle there have to be more, because the diagonals cross the lines of the two horizontal lines. I begin in the upper left corner and . . . trace the diagonal . . . and it it goes a level deeper, and forms a triangle where it crosses the second horizontal line. I didn’t really search for the horizontal line, but looked how the diagonal runs, whether it could form a triangle somewhere. . . . I never have the whole image, but only - well, I often lose the whole image. I trace lines, of which I should already know how it runs. . . . Then I felt reminded of the symmetry, and the thought comes up that there have to be four of them.
The subject has an idea – we want to call it a prototype – of a certain type of triangle which only half of the subjects found. It is being formed by one of the vertical sides, an adjacent diagonal, and the horizontal line which is further away from it; this description would already suffice to deduce a triangle. But for the subject, this is only a hypotheses so far which he tries to instantiate: he traces the diagonal and tries to coordinate it with the horizontal line. But he cannot simply see this line and the intersection. Instead, he makes the horizontal line up again from the initial figure, and coordinates the intersection with the diagonal. Another interesting aspect about his description is the intuition of symmetry, which provides a heuristic to look out for more triangles.
We want to briefly paraphrase our findings. Focusing on the assemble operations, we can see that different perspectives are being assembled over and over. This process is guided in a very fine-grained manner by a scaffolding of the task structure (triangles) and on-the-fly produced prototypes and heuristics. We consider these to be ontologically equivalent, because all guide the assembly of elements of the initial figure and thereby provide a tightly meshed, heterarchical organization of mental operations. Because all of these guides path the way to assemble components, we suggest that an assembled perspective (or figure) is defined by one’s orientation towards certain operations. Eventually, an assembled perspective confirms the instantiation of a certain property or relation.
Based on our exploration of visuospatial reasoning we want to propose a renewed interpretation of mental imagery. In imagery, one actively attempts to assemble an intended structure; but one does not simply see it. One can indeed think of the whole process as being constituted through operations of type (a) and (b) from above, but with a reversed order. We do not make up an image to see something, but we want to see something, and in the absence of a visual stimulus we have to construct an orientation that fits.
Is the visual metaphor of depictive theory – imagery as internalized perception – therefore invalid? This only holds if we think of seeing as a one-way, passive uptake of input. But in an account of active perception, like that of Rensink, he states that one looks at scenes in a structural manner: “scene representations are no [. . . ] structures built up from eye movements and attentional shifts, but rather, are structures that guide such activities [7, p. 36]”.
With regard to the top-down guidance that we found in imagery we are sympathetic
to such a conceptualization of vision. But do we therefore have to rephrase and say,
somewhat awkward, perception is externalized imagery? Based on our observations there
would be a missing piece, which are the overly present dynamic processes of assembling
figures. We think it holds that imagery has its ontogenetic foundation in intentionally
guided visual operations, and go with Piaget [
Our results are preliminary. In order to further our understanding of assembly
processes, we are currently executing a study on the development of spatial orientation in
familiar environments. Our approach also shows promising relations to a recent study,
where [