=Paper=
{{Paper
|id=Vol-1419/paper0002
|storemode=property
|title=Gestalt Effects in Visual Working Memory
|pdfUrl=https://ceur-ws.org/Vol-1419/paper0002.pdf
|volume=Vol-1419
|dblpUrl=https://dblp.org/rec/conf/eapcogsci/KalamalaSOC15
}}
==Gestalt Effects in Visual Working Memory==
https://ceur-ws.org/Vol-1419/paper0002.pdf
Gestalt Effects in Visual Working Memory
Patrycja Kałamała (patrycja.kalamala@gmail.com)
Aleksandra Sadowska (aleksandrowska@gmail.com)
Wawrzyniec Ordziniak (wordziniak@gmail.com)
Adam Chuderski (adam.chuderski@gmail.com)
Institute of Philosophy, Jagiellonian University in Krakow
Grodzka 52, 31-044 Krakow, Poland
Abstract The crucial attribute of VWM consists of its very limited
capacity with regard to the actively maintained objects
The study investigated whether the congruence between the (probably 4 or less objects; Luck & Vogel, 1997), but sub-
shape of the single target and the shape of the overall pattern
of stimuli, as well as the regularity (equal distances among
stantial capacity concerning the number and precision of
stimuli) of that pattern, could facilitate the maintenance of perceptual features constituting these objects (multiple-
information in visual working memory (VWM). We observed feature objects can be maintained and recognized almost as
strong evidence in favor of the congruency effect, and effectively as single-feature objects; Luck & Vogel, 1997;
moderately positive evidence for the regularity effect. Both but see Oberauer & Eichenberger, 2013). This differentiated
effects were relatively weak, but easily identifiable with the impact of the number of objects versus features on VWM
quite large samples we examined. These data support and
capacity is compatible with the fact that VWM subsystem
largely extend the existing evidence showing that Gestalt
principles of perceptual organization, which are well known to responsible for maintaining object features is located within
organize visual perception, influence also the active mainte- the superior parietal lobule, whereas the binding of com-
nance and access of information in VWM during the absence plete objects from those features most probably takes place
of perceptual stimulation. within the inferior parietal lobule (Xu & Chun, 2009). In
order to explain how single features can be bound into
Introduction objects, and maintained univocally, oscillatory computa-
The last 40 years of research in cognitive science has tional models have been developed (Chuderski, Andrelczyk
yielded substantial knowledge on the key role of working & Smoleń, 2013; Raffone & Wolters, 2001).
memory (WM) for human cognition. WM is a hypothetical An important area of evidence regarding factors that
cognitive mechanism responsible for the active maintenance influence actual VWM capacity pertains to the influence of
of information and its goal-driven manipulation for the global organization of perceptual scene, that is, the fact that
purpose of the current task (Baddeley & Hitch, 1974). WM objects are not stored in memory independently from other
has been shown to be involved in such mental processes as items, but there exist substantial contextual effects (Brady,
problem solving, thinking, reasoning, cognitive control, Konkle, & Alvarez, 2011). For example, when context of an
encoding and retrieving information in/from long-term item (e.g., surrounding objects) changes or disappears
memory, and many others. Although the debate on the between the to-be-memorized scene and the probe scene,
fundamental mechanisms that determine the “workings” of retrieval of this item is worse than when the unchanged con-
working memory has been dominated by verbal paradigms text accompanies the probe (Jiang, Olson, & Chun, 2000).
of WM measurement, re-search conducted during last ten or Also statistical distribution of features is important for
so years is converging at the crucial role of visual working retrieval, for instance it is easier to reject a false probe if its
memory (VWM) in underlying many functions of working features differ much from the dominant features in a scene
memory. Models of VWM generally assume that VWM is a (e.g., to reject a new cold-color probe if all objects in a
relatively simple mechanism, which operates on visual scene were shown in warm colors; Brady et al., 2011).
representations of objects (or bundles of features defining Context also influences how we recall individual items, as
these objects) and spatial relations among them (Clevenger recall of items that possessed an extreme value of a
& Hummel, 2014; Luck & Vogel, 1997). Although simple, particular visual feature (e.g., size) is often biased toward an
during the evolu-tion of human mind this mechanism, average value of that feature in a display (Brady & Alvarez,
primarily responsible for the continuity of perception and 2011). In total, all these context effects suggest that people
the spatial orientation, most probably has been adapted in encode in VWM not only particular items, but also (or –
the service of more complex cognition, including the even – primarily) encode their ensemble in a way that is
construction of abstract representations (see Cowan et al., able to compress redundant and structured information from
2011), processing relat-ions (Clevenger & Hummel, 2014), a display into concise but very informative higher-level
as well as using mental models and simulations to represent representation of ensemble, which then can be used to
hypothetical states of the world (Johnson-Laird, 2006). predict features of individual items (Alvarez, 2011).
42
One particularly interesting type of context/ensemble Goals of the study
effects in VWM regards the influence of satisfying (or not) The aim of the study was to examine two other contextual
the Gestalt principles of perceptual organization (Laws of effects that relied on Gestalt principles, which have not been
Pragnanz, Proximity, Similarity, Closure, Symmetry, and experimentally tested yet. Our first hypothesis predicted a
Continuity). Some studies have demonstrated that satisfying positive effect of matching between a stimulus shape and a
such principles by the group of objects not only helps in shape of the complete pattern of stimuli. The second hypo-
perceiving them in a particular way, but also facilitates their thesis was that regularity of the pattern of stimuli, under-
retrieval from VWM (that is, Gestalt principles “work” even stood as equal distances among stimuli, would increase the
when objects are not accessible perceptually). For instance, accuracy of retrievals from VWM, in comparison to
objects displayed in proximity to an object that had been irregular patterns, in which there exist unequal distances
cued were more likely reported than distant objects among stimuli. Both these hypotheses are summarized in
(Woodman, Vecera, & Luck, 2003), and the overall number Figure 1. Evidence in favor of both of them will extend our
of reported objects was larger if they were grouped in knowledge on Gestalt effects in VWM. In two experiments,
preceding display than when they were not grouped (Xu & we applied the widely-used change detection task, in which
Chun, 2007; for an analogous result pertaining to grouping a participant has to decide if the cued object in a subsequent
by similarity see Peterson & Berryhill, 2013). pattern of stimuli was either the same or different than a
Another such an example pertains to the facilitating role stimulus on the same location in the preceding pattern
of symmetry of the layout of objects for their memorization (Cowan, 2001). We expected response accuracy to be higher
in VWM. Kemps (2001), using the Corsi blocks test (tap- when a stimulus shape matches than mismatches the shape
ping manually a set of objects from the 5 × 5 matrix in the of the pattern (Experiment 1), as well as for regular than
same sequence as they were previously highlighted), has irregular patterns (Experiment 2).
demonstrated that the recall was better when the sequence
was spatially symmetrical than it was not. This result was Experiment 1
later replicated by Rossi-Arnaud, Pierroni, and Baddeley
(2006), who additionally showed that symmetry along the
Participants
vertical axis was more effective than along the horizontal
and diagonal axis, although all three types of symmetry A total of 34 women and 26 men participated (60 people).
increased recall as long as the target items were highlighted All of them were recruited via emails or adds on social
simultaneously (as this facilitated symmetry detection). All networking webpages. Mean age was 22.5 years (SD = 5.3,
these results together suggest that VWM contents are range 18 – 46). For a three hour participation each person
globally and hierarchically structured, in line with the received an equivalent of 5 euro in local currency. Each
proposals of the historical Gestalt approach. person had normal or corrected-to-normal vision and no
history of neurological problems, filled a written consent to
participate, and was informed that she or he could stop and
leave the laboratory at will. Participants were tested in a
cognitive psychology lab, in groups of a few people, under
the supervision of experimenter.
Materials and procedure
Each of 96 trials of the change-detection task consisted of a
virtual array filled with either five or six stimuli (i.e., only
some cells in the array were filled). The stimuli were sixteen
figures (e.g., a square, circle, rhombus, cross etc.), each
approximately 3 × 3 cm in size. The array was presented for
2 s, and then followed by a mask of the same size as the
array, presented for 0.8 s. In random 50% of trials, the
second array was identical to the first one, while in the
remaining trials both arrays differed by exactly one item at
one position. If they differed, then the new item was
highlighted by a square red border. If they were identical, a
Figure 1: Schematic illustration of stimuli patterns used in random item was highlighted. The task was to press one of
Experiments 1 & 2. A lower retrieval accuracy is expected two response keys depending on whether the highlighted
for mismatching (top left) and irregular patterns (top right) item differed or not in two arrays. The second array was
than for matching (bottom left) and regular patterns (bottom shown until a response was given or eight seconds elapsed.
right). Arrows indicate targets that match the pattern or not. The trials were self-paced.
43
The sole independent variable was whether an item from a
to-be-highlighted location in the first array (the target item)
had or had not the same shape as the complete pattern of
stimuli in the array. For example, the stimuli in the pattern
could form a shape of X, and the target could be either an X
figure (the matching condition) or a different figure (the
mismatching condition). See Figure 2 for illustration of the
sequence of events in a change trial of the matching
condition. In total, there were 8 training trials and as much
as 96 experimental trials, 48 trials per each condition,
randomly intermixed.
The score on this task is the estimated sheer capacity of
VWM (Cowan, 2001) that is based on the proportion of hits
(H, correct responses for arrays with one item changed) and
the proportion of false alarms (FA, incorrect responses for
unchanged arrays). The capacity of VWM is estimated to be
k items (out of N items of a memory load), on the assum-
ption that a participant produces a correct hit or avoids a
false alarm only if a cued item is transferred to his or her
VWM (with the k/N chance). If a non-transferred item is
cued, then a participant is assumed to be guessing the
answer. In consequence, the following formula evaluates the
score on the task for each N: k = N × (H – FA). The total
score on this task was the mean from the values of k in the
the five- and six-stimulus conditions, and it was an estimate
of how many items the participants actually memorized
successfully in their VWM. Such a measure also effectively
corrects for response bias (i.e., an increased tendency for
making either omission or commission errors).
Results and discussion
The mean proportion of errors was M = .73 (SD = .11).
There was a higher tendency to make omission than
commission errors, indicated by a higher accuracy in the no-
change condition (M = .79) than in the change condition
(M = .66), t(59) = 4.45, p < .001.
Most importantly, the matching condition yielded a
significantly higher k value (M = 2.74, SD = 1.10, range 0 –
4.81) than the mismatching condition (M = 2.47, SD = 1.06,
range 0 – 4.35), t(59) = 2.28, p = .030. This result indicated
that although on average participants were able to
effectively hold in their WM about two and a half object
(which is close to previous estimates; e.g., Chuderski,
Taraday, Nęcka, & Smoleń, 2012; Vogel, Woodman, &
Luck, 2001), the match between the target stimulus and the
overall pattern of stimuli increased the VWM capacity by a Figure 2: Example sequence of events in a change trial of
quarter of object on average (~10%). the matching condition in Experiment 1. The to-be-encoded
Thus, the present experiment provides the first, as far as array, presented for 2 s, is replaced by the mask of the same
we know, positive evidence that the Gestalt-like effect of size that is then replaced by another array, in which an
matching between the pattern of stimuli and the shape of a object matching the pattern of stimuli in the first array is
particular stimulus increases the likelihood of effectively substituted with another object, and highlighted with the red
encoding/retrieving that stimulus in/from VWM. These border. In the no-change trials, the matching object was
results suggest that participants encoded not only individual shown also in the second array. In the mismatching
objects, but also some ensemble representation of the condition, in both the change and no-change trials the target
higher-level pattern constituted by these objects. object shape did not match the pattern of stimuli.
44
Experiment 2
Participants
A total of 36 women and 29 men participated (65 people).
All of them were also recruited via emails or adds on social
networking webpages. Mean age was 22.8 years (SD = 4.9,
range 18 – 44). Testing conditions and gratification was the
same as in Experiment 2.
Materials and procedure
The same task was used as in Experiment 1. However, this
time the key experimental condition consisted of showing,
in both arrays presented, either the regular (distances
between neighboring stimuli in the pattern were equal; the
regular condition) or irregular patterns of stimuli (such
distances were random; the irregular condition). Similarly as
in Experiment 1, there were 48 trials per condition. The
dependent variable was above described Cowan’s k value.
Figure 3 presents example patterns of stimuli for the regular
and irregular conditions.
Results and discussion
In Experiment 2, the mean proportion of errors was M = .73
(SD = .08). Again, there was a higher tendency to make
omission than commission errors, indicated by a
significantly higher accuracy observed in the no-change
condition (M = .77) than in the change condition (M = .69),
t(64) = 2.97, p = .004.
Regarding the key manipulation, the regular condition
resulted in a slightly higher k value (M = 2.60, SD = 1.01,
range 0.46 – 4.35) than the irregular condition (M = 2.48,
SD = 1.06, range -0.23 – 4.58), however this difference was
not significant, t(64) = 0.90, p = .37. Closer investigation
revealed that the difference in accuracy between the regular
and irregular conditions was indeed significant for the no-
change trials, M = .79 and M = .75, respectively,
t(64) = 2.86, p = .005, but not for the change trials, M = .68
and M = .70, respectively, t(64) = 1.00, p = .32.
It is not clear why the effect of regularity showed up only
for the repeated arrays, but not for the changed ones. A
possible explanation is that this effect in VWM was
relatively labile (perhaps due to regularity of the pattern
participants were able to divide the moment-to-moment
attention among more objects), and the sudden change in
perceptual field strongly attenuated this effect, so it
appeared only when the same pattern of stimuli reoccurred. Figure 3: Example sequence of events in a no-change trial
However, a more reliable replication of this study is of the irregular condition in Experiment 1. The to-be-
necessary to be able to derive any firmer conclusions. encoded array, presented for 2 s, is replaced by the mask of
Anyway, Experiment 2 brought some initial support for the same size that is then replaced by the same array again,
the positive influence of Gestalt-like regularity on the in which a random object is highlighted with the red border.
number of objects held in VWM, being another example of In the change trials, that random object was changed in the
VWM capacity increase resulting from a possible encoding second array. The regular condition trials was analogous to
of some ensemble representation of the higher-level pattern that from Figure 2.
constituted by the objects displayed.
45
Conclusion An even more general theoretical consequence of the
Our hypotheses assumed that two novel Gestalt-like effects, research on Gestalt effects in VWM pertains to the crucial
the effect of matching between the shape of the single target role of VWM in abstract thinking and reasoning (e.g., strong
and the shape of the overall pattern of stimuli (i.e., a context correlations between the former and the latter; see
for that target shape), as well as the regularity (in the form Chuderski et al., 2012). If WM is so important for high-
of equal distances among stimuli) of that pattern, could level cognition, and at the same time it is so much related to
facilitate the maintenance and later retrieval of information perceptual mechanisms and representations, then it is likely
from VWM. We observed strong evidence in favor of the that substantial part of our high-level, abstract cognition
matching effect, and moderately positive evidence for the also relies to large extent on such a perceptual “engine” (see
regularity effect (it showed up only for no-change trials). Clevenger & Hummel, 2014). The seminal work on the role
Both effects were relatively weak, but easily identifiable of iconic mental models in reasoning (Johnson-Laird, 2006),
with the quite large samples we examined. or the role of perceptual symbol systems for the human
These data support and largely extend the existing conceptual system and creativity (Barsalou & Prinz, 1997)
evidence (e.g., Jiang et al., 2000; Kemps, 2001; Peterson & strongly suggest that this may be the case.
Berryhill, 2013; Woodman et al., 2003; Xu & Chun, 2007) The present work should be treated as a very initial
showing that Gestalt principles of perceptual organization, investigation of the matching and regularity effects on the
like the tendency to perceive and interpret environment in workings of VWM. Future work is needed to obtain a
the simple, orderly, and regular way (Law of Pragnanz), and stronger and replicable evidence for those two effects in the
the influence of such attributes of perceptual objects as change detection task, as well as validate these effects in
proximity, similarity, closure, and continuity for their other types of VWM tasks (to rule out a possibility that
grouping into coherent wholes, which are well-known to these Gestalt effects result from some unknown peculiarities
organize visual perception, influence also the active of the change detection task). It will also be interesting to
maintenance and access of information in VWM during the test what factors moderate these effects, for example
absence of perceptual stimulation. These results have crucial whether they show up for different types of materials or
significance for our understanding of the mechanisms and under various memory loads. Nevertheless, the present
function of one of the crucial elements of human mind study delineates a promising direction of research on the
architecture – working memory (i.e., its visual component). VWM mechanisms and representations. In general, the
One theoretical consequence of the previous studies as number of studies on Gestalt effects in VWM, although
well as the current study is that most probably represent- potentially important ones, is relatively scarce. Thus, it
tation of information in VWM does not consist of isolated seems that such a direction should be more intensively
representations of objects in a memorized scene, but it also followed in future.
includes the pattern of their mutual relations (see Clevenger
& Hummel, 2014), the overall layout (see Rensink, 2000),
Acknowledgments
and general statistical properties encoded into some This work was sponsored by the National Science Centre of
ensemble representation of the visual pattern (see Alvarez, Poland (grant no. 2014/01/D/HS6/01234).
2011; Brady et al., 2011). Although early research on VWM
was primarily focused on uncovering VWM representation References
of the single visual objects, as well as their maximal number Alvarez, G. A. (2011). Representing multiple objects as an
that can be simultaneously processed by humans (leading to ensemble enhances visual cognition. Trends in Cognitive
estimates of VWM capacity of about three or four objects; Sciences, 15, 122-131.
see Cowan, 2001; Vogel et al., 2001), currently an Baddeley, A.D., & Hitch, G. (1974). Working memory. In
increasing evidence implicates that representation of visual G.H. Bower (Ed.), The psychology of learning and
information in WM is highly hierarchical, encompassing the motivation: Advances in research and theory (Vol. 8,
binding of elementary features into composite objects, as pp. 47–89). New York: Academic Press.
well as binding of objects into groups and ensembles (for a Barsalou, L. W., & Prinz, J. J. (1997). Mundane creativity in
seminal model of such binding see Hummel & Biederman, perceptual symbol systems. In T. B. Ward, S. M. Smith,
1992). It seems that only such hierarchical representations & J. Vaid (Eds.), Creative thought: An investigation of
allow holistic and meaningful interpretation of perceptual conceptual structures and processes (pp. 267-307).
data (Brady et al., 2011). Moreover, such representations Washington, DC: APA.
more efficiently compress visual data, which often include a Brady, T. F., & Alvarez, G. A. (2011). Hierarchical
lot of structured organization and redundancy (Alvarez, encoding in visual working memory: Ensemble statistics
2011). Overall, encoding (in perception) and actively bias memory for individual items. Psychological Science,
maintaining (in VWM) visual information seem to be more 22, 384-392.
complex processes than they were initially considered.
46
Brady, T. F., Konkle, T., & Alvarez, G. A. (2011). A review Luck, S. J., & Vogel, E. K. (1997). The capacity of visual
of visual working memory capacity. Beyond individual working memory for features and conjunctions. Nature,
objects and towards structured representations. Journal of 390, 279-281.
Vision, 11, 1-34. Oberauer, K., & Eichenberger, S. (2013). Visual working
Chuderski, A., Andrelczyk, K., & Smoleń, T. (2013). An memory declines when more features must be
oscillatory model of individual differences in working remembered per object. Memory & Cognition, 41, 1212–
memory capacity and relational integration. Cognitive 1227.
Systems Research, 24, 87-95. Peterson, D. J., & Berryhill, M. E. (2013). The Gestalt
Chuderski, A., Taraday, M., Nęcka, E., & Smoleń, T. principle of similarity benefits visual working memory.
(2012). Storage capacity explains fluid intelligence while Psychonomic Bulletin & Review, 20, 1282-1289.
executive control does not. Intelligence, 40, 278-295. Raffone, A., & Wolters, G. (2001). A cortical mechanism
Clevenger, P. E., & Hummel, J. E. (2014). Working for binding in visual memory. Journal of Cognitive
memory for relations among objects. Attention, Neuroscience, 13, 766-785.
Perception, & Psychophysics, 76, 1933-1953. Rensink, R. A. (2000). The dynamic representation of
Cowan, N. (2001). The magical number 4 in short-term scenes. Visual Cognition, 7, 17–42.
memory: A reconsideration of mental storage capacity. Rossi-Arnaud, C., Pieroni, L., & Baddeley, A. D. (2006).
Behavioral and Brain Sciences, 24, 87-114. Symmetry and binding in visuo-spatial working memory.
Cowan, N., et al. (2011). A neural region of abstract Neuroscience, 139, 393–400.
working memory. Journal of Cognitive Neuroscience, 23, Woodman, G. F., Vecera, S. P., & Luck, S. J. (2003).
2852-2863. Perceptual organization influences visual working
Hummel, J. E., & Biederman, I. (1992). Dynamic binding in memory. Psychonomic Bulletin & Review, 10, 80–87.
a neural network for shape recognition. Psychological Vogel, E. K., Woodman, G. F., & Luck, S. J. (2001).
Review, 99, 480-517. Storage of features, conjunctions, and objects in visual
Jiang, Y., Olson, I. R., & Chun, M. M. (2000). Organization working memory. Journal of Experimental Psychology:
of visual-short term memory. Journal of Experimental Human Perception and Performance, 27, 92–114.
Psychology: Learning, Memory, & Cognition, 26, 683– Xu, Y., & Chun, M. M. (2007). Visual grouping in human
702. parietal cortex. PNAS, 104, 18766–18771.
Johnson-Laird, P. N. (2006). How we reason? Oxford: Xu, Y., & Chun, M. M. (2009). Selecting and perceiving
Oxford University Press.
multiple visual objects. Trends in CogSci, 13, 167-174.
Kemps, E. (2001). Complexity effects in visuo-spatial
working memory: Implications for the role of long-term
memory. Memory, 9, 13-27.
47