Gestalt Effects in Visual Working Memory

Abstract

The study investigated whether the congruence between the shape of the single target and the shape of the overall pattern of stimuli, as well as the regularity (equal distances among stimuli) of that pattern, could facilitate the maintenance of information in visual working memory (VWM). We observed strong evidence in favor of the congruency effect, and moderately positive evidence for the regularity effect. Both effects were relatively weak, but easily identifiable with the quite large samples we examined. These data support and largely extend the existing evidence showing that Gestalt principles of perceptual organization, which are well known to organize visual perception, influence also the active maintenance and access of information in VWM during the absence of perceptual stimulation.

Introduction

The last 40 years of research in cognitive science has yielded substantial knowledge on the key role of working memory (WM) for human cognition. WM is a hypothetical cognitive mechanism responsible for the active maintenance of information and its goal-driven manipulation for the purpose of the current task (Baddeley & Hitch, 1974) . WM has been shown to be involved in such mental processes as problem solving, thinking, reasoning, cognitive control, encoding and retrieving information in/from long-term memory, and many others. Although the debate on the fundamental mechanisms that determine the “workings” of working memory has been dominated by verbal paradigms of WM measurement, re-search conducted during last ten or so years is converging at the crucial role of visual working memory (VWM) in underlying many functions of working memory. Models of VWM generally assume that VWM is a relatively simple mechanism, which operates on visual representations of objects (or bundles of features defining these objects) and spatial relations among them (Clevenger & Hummel, 2014; Luck & Vogel, 1997) . Although simple, during the evolu-tion of human mind this mechanism, primarily responsible for the continuity of perception and the spatial orientation, most probably has been adapted in the service of more complex cognition, including the construction of abstract representations (see Cowan et al., 2011) , processing relat-ions (Clevenger & Hummel, 2014) , as well as using mental models and simulations to represent hypothetical states of the world (Johnson-Laird, 2006) .

The crucial attribute of VWM consists of its very limited capacity with regard to the actively maintained objects (probably 4 or less objects; Luck & Vogel, 1997) , but substantial capacity concerning the number and precision of perceptual features constituting these objects (multiplefeature objects can be maintained and recognized almost as effectively as single-feature objects; Luck & Vogel, 1997; but see Oberauer & Eichenberger, 2013) . This differentiated impact of the number of objects versus features on VWM capacity is compatible with the fact that VWM subsystem responsible for maintaining object features is located within the superior parietal lobule, whereas the binding of complete objects from those features most probably takes place within the inferior parietal lobule (Xu & Chun, 2009) . In order to explain how single features can be bound into objects, and maintained univocally, oscillatory computational models have been developed (Chuderski, Andrelczyk & Smoleń, 2013; Raffone & Wolters, 2001) .

An important area of evidence regarding factors that influence actual VWM capacity pertains to the influence of global organization of perceptual scene, that is, the fact that objects are not stored in memory independently from other items, but there exist substantial contextual effects (Brady, Konkle, & Alvarez, 2011) . For example, when context of an item (e.g., surrounding objects) changes or disappears between the to-be-memorized scene and the probe scene, retrieval of this item is worse than when the unchanged context accompanies the probe (Jiang, Olson, & Chun, 2000) . Also statistical distribution of features is important for retrieval, for instance it is easier to reject a false probe if its features differ much from the dominant features in a scene (e.g., to reject a new cold-color probe if all objects in a scene were shown in warm colors; Brady et al., 2011) . Context also influences how we recall individual items, as recall of items that possessed an extreme value of a particular visual feature (e.g., size) is often biased toward an average value of that feature in a display (Brady & Alvarez, 2011) . In total, all these context effects suggest that people encode in VWM not only particular items, but also (or – even – primarily) encode their ensemble in a way that is able to compress redundant and structured information from a display into concise but very informative higher-level representation of ensemble, which then can be used to predict features of individual items (Alvarez, 2011) .

One particularly interesting type of context/ensemble effects in VWM regards the influence of satisfying (or not) the Gestalt principles of perceptual organization (Laws of Pragnanz, Proximity, Similarity, Closure, Symmetry, and Continuity). Some studies have demonstrated that satisfying such principles by the group of objects not only helps in perceiving them in a particular way, but also facilitates their retrieval from VWM (that is, Gestalt principles “work” even when objects are not accessible perceptually). For instance, objects displayed in proximity to an object that had been cued were more likely reported than distant objects (Woodman, Vecera, & Luck, 2003) , and the overall number of reported objects was larger if they were grouped in preceding display than when they were not grouped (Xu & Chun, 2007; for an analogous result pertaining to grouping by similarity see Peterson & Berryhill, 2013) .

Another such an example pertains to the facilitating role of symmetry of the layout of objects for their memorization in VWM. Kemps (2001) , using the Corsi blocks test (tapping manually a set of objects from the 5 × 5 matrix in the same sequence as they were previously highlighted), has demonstrated that the recall was better when the sequence was spatially symmetrical than it was not. This result was later replicated by Rossi-Arnaud, Pierroni, and Baddeley (2006), who additionally showed that symmetry along the vertical axis was more effective than along the horizontal and diagonal axis, although all three types of symmetry increased recall as long as the target items were highlighted simultaneously (as this facilitated symmetry detection). All these results together suggest that VWM contents are globally and hierarchically structured, in line with the proposals of the historical Gestalt approach.

Goals of the study

The aim of the study was to examine two other contextual effects that relied on Gestalt principles, which have not been experimentally tested yet. Our first hypothesis predicted a positive effect of matching between a stimulus shape and a shape of the complete pattern of stimuli. The second hypothesis was that regularity of the pattern of stimuli, understood as equal distances among stimuli, would increase the accuracy of retrievals from VWM, in comparison to irregular patterns, in which there exist unequal distances among stimuli. Both these hypotheses are summarized in Figure 1. Evidence in favor of both of them will extend our knowledge on Gestalt effects in VWM. In two experiments, we applied the widely-used change detection task, in which a participant has to decide if the cued object in a subsequent pattern of stimuli was either the same or different than a stimulus on the same location in the preceding pattern (Cowan, 2001) . We expected response accuracy to be higher when a stimulus shape matches than mismatches the shape of the pattern (Experiment 1), as well as for regular than irregular patterns (Experiment 2).

Experiment 1

Participants

A total of 34 women and 26 men participated (60 people). All of them were recruited via emails or adds on social networking webpages. Mean age was 22.5 years (SD = 5.3, range 18 – 46). For a three hour participation each person received an equivalent of 5 euro in local currency. Each person had normal or corrected-to-normal vision and no history of neurological problems, filled a written consent to participate, and was informed that she or he could stop and leave the laboratory at will. Participants were tested in a cognitive psychology lab, in groups of a few people, under the supervision of experimenter.

Materials and procedure

Each of 96 trials of the change-detection task consisted of a virtual array filled with either five or six stimuli (i.e., only some cells in the array were filled). The stimuli were sixteen figures (e.g., a square, circle, rhombus, cross etc.), each approximately 3 × 3 cm in size. The array was presented for 2 s, and then followed by a mask of the same size as the array, presented for 0.8 s. In random 50% of trials, the second array was identical to the first one, while in the remaining trials both arrays differed by exactly one item at one position. If they differed, then the new item was highlighted by a square red border. If they were identical, a random item was highlighted. The task was to press one of two response keys depending on whether the highlighted item differed or not in two arrays. The second array was shown until a response was given or eight seconds elapsed. The trials were self-paced.

The sole independent variable was whether an item from a to-be-highlighted location in the first array (the target item) had or had not the same shape as the complete pattern of stimuli in the array. For example, the stimuli in the pattern could form a shape of X, and the target could be either an X figure (the matching condition) or a different figure (the mismatching condition). See Figure 2 for illustration of the sequence of events in a change trial of the matching condition. In total, there were 8 training trials and as much as 96 experimental trials, 48 trials per each condition, randomly intermixed.

The score on this task is the estimated sheer capacity of VWM (Cowan, 2001) that is based on the proportion of hits (H, correct responses for arrays with one item changed) and the proportion of false alarms (FA, incorrect responses for unchanged arrays). The capacity of VWM is estimated to be k items (out of N items of a memory load), on the assumption that a participant produces a correct hit or avoids a false alarm only if a cued item is transferred to his or her VWM (with the k/N chance). If a non-transferred item is cued, then a participant is assumed to be guessing the answer. In consequence, the following formula evaluates the score on the task for each N: k = N × (H – FA). The total score on this task was the mean from the values of k in the the five- and six-stimulus conditions, and it was an estimate of how many items the participants actually memorized successfully in their VWM. Such a measure also effectively corrects for response bias (i.e., an increased tendency for making either omission or commission errors).

Results and discussion

The mean proportion of errors was M = .73 (SD = .11). There was a higher tendency to make omission than commission errors, indicated by a higher accuracy in the nochange condition (M = .79) than in the change condition (M = .66), t(59) = 4.45, p < .001.

Most importantly, the matching condition yielded a significantly higher k value (M = 2.74, SD = 1.10, range 0 – 4.81) than the mismatching condition (M = 2.47, SD = 1.06, range 0 – 4.35), t(59) = 2.28, p = .030. This result indicated that although on average participants were able to effectively hold in their WM about two and a half object (which is close to previous estimates; e.g., Chuderski, Taraday, Nęcka, & Smoleń, 2012; Vogel, Woodman, & Luck, 2001) , the match between the target stimulus and the overall pattern of stimuli increased the VWM capacity by a quarter of object on average (~10%).

Thus, the present experiment provides the first, as far as we know, positive evidence that the Gestalt-like effect of matching between the pattern of stimuli and the shape of a particular stimulus increases the likelihood of effectively encoding/retrieving that stimulus in/from VWM. These results suggest that participants encoded not only individual objects, but also some ensemble representation of the higher-level pattern constituted by these objects.

Experiment 2

Participants

A total of 36 women and 29 men participated (65 people). All of them were also recruited via emails or adds on social networking webpages. Mean age was 22.8 years (SD = 4.9, range 18 – 44). Testing conditions and gratification was the same as in Experiment 2.

Materials and procedure

The same task was used as in Experiment 1. However, this time the key experimental condition consisted of showing, in both arrays presented, either the regular (distances between neighboring stimuli in the pattern were equal; the regular condition) or irregular patterns of stimuli (such distances were random; the irregular condition). Similarly as in Experiment 1, there were 48 trials per condition. The dependent variable was above described Cowan’s k value. Figure 3 presents example patterns of stimuli for the regular and irregular conditions.

Results and discussion

In Experiment 2, the mean proportion of errors was M = .73 (SD = .08). Again, there was a higher tendency to make omission than commission errors, indicated by a significantly higher accuracy observed in the no-change condition (M = .77) than in the change condition (M = .69), t(64) = 2.97, p = .004.

Regarding the key manipulation, the regular condition resulted in a slightly higher k value (M = 2.60, SD = 1.01, range 0.46 – 4.35) than the irregular condition (M = 2.48, SD = 1.06, range -0.23 – 4.58), however this difference was not significant, t(64) = 0.90, p = .37. Closer investigation revealed that the difference in accuracy between the regular and irregular conditions was indeed significant for the nochange trials, M = .79 and M = .75, respectively, t(64) = 2.86, p = .005, but not for the change trials, M = .68 and M = .70, respectively, t(64) = 1.00, p = .32.

It is not clear why the effect of regularity showed up only for the repeated arrays, but not for the changed ones. A possible explanation is that this effect in VWM was relatively labile (perhaps due to regularity of the pattern participants were able to divide the moment-to-moment attention among more objects), and the sudden change in perceptual field strongly attenuated this effect, so it appeared only when the same pattern of stimuli reoccurred. However, a more reliable replication of this study is necessary to be able to derive any firmer conclusions.

Anyway, Experiment 2 brought some initial support for the positive influence of Gestalt-like regularity on the number of objects held in VWM, being another example of VWM capacity increase resulting from a possible encoding of some ensemble representation of the higher-level pattern constituted by the objects displayed.

Conclusion

Our hypotheses assumed that two novel Gestalt-like effects, the effect of matching between the shape of the single target and the shape of the overall pattern of stimuli (i.e., a context for that target shape), as well as the regularity (in the form of equal distances among stimuli) of that pattern, could facilitate the maintenance and later retrieval of information from VWM. We observed strong evidence in favor of the matching effect, and moderately positive evidence for the regularity effect (it showed up only for no-change trials). Both effects were relatively weak, but easily identifiable with the quite large samples we examined.

These data support and largely extend the existing evidence (e.g., Jiang et al., 2000; Kemps, 2001; Peterson & Berryhill, 2013; Woodman et al., 2003; Xu & Chun, 2007) showing that Gestalt principles of perceptual organization, like the tendency to perceive and interpret environment in the simple, orderly, and regular way (Law of Pragnanz), and the influence of such attributes of perceptual objects as proximity, similarity, closure, and continuity for their grouping into coherent wholes, which are well-known to organize visual perception, influence also the active maintenance and access of information in VWM during the absence of perceptual stimulation. These results have crucial significance for our understanding of the mechanisms and function of one of the crucial elements of human mind architecture – working memory (i.e., its visual component).

One theoretical consequence of the previous studies as well as the current study is that most probably representtation of information in VWM does not consist of isolated representations of objects in a memorized scene, but it also includes the pattern of their mutual relations (see Clevenger & Hummel, 2014) , the overall layout (see Rensink, 2000) , and general statistical properties encoded into some ensemble representation of the visual pattern (see Alvarez, 2011; Brady et al., 2011) . Although early research on VWM was primarily focused on uncovering VWM representation of the single visual objects, as well as their maximal number that can be simultaneously processed by humans (leading to estimates of VWM capacity of about three or four objects; see Cowan, 2001; Vogel et al., 2001) , currently an increasing evidence implicates that representation of visual information in WM is highly hierarchical, encompassing the binding of elementary features into composite objects, as well as binding of objects into groups and ensembles (for a seminal model of such binding see Hummel & Biederman, 1992) . It seems that only such hierarchical representations allow holistic and meaningful interpretation of perceptual data (Brady et al., 2011) . Moreover, such representations more efficiently compress visual data, which often include a lot of structured organization and redundancy (Alvarez, 2011) . Overall, encoding (in perception) and actively maintaining (in VWM) visual information seem to be more complex processes than they were initially considered.

An even more general theoretical consequence of the research on Gestalt effects in VWM pertains to the crucial role of VWM in abstract thinking and reasoning (e.g., strong correlations between the former and the latter; see Chuderski et al., 2012) . If WM is so important for highlevel cognition, and at the same time it is so much related to perceptual mechanisms and representations, then it is likely that substantial part of our high-level, abstract cognition also relies to large extent on such a perceptual “engine” (see Clevenger & Hummel, 2014) . The seminal work on the role of iconic mental models in reasoning (Johnson-Laird, 2006) , or the role of perceptual symbol systems for the human conceptual system and creativity (Barsalou & Prinz, 1997) strongly suggest that this may be the case.

The present work should be treated as a very initial investigation of the matching and regularity effects on the workings of VWM. Future work is needed to obtain a stronger and replicable evidence for those two effects in the change detection task, as well as validate these effects in other types of VWM tasks (to rule out a possibility that these Gestalt effects result from some unknown peculiarities of the change detection task). It will also be interesting to test what factors moderate these effects, for example whether they show up for different types of materials or under various memory loads. Nevertheless, the present study delineates a promising direction of research on the VWM mechanisms and representations. In general, the number of studies on Gestalt effects in VWM, although potentially important ones, is relatively scarce. Thus, it seems that such a direction should be more intensively followed in future.

Acknowledgments

This work was sponsored by the National Science Centre of Poland (grant no. 2014/01/D/HS6/01234).

References

Alvarez , G. A. ( 2011 ). Representing multiple objects as an ensemble enhances visual cognition . Trends in Cognitive Sciences , 15 , 122 - 131 .

Baddeley , A.D. , & Hitch , G. ( 1974 ). Working memory . In G.H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 8 , pp. 47 - 89 ). New York: Academic Press.

Barsalou , L. W. , & Prinz , J. J. ( 1997 ). Mundane creativity in perceptual symbol systems . In T. B. Ward , S. M. Smith , & J. Vaid (Eds.), Creative thought: An investigation of conceptual structures and processes (pp. 267 - 307 ). Washington, DC: APA.

Brady , T. F. , & Alvarez , G. A. ( 2011 ). Hierarchical encoding in visual working memory: Ensemble statistics bias memory for individual items . Psychological Science , 22 , 384 - 392 .

Brady , T. F. , Konkle , T. , & Alvarez , G. A. ( 2011 ). A review of visual working memory capacity. Beyond individual objects and towards structured representations . Journal of Vision , 11 , 1 - 34 .

Chuderski , A. , Andrelczyk , K. , & Smoleń , T. ( 2013 ). An oscillatory model of individual differences in working memory capacity and relational integration . Cognitive Systems Research , 24 , 87 - 95 .

Chuderski , A. , Taraday , M. , Nęcka , E. , & Smoleń , T. ( 2012 ). Storage capacity explains fluid intelligence while executive control does not . Intelligence , 40 , 278 - 295 .

Clevenger , P. E. , & Hummel , J. E. ( 2014 ). Working memory for relations among objects . Attention, Perception, & Psychophysics , 76 , 1933 - 1953 .

Cowan , N. ( 2001 ). The magical number 4 in short-term memory: A reconsideration of mental storage capacity . Behavioral and Brain Sciences , 24 , 87 - 114 .

Cowan , N. , et al. ( 2011 ). A neural region of abstract working memory . Journal of Cognitive Neuroscience , 23 , 2852 - 2863 .

Hummel , J. E. , & Biederman , I. ( 1992 ). Dynamic binding in a neural network for shape recognition . Psychological Review , 99 , 480 - 517 .

Jiang , Y. , Olson , I. R. , & Chun , M. M. ( 2000 ). Organization of visual-short term memory . Journal of Experimental Psychology: Learning , Memory, & Cognition, 26 , 683 - 702 .

Johnson-Laird , P. N. ( 2006 ). How we reason? Oxford: Oxford University Press.

Kemps , E. ( 2001 ). Complexity effects in visuo-spatial working memory: Implications for the role of long-term memory . Memory , 9 , 13 - 27 .

Luck , S. J. , & Vogel , E. K. ( 1997 ). The capacity of visual working memory for features and conjunctions . Nature , 390 , 279 - 281 .

Oberauer , K. , & Eichenberger , S. ( 2013 ). Visual working memory declines when more features must be remembered per object . Memory & Cognition , 41 , 1212 - 1227 .

Peterson , D. J. , & Berryhill , M. E. ( 2013 ). The Gestalt principle of similarity benefits visual working memory . Psychonomic Bulletin & Review , 20 , 1282 - 1289 .

Raffone , A. , & Wolters , G. ( 2001 ). A cortical mechanism for binding in visual memory . Journal of Cognitive Neuroscience , 13 , 766 - 785 .

Rensink , R. A. ( 2000 ). The dynamic representation of scenes . Visual Cognition , 7 , 17 - 42 .

Rossi-Arnaud , C. , Pieroni , L. , & Baddeley , A. D. ( 2006 ). Symmetry and binding in visuo-spatial working memory . Neuroscience , 139 , 393 - 400 .

Woodman , G. F. , Vecera , S. P. , & Luck , S. J. ( 2003 ). Perceptual organization influences visual working memory . Psychonomic Bulletin & Review , 10 , 80 - 87 .

Vogel , E. K. , Woodman , G. F. , & Luck , S. J. ( 2001 ). Storage of features, conjunctions, and objects in visual working memory . Journal of Experimental Psychology: Human Perception and Performance , 27 , 92 - 114 .

Xu , Y. , & Chun , M. M. ( 2007 ). Visual grouping in human parietal cortex . PNAS , 104 , 18766 - 18771 .

Xu , Y. , & Chun , M. M. ( 2009 ). Selecting and perceiving multiple visual objects . Trends in CogSci , 13 , 167 - 174 .