The paper formulates an alternative view about the core function of attention claiming that attention is not selection but the deployment of extra processing capacity. This way of thinking about attention has greater explanatory power, since it proposes a common implementation both for selection and modulatory effects, and it offers a unificatory perspective on the workings of perception and cognition.
In recent years, the theoretical and empirical literature on
attention has gone through an exponential growth not just in
sheer volume but also in diversity, which has lead many to
claim that attention, after all, is not a unitary mechanism but
rather a feature of “multiple perceptual and cognitive
control mechanisms”
Traditionally, the core function of attention is claimed to
be selection: attention selects the relevant ones from the
pool of concurrently present stimuli
We call this view, according to which attention is a form
of selection, the Selection View (SV). Our aim in this paper
is to challenge this traditional understanding. In what
follows we shall argue that both the selective aspect of
attention, and its modulatory effects
According to SV, the core function of attention is selection:
it functions over information processing channels and
decides which one of these can reach further processing.
This view is in the centre of the early vs. late selection
debate that dominated much of the research on attention in
the 20th century. Early selectionists claim that basic
physical features of all stimuli are detected and processed
pre-attentively, and attention selects a few of these channels
for categorical processing
It needs to be acknowledged that SV is not a homogenous
position—there are many varieties, even beyond the
question of what point of perceptual processing attention
(that is, selection) presents a bottleneck. Originally, SV
pictured attention as a single mechanism dividing
perceptual-cognitive processing into a pre- and a
postattentive stage. However, theorists nowadays argue that
instead of one single or a few major stages of selection
filtering effects occur throughout the processing stream
These many possible stages or forms of selection are
sometimes brought under the same umbrella by the general
characterisation that attention is selection for further
processing, where the processing in question takes place
further up in the perceptual-cognitive hierarchy
Thinking of attention as selection has been the mainstream
view. But there is an alternative way of thinking about
attention, one that is often implicit in works of vision
scientists. It has long been known that the focus of spatial
attention is able to enhance the processing of visual stimuli
These studies suggest that attention is able to directly
affect how much processing capacity is to be allocated to
different stimuli (even right at the very entry level of the
perceptual system). From this perspective, attention is not a
gatekeeper but rather an information processing booster that
is able to modulate perceptual processing by affecting the
allocation of processing resources. This reinterpretation
resonates quite well with how Marisa Carrasco herself
summarises the moral of her research: “attention is involved
in distributing resources across the visual field”
This is the way of thinking about attention that we call DEP. According to this alternative account, the core function of attention is the allocation of extra processing capacity: when one voluntarily attends to a specific spatial location or when a particular stimulus automatically captures one’s attention what happens is that extra processing resources get allocated to the specific spatial location or particular stimulus. That is, attention increases the allocation of processing resources to the attended region or stimulus. It might be the case that a particular stimulus has already been processed before a novel engagement of attention, so a certain amount of processing capacity has already been allocated to it. However, when attention shifts to this stimulus extra resources get deployed facilitating the processing of the stimulus. Similarly, when one voluntarily withdraws one’s attention from a specific region or when one’s attention gets automatically disengaged from a particular stimulus the processing capacity allocated to the region or stimulus in question decreases. That is, according to DEP, the voluntary and automatic engagement and
Before moving on, a clarification is in order. So far, we
have been talking about the allocation of certain resources,
but we haven’t elucidated what resources we have in mind.
Here we propose that processing resources are best to be
understood as the natural processing capacity of the
perceptual system, i.e. representational encoding via spike
generation. Stimuli are processed through a hierarchy of
neuronal circuits that encodes stimulus-features as specific
firing patterns. Spike generation has a particular energy cost
One might wonder at this point whether the difference between SV and DEP is merely verbal. We believe that the two views are substantially different: they have different explanatory power (see this section), and integrative potential (see next section). On these grounds, we argue that DEP is preferable to SV.
As it has become increasingly evident in recent years, the
modulatory effect of attention on neural activity is a general
phenomenon. As it is sometimes put, nowadays it is
“overwhelmingly apparent” that attention modulates
neuronal responses across many stages of the
perceptualcognitive system
That is, the ability to modulate neural activity seems to be a key feature of attention—a mark of the operation of attention that, arguably, is just as widespread and fundamental as selection itself. It seems as though modulation was just as common and just as core a function of the many different forms of attention as selection is often argued to be.
Note, however, that SV sees modulation and selection as
two distinct features of attention that require two distinct
independent explanations. As some of the proponents of SV
explicitly acknowledge
Here we propose that DEP is preferable over SV because contrary to the latter the former is able to account for both selection and modulation on the basis of a single mechanism, namely the eponymous active management of processing resources.
As we have seen, DEP is motivated by the modulatory effects of attention, and therefore accounting for modulation within this framework is quite straightforward. If what attention is is the deployment of extra processing capacity, then the processing of attended information is enhanced, compared to the baseline level of processing preceding the allocation of attention. Similarly, if the disengagement of attention consists in a drop of the available processing capacity, then the processing of newly unattended information diminishes compared to the pre-disengagement level. That is, attention as the deployment of extra processing directly affects the neural activity relevant for the encoding of quality representations.
According to DEP, selection is also a consequence of the active management of processing resources. In fact, within this new framework there is a gradual shift between modulation and selection, with modulatory effects coming in various degrees and (full-blown) selection being at the far end of the same continuum.
It is a fundamental characteristic of attention that it is not
possible to attend to too many things at once or to distribute
attention over a large region of the visual field without a
decrement in processing efficiency. That is, selection seems
to be inevitable. And indeed, given our understanding of
processing resources, selection is a consequence of the
allocation of processing capacity, since energy resources
required for spike generation and representational encoding
are very limited.
So when the capacity limit is reached and all the resources are allocated then any novel act of voluntarily attending or episode of automatic attentional capture must necessarily co-occur with a withdrawal of some processing capacity from unattended regions. That is, the active management of processing resources, when operating near limit, consists in the joint allocation and withdrawal of resources: shifting attention is shifting the allocation of (i.e. re-allocating) processing capacity. Allocating extra resources to a particular stimulus decreases the amount of processing capacity that can be deployed elsewhere—this is how the particular stimulus in question gets selected and why unattended stimuli ‘fade away’.
Note that although in the case of near-limit operation all
novel instances of allocating extra processing resources to
certain stimuli co-occur with the withdrawal of some
processing capacity from other stimuli, this does not
necessarily mean the full withdrawal of attention from
unattended stimuli (i.e. full-blown selection). There very
well might be cases where only part of the processing
capacity already allocated to some stimuli gets withdrawn
with a novel engagement of attention at another location. In
these cases some (residual or excess) processing resources
remain allocated to the original (now unattended) stimuli
due to which these stimuli are still processed to a certain
degree. These are the cases of partial or distributed attention
Full-blown selection (i.e. the total blocking of the processing of unattended stimuli) only happens if all processing resources are consumed by a single stimulus (or a single set of stimuli—the attended one) and thus no processing capacity could be allocated to other stimuli.
According to DEP, thus, the core function of attention is the amplification of representational encoding of salient or task relevant stimuli via the allocation of extra processing resources. Selection effects are only by-products of this amplification and especially the corresponding attenuation of the representational encoding of inconspicuous or irrelevant stimuli that results from the fact that processing resources are of limited capacity.
Note that according this view, selection is far from being the core function of attention—it is not even a necessary consequence of the mechanisms underlying the allocation of attention. In systems without resource-limits, or (more realistically) in systems operating far from their limits, there can be spare capacity at the system’s disposal to be allocated to new stimuli. In these cases, though attention— as the deployment of extra processing resources—is very much in operation, no selection effects occur, since no processing capacity needs to be withdrawn from unattended stimuli.
This way of thinking about selection, however, might be
called into doubt by the very low level studies that have
originally motivated DEP. In a series of studies, Carrasco
and colleagues show that even in the case of very simple,
non-cluttered displays with only two stimuli, when attention
facilitates contrast sensitivity and acuity at the attended
location, trade-offs (decreased contrast sensitivity and
acuity—compared to the baseline) appear at the
nonattended location
On the face of it, this finding is in tension with the idea that selection effects occur only when the system operates near limit (i.e. when perceptual resources are fully allocated). The problem is this: in the experimental paradigms used in the studies above, the displays, typically containing only two Gabor-patches, are so simple that their capacity requirement in the baseline condition could hardly exceed, or even get close to, the capacity limit. So when one attends to the location of one of the stimuli extra processing capacity could very well be deployed without the necessary withdrawal of processing resources from other locations. That is, this finding seems to show that selection effects can occur even if the full processing capacity is only partly allocated.
We, however, think that these results are, in fact, perfectly compatible with DEP. What they suggest, is that the limited processing capacity (at least of early visual processing, see below) is always fully allocated. That is, we interpret the Carrasco studies in question as indicators that early visual processing always operates near limit. The low levels of the visual processing stream seem to be unable to reserve spare (unallocated) processing capacity that could be deployed without any occurrent cost. Therefore, processing capacity is fully allocated even when subjects are faced with the simple displays in the Carrasco studies. When attention is captured at a certain location overlapping with one of the stimuli, and extra resources get allocated to the processing of this stimulus, then resources from other locations must be withdrawn.
The claim that the limited processing capacity of low
level visual perception is always fully allocated finds plenty
of support in the literature. For example,
To sum up, selection effects do not necessarily follow from the deployment of extra processing, but whenever a (sub-)system operates near its capacity limit selection effects will always occur. As the Carrasco studies, energy considerations, or observations related to spontaneous activity suggest, early vision might very well be such a subsystem. If so, then shifts in attention are always accompanied by the re-distribution of the limited processing capacity of low level visual perception—it will be fully allocated all the time, but slightly differently: the actually attended location will receive more resources, the actually unattended locations will receive less resources.
Finally, note that this picture is compatible with the active
inhibition of unattended stimuli
According to DEP, attention is the deployment of extra processing capacity. Since the processing capacity in question is, in fact, the set of available energy resources required for spike generation, the deployment of extra processing capacity results in enhanced representational encoding. Enhanced representational encoding is a form of modulation, so modulatory effects straightforwardly follow from the deployment of extra processing capacity.
If the system operates near limit (i.e. if the resources are always fully allocated), then the deployment of extra processing capacity will lead to the withdrawal of processing capacity from unattended stimuli (via, say, active inhibition). The withdrawal of some processing capacity from unattended stimuli decreases the quality of the representational encoding of these stimuli. Diminished representational encoding of unattended stimuli is equally a form of modulation, but it is also a form of selection: it is what is called partial (or incomplete, or inefficient) selection.
Finally, if the attended stimulus consumes all processing capacity then there is no residual capacity that could spill out to unattended stimuli, so unattended stimuli are not processed at all. They are stopped being representationally encoded. This is the case of full-blown selection.
In other words, if we accept DEP and the further assumption about the near-limit operation of at least certain perceptual sub-systems, then we become able to explain both the modulatory effects and the selective nature of attention. Therefore, since SV fails to explain why attention entails the modulation of signal processing, DEP is preferable because of its greater explanatory scope.
In this section, we would like to point out that attention, as
defined by DEP, is a natural ally of modern approaches to
higher cognitive processes, especially the functioning of
working memory. Working memory is generally thought of
as the site where information conveyed by the senses and
processed by the perceptual system is brought together and
gets evaluated in accordance with the needs of ongoing
tasks
Perhaps the most characteristic feature of working memory
functions is that they are of limited capacity. In recent years,
the nature of this limited capacity has hotly been debated
Classically, the limited capacity of working memory is
interpreted as a limit in the number of discrete
representations that can actively be maintained to serve the
needs of ongoing tasks. The observations according to
which working memory is able to hold only 3-7 items at
once
Contrary to this, resource models conceptualise working
memory as a limited resource that can flexibly be
distributed between all the items in the input
Note that SV with its item-based attentional shifts is a quite good match for classical slot-based accounts of working memory. Attention, according to SV, operates over information processing channels that process characteristics of individual physical features or objects. In this context, selecting a channel and blocking others amounts to passing on information about certain features or objects while filtering out others—providing ideal input for the fixed slots of working memory. The feature or object the information of which can get through the bottleneck will be represented in working memory, whereas those that are screened off will not be remembered. Similarly, shifting attention from one feature to another means that the filtering mechanism gets repositioned to the corresponding information processing channel passing on information about a new feature and blocking others—providing just the right sort of input to fill up another slot of working memory. That is, slot-based accounts of working memory and attention as defined within SV are natural allies. Attention as it is jumping in shifts from one feature or object to another delivers exactly that kind of information that is going to end up in working memory. Slots store what attention selects.
Resource models of working memory break with the
notion of all-or-nothing representational encoding. They
claim that working memory can and often does maintain
partial (low quality) information about items that are not
necessarily given priority in accordance with actual task
demand. Note that by making this claim, and especially by
anchoring the notion of the limit of working memory to
resources required for creating quality representations,
resource models of working memory commit themselves to
the very same principles that underly DEP. Also note that
the emphasis of DEP on the continuous and flexible
reallocation of these resources is also shared by the resource
models of working memory. In fact, it seems that the very
mechanism that DEP points out as the implementation of all
attentional effects, is also able to implement the functioning
of the working memory as recent resource models describe
it. Reported overlap between attentional and working
memory functions
That is, whereas SV complements slot-based accounts of working memory quite well, it doesn’t really match modern resource models. However, DEP re-conceptualises attention in a way that makes it a perfect fit for this latter approach to working memory that has become increasingly dominant in recent years.
Finally, let’s step back and take a bird-eye-view of what has been argued for so far. Seeing the re-allocation of processing capacity as the fundamental mechanism implementing attentional effects re-defines the relationship between attention, perception, and cognition.
First, it is an essential feature of perceptual processing that its resources get continuously re-allocated. In the course of this re-allocation the processing of certain stimuli receive extra capacity making these stimuli the attended ones. Attention, thus, is not an extra mechanism working on top of perception (as the traditional SV claimed it to be), but rather an aspect or a result of how perception itself works. That is, attentional effects are just aspects of the natural unfolding of how the perceptual system does what it does.
Second, recent approaches to working memory teach us that even higher cognitive functions share the common characteristic of flexibly allocating processing resources. It seems, thus, that the fundamental principles our proposal points out underlie a broad range of mental processes from attention through perception to cognition.
This work was supported by the FWO Postdoctoral Fellowship 12B3914N and the DFF - EU MCA - COFUND Mobilex grant 1321-00165 (PF), and the EU FP7 CIG grant PCIG09-GA-2011–293818 and the FWO Odysseus grant G.0020.12 (BN).