=Paper=
{{Paper
|id=Vol-1419/paper0056
|storemode=property
|title=Access and Use of Contextual Expectations in Visual Search during Aging
|pdfUrl=https://ceur-ws.org/Vol-1419/paper0056.pdf
|volume=Vol-1419
|dblpUrl=https://dblp.org/rec/conf/eapcogsci/BorgesC15
}}
==Access and Use of Contextual Expectations in Visual Search during Aging==
https://ceur-ws.org/Vol-1419/paper0056.pdf
Access and Use of Contextual Expectations in Visual Search during Aging
Miguel Borges (m.borges@student.maastrichtuniversity.nl) 1
Moreno I. Coco (micoco@psicologia.ulisboa.pt)
University of Lisbon, Department of Psychology,
Alameda da Universidade, 1600-214 Lisboa, Portugal
Abstract extent in short visual-working memory (Brockmole, Parra,
Expectations are routinely utilized by the cognitive system to
Sala, & Logie, 2008).
facilitate processing and optimize responses. However, as we Most of the above studies on aging focused on low-level
age, the access and use of expectations might undergo signifi- features, with few other studies approaching this topic from
cant changes. In an eye-tracking study, we investigate how a naturalistic perspective, where knowledge-based rather
search performance in young and old age is influenced by than stimulus-based processing is examined (Park, 2012).
contextual priming of expectations, and the contextual consis- Thus, it remains an open question whether knowledge-based
tency of the target object. We demonstrate that older partici-
expectancy mechanisms undergo similar changes to those
pants congruently primed perform significantly worse, espe-
cially when the target object is inconsistent with the scene observed on low-level stimulus based information. In this
context (e.g., exposed to a ‘restaurant’ scene before perform- study, we precisely investigate the role of expectations by
ing the search in another ‘restaurant’ scene, but looking for an comparing younger and older observers in a visual search,
‘iron’ rather than ‘bread’). Even though congruency and con- contextual priming task situated in naturalistic scenes.
sistency differentially impact on the eye-movement responses Priming paradigms are used to probe, and activate, se-
of the two age groups, they do not display any interaction. mantic representations in memory. For example, if a prime
Overall, this study suggests that reliance on mechanisms of
contextual expectations becomes stronger when we get older. word is semantically related to a target word (plant-grass),
participants' lexical decision responses are shorter than
Keywords: aging; active vision; visual search; contextual when the prime word does not belong to the same semantic
expectations; priming; eye-movements. category (Stanovich & West, 1981).
Effects of priming manifest also during visual attention.
Introduction Mudrik and Koch (2013), for example, show in a congru-
Plates and forks are more likely to be found in kitchens than ency detection task that contextual priming occurs uncon-
in bathrooms. Expectations of this sort are constantly gener- sciously, with scenes presented for as little as 33ms. More-
ated by our cognitive system to predictively anticipate the over, repeated exposure to the same visual scene can im-
state of events, and pro-actively generate appropriate re- prove the speed of search, but only for the same target ob-
sponses (Bar, 2007; Clark, 2013). ject (Vo & Wolfe, 2012). Additionally, as aging impacts on
Visual attention is a cognitive modality that relies heavily executive control and inhibition suppression, search per-
on such contextual expectations to maximize performance in formance in seniors would benefit from valid cues, and
visual tasks (Torralba, Oliva, Castelhano, & Henderson, conversely, take longer to recover from invalid cues (e.g.,
2006). In particular, the allocation of visual attention in Müller-Oehring, et. al. 2013)
search tasks improves when semantic information of the The main goal of the current study is to establish how
target is combined with the contextual information wherein contextual expectations are utilized by the visual system by
the target is embedded (Malcolm & Henderson, 2010). younger (under 25 years) and senior (over 65 years) parti-
Moreover, this joint use of target and context information cipants during a search task. In particular, using a priming
can occur at early stages of search, as long as the quality and paradigm, we probe the congruency of contextual informa-
quantity of the contextual information available is adequate tion between two scenes (the prime scene and the search
(Spotorno, Malcolm, & Tatler, 2014). scene), sequentially presented (e.g., bathroom-bathroom,
However, as we age, the visual system deteriorates and versus, kitchen-bathroom), and the consistency of the target
performance on visual tasks starts to decrease (Potter, Gre- object with the search scene (e.g., a toothpaste versus a
aly, Elliott & Andrés, 2012; Brockmole and Logie, 2013). In fork). In line with research showing that seniors have diffi-
particular, visual search for a target, manipulated on low- culties with inconsistent cues, we expect them to display a
level features (e.g., color, shape and orientation) and embed- significantly worst search performance when the target ob-
ded in an array of distractors (set of increasing sizes) wors- ject is inconsistent with the primed information. Their diffi-
ens with increasing age (Humphrey & Kramer, 1997), even culties with inhibition suppression would imply an in-
though the age-related differences do not manifest to same creased difficulty to detect a target object when it inconsist-
ently deviates from the congruently primed contextual ex-
pectations.
1
MB is now at Maastricht University, FPN
358
Figure 2: Trial run. Target object and Prime scene are congruent
with the search scene (restaurant-bread).
Figure 1: Design. Participants are primed (left column) with
either a congruent (restaurant) or an incongruent scene (pool hall), group scored an average of 27.1 out of 30 on the MoCA (SD
and then see a search scene (right column) wherein a consistent or = 1.83), while the elderly group scored an average of 24 out
inconsistent target is placed. The target objects (consistent = ‘bas- of 30 (SD = 2.81). The groups significantly differ in their
ket of bread’; inconsistent = ‘an iron’) are highlighted in red, and performance, t(32.8) = 4.46, p < 0.001), however, none of
zoomed in.
the participants in the elderly group performed below the
cut-off score of 19.
Our study Design
In this eye-tracking study, we compare the behavioural per- Our 2x2x2 experimental design crosses object Consistency
formance (accuracy and response time), and attentional re- (Consistent or Inconsistent), contextual Congruency (Con-
sponses (search latency, first fixation and total gaze) of gruent, Incongruent) between the prime scene and search
younger and older participants in a visual search task situ- scene, and a between-participants age Group condition
ated in naturalistic scenes. We investigate how contextual (younger or older participant). Refer to Figure 1 for an ex-
priming and the semantic consistency of the search target ample of the design material and Figure 2 for the trial run.
mediates performance during aging. For example, a participant could be primed with a Restaur-
ant scene, then presented with another Restaurant scene, and
Method previously asked to search for a basket of bread. This ex-
ample corresponds to the Congruent and Consistent experi-
Participants mental condition.
Fifty-four participants Portuguese native speakers, divided
into two age groups: 32 younger (28 women), age 22.63 Material
(SD = 9.03), University students; and 22 older volunteers We utilize 42 experimental scenes, 42 filler scenes and 84
(19 women), age 67.9 (SD = 4.75), mostly recruited from prime scenes, which are all naturalistic, real-world
senior universities, with no medical history of photographs. In particular, the 42 experimental items were
neuropsychological illness Both groups had normal or drawn from 7 different indoor scenarios (Restaurant,
corrected to normal vision. The study was approved by the Church, Office, Waiting Room, Bathroom, Living Room,
Ethics Committee of the Department of Psychology, and Kitchen) with 6 items per scenario. The scenes were
University of Lisbon (FPUL). captured in Lisbon, and surrounding areas, with a congruent
All participants underwent the Montreal Cognitive As- or an incongruent object placed by the photographer (refer
sessment Test - Portuguese Version (MoCA) (Simões et al., to section Validity pre-tests of experimental material for
2008), used as a criteria to detect potential mild cognitive details on the validation of the consistency between the
deficit, especially for the older group. The MoCA was ap- target objects and the search scene). The position of the
plied on paper with instructions given orally. The young target object was counterbalanced with left-right conditions
to avoid the development of search strategies. We decided to
take photographs of the scenes to obtain a naturalistic visual
saliency, and avoid uncontrolled artifacts (e.g., brightness or
359
contrast) that could be introduced when the target object is Equipment
digitally inserted into the scene. Material was distributed in Visual stimuli were presented on a LG Flatron L194ws 19-
4 lists using a Latin-Square Design. inch LCD screen with a 60 Hz refresh rate, 55cm away from
Primes and fillers were selected from various freely participants' eyes. Eye-movements were recorded using an
available online databases, such as the SUN database (Xiao, SMI IVIEW X™ HI-SPEED eye-tracker at a sampling rate
Hays, Ehinger, Oliva, & Torralba, 2010) or Flickr. The of 1250 Hz on a 21” screen (1024 x 768 px. image resolu-
primes were selected on the basis of their Congruency with tion). Viewing was binocular but only the participant’s dom-
respect to the experimental scenes (refer to next section). inant eye was tracked (determined by a prior parallax test).
Moreover, we made sure that the prime scene never Chin rest, and forehead support were used to stabilize the
contained the target object. The same databases were used head position, and keep viewing distance constant. Parti-
for the filler scenes, and we have chosen scenes belonging cipants logged their answers regarding the presence (or not)
to similar contexts. No filler trials did included the target of the target object in the scene, using a Logitech Cordless
object. Rumblepad 2 controller. They used 3 buttons for: “Yes”,
“No” and to advance to the next trial. The experimental pro-
Validity pre-tests of experimental material tocol was controlled by ExperimentCenter 3.2 (SMI).
Firstly, we asked 10 students of the FPUL (6 women)
between the ages of 20-25 years (mean = 21.5, SD = 1.433)
to name 8 objects they considered typical of each of the 7 Procedure
contexts. 198 unique objects were produced overall, and Participants were first presented with the target object (a
among those we selected the 8 most nominated objects for written cue for 500ms). Then, a fixation cross appeared in
each context (e.g., pillow for context bedroom), as the the center of the screen for 1000ms after which the prime
consistent targets. The 8 inconsistent objects, instead, were scene is presented for 250ms, followed by a mask 2 of the
drawn from another pool of objects never mentioned by any same scene for 250ms. The mask is utilized to disrupt feed-
of the participants. A new group of 20 participants (12 back between lower and higher visual areas and keep the
women, mean age = 21.2, SD = 1.23), was then asked to perception of the prime scene on a subliminal level (Enns &
judge the consistency between the target objects previously Di Lollo, 2000). Finally, the search scene appears and the
selected (8 consistent, 8 inconsistent) and the embedding participants can inspect it until they self-terminate the trial
context on a 6-point Likert scale (1 – Totally inconsistent; 6 by stating whether they saw (or not) the target object.
– Totally consistent). We selected the 6 objects that had Each block contained 42 experimental trials and 42 filler
highest (consistent) and lowest (inconsistent) scores for trials, for a total of 84 trials, which were randomly presented
each context (e.g., Soap – Bathroom: mean = 5.9, SD = to each participant. The target object was present in the ex-
0.31; Flashlight-Bathroom: mean = 1.3, SD = 0.67). Finally, perimental trials, and absent in the filler items guaranteeing
a new group of 12 participants (6 women, mean age = 22.5, a balanced 50-50 distribution of yes/no responses, as well as
SD = 1.87) was asked to respond to another Likert scale keeping the participants engaged in the search task. 4 prac-
questionnaire and rate: (a) the contextual congruency tice trials were administered at the beginning of the experi-
between the prime scene and the search scene, (b) the mental session to familiarize participants with the task. A
typicality of the prime scene and search scene given their 15-points calibration procedure was performed at the begin-
context, (c) the consistency of the target object within the ning of the trial; and repeated every 10 trials (8-points) to
search scene, and (d) the ease of recognizing the target ensure accurate tracking. The calibration was accepted when
object given its linguistic denotation. angle threshold was ≈ 0.8/1.2 for x and y respectively. On
The purpose of this last questionnaire was to empirically average, the younger population had 0.61º ± 0.42º deviation
validate the Congruency and Consistency manipulations. error on the x-axis. and 0.72º ± 0.58º on the y-axis. The
Results show that: (a) the difference between congruous older population had 0.82º ± 0.60º deviation error on the x-
prime scene/search scene (mean = 5.4, SD = 1.09) and their axis and 1.12º ± 0.79º on the y-axis. The experiment was ex-
incongruent version (mean = 1.5, SD = 0.92), was statistic- plained using written instructions and the eye-tracking ses-
ally significant (t(489) = 44, p < 0.001); (b) both prime sion lasted for approximately twenty to thirty minutes.
scene (mean = 5, SD = 0.87) and search scene (mean = 5 ,
SD = 0.86) were considered to be highly representative ex- Analyses
amples of their context (t(503) = 0.167, p = 0.87); (c) the The performance measures analyzed are: (a) response ac-
target objects consistent with the search scene (mean = 5.1, curacy, a binomial variable for correct and incorrect re-
SD = 1.6) were rated as significantly more likely than their sponses (1, 0), and on correct trials only, (b) the response
inconsistent counterparts (mean = 1.7, SD = 1.2) t(470) = time to pass to the next trial (in ms.). From the eye-move-
28, p < 0.001); and (d) the target objects were rated as ment responses to the target object, only for correct trials
highly identifiable by their linguistic denotation (mean = 5.3
SD = 1.7). 2
The masks were generated by splitting the prime scene into 12
by 9 quadrants, 100x100 pixel squares, and randomly shuffle them.
360
Figure 3: Means and SE for accuracy and reaction time, grouped Figure 4: Means (SE) of search latency, first fixation and total
by Congruency of the prime scene with the target scene duration, grouped by Congruency of the prime scene with the
(Congruent/Incongruent), Consistency of target object search scene (congruent/incongruent), Consistency of target object
(consistent/inconsistent) and age Group (older/younger). We (consistent/inconsistent) and age Group (older/younger). We
distinguish between the different experimental conditions using distinguish between the different experimental conditions using
colour and density of lines. color and density of lines.
we compute: (c) the search latency in ms. (i.e., the time participants are overall more accurate than older participants
from scene onset until the target object is looked at for the [F (1,1153) = 44.4, p < 0.001]. Moreover, we find a signifi-
first time), (d) the duration of the first fixation in ms., and cant two-ways interaction Consistency:Group, whereby
(e) the total duration of fixations on target-object in ms. older participants detect more accurately inconsistent than
These eye-movement measures are commonly used to consistent object [F (1,1153) = 4.22 p < 0.05] Moreover, we
characterize attentional performance in search tasks observe a significant three ways interaction
(Malcolm & Henderson, 2010). Fixations to the target Congruency:Consistency:Group, whereby response accu-
object AOI were mapped using BeGaze (SMI). Statistical racy of older participants drastically drops when prime
tests on the dependent measures were carried out using scene and search scene are Congruent, but the target object
mixed-ANOVAs where the between-participant independent is Inconsistent with the search scene [F (1,1153) = 14.07,
variables are Congruency between the prime scene and the p < 0.001].
search scene (2 levels: congruent and incongruent), object
Consistency (2 levels: consistent and inconsistent) and the Reaction time
within-participant Group variable (2 levels: younger and
older). All analyses are performed using R (ver. 3.0.2, R Scenes where the target object was consistent with the
Core Team, Austria). context of the search scene took longer to be responded [F
(1, 1109) = 17.83, p < 0.0001]. Older participants took
longer to respond than younger participants [F (1, 1109) =
Results 309.82, p < 0.0001]. When looking at the two-ways
interaction, we find that a search scene containing a
Accuracy Consistent target object, which was Congruently primed [F
In Table 1, we report means and standard deviation for each (1, 1109) = 8.98, p < 0.01] yielded faster response times
dependent measure across all experimental conditions, than a scene with an Inconsistent target object; especially in
which are also visualized in Figure 3 and 4. We observe a older participants [F (1, 1109) = 6.77, p < 0.01].
significant main effect of Congruency, whereby prime
scenes congruent with search scenes improve the detection
of the target object [F (1, 1153) = 10.2, p < 0.01]. Younger
361
Table 1: Mean results. Means and standard deviations as a function of the two Prime context conditions and two Object consistency con -
ditions for each of the age groups.
Search latency more strongly the case when the object is also inconsistent.
Young participants are overall faster than older participants By probing an unrelated contextual template, participants
to look at target object for the first time [F (1, 1109) = may become more aware of inconsistency within the search
95.81, p < 0.0001]. Search latency is faster when the object scene. The most interesting result, however, is that the
is Inconsistent with the search scene, compared to when it is detection performance of older participants abruptly drops
Consistent [F (1, 1109) = 24.06, p < 0.001]. However, when when prime and search scenes are congruent, but the target
prime scene and target scene are Congruent, and the target object is inconsistent with the search scene. This result
object is Consistent, we observe significantly shorter search confirms previous literature showing that older adults have
latencies [F (1, 1109) = 4.19, p < 0.05]. more difficulties to inhibit invalid cued information (Müller-
Oehring, 2013). Another possibly related explanation is that
First fixation older participants might rely more strongly on contextual
expectations than younger participants. So, when primed
On the duration of the first fixation, we only find that: (1) with an appropriate context (i.e., restaurant-restaurant), they
consistent target objects are fixated the first time for longer generate strongly structured expectations for consistent
than inconsistent objects [F (1, 1109) = 4.42, p < 0.05]; and information (e.g., finding the bread on the table of a
(2) older participants linger on the target object for longer restaurant). Thus, when the target object violates such
than younger participants [F (1, 1109) = 5.05, p < 0.05]. expectations, they do not manage to inhibit contextual
priming, and the target escapes their attention. These results
Total fixation
are largely corroborated by the response times on correct
On the total fixation duration to the target object, we find responses, where again we find faster response times with
main effects of Consistency, whereby participants fixated an incongruent primes for inconsistent target objects. Crucially,
Inconsistent target object for longer than when it was Con- older participants are much slower to respond with
sistent [F (1, 1109) = 9.10, p < 0.01]; and of Group, with inconsistent objects after being primed with contextually
younger participants displaying shorter total duration of fix- congruent information. As just said but from a slightly
ation than older participants [F (1, 1109) = 32.96, different angle, this result might indicate that young
p < 0.001]. Furthermore, a two-way interaction effect participants better adapt contextual expectations to the
between these two variables was also observed, whereby actual search scene, and hence are better able to resolve the
older participants fixated the target object overall less when mismatch generated by the inconsistency of the target object
it was Consistent with the scene [F (1, 1109) = 19.92 with the primed expectations. On eye-movement responses,
p < 0.001]. older participants compared to younger participants are
overall slower locating the target object and spend more
Discussion time processing its identity during the first fixation and
Both age groups are clearly influenced by contextual overall. Moreover, inconsistent objects are found faster than
priming, object consistency, as well as by their interaction. consistent object irrespective of the group, in line with
In particular, correct detections of the target object strongly previous literature manipulating this experimental factor
depend on the type of priming participants receive. (Bonitz & Gordon, 2008). Both groups improve their search
Incongruent primes improve target detection, but this is latency if the priming is congruent, and the object is
362
consistent. This indicates that appropriate contextual award to MIC (SFRH/BDP/88374/2012), are gratefully
priming can improve performance, but only if the object to acknowledged.
be looked at is consistent with the primed information.
Crucially, however, we did not find a significant three-ways References
interaction between Congruency, Consistency and Group; Bar, M., 2007. The proactive brain: using analogies and
which might indicate that aging does not really mediate the associations to generate predictions. Trends in cognitive
joint influence of Consistency and Congruency on sciences 11 (7), 280–289.
attentional responses. Bonitz, V. S., & Gordon, R. D. (2008). Attention to smoking
related and incongruous objects during scene viewing.
Conclusion Acta Psychologica, 129, 255–263.
Expectations are routinely used by the cognitive system to Brockmole, J., Parra, M., Sala, S., Logie, R. (2008). Do
anticipate information about real-word events, and optimize binding deficits account for age-related decline in visual
appropriate action-responses (Bar, 2007; Clark, 2013). working memory? Psychonomic Bulletin & Review,
Visual attention is one of the components that can benefit 15(3), 543-547.
from contextual expectations in tasks such as visual search, Brockmole, J. R., & Logie, R. H. (2013). Age-related
(Malcolm & Henderson, 2010; Spotorno et. al. 2014). change in visual working memory: a study of 55,753 par-
Aging, however, might impair attentional performance, and ticipants aged 8–75. Frontiers in Psychology, 4.
consequently change the way contextual expectations are Clark, A., 2013. Whatever next? predictive brains, situated
accessed, and subsequently used, to allocate attention on agents, and the future of cognitive science. Behavioral
scenes. For example, visual performance in search tasks, and Brain Sciences 36 (03), 310 181–204.
manipulating low-level features, appears to degrade with Enns, J. & Di Lollo, V. (2000). What's new in visual
increasing age (Humphrey & Kramer, 1997). An open masking? Trends in Cognitive Sciences, 4(9), 345-352.
challenge was to investigate whether knowledge-based Humphrey, D. G., & Kramer, A. F. (1997). Age differences
expectations undergo similar degradation, and especially, in visual search for feature,conjunction, and triple-con-
how attentional mechanisms would be differently used. junction targets. Psychology and Aging, 12, 704–717.
In the current study, we used a contextual priming Malcolm, G. L., & Henderson, J. M. (2010). Combining
paradigm to probe expectations, and compared visual top-down processes to guide eye movements during real-
performance in a search task between younger and older world scene search. Journal of Vision, 10(2), 4.
observers. We demonstrated that older observers rely more Mudrik, L., & Koch, C. (2013). Differential processing of
strongly on consolidated expectations than younger invisible congruent and incongruent scenes: A case for un-
observers. In particular, when the prime and search scene conscious integration. Journal of Vision, 13(13), 24.
overlapped in context, older participants had a worse Müller-Oehring, E. M., Schulte, T., Rohlfing, T., Pfeffer-
performance than the younger, especially if the target object baum, A., & Sullivan, E. V. (2013). Visual search and the
was inconsistent with the primed expectations. This result aging brain: discerning the effects of age-related brain
suggests that older observers might have more difficulties volume shrinkage on alertness, feature binding, and atten-
re-computing the semantics of individual objects when tional control. Neuropsychology, 27(1), 48.
primed, and hence compensate for unexpected information. Park, Denise, ed. Cognitive aging: A primer. Psychology
Nevertheless, eye-movement responses did not show a Press, 2012.
reliable difference in this interaction between congruency Potter, L. M., Grealy, M. A., Elliott, M. A., & Andrés, P.
and consistency due to aging. Both groups experienced (2012). Aging and performance on an everyday-based vis-
better search performance when the target object was ual search task. Acta Psychologica, 140(3), 208-217.
consistent with expectations, but no difference was observed Spotorno, S., Malcolm, G. L., & Tatler, B. W. (2014). How
in this interaction due to age. context information and target information guide the eyes
This study opens intriguing questions on the role of from the first epoch of search in real-world scenes.
primed contextual expectations in clinical populations Journal of Vision, 14(2), 7.
affected by neuro-degenerative disease (e.g., Alzheimer); Stanovich, K. E., & West, R. F. (1983). On priming by a
and especially on how would behavioural patterns compare sentence context. Journal of Experimental Psychology:
with a normally aging population. General, 112(1), 1.
Overall, this work confirms the important role that Torralba, A., Oliva, A., Castelhano, M., & Henderson, J.
contextual information plays on the active allocation of (2006). Contextual guidance of eye movements and atten-
visual attention and shows that reliance on it may become tion in real-world scenes:The role of global features in ob-
stronger as we age. ject search. Psychological review, 4(113), 766–786.
Võ, M. L. H., & Wolfe, J. M. (2012). When does repeated
Acknowledgments search in scenes involve memory? Looking at versus
Fundação Amadeu Dias for student award to MB, and looking for objects in scenes. Journal of Experimental
Fundação para a Ciência e Tecnologia research fellowship Psychology: Human Perception and Performance, 38(1),
23.
363