=Paper=
{{Paper
|id=Vol-1419/paper0029
|storemode=property
|title=The Influence of Cognitive Load and Amount of Stimuli on Entropy through Eye Tracking Measures
|pdfUrl=https://ceur-ws.org/Vol-1419/paper0029.pdf
|volume=Vol-1419
|dblpUrl=https://dblp.org/rec/conf/eapcogsci/FabioIEMCCSF15
}}
==The Influence of Cognitive Load and Amount of Stimuli on Entropy through Eye Tracking Measures==
https://ceur-ws.org/Vol-1419/paper0029.pdf
The Influence of Cognitive Load and Amount of Stimuli on Entropy through Eye
tracking measures
Rosa Angela Fabio (rafabio@unime.it)
Department of Cognitive Science, University of Messina, 6 Via Concezione
Messina, Italy
Chiara Incorpora (c.incorpora@unicatt.it)
Department of Psychology, Catholic University of Milan, 1 Largo Gemelli
Milan, Italy
Antonino Errante (antonino.errante89@yahoo.it)
Department of Neuroscience, University of Parma, 6 Via Volturno
Parma, Italy
Nasrin Mohammadhasni (n.mohammadhasani@modares.ac.ir)
Department of Humanities, University of Tarbiat Madares, Jalal Ale Ahmad Highway
Tehran, Iran
Tindara Caprì (tindaracapri@gmail.com)
Department of Cognitive Science, University of Messina, 6 Via Concezione
Messina, Italy
Cristina Carrozza (ccarrozza@unime.it)
Department of Cognitive Science, University of Messina, 6 Via Concezione
Messina, Italy
De Santis Sandro (desantis@unipa.it)
Department of Psychology, University of Palermo, Via Centrale
Palermo, Italy
Alessandra Falzone (amfalzone@unime.it)
Department of Cognitive Science, University of Messina, 6 Via Concezione
Messina, Italy
Abstract
Entropy can be described qualitatively as a measure of energy
Introduction
dispersal. The concept itself is linked to disorder: entropy is a The term “entropy” comes from the Greek εντροπία “a
measure of disorder, and nature tends toward maximum turning toward, from εν- "in" + τροπή "a turning", and is a
entropy for any isolated system. One of the fields in which measure of the unavailability of a system’s energy to do
energy dispersal can be quantified is eye scanning in visual
work. From an evolutionary perspective, the fundamental
search. It is well known that visual research time in eye
scanning is influenced by the number of targets to explore: goal of a nervous system is to integrate appropriate
the higher the number of targets, the longer the exploration perceptual frames and behavioral responses with the steady
time. The aim of this study is to understand whether the flow of sensory information, so that biological needs can be
exploration time on non target stimuli depends on cognitive adequately satisfied (Swanson, 2003). Consequently, there
load and on the number of distracting stimuli. 26 voluntary are two primary domains of uncertainty that must be
students (mean age and standard deviation: 23.53 and 3.2) contended with from a psychological perspective:
were involved in the study. Eye-Tracker technology was used
with an intuitive and accessible graphic interface. The
uncertainty about perception and uncertainty about action
subjects were asked to detect and look at a target, as quickly (Hirsh, Mar, & Peterson, 2012).
as possible. During this task, in the first study subjects were In psychology, researchers have used entropy to measure
also asked to listen to and repeat a list of numbers read aloud basic cognitive limits (Miller's [1956] magical number
by an experimenter. In the second study distracting stimuli seven plus or minus two) and, as a stimulus property, to
were manipulated by increasing their number. Results showed predict aesthetic preferences (Berlyne, 1974b). Another
that both the amount of cognitive load and the number of field in which energy dispersal can be quantified is visual
distracting stimuli increase the entropy of eye movement.
Results are discussed in terms of entropy theories. scanning related to eye movement. Wang et al. (2010)
propose a biology-inspired bottom-up computational model
Keywords: entropy; eye tracking; top down processes, of attention based on visual salience. The authors propose a
cognitive load; bottom up elaboration;
new visual scanning measure derived from the principle of
information maximization. This principle suggests that the
human visual system (HVS) tends to focus on the most
199
informative points on an image in order to efficiently entropy could be associated with higher mental workload as
analyze the scene. They simulate the computational function well (Kruizinga, Mulder, & de Waard, 2006). Other more
of visual saliency in the brain in which the saliency is recent studies (Camilli, Nacchia, Terenzi & Di Nocera,
defined as Site Entropy Rate (SER) based on the principle 2008) using spatial statistics algorithms report that when the
of information maximization. The experiments demonstrate mental workload is high, eye fixation is dispersed, while
that the proposed model achieves the state-of-art when mental workload is low, eye fixation is clustered.
performance of saliency detection. In this study entropy of visual scanning was defined by:
The basic assumption of eye movement is that the DTNT = TT – DTT, where TT is the total duration of time
observer’s attention is usually held only by certain elements search and DTT is the fixation time on the target stimulus.
of the picture, and so eye movements reflect human thought The rationale to measure entropy in such a way is that the
processes; so the observer's thought may be followed to amount of energy dispersal can be defined by randomness of
some extent by recording eye movements (the thought search behavior, i.e. by the time lost to find the correct
accompanying the examination of the particular object). target, the time lost in coming back to distracting stimuli
From these records it is easy to determine “which elements and the time lost in fixating the eye on one or more
attract the observer's eye (and, consequently, his thought), distracting stimuli. By using an implicit free-viewing task to
in what order, and how often” (Yarbus, 1967). In 1980, Just search for the target, we can compare the distribution of
and Carpenter formulated the influential Strong eye-mind attention across a range of tasks.
Hypothesis, the hypothesis that “there is no appreciable lag Specifically the purpose of this study is to understand
between what is fixated and what is processed”. If this whether the energy dispersal of the eye scanning
hypothesis is correct, then when a subject looks at a word or (exploration time on non target stimuli, entropy) also
an object, he or she also thinks about it (processes it depends on the intensity of the cognitive load and on the
cognitively), and for exactly as long as the recorded fixation number of distracting stimuli.
lasts. During the 1980, the eye-mind hypothesis was often
questioned in the light of covert attention (Posner, 1980), Study 1
that is the attention to something at which one is not
looking, which people often do. If covert attention is Method
common during eye tracking recordings, the resulting scan Subjects
path and fixation patterns will often not show where our 26 voluntary students (16 females and 10 males) attending
attention has been, but only where the eye has been looking, courses at the Catholic University of the Sacred Heart of
and so eye tracking will not indicate cognitive processing. Milan. The age of the subjects was between 19 and 31
According to Hoffman (1998), the current consensus is that (mean and standard deviation: 23.53 and 3.2). The sample
visual attention is always slightly (100 to 250 ms) ahead of comprised normal-sighted subjects, all with visus values
the eye. But as soon as attention moves to a new position, between 0.9, and 1 with their usual visual correction where
the eyes will want to follow (Deubel & Schneider, 1996). applicable.
Based on this current consensus, in this work we use the
Eye Tracker to analyze visual search. Task specifications
In the view of Siemens (2009), educators need to embrace Eye-Tracker consists of an instrument that is
the unpredictability because randomness in codifying transportable, works without further equipment and can be
information can lead to deficit in learning. The problem is used in normal room light conditions. In this study, we used
that all entropy decreasing-transformations can’t leave any the model “iAble© - MyTobii®” (“D10” version), a system
trace (e.g. in a memory) of them having happened of ocular and vowel control. The system was composed by
(Maccone, 2008) an eye-tracker, by a computer and related software. The
Visual search related to visual scanning randomness was graphic interface was highly intuitive and accessible, and
related to entropy processes. Visual search is influenced by the controls were projected according to a multimodal input
both top-down and bottom-up processes. Hirsh, Mar, & (vocal and ocular command).
Peterson (2012) proposed a computational model of entropy The interface’s efficacy was validated thanks to
mainly based on bottom-up processing. usability’s studies and “beta test” on the field, perceived
In this study, with reference to bottom-up processes, the primary with subjects.
number of items to explore in order to find the target is The picture used for the eye tracker’s first task consisted
important: the higher the number of targets, the longer the of three different complex images, in which a little yellow
exploration time. With reference to top-down processes, the duck was hidden. The first image showed a kitchen, the
role of cognitive load is controversial. Some studies (e.g., second showed a country view, and the third represented a
Hilburn, Jorna, Byrne, & Parasuraman, 1997) report that supermarket (for the image, see fig. 1 at the end of the
visual scanning randomness (or entropy) is related to mental paper).
workload: high task load conditions would generate less The complex images used in this part of the research were
randomness than would low task load conditions. Other divided into two groups, 5 for the first test and 3 for the
studies show an opposite pattern, namely that higher second one.
200
Before beginning the experiment, the perceptual salience 7) Presentation of a complex image in which the target
of this complex stimuli was weighted. An independent stimulus was hidden (entire cognitive load);
sample of 12 subjects of the same age of the study sample 8) Last slide in which “thank you” appears.
judged the complexity of the ten complex cards within The whole experiment took about 40 minutes. Four
which there were those of the fig. 1; the judgment ranged measures of task performance were recorded: the time
from 1 (not complex) to 10 (very complex). The results of before correct fixation (DBF); the number of fixations on
the evaluation of the three final complex stimuli were very target stimulus (NFT); the duration of fixation time on the
similar. The means were 7.2, 7.8, 7.1 respectively. correct target stimulus (DTT); the duration of fixation time
on the non-target stimuli (DTNT).
Procedure This last measure was obtained from: DTNT = TT – DTT,
The tasks of the experiment were administered during a where TT is the total duration of time search and DTT is the
break between ordinary lessons, in a university laboratory. fixation time on the target stimulus. Subjects had ninety
The setting was a 4x5 meters room, with a table in the seconds to complete each of the three tests, consisting in
center; on that table the eye-tracker was placed and in front finding the hidden yellow duck.
of it, at a 50 centimeters distance, sat the subject. In the
room there were normal light conditions. The experimenter Results
remained always in the room, ready to answer in case of The visual scanning data were analyzed by comparing the
questions. Each subject completed consent forms prior to time before correct fixation (DBF), the number of fixations
participation in the study on target stimulus (NFT), the duration of fixation time on
Memory load was then determined for each subject using the target stimulus (DTT), and the duration of fixation time
the test of digit span of the Wechsler scale, in a silent on the non-target stimuli (DTNT). After verifying normal
classroom. Two different tasks were then administered. In data distribution, analysis of variance was applied. A
this first task, subjects were instructed to observe a target significance of .05 was considered.
stimulus (fig. 1: “target”), a little yellow duck. Then, the With reference to the first parameter, the time before
subjects were asked to spot, as quickly as possible, the little correct fixation (DBF), there is a significant effect on
yellow duck that was hidden in a complex image. At the cognitive load, F (2, 50) = 13.43, p <.01. This result
same time, participants were asked to listen to and repeat a suggests that the higher the cognitive load, the longer the
list of numbers read aloud by an experimenter. Memory time spent to find the target for the first time. More
load was in fact manipulated by increasing or decreasing the specifically, post hoc analysis shows statistical differences
memory set, thanks to three different conditions: between the absence of cognitive load and full load, t (25) =
−visual scanning without any cognitive interference due 23.56, p< .001 (Tab. 1).
to other simultaneous tasks (no load = 0 digits); With reference to the second parameter, the number of
−visual scanning with a contemporaneous second task fixations on target stimulus (NFT), there is a significant
that involved half of the cognitive subject’s load (half load = effect on cognitive load, F (2, 50) = 25.9, p <.001. This
(span-1)/2 digits); result indicates that the higher the cognitive load, the lower
−visual scanning with a contemporaneous second task the number of fixations on target stimulus. Post hoc
that involved the entire subject’s cognitive load (full load = comparisons show a significant effect between no load and
span -1 digits). half load, t (25) = 7.288, p< .001 (Tab. 2), between no load
Every complex image was presented for 90 seconds. The and full load, t (25) = 18.46, p< .001 (Tab. 3), and between
total length of the second working session was about 10 half load and full load conditions, t (25) = 11.179, p< .001
minutes. The slide’s sequence was the following: (Tab. 1).
1) Verbal instructions: “Look at this little yellow duck The same trend, with reverse results, can be observed on
and try to memorize it”; the parameter DTT, duration of fixation time on the target
2) Presentation of the target stimulus; stimulus, in which there is an effect on cognitive load, F (2,
3) Verbal instructions: “Try to spot, as quickly as 50) = 4.32, p <.002. This result indicates that the higher the
possible, the little yellow duck that you have seen before, cognitive load, the longer the duration of fixation. Post hoc
and stare at it until the change of the picture”; comparisons show a significant effect between no load and
4) Presentation of a complex image in which the target full load conditions, at (25) = 14.038, p< .001 (Tab. 1).
stimulus was hidden (no cognitive load); Finally, with reference to the parameter that indicates the
5) Verbal instructions: “Try to spot, as quickly as entropy level, DTNT = TT – DTT, there is a significant
possible, the little yellow duck that you have seen before, effect on cognitive load, F (2, 50) = 25.91, p <.001. This
and stare at it until the change of the picture; at the same result suggests that the higher the cognitive load, the longer
time, repeat after me the list of the numbers I’m reading to the total time spent in looking at non target stimuli. Post hoc
you”; comparisons show a significant effect between no load and
6) Presentation of a complex image in which the target half load, t (25) = 7.28, p< .001, between no load and full
stimulus was hidden (half cognitive load); load, t (25) = 18.46, p< .001, and between half load and full
load, t (25) = 11.179, p< .001 (table 1).
201
1) Verbal instructions: “Look at this little square and try
to memorize it”;
2) Presentation of the target stimulus for five seconds;
Table 1 Means and standard deviations (S.D) of the four 3) Verbal instructions: “Try to find, as quickly as
parameters of eye tracking related to the cognitive load possible, the little square that you have seen before, and
Conditions DBT NFT DTT DTNT = stare at it until the change of the slide”;
TT-DTT 4) Presentation of the target stimulus with 5 distracting
stimuli;
No load 10,666 31,902 39,962 28,098 5) Presentation of the target stimulus with 10 distracting
(2,9) (3,3) (5,3) (3,3) stimuli;
Half load 11,106 24,614 34,231 35,386 6) Presentation of the target stimulus with 15 distracting
(2,5) (2,9) (4,6) (2,9) stimuli;
7) Presentation of the target stimulus with 20 distracting
Full load 34,231 13,435 25,923 46,565 stimuli;
(4,6) (2,5) (4,6) (2,5) 8) Presentation of the target stimulus with 25 distracting
stimuli.
Results
Study 2 The visual scanning data were analyzed by comparing the
The aim of this second study is to analyze whether energy time before correct fixation (DBF), the number of fixations
dispersal of eye movements is related not only to top-down on target stimulus (NFT), the duration of fixation time on
factors such as cognitive load, but also to objective factors the target stimulus (DTT), and the duration of fixation time
such as the number of distracting stimuli. on the non-target stimuli (DTNT). After verifying normal
data distribution, analysis of variance was applied. A
Method significance of .05 was considered.
With reference to the first parameter, the time before
Participants correct fixation (DBF), there is a significant effect on the
The same 26 voluntary students in the first study were number of distracting stimuli, F (4, 100) = 11.49, p <.01.
engaged. This result suggests that the higher the number of stimuli
(from 5 to 25), the longer the time spent to find the target
Task specifications for the first time. With reference to the second parameter,
The picture used for the eye tracker’s second task the number of fixations on target stimulus (NFT), there is a
consisted of an increasing series of little squares (0,5 x 0,5 significant effect on the number of distracting stimuli, F
cm) with a little segment starting from one corner, (4,100) = 6.214, p <.014. This result suggests that the higher
respectively the left and right corners at the top and the left the number of stimuli, the lower the number of fixations on
and right corners at the bottom of the squares. The target target stimulus. The same trend, with reverse results, can be
stimulus was the square with the segment situated in the shown on the parameter DTT, duration of fixation time on
right corner at the top (fig. 2: “target”). All the other the target stimulus, in which there is an effect on the number
combinations of squares were considered distracting stimuli of distracting stimuli, F (4,100) = 6.62, p <.01. Those results
(fig.: 2: “target and 5, 10, 15, 20, 25 distracting stimuli”). indicate that the higher the number of stimuli, the lower the
Stimuli were presented in five different combinations, in duration of fixation. Finally, with reference to the parameter
particular with 5, 10, 15, 20, 25 distracting stimuli. Subjects DTNT, the fixation time on the non target stimulus, there is
had ninety seconds to complete the test, consisting in a significant effect on the number of distracting stimuli,
finding the correct target stimulus.
F (4,100) = 6.63, p < .01. This result suggests that the higher
the number of stimuli, the longer the fixation time on the
Procedure
non target stimuli. With reference to DTNT, post-hoc
In this second task, subjects were asked to find, as quickly
comparisons show a significant effect between 5 and 20
as possible, the stimulus target within a series of distracting
distracting stimuli, t (25) = 4.617, p< .001, and between 5
stimuli. In the first test the subjects were instructed to
and 25 distracting stimuli, t (25) = 4.348, p< .001.
observe a target stimulus (fig. 2: “target”), a square with a
segment starting from the upper right corner, and upper-
Table 2 Means and standard deviations (S.D.) of the four
slanting. Then they had to find the correct target they had
parameters of eye tracking related to the amount of
seen before, in an image in which the target appeared with
distracting stimuli.
an increasing series of distracting stimuli (respectively 5,
10, 15, 20, 25 distracting stimuli). Every slide was presented
Number of DBT NFT DTT DTNT=
for 90 seconds. The total length of the first working session distracting TT-DTT
was about 10 minutes. The slide’s sequence was the stimuli
following:
202
5 1,352 2,667 4,346 11,665 technology can help us to visualize the eye movement and
(.17) (.25) (.58) (3.25) the factors that can maximize information through the
clustering process that reduce entropy.
10 1,689 2,100 3,385 29,00
(.27) (.24) (.48) (4.24)
15 2,478 1,992 4,000 45,12 References
(.25) (.25) (.68) (3.25) Berlyne, (1974). Studies in the new experimental aesthetics:
steps toward an objective psychology of aesthetic
20 3,160 1,370 1,923 72,6 appreciation. Washington, DC: Hemisphere Publishing
(.28) (.24) (.41) (6.24)
Corporation.
25 3,231 1,270 2,115 87,25 Camilli, M., Nacchia, R., Terenzi, M., & Di Nocera, F.
(.35) (.28) (.52) (3,73) (2008). ASTEF: A simple tool for examining fixations.
Behavior Research Methods, 40 (2), 373-382
Deubel, H., & Schneider, W.X. (1996). Saccade target
selection and object recognition: Evidence for a common
Discussion attentional mechanism. Vision Research, 36, 1827-1837.
The results of both studies indicate that energy dispersal of Hamilton, P. (1979). Process Entropy and Cognitive
eye movement depends on both top-down and bottom-up Control: Mental Load in Internalised Thought Processes.
factors. In this case, both the amount of cognitive load and Mental Workload. Stirling, Scotland.
the number of distracting stimuli influence energy dispersal. Hoffman, J. E. (1998). Visual attention and eye movements.
The parameters here examined are not independent ones. In H. Pashler (ed.), Attention, Hove, UK: Psychology
They are related and DTNT derive from basic parameters: Press.
the lower the time before correct fixation (DBF), the higher Hilburn, B., Jorna, P. G., Byrne, E. A., & Parasuraman, R.
the number of fixations on target stimulus (NFT), the higher (1997). The effect of adaptive air traffic control (ATC)
the duration of fixation time on the target stimulus (DTT), decision aiding on controller mental workload. In M.
and the lower the entropy (DTNT). Mouloua & J. M. Koonce (Eds.), Human–automation
Cognitive load not only affects complex cognitive factors, interaction: Research and practice. Mahwah, NJ:
but the analysis of the four parameters suggests that it is Erlbaum
relevant also in visual scanning. Particularly, time “lost” or Hirsh, J. B., Mar, R. A., & Peterson, J. B. (2012).
dispersed, id est time spent on search on not target stimuli, Psychological entropy: A framework for understanding
is high both on the time before correct fixation (DBF) and uncertainty-related anxiety. Psychological Review, 119,
on the entropy level. The present results are in line with 304-320.
evidence from Kruizinga, Mulder, & de Waard (2006), Kruizinga, A., Mulder, B., & de Waard, D. (2006). Eye scan
underlying that higher entropy could be associated with patterns in a simulated ambulance dispatcher’s task. In D.
higher mental workload, and with evidence from Camilli, de Waard, K. A. Brookhuis, & A. Toffetti (Eds.),
Nacchia, Terenzi & Di Nocera (2008) that highlighted that Developments in Human Factors in Transportation,
when the mental workload is high eye fixation is dispersed, Design, and Evaluation, Maastricht: Shaker.
when the mental workload is low eye fixation is clustered. Maccone, M. (2014) Evolution and History in a new
In the area of research on distraction, this question is very “Mathematical SETI” model. Acta Astronautica, 93, 317-
important. From a bottom-up perspective, considerable 344.
evidence suggests that distracting stimuli can interfere by Miller, G. A., (1956). The Magical Number Seven, Plus or
having common qualities with the target, such as color Minus Two: Some Limits on Our Capacity for Processing
(Stroop, 1935) or orientation (Joseph & Optican, 1996). Information, The Psychological Review, 63, 81-97.
Others have looked at adjacent distracters (proximity) and Just, M. A., & Carpenter, P. A. (1980). A theory of reading:
their influence (Flowers & Wilcox, 1982). from eye fixation to comprehension. Psychological
To understand what causes distracters to be detrimental to Review, 87, 329–354.
task performance, eye tracker technology can be useful. Posner, M. I. (1980). Orienting of attention. Quarterly
Further research with eye tracker can give us information on Journal of Experimental Psychology, 32, 3-25.
entropy styles and on how the energy dispersal may impact Yarbus, A. L. (1967). Eye Movements and Vision. Plenum.
the distraction and the related learning deficits. New York.
In a task requiring more effortful attention, such as a
visual search task, distracters that are processed
automatically may provide more interference and enhance
entropy. Another factor is that visual search tasks require
more effort for attention control when distracters are
randomized and unpredictable (Michael, Kiefer, &
Niedeggen, 2012; Neo & Chua, 2006). Eye tracker
203
Fig. 1 Picture used for the eye tracker’s first study and target
Fig. 2 Target stimulus and distracting stimuli in the five
levels of complexity
204