ET4S 2014
Location Dependent Fixation Analysis with Sight Vectors.
Locomotion as a Challenge in Mobile Eye Tracking
Daniel Müller-Feldmeth1*, Sarah Schwarzkopf1*, Simon J. Büchner2, Christoph
Hölscher4, Gregor Kallert3, Rul von Stülpnagel1, and Lars Konieczny1
1
Center
for
Cognitive
Science,
Freiburg,
Germany
{daniel, sarah.schwarzkopf, rul.von.stuelpnagel,
lars}@cognition.uni-freiburg.de
2
University
College
Freiburg,
Germany
buechner@ucf.uni-freiburg.de
3
Fraport
AG,
Frankfurt,
Germany
g.kallert@fraport.de
4
ETH
Zurich,
Switzerland
choelsch@ethz.ch
*equal
contribution,
alphabetical
order
Abstract.
Mobile
eye
tracking
has
become
a
fruitful
method
for
spatial
research.
Body
movement
and
orientation
as
well
as
the
complexity
of
real-‐world
surround-‐
ings
have
a
strong
influence
on
the
processing
of
environmental
information
that
can
be
captured
by
mobile
eye
tracking
devices.
On
a
methodological
level,
perceiv-‐
er
locomotion
is
both
a
challenge
due
to
the
complexity
of
the
data,
as
well
as
a
val-‐
uable
resource
of
information.
In
this
paper
we
propose
a
new
approach
to
inte-‐
grate
observer
location
information
and
fixation
data
using
sight
vectors.
This
meth-‐
od
is
a
crucial
step
towards
furthering
the
analysis
of
mobile
eye
tracking
data
and
the
understanding
of
the
perception
of
moving
observers
in
complex
environments.
Keywords:
Mobile
Eye
Tracking,
Situated
Cognition,
Locomotion,
Sight
Vectors,
Visuospatial
Perspective.
1
Mobile
Eye
Tracking
during
Locomotion
In the last years mobile eye tracking has become a popular method of veering away
from studies in the laboratory and investigating eye movements in real environments.
This move is important insofar as it is yet unclear to what extent the results obtained
from eye tracking studies in the lab bear external validity so that conclusions drawn
from them can be transferred to the real world [1, 2]. However, using mobile eye
tracking poses new challenges for data analysis, in particular in spatial tasks where
persons can move freely through a complex environment. Both the complexity of the
environment as well as the movement of the observer are, on the one hand, aspects
that can hardly be accounted for in lab studies, but, on the other hand, make it neces-
ET4S 2014, September 23, 2014, Vienna, Austria
Copyright © 2014 for the individual papers by the papers' authors. Copying permitted for private and
academic purposes. This volume is published and copyrighted by its editors.
67
ET4S 2014
sary to develop new methods that allow an integrated analysis of locomotion and
fixation patterns [3].
Findings from lab studies with spatial tasks in environmental space are often
limited in that simple stimuli with little visual clutter are used whereas humans navi-
gating space in the real world must deal with complex perceptual input. Rich static
images e.g. [4] or videos [2] of the environment are more closely related to real-world
visuospatial complexity, but still lack the aspect of free locomotion. While walking,
the body’s movement and its orientation continuously change the visuospatial per-
spective on a scene. Observations from a comparison of eye tracking data during a
navigation task in a lab and a field study indicate that body orientation and locomo-
tion have a strong impact on the perception of signs that cannot be captured in lab
studies [1]. In addition, gaze patterns differ significantly between walking a route and
watching a video of the same route from the walker’s perspective [2]. Thus, gaze
behavior during locomotion in real-world settings must inevitably be investigated to
understand how people use visual information while navigating, as well as to evaluate
if and how lab-based experimental settings can be used as a valid alternative.
The rapid advancement of technology over the past decade has brought about
mobile eye tracking devices that are light and efficient enough to provide high resolu-
tion fixation data of perceivers moving in real environments while only minimally
interfering with the users’ perception and task. However, when different participants
move freely, they perceive diverse objects and environmental features from varying
visuospatial perspectives
[1],
[5]
which
will
in
turn
influence
their
decisions
and
trajectories. To tackle this challenge, new methods for data analysis are required,
particularly for spatial research where locomotion is an inherent part of the task: Body
movement and eye movements have to be integrated.
To date, only a few studies have used mobile eye tracking in connection with
locomotion (e.g. [1], [6]) or even combined mobile eye tracking with location track-
ing ([3], [7]). Mobile eye tracking data is typically analyzed by mapping fixations on
reference images that resemble the viewpoint of the participant. By defining areas of
interest, this method allows the proportion of attention participants pay to particular
objects or scenes to be quantified, and has proven valuable in a wide area of research.
However, to investigate gaze behavior during locomotion, this method bears severe
limitations. Analysis is typically restricted to a limited set of locations or decision
points on the trajectory of a person navigating through the environment ([3], [7]). A
step further is to analyze gaze allocation for different objects or object categories over
time ([2], [7]). However, these methods still do not take into account information
about a perceiver’s actual position in and movement through the environment in a
way that allows exploration of the complex interaction of bodily movement and gaze
behavior during locomotion. We will outline a new method that attempts to integrate
both participants’ location and their fixations within the same coordinate system. This
method allows us to construct sight vectors that can be used to visualize and analyze
the integrated dynamics of locomotion and gaze behavior during navigation.
68
ET4S 2014
2
Location
Dependent
Fixation
Analysis
To analyze the location dependent fixation data we developed sight vectors as a
method to integrate locomotion data with gaze data. We tested this method in a field
study. 29 participants (13 female, 16 male) aged 21-57 years (M=33.2, SD=11.9)
performed a wayfinding task at Frankfurt Airport ("Find Gate A5!"). The area we
tested was a staircase in which participants left one flight of stairs/escalator, then had
to turn around 180 degrees and continued descending down a second flight of
stairs/escalator (see Figure 1). The choice whether to take the stairs (n=12) or the
escalator (n=17) was made by the participants. The target location is indicated by
three signs. While signs 1 and 3 direct passengers to the stairs/escalator, sign 2 directs
them to the elevator. This scenario involved a large number of body movement oppor-
tunities within a small area in a short testing time (10.5 – 26.7 seconds).
During the task, we measured participants’ gaze behavior using mobile eye
tracking glasses (SMI). To analyze the recorded data, in a first step we manually cod-
ed fixations on a floor plan of the environment. As we were mainly interested in the
perception of signs and exit points and their interaction with navigation, only fixations
on these objects of interest were considered in the analysis. In a second step, we cod-
ed the participants’ locations in the room for every coded fixation. These two steps
provided us with two x/y coordinates per time stamp that could be used to compute a
sight vector indicating not only the destination of a gaze, but also its origin. These
sight vectors can be utilized to analyze the attention dynamics in moving perceivers
and to identify the viewpoints from which a sign catches attention and can be inter-
preted.
Sign 2
escalator ●●●●●●
●●●
●●
●●● ●●
●
●
stairs
●
●
●
●● ●●● ●●
●●● ●●
● ●● ● ● ● ●
● ● ●
● ●
●● ● ●● ● ●
● ●
●●
●
●
●● ● ●
Sign 1●
●● ● ● ●● ●
Sign 3
Fig. 1. Locomotion trajectories and sight vectors of two participants, one coming from the esca-
lator (blue), one from the stairs (red), but both continuing onto the escalator.
Using the same coordinate system for both fixations and locations, sight vec-
tor patterns indicate how the navigated space is being scanned during locomotion.
Figure 1 shows trajectories and sight vectors for two participants, one coming from
the escalator; the other coming down the stairs. The sight vector pattern (a) illustrates
the influence of the different visuospatial perspectives of the two trajectories on fixa-
tion patterns, and (b) enables an inspection of the sequence of fixations dependent on
69
ET4S 2014
the changing location. Furthermore, fixation times for particular objects can be calcu-
lated using sight vectors. Figure 2 shows the sight vectors on the three signs as well as
a representation of the location from where a sign is being fixated and for how long:
Colored regions represent the fixation time spent on the corresponding sign at a par-
ticular location. Figure 2a (left hand side) shows the fixation times of the participants
using the escalator; Figure 2b (right hand side) represents participants using the stairs.
It is clearly visible that the order in which the signs are fixated differs be-
tween the two groups. When coming down the escalator, sign 1 is visible first; short-
ly after that sign 3 becomes accessible providing the information to find the correct
Escalator Stairs
0
0
● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●
●● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
● ● ● ●
●
●
●
● ● ● ●
200
200
●
● ● ●
●
●
●
● ●
Sign 2
●
Sign 2
● ●
● ●
●
●
●
●
●
● ●
●
●
● ●
●
400
400
●
● ●●● ●● ●●● ● ●
●● ●●● ●●●●
●●
●●●
●● ●●
●● ●
●●
●●●
●●●●
●●
●●●● ●●● ●●
●● ● ● ● ● ●● ●
●●●●● ● ●
● ●
●
●●●●●●
●● ●● ●●●●●●●● ●●●●●● ●●●●●●●●●●● ●● ● ● ● ● ● ● ● ●● ●
● ● ●●
●●●● ● ●●
●●
●●●●●
●●
●
●●
●●
●●●●
●●
●
●●●
●●
● ●●●●●
●
●●
●●
●● ●●●●●● ●●●
● ●●●●●● ●●●● ● ●●● ● ●●●●● ● ● ●●
●●●
●● ●
●●● ● ●● ● ● ● ● ●
●●●●●●●
● ● ● ● ●● ● ●
● ●
● ● ● ● ●
● ● ● ● ● ●● ● ● ● ●
● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ●
● ● ● ● ●
● ● ●● ● ●● ●
●
● ● ●
● ●● ● ●
●
● ●● ● ●
● ● ●● ● ●
● ●
● ● ●
●● ●
● ● ●
● ● ●●● ●● ●●●● ●● ●● ● ●● ● ●
●● ● ● ● ●
●● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ●●●●●● ●●●●
● ●● ●●● ● ● ● ● ●● ●●●●●●●●● ● ●●●● ● ● ●● ● ● ● ● ● ●● ● ●
●● ● ● ● ●
●● ●
●
● ● ● ● ● ●● ● ●●
● ●
●●● ●● ● ● ● ● ●● ● ● ●●● ●
●
● ●
●
●●●● ● ●●● ●● ● ● ● ●●
Sign 1 Sign 1
● ● ●●●●●● ● ● ●● ●● ●●● ●● ● ● ●
● ●● ● ●● ●● ● ●
●
●●
● ●●● ●
● ●● ● ● ● ●● ●● ● ●
● ●● ● ● ● ● ● ●
● ● ● ●
● ● ● ●● ●● ● ●
●
● ● ● ●●●● ●● ● ●
● ●● ●
●
●
●
● ● ● ● ● ●
●● ●
●● ● ● ● ●●
● ● ● ●
● ● ●● ● ● ● ●
● ●●● ● ●
● ●● ●●
● ● ● ● ●● ● ●● ●●● ●●● ●●●●●● ● ● ●
● ●●● ● ● ●
● ● ● ●● ● ● ● ●● ● ●●● ●●● ●●
● ●● ● ● ● ● ●● ● ● ● ●
● ● ●● ● ● ● ●●
● ● ●●●● ●●● ●● ●●
● ●●●●● ● ● ● ●
● ● ● ● ● ● ● ● ●
● ● ●● ● ●
●● ● ● ● ● ● ● ●● ●● ●
●● ● ● ● ● ● ●●
● ● ●● ● ●● ●
● ●● ● ● ●
●● ● ● ● ● ● ● ●
● ● ● ● ●
● ● ● ● ●
●
● ● ●
● ●●
600
●
600
●
● ●
●
● ● ●
● ● ●
● ● ● ● ● ● ● ●● ● ● ●●● ● ● ●
● ● ●● ●
● ●
● ● ● ● ● ● ●
●
Sign 3 Sign 3
●
●
● ●
●
●
● ●
800
800
Fig. 2. Sight vectors and fixation time on three signs. Saturation of the colored regions represents
the fixation time spent on 500
0 the corresponding sign
0 at each
1000 location. Left hand
500side1500
(2a): Partici- 1000
pants using the escalator. Right hand side (2b): Participants using the stairs.
way. The acute angle to sign 2 makes it difficult to read and therefore it is only fixat-
ed by the few participants near it. In contrast, participants using the stairs first fixate
both sign 1 and sign 2 (which compete with each other) before sign 3, resulting in a
higher number of detours and irritation for some participants.
3
Outlook
We presented a method to integrate locomotion data with gaze behavior data. We
showed that it can be useful for qualitative approaches, but it is especially important
as the first step for a quantitative analysis of mobile eye tracking data during locomo-
tion. In our study, we were able to identify areas from where particular objects are
most likely to be fixated. In our future work, we will extend this approach by integrat-
ing head and body orientation in the analyses.
So far, coding has been done manually, using the video data provided by the eye
tracking system. Both location and fixation data can be annotated within the same
environment, which ensures easy synchronization and integration of both sources of
information. Using the scene camera images to determine the location as well as ori-
entation of a participant also has the advantage that no external tracking device is
necessary, and is thus also feasible in environments or tasks where such tracking de-
vices (e.g., based on GPS) are not available or do not provide the required resolution.
However, an interesting continuation to make the method more easily applicable will
70
ET4S 2014
be to (a) explore alternative ways to track body and head movements automatically
and (b) to employ object recognition algorithms to be able to at least partly automa-
tize fixation mapping [5],[8].
Acknowledgements. This work was funded by the DFG project SFB TR/8 Spatial
Cognition. Our special thanks go to Carina Hoppenz, Saskia Leymann and Jana
Wendler for their committed help with the study conduction and with semantic gaze
mapping as well as to Stephanie Nicole Schwenke for article proofreading. We also
thank the FRIAS and especially Peter Auer for the provision of the two eye tracking
systems. Last but not least, our thanks go to many Fraport employees for their volun-
tary study participation.
References
1. Schwarzkopf, S., von Stülpnagel, R., Büchner, S. J., Konieczny, L., Kallert, G.,
and Hölscher, C. (2013). What Lab Eye Tracking Tells us About Wayfinding. A
Comparison of Stationary and Mobile Eye Tracking in a Large Building Scenar-
io. In Kiefer, P., Giannopoulos, I., Raubal, M., and Hegarty, M., editors, Eye
Tracking for Spatial Research, Proceedings of the 1st International Workshop
(in conjunction with COSIT 2013), 2013.
2. Foulsham, T., Walker, E., and Kingstone, A. (2011). The Where, What and
When of Gaze Allocation in the Lab and the Natural Environment. Vision Re-
search, 51(17):1920–1931.
3. Kiefer, P., Straub, F., and Raubal, M. (2012). Towards Location-aware Mobile
Eye Tracking. In Spencer, S. N., editor, Proceedings of the Symposium on Eye
Tracking Research and Applications, ETRA ’12, pages 313–316, New York, NY,
USA. ACM.
4. Wiener, J.M., Hölscher, C., Büchner, S., & Konieczny, L. (2012). Gaze Behav-
iour During Space Perception and Spatial Decision Making. Psychological Re-
search, 76(6), 713-29.
5. Brône, G., Oben, B., and Goedemé, T. (2011). Towards a More Effective Meth-
od for Analyzing Mobile Eye-Tracking Data: Integrating Gaze Data With Object
Recognition Algorithms. In Proceedings of the 1st International Workshop on
Pervasive Eye Tracking; Mobile Eye-based Interaction, PETMEI ’11, pages 53–
56, New York, NY, USA. ACM.
6. Patla, A. and Vickers, J. (2003). How far Ahead do we Look When Required to
Step on Specific Locations in the Travel Path During Locomotion? Experimental
Brain Research, 148(1):133–138.
7. Cave, A. R., Blackler, A. L., Popovic, V., and Kraal, B. J. (2014). Examining
Intuitive Navigation in Airports. In Proceedings of Design Research Society
Conference 2014, Umea, Sweden.
8. Abbott, W.W., Thomik, A.A.C., & Faisal, A.A. (2013). Embodied Attention for
Gaze Analysis in Daily Life Activities. In Proceedings of the First International
Workshop on Solutions for Automatic Gaze-Data Analysis 2013 (SAGA 2013).
71