Mobile eye tracking has become a fruitful method for spatial research. Body movement and orientation as well as the complexity of real-‐world surround-ings have a strong influence on the processing of environmental information that can be captured by mobile eye tracking devices. On a methodological level, perceiv-er locomotion is both a challenge due to the complexity of the data, as well as a val-uable resource of information. In this paper we propose a new approach to inte-grate observer location information and fixation data using sight vectors. This meth-od is a crucial step towards furthering the analysis of mobile eye tracking data and the understanding of the perception of moving observers in complex environments. In the last years mobile eye tracking has become a popular method of veering away from studies in the laboratory and investigating eye movements in real environments. This move is important insofar as it is yet unclear to what extent the results obtained from eye tracking studies in the lab bear external validity so that conclusions drawn from them can be transferred to the real world [1, 2]. However, using mobile eye tracking poses new challenges for data analysis, in particular in spatial tasks where persons can move freely through a complex environment. Both the complexity of the environment as well as the movement of the observer are, on the one hand, aspects that can hardly be accounted for in lab studies, but, on the other hand, make it neces-
sary to develop new methods that allow an integrated analysis of locomotion and fixation patterns [3].
Findings from lab studies with spatial tasks in environmental space are often limited in that simple stimuli with little visual clutter are used whereas humans navigating space in the real world must deal with complex perceptual input. Rich static images e.g. [4] or videos [2] of the environment are more closely related to real-world visuospatial complexity, but still lack the aspect of free locomotion. While walking, the body’s movement and its orientation continuously change the visuospatial perspective on a scene. Observations from a comparison of eye tracking data during a navigation task in a lab and a field study indicate that body orientation and locomotion have a strong impact on the perception of signs that cannot be captured in lab studies [1]. In addition, gaze patterns differ significantly between walking a route and watching a video of the same route from the walker’s perspective [2]. Thus, gaze behavior during locomotion in real-world settings must inevitably be investigated to understand how people use visual information while navigating, as well as to evaluate if and how lab-based experimental settings can be used as a valid alternative.
The rapid advancement of technology over the past decade has brought about mobile eye tracking devices that are light and efficient enough to provide high resolution fixation data of perceivers moving in real environments while only minimally interfering with the users’ perception and task. However, when different participants move freely, they perceive diverse objects and environmental features from varying visuospatial perspectives [1], [5] which will in turn influence their decisions and trajectories. To tackle this challenge, new methods for data analysis are required, particularly for spatial research where locomotion is an inherent part of the task: Body movement and eye movements have to be integrated.
To date, only a few studies have used mobile eye tracking in connection with locomotion (e.g. [1], [6]) or even combined mobile eye tracking with location tracking ([3], [7]). Mobile eye tracking data is typically analyzed by mapping fixations on reference images that resemble the viewpoint of the participant. By defining areas of interest, this method allows the proportion of attention participants pay to particular objects or scenes to be quantified, and has proven valuable in a wide area of research. However, to investigate gaze behavior during locomotion, this method bears severe limitations. Analysis is typically restricted to a limited set of locations or decision points on the trajectory of a person navigating through the environment ([3], [7]). A step further is to analyze gaze allocation for different objects or object categories over time ([2], [7]). However, these methods still do not take into account information about a perceiver’s actual position in and movement through the environment in a way that allows exploration of the complex interaction of bodily movement and gaze behavior during locomotion. We will outline a new method that attempts to integrate both participants’ location and their fixations within the same coordinate system. This method allows us to construct sight vectors that can be used to visualize and analyze the integrated dynamics of locomotion and gaze behavior during navigation.
To analyze the location dependent fixation data we developed sight vectors as a method to integrate locomotion data with gaze data. We tested this method in a field study. 29 participants (13 female, 16 male) aged 21-57 years (M=33.2, SD=11.9) performed a wayfinding task at Frankfurt Airport ("Find Gate A5!"). The area we tested was a staircase in which participants left one flight of stairs/escalator, then had to turn around 180 degrees and continued descending down a second flight of stairs/escalator (see Figure 1). The choice whether to take the stairs (n=12) or the escalator (n=17) was made by the participants. The target location is indicated by three signs. While signs 1 and 3 direct passengers to the stairs/escalator, sign 2 directs them to the elevator. This scenario involved a large number of body movement opportunities within a small area in a short testing time (10.5 – 26.7 seconds).
During the task, we measured participants’ gaze behavior using mobile eye tracking glasses (SMI). To analyze the recorded data, in a first step we manually coded fixations on a floor plan of the environment. As we were mainly interested in the perception of signs and exit points and their interaction with navigation, only fixations on these objects of interest were considered in the analysis. In a second step, we coded the participants’ locations in the room for every coded fixation. These two steps provided us with two x/y coordinates per time stamp that could be used to compute a sight vector indicating not only the destination of a gaze, but also its origin. These sight vectors can be utilized to analyze the attention dynamics in moving perceivers and to identify the viewpoints from which a sign catches attention and can be interpreted.
Sign 2 escalator stairs ●●●●●●● ●● ● ● ● ●●●● ●●●●● ● ●●●●●● ●●● ● ● ●
S●ign 1 ●● ● ●● ●●●●●●● ● ● ●● ● ● ●● ●
Sign 3
Using the same coordinate system for both fixations and locations, sight vector patterns indicate how the navigated space is being scanned during locomotion. Figure 1 shows trajectories and sight vectors for two participants, one coming from the escalator; the other coming down the stairs. The sight vector pattern (a) illustrates the influence of the different visuospatial perspectives of the two trajectories on fixation patterns, and (b) enables an inspection of the sequence of fixations dependent on 0 0 0 2 0 0 4 0 0 6 0 0 8 the changing location. Furthermore, fixation times for particular objects can be calculated using sight vectors. Figure 2 shows the sight vectors on the three signs as well as a representation of the location from where a sign is being fixated and for how long: Colored regions represent the fixation time spent on the corresponding sign at a particular location. Figure 2a (left hand side) shows the fixation times of the participants using the escalator; Figure 2b (right hand side) represents participants using the stairs.
It is clearly visible that the order in which the signs are fixated differs between the two groups. When coming down the escalator, sign 1 is visible first; shortly after that sign 3 becomes accessible providing the information to find the correct Escalator 0 Stairs ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●● ●●●●●●●●●●●●●●●●●● ● S●● i● g●●n● S●1●●i● ● g●●●●●●●●●●●● n 204002000
Sign 3 6 0 0 8 ●● ● ● ● ● ● ●● ● ●●● ●● ● ● ●● ●● ●● ● ●● ● ● ● ● ● ●● ●● ●
● S●i●g●n 2
● ● ● ● ● ● ● ● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●● ●●● ●● ● ●● S●●●●●●i● g●●●n ●1 ● ●● ●●● ● S●●● ● ●●i●g●●●● n●●●●●●●● ●●●3●●●●●●● ● ● ● ●● ● ● ● ● ●●● ● ● ● ● ● ● ● ● way. The acute angle to sign 2 makes it difficult to read and therefore it is only fixated by the few participants near it. In contrast, participants using the stairs first fixate both sign 1 and sign 2 (which compete with each other) before sign 3, resulting in a higher number of detours and irritation for some participants. 3 Outlook
We presented a method to integrate locomotion data with gaze behavior data. We showed that it can be useful for qualitative approaches, but it is especially important as the first step for a quantitative analysis of mobile eye tracking data during locomotion. In our study, we were able to identify areas from where particular objects are most likely to be fixated. In our future work, we will extend this approach by integrating head and body orientation in the analyses.
So far, coding has been done manually, using the video data provided by the eye tracking system. Both location and fixation data can be annotated within the same environment, which ensures easy synchronization and integration of both sources of information. Using the scene camera images to determine the location as well as orientation of a participant also has the advantage that no external tracking device is necessary, and is thus also feasible in environments or tasks where such tracking devices (e.g., based on GPS) are not available or do not provide the required resolution. However, an interesting continuation to make the method more easily applicable will be to (a) explore alternative ways to track body and head movements automatically and (b) to employ object recognition algorithms to be able to at least partly automatize fixation mapping [5],[8].
Acknowledgements. This work was funded by the DFG project SFB TR/8 Spatial Cognition. Our special thanks go to Carina Hoppenz, Saskia Leymann and Jana Wendler for their committed help with the study conduction and with semantic gaze mapping as well as to Stephanie Nicole Schwenke for article proofreading. We also thank the FRIAS and especially Peter Auer for the provision of the two eye tracking systems. Last but not least, our thanks go to many Fraport employees for their voluntary study participation.