Oculomotor disturbance (OMD) is a common vision problem, meaning that the left and right eye do not cooperate properly, i.e., by having a common gaze point. Eye tracking technology (ET) promises support for identifying problematic eye coordination. Data from the eye tracker is based on time series of screen positions and the stimuli movements, which are recorded, following a structural pattern. A vision specialist can analyze and interpret the graphical plots of the time series visually to get a better understanding of the problems related to gaze movements. However, this is tedious and time consuming due to the huge amount of data collected by ETs, or the necessity for replaying the tests. This paper explores a method to automatically analyzing the results of a screening to indicate potential OMD problems by applying Symbolic Aggregate Approximation (SAX) and pinpointing relevant features for OMD. The potential benefits of the method are investigated via examples considering the distance between left and right gaze points. This indicates promising results for faster examining large data sets and discussing possibilities for future extensions for considering eye movement parameters based on real-time measurements of the distance between the stimuli and the gaze points.
Eye tracking technology (ET) gives the ability to explore and measure gaze points and eye
movements for diferent purposes, e.g. attention management, marketing research, or
investigating vision disturbances. ET data are time series of positions, usually on a computer screen,
visualized using diferent methods like scan paths, heat maps, fixation points etc. [
SAX has been used for many purposes, e.g. analyzing ET data for games [
A time series is a sequence of data points or observations collected and recorded over regular time
intervals, denoted = {} where each is a single number or a composite structure. Time
series data mining aims to perform clustering, classification, and anomaly detection, to mention
a few. Several recent papers suggest new methods to explore time series, Karamitopoulos and
Evangelidis [
SAX is a method for reducing the dimensionality of a time series by applying Piecewise Aggregate Approximation (PAA) and assigning symbols to the calculated aggregates [15], as proposed by Yi and Faloutsos [16]. Let = {} be a time series of real numbers with length , the PAA algorithm calculates the mean value of all values in a window of length called “frames”, which reduces C into a time series ′ = {′} of length ⌈/⌉. Thus ′ = 1 ∑︁ =(− 1)+1
These means are the final data-reduced representation of PAA and used in the discretization process described below. If n is not divisible by w, some points will be dificult to place. One solution consists of splitting the afected points into parts and placing them in each frame [ 15], although this is a minor detail for time series having much larger than .
SAX transforms a time series into a discrete string of symbols, based on the PAA [15]. The symbols assignment process uses the statistical distribution of the data set and divides it into a frames (intervals) of equal probability, and assigns a symbol to each frame, as seen in Figure 1.
Each frame having equal probability ensures that the distribution of the aggregated data will be similar to the distribution of the original data. If the data are standardized and normally distributed, the frames can be given by break points derived by the formula for the normal distribution with mean 0 and standard deviation 1. If the real distribution is unknown, and the data volume is suficiently large, reasonable break points can also be calculated from a cumulative histogram of the data set approximating the distribution, dividing the frequency (“the probability”) axis into equal parts, and reading of the corresponding break points on the value axis. If is an odd number, the middle interval will be assigned to the middle symbol, while if is even the mean value will be the break point between the two intervals on each side of the mean. This is illustrated in Figure 2.
The data set is from the SecEd project, screening school children in Tanzania. The software used is C&Look, using the Tobii C4 eye tracker with 60 Hz recording frequency. All screening data are anonymized. The master thesis this paper is based on investigates many time series, but for this paper we only present two representative sets of time series, due to space limitation. The distance between the left and right gaze point are measured using the Euclidean distance between the screen positions given by the eye tracker for each time stamp. The normalization of the distance values to the Z-score is done using the mean and standard deviation calculated across all the time series from the set of screenings available. Thus any bias will be potential measurement errors from the eye tracker.
Calculating the SAX transformation relies on two parameters: the frame size and the alphabet
size . Below, we discuss how each of them afects the ability to characterize ET time series and
discover various features, in particular anomalies, as shown in Figures 3 and 4 [
The frame size is the factor reducing the dimensionality of the time series of length to ⌈/⌉. There is a trade-of between dimensionality reduction and the ability to discover small features in the time series. For vision screening must be chosen in cooperation with a vision expert to reflect the time length of the relevant features that should be discovered. Figure 3 shows the result of transforming gaze data using three diferent frame sizes: = 5, = 15 and = 30. With an eye tracker frequency of 60 Hz, each frame covers data from 1/12, ¼, and ½ second, respectively. From the top, Figure 3 shows the x direction of movements of the stimulus across the screen, i.e. a saccadic movement where the object jumps 10 steps across the screen, then does the same in a second line, and so on. The second graph shows the gaze point in the x direction for left and right eye, respectively. The third graph shows the distance between the two gaze points, while the fourth one shows the same data normalized to Z-score. The three lower plots show the result after performing PAA with three diferent values of . It shows that the areas where the gaze points difer most are clearly shown on each of the lower graphs having values above 0, i.e. distance being larger than expected across the entire distribution. The shortest features are lost with = 15 and = 30.
The alphabet size decides into how many parts the distribution of the data are split. This also afects how small features can be detected, but this time on the gaze distance value scale. Again, there is a trade-of between detail and readability since a low value of is easy to interpret but may hide the small details in varying gaze distances, while a larger value of will pick up more detail, but can be harder interpret. For vision screening, must be chosen in cooperation with a vision expert to reflect the smallest distance of interest for relevant features. Figure 4 shows the result of transforming gaze data using three diferent frame sizes: = 5, = 5, and = 8. From the top, the figure shows the x-direction of smooth pursuit across the screen. As in figure 3, the second and third graphs show the gaze point in the x direction and the gaze distance for the left and right eye, respectively, while the fourth shows the distance data normalized to Z-score. The three lower plots show the result after performing SAX with the two diferent values of . It can be seen that the areas where the gaze points difer most are clearly identifiable on each of the lower graphs having values above 0, i.e. distance being larger than expected across the entire distribution.
By transforming the time series into a symbol sequence with SAX, interesting patterns in the
symbol sequence can occur. Symbols representing “normality”, i.e. having a value close to
the mean value of the overall distribution, depends on the value of , and also on the shape
of the distribution. When the data are normally distributed, or more generally symmetrically
distributed, and is an odd number, the “middle” symbol covers the mean of the distribution,
otherwise the two middle symbols are on each side of the mean. Assuming = 5, with symbols
{a, b, c, d, e}[
As a first step Borgli [ 17] studied using SAX for the distance between left and right gaze point. The method can be extended to study the distance between the stimulus and the gaze points, indicating how well the gaze is able to follow the stimulus. This is most easily done if the stimulus moves in a deterministic manner, e.g. with software like C&Look. If the stimulus is non-deterministic it is more dificult to compare the stimulus movement and the gaze points.
The SAX paper [15] also defines a distance measure between symbol sequences, which can be used to compare two screenings with sequence alignment methods from bioinformatics, or even by using machine learning. Although we have only studied a small data set, we recon that the method scales well, as the computations only applies to limited number of time series per screening, and the break points are pre-calculated.
This paper has studied a limited data set, due to limited space, but illustrates that SAX is a viable method to categorize time series for vision screening (research question 1). Borgli’s master thesis [17] has many more examples, and his findings are fully in line with this paper. Applying SAX show promising results for detecting anomalies in screening data and characterize a screening result as probably being normal or showing a potential OMD problem. As Figures 3 and 4 illustrate, the symbol patterns and the PAA both pinpoint relevant patterns in the ET data indicating potential anomalies (research question 2). However, the thresholds and patterns indicating a suspected OMD problem must be decided in close cooperation with vision experts.
We want to thank the Seced project (number-267524) for being able to use data collected by using the C&Look software, and examinations from vision experts. We are grateful to Henrik Borgli for his interest and work with his MSc project supervised by the other coauthors. [15] K. Lin, E. Keogh, L. Wei, S. Lonardi, Experiencing SAX: a novel symbolic representation of time series, Data Mining and knowledge discovery 15 (2007) 507–144. [16] B.-K. Yi, C. Faloutsos, Fast time sequence indexing for arbitrary Lp norms, in: Proc. of the 26th International Conference on Very Large Data Bases, 2000, p. 385–394. [17] H. Borgli, Analyzing time series from eye tracking using Symbolic Aggregate Approximation, Master’s thesis, University of Bergen and Western Norway University of Applied Sciences, 2022. MSc Thesis in Software Engineering,.