Towards a Framework for Visualization and Analysis of Eye Tracking Data for Functional Vision Screening Qasim Ali 0 Carsten G. Helgesen 0 Ilona Heldal 0 Western Norway University of Applied Sciences , Inndalsveien 28, 5063, Bergen , Norway

Eye-tracking technologies ofer a valuable means of collecting data on gaze movements, which can be utilized to assess functional vision problems. However, analyzing gaze data and presenting trustworthy interpretations is a challenge today. The analysis depends on the eye-tracking technologies, available software for vision screening, gaze metrics, and stimuli data. This paper presents a novel framework for visualizing and analyzing data obtained from diferent eye trackers and vision screening software ordered in a pipeline. This framework is flexible and can be scaled up and extended with gaze metrics and data-cleaning mechanisms to facilitate vision experts and researchers in further optimizations. It illustrates through a description of the pipeline the necessary steps toward producing comprehensive results on gaze metrics and a range of visualizations, such as heatmaps, scan paths, and fixation points, through a graphical interface, making data analysis and interpretation more eficient and user-friendly.

 eol>Eye-tracking data analysis framework functional vision screening 1. Introduction

In recent years, there has been a growing interest among researchers to utilize eye-tracking technology using computer programs to assess oculomotor behaviour with the help of serious games and reading tasks [ 1 ]. Eye-tracking (ET) technologies provide helpful information and enable researchers to study eye movements in psychology, neurology, cognition, engineering, and education [ 2 ]. A bibliometric analysis showed significant research conducted on diverse facets of eye movements, encompassing visual attention, saccades, reading and visual search [ 3 ].

An eye tracker provides raw data on gaze that needs to be analyzed to obtain meaningful metrics on eye movements, such as fixations, saccades, smooth pursuits, blinks, pupil size, and latency [ 4 ]. While numerous algorithms have been developed and optimized [ 5 ] to improve the accuracy and precision of eye movement detection, interpreting the results obtained from these algorithms can still be challenging, especially when high levels of precision and accuracy are required to draw reliable conclusions. Measuring oculomotor problems relies on the chain of such eye movements, which are dependent on computerized vision screening tasks. Some studies [ 6, 7 ] have illustrated the analysis of ET data for functional vision screening, but they are limited to a specific ET, analyzing the tasks under study, a specific screen size, and a fixed threshold for calculating fixations and saccades. In addition, there is also no specific pipeline for the data analysis of ET for functional vision screening.

The overall aim of this paper is to propose a concept of a novel framework for the evaluation of functional vision screening by the implementation of a data analysis pipeline that consists of some existing algorithms for computing gaze metrics and visualization techniques on unprocessed ET data. The input of the system is data from any ET, stimuli data and dimensions of the screen where the data is collected, and the output of the system would be the gaze metrics, analysis of gaze data with respect to the movements of stimuli, and visualizations such as heatmaps, fixations points, and scan path. Even though the optimal threshold values of ET algorithms can vary among clinicians and vision tests, a framework with a graphical interface visualizing this will enable adjusting the parameters accordingly. Such a framework can regenerate the simulation (replay) of the animated vision screening task, which can give a better understanding of the gaze response to the stimuli. Thus, the proposed framework provides a pipeline of ET data analysis, visualization and integration with any eye-tracker data.

 2. Background 2.1. Functional vision screening and gaze metrics

In recent years, a growing body of research has demonstrated the potential of ET technologies for functional vision screening. The quantification of ET data for functional vision screening has recently been used to evaluate the efectiveness of this technology with vision experts [ 6, 7 ]. Recent research studies have highlighted the potential benefits of ET technologies utilized to make such measurements for the diagnosis of, e.g., strabismus and nystagmus [ 8, 9 ]. By analyzing eye movements and measuring deviations from normal eye behaviour patterns, ET technology can provide objective and quantitative measures of how vision functions. In this paper, we utilize a computer software called C&Look [ 10 ] that uses an eye tracker to complement vision experts in providing essential evidence for OMD (oculomotor dificulties, a problem with coordination of left and right eyes), a functional vision assessment (FVA) [ 11 ].

An eye tracker provides raw data on gaze points, and algorithms must be applied to handle and visualize the results in order to extract important characteristics or gaze metrics from the data. For example, fixations refer to specific points in continuous eye movement data that indicate where the eye is focused on a visual display. Saccades are the fast eye movements between fixation points [ 12 ]. Fixations are obtained by taking discrete samples of the eye’s position [ 13 ]. Salvucci et al. [ 14 ] have proposed a taxonomy to classify fixation identification algorithms based on how they use spatial and temporal information in ET protocols. The optimal threshold for the minimum fixation duration varies depending on the nature of the task. For example, it is measured at 225 milliseconds for reading, 275 milliseconds for visual search, and 400 milliseconds for hand-eye coordination [ 15 ]. Among the other gaze metrics, latency, velocity, and amplitude are particularly relevant in functional vision screening.

2.2. Eye-tracking data analysis studies

PyTrack [ 16 ] is an eye-tracking data analysis toolkit that can extract gaze metrics and provide visualizations of the results. However, it does not currently support dynamic stimuli or areas of interest (AOIs). Duchowski proposed a data analysis pipeline [ 17 ] that involves averaging left and right gaze values. However, averaging positions of gaze points which are apart may result in a misleading representation as the result does not reflect any of the two gaze points. Therefore, a more comprehensive analysis should be conducted on each left and right gaze point to ensure accurate measurements. Kumar et al. [ 18 ] has developed a system for detecting OMD in stroke patients using a low-frequency eye tracker, 30 Hz. This system relies on gaze metrics, including fixation, smooth pursuit, and blinking, in response to both static and dynamic visual stimuli. Jayawardena [ 19 ] has put forward a proposed architecture called the Real-Time Advanced Eye Movements Analysis Pipeline (RAEMAP), which aims to utilize ET measures as a valid method for psychophysiological evaluation. The goal is to demonstrate the efectiveness of RAEMAP in diagnosing ADHD in real time.

3. Eye-tracking data analysis framework

We are developing a framework consisting of four phases to quantify and interpret ET data to measure OMD (see figure 1). The proposed framework is compatible with recorded eye-tracking data for vision screening purposes.

3.1. Data source and data transformation

The framework requires data from multiple sources. Firstly, ET data will be used to calculate gaze metrics such as fixations, saccades, smooth pursuit, and pupil size. Secondly, stimulus data will be used to map ET data into areas of interest (AOIs) within the stimuli. In vision screening tasks, the AOI is either the stimuli itself or some specific region inside the stimuli. Lastly, vision screening software can be used on diferent screen sizes, such as laptops and monitors of varying sizes. Therefore, information regarding screen size is crucial to converting the data into a suitable format and common units.

Eye trackers provide a stream of data (a time series) with a frequency depending on the Hertz (Hz) rate of the eye tracker. The second phase of the framework will handle the data transformation of the eye tracker, stimuli data and screen dimensions. Vision screening software can be developed in diferent environments and programming packages. Therefore, the coordinate systems represent the data in diferent shapes and formats, e.g., normalized coordinates, pixel coordinates, word space and camera space. Diferent applications, such as Unity, Windows Form, or Java, might use diferent coordinate systems to represent stimuli. The collected data must be transformed into a common coordinate system, and time series data must be transformed to be on coordinated time points [ 20 ].

3.2. Pre-processing and analyzer

The data stream can contain artefacts that need to be cleaned before running the algorithms to detect gaze metrics and visualizations. The common problems are invalid, or missing data of gaze points. Therefore, the data from the eye tracker and stimuli will be cleaned in the third phase of the framework.

In the analyzer phase, our framework will employ several established quantitative analysis techniques. These techniques will involve the extraction of gaze metrics, conducting statistical and descriptive analyses, and visualizing the results [ 21 ]. The process will be structured following a hierarchical approach, as illustrated in figure 2. The statistical and descriptive analysis will compute the (1) mapping of gaze points with the stimuli position and compare them, (2) a percentage of gaze points following the path of stimuli and (3) calculate valid fixation points, only those fixation points where the fixation is exactly on the stimuli. The framework will be able to produce a detailed report that includes expected and calculated values of stimuli position and gaze position as comprehensive visualizations for the vision screening task. Overall, this approach will facilitate a thorough and rigorous examination of the data collected during the vision screening task.

4. Implementation of toolkit 5. Limitations and future directions

The aim of this paper was to show the overall concept of a framework. Currently, the framework has implemented the fixations detection algorithm and visualization of gaze data. However, the framework is scalable and flexible to integrate more modules. This involves implementing the algorithm for transforming screen coordinates in Phase 2 of the framework on the original data from both eye trackers and stimuli, as well as coordinating time series data. For Phase 4 of the framework, remaining gaze metrics such as latency, pupil size, statistical analysis and reporting mechanisms must also be developed and integrated.

Although there is no currently accepted benchmark for assessing functional vision problems using ET technology and computerized software, our research aims to determine the expected values of stimuli position and the calculated position of the gaze point over a specified period of time along gaze metrics. This data can contribute to defining a benchmark necessary for vision experts in documenting and measuring objective eye behaviour metrics.

 6. Discussion

The study highlights the proposed framework and current state of work to compute gaze metrics and visualizations such as heatmaps, scan paths, and fixation points in presenting the gaze data. The graphical interface of the framework makes the pipeline of data analysis and interpretation of ET data more eficient and user-friendly for vision experts and researchers.

Previous research has investigated the application of ET data quantification for assessing OMD and measuring functional vision [ 18 ]. Clinicians or vision experts can efectively identify potential abnormalities by analyzing the overall gaze metrics, statistical graphs, and recording.

Some of the existing ET data analysis pipelines have incorporated essential algorithms and a sequential set of procedures that facilitate the analysis of raw data and extraction of key features [ 23, 24 ]. This framework provides a package that consists of a data analysis pipeline and a graphical interface for the navigation and interaction with the framework. The data analysis pipeline is a sequential phase that will be executed to generate quantitative results for i.e, fixtions and visualize i.e, scanpath, heatmap and fixations points. Currently, the data analysis pipeline has implemented the I-DT algorithm [ 25 ] to extract fixations and saccades gaze metrics and pupil size has already been calculated in the stream of ET data provided by most of the eye trackers, for example, Tobii eye trackers provide pupil size in real-time. However, further development is required to implement the analysis of additional gaze metrics and perform statistical data analysis.

7. Conclusion

ET technologies provide a valuable means of collecting data from gaze movements, which can be utilized to assess functional vision problems. In this article, we presented the concept and current implementation of the framework which combine the data analysis pipeline for the ET data. It provides a detailed overview of all the steps of the proposed framework and shows the data visualization in an interactive graphical interface implemented in Python. The components of the framework are flexible and can be scaled up and extended with further gaze metrics mechanisms to facilitate vision experts and researchers in further optimizations. Thus, the next step involves implementing statistical data analysis within the framework to enhance its capabilities further

 8. Acknowledgments

Thanks to the Seced project (number-267524) for collecting the data we examined for using C&Look.

 [1] Q. Ali , I. Heldal, C. G. Helgesen , G. Krumina, C. Costescu , A. Kovari , J. Katona , S. Thill , Current challenges supporting school-aged children with vision problems: A rapid review , Applied Sciences 11 ( 2021 ). URL: https://www.mdpi.com/2076-3417/11/20/9673. doi: 10 . 3390/app11209673. [2] A. T. Duchowski , A breadth-first survey of eye-tracking applications , Behavior Research Methods, Instruments, & Computers 34 ( 2002 ) 455 - 470 . doi: 10 .3758/bf03195475. [3] Q. Ali , I. Heldal, C. G. Helgesen , A bibliometric analysis and visualization of the use of eyetracking technologies for vision screening , in: 2021 International Conference on e-Health and Bioengineering (EHB) , 2021 , pp. 1 - 4 . doi: 10 .1109/EHB52898. 2021 . 9657547 . [4] B. Mahanama , Y. Jayawardana , S. Rengarajan , G. Jayawardena , L. Chukoskie , J. Snider , S. Jayarathna , Eye movement and pupil measures: A review , Frontiers in Computer Science 3 ( 2022 ). URL: https://doi.org/10.3389/fcomp. 2021 . 733531 . doi: 10 .3389/fcomp. 2021 . 733531 . [5] S. Akshay , Y. J. Megha , C. B. Shetty , Machine learning algorithm to identify eye movement metrics using raw eye tracking data , in: 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT) , 2020 , pp. 949 - 955 . doi: 10 .1109/ICSSIT48917. 2020 . 9214290 . [6] P. Blignaut , E. J. van Rensburg , M. Oberholzer , Visualization and quantification of eye tracking data for the evaluation of oculomotor function , Heliyon 5 ( 2019 ) e01127 . URL: https://europepmc.org/articles/PMC6348242. doi: 10 .1016/j.heliyon. 2019 .e01127. [7] P. Blignaut , E. J. van Rensburg , M. Oberholzer , Visualization and quantification of eye tracking data for the evaluation of oculomotor function , Heliyon 5 ( 2019 ) e01127 . URL: https://doi.org/10.1016/j.heliyon. 2019 .e01127. doi: 10 .1016/j.heliyon. 2019 .e01127. [8] D. Giordano , C. Pino , C. Spampinato , M. Di Pietro , A. Reibaldi , Eye tracker based method for quantitative analysis of pathological nystagmus , in: 2011 24th International Symposium on Computer-Based Medical Systems (CBMS) , 2011 , pp. 1 - 6 . doi: 10 .1109/CBMS. 2011 . 5999041 . [9] U. Saisara , P. Boonbrahm , A. Chaiwiriya , Strabismus screening by eye tracker and games , in: 2017 14th International Joint Conference on Computer Science and Software Engineering (JCSSE) , 2017 , pp. 1 - 5 . doi: 10 .1109/JCSSE. 2017 . 8025956 . [10] M. G. Eide , I. Heldal, C. G. Helgesen , G. Birkeland Wilhelmsen , R. Watanabe , A. Geitung , H. Soleim , C. Costescu , Eye-tracking complementing manual vision screening for detecting oculomotor dysfunction , in: 2019 E-Health and Bioengineering Conference (EHB) , 2019 , pp. 1 - 5 . doi: 10 .1109/EHB47216. 2019 . 8969956 . [11] Q. Ali , I. Heldal, M. G. Eide , C. G. Helgesen , G. B. Wilhelmsen , Using eye-tracking technologies in vision teachers' work - a norwegian perspective , in: 2020 International Conference on e-Health and Bioengineering (EHB) , 2020 , pp. 1 - 5 . doi: 10 .1109/EHB50910. 2020 . 9280169 . [12] S. Stuart , B. Galna , S. Lord , L. Rochester , A. Godfrey , Quantifying saccades while walking: Validity of a novel velocity-based algorithm for mobile eye tracking , in: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society , IEEE, 2014 . URL: https://doi.org/10.1109/embc. 2014 . 6944931 . doi: 10 .1109/embc. 2014 . 6944931 . [13] Z. Bylinskii , M. A. Borkin , N. W. Kim , H. Pfister , A. Oliva , Eye fixation metrics for large scale evaluation and comparison of information visualizations , in: M. Burch , L. Chuang , B. Fisher, A. Schmidt , D. Weiskopf (Eds.), Eye Tracking and Visualization , Springer International Publishing, Cham, 2017 , pp. 235 - 255 . [14] D. D. Salvucci , J. H. Goldberg , Identifying fixations and saccades in eye-tracking protocols , in: Eye Tracking Research & Application, 2000 . [15] P. Blignaut , Fixation identification: The optimum threshold for a dispersion algorithm , Attention, Perception, & Psychophysics 71 ( 2009 ) 881 - 895 . URL: https://doi.org/10.3758/ app.71.4.881. doi: 10 .3758/app.71.4.881. [16] U. Ghose , A. A. Srinivasan , W. P. Boyce , H. Xu , E. S. Chng , PyTrack: An end-to-end analysis toolkit for eye tracking , Behavior Research Methods 52 ( 2020 ) 2588 - 2603 . URL: https://doi.org/10.3758/s13428-020-01392-6. doi: 10 .3758/s13428-020-01392-6. [17] A. T. Duchowski , Eye Tracking Methodology, Springer International Publishing, 2017 . URL: https://doi.org/10.1007/978-3- 319 -57883-5. doi: 10 .1007/978-3- 319 -57883-5. [18] D. Kumar , A. Dutta , A. Das , U. Lahiri, SmartEye: Developing a novel eye tracking system for quantitative assessment of oculomotor abnormalities , IEEE Transactions on Neural Systems and Rehabilitation Engineering 24 ( 2016 ) 1051 - 1059 . URL: https://doi.org/10.1109/ tnsre. 2016 . 2518222 . doi: 10 .1109/tnsre. 2016 . 2518222 . [19] G. Jayawardena , RAEMAP: Real-time advanced eye movements analysis pipeline , in: ACM Symposium on Eye Tracking Research and Applications , ACM, 2020 . doi: 10 .1145/ 3379157.3391992. [20] S. Berbuir , Towards analyzing C & Look Data, HVL internal report , Discussion paper, HVL, Bergen, Norway, 2022 . [21] Q. Ali , Analysis of eye tracking data: Supporting vision screening with eye tracking technologies , in: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications , ETRA '23, Association for Computing Machinery, New York, NY, USA, 2023 . doi: 10 .1145/3588015.3589536. [22] G. Drusch , J. Bastien , S. Paris, Analysing eye-tracking data: From scanpaths and heatmaps to the dynamic visualisation of areas of interest, Advances in science, technology, higher education and society in the conceptual age : STHESCA 20 ( 2014 ) 25 . [23] N. A. Gehrer , M. Schönenberg , A. T. Duchowski , K. Krejtz , Implementing innovative gaze analytic methods in clinical psychology: A study on eye movements in antisocial violent ofenders , in: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications , ETRA '18, Association for Computing Machinery, New York, NY, USA, 2018 . doi: 10 .1145/3204493.3204543. [24] A. T. Duchowski , S. Jörg , T. N. Allen , I. Giannopoulos, K. Krejtz , Eye movement synthesis, in: Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications , ETRA '16, Association for Computing Machinery, New York, NY, USA, 2016 , p. 147 - 154 . doi: 10 .1145/2857491.2857528. [25] D. D. Salvucci , J. H. Goldberg , Identifying fixations and saccades in eye-tracking protocols , in: Proceedings of the 2000 Symposium on Eye Tracking Research & Applications , ETRA '00, Association for Computing Machinery, New York, NY, USA, 2000 , p. 71 - 78 . doi: 10 . 1145/355017.355028.