Eye-tracking is the measurement of eye movements and determination of the gaze, which is a location where the person is looking. Eye movement has been of interest to scientists since the 19th century. One of the first important milestones was the discovery by Louis Émile Javal that reading is not a smooth movement over a text, but a series of short stops (fixations) and quick jumps (saccades). Since the 1970s, research on gaze tracking in reading has gained popularity, especially the Rayner study [1]. An important idea is the hypothesis put forward in 1980 by Just and Carpenter about the strong eye-mind, according to which what the gaze is fixed on is processed [2]. There are also other hypotheses, in particular the immediacy hypothesis, according to which the eyes do not move until all processing is completed. Moreover, there are opposing ideas that eye movements do not really reflect moment-by-moment cognitive processing demands during reading. In any case, scientists agree that gaze tracking can provide important data about reading.
Initially eye movement was studied to determine its role in reading and language comprehension. Later psycholinguists began to use this technology to organize the process of language learning for different social groups, to improve cross-cultural communication in business, as well as for many theoretical questions about understanding how people perceive language. Eye-tracking is not only used in reading text, but also in listening to music, typing, and visual search. There are many applications of this technology [3].
The technology is widely used in education, both in the form of student eye tracking (tasks to determine student engagement, academic performance, and skills assessment) and in the form of teacher eye tracking (professional eye tracking). In addition, there is an idea to explore the methodology of performing tasks with eye tracking with the possibility of turning them into educational material. Eye movement has also helped in the training of some professions, such as pilots, by developing methods for modeling complex realized events, detecting errors, and assessing consequences.
Marketing, human-computer interaction, and neuroergonomics are also well-known applications of eye tracking for market research, audience research, as an input device, and for adaptation in the workplace, respectively [4]. There are ideas for using tracking data to improve NLP models, to evaluate them, and to explore the possibility of applying the models to other languages and problems [5][6][7][8]. There are also many other applications of gaze tracking systems, such as research on perception and cognitive processes, lie detection [9][10], and detection of deepfakes [11][12].
Another interesting use of the system is to diagnose a person's mental state, which is especially relevant in times of full-scale war. An example of the realization of such an idea is the Ukrainian startup Anima [13]. Indeed, many Ukrainians, especially the military, suffer from constant stress that puts a strain on their psychological state. It is important to constantly monitor and prevent possible deterioration of psychological health in order to understand the readiness of the military for combat. Gaze tracking systems can help with this, as a shifted distribution of attention to stimuli of different emotional colors may indicate certain deviations. This idea can be expanded to diagnose various other mental illnesses, as there are already publications describing ways to diagnose them by studying a person's gaze.
Calibration is necessary to adapt the system algorithm before applying it to the person sitting in front of the eye-tracker. Some systems do not need calibration, but most still require it. Calibration is possible when the point the person is looking at is known. Mobile device calibrations vary, and their instructions are in the manuals. For a screen-based calibration, you need to focus your gaze on the targets shown on the screen. The more targets in different parts of the screen are analyzed, the more accurate the system will be, but this can be time consuming and inconvenient for some experiments.
In our experiment, we want to collect data that can be useful to different professionals interested in the application of eye tracking in their field, and to evaluate the quality and accuracy of the resulting system.
Since in our experiment we will not use special high-precision equipment that may not require calibration, and the format will be screen-based, we will use points from different parts of the screen for calibration, namely 3 rows of 3 points, with one point appearing in the center of the screen and the rest in the perimeter of the screen with 4 points in the corners.
Eye-tracking reading experiments often use fixations (place, duration) and saccades (length, duration, start and end point) as captured primitives. For more detailed experiments, the number of fixations per 100 words, whether they are progressive or regressive, and the frequency of regressions are observed.
There are eye-tracking datasets with 2 different types of input data: the original video with the specified characteristics of the experiment and already processed format, where with the original parameters of the eyes the resultant point on the screen is indicated [14].
Two methods are used for text experiments: rapid serial visual presentation (RSVP), in which the words are presented at a set rate in the same location, and self-paced reading, in which only several words are presented on the screen at a time and readers advance the text by pushing a button. There are variations of the self-paced method. Some experiments include the ability to look back in the text. Some display only one word, while others display several at once. Sometimes the words are in the same place, and sometimes they are presented spatially, as in a normal text. There are also studies that consider a text as a time-series process [15] or based on an attention mechanism [16].
As for textual datasets, they often consist of a single sentence and are monolingual, although there are also longer texts and multilingualism available, such as GECO [17] and MECO [18] datasets [19]. It is also important to note that most of the existing datasets are in English. The Ukrainian language is considered to be a low resource language with very few existing datasets [20]. Therefore, the text in Ukrainian will be used in our experiment.
Hardware differs by specialty, human interface, scope of tracking, and technical characteristics.
Special trackers refer to any additional gaze-detection hardware, when non-special is a standard webcam. The interface can be head-stabilized, remote, and mobile (also called "head-mounted"). In terms of tracking area, devices are differentiated into those that use a computer screen as the stimulus area; those that can operate in more complex geometries, such as a multi-screen booth; and those that can operate in the real world. Technical characteristics include accuracy, sampling rate, resolution, etc.
We divided trackers by human interface, additionally adding non-special devices (webcams). We get the following groups: head-stabilized, remote, mobile and non-special.
Head-stabilized eye tracking is usually much more accurate than other types of trackers, so they are used in neurophysiological experiments, where participants' comfort is less important than system accuracy and precision, as well as in experiments with animals. However, such trackers are uncomfortable and not designed for experiments where natural responses with head movements are important.
Remote systems consist of a camera and an infrared source. Most often such cameras are mounted under the screen, as the pupil is more visible from that position due to the shape of the eye and eyelids. Remote eye tracking is useful in experiments with infants and in natural interactions. Limitations for such trackers are a fixed working area beyond which gaze tracking is difficult, the tolerance of head movements affecting the inaccuracy of results, sunlight reflected in participants' eyes, and the problem of capturing more than one participant.
Mobile gaze tracking is often in the form of goggles that include a scene recording camera, gaze capture cameras, and illuminators. Such devices are a great option for real-world research. There is the problem of sunlight, but it is solvable. In addition there is a problem of the difficulty of tracking the gaze at the periphery and the use of a relative coordinate system that differs greatly from participant to participant.
Non-special equipment does not have infrared beams or a head stabilizer, so its effectiveness is orders of magnitude lower. However, a significant advantage is the popularity of web cameras.
As a result of comparing different hardware, it was concluded that greater accuracy is given by special devices, especially with a head lock. But since webcams are much more common and do not require additional costs, we would like to investigate camera accuracy. The camera used in the experiment has a frame rate of 30 FPS and a resolution of 1280x720.
These days, there are various programs that allow eye-tracking, both with special eye trackers and with general webcams [21]. Our focus was on those of them that can be applied to general webcams. A comparison of different eye-tracking software is shown in Table 1. Although some software can detect, for example, saccades, it can only be achieved using special equipment, because the frequency of the web-cameras is not sufficient for this.
Having compared and worked with the above-mentioned programs, it was decided to use WebGazer.js for the experiment. Although this software works relatively roughly and cannot capture saccades and fixations, due to its well-developed community and available documentation it is excellent for conducting small experiments.
In preparation for an upcoming experiment, the task at hand involves the careful and meticulous preparation of data. To accomplish this, we have developed a specialized service designed to generate custom images with specific parameters. These parameters include the picture resolution, aimed at optimizing the display quality on the user's screen, as well as the desired text, font style, font size, spacing between words, spacing between sentences, distance to the screen border, and the desired position of the text on the screen.
By leveraging our innovative service, a seamless process is established for creating a high-quality PNG image that precisely adheres to the specified parameters. Additionally, a corresponding JSON file is generated, which contains metadata that comprehensively describes the various parameters of the image. This includes vital information such as the exact placement coordinates for each word within the image.
By providing this comprehensive and meticulously detailed output, our service ensures that the necessary data is readily available for the future experiment, enabling a smooth and accurate analysis of the acquired information.
The final generated image contains the text itself and additional borders. The word boundaries calculated by the program according to the text metadata are displayed in green.
Figure 1 shows the generated final image, using the data taken from the metadata.
The primary objective of this paper is not to conduct a comprehensive study on eye movement during text-related tasks. Instead, its main focus is to introduce a system that can potentially facilitate such experiments. The paper includes an experiment that serves two purposes: firstly, to showcase the viability of employing an eye-tracking system, and secondly, to examine the feasibility of substituting costly eye trackers with a simpler webcam solution.
The goal of the experiment is to determine the extent to which webcams can be useful using WebGazer.js to determine the position in the text, where the participant focuses. For this purpose, Ukrainian text was taken, which was projected on a 14'' FullHD screen with different font sizes and line spacing. The result of the experiment was to select the optimal ratio of text parameters.
At the stage of preparation for the experiment 16 variants of text with Calibri font and font size from 24 pt to 32 pt with the step of 4 pt, and with the line spacing from 1.5 to 3 with the step of 0.5 were formed.
The experiment involved people with different eye colors without special features (e.g., glasses). Each participant had to wait for the camera to capture their face and then underwent calibration at 9 points after starting the app. During the calibration, each point had to be clicked 5 times while looking at it. After a successful calibration, the system's accuracy was calculated when looking at the center point and displayed on the screen. The system-determined point of view was displayed as a blue dot. The text parameters could be changed from the top menu bar. During the experiment, attention was paid to how accurately the place where the gaze was directed could be determined. For this purpose, hits in a word were examined.
Percentages of hits in the word was calculated using the following equation
where 𝑁 ! is the fraction of hits, i.e. the number of hits on the container that contains the word, and 𝑁 " is the number of the number of times the container was missed calculated for 5 seconds of gaze. The hit or miss accuracy is evaluated at regular intervals of 10 milliseconds.
To activate the accuracy calculation function, we used the right mouse click at the place where the gaze is actually directed.
Heat maps in eye tracking play a crucial role in visualizing and analyzing gaze focus data. By utilizing color intensity, heat maps provide a graphical representation that indicates the level of eye attention in specific areas of an image or screen. This enables researchers and practitioners to gain insights into the patterns of visual exploration and focus during various tasks.
Real-time display of eye movements is particularly valuable in tasks that require immediate feedback or interaction. The inclusion of real-time heat maps allows for the visualization of the current gaze location on the screen, providing instant visual feedback to both the user and the experimenter.
One of the advantages of heat maps is their ability to accumulate information over time. By highlighting areas on the screen where the gaze was directed for longer durations with stronger and more noticeable colors, heat maps effectively capture the salient regions of interest. This feature aids in reducing noise or transient eye movements, as the emphasis is placed on areas that received sustained attention.
Heat maps can be employed in various domains, including usability testing, user experience research, website optimization, and advertising analysis. They provide a valuable tool for understanding visual attention patterns, optimizing user interfaces, and enhancing the overall user experience.
Figure 2 shows the heat map display. The image clearly demonstrates the informative nature of heat maps, as they vividly depict the visual information regarding the focus of attention.
During the experiment, the program collects important data about the user, in particular the coordinates of his gaze, namely the x and y coordinates and the corresponding timestamps. This data is then organized and stored in a structured format in a CSV (Comma-Separated Values) file. The collected data is then transmitted to a server for further processing and analysis.
Figure 3 shows the application during the accuracy assessment at a particular location. Despite the expected linearity of the data, we got a different result. We can explain it by the inaccuracies of the program, lack of experiments and by the internal processes of the brain, which lead to uncontrolled eye movements.
According to the results of the experiment, a word can be caught with a webcam with a font size not less than 32 pt and a line spacing not less than 2.5 with an accuracy more than 84%, but to determine the letter that the participant is looking at, the webcam is not enough.
The results of the experiment showed that a webcam can be used to recognize a line that reads text. However, compared to the results that can be obtained using special equipment, webcams do not work accurately. The plan is to experiment with more accurate special trackers in order to separate the letter at which the gaze is directed with greater accuracy.
The experiment yielded a discovery regarding the feasibility of utilizing webcams for eye-tracking in the context of reading texts on general devices. This finding highlights a crucial advancement in the field, as it demonstrates the successful adaptation of commonly available webcams for capturing and analyzing eye movement data. The ability to utilize webcams for eye-tracking opens up new possibilities for researchers and practitioners, as it eliminates the need for specialized and expensive eye-tracking equipment. This not only expands the accessibility of eye-tracking technology but also presents opportunities for conducting eye-tracking studies on a larger scale and in diverse settings.
The implications of this finding extend beyond the scope of the experiment and have the potential to shape future research methodologies and applications in the field of eye-tracking.
This publication is based upon work from COST Action CA21131, supported by COST (European Cooperation in Science and Technology).