EToS-1: Eye Tracking on Shopfloors for User Engagement with Automation

Abstract

Mixed Reality (MR) is becoming an integral part of many context-aware industrial applications. In maintenance and remote support operations, the individual steps of computer-supported (cooperative) work can be defined and presented to human operators through MR headsets. Tracking of eye movements can provide valuable insights into a user's decision-making and interaction processes. Thus, our overarching goal is to better understand the visual inspection behavior of machine operators on shopfloors and to ifnd ways to provide them with attention-aware and context-aware assistance through MR headsets that increasingly come with eye tracking (ET) as a default feature. Toward this goal, in two industrial scenarios, we used two mobile eye tracking devices and systematically compared the visual inspection behavior of novice and expert operators. In this paper we present our preliminary findings and lessons learned.

Keywords

• eye tracking
• mixed reality
• industrial operations
• CSCW
• automation
• user engagement

1. Introduction

Bektaş); https://interactions.ics.unisg.ch (J. Strecker); CEUR achieved simultaneously [ 8 ]. Moreover, the research gap at the intersection of user experience, automation, and work indicates that more efort is necessary to understand how well human operators interact with systems and engage with their work in automated environments [ 9 ]. In a recent review [ 3 ], researchers presented emerging technologies and concepts (e.g., mixed reality headsets, gaze-contingent displays, and digital companions) that can capture the contextual and cognitive state of users thus potentially improving the level of engagement in their social or work environments. The definition of technology engagement may change in diferent contexts, but it is generally linked with the focus of attention [ 9 ]; in this context, mixed reality experiences can facilitate reaching high levels of engagement in diferent activities [ 3, 10 ].

Eye-tracking (ET) has become afordable and useful for studying human decision-making processes and visual inspection behavior [ 11 ]. Electrooculography [ 12 ] or video-based ET [ 13 ] can be used for the non-intrusive assessment of humans’ cognitive state (e.g., their level of engagement, attention, cognitive load, or fatigue). On various displays, ET can also guide an instant (gaze-contingent) adjustment of the information load [ 14, 15, 16 ] or a context-specific adaptation of the information content [ 17, 18 ]. Therefore, the use of Mixed Reality (MR) headsets that enable ET out of the box or with additional sensors is becoming more widespread [ 3, 19 ]. The seamless integration of ET with MR headsets would allow us to improve computer support for human users [ 20, 21 ], by means of gaze-contingent and context-aware assistance [ 3, 22 ].

Head-mounted eye trackers or mobile ET glasses can be used to provide their users with assistance in everyday activities such as reading [ 23 ] or navigation [ 24 ] where users’ can freely move their head and body. Whereas, in more controlled environments such as laboratory experiments with desktop-mounted or remote eye trackers, users often are required to sit and preserve their head position [ 16 ] or use a chin-rest [ 25 ]. In mobile settings, the accuracy of the ET data might therefore be susceptible to dynamic movements of the user (i.e., the glasses may slip) or changing illumination conditions [ 26 ]. In dynamic scenes, the assessment of gaze recordings on moving objects is hard to automate and depends on manual input [ 27 ]. In such recordings, the size and location of the areas of interest must be adjusted in multiple frames, which can be laborious based on factors such as the length of the recording or the number of objects.

Recently, Ehinger et al. compared a high-end remote eye tracker with a mobile one and found that the latter can provide reliable measurements (e.g., accuracy and pupil dilation) that are suitable for general ET research [ 28 ]. However, existing ET-enabled MR headsets exhibit a trade-of between their mobility and ET capability (Figure 2). For example, some video seethrough MR headsets (e.g., HTC VIVE Pro Eye [ 29 ], Varjo XR-3 [ 30 ]) make use of ET sensors with high sampling rate and accuracy, but remain wired to a computer. Wireless MR headsets (e.g., HoloLens 2 [ 31 ]) are more suitable for mobile settings (e.g., outdoors or on shopfloors), but their ET features are inferior to those of dedicated mobile eye trackers that permit sub-degree accuracy, high sampling rate, and low latency [ 32 ].

To better understand the potential and challenges of utilizing an ET-enabled MR headset on shopfloors, we conducted an initial assessment across two industrial scenarios. In this paper, we present our preliminary findings from these scenarios that involve actual operators and shed light on gaze-based assessment on shopfloors.

2. EToS: Eye Tracking on the Shopfloor

We are currently involved in a project that aims to develop MR solutions to provide attentionand context-aware support to operators in real scenarios that are performed on the industrial shopfloors. In this project, we have two application partners (APs; both are industrial companies) that voluntarily contribute to our research and provide feedback on our MR solutions. The ifrst application scenario (with AP1) focuses on machine tooling operations that involves the semi-automated measurement of work pieces at predefined intervals. Here a human operator visually inspects components, uses a task-specific measurement head (i.e., the operator needs to select among available measurement heads), and considers diferent pieces of information in printed or digital form. According to AP1, these visual inspection processes are critical and require a high level-of-engagement of the operator because human error may prolong the production (due to re-measurement or even damage to components), thus reduction of outages is a key performance indicator (KPI).

AP2 is specialized in producing industrial machines; our second application scenario is the manual cleaning of an optical instrument, that needs to be performed at regular intervals to maintain output quality and avoid overheating of the machine. The primary KPI of AP2 is the reduction of the total number of support and repair requests with respect to this piece of equipment. This scenario includes the engagement of the operator with a (currently) nonautomated operation. However, it will help us to better understand the potential of ET-enabled MR in remote-support operations that may involve an automated-suggestion of help instructions (as envisioned in [ 1 ]).

In both scenarios, human operators need to follow procedural workflows that are documented in printed or digital form. Given the current state of the art, individual steps of these workflows can be presented through MR headsets to provide users with assistance. We take a closer look at those steps that require human visual inspection because we want to better understand how mobile MR+ET might enable real-time assessment and assistance to support human users. Together with our APs, we performed two experiments that are each composed of three phases: During the Briefings, we asked an expert operator to describe the individual steps of the particular scenario of interest (Figures 3 and 4). We transformed this description to a sequence diagram and asked the same expert to identify individual steps that require manual (or hands-based) interaction and visual inspection. The Data Collection phase, which we report on in more detail in Section 2.3, considered four setups (with/without MR and ET). In Debriefing, we showed the participants their own recordings and noted their feedback.

2.1. Participants

In the first scenario, an expert and a novice operator (both male and 45 years old) participated. They did not wear prescription glasses and had no experience with MR headsets. In the second scenario, a 49-year-old male expert participated. He did not wear prescription glasses and had some familiarity with MR headsets. We communicated with the participants in German at all times.

2.2. Apparatus and Material

In EToS-1, we used a tripod-mounted external camera for recording the participants from a spectator’s point of view. We used the Microsoft HoloLens2 (HL2) as the ET-enabled MR headset. Its frontal scene camera captures 30 frames per second (fps) at 1920 × 1080px resolution with a 64.69° horizontal FOV [ 37 ]. The built-in eye tracker has a sampling rate of 30Hz and 1.5-3° accuracy [ 31 ]. For recording the HL2’s ET data, we combined the Augmented Reality Eye Tracking Toolkit for Head Mounted Displays (ARETT) [ 32 ] with an application that we developed in the Unity 3D game engine. We used the Tobii Glasses 3 (TG3) as the mobile eye tracker [ 33 ]. With a 106° diagonal field-of-view, the TG3 can record gaze data at a 50 or 100 Hz sampling rate and 0.6° accuracy [ 33 ]. The front camera of the TG3 captures scenes in 1920 × 1080px resolution and 25 fps with a 95° horizontal FOV. We used iMotions to analyze gaze and scene recordings of the TG3 [ 38 ]. The head unit of the TG3 weighs approximately 76 grams, whereas the HL2 weighs approximately 566 grams.

We assessed operators’ comfort during the experimental sessions with a Comfort Questionnaire (CQ). The CQ contains six selected statements from the Comfort Rating Scale [ 39 ] and four additional statements that are specific to our context. The final CQ measures the comfort of wearable devices on a 7-point Likert-scale (i.e., an overall score of 7 indicates that the device is very comfortable) and contains the following statements which were presented to the participants in German: a) I feel tense or on edge because I am wearing the device [ 39 ]. b) I can feel the device moving [ 39 ]. b) The device is painful to wear [ 39 ]. d) I feel strange wearing the device [ 39 ]. e) The device afects the way I move [ 39 ]. f) I do not feel secure wearing the device [ 39 ]. g) Wearing the device distracts me from my work. h) I can imagine to wear the device in the future. i) The device is too heavy. j) The device restricts my field-of-view.

2.3. Procedure

In the first scenario, the semi-automated measurement of a work-piece, operators move between workbenches and must coordination of their hand and eye movements. We decomposed this scenario into the following steps (Figure 3): a) Load the measurement program on the computer. b) Clamp the work piece on the measurement machine. c) Check if the correct measurement head (MH) is mounted. If not, instruct the machine to change it. d) Start the measurement in lower speed to test the MH and if applicable increase the speed. e) Measure the work piece (automatic).

After the procedure, the operator records the measurements and proceeds with the work piece. We hypothesized that in c), an assessment of operators’ eye movements with respect to the MH and help materials (i.e., dynamic areas of interests or in short dynamic-AOIs) would be beneficial to better understand causes of human errors.

The second scenario, cleaning an optical instrument, requires the operator to sit throughout the task and coordinate eyes and hands across the following steps (Figure 4): a) Open the protective case. b) Unscrew, remove and clean the protective glass drawer. c) Check and clean the sealing. d) Clean the lens, inspect stains with green light and repeat this step if necessary. e) Reassemble the pieces.

In each scenario, we collected data across four diferent setups: • Setup 1: An external camera recording allows us capturing operators’ behavior under (almost) natural conditions. • Setup 2: Scene recording, HL2 without eye tracking, and CQ. In this setup, we introduce the HL2 to the participant. During the operation, the front camera of the HL2 records the scene from the operators’ view point. • Setup 3: Scene recording and HL2 including the gaze recording using ARETT [ 32 ]. • Setup 4: Scene recording, TG3, and CQ. This starts with an introduction of the TG3 to the participants. The TG3 is then used to record the scene and collect gaze data during the procedure.

Both experts worked through all of these setups in the respective scenario; to minimize learning efects, the novice operator in the first scenario was exposed only to the last two setups. At the end of Setups 2 and 4, each operator answered the comfort questionnaire.

3. Results

In EToS-1, we aimed at a preliminary assessment of mobile ET in user engagement with automation. Specifically, we compared a) Visual inspection behavior of an Expert vs. a Novice, b) Implications of employing a mobile eye tracker (i.e., no MR) vs. a low-fidelity eye tracker (HoloLens 2), and c) diferences between mobile (i.e., measurement scenario) vs. sedentary operation (i.e., cleaning scenario).

Expert vs. Novice: In the recordings of the semi-automated measurement, we were able to identify diferences based on operators’ expertise such as when and for how long an operator was engaged with the individual steps of the workflow. The gaze recordings show that the expert reached the step for visual inspection of the measurement head (Figure 3-c) after 109 seconds whereas the novice needed 225 seconds. Both operators then completed Step c in approximately 5 seconds. In contrast to the expert, the novice used 1 second less to check the already mounted measurement head and instead inspected wall-mounted support manuals. The novice completed the procedure (Figure 3-e) after 8 minutes in Setup 3 and after 6 minutes in Setup 4, respectively, and the expert needed 4 minutes in both setups.

Eye Tracking with the TG3 vs. HL2 in Dynamic Environments: For Setup 3, the definition of AOIs in 3D was done in a pre-processing step while building the HL2 application. Then, ARETT allowed us to assess whether the collected gaze data were within a particular AOI and calculate AOI-related measures accordingly. The AOI preparation and post-processing took us approximately 1.5 hours, independent of the recordings’ length.

We assessed the recordings of Setup 4 and identified the previously discussed steps of the two scenarios (Figures 3 and 4). Then we defined AOIs (i.e., 2D bounding boxes), and assessed the eye movements (e.g., dwell time, fixation duration and count, and response ratio) of the operators. The duration of this post-hoc assessment took approximately ten times as long as the average duration of the recordings (e.g., 60 minutes for a 6 minutes recording). In the cleaning scenario, the operator needed about 12 minutes in all setups, thus we did not observe an efect of the devices on the task duration.

Mobile vs. Sedentary Operation: In the sedentary operation (i.e., cleaning), the HL2 scene recordings showed that the objects of interest, e.g., those held by the operator, were often remaining beyond the HL2’s FOV. This was due to the operator’s proximity to those objects and the position of the HL2’s camera (Figure 1-b). On the other hand, with the TG3 this problem did not occur because the TG3’s camera is closer to the operators’ head. While performing the measurement procedure, operators frequently move between their desk and the workbench. The relevant objects are then not as close to the operator as in the cleaning operation, thus we were able to record them with both the HL2 and TG3.

General Comfort and Acceptance: Overall, the TG3 (mean=6.0) is perceived as being more comfortable than the HL2 (mean=4.6). Both experts rated both devices as more comfortable (combined means: 5.25 (HL2), 6.3 (TG3)) than the novice (means: 2.8 (HL2), 5.5 (TG3)). Both experts agreed that they could imagine to wear the HL2 in the future, and all three participants indicated the same for the TG3.

4. Discussion

Mobile eye trackers are useful tools for an assessment of users’ visual inspection behavior and their level-of-engagement in dynamic work environments such as on shopfloors. Mobile MR headsets are increasingly used on shopfloors to provide operators with assistance and contextual information. Here we reflect upon our observations and discuss the implications of utilizing two mobile ET-enabled devices in the assessment of user engagement on shopfloors.

Expert vs. Novice: In this initial attempt, EToS-1 allowed us to observe some diferences in the engagement and visual inspection behavior of operators on the shopfloor based on their experience level, which also makes sense intuitively. However, due to the small number of observations, our findings are hard to generalize. In a follow-up study (EToS-2), we plan to extend our evaluation to include attention-aware features and repeat the assessment with a larger sample size. In EToS-2, our prototypes (i.e., currently being developed) will allow us to assess the level-of-engagement of the operators based on their eye movements. For instance, in a teaching or training scenario, the expert operator will be able to observe the point-of-interest of the novice, check how the novice follows and performs the individual steps of a workflow, and provide respective feedback.

Eye Tracking with the TG3 vs. HL2 in Dynamic Environments: In iMotions, the manual adjustment of dynamic AOIs and the analysis of the TG3 data were time consuming yet straightforward as the underlying hardware and software are being developed and used by practitioners for many years. On the other hand, there are open-source solutions available for HL2. For example, ARETT [ 32 ] can be used for the data acquisition and analysis but it requires more programming work than the commercially available software for TG3. We believe that it is beneficial to explore the mobile eye tracking features of the HL2 on the shopfloor, because future mobile and gaze-enabled MR applications will run on the HL2 or similar headsets.

Mobile vs. Sedentary Operation: Especially in sedentary operation, it is important for practitioners to note that some relevant items might remain beyond the FOV of the HL2’s camera. In such cases, it is hard to provide attention- and context-aware assistance without restricting users’ natural behavior. However, in a mobile operation (e.g, measurement) this could be less problematic.

General Comfort and Acceptance: The TG3 weights less than the HL2, thus, it is not surprising that the participants preferred wearing it over HL2. This poses one requirement for future headsets to be light-weight. In the debriefing, our APs stated that MR headsets could be more easily accepted if the operators could see a direct benefit of them in their work. Therefore, in our subsequent research, we will focus on gaze-contingent and MR-based assistance. Moreover, for exploring the suitability of MR devices in an industrial context, recent research suggests taking additional concerns of workers into account, such as risk of distraction, (perceived) loss of competence, or privacy [40].

Other Implications: To manage the recordings, the TG3 needs a wireless connection to a computer. The management of the HL2’s recordings with ARETT can be done using its Web interface. In EToS-1, we noticed that wireless connectivity can be an issue depending on the physical conditions of the shopfloor. A portable WiFi router can be used for overcoming this problem. The scene recordings of the measurement scenario showed that the novice user preferred to be close to the task-specific objects. In future studies, it would be beneficial to assess the spatial proximity of the operator to objects or areas of interest (e.g., dangerous or moving parts).

5. Conclusion

EToS-1 was our first attempt at conducting mobile-eye-tracking research in a real industrial environment without enabling MR features. In EToS-1 we were able have an initial look at the eye-tracking related shortcomings of HL2 on the shopfloors. Currently, we are working on the development of attention- and context-aware features of a mixed-reality-testbed that is dedicated to assisting operators on the shopfloor. Based on the experience we collected in EToS-1, this testbed will allow us to identify situations (e.g., in maintenance, remote support or training) where operators should benefit from MR-based and automated assistance. We will report on our experimental findings in a follow-up publication (i.e., EToS-2).

Acknowledgments

This project was funded by the Swiss Innovation Agency Innosuisse (Project #48342.1 IP-ICT). We thank our application partners and operators for taking the time for participating in the experiments and for providing us with feedback. doi:1 0 . 1 5 1 8 / 0 0 1 8 7 2 0 0 5 3 6 5 3 8 7 5 . [40] K. E. Schein, P. A. Rauschnabel, Augmented Reality in Manufacturing: Exploring Workers’ Perceptions of Barriers (2021) 1–14. doi:1 0 . 1 1 0 9 / T E M . 2 0 2 1 . 3 0 9 3 8 3 3 .

References

[1] M. Baldauf , P. Fröhlich , S. Sadeghian , P. Palanque , V. Roto , W. Ju , L. Baillie , M. Tscheligi , Automation Experience at the Workplace, Association for Computing Machinery , New York, NY, USA, 2021 . URL: https://doi.org/10.1145/3411763.3441332.

[2] G. Fischer , Exploring design trade-ofs for quality of life in human-centered design , Interactions 25 ( 2017 ) 26 - 33 . URL: https://doi.org/10.1145/3170706. doi:1 0 . 1 1 4 5 / 3 1 7 0 7 0 6 .

[3] J. Orlosky , M. Sra , K. Bektaş , H. Peng , J. Kim , N. Kos'myna, T. Höllerer, A. Steed , K. Kiyokawa , K. Akşit , Telelife: The future of remote living , Frontiers in Virtual Reality 2 ( 2021 ) 147 . URL: https://www.frontiersin.org/article/10.3389/frvir. 2021 . 763340 . doi:1 0 . 3 3 8 9 / f r v i r . 2 0 2 1 . 7 6 3 3 4 0 .

[4] K. Bektaş , J. Kim , H. Peng , K. Kiyokawa , A. Steed , T. Höllerer , N. Kos'myna, M. Sra , J. Orlosky , K. Akşit , Telelife: A vision of remote living in 2035, in: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems , CHI '22, Association for Computing Machinery, New York, NY, USA, 2022 . doi:1 0 . 1 1 4 5 / 3 4 9 1 1 0 1 . 3 5 1 6 5 0 5 .

[5] R. Parasuraman , T. Sheridan , C. Wickens , A model for types and levels of human interaction with automation , IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 30 ( 2000 ) 286 - 297 . doi:1 0 . 1 1 0 9 / 3 4 6 8 . 8 4 4 3 5 4 .

[6] S. of Automotive Engineers, Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles sae report j3016 ( 2021 ). doi:1 0 . 4 2 7 1 / J 3 0 1 6 _ 2 0 2 1 0 4 .

[7] A. Schmidt , T. Herrmann, Intervention user interfaces: A new interaction paradigm for automated systems , Interactions 24 ( 2017 ) 40 - 45 . URL: https://doi.org/10.1145/3121357. doi:1 0 . 1 1 4 5 / 3 1 2 1 3 5 7 .

[8] B. Shneiderman , Human-centered artificial intelligence: Reliable, safe & trustworthy, International Journal of Human-Computer Interaction 36 ( 2020 ) 495 - 504 . URL: https://doi.org/10.1080/10447318. 2020 . 1741118 . doi:1 0 . 1 0 8 0 / 1 0 4 4 7 3 1 8 . 2 0 2 0 . 1 7 4 1 1 1 8 . a r X i v : h t t p s : / / d o i . o r g / 1 0 . 1 0 8 0 / 1 0 4 4 7 3 1 8 . 2 0 2 0 . 1 7 4 1 1 1 8 .

[9] V. Roto , P. Palanque , H. Karvonen , Engaging automation at work - a literature review , in: B. R. Barricelli , V. Roto , T. Clemmensen , P. Campos , A. Lopes , F. Gonçalves , J. AbdelnourNocera (Eds.), Human Work Interaction Design. Designing Engaging Automation , Springer International Publishing, Cham, 2019 , pp. 158 - 172 .

[10] R. Lindgren , M. Tscholl , S. Wang , E. Johnson, Enhancing learning and engagement through embodied interaction within a mixed reality simulation , Computers & Education 95 ( 2016 ) 174 - 187 . URL: https://www.sciencedirect.com/science/article/pii/S036013151630001X. doi:h t t p s : / / d o i . o r g / 1 0 . 1 0 1 6 / j . c o m p e d u . 2 0 1 6 . 0 1 . 0 0 1 .

[11] J. L. Kröger , O. H.-M. Lutz , F. Müller , What Does Your Gaze Reveal About You? On the Privacy Implications of Eye Tracking , Springer International Publishing, Cham, 2020 , pp. 226 - 241 . URL: https://doi.org/10.1007/978-3- 030 -42504-3_ 15 . doi:1 0 . 1 0 0 7 / 9 7 8 - 3 - 0 3 0 - 4 2 5 0 4 - 3 _ 1 5 .

[12] N. Kosmyna , C. Morris , U. Sarawgi, T. Nguyen , P. Maes , Attentivu: A wearable pair of eeg and eog glasses for real-time physiological processing , in: 2019 IEEE 16th International Conference on Wearable and Implantable Body Sensor Networks (BSN) , 2019 , pp. 1 - 4 . doi:1 0 . 1 1 0 9 / B S N . 2 0 1 9 . 8 7 7 1 0 8 0 .

[13] M. I. Ahmad , I. Keller, D. A. Robb , K. S. Lohan , A framework to estimate cognitive load using physiological data, Personal and Ubiquitous Computing ( 2020 ) 1 - 15 . doi:1 0 . 1 0 0 7 / s 0 0 7 7 9 - 0 2 0 - 0 1 4 5 5 - 7 .

[14] K. Bektaş , A. Çöltekin , Area of Interest Based Interaction and Geovisualization with WebGL , The Graphical Web Conference ( 2012 ) 1 - 14 .

[15] K. Bektaş , A. Çöltekin , J. Krüger , A. T. Duchowski , A Testbed Combining Visual Perception Models for Geographic Gaze Contingent Displays , in: E. Bertini, J. Kennedy , E. Puppo (Eds.), Eurographics Conference on Visualization (EuroVis) - Short Papers , 2015 , pp. 67 - 71 . URL: https://diglib.eg.org/handle/10.2312/eurovisshort.20151127. 067 - 071 . doi:1 0 . 2 3 1 2 / e u r o v i s s h o r t . 2 0 1 5 1 1 2 7 .

[16] K. Bektaş , A. Çöltekin , J. Krüger , A. T. Duchowski , S. I. Fabrikant , Geogcd: Improved visual search via gaze-contingent display , in: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications , ETRA '19, Association for Computing Machinery, New York, NY, USA, 2019 . URL: https://doi.org/10.1145/3317959.3321488. doi:1 0 . 1 1 4 5 / 3 3 1 7 9 5 9 . 3 3 2 1 4 8 8 .

[17] J. Grubert , T. Langlotz , S. Zollmann , H. Regenbrecht , Towards pervasive augmented reality: Context-awareness in augmented reality , IEEE transactions on visualization and computer graphics 23 ( 2016 ) 1706 - 1724 . doi:1 0 . 1 1 0 9 / T V C G . 2 0 1 6 . 2 5 4 3 7 2 0 .

[18] D. Lindlbauer , A. M. Feit , O. Hilliges , Context-aware online adaptation of mixed reality interfaces , in: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology , 2019 , pp. 147 - 160 . doi:1 0 . 1 1 4 5 / 3 3 3 2 1 6 5 . 3 3 4 7 9 4 5 .

[19] A. L. Gardony , R. W. Lindeman , T. T. Brunyé, Eye-tracking for human-centered mixed reality: promises and challenges , in: B. C. Kress , C. Peroz (Eds.), Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR , MR), volume 11310 , International Society for Optics and Photonics, SPIE , 2020 , pp. 230 - 247 . URL: https://doi.org/10.1117/12.2542699. doi:1 0 . 1 1 1 7 / 1 2 . 2 5 4 2 6 9 9 .

[20] M. Billinghurst , H. Kato , Collaborative mixed reality , in: Proceedings of the First International Symposium on Mixed Reality , 1999 , pp. 261 - 284 .

[21] T. Piumsomboon , A. Dey , B. Ens , G. Lee , M. Billinghurst , The efects of sharing awareness cues in collaborative mixed reality , Frontiers in Robotics and AI 6 ( 2019 ). URL: https: //www.frontiersin.org/article/10.3389/frobt. 2019 . 00005 . doi:1 0 . 3 3 8 9 / f r o b t . 2 0 1 9 . 0 0 0 0 5 .

[22] K. Bektas , Toward a pervasive gaze-contingent assistance system: Attention and contextawareness in augmented reality , in: ACM Symposium on Eye Tracking Research and Applications , ETRA '20 Adjunct, Association for Computing Machinery, New York, NY, USA, 2020 . URL: https://doi.org/10.1145/3379157.3391657. doi:1 0 . 1 1 4 5 / 3 3 7 9 1 5 7 . 3 3 9 1 6 5 7 .

[23] K. Kunze , Y. Utsumi , Y. Shiga , K. Kise , A. Bulling , I know what you are reading: Recognition of document types using mobile eye tracking , in: Proceedings of the 2013 International Symposium on Wearable Computers , ISWC '13, Association for Computing Machinery, New York, NY, USA, 2013 , p. 113 - 116 . URL: https://doi.org/10.1145/2493988.2494354. doi:1 0 . 1 1 4 5 / 2 4 9 3 9 8 8 . 2 4 9 4 3 5 4 .

[24] I. Giannopoulos , P. Kiefer , M. Raubal , Gazenav: Gaze-based pedestrian navigation , in: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services , MobileHCI '15, Association for Computing Machinery, New York, NY, USA, 2015 , p. 337 - 346 . URL: https://doi.org/10.1145/2785830.2785873. doi:1 0 . 1 1 4 5 / 2 7 8 5 8 3 0 . 2 7 8 5 8 7 3 .

[25] T. Templier , K. Bektaş , R. H. R. Hahnloser , Eye-Trace: Segmentation of Volumetric Microscopy Images with Eyegaze , Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems ( 2016 ) 5812 - 5823 . URL: http : / / dl.acm.org / citation.cfm?id = 2858036 .2858069{%}5Cnhttp : / / doi.acm. org / 10.1145 / 2858036.2858578. doi:1 0 . 1 1 4 5 / 2 8 5 8 0 3 6 . 2 8 5 8 5 7 8 .

[26] P. Majaranta , A. Bulling , Eye Tracking and Eye-Based Human-Computer Interaction , Springer London, London, 2014 , pp. 39 - 65 . URL: https://doi.org/10.1007/978-1- 4471 -6392- 3 _3. doi:1 0 . 1 0 0 7 / 9 7 8 - 1 - 4 4 7 1 - 6 3 9 2 - 3 _ 3 .

[27] M. Friedrich , N. Rußwinkel , C. Möhlenbrink , A guideline for integrating dynamic areas of interests in existing set-up for capturing eye movement: Looking at moving aircraft , Behavior research methods 49 ( 2017 ) 822 - 834 .

[28] B. V. Ehinger , K. Groß , I. Ibs , P. König , A new comprehensive eye-tracking test battery concurrently evaluating the pupil labs glasses and the eyelink 1000, PeerJ 7 ( 2019 ) e7086 .

[29] VIVE , VIVE Pro Eye Specs | VIVE United States , 2022 . URL: https://www.vive.com/us/ product/vive-pro-eye/specs/.

[30] Varjo , Varjo XR -3 - The industry's highest resolution XR headset | Varjo , 2022 . URL: https://varjo.com/products/xr-3/.

[31] Microsoft , Eye tracking on HoloLens 2 , 2022 . URL: https://docs.microsoft.com/en-us/ windows/mixed-reality/design/eye-tracking.

[32] S. Kapp , M. Barz , S. Mukhametov , D. Sonntag , J. Kuhn, ARETT: Augmented Reality Eye Tracking Toolkit for Head Mounted Displays, Sensors 21 ( 2021 ) 2234 . doi:1 0 . 3 3 9 0 / s 2 1 0 6 2 2 3 4 .

[33] Tobii Pro , Tobii Pro Glasses 3 , 2022 . URL: https://www.tobiipro.com/product-listing/tobiipro-glasses- 3 /.

[34] Pupil Labs , Pupil Core - Eye tracking platform technical specifications - Pupil Labs , 2022 . URL: https://pupil-labs.com/products/core/tech-specs/.

[35] Tobii Pro , Screen based eye tracker for research | Tobii Pro Fusion , 2019 . URL: https: //www.tobiipro.com/product-listing/fusion/.

[36] EyeLink, EyeLink 1000 Plus Technical Specifications , 2017 . URL: https : / / www.sr - research.com/wp-content/uploads/2017/11/eyelink-1000 - plus-specifications.pdf.

[37] Microsoft , Locatable camera overview - Mixed Reality , 2022 . URL: https : / / docs.microsoft.com/en-us/windows/mixed-reality/develop/advanced -concepts/locatablecamera-overview.

[38] iMotions, iMotions Platform, 2022 . URL: https://imotions.com/platform/.

[39] J. F. Knight , C. Baber , A Tool to Assess the Comfort of Wearable Computers , Human Factors: The Journal of the Human Factors and Ergonomics Society 47 ( 2005 ) 77 - 91 .