Area of Interest Based Assessment of Software Interface Usability for Human-Computer Interaction Using Eye-Tracking Oleksandr Gordieiev1, Vyacheslav Kharchenko2, Daria Gordieieva1, Inna Kondius1 and Nataliia Lishchyna1 1 Lutsk National Technical University, Lvivska Str., 75, Lutsk, 43018, Ukraine 2 National Aerospace University "Kharkiv Aviation Institute", Chkalov Str.,17, Kharkiv, 61070, Ukraine Abstract Non-functional requirements to software quality (SWQ) is described in general by SWQ models characteristics. Last the most famous software quality model ISO/IEC 25010 includes eight characteristics: functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability and portability. Usability as a SWQ characteristic must include subcharacteristics, inherent to software user interface quality as static object and also as subcharacteristics of process user interaction, i.e. is user-computer interaction in real time. In existing models of quality and quality assessment of usability the peculiarities of human- computer interaction (HCI) in real time are not taken into account in real time. Software interface usability model (SIUM) for HCI and models for its assessment are suggested. Such models are interconnected through a single nomenclature of subcharacteristics. HCI SIUM consists of two parts and includes a set of metrics, which correspond to the defined under subcharacteristics. Particularity of model is that all primitives for calculation software interface usability metrics for HCI obtained only with use of software and hardware complex of the eye-tracker. Keywords Interaction quality, software usability, human-computer interaction, metrics usability. 1. Introduction User interacts with the software through the interface. Software interface quality is determined, on the one hand, by usability characteristic as static object, and on the other, – with user interaction. Definitions of usability and human-computer interaction and interconnection have been discussed by many researchers and presented in the different standards. Usability is a degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [1]. Human-computer interaction (HCI) is a process of interaction between computer or mobile device and user through interface, when the user, analyzing the information obtained (predominantly visual), interacts with computer through interface, using a keyboard, mouse, webcam, etc (Fig. 1). It is experimentally established that the quality of HCI depends on not only by user interface quality and research experience, but also human-computer interaction quality in real time [2,3]. Interaction is direct user interaction with the software through the click of a computer mouse. At the same time, the user is in the process of constant «visual» interactivity with the software interface [4- 6]. ________________________ IntelITSIS’2022: 3nd International Workshop on Intelligent Information Technologies and Systems of Information Security, March 23–25, 2022, Khmelnytskyi, Ukraine EMAIL: oleksandr.gordieiev@gmail.com (O. Gordieiev); v.kharchenko@csn.khai.edu (V. Kharchenko); gordeyeva.daria@gmail.com (D. Gordieieva); innastk@ukr.net (I. Kondius); lischyna@gmail.com (N. Lishchyna) ORCID: 0000-0003-2517-9388 (O. Gordieiev); 0000-0001-5352-077X (V. Kharchenko); 0000-0003-0474-3138 (D. Gordieieva); 0000- 0003-3416-7506 (I. Kondius); 0000-0002-5200-536X (N. Lishchyna) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) Figure 1: The relationship between usability and human-computer interaction: 1 - use of the user interface; 2 - receiving information from the software The standards [1, 7-9] describe software quality models, requirements to no-functional software characteristics, life cycle processes and so on. The standard [8] defines main definitions and concepts for HCI including usability as the most important characteristic. The standards [10, 11] formulate requirements to software quality and measurement of quality in use. Authors of [12, 13] developed software quality models based on the standards and discussed metrics and techniques of assessment of software for different applications considering usability characteristic [14-16]. Last years researchers analyze application of dynamical techniques for HCI assessment and implementation, especially eye tracking for bio-monitored integrated circuits [17], pedestrian-automated vehicles [18], electronic chart displays [19], smart home [20, 21], industry and transport [22]. Evolution of the software quality models in context of usability and security [23] and model for assessment of HCI considering software interface usability [24-26] are discussed taking into account structure of general and individual requirement quality models [27]. Authors of [28] suggested concept of using eye-tracking to assess and provide safety, security and usability requirements to HCI. It can be concluded that according with known software quality models and software usability quality assessment user interface is considered as a static object and its interactivity is not taken into account at all, especially in real time. There is a gap between application of eye-tracking for dynamical HCI and models for usability assessment. The aim of the article is developing quality model of software interface usability for human- computer interaction and model for assessment of quality considering application of eye-tracking technology (ET). In this work it’s used concepts described in [22, 28] such as: - eye-tracking (oculagraphy) – the process of determining the coordinates of the gaze, i.e the point of intersection of the optical axis of the eyeball and the area of the object of observation or screen, which depicts a visual stimulus. Eye-tracker – a device that supports the process of determining the orientation of the optical axis of the eyeball in space, i.e. a device designed to track the eyes of the respondent; - visual route – a type of visualization of eye-tracking data, which is a map showing the location, order and time spent by the respondent in response to a stimulus, for example, a web page, printed announcement or video. The sequence of view points is indicated by numbers. The time spent on the look is given as the duration of the fixation and corresponds to the diameter of the fixation circle: the longer you look, the larger the diameter of the circle; - research scenario – a pre-developed step-by-step plan of research, which is developed by the researcher. The paper is structuring as follows: section 2 describe HCI usability model as two-level hierarchy of characteristics and subcharacteristics; section 3 analyses features of HCI by use of eye-tracking technology and types of interaction; model for HCI usability assessment based on application of ET is suggested in section 4. Section 5 discusses examples of use of developed models and section 6 concludes the paper and describes future research directions. 2. HCI usability model For developing model we use eye-tracking technology providing assessment of visual interactivity and direct interactivity with user interface in real time [6]. An approach to assessment is based on concept of area of interest (AoI). AoI is a limited area (perimeter of a rectangle, circle, oval, etc.) of the object under study (for example, website pages) for which it is necessary to calculate the metrics of eye-tracking. This area can be a navigation bar, software, a paragraph of text, a product on a shelf, a billboard or a sign at the airport. Set of characteristics, which define HCI usability quality and structure of its model are presented on Fig.2. Characteristics of the human-machine interaction usability quality are divided into two groups: 1) characteristics of a complex making a decision; 2) characteristics of interactive attention. 1. Interactive quality 1.1 Complex making a decision 1.2.Interactive attention 1.1.1 Visibility and target search 1.2.1 Cognitive processing 1.1.2 Goal recognition 1.2.2 Emotional arousal 1.1.3 Decision making 1.2.3 Attention in the field of interest Figure 2: Structural diagram of the software interface usability model for human-computer interaction The first group includes the following characteristics: • complex making a decision determines the success and speed of finding the desired goal; • goal recognition determines the difficulty of target recognition; • decision making characterizes the complexity (simplicity) and speed of decision making. The second group includes following characteristics: • cognitive processing determines the speed of information processing by the respondent; • emotional arousal characterizes the level of excitation in the area of interest and outside this area and the dynamics of excitation; • attention in the field of interest determines the level and stability of attention in the field of interest. 3. HCI using eye-tracking Process of eye-tracking assumes the presence of an eye-tracker and the presence of a respondent and a researcher. According to the pre-designed scenario, the respondent performs actions on the computer, and the eye-tracker records the movement of his eyes during this process. To do this, the eye-tracker illuminates the respondent's eyes with infrared light and records the reflection of infrared light from the retina of his eyes. This procedure allows the eye-tracker to find the center of the respondent's pupil and allows you to analyze the reflection of infrared light from the cornea. Each eye of respondent has reflection of light from the cornea. If respondent keeps his head still and looks left, right, up and down, the reflection moves along with the pupil. The distance between the center of the pupil and the reflection of light is changing. Thus, the point to which the respondent's gaze is directed can be determined through the position of the center of the pupil relative to the reflection of the cornea. If respondent moves his head, looking at the same place, the distance between the center of the pupil and the reflection of the cornea remains unchanged. Even if respondent is moving, the eye-tracker will determine that he is looking at the same point. Modern commercial eye-trackers consist of two components: light source close to infrared, which creates a reflection in the person eye, and a video camera sensitive to infrared light (Fig. 3). The camera focuses on the respondent's eyes and records the reflection. Using software that supports the work of the eye-tracker, location of view is calculated and superimposed on an object, such as a web page. Eye-tracker uses a length of wave which invisible to humans, and therefore does not distract their attention, but is reflected by the eye. Figure 3: Schematic representation of the eye-tracker: visual information range 180 °, focus of attention 2 ° The respondent's eyes move an average of three to four times per second. Such rapid eye movements are called saccades and they are the fastest movements performed by the external parts of the human body. To prevent clouding, human vision is almost completely suppressed during saccades. Visual information is perceived only when the eyes are relatively still and focused on an object, i.e. there is a fixation. It lasts from 0.1 to 0.5 s, after which the eyes move again (through the saccades) to the next part of the field of view. Thus, human vision is in constant motion, from the current fixation through the saccade to a new fixation. Fig. 4 shows the visual route of the respondent, who looks at the registration form of the conference. Fixations are given in the form of points, and saccades - in the form of lines connecting the points of fixation. The size of the point is proportional to the duration of fixation. Eye-tracking involves interactive evaluation not of the entire user interface, but of its limited area, such as a text box, control, etc. There can be several such areas on one object (Fig. 4). The interactive quality of usability of HCI complements is the basic part of usability. When describing the model of evaluation of interactive interaction, it should be taken into account that according to the research scenario, the user is involved in two parallel processes: • visual interaction, when the movement of his eyes is traced, which is described by a sequence of alternating saccades and fixations. Since saccade is a logical transition between fixations, we will consider only fixations. This process is called visual interactivity; • direct interaction with the user interface using a computer mouse and pressing its left button while controlling the interface. This process is called real interactivity. Figure 4: Areas of interest (1, 2) and visual route of the respondent 4. AoI-ET based HCI usability assessment As part of assessment processes, the necessary measurements will carry out at certain discrete time intervals. For visual interaction the following should be fixed: the counter of discrete measurements, time of fixing of measurement, coordinates of focus of attention. Real interaction includes all measured values of visual fixation and the parameter (event) – pressing the left button of a computer mouse is added. Thus, to describe this model in accordance with the visual and real interaction, we introduce the following three sets of model elements: VI – set of data of visual interaction of the user with the user interface (1):  VI = N ,ti ,xi , yi  N  i =1 , (1) where N is the number of discrete measurements (may coincide with the number of fixations), ti is the measurement time, xi, yi – are the fixation coordinates; RI – a set of user interaction data with the user interface using a computer mouse (2):  N    RI =  N , t j ,x j , y j ,c j , j =1  (2) where xj, yj – coordinates of the mouse cursor, cj – the event of pressing the left mouse button of a computer mouse (mouse button can be pressed – 1, or not pressed – 0); AoI – many areas of interest (3): AoI = aoin , j =1 (3) where aoij – is an area of interest. Note that it is necessary to separate the data recorded in the area of interest and beyond. In the plane of the coordinate axes we introduce values to describe the area of interest: on the abscissa axis – xj1, xj2, on the ordinate axis – yj1, yj2 (Fig. 5). Figure 5: Area of interest in the coordinate plane To describe the impact of the values of the coordinates x and y in the area of interest, we use the following characteristic functions (4,5): 1,( x  x j1 )  ( x  x j 2 )    W jx ( x ) =  , (4)  0 ,( x  x )  ( x  x )   j1 j 2   1 ,( y  y j1 )  ( y  y j 2  ) W jy ( y ) =  .  0,( y  y j1 )  ( y  y j 2 ) (5)   Such ratios describe the hit (value 1) or miss (value 0) of x and y values in the area of interest. Usually the area of interest is approximately equal to the user interface element. In practice, there is no clear equality, as the expert determines the location of the area of interest not by coordinates, but visually. Therefore, the coordinates of the user interface element and the area of interest are approximately equal (6,7): x j1  spsuie1  SPSUIE , x j 2  fpsuie1  FPSUIE , (6) y j1  spsuie2  SPSUIE , y j 2  fpsuie2  FPSUIE , (7) where SPSUIE – Start Position of Software User Inter face Element; spsuie – the x and y coordinates of the upper left corner of the user interface element; FPSUIE – Final Position of Software User Interface Element; fpsuie – the x and y coordinates of the lower right corner of the user interface element. 5. Case study Consider a real example of research into the quality of the user interface of the conference website «Dependable Systems, Services and Technologies» [29] (Fig. 6). Let's set the initial values: area of interest – the element of the user interface «Explanatory information». Clicks of a computer mouse are marked with numbers on the fixation points. Figure 6: Correspondence of the AoI to the user interface element Assuming that the measurements are performed once per second, we obtain the following values of the sets VI (8) and RI (9):  1,38, 4 17 , 2,38, 4 17 , 3,8,3 17 ,   1  2  3    4,8,34 ,5,8,35 ,6 ,24 ,36 , 17 17 17    7 ,24,317 ,8,24 ,317 ,9 ,20 ,217 ,   7 8 9  VI =   (8) 10,20,217 ,11,20,217 ,12,26,417 ,  10 11 12   17  13,26,413 ,14,26,414 ,15, 26, 415 , 17 17   16,30,317 ,17 ,30,317   16 17   1,14,8,0 17 , 2,14,8,0 17 , 3,14,8,0 17 ,   1  2  3    4,14,8,04 ,5,14,8,05 ,6,14,8,16 , 17 17 17    7 ,14,8,017 ,8, 26,8,017 ,9, 26,8,017 ,   7 8 9  RI =   (9) 10,26,8,017 ,11,26,8,017 ,12,26,8,017 ,  10 11 12    13,26,8,113 ,14,26,8,014 ,15,30,5,015 ,  17 17 17   16,30,5,117 ,17 ,30,5,017   16 17  Analyzing the obtained values of sets VI and RI, we can say the following: the number of discrete measurements to describe the visual interactivity of set VI is 17. Usually several discrete measurements correspond to one fixation: - fixation № 1 corresponds to measurements 1 and 2; - fixation № 2 corresponds to measurement 3-5; - fixation №3 corresponds to measurement 6-8; - fixation №4 corresponds to measurement 9-11; - fixation №5 corresponds to measurement 12-15; - fixation №6 corresponds to measurement 16, 17. For the RI set: there were three clicks on the left mouse button during 6, 13 and 16 discrete measurements. A nomenclature of metrics for assessing the quality of usability of software for human-computer interaction has been developed. It was previously established that all metrics by logic can be divided into two major groups: metrics of virtual interaction and metrics of real interaction. A detailed analysis of the metrics showed that they can be classified in more detail and accurately at the level of groups of successive stages. Thus, all metrics are classified as follows: - the group of metrics «Complex making a decision» is divided by the sequence of stages «Visibility and target search», «Goal recognition» and «Decision making»; - the metric group «Interactive Attention» is divided by sequence of stages «Cognitive processing», «Emotional arousal» and «Attention in the field of interest». In order to present the relationship between the metrics and the processes of virtual and real interactivity, for each metric there is a relationship with the corresponding process VI or RI. An example of a metric description is given in the table 1. Table 1 Description of interactive quality assessment metrics Identifier VI/RI Metrics Description Formula (primitives) IM1 VI SSGU – user Calculated based on the SSGU=UI / AU, targeting success ratio of target users to where UI – the number of users the total number of who pay attention to the area of users interest, AU – total number of users Conclusions The proposed models describe the interactive quality of usability of the software interface for human-computer interaction, as well as provide its quantitative assessment using certain groups of metrics and indicators. These models allow increasing the accuracy, completeness and trustworthiness of the assessment. The peculiarity of the model is that all primitives for calculating metrics have been obtained using hardware and software complex eye-tracker only. This allows you to get the most authenticity initial data (primitives) for the calculation and, in the end, a authenticity result. It is planned to develop a tool that will support the process of assessing the interactive quality of the usability of the software interface for human-computer interaction. It is advisable to implement the results in a new and promising direction of cyber interfaces [30]. References [1] International Organization for Standardization, International Electrotechnical Commission (2011). International standard ISO/IEC 25010:2011. Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – System and software quality models, 34 p. [2] Singh, H., Mahmoud, Q. (2020). NLP-Based Approach for Predicting HMI State Sequences Towards Monitoring Operator Situational Awareness. Sensors 2020. Vol. 20(11), P. 1-33. https://doi.org/10.3390/s20113228 [3] Christina K., Schoch S., Naujoks F., Hergeth S., Keinath A. and Neukum A. Standardized Test Procedure for External Human–Machine Interfaces of Automated Vehicles. Information 2020, Vol. 11(3). P. 1-19. https://doi.org/10.3390/info11030173 [4] Bissoli A., Lavino-Junior D., Sime M., Encarnação L, Bastos-Filho T. A Human–Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things. Sensors 2019, Vol. 19(4). P.1-26. https://doi.org/10.3390/s19040859 [5] Martinez-Marquez, D., Pingali S., Panuwatwanich K., Stewart R, Mohamed S., Application of Eye Tracking Technology in Aviation, Maritime, and Construction Industries: A Systematic Review. Sensors 2021. Vol. 21(13). https://doi.org/10.3390/s21134289 [6] Paing, M., Juhong A., Pintavirooj C. Design and Development of an Assistive System Based on Eye Tracking. Electronics 2022, 11, 535. https://doi.org/10.3390/electronics11040535 [7] International Organization for Standardization, International Electrotechnical Commission (2001), International standard ISO/IEC 9126-1:2001. Software engineering – Product quality – Part 1: Quality model, 25 p. [8] International Organization for Standardization, International Electrotechnical Commission (2018), International standard ISO 9241-11:2018. Ergonomics of human-system interaction – Part 11: Usability: Definitions and concepts, 29 p. [9] International Organization for Standardization, International Electrotechnical Commission (2006), International standard ISO/IEC 14764:2006. Software Engineering – Software Life Cycle Processes – Maintenance, 44 p. [10] International Organization for Standardization, International Electrotechnical Commission (2016), International standard ISO/IEC 25022:2016 Systems and software engineering – Systems and software quality requirements and evaluation (SQuaRE) – Measurement of quality in use, 41 p. [11] International Organization for Standardization, International Electrotechnical Commission (2016), International standard ISO/IEC 25023:2016 Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Measurement of system and software product quality, 45 p. [12] Rawashdeh, A., Bassem, M. (2006). New Software Quality Model for Evaluating COTS Components, Journal of Computer Science, Vol. 2(4). P.373-381. [13] Upadhyay N., Despande B., Agrawal V. (2011). Towards a Software Component Quality Model, In proceedings of the 1st International Conference on Computer Science and Information Technology, P. 398-412. http://doi.org/10.1007/978-3-642-17857-3_40 [14] Seffah A., Donyaee M., Kline, R., Padda H. (2006). Usability measurement and metrics: A consolidated model, Software Quality Journal, Vol. 14(2). P.159-178. http://doi.org/10.1007/s11219-006-7600-8 [15] Montero F., Lopez-Jaquero V., Lozano M., Gonzalez P. (2003), Usability and web site evaluation. Quality models and user testing evaluations, In proceedings of the 5th International Conference on Enterprise Information Systems, P. 525-528. [16] Musabini A., Nguyen, K., Rouyer R., Lilis Y. (2020), Influence of Adaptive Human–Machine Interface on Electric-Vehicle Range-Anxiety Mitigation. Multimodal Technol. Interact. 2020, Vol. 4(1). P. 1-22. https://doi.org/10.3390/mti4010004 [17] Massin L., Lahuec, C., Seguin F., Nourrit V., de Bougrenet de la Tocnaye J.-L. Multipurpose Bio-Monitored Integrated Circuit in a Contact Lens Eye-Tracker. Sensors 2022, Vol. 22(2). P.1- 11. https://doi.org/10.3390/s22020595 [18] Guo F., Lyu W., Ren Z., Li M., Liu Z., A Video-Based, Eye-Tracking Study to Investigate the Effect of eHMI Modalities and Locations on Pedestrian–Automated Vehicle Interaction. Sustainability 2022. Vol. 14(9), P. 1-21. https://doi.org/10.3390/su14095633 [19] Arslan O., Atik O., Kahraman S. (2021). Eye tracking in usability of electronic chart display and information system. The Journal of Navigation, Vol. 74(3), P.594-604. https://doi.org/10.1017/S0373463320000624 [20] Bissoli A., Lavino-Junior D., Sime M., Encarnação L., Bastos-Filho T. (2019). A Human– Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things. Sensors, Vol. 19(4), P. 1-26. https://doi.org/10.3390/s19040859 [21] Paing M.P., Juhong A., Pintavirooj C. (2022) Design and Development of an Assistive System Based on Eye Tracking. Electronics, Vol. 11(4), P.1-20. https://doi.org/10.3390/electronics11040535 [22] Martinez-Marquez D., Pingali S., Panuwatwanich K. Stewart R.A. Mohamed S. (2021) Application of Eye Tracking Technology in Aviation, Maritime, and Construction Industries: A Systematic Review. Sensors, Vol. 21 (13), P. 1-40. https://doi.org/10.3390/s21134289 [23] Gordieiev O., Kharchenko V., Leontiiev K. (2018). Usability, security and safety interaction: profile and metrics based analysis, In, proceedings of the 13 International Conference on Dependability and Complex Systems (DepCoS-RELCOMEX), P. 238-247. http://doi.org/10.1007/978-3-319-91446-6_23. [24] Godieiev O., Kharchenko V., Vereshchak K. Usable Security Versus Secure Usability: an Assessment of Attributes Interaction, In proceedings of the 13th International Conference on ICT in Education, Research and Industrial Applications. Integration, Harmonization and Knowledge Transfer (ICTERI), 2017. P. 727-740. [25] Feby Artwodini Muqtadiroh, Hanim Maria Astuti, Eko Wahyu Tyas Darmaningrat, Fenty Rizky Aprilian (2017). Usability Evaluation to Enhance Software Quality of Cultural Conservation System Based on Nielsen Model, Procedia Computer Science, Vol. 124, P. 513-521. https://doi.org/10.1016/j.procs.2017.12.184. [26] Gordieiev O. (2020). A models and assessment of quality of human-computer interaction software interface usability. Radioelectronic and Computer Systems, Volume 3 (95), P. 84-96. [27] Gordieiev O., (2020). Software individual requirement quality model. Radioelectronic and Computer Systems, Volume 2 (94), P. 48-58. [28] K. Bobrovnikova, S. Lysenko, B. Savenko, P. Gaj, O. Savenko. Technique for IoT malware detection based on control flow graph analysis. Radioelectronic and Computer Systems, 2022(1), pp. 141–153. [29] Web-site conference «Dependable Systems, Services and Technologies» www.dessert.ieee.org.ua [30] Zafar S., Nazir M.K., Bakhshi T., Khattak H., Khan S., Bilal M., Choo K.R., Kwak K., Sabah A. (2021). A Systematic Review of Bio-Cyber Interface Technologies and Security Issues for Internet of Bio-Nano Things. IEEE Access, 9, P. 93529-93566. http://doi: 10.1109/access.2021.3093442.