Evaluating the Ergonomics of Online Store User Interfaces Based on Visual Analytics Konstantin Dergachev 1 and Elena Dergacheva 1 1 Bryansk State Technical University, bulv. 50 let Oktyabrya 7, Bryansk, 241035, Russia Abstract The volume of the online commerce market is growing steadily; however, the online store conversion remains at a fairly low level. One of the reasons for the low conversion rate of online stores is the lack of the ergonomic design. The interface expert assessment and usability testing allow identifying interface errors of existing sites. The methodology is proposed for testing the ergonomics of the graphical user web interface of online stores based on choosing alternatives and visual analytics. The methodology is based on combining expert assessment of user interfaces with usability testing throughout the design process. Expert assessment is a quick way to find the main ergonomic problems, which is recommended to be used at the stages of developing technical specification, prototyping and final testing. The expert assessment mechanism of ergonomics is improved based on interface checklists and visual analysis of results. Within each of the ergonomics criteria (speed, errors, subjective satisfaction, comprehensibility), uniquely defined characteristics are selected to evaluate the ergonomics of the online store interface. Each ergonomic characteristic is assigned a weighting coefficient that determines its importance. The degree and quality of implementing the characteristics on a 0 to 5 scale are taken into account. Acceptable test results are obtained for the peer review method of the online stores interfaces for comparing the interfaces of Eldorado, M.Video and DNS sites. Keywords 1 Graphical user interface, Web interface, ergonomics, online store, interface design, prototyping, checklist, evaluation methodology, decision making, multi-criteria evaluation, visual analytics 1. Introduction The volume of online commerce has significantly grown recently. There are approximately 1.92 billion online shoppers worldwide. In 2021 e-commerce sales amounted for about 15% of all global retail sales. Analysts predict the upturn of this indicator to 22% in 2023 [1, 2]. The growth trend of the e-commerce sector is supported by the coronavirus pandemic as people are increasingly beginning to give preference to electronic shopping [3]. As a result, new online stores appear daily [1]. Despite the growing number of online stores, the world statistics states that only 2.58% of their visitors make purchases in the end [1]. Work [4] indentifies the Internet commerce concerns associated with the problems of user interfaces, namely the chaos and information overload of sites and user- unfriendly navigation. Therefore, the urgent task is to increase the conversion by optimising the site user interface, increasing its usability. Leading companies are paying more and more attention to optimising the user interfaces of their online stores, realising the importance of their usability. The interface affects the product retrieval rate, its presentation convenience, and, as a result, the decision to buy it, to reuse the service and to form an opinion about the seller. For instance, simplifying the registration process for adidas.com users led to GraphiCon 2021: 31st International Conference on Computer Graphics and Vision, September 27-30, 2021, Nizhny Novgorod, Russia EMAIL: kv.dergachev@gmail.com (K. Dergachev); eadergacheva2013@yandex.ru (E. Dergacheva) ORCID: 0000-0002-2016-8981 (K. Dergachev); 0000-0003-4562-2914 (E. Dergacheva) ©️ 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) increasing the number of accounts by almost 19%, which has ultimately enhanced revenues by more than $1 million [5]. Interface problems can be divided into design ergonomics concerns and the problems of interface incompatibility with the user activity structure. In relation to many online store sites, the second problem is successfully solved, due to the sufficient simplicity and standardising the process of finding and purchasing goods. If the user non-standard behaviour on the site is planned, related to the specifics of the online store subject area and the features of the purchase, then the interaction must be carefully checked on the system potential users. The problem of improving design ergonomics for many sites comes to the fore and is subject to more detailed study. The presence and properties of the Web interface individual components are regulated by the quality manual for Web interfaces ISO 9241-151: 2008 [7], however, aspects of evaluating the ergonomics of the online store Web interface are not affected by this and other standards [7, 8]. Therefore, the problem of creating a methodology for testing online store ergonomics is urgent. Comparative expert analysis of the speed of similar site users can be carried out using the GOMS method [9]. However, this method has significant limitations, does not allow identifying problems and ways for their solution, and is not intended for a comprehensive assessment of the interface ergonomics. In work [10], a method is proposed for formalizing the interface quality assessment relative to similar programme interfaces. However, the analogs under consideration may not be ideal, which may lead to decreasing the quality level and complicating the expert assessment procedure. Ergonomics testing is based on selecting and using many criteria and characteristics, the set of which can vary for each site. The general set of requirements for the website interfaces is determined by the standard [7]. When designing and evaluating each website, the standard requirements are to be transformed into a set of easily verifiable and understandable ergonomic criteria and characteristics. This set is developed by expert designers, domain specialists, constructors, customers, and other decision makers. Differences in viewpoints, as a rule, are observed both among the groups of decision- makers and within each group, which can lead to emerging redundant criteria, differences in evaluating the interface quality. The subjectivity of the criteria for evaluating the interface and interpreting the results of testing interfaces introduced by decision-takers makes the task of multi-criteria evaluation and choice of the alternatives urgent. The use of visual analytics methods allows reducing the number of alternatives and criteria, can increase the objectivity of the proposed assessment methods, as well as the objectivity of interpreting the study results of the interface ergonomics [11]. Applying specialized methods of data visualization can also improve the analysis efficiency of large amounts of data [12-15]. Using visual analytics is especially important for the comparative analysis of a few analog interfaces for several dozen characteristics at the stage of developing a technical task, as well as at the stage of evaluating several options for interface prototypes of the designed application. Thus, the task arises of creating a methodology for evaluating the interface ergonomics, which will allow determining a set of adequate assessment criteria, objectively evaluate by criteria, clearly present the study results and draw a conclusion about the advantages, disadvantages and directions of improving the interface. 2. Relationship between the interface design and ergonomics evaluation A generalized scheme for designing a user interface, on the basis of which a methodology for evaluating ergonomics has been developed, is shown in Figure 1. Figure 1: Generalized BPMN-scheme for designing an ergonomic interface The developed methodology is based on the following formulated principles: 1. The expert assessment and usability testing of the interface should be applied in the design, starting from developing technical specifications, and then when checking the correctness of the first and subsequent versions of the prototype, since with each new iteration, fixes in the interface are more expensive than at the previous one. 2. The biggest problems of analog interfaces or the system being developed can be discovered by experts involved in the design process. The expert conducts a preliminary check of the interface compliance with the technical specification requirements and ergonomics criteria. Therefore, starting from the early design stages, it is advisable to involve experts to evaluate the decisions made within the developed prototypes. Expert assessment usually precedes usability testing. 3. The procedure for evaluating the interface quality by an expert is heuristic and subjective; therefore it is necessary to formalize this process in order to ensure the objectivity and evidence of the expert conclusions with a minimum loss in the examination speed. To ensure the reliability and consistency of the research findings, it is necessary to provide for a multi-criteria evaluation of the expert judgment conformity and, if necessary, take measures aimed at increasing the conformity [16]. In addition, it is advisable for an expert to have visual analytics tools at his disposal. 4. Interface design is an iterative process focused on improving different versions of the interface prototype by periodic peer review and user testing [17, 18]. 5. The effectiveness of the interface solutions of each prototype is checked by means of the interface expert assessment and / or usability testing [9]. If testing shows acceptable results of compliance with the technical task requirements or with its abbreviated version that is the checklist, then one proceeds to developing a better version of the prototype, otherwise (if ergonomic problems are found), the current version is corrected or developed anew [17]. 6. Usability testing of the interface is carried out by potential users of the online store who are familiar with the subject area. It is recommended to involve test groups of 4-5 participants. This study is advisable to carry out after eliminating expert comments. A promising area of interface testing is biometric assessment and tracking the user view direction. These techniques make it possible to reveal the user hidden subconscious motives [19]. User interface ergonomics depends on the quality of carrying out individual work within the interface design framework. At the first design stage, the features of the subject area and analog interfaces are studied. Analysing the interfaces of analogous systems involves the expert selection of the evaluation criteria. Given the subjectivity and differences in the expert opinions, this task of multi- criteria evaluation will require involving the resources of the software analytical system to select objective criteria and the degree of their importance (the process of “Analysis of opinions”). Then ranking the evaluation criteria is performed and the interface is directly analyzed. The interface analysis can be performed both by an expert and by an analytical system that carries out the analysis on the rule base and the estimation model. Further, according to the analysis results, one obtains numerical and visualized data that reveal the advantages and disadvantages of the interface under consideration. The data received are used to develop the technical specification and interface checklist. After the expert assessment, usability testing of similar interfaces can be used, which helps to identify their objective additional flaws and to confirm the problems found by the experts. The analysis of user requirements, as a rule, is based on observing user activities according to pre- developed scenarios and questionnaires, allowing identifying goals, objectives, and typical examples of applying the interface. At the high-level design, the designer arranges navigation links between the interface blocks. Testing users by the classification method of card sorting helps one to determine the navigation links by the user view, to group functions in menus and on separate pages, to determine the product catalogue structure. In this situation, the “Analysis of opinions” block performs the processing and interpreting of statistical information obtained as a result of card sorting. Low-level design is the stage at which expert assessment involves estimating the quality and identifying shortcomings of the paper, presentation, pseudo-real and real versions of the prototype. The peer review scenario at this stage is similar to studying the analogs, with the exception of its iteration. Evaluating the prototype version interface is carried out within the framework of the previously developed technical task. Visualizing the level of the criteria fulfilment allows one to understand the degree of the project readiness and to determine the places where the interface needs improving. Usability testing allows one to determine the prototype compliance with the technical task requirements and is the main way to determine the ergonomics of the developed interface. 3. Methodology for evaluating ergonomics In checking the interface ergonomics, expert assessment takes a dominant position, especially at the initial design stages when developing technical specifications. The mechanism of checklists [20], supplemented by a visual evaluation model, allows having a quick and accurate assessment of the interface ergonomics or its prototype ergonomics. A checklist is a list of requirements for individual properties and interface elements that have their own weighting coefficients. If the requirement is met, its weight is added to the total amount; if it is not met, the weight is not added. The requirements are not always met unambiguously and at a high level, therefore it is proposed to take into account the degree of the requirement fulfilment on a six-point scale (0 – the requirement is not met, 5 – the requirement is met in full). An interface checklist is developed for each project, but a universal basic checklist grounded on the basic ergonomic requirements for such sites can be proposed for designing an online store site. To evaluate the ergonomics of the graphical user interface of online store sites, it is proposed: 1. To select uniquely defined characteristics (criteria) for evaluating the interface ergonomics. 2. To assign to the ergonomics characteristics j quantitative values of the weighting coefficients 𝑘𝑗 determining the importance of the characteristics. The sum of the weighting factors must be equal to 1. 3. To choose a scale for evaluating the ergonomics characteristics (for example, a six-point scale from pj=0 which means that the characteristic is not implemented until pj=5 which means that the characteristic is implemented at the highest level). 4. To calculate the integral estimation of ergonomics using the formula for determining the ergonomics level of the V interface: 𝑛 𝑉 = ∑ 𝑝𝑗 𝑘𝑗 (1) 𝑗=1 Formula (1) is only one of the possible options for obtaining an aggregate estimation; it is possible to use other types of decision rules, as well as their combinations. 5. To conduct ranking and clustering of ergonomic characteristics according to the main criteria of ergonomics (user speed, reduction in the number of errors, subjective satisfaction, clarity of the interface). 6. To build a visual evaluation model that allows one to objectively and visually present the results of the ergonomics test. 7. To draw conclusions about the shortcomings of the analyzed sites. Thus, the maximum possible value of user interface ergonomics level is Vmax=5. The ergonomics of any online store interface is based on the following criteria: 1. User speed 2. Reduction of user errors 3. User subjective satisfaction 4. Interface comprehensibility Each of the ergonomics criteria is implemented through various characteristics of a variety of interface elements and interface solutions determined by the user mental model, subject area and standard [7]. The problem of identifying the interface individual characteristics and especially determining their weighting coefficients is associated with the mutual influence of various ergonomic characteristics on different criteria of ergonomics. For example, fewer data entry errors tend to result in increasing the speed and user higher subjective satisfaction. Due to their versatility, such characteristics can have higher weighting coefficients characterising their importance to ensure the ergonomics of the site user interface. To test the methodology, the site interfaces of Eldorado [21], M.Video [22] and DNS [23] online stores are analysed. The comparison results are presented in Table 1-4. Table 1 Evaluating the quality of the online store interfaces by user speed № Ergonomic features / interface solutions kj pj Eldorado M.Video DNS 1. Interface quietness 0,04 2 4 4 2. Restricting on decision making (shopping cart) 0,03 3 5 5 3. Minimizing data entry 0,03 2 4 4 4. Navigating 0,03 4 4 4 5. Sorting and filtering search results 0,03 5 4 4 6. Combining typical actions (purchase) 0,02 5 2 4 7. Grouping related items 0,02 4 4 4 8. Simplicity (number of elements) 0,02 3 3 4 9. System memory (information storage) 0,02 4 5 4 10. Searching hints 0,02 5 4 4 11. Size of interface elements 0,015 3 4 5 12. Searching by part of a word 0,015 5 4 5 13. Searching history 0,015 4 3 5 14. Arranging items in order of importance 0,015 4 3 4 15. Need for vertical scrolling 0,01 4 3 4 16. Product browsing history 0,01 5 1 5 Table 2 Evaluating the quality of the online store interfaces by user error reduction Ergonomic features / interface solutions kj pj Eldorado M.Video DNS 1. Resistance to typos when searching 0,03 4 3 4 2. Using specialized elements for the data entry 0,03 4 4 3 3. Blocking erroneous actions 0,025 4 4 4 4. Indicating errors 0,015 4 4 4 5. Searching with the wrong keyboard layout 0,01 5 5 5 Table 3 Evaluating the quality of the online store interfaces by user subjective satisfaction № Ergonomic features / interface solutions kj pj Eldorado M.Video DNS 1. Integrity of style 0,05 1 5 4 2. Modular grid and visual constants 0,035 3 4 4 3. Rounded corners 0,03 2 5 4 4. Colour scheme 0,03 3 4 4 5. Fonts 0,025 2 5 4 6. “Airiness” of the interface (distance between the 0,02 3 4 4 elements) 7. Ease of user authentication 0,02 4 3 4 8. Content 0,02 4 4 4 9. Adaptability 0,02 4 4 4 10. Designing navigation links 0,02 4 3 4 11. Feedback (sending comments, questions, ratings) 0,015 5 5 5 12. Loading speed and indication 0,015 3 3 4 13. Using the golden ratio 0,01 2 5 5 14. Absence of “broken links” 0,01 5 5 5 15. Location information 0,01 5 3 5 Table 4 Evaluating the quality of the online store interfaces by the interface comprehensibility № Ergonomic features / interface solutions kj pj Eldorado M.Video DNS 1. Pictogram comprehensibility 0,035 5 3 4 2. Standard nature of elements 0,035 4 4 4 3. Changing properties of elements during the 0,03 4 3 4 interaction 4. Compliance with the industry standards 0,03 3 4 5 5. Displaying the access path 0,025 4 4 4 6. Visible ownership of controls to an object 0,02 4 4 4 7. Reference for typical actions 0,015 4 4 4 8. Tooltips 0,015 4 3 4 9. Subject area reference 0,015 5 3 3 As a result of the calculations according to formula 1, we will determine the level of the interface quality of the compared sites. Eldorado website has its interface ergonomics score of VE=3,575. M.Video website shows the second result, that is VM=3,88. DNS site has the best level of the interface ergonomics, that is VD =4,155 among those considered websites. The integral value of the interface quality level gives us insights into the necessity to improve the site. Making a decision about the characteristics that need to be improved requires clustering, ranking and comparing individual ergonomic characteristics. It is proposed to evaluate ergonomics within the four identified clusters that meet the main ergonomics criteria. A radar chart is chosen as a visual benchmark for comparing multiple interfaces. The choice of radar diagram is due to the fact that the evaluated characteristics are not related to each other, as well as the fact that the area bounded by the diagram line is proportional to the quality level of the interface. Let's construct separate diagrams for the characteristics of each of the four ergonomics criteria. The quality level of each feature is determined by multiplying the rating of each site by the feature weighting coefficient. As a result, based on the data in tables 1-4, we will obtain the corresponding diagrams for evaluating the ergonomic characteristics of online stores (Figure 2). 1 1 a) 16 0,2 2 b) 0,15 15 0,15 3 0,1 0,1 14 4 0,05 5 0,05 2 13 0 5 0 12 6 11 7 10 8 4 3 9 1 1 c) 15 0,25 2 d) 0,2 0,2 9 2 14 3 0,15 0,15 0,1 0,1 13 4 0,05 0,05 8 3 0 0 12 5 11 6 7 4 10 7 6 5 9 8 Eldorado M.Video DNS MAX Figure 2: Diagram of the ergonomics level of characteristics that affect: a) user speed; b) number of user errors; c) user subjective satisfaction; d) interface comprehensibility Analyzing the deviation of the line corresponding to a specific interface from the line of the maximum quality level (MAX), as well as the area formed by this line, it is possible to draw a conclusion about the level of the interface quality. The closer the points of the broken curve to the centre of the diagram, the lower the overall level of quality of this criterion implementation in the interface of the site in question. Using such diagrams, one can solve two types of problems: 1. Improving the interface of one of the analyzed sites (determining the ergonomic characteristics by which the site lags behind the maximum indicator and the competitors). 2. Designing the interface of a new site (determining the best ergonomic solutions or improving those available in analogs). Let's show by example approaches to solving the problem of improving the site of Eldorado online store. Figure 2a shows that M.Video and DNS sites are leading in terms of user speed. It is advisable to take ergonomic characteristics № 2, 11, 12, 13 of DNS site and № 9 of M.Video site as standards of implementation. It is necessary to pay attention to the characteristics of the leading interfaces; the points of their diagrams are as distant as possible from the maximum values. They are the best of the considered analogs, but their improvement will provide the most significant gain in ergonomics. Thus, we can recommend to improve and use ergonomic characteristics № 1, 3, 8 of DNS leader site. The characteristics of Eldorado site № 4, 7, 14 are rated on par with those of the competitors, but they need improving. The characteristics of Eldorado site № 5, 6 10, 12, 16 are maximum and do not need any improvement. The task of designing a new site interface requires knowledge of the main disadvantages and advantages of the similar sites. From the diagram in Figure 2, we can see that it is advisable to take characteristics № 2, 11, 12, 13, 16 of DNS site, characteristics № 5, 6, 10, 12 of Eldorado site and characteristics № 9 of M.Video site as standards of implementation. Then we consider the characteristics of the leading interfaces, as the points of their diagrams are distant from the maximum values. Thus, we can recommend to improve and use ergonomic characteristics № 1, 3, 4, 7, 8, 14 of DNS leader site. 4. Conclusion Thus, the proposed methodology makes it possible to determine the absolute value of the user interface ergonomics level for the online store website, as well as the relative level of interface quality in the case of comparing several sites. Presenting visual analysis tools obtained in the process of evaluating the ergonomics of the alternative site allows one to involve expert cognitive capabilities in the analysis process and to perceive the solution fully, entirely using all the evaluation criteria proposed in the work. The described technique can be applied at the initial design stages as an express analysis for setting development goals, as well as at the stage of designing and testing interface prototypes to evaluate the decisions made as an addition to usability testing methods and independently of them. With the help of expert assessment according to the methodology proposed in this work, it is possible to quickly determine the strengths and weaknesses of user interfaces, which are advisable to take into account when creating new sites for online stores or improving existing sites. 5. References [1] I. Stevanovic, eCommerce Statistics & the Industry’s Vibrant Future, 2019. URL: https://kommandotech.com/statistics/ecommerce-statistics/. [2] E. Dergacheva, Visualizing Socio-techno-natural Processes: Issues and Challenges, in: CEUR Workshop Proceedings of the 29th International Conference on Computer Graphics and Vision (2019), Vol. 2485, pp. 168-172. URL: http://ceur-ws.org/Vol-2485/paper38.pdf. [3] E.A. Dergacheva, E.S. Demidenko, Visualizing Global Socio-Technogenic Human Transformation: Digital Challenges of Living Earth, in: CEUR Workshop Proceedings of the 30th International Conference on Computer Graphics and Vision (2020). Proceedings of the 30th International Conference on Computer Graphics and Machine Vision, Vol. 2744, pp.44-1 – 44-14. URL: http://ceur-ws.org/Vol-2744/paper44.pdf. doi: https://doi.org/10.51130/graphicon-2020-2- 3-44. [4] E.V. Matuzenko, O.A. Glazunova, A.A. Izvarin, Trends, Problems and Prospects for the Development of Electronic Commerce in the Sphere of Internet Trade, in: Bulletin of the Belgorod University of Cooperation, Economics and Law, Vol. 1 (86), 2021, pp. 197-206. doi: 10.21295/2223-5639-2021-1-197-206. [5] V.V. Evenko, A.A. Puzacheva, D.D. Belikova, Ergonomic Adjustments of a Website Due to Changes in Preferences and Mechanisms of Information Perception by People, in: CEUR Workshop Proceedings of the 29th International Conference on Computer Graphics and Vision Vol. 2485, 2019, pp. 211-214. doi: 10.30987/graphicon-2019-2-211-214. [6] www.accenture.com, Data-driven decision making delivers results, 2020. URL: https://www.accenture.com/ru-ru/case-studies/digital/success-adidas-data-driven-experience- design. [7] ISO 9241-151:2008 Ergonomics of human-system interaction – Part 151: Guidance on World Wide Web user interfaces. [8] ISO 9241-210:2010 Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems. [9] V.A. Zeng, Evaluating the Design Quality of New Generation User Interfaces, in: Bulletin of Tula State University. Technical sciences, Vol. 12, 2019, pp. 404-410. [10] I.A. Ponomarev, Developing Models and Algorithms for Multi-criteria Evaluation of the Graphical User Interface Quality, Moscow, IPK MIET, 2006. [11] A.A. Zakharova, D.A. Korostelyov, Application of Visual Analytics Methods to Reduce the Dimensionality of Decision-making Problems, Scientific Visualization. Vol. 12, 2020, Vol. 4. pp. 23-32. doi: 10.26583/sv.12.4.03. [12] T.Runkler, Data Visualization. In: Data Analytics. Springer Vieweg, Wiesbaden (2016). doi: 10.1007/978-3-658-14075-5_4. [13] A.G.Podvesovskii, R.A. Isaev, Visualization Metaphors for Fuzzy Cognitive Maps. Scientific Visualization 10(4), (2018), 13-29. doi: 10.26583/sv.10.4.02. [14] A.A.Zakharova, A.V. Shklyar, Informative features of data visualization tasks. Scientific Visualization, 7 (2), (2015), 73-80. [15] A.A. Zakharova, A.V. Shklyar, Y.S. Rizen, Measurable features of visualization task Scientific Visualization, 8 (1), (2016), 95-107. [16] A. Podvesovskii, O. Mikhaleva, et. al., Model of Control of Expert Estimates Consistency in Distributed Group Expertise. in: A. Kravets et. al. (eds) Creativity in Intelligent Technologies and Data Science. CIT&DS 2017. Communications in Computer and Information Science; 754. Cham: Springer, 2017, pp. 361-374. doi: 10.1007/978-3-319-65551-2_26. [17] O.V. Sayapin, O.V. Tikhanychev, S.V. Chiskidov, I.A. Bystrakova, Development of Applied Programme Interfaces: Modelling or Prototyping, in: Applied Informatics, Vol. 15, No. 1 (85), 2020, pp. 47-56. doi: 10.24411/1993-8314-2020-10004. [18] O.V. Tikhanychev, User Interfaces in Automated Systems: Development Problems, in: Software systems and computational methods, Vol. 2, 2019, pp. 11-22. doi: 10.7256/2454- 0714.2019.2.28443. [19] D.A. Kostyuk, O.O. Latiy, A.A. Markina, Approach to Biometric Evaluation of the Graphical User Interface Ergonomics, in: Bulletin of Brest State Technical University. Physics, mathematics, computer science, Vol. 5 (101), 2016, pp. 46-49. [20] D.P. Soroka, Evaluating Method Analysis of the User Interface Ergonomic Indicators In the world of scientific discoveries, No. 10-2 (70), 2015, pp. 894-906. [21] Eldorado.ru, 2021. URL: https://www.eldorado.ru [22] M.Video, 2021. URL: https://www.mvideo.ru [23] DNS, 2021. URL: https://www.dns-shop.ru