The volume of the online commerce market is growing steadily; however, the online store conversion remains at a fairly low level. One of the reasons for the low conversion rate of online stores is the lack of the ergonomic design. The interface expert assessment and usability testing allow identifying interface errors of existing sites. The methodology is proposed for testing the ergonomics of the graphical user web interface of online stores based on choosing alternatives and visual analytics. The methodology is based on combining expert assessment of user interfaces with usability testing throughout the design process. Expert assessment is a quick way to find the main ergonomic problems, which is recommended to be used at the stages of developing technical specification, prototyping and final testing. The expert assessment mechanism of ergonomics is improved based on interface checklists and visual analysis of results.
2021 Copyright for this paper by its authors.
increasing the number of accounts by almost 19%, which has ultimately enhanced revenues by more
than $1 million [
Interface problems can be divided into design ergonomics concerns and the problems of interface incompatibility with the user activity structure. In relation to many online store sites, the second problem is successfully solved, due to the sufficient simplicity and standardising the process of finding and purchasing goods. If the user non-standard behaviour on the site is planned, related to the specifics of the online store subject area and the features of the purchase, then the interaction must be carefully checked on the system potential users.
detailed study.
The presence and properties of the Web interface individual components are regulated by the quality
manual for Web interfaces ISO 9241-151: 2008 [
method [
programme interfaces. However, the analogs under consideration may not be ideal, which may lead to decreasing the quality level and complicating the expert assessment procedure.
Ergonomics testing is based on selecting and using many criteria and characteristics, the set of which
can vary for each site. The general set of requirements for the website interfaces is determined by the
standard [
The subjectivity of the criteria for evaluating the interface and interpreting the results of testing
interfaces introduced by decision-takers makes the task of multi-criteria evaluation and choice of the
alternatives urgent. The use of visual analytics methods allows reducing the number of alternatives and
criteria, can increase the objectivity of the proposed assessment methods, as well as the objectivity of
interpreting the study results of the interface ergonomics [
Applying specialized methods of data visualization can also improve the analysis efficiency of large
amounts of data [
Thus, the task arises of creating a methodology for evaluating the interface ergonomics, which will allow determining a set of adequate assessment criteria, objectively evaluate by criteria, clearly present the study results and draw a conclusion about the advantages, disadvantages and directions of improving the interface.
The developed methodology is based on the following formulated principles:
1. The expert assessment and usability testing of the interface should be applied in the design,
starting from developing technical specifications, and then when checking the correctness of the first
and subsequent versions of the prototype, since with each new iteration, fixes in the interface are
more expensive than at the previous one.
2. The biggest problems of analog interfaces or the system being developed can be discovered by
experts involved in the design process. The expert conducts a preliminary check of the interface
compliance with the technical specification requirements and ergonomics criteria. Therefore,
starting from the early design stages, it is advisable to involve experts to evaluate the decisions made
within the developed prototypes. Expert assessment usually precedes usability testing.
3. The procedure for evaluating the interface quality by an expert is heuristic and subjective;
therefore it is necessary to formalize this process in order to ensure the objectivity and evidence of
the expert conclusions with a minimum loss in the examination speed. To ensure the reliability and
consistency of the research findings, it is necessary to provide for a multi-criteria evaluation of the
expert judgment conformity and, if necessary, take measures aimed at increasing the conformity
[16]. In addition, it is advisable for an expert to have visual analytics tools at his disposal.
4. Interface design is an iterative process focused on improving different versions of the interface
prototype by periodic peer review and user testing [17, 18].
5. The effectiveness of the interface solutions of each prototype is checked by means of the interface
expert assessment and / or usability testing [
familiar with the subject area. It is recommended to involve test groups of 4-5 participants. This study is advisable to carry out after eliminating expert comments. A promising area of interface testing is biometric assessment and tracking the user view direction. These techniques make it possible to reveal the user hidden subconscious motives [19].
User interface ergonomics depends on the quality of carrying out individual work within the interface design framework. At the first design stage, the features of the subject area and analog interfaces are studied. Analysing the interfaces of analogous systems involves the expert selection of the evaluation criteria. Given the subjectivity and differences in the expert opinions, this task of multicriteria evaluation will require involving the resources of the software analytical system to select objective criteria and the degree of their importance (the process of “Analysis of opinions”). Then ranking the evaluation criteria is performed and the interface is directly analyzed. The interface analysis can be performed both by an expert and by an analytical system that carries out the analysis on the rule base and the estimation model. Further, according to the analysis results, one obtains numerical and visualized data that reveal the advantages and disadvantages of the interface under consideration. The data received are used to develop the technical specification and interface checklist. After the expert assessment, usability testing of similar interfaces can be used, which helps to identify their objective additional flaws and to confirm the problems found by the experts.
The analysis of user requirements, as a rule, is based on observing user activities according to predeveloped scenarios and questionnaires, allowing identifying goals, objectives, and typical examples of applying the interface.
At the high-level design, the designer arranges navigation links between the interface blocks. Testing users by the classification method of card sorting helps one to determine the navigation links by the user view, to group functions in menus and on separate pages, to determine the product catalogue structure. In this situation, the “Analysis of opinions” block performs the processing and interpreting of statistical information obtained as a result of card sorting.
Low-level design is the stage at which expert assessment involves estimating the quality and identifying shortcomings of the paper, presentation, pseudo-real and real versions of the prototype. The peer review scenario at this stage is similar to studying the analogs, with the exception of its iteration. Evaluating the prototype version interface is carried out within the framework of the previously developed technical task. Visualizing the level of the criteria fulfilment allows one to understand the degree of the project readiness and to determine the places where the interface needs improving. Usability testing allows one to determine the prototype compliance with the technical task requirements and is the main way to determine the ergonomics of the developed interface.
In checking the interface ergonomics, expert assessment takes a dominant position, especially at the initial design stages when developing technical specifications. The mechanism of checklists [20], supplemented by a visual evaluation model, allows having a quick and accurate assessment of the interface ergonomics or its prototype ergonomics. A checklist is a list of requirements for individual properties and interface elements that have their own weighting coefficients. If the requirement is met, its weight is added to the total amount; if it is not met, the weight is not added. The requirements are not always met unambiguously and at a high level, therefore it is proposed to take into account the degree of the requirement fulfilment on a six-point scale (0 – the requirement is not met, 5 – the requirement is met in full). An interface checklist is developed for each project, but a universal basic checklist grounded on the basic ergonomic requirements for such sites can be proposed for designing an online store site.
To evaluate the ergonomics of the graphical user interface of online store sites, it is proposed: 1. To select uniquely defined characteristics (criteria) for evaluating the interface ergonomics. 2. To assign to the ergonomics characteristics j quantitative values of the weighting coefficients determining the importance of the characteristics. The sum of the weighting factors must be equal to 1. 3. To choose a scale for evaluating the ergonomics characteristics (for example, a six-point scale from pj=0 which means that the characteristic is not implemented until pj=5 which means that the characteristic is implemented at the highest level). interface.
Each of the ergonomics criteria is implemented through various characteristics of a variety of
interface elements and interface solutions determined by the user mental model, subject area and
standard [
weighting coefficients is associated with the mutual influence of various ergonomic characteristics on different criteria of ergonomics. For example, fewer data entry errors tend to result in increasing the speed and user higher subjective satisfaction. Due to their versatility, such characteristics can have higher weighting coefficients characterising their importance to ensure the ergonomics of the site user
To test the methodology, the site interfaces of Eldorado [21], M.Video [22] and DNS [23] online stores are analysed. The comparison results are presented in Table 1-4.
To calculate the integral estimation of ergonomics using the formula for determining the ergonomics level of the V interface: = ∑ to use other types of decision rules, as well as their combinations.
Thus, the maximum possible value of user interface ergonomics level is Vmax=5. The ergonomics of any online store interface is based on the following criteria: 14.
kj 0,04 0,03 0,03 0,03 0,03 0,02 0,02 0,02 0,02 0,02 0,015 0,015 0,015
kj 0,035 0,035 0,03 0,03 0,025 0,02 0,015 0,015 0,015
quality of the compared sites. Eldorado website has its interface ergonomics score of VE=3,575. M.Video website shows the second result, that is VM=3,88. DNS site has the best level of the interface ergonomics, that is VD =4,155 among those considered websites.
The integral value of the interface quality level gives us insights into the necessity to improve the site. Making a decision about the characteristics that need to be improved requires clustering, ranking and comparing individual ergonomic characteristics.
ergonomics criteria. A radar chart is chosen as a visual benchmark for comparing multiple interfaces. The choice of radar diagram is due to the fact that the evaluated characteristics are not related to each other, as well as the fact that the area bounded by the diagram line is proportional to the quality level of the interface.
Let's construct separate diagrams for the characteristics of each of the four ergonomics criteria. The quality level of each feature is determined by multiplying the rating of each site by the feature weighting coefficient. As a result, based on the data in tables 1-4, we will obtain the corresponding diagrams for evaluating the ergonomic characteristics of online stores (Figure 2).
a) 13 14 12 15 11 16 10 0,2 0,15 0,1 0,05 0 1
4 5 4 4 4 4 3 2 5 4 4 3 4 4 4 4 5 0,15 0,1 0,05 0 1 pj 3 4 3 4 4 4 4 3 3 4 3 c) 13 12 11 14 0,25 0,2 0,15 0,1 0,05 0 10 7 9 8 3 6 4 5 d) 8 7 9 2 0,2 0,15 0,1 0,05 0 4 3 6 Eldorado DNS 5
maximum quality level (MAX), as well as the area formed by this line, it is possible to draw a conclusion about the level of the interface quality. The closer the points of the broken curve to the centre of the diagram, the lower the overall level of quality of this criterion implementation in the interface of the site in question.
1. Improving the interface of one of the analyzed sites (determining the ergonomic characteristics by which the site lags behind the maximum indicator and the competitors).
those available in analogs).
Let's show by example approaches to solving the problem of improving the site of Eldorado online store. Figure 2a shows that M.Video and DNS sites are leading in terms of user speed. It is advisable to take ergonomic characteristics № 2, 11, 12, 13 of DNS site and № 9 of M.Video site as standards of implementation.
It is necessary to pay attention to the characteristics of the leading interfaces; the points of their diagrams are as distant as possible from the maximum values. They are the best of the considered analogs, but their improvement will provide the most significant gain in ergonomics. Thus, we can recommend to improve and use ergonomic characteristics № 1, 3, 8 of DNS leader site.
The characteristics of Eldorado site № 4, 7, 14 are rated on par with those of the competitors, but they need improving. The characteristics of Eldorado site № 5, 6 10, 12, 16 are maximum and do not need any improvement.
advantages of the similar sites. From the diagram in Figure 2, we can see that it is advisable to take characteristics № 2, 11, 12, 13, 16 of DNS site, characteristics № 5, 6, 10, 12 of Eldorado site and characteristics № 9 of M.Video site as standards of implementation. Then we consider the characteristics of the leading interfaces, as the points of their diagrams are distant from the maximum values. Thus, we can recommend to improve and use ergonomic characteristics № 1, 3, 4, 7, 8, 14 of
Thus, the proposed methodology makes it possible to determine the absolute value of the user interface ergonomics level for the online store website, as well as the relative level of interface quality in the case of comparing several sites. Presenting visual analysis tools obtained in the process of evaluating the ergonomics of the alternative site allows one to involve expert cognitive capabilities in the analysis process and to perceive the solution fully, entirely using all the evaluation criteria proposed in the work. The described technique can be applied at the initial design stages as an express analysis for setting development goals, as well as at the stage of designing and testing interface prototypes to evaluate the decisions made as an addition to usability testing methods and independently of them. With the help of expert assessment according to the methodology proposed in this work, it is possible to quickly determine the strengths and weaknesses of user interfaces, which are advisable to take into account when creating new sites for online stores or improving existing sites.
[16] A. Podvesovskii, O. Mikhaleva, et. al., Model of Control of Expert Estimates Consistency in
Springer, 2017, pp. 361-374. doi: 10.1007/978-3-319-65551-2_26. [17] O.V. Sayapin, O.V. Tikhanychev, S.V. Chiskidov, I.A. Bystrakova, Development of Applied
2020, pp. 47-56. doi: 10.24411/1993-8314-2020-10004. [18] O.V. Tikhanychev, User Interfaces in Automated Systems: Development Problems, in: Software systems and computational methods, Vol. 2, 2019, pp. 11-22. doi: 10.7256/24540714.2019.2.28443. [19] D.A. Kostyuk, O.O. Latiy, A.A. Markina, Approach to Biometric Evaluation of the Graphical User Interface Ergonomics, in: Bulletin of Brest State Technical University. Physics, mathematics, computer science, Vol. 5 (101), 2016, pp. 46-49. [20] D.P. Soroka, Evaluating Method Analysis of the User Interface Ergonomic Indicators In the world of scientific discoveries, No. 10-2 (70), 2015, pp. 894-906. [21] Eldorado.ru, 2021. URL: https://www.eldorado.ru [22] M.Video, 2021. URL: https://www.mvideo.ru [23] DNS, 2021. URL: https://www.dns-shop.ru