Introduction

Any design object needs effective presentation, in which structuring of textual and visual information is highly important [1]. Many researchers and designers have been looking for the principles of harmonious organization of compositional elements in architecture and website design. For instance, visual appearance of web user interfaces (UIs) is known to affect behavior of users, and its analysis can help to improve usability and thus increase KPIs of the website, such as e.g. conversion rate [2]. The visual complexity assessment helps to identify and describe problems in the website UI. Visual complexity is affected by the number of elements in an object or image, their structural relations, the detail of the information that these elements provide, etc. [3]. It has been scientifically proven that aesthetic preferences for the visual complexity of web pages are influenced by users' age and previous experience [4]. However, our article focuses on the dependence of visual complexity in web UI screenshots: namely the common compositional elements in web pages (buttons, texts, lists, etc.), as well as multimedia elements (images, videos, etc.).

The identification of UI elements in a website page screenshot for further assessment of visual complexity can be obtained through either manual labeling or automated recognition process [5]. Automation of any process makes it possible to simplify it and helps to free a person from routine and tedious tasks, but often it involves additional costs and resources (time, labor, etc.), especially at the initial stage. Table 1 shows a comparison of the automatic and manual methods with respect to UI labeling.

Thus, the purpose of the current work is to determine the types of elements that affect the subjective perception of websites, as well as to compare the models built with the factors' values obtained via automated vs. manual labeling of web UI screenshots.

The Study Description 2.1. The Manual Labeling

In the experiment, the subjects were offered about 500 website interface screenshots and asked to label UI elements in them: highlight the element in a box and identify the elements' type. They were using a dedicated software tool, LabelImg (see in Fig. 1). In total, 11 human labelers took part in this activity, after providing informed consent.

The Automated Labeling

In addition to the manual labeling of the screenshots, we also performed their automated analysis, using our dedicated Visual Analyzer (VA) software tool, available at http://va.wuikb.info/ and described in detail in [6]. It identifies UI elements in images based on previously trained ML models (see in Fig. 2), but our previous studies suggest that its accuracy is somehow deficient, particularly in determining the type of each UI element. In Table 2 we show the types of UI elements that had been identified in manual and automated labeling that we performed. As one can seen from it, the main difference between the two methods lies in determining the types of elements. In the manual labeling, the participants were able to rather successfully identify 26 different UI types, while for the automated labeling, there were 8 types, resulting from the pre-trained ML models. In addition, due to high visual diversity of today's web designs, the accuracy of the automated type detection was the most problematic dimension in the semantic-spatial analysis that our VA tool performed (see [6] for more detail).

From the labeling datasets, the heights and widths of the elements were calculated (also by the elements' types: text paragraphs, buttons, images, panels, etc.), and there was also the data on the height and width of the screenshot. Based on this information, areas of each UI he element was calculated, as well as the share occupied by this element in the total screenshot space (by the elements' types: texts, images, background, etc.).

Thus, there were 2 groups of factors in the behavior models that we further constructed for complexity, aesthetics and orderliness that were the dependent variables in the study:

1) the number of elements: the number of elements of a certain type located in one screenshot; 2) the proportions: the shares of the total area of the elements' types to the total area of the screenshot.

The Subjective Perception Evaluation

For each screenshot, we also had evaluations of complexity, orderliness and aesthetics, provided by another 137 participants (67 female, 70 male). The majority of them were Russians (89.1%), while the rest were from Bulgaria, Germany, South Africa, etc. More details on the participants and the procedure can be found in one of our previous works [7].

Each of the subjective perception dimensions was assessed on a scale from 1 to 10. Then the average values for each indicator were found using formulas (1)-( 3), where i is the number of web pages, n j is the number of participants who provided the evaluations for the i-th website.

∑

where indicator of the aesthetics of the interface of the i-th web page, estimation of the aesthetics of the interface of the i-th web page ∑

where indicator of the complexity of the interface of the i-th web page, estimation of the complexity of the interface of the i-th web page ∑

where indicator of the ordering of the interface of the i-th web page, estimation of the ordering of the interface of the i-th web page.

Complexity is the number of elements on the screen and their arrangement. Orderliness is an ordered data structure of the web interface that allows the user to easily find the information they need. Aesthetics is an assessment of the attractiveness of a product by the user. This indicator is important because the aesthetics of any web interface has a strong impact on the user, even when he tries to evaluate the functionality of the system [8].

Results

Using the SPSS statistical analysis software, we build linear regression user behavior models (as representing the most universal ML method) with the factors resulting from the manual and the automated labeling. The dependent variables in the models were the three subjective perception evaluation scores, while the independent variables were the factors resulting from the two labeling processes. The step-by-step method of the analysis allowed stepwise inclusion of the factors in the models, thereby discarding those that did not make a significant contribution to explaining the dependent variables.

The results of the regression analysis are presented in Tables 3-5, each corresponding to a different subjective perception dimension. The null hypothesis H 0 was as usual in the regression analysis, that the regression equation is not significant. For each of the dependent variable, the models turned out to be significant (p < 0.05), while the significances for the factors selected by the step-by-step method are shown in the respective columns of the tables. As one can note from Table 3, the perceived complexity of websites was most influenced by the amount of text, the number of radio buttons, the number of buttons and tabs. The subjective orderliness (Table 4) was most influenced by the proportion of radio buttons and buttons. The perception of aesthetics (Table 5) was most influenced by the proportion and amount of text on the page, the number of buttons, labels and images. Despite the fact that the constructed models had rather low determination coefficients (R 2 ), it is still possible to draw a general conclusion about which elements affect the assessment of website perception among users, and whether the degree of this dependence is influenced by the labeling method (Table 6).

As one can see from Table 6, the share of text on the page was significant for all the three subjective perception dimensions. Moreover, the presence of the text had the greatest impact on the assessment of complexity and aesthetics. At the same time, the obtained values for the automated and the manual labeling did not differ much (4-10%), but with the automated labeling, the models' quality indexes, as represented by R 2 s, were somehow superior. the share of the text area, text style (font, size, etc.), and the presence of pictures that dilute the text, and many other factors are also important. The number of images, their size, quality and position on the web page affect the overall website subjective perception, including the assessment of the aesthetics and orderliness. As a general rule, these indicators are interchangeable: the lack of text is often compensated by a variety of graphic elements and images. Therefore, in order for web UI to comply with usability standards and to be simpler and more understandable for its users, it is necessary not to clutter the interface with a large number of elements and break long texts into smaller parts. At the same time, the number of element types did not significantly affect any of the subjective perception dimensions.

The quality of the user behavior models built on the factors resulting from the automated labeling was slightly higher than for the manual one, which suggests feasibility of our UI visual analysis tool. In general, this may indicate that the automation of the labeling process makes sense, but it requires high costs and the presence of certain knowledge to implement it. After all, our VA software was initially trained with the data once provided by human labelers [6].

It should be noted that the constructed linear models have low R 2 coefficients, so they should not be used in production. The goal of our current study was merely to compare the two labeling methods, while for real user behavior modeling, more advanced ML methods and architectures should be used.

Our plans for further research include investigation of the effects of web page layouts (the size of the element, its type and occupied area) on the subjective assessment of the perception of the site, but also the colors, fonts, types of buttons, animations, etc.