User Interface Design Based on Color Schemes of Paintings’ Digital Reproductions Anastasia Mandrik1[0000−0001−8204−7862] , Larisa Sopronenko1[0000−0002−0152−5808] , Nina Rushchenko1[0000−0003−1230−5410] , and Aleksei Lavrov1[0000−0002−5315−8915] ITMO University, Saint-Petersburg, Russia mandrik.anastasi@mail.ru, so lar@mail.ru, rushchenko@mail.ru, lavrov@itmo.ru Abstract. The article discusses the creating of user interface using art works by artists of the post-impressionist era. A perception of colors is investigated. An experiment was conducted on two samples in three steps: collection of data on respondents, including a test to determine deviations in level of color vision, the test for recognition of the author of the image by compositional color scheme, the test for recognition of the author by non-compositional color scheme. A pilot experiment was conducted on 23 respondents, 45 people took part in main experiment. It was found that most respondents, both at the step of recognition by compositional color scheme and at the step of recognition only by non- compositional color scheme, recognize the works of some artists. Accord- ing to results of comparing the data, it was revealed that the number of correctly recognized images depends on the deviations of color vision only when recognized by the non-compositional color scheme. The results of the study are used to create color schemes of user interfaces. Keywords: digital color scheme of the image · color vision · influence of color on perception · cognitive portal. In the era of digitalization and the inevitable transformation of aesthetic values, the perception of art in society goes along the path of generalizing knowledge. Unfortunately, at present, public interest in visual arts is for informational pur- poses only. In arts’ work not only the peculiarities of compositional construction are incorporated, but also color combinations carry most part of information [1]. Aesthetic perception is associated with a person’s view of the world and the ability to see beauty in it, which has a sufficient effect on the attitude and how a person perceives and treates the information that comes to him, a sufficiently large amount of which he receives visually. Therefore, it is important to consider factors that influence the appropriate perception. Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). A. Mandrik et al. Also, don’t forget that the color scheme of the image (the mutual position and the ratio of colors among themselves, the amount of one color or another) carries a kind of message from the author, and at the same time is a holistic and harmonized tool. In addition, the color scheme is unique and individual for each author, it has its own combinations, not dissimilar like others [2, 3]. Color in user interface can control the user’s attention, help to recognize interface elements, express the meaning of indicators in complex professional systems, as well as be used for visual grouping of similar objects [4]. The article [5] discusses the design of human-computer interaction with smartphones, taking into account the color semantics, the use of which in the design of the user interface will better meet the needs of users and improve the user experience. The paper [6] examines the color preferences of mobile device users. In accor- dance with the PCCS color system, three groups of colors (warm, cold, neutral) are defined, which are divided into 18 colors used in the user interface (UI). The authors received various combinations of colors in the UI, which are preferred by users of different gender, age and level of education, etc. In accordance with the research results, the article offers some suggestions about the color design of the UI. The work [4] tested the hypothesis that people see and perceive colors in the image in accordance with their color preferences, which were determined by the Lüscher color test. The study revealed that the dependence of the visibility of the colors in the image on their attractiveness is negligible. The presence of favorite colors in the images also did not affect the preferences of the image as a whole. In our study an experiment was planned and conducted to find out whether the possession of basic knowledge on the history of art and color science, as well as the level of color vision, influence the recognition of authors of paintings of the post-impressionism era by a simplified compositional and non-compositional color schemes. The experiment was carried out on the basis of the works of postimpression- ists artists – Paul Gauguin, Vincent van Gogh, Paul Cezanne, Paul Signac and Georges Seurat. These artists strived in their work not so much to show as to interpret what they saw, using each their own individual compositional and coloristic solution. In particular, Seurat was fascinated by the theory of color and sought to combine creativity with scientific achievements, considered it possible to explain art from a scientific point of view. Post-impressionists believed that colors should not mix on a palette, like academics, and not on canvas, like impressionists, but directly in the face of the viewer, the intended color arises when considering brush strokes of different colors located next to each other. Thus, we can say that color plays an important role in the work of post- impressionists, so reproductions of their works can be used as mainstay mate- rial for the experiment. In addition, open electronic sources contain a sufficient UI Design Based on Color Schemes of Paintings’ Digital Reproductions number of reproductions of paintings by post-impressionists. To study the depen- dence of the recognition of authors on the deviation of color vision, an experiment was planned. A pilot version of the experiment was conducted with participation of second- year students of the educational program ”Computer Graphics and Design” at ITMO University. Such students have basic knowledge of art history, which helps to minimize differences in answers due to unpreparedness. In addition, students studying in this area have basic knowledge of color science. The study included a test of the level of color vision and an experiment, consisting of three steps for ease of data collection and analysis. First of all, a poll was offered, consisting of nine questions about respondents’ personal data (age, gender, educational level), as well as about their knowledge in the field of art history and color science. In the pilot experiment data were obtained from 23 respondents. A basic experiment was conducted with 45 re- spondents. Based on the collected data, diagrams were constructed to show the distribution of data (fig. 1). Pilot experiment data Age Gender 11 1 3 7 16 17 18 19 20 21 26 Women Men Experimental data Age Gender 111 2 3 10 10 21 24 4 11 1 16 18 19 20 21 Women Men 22 23 24 28 38 Fig. 1. Characteristics of the samples A. Mandrik et al. After that, it was necessary to find out the level of color vision of respon- dents for further analysis of experiment results, since the quality of color vision can affect the results of subsequent experiments. The test from the PANTONE Color Institute was conducted on the online platform at the address in the In- ternet: https://www.colormunki.com/game/huetest kiosk for determining color deviation. During the test respondents collect four different color spectra, each of which contains 22 colors (colored tiles). Respondents need to arrange the tiles (interactive elements of a given color) in the right order between two stationary color tiles (the beginning and the end of the spectrum), that is, it is necessary to collect four 22-color sequences. After compiling the spectra, a program analyzes data and presents a numerical indicator of the deviation of color vision. Ideal color vision has zero deviation, accordingly, each incorrectly positioned color in- creases this number. The results of the test for determining deviations of color vision of pilot and basic samples are presented in figure 2. 30 25 12 20 15 10 17 13 14 5 4 1 6 0 2 0 1-10 11-50 51-100 101-1000 Pilot experiment data Experimental data Fig. 2. Color vision deviations of respondents Upon completion of the test, respondents were shown a sequence of repro- ductions of 30 works by post-impressionist artists, with the indicating the author and title (fig. 3). Parameters of reproductions: size 3508x2480 pixels, resolution 300 pixels per inch. A week after the respondents looked at the sequence of illustrations, they were asked to look at a new sequence containing other works by the same authors, written by them at the same time. The works are a simplified compositional and color scheme of the image (fig. 4). Such images include a set of uniformly colored closed outlines, situated correctly relative to the original image composition, in which the using colors number is ten (the most common colors in the original image). Respondents were asked to indicate the author of the picture, the scheme of which is being considered. UI Design Based on Color Schemes of Paintings’ Digital Reproductions Fig. 3. Example of image shown at the first step of the experiment (reproduction of a painting by Paul Gauguin ”When Will You Marry?” with the author’s name and title) Fig. 4. Example of image at the second step (compositional color scheme of a painting by Paul Gauguin ”Tahitian Women on the Beach”) A. Mandrik et al. A percentage of authors recognition and confusion was calculated (fig. 5). According to the results of experiments on recognizing and determining the level of color vision, we can conclude that, in the presence of composition and color, respondents are looking for general patterns of color, composition, and previous experience. In addition, if respondents have deviations in color vision, they tend to rec- ognize images worse if these images are based on the colors of a poorly visible part of the spectrum (for such a respondent). However, it should be noted that the vast majority of respondents have about zero deviations in the field of color vision [7]. 345 295 245 68% 66% 195 145 37% 29% 95 24% 45 0 Vincent van Gogh Paul Gauguin Paul Cezanne Paul Signac Georges Seurat Vincent van Gogh Paul Gauguin Paul Cézanne Paul Signac Georges Seurat difficult to answer Fig. 5. The recognition and confusion of authors by compositional color scheme To answer the question about the recognition of non-compositional color schemes, it was decided to re-test the recognition of authors by non-compositional color schemes of images. Respondents were invited to view a sequence of 30 re- productions of paintings by post-impressionist artists once again, shown in the first stage. Then they needed to see a new sequence containing non-compositional color schemes of the works shown in the second stage of the experiment. Non-compositional color schemes were presented as blocks of ten most com- monly used colors in the original painting of the author (fig. 6). Respondents were required to indicate the author of the picture, which was represented by the shown non-compositional color scheme. UI Design Based on Color Schemes of Paintings’ Digital Reproductions Fig. 6. Example of image shown in the third step (non-compositional color scheme) If there is only a non-compositional color scheme, respondents compare what they saw and only their own previous experience. Similarly to the second stage, the percentage of recognition and confusion of authors by non-compositional color scheme was calculated (fig. 7). 200 150 63% 59% 51% 100 37% 28% 50 0 Vincent van Gogh Paul Gauguin Paul Cezanne Paul Signac Georges Seurat Vincent van Gogh Paul Gauguin Paul Cézanne Paul Signac Georges Seurat difficult to answer Fig. 7. The recognition and confusion of authors by non-compositional color scheme We noticed that at third step respondents who have deviations in color vision will usually recognize less authors than those who have zero deviations. Based on all collected data, it is possible to test the hypothesis that deviations in color vision correlates with the recognition of authors of the works. Spearman’s rank correlation coefficient with two-tailed test of statistical significance was chosen for data analysis. A. Mandrik et al. The first experimental hypothesis: the level of color vision correlates with the recognition of authors of works by compositional color scheme. It has not been confirmed by the experimental data (p-value = 0.14). There may actually be a dependency, but it is not strong enough to make a significance correlation in the sample of this size. The second experimental hypothesis: the level of color vision correlates with the recognition of authors of works by non-compositional color scheme. This hypothesis was confirmed at a confidence level of 95% (p-value = 0.03). It was found out that the percentage of recognition of the authors by non- compositional color scheme and by compositional color scheme is about the same (Fig. 8). 100% 96% 90% 80% 75% 70% 60% 50% 47% 44% 40% 30% 20% 10% 4% 0% 0% compositional color scheme non-compositional color scheme Fig. 8. Comparison of the recognition percentage by two image presentation options As a result of the experiment, it was revealed that the work of artists such as Paul Gauguin and Vincent van Gogh were found by the majority of respon- dents, regardless of the scheme in which the image was presented. The artists Georges Seurat, Paul Cezanne and Paul Signac were not distinguishing by the compositional color scheme, however, when recognizing the author only by the color scheme, Paul Signac is distinguished by the majority of respondents. In the process of analyzing the obtained data, a hypothesis was put forward that respondents, especially those who do not have knowledge of art history, can distinguish between the images themselves, but confuse the names of artists. In addition, by the results of comparing the data obtained, it was found that the number of correctly recognized images depends on color vision deviations only when recognized by the color scheme. UI Design Based on Color Schemes of Paintings’ Digital Reproductions Based on the results gotten, harmonious color schemes of user interfaces for various digital products have been developed, which are improve user experience. A prototype of the user interface of cognitive portal was designed. This cognitive portal is aimed at: – the learning about the cognitive processes (perception of art, as well as general development), – the learning of property of color, like a marker to identify of the artist, – the setting of key points in the design of the interface, – the getting of new knowledge in the field of art history, – and some psychological relax. The key section of the portal is the block using the previously obtained color schemes of artists’ works. So, the user is invited to apply the color schemes of artists of the chosen era to designed user interface. The portal is working with user interfaces for Android, iOS or web applications. Figure 9 shows the interface of the cognitive portal section. On the current screen, the user chooses an artist depending on the previously selected direction. The choice of application type and filling method are also available. Fig. 9. Interface for selection of color solution A. Mandrik et al. The current section is made on the coloring principle, so the user needs to select a color and apply it to any part of the application user interface that was selected from the panel on the right. By default, the user is offered the option of solid filling. However, either in the real world and at the canvases of artists it is rarely possible to find a solid color. Therefore, it was decided to allow the user to choose the fill method. Thus, user can select one of the three available fill options: – solid fill, – transparent gradient, – two-color gradient. A transparent gradient is a color with a consistent change in saturation. A fragment of the palette with a transparent interface is shown in fig. 10. Прозрачный градиент Fig. 10. Fragment of a palette with transparent gradients Two-color gradients combine two colors from the main palette. Among the gradients there are both combinations of slightly different colors and pairs of complementary colors. The most successful pairs of such gradients made up the proposed palette (fig. 11). Двухцветный градиент Fig. 11. Fragment of a palette with two-color gradients Thus, the color schemes of artists used by them in their works can be applied in the currently relevant area of interface design as harmonious color combina- tions. UI Design Based on Color Schemes of Paintings’ Digital Reproductions References 1. Castelhano, M. S., Henderson, J. M. The Influence of Color on the Perception of Scene Gist. In: Journal of Experimental Psychology: pp. 660-675. American Psy- chological Association, USA (2008) 2. Lysova, E. V., Sychkov, F. V. Language of Color. Expressive Possibilities and Speci- ficity of Perception. In: Regionology. No.1. pp. 229-240. (2011) 3. Poleva, N. S. Color as the Language of Painting in the Psychology of Art of N. N. Volkov. In: Cultural-historical psychology. Vol.3, No.2, pp. 93-100. Moscow State University of Psychology and Education, Russia (2007) 4. Shishkin R.V., Lokalov V. A. Factors Influencing the Perception of the Color Struc- ture of a Computer Image. In: Almanac of Scientific Works of Young Scientists of ITMO University. Volume 1. pp. 313-315. ITMO University, St. Petersburg, Russia (2019) 5. Xiaoxiao B., Wenming L. Research on the application of color semantics in the human-computer interaction design of smartphones. In: 6th International Forum on Industrial Design (IFID 2018), MATEC Web Conf (2018) 6. Jin J., Shen H. Research About UI Color Preference for Mobile Terminal Based on the User’s Visual Thinking and Perception. In: Man-Machine-Environment Sys- tem Engineering. MMESE 2018. Lecture Notes in Electrical Engineering, vol 527. Springer, Singapore (2019) 7. Mandrik A. D., Sopronenko L. P. Study of the Influence of Color on the Perception of Works of Art. In: Social Transformations: collection of scientific articles / Vol. 30: The First All-Russian Congress on Color, pp. 110-116. Publishing house of Smolensk State University, Russia (2019)