In recent years, an increasing number of software systems offer users the ability to switch from the more conventional light color background graphical user interface (GUI) to the dark mode. In this mode, black, dark grey, or other similar colors are used for background, while text and other interface objects are typically displayed in white or other light colors. However, the impact of the mode on user experience and usability of systems is still under-researched, meaning that the dark mode is often implemented without knowing if it won't cause unwanted effects. This is especially important on the web, where GUI design trends are particularly dynamic, while interface changes can be implemented relatively quickly and completely onesidedly by system owners. In this paper, preliminary results of the study aimed at determining the connections between GUI mode and web system usage efficiency or productivity are presented. The study was conducted in the form of an experimental survey and provided valuable insights that could be valuable to GUI developers and web system owners when making informed decisions on usage of dark and/or light color modes.
2023 Copyright for this paper by its authors. CEUR
ceur-ws.org
This way, two characteristics of systems usability – usage efficiency (or productivity) and aesthetics – were addressed. Most importantly, the study attempted identifying potential links between color mode, work efficiency, and such parameters as system user age, devices used, and web browsing experience. The findings serve as the basis for GUI development guidelines that could be relevant to system developers and owners looking to better understand and/or optimize efficiency and aesthetic appeal of their systems by making informed decisions regarding the GUI mode.
The remainder of this paper is structured as follows. Section 2 presents an overview of main known studies in relationships between GUI modes and usability of systems. Plan of the study, main hypotheses, structure of the experimental survey and tool used are discussed in Section 3. The results of the survey as well as key preliminary findings with regards to the outlined hypotheses are given in Section 4, followed by conclusions and directions for future work.
Research on the impact of dark and light graphical user interface modes on system usability has been ongoing for several years. Many of the studies focus on psychological impact of different color modes, but as usability is a multifaceted concept encompassing learnability, efficiency, aesthetics, and other characteristics, such findings are also relevant to systems requirements analysts and GUI designers.
In a study by Koning and Junger it was hypothesized that a dark user interface could lead to more
dishonest behavior due to the perceived anonymity of the user [
Nazeriha and Jonsson [
Emotional and semantic effects of color on digital content was also investigated by Löffler et al.
who attempted understanding the influence of black, white, and gray user interface backgrounds on the
perception of sentiment in chat messages on Twitch.tv [
Much of the work done on modern systems involves text. Therefore, researchers have been trying
to understand how color mode influences the ease with which the user can understand it or its
readability. Nyqvist and Rutqvist [
We think it is especially important to understand whether the color mode influences productivity of
users. Pedersen et al. addressed this question in a study involving a controlled experiment with a visually
intensive text entry task using a virtual keyboard with an unfamiliar layout [
Results of several studies have been summarized in a report by Nielsen Norman Group [
Overall, these studies highlight the complex relationship between the color theme and user behavior, trust, readability, efficiency, and productivity. While there is no clear consensus on whether dark or light mode is better, there is enough ground to conclude that the choice of mode does influence user experience. It is, however, not entirely clear how exactly it impacts various aspects of usability, including speed of work (efficiency), and how that impact, if there is any, depends on the characteristics of users and nature of tasks.
The intended plan for conducting research consisted of the following stages: 1. Formulation of hypotheses and definition of variables to be measured. 2. Defining the structure of the study. It can immediately be noted that, based on the hypothesis defined, mixed survey was chosen as the experiment method. The remaining steps were as follows: 1.1. Defining a list of experimental tasks required to measure usability characteristics outlined in the hypotheses. 1.2. Defining the questions that would be posed to study participants. 1.3. Choosing a survey system. 1.4. Creating and implementing GUI elements (i.e., fully functional pages) required to perform the defined experimental tasks.
1.5. Implementing the experimental survey on a chosen system. 3. Publishing the survey, sending out participation requests, and data collection. 4. Analysis and summarization of results. 3.2.
In this study, the main goal was to measure usability in terms of efficiency of work in both dark and light mode on non-leisure related websites. The key variable determining efficiency was speed of performing typical tasks (correctly!). In addition to that, we were interested in whether the efficiency of dark and light GUI modes would be dependent on user characteristics. User age, web browsing experience, color mode preference, and device type used were chosen as variables that could influence efficiency. Based on these decisions, five core hypotheses were formulated and their generalized expressions are given below. Note that in all cases the goal is to look for dependencies. However, in this paper we focus on the preliminary findings. Therefore, presented hypotheses are simplified versions of statistical dependency hypotheses. Consequently, statistical findings given below were limited to basic data necessary to make preliminary observations.
H1. Color mode (M1, M2) affects the speed of working with the system (W*).
H2. Color mode’s (M1, M2) influence on the speed of working with the system (W*) depends on the age of the user (A).
H3. Color mode’s (M1, M2) influence on the speed of working with the system (W*) depends on the user’s browsing experience (E).
H4. Color mode’s (M1, M2) influence on the speed of working with the system (W*) depends on the type of device used (D).
H5. Color mode’s (M1, M2) influence on the speed of working with the system (W*) depends on the user’s color mode preference (P).
Here, task performance speed (W*) refers to the time it takes for a user to complete a task within a system or interface. Two typical tasks were chosen for this study: finding an object in a list given on screen W1 (e.g., finding a product in an e-store catalogue), and filling in a complex form W2 (e.g., registering on the system or submitting order data).
In addition to the above, we wanted to compare user preferences to their opinion on whether they thought websites should have dark mode available or not. Finally, any thoughts participants had on this topic were also to be collected.
3.3.
The study was defined as a mixed anonymous survey, following the steps outlined in Section 3.1. The survey questions and possible answers are presented in Figure 1. To simulate realistic work with web information systems, clothes e-shopping scenario was chosen as an experimental domain. A full product purchase process was to be simulated. However, key measured efficiency criteria concerned measuring the speed of performing two tasks as defined in Section 3.2: W1 – discoverability of information; finding the white V-neck t-shirt in the catalogue of products (see Figure 2 for pages created for this task); W2 – form filling efficiency; filling in the order requisites with the given data (see Figure 3).
In order not to give away the objectives of the study, questions 4, 5, and 6 were to be presented after the tasks have been carried out by the participant. Control questions on participant’s age and web browsing experience were posed before the tasks. Each participant was to carry out the experimental tasks only once, on either all-dark, or all-light version of the experimental site.
As a platform for the experiment, a wgsite testing tool provided by E-bros company was chosen. To accommodate the needs of this study, the tool was extended with a new configurable usability study module. The module's primary objective was to gather data on participant performance and behavior during a given task by recording timestamps for each stage of the task.
The entire survey, including experimental pages, was implemented in both Lithuanian and in English with participants free to switch to the language of their choice. To ensure unbiased results, the system presented both GUI designs (dark and light mode) to a similar number of participants. Black-white and white-black background and foreground colors were used for dark and light versions of the experimental site. Multiple participations from the same IP address were not allowed.
Note that the system recorded all time stamps of each participant, but for the purpose of this paper we only consider times it took users to carry out tasks W1 and W2.
Study invitation was disseminated on Reddit (several UI/UX related groups), Facebook (several survey related groups), also on the personal ResearchGate account of one of the authors as well as among a second- and third-year students at Kaunas University of Technology enrolled in a GUI related course. Invitation did not include any hints on what exactly the object of the study was.
At the time of writing this paper, the data from more than 70 participants has been recorded, but incomplete and unreliable results were excluded from the analysis. As a result, a total of 62 responses were included in the calculations presented below. 53% of the participants received the dark mode variant while 47% received the light mode.
As depicted in Figure 4, out of the 63 participants, 55% used smartphones while the remaining 45% used computers during the experiment. Overall, there were 15 participants who were presented with a light mode version of experimental site on smartphones, 18 who got light mode on computers, 10 – dark mode on smartphones, and 19 – dark mode on computers. In terms of age, almost 68% of the participants were under 32 years old (see Figure 5).
20 10 0
Similarly, majority of participants reported high web browsing experience (63% browsed for more than 2 h/day), and only one participant had relatively title experience (less than 0.5 h/day) (see Figure 6). Finally, participants were also asked to indicate which graphical interface mode they preferred the most. Most indicated they were fine with whatever was the default mode of the system they were using, and 29% expressed preference towards the dark mode. Table 1 (average time taken in seconds and average standard deviation (SD)). The table does not include results based on participant age as they were analyzed in more detail. Below, all of these results are overviewed from the perspective of hypotheses H1-5.
Hypothesis H1. The average product opening time when using the light mode was 2.25 s (8.2 %) faster than in the dark mode, and the average form filling time was 4.59 s (4 %) faster (Figure 7). These results provide weak support to the hypothesis H1 as tasks were completed faster in light mode.
31 24 Can't answer 28.44
Hypothesis H2. There was no significant impact of age on product opening time in either of the two modes, as indicated by a Pearson correlation coefficient of -0.096 in dark mode (Figure 8) and -0.17 in light mode (Figure 9). The Pearson correlation coefficient is a measure of the linear relationship between two variables, ranging from -1 (perfect negative correlation) to 1 (perfect positive correlation), with 0 indicating no correlation. In this case, the coefficients indicate a weak negative correlation between age and product opening time in dark and light modes, suggesting that there is no meaningful dependency between these variables. Speaking of form filling task W2, no correlation between time taken and age was observed in the light mode condition (correlation 0.0306, Figure 11). However, in the dark mode condition correlation was 0.508 (Figure 10), indicating moderately strong positive correlation between age and form filling time in the given context. In other words, as age increased, the time taken to fill out the form also tended to increase. These findings support rejecting hypothesis H2 in all but form filling in the dark mode scenario. More research is required to test this seemingly negative effect dark mode had on older users further.
114,59 110,00 31
28,45
Light mode 0 10 20 50 60
70 30
40 Age 400 )300 s (e200 m iT100 0
Age
R² = 0,0009 0 10 20 30 40 50 60 70
Hypothesis H3. Preliminary results expressed in average time taken indicate that product was found and opened faster by more experienced participants (Figure 12), the product detection and opening time was faster among more experienced participants in both GUI conditions. In dark mode, the difference in product opening times between experienced and non-experienced participants was 4.54 s indicating that those who spent more time browsing could spot and open the product 13.2% faster. In light mode, the difference was 1.66 s (5.63%). Less experienced participants also took 21.81% longer on average to fill out the form in the dark mode condition. However, in the light mode group, less experienced users actually took slightly less time than more experienced ones. These results confirm the hypothesis H3 indicating that user browsing experience (E) affects the speed of working with the system in both tested color modes, but further research to understand the specific influence the experience has in different scenarios is needed.
)50,00 s ( e iTm0,00
Product opening W1
Hypothesis H4. The average product opening time clearly differed between different devices, as shown in Figure 13. On smartphones, in dark mode, the product's opening time was 1.56 times faster than in light mode, with an average time of 21.8 s compared to 34.13 s in light mode. However, for computer users, product spotting and opening task took significantly less time on average to complete than in light mode. Form filling in dark mode was slightly faster on average than in light mode on computers, while on smartphones the form filling average time in dark mode was 20.1% longer. These findings support hypothesis H4 and it can be preliminarily confirmed that the effect color mode has on work efficiency depends on the type of device used.
Hypothesis H5. Preliminary results given in Figure 14 confirm this hypothesis and show that the preferred mode has an impact on which color mode will be most efficient. Participants who prefer dark mode spotted and opened the product faster in this mode than respondents who prefer light or default mode. However, the form filling task was completed faster by participants who preferred the default mode in both conditions. Interestingly, participants who prefer dark mode took the longest on average to complete the form filling task in this mode and longer than when working in light mode. Same effect was observed with those favoring light mode: they took the longest to complete task W2 in this mode and were more efficient in the dark mode condition.
Speaking of the limitations that could have compromised the results, it is important to point out the uneven distribution by age among the participants. The study was also performed in a non-laboratory environment, so there could have been unknown factors influencing efficiency. Also, the study focused on testing website usage, so there is no certainty these findings would apply to desktop applications.
Although preliminary results of the study indicated no significant or meaningful difference in overall task completion time between dark and light mode, when other variables were considered, certain trends and dependencies emerged. All five formulated hypotheses were confirmed (to a varying degree), indicating that the software GUI color mode(s) should be selected based on the typical user age, browsing experience, device type, and color mode preference.
Further, studies should be conducted to verify these preliminary results and investigate other factors that may affect the speed of working with the system. For example, studies could explore the impact of color contrast, font size, and screen resolution on different user groups. Task-specific investigations should also be carried out to explore other scenarios, such as data entry, navigation, and content consumption. Sample size should also be extended. This could help to identify more specific patterns and dependencies between user characteristics and preferable GUI color modes. In addition, future research could explore additional factors that could affect the speed of working with the system, such as cognitive load, motivation, and emotional state.
Overall, the study highlights the importance of considering the user's individual characteristics and preferences when designing UI modes to optimize their work speed and efficiency with the system. The findings of this research have practical implications for designers and developers of digital software systems. By considering the user's characteristics and device usage when choosing most suitable GUI color mode(s), they can improve user experience and work efficiency.
We would like to acknowledge E-bros company for allowing us to implement additional functionality in their in-house testing system. Their support and cooperation have been crucial in making this study possible. 7. References