Learning System User Interface Preferences: An Exploratory Survey Timo Hynninen Abstract Antti Knutas User experience is a key aspect when designing a Arash Hajikhani software product. This applies especially when the use Lappeenranta University of of the service requires a cognitive load from the user, Technology such as in online learning systems. In this paper, we Lappeenranta, Finland present initial results that can serve as source material timo.hynninen@lut.fi for creating preference profiles for users, based on their antti.knutas@lut.fi personal information and teamwork preferences to arash.hajikhani@lut.fi enhance usability aspects of software systems. Based on our studies on student behavior during a learning Jussi Kasurinen experience, we present the plan for a solution which South-Eastern Finland University combines these two approaches. of Applied Sciences Kotka, Finland Author Keywords jussi.kasurinen@xamk.fi User interface; adaptive systems; survey; Belbin; Yee; human factors; learning system; profiling; clustering. ACM Classification Keywords Copyright is held by the author/owner(s). H.1.2. User/Machine Systems: Human factors. CHItaly ’17, September 18-20, 2017, Cagliari, Italy. H.5.2. User Interfaces: Evaluation/Methodology. H.5.2. User Interfaces: Benchmarking. Introduction User experience is a key aspect when designing a product or a service aimed towards the general population. In this context, the user experience means both the usability and learnability of the user interface, and the provided satisfaction and emotional impact from being exposed to the contents of the service [11]. To realize these research goals we conducted an Often the primary aim in design is to maximize the exploratory empirical usability survey with several usability and the user experience aspects to the best different basic user interface components that could be possible degree. However, the problem of this approach featured in online learning systems, such as Moodle 1. is that the people tend to like different things, and We formed a focus group consisting of volunteer generally behave differently from each other. To university students, and asked the participants to mitigate this problem in the user interface design, individually evaluate the different user interface many services allow users to modify the interface to elements according to their own preferences. During their liking. However, in many services the customers the same study, the participants were also asked to fill may not even be aware of all possible modifications out teamwork and online game preference profile that can be done [3]. Some users consider usability questionnaires to examine if their preferences could be tutorials, mentors or mandatory visits to see the help matched to the motivational aspects of using an online systems irritating, or simply lack the computer skills to system, or preferred working styles. independently learn to use anything more complicated Figure 1: A set of examples of than the simple web services [4]. Research Setup the UI elements: Editor with To understand what fundamental user interface several support views (top), and At the same time, there are studies which show that solutions our test group liked and used, we created a Editor with one support view in the adaptive UI generation is possible [9], and various test with 153 test cases of different types of user window (bottom). approaches towards this objective have been studied interface solutions, layouts, input devices, elements [10]. Previous research has also established that the and color schemes. users of online systems can be profiled [1], and these profiles can be used to create customized approaches The concept was to measure the usefulness, efficiency to enhance user experience [8] in computer-supported and likeability of the different user interface schemes, learning. The possibility of creating an adaptive following the usability categories by Rubin [11]. The interface raises questions: Which kind of interface participants were requested to evaluate the user elements should the designers try present to the users? interface elements, and grade them for efficiency, Which kind of approaches should be presented to the likeability and usefulness. After this, the participants users from a multitude of options? were showed two user interface elements from which to choose the one they preferred. The element of To summarize, the main research questions in this learnability was not measured, since our test set was study are: composed of images depicting different elements and layouts as illustrated in figure 1. The test image set and 1. Are there distinct preferences for certain interface types, and 2. Are there distinct clusters of interaction preferences amongst the users? 1 https://moodle.org/ Item Avg Std survey materials can be downloaded from the online results that were high on average. The results of the Mouse and 4.55 0.66 appendix 2. two tables correlate highly. The respondents seem to Keyboard think that traditional input methods such as the mouse Mouse 4.32 0.86 The data was collected in controlled sessions from the and keyboard are usable and efficient. Desktop Keyboard 4.19 0.86 volunteers. In total, we collected profiles from 31 computer was perceived to be more efficient as an participants. The goal in analyzing the data was to input method, but on the other hand laptop was Laptop 3.97 0.93 establish whether there are varying user interface considered to be more usable overall. On the UI design Table 1: Items where perceived preferences for different types of users. To establish selections, the traditional tools such as hyperlinked usability was high. the user profiles for the participating volunteers, they text, desktop icons and a selection of the simplified were asked to fill questionnaires for two profiling versions of tools (login, search tool) reached universal methods: The Belbin teamwork profile [5] and Yee's high scores. Table 3 presents a list of detected clusters Item Avg Std online game motivations [13]. The results were and their centroids. The table cells have been colored analyzed to discover any recurring patterns between for clarity. Red indicates large values and blue indicates Mouse and 4.65 0.60 Keyboard the respondents with the k-means clustering algorithm, low values. Column C1 presents values present in the which is a statistical analysis method for automatically first cluster, column C2 in the second cluster and Keyboard 4.61 0.61 partitioning a dataset into a specified number of groups column Dev is the calculated difference between the Desktop 4.58 0.71 [7,12]. two columns. Mouse 4.35 0.74 Hyperlinks 4.23 0.91 Results and Implications When investigating the cluster values, the clearest line Simple 4.23 1.01 First, the results were sorted to discover universal high of division is between respondents who rated Login-screen scores from the data. The usability aspects of the themselves motivated in play and respondents who did Desktop 4.26 1.29 different layouts, input devices and designs did not not feel highly motivated to play in any category. The icons include any surprises; Table 1 presents the results that division can be seen clearly in three last rows in Table Simple 4.06 1.13 were high on average. Results are presented on a 3. Respondents who had high motivations regarding search range of 0 to 5. For using any system, the most gameplay (C1) were also biased towards Belbin universal high scores were given to the mouse and coordinator, shaper and resource investigator profiles Table 2: Items where perceived keyboard, and laptop system. Any other UI compared to the other cluster. On the other hand, efficiency was high. arrangement (different touch screen layouts, pens, respondents who were not motivated to play games tablets, OS styles) did not reach high average without (C2) had higher Belbin profile preferences in significant deviation. implementer, team worker and complete finisher. It should be noted that the cluster C2 was still slightly The respondents were also queried on the perceived interested in the social aspects of gameplay. efficiency of the input methods. Table 2 presents the 2 http://www2.it.lut.fi/GRIP/datatools/UI- images/UIelements_Cyberlab.zip Profile category C1 C2 Dev layouts with minimal amount of added views or Cluster Analysis Results Implementer 0,35 0,46 0,11 elements. Also, plain presentation of the data was and Validity Evaluation Coordinator 0,45 0,29 0,16 usually considered more likeable and efficient; users strongly preferred simple dropdown menus over Shaper 0,54 0,48 0,06 context-classified, and simple text over hypertext. In K-means clustering analysis Plant 0,41 0,42 0,01 general, most of the universal low scores in usability resulted in two clusters of Resource Investigator 0,46 0,30 0,16 represented complex, clustered or item-saturated user student profiles that share Monitor Evaluator 0,41 0,40 0,01 interface views. same preference sets in Belbin and Yee tests. The Team Worker 0,34 0,43 0,09 Complete Finisher 0,34 0,44 0,10 A threat to the validity of this study is the overfitting of average silhouette coefficient the target population. Our method of collecting answers for combined profile clusters Specialist 0,39 0,37 0,02 steered the system towards 20-35 year old, educated for Yee and Belbin combined Yee: Achievement 0,70 0,07 0,62 and technologically savvy audiences, since we collected is 0.45. This is at the same Yee: Social 0,58 0,20 0,37 most of our data from the university and college. level of clustering as Yee: Immersion 0,58 0,06 0,52 Acknowledging this limitation, we intend to diversify individual Yee (0.46) or our sample population in the further studies with actual individual Belbin profile Table 3: List of detected clusters and their centroids. test scenarios, where the learnability and intuitivism clusters (0.46), The silhouette value of 0.45 Discussion and Conclusion have larger impact on the results. In this paper we discussed the applicability of Belbin [5] means that the data point and Yee [13] profiles to usability preferences and The presented empirical results provide valuable data cluster is medium-weak, with performed an exploratory survey of a learning system for interface design decision-making, for example in the value of 0.51 being a limit to user preferences. Our study indicates that the users model based approaches [10] or adaptive a medium coverage cluster can be profiled and these profiles can be used to tailor recommender systems [6]. The presented approach [7]. user interfaces. Also, per the survey we observed that has potential to provide a method for providing there are clear differences in the user preferences datasets for adaptive interface systems, like the one between the different types of interfaces, with presented by Ahmed et al. [2]. The initial analysis and traditional input methods being preferred the most. small-scale tests indicate, that this approach could be feasible, since there are differences in the preferences The analysis of the data showed meaningful patterns between the different user groups, especially when that can be used to divide users into distinct types. observing the motivational aspects. There is also a clear order of respondent preferences in perceived efficiency and usability of inputs. As for future work our intention is to continue with this Additionally, the k-means clustering produced two concept, and extend the profiling test into a complete geometrically distinct groups in Belbin and Yee profile test setting with usability-related case activities and results. Especially in mobile platform layouts the more detailed user profiling. respondents seemed to prefer simple, systematic References Gamification Design. International Journal of 1. Gediminas Adomavicius and Alexander Tuzhilin. Human Capital and Information Technology 1999. User profiling in personalization applications Professionals (IJHCITP) 7, 3: 47–62. through rule discovery and validation. In 9. Vìctor López-Jaquero, Francisco Montero, Antonio Proceedings of the fifth ACM SIGKDD international Fernández-Caballero, and Marìa D. Lozano. 2004. conference on Knowledge discovery and data Towards Adaptive User Interfaces Generation. In mining, 377–381. Enterprise Information Systems V, Olivier Camp, 2. Ejaz Ahmed, Nik Bessis, and Yong Yue. 2010. Joaquim B. L. Filipe, Slimane Hammoudi and Mario Customizing interactive patient’s diagnosis user Piattini (eds.). Springer Netherlands, 226–232. interface. In Digital Information Management 10. Vivian Genaro Motti, Dave Raggett, and Jean (ICDIM), 2010 Fifth International Conference on, Vanderdonckt. 2013. Current Practices on Model- 536–539. based Context-aware Adaptation. In CASFE, 17–23. 3. Pierre A. Akiki, Arosha K. Bandara, and Yijun Yu. 11. Jeffrey Rubin and Dana Chisnell. 2008. Handbook 2013. RBUIS: simplifying enterprise application of usability testing: how to plan, design and user interfaces through engineering role-based conduct effective tests. John Wiley & Sons. adaptive behavior. In Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive 12. Kiri Wagstaff, Claire Cardie, Seth Rogers, Stefan computing systems, 3–12. Schrödl, and others. 2001. Constrained k-means clustering with background knowledge. In ICML, 4. Vânia Paula de Almeida Neris and M. Cecília C. 577–584. Baranauskas. 2010. Making interactive systems more flexible: an approach based on users’ 13. Nick Yee. 2006. Motivations for play in online participation and norms. In Proceedings of the IX games. CyberPsychology & behavior 9, 6: 772– Symposium on Human Factors in Computing 775. Systems, 101–110. 5. R. Meredith Belbin. 2012. Team roles at work. Routledge. 6. Catherine Inibhunu, Scott Langevin, Scott Ralph, Nathan Kronefeld, Harold Soh, Greg A. Jamieson, Scott Sanner, Sean W. Kortschot, Chelsea Carrasco, and Madeleine White. 2016. Adapting level of detail in user interfaces for Cybersecurity operations. In Resilience Week (RWS), 2016, 13– 16. 7. Leonard Kaufman and Peter J. Rousseeuw. 2009. Finding groups in data: an introduction to cluster analysis. John Wiley & Sons. 8. Antti Knutas, Jouni Ikonen, Dario Maggiorini, Laura Ripamonti, and Jari Porras. 2016. Creating Student Interaction Profiles for Adaptive Collaboration