Learning System User Interface Preferences: An Exploratory Survey

Abstract

User experience is a key aspect when designing a software product. This applies especially when the use of the service requires a cognitive load from the user, such as in online learning systems. In this paper, we present initial results that can serve as source material for creating preference profiles for users, based on their personal information and teamwork preferences to enhance usability aspects of software systems. Based on our studies on student behavior during a learning experience, we present the plan for a solution which combines these two approaches.

-

from being exposed to the contents of the service [ 11 ]. Often the primary aim in design is to maximize the usability and the user experience aspects to the best possible degree. However, the problem of this approach is that the people tend to like different things, and generally behave differently from each other. To mitigate this problem in the user interface design, many services allow users to modify the interface to their liking. However, in many services the customers may not even be aware of all possible modifications that can be done [ 3 ]. Some users consider usability tutorials, mentors or mandatory visits to see the help systems irritating, or simply lack the computer skills to independently learn to use anything more complicated than the simple web services [ 4 ].

At the same time, there are studies which show that the adaptive UI generation is possible [ 9 ], and various approaches towards this objective have been studied [ 10 ]. Previous research has also established that the users of online systems can be profiled [1], and these profiles can be used to create customized approaches to enhance user experience [ 8 ] in computer-supported learning. The possibility of creating an adaptive interface raises questions: Which kind of interface elements should the designers try present to the users? Which kind of approaches should be presented to the users from a multitude of options? To summarize, the main research questions in this study are: 1. Are there distinct preferences for certain

interface types, and 2. Are there distinct clusters of interaction preferences amongst the users? To realize these research goals we conducted an exploratory empirical usability survey with several different basic user interface components that could be featured in online learning systems, such as Moodle 1. We formed a focus group consisting of volunteer university students, and asked the participants to individually evaluate the different user interface elements according to their own preferences. During the same study, the participants were also asked to fill out teamwork and online game preference profile questionnaires to examine if their preferences could be matched to the motivational aspects of using an online system, or preferred working styles.

Research Setup

To understand what fundamental user interface solutions our test group liked and used, we created a test with 153 test cases of different types of user interface solutions, layouts, input devices, elements and color schemes.

The concept was to measure the usefulness, efficiency and likeability of the different user interface schemes, following the usability categories by Rubin [ 11 ]. The participants were requested to evaluate the user interface elements, and grade them for efficiency, likeability and usefulness. After this, the participants were showed two user interface elements from which to choose the one they preferred. The element of learnability was not measured, since our test set was composed of images depicting different elements and layouts as illustrated in figure 1. The test image set and 1 https://moodle.org/

Item

Mouse and Keyboard Keyboard Desktop

Mouse Hyperlinks

Simple Login-screen

Desktop icons Simple search 4.32 4.19 3.97 0.86 0.86 0.93 Avg 4.65 4.61 4.58 4.35 4.23 4.23 4.26 4.06 survey materials can be downloaded from the online appendix 2.

The data was collected in controlled sessions from the volunteers. In total, we collected profiles from 31 participants. The goal in analyzing the data was to establish whether there are varying user interface preferences for different types of users. To establish the user profiles for the participating volunteers, they were asked to fill questionnaires for two profiling methods: The Belbin teamwork profile [ 5 ] and Yee's online game motivations [ 13 ]. The results were analyzed to discover any recurring patterns between the respondents with the k-means clustering algorithm, which is a statistical analysis method for automatically partitioning a dataset into a specified number of groups [ 7,12 ].

Results and Implications

First, the results were sorted to discover universal high scores from the data. The usability aspects of the different layouts, input devices and designs did not include any surprises; Table 1 presents the results that were high on average. Results are presented on a range of 0 to 5. For using any system, the most universal high scores were given to the mouse and keyboard, and laptop system. Any other UI arrangement (different touch screen layouts, pens, tablets, OS styles) did not reach high average without significant deviation.

The respondents were also queried on the perceived efficiency of the input methods. Table 2 presents the 2 http://www2.it.lut.fi/GRIP/datatools/UIimages/UIelements_Cyberlab.zip results that were high on average. The results of the two tables correlate highly. The respondents seem to think that traditional input methods such as the mouse and keyboard are usable and efficient. Desktop computer was perceived to be more efficient as an input method, but on the other hand laptop was considered to be more usable overall. On the UI design selections, the traditional tools such as hyperlinked text, desktop icons and a selection of the simplified versions of tools (login, search tool) reached universal high scores. Table 3 presents a list of detected clusters and their centroids. The table cells have been colored for clarity. Red indicates large values and blue indicates low values. Column C1 presents values present in the first cluster, column C2 in the second cluster and column Dev is the calculated difference between the two columns.

When investigating the cluster values, the clearest line of division is between respondents who rated themselves motivated in play and respondents who did not feel highly motivated to play in any category. The division can be seen clearly in three last rows in Table 3. Respondents who had high motivations regarding gameplay (C1) were also biased towards Belbin coordinator, shaper and resource investigator profiles compared to the other cluster. On the other hand, respondents who were not motivated to play games (C2) had higher Belbin profile preferences in implementer, team worker and complete finisher. It should be noted that the cluster C2 was still slightly interested in the social aspects of gameplay.

Cluster Analysis Results and Validity Evaluation

K-means clustering analysis resulted in two clusters of student profiles that share same preference sets in Belbin and Yee tests. The average silhouette coefficient for combined profile clusters for Yee and Belbin combined is 0.45. This is at the same level of clustering as individual Yee (0.46) or individual Belbin profile clusters (0.46), The silhouette value of 0.45 means that the data point cluster is medium-weak, with value of 0.51 being a limit to a medium coverage cluster [ 7 ].

Profile category

Implementer Coordinator

Shaper

Plant Resource Investigator Monitor Evaluator

Team Worker Complete Finisher

Specialist Yee: Achievement

Yee: Social Yee: Immersion C1 0,35 0,45 0,54 0,41 0,46 0,41 0,34 0,34 0,39 0,70 0,58 0,58

C2 0,46 0,29 0,48 0,42 0,30 0,40 0,43 0,44 0,37 0,07 0,20 0,06

Dev 0,11 0,16 0,06 0,01 0,16 0,01 0,09 0,10 0,02 0,62 0,37 0,52

Discussion and Conclusion

In this paper we discussed the applicability of Belbin [ 5 ] and Yee [ 13 ] profiles to usability preferences and performed an exploratory survey of a learning system user preferences. Our study indicates that the users can be profiled and these profiles can be used to tailor user interfaces. Also, per the survey we observed that there are clear differences in the user preferences between the different types of interfaces, with traditional input methods being preferred the most. The analysis of the data showed meaningful patterns that can be used to divide users into distinct types. There is also a clear order of respondent preferences in perceived efficiency and usability of inputs.

Additionally, the k-means clustering produced two geometrically distinct groups in Belbin and Yee profile results. Especially in mobile platform layouts the respondents seemed to prefer simple, systematic layouts with minimal amount of added views or elements. Also, plain presentation of the data was usually considered more likeable and efficient; users strongly preferred simple dropdown menus over context-classified, and simple text over hypertext. In general, most of the universal low scores in usability represented complex, clustered or item-saturated user interface views.

A threat to the validity of this study is the overfitting of the target population. Our method of collecting answers steered the system towards 20-35 year old, educated and technologically savvy audiences, since we collected most of our data from the university and college. Acknowledging this limitation, we intend to diversify our sample population in the further studies with actual test scenarios, where the learnability and intuitivism have larger impact on the results.

The presented empirical results provide valuable data for interface design decision-making, for example in the model based approaches [ 10 ] or adaptive recommender systems [ 6 ]. The presented approach has potential to provide a method for providing datasets for adaptive interface systems, like the one presented by Ahmed et al. [ 2 ]. The initial analysis and small-scale tests indicate, that this approach could be feasible, since there are differences in the preferences between the different user groups, especially when observing the motivational aspects.

As for future work our intention is to continue with this concept, and extend the profiling test into a complete test setting with usability-related case activities and more detailed user profiling.

References

1. Gediminas Adomavicius and Alexander Tuzhilin. 1999. User profiling in personalization applications through rule discovery and validation. In Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, 377–381.

References

2. Ejaz Ahmed , Nik Bessis, and Yong Yue . 2010 . Customizing interactive patient's diagnosis user interface . In Digital Information Management (ICDIM) , 2010 Fifth International Conference on, 536 - 539 .

3. Pierre A. Akiki , Arosha K. Bandara , and Yijun Yu . 2013 . RBUIS: simplifying enterprise application user interfaces through engineering role-based adaptive behavior . In Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems , 3 - 12 .

4. Vânia Paula de Almeida Neris and M. Cecília C. Baranauskas . 2010 . Making interactive systems more flexible: an approach based on users' participation and norms . In Proceedings of the IX Symposium on Human Factors in Computing Systems , 101 - 110 .

5. R. Meredith Belbin . 2012 . Team roles at work . Routledge.

6. Catherine Inibhunu , Scott Langevin , Scott Ralph, Nathan Kronefeld, Harold Soh, Greg A. Jamieson , Scott Sanner, Sean W. Kortschot, Chelsea Carrasco, and Madeleine White . 2016 . Adapting level of detail in user interfaces for Cybersecurity operations . In Resilience Week (RWS) , 2016 , 13 - 16 .

7. Leonard Kaufman and Peter J. Rousseeuw . 2009 . Finding groups in data: an introduction to cluster analysis . John Wiley & Sons.

8. Antti Knutas , Jouni Ikonen, Dario Maggiorini, Laura Ripamonti, and Jari Porras . 2016 . Creating Student Interaction Profiles for Adaptive Collaboration

9. Vìctor López-Jaquero , Francisco Montero, Antonio Fernández-Caballero, and Marìa D. Lozano . 2004 . Towards Adaptive User Interfaces Generation . In Enterprise Information Systems V, Olivier Camp , Joaquim B. L. Filipe , Slimane Hammoudi and Mario Piattini (eds.) . Springer Netherlands, 226 - 232 .

10. Vivian Genaro Motti, Dave Raggett, and Jean Vanderdonckt . 2013 . Current Practices on Modelbased Context-aware Adaptation . In CASFE , 17 - 23 .

11. Jeffrey Rubin and Dana Chisnell . 2008 . Handbook of usability testing: how to plan, design and conduct effective tests . John Wiley & Sons.

12. Kiri Wagstaff , Claire Cardie, Seth Rogers, Stefan Schrödl, and others. 2001 . Constrained k-means clustering with background knowledge . In ICML , 577 - 584 .

13. Nick Yee . 2006 . Motivations for play in online games . CyberPsychology & behavior 9 , 6: 772 - 775 .