Machine Learning in the Support of Context-Aware Adaptation Vivian Genaro Motti, Nesrine Mezhoudi, Jean Vanderdonckt LILab – Louvain Interaction Laboratory - Université catholique de Louvain Place des Doyens 1 – Louvain-la-Neuve 1348 vivian.motti@uclouvain.be ABSTRACT use. Therefore, in the current computational landscape, of Adapting user interfaces according to the context of use ubiquity, mobility, pervasiveness and context-awareness, aims at improving the usability levels of an application and adaptation becomes an inevitable solution. However, enhancing the user experience, mainly by optimizing the efficiently implementing adaptation taking several contexts users’ interaction and reducing their errors. However, given into account to match them with appropriate techniques is a the significant amount of information involved, adapting challenge. UIs often demands complex inferences. Because the context information is extensive, it is hard to prioritize it to decide In this sense, machine learning (ML) as a domain capable the best adaptation techniques. Moreover, dealing with of supporting the solution of complex problems, is able to recurrent trade-offs, e.g. adaptability vs. performance, is not provide significant help [Alpaydin, 2004], [Bishop, 2006], simple. To aid the adaptation decisions, machine learning [Barber, 2010]. Although machine learning is able to algorithms can be applied to support reasoning, inferences provide important benefits for CAA, so far there is no and also to deal with complex or fuzzy information. guidance that effectively supports developers in finding the Although ML can provide several benefits for CAA, there best approaches to solve their recurrent problems in CAA. is no agreed framework that aids developers in applying it. This paper presents a roadmap that guides stakeholders in Thus, aiming to fill such a gap, this paper defines potential the application of machine learning algorithms in the scenarios of CAA where ML can be successfully applied, domain of context-aware adaptation. Potential application presenting their common requirements and main trade-offs. scenarios are exemplified, supporting the development of Author Keywords similar applications. We present a set of potential Context-aware Adaptation, Adaptivity, Adaptability. approaches that support the development of adaptive and adaptable applications in an optimal fashion. This roadmap ACM Classification Keywords aims at guiding the development of CAA in all its phases, D.2.2 [Software Engineering]: User interfaces. H.1.2 considering different application domains, context [Information Systems]: Human factors. H.5.1 information and scenarios of use. We also identified the [Information Interfaces and Presentation]: Multimedia main trade-offs commonly encountered and commons Information Systems. H5.2 [Information interfaces and requirements for applying ML for context-aware presentation]: User Interfaces – User-centered design. adaptation. General Terms This paper is organized as follows: the Section 2 discusses Human Factors; Design. related works; Section 3 presents the roadmap; Section 4 INTRODUCTION presents common requirements and the design decisions; Nowadays users of many different profiles are interacting Section 5 discusses the contributions, and Section 6 with several device types and from varied environments, presents the final remarks and the future works. resulting in significantly heterogeneous contexts of use in RELATED WORKS which the interactions to take place. As such, it is not Since the early 90’s ML has been applied to support scalable for developers to implement dedicated versions of different façades of CAA. Although these works are UIs and applications that are suitable for and accommodate dedicated to explore distinct applications of machine all constraints and characteristics of each specific context of learning for context-aware adaptation, they are scattered, each one focusing on a specific application of adaptation at Permission to make digital or hard copies of all or part of this work for a time without a unified view of their potential benefits. personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies CAA can benefit from ML potential especially during two bear this notice and the full citation on the first page. To copy otherwise, distinct phases: the inferences about context information, or republish, to post on servers or to redistribute to lists, requires prior and the CAA design decisions. Examples of ML application specific permission and/or a fee. during the inference phase include: finding patterns in user interaction histories and clustering data, as users’ profiles [Jennings and Higuchi, 1993]. Examples of ML applied for pages [Smith and Blandford, 2002]. Although it is the most the design decisions of the CAA process include: predicting common approach adopted for context-aware adaptation the user behavior [Mitrovic, 2007], [Mitrovic, 2009], given its simplicity, it has also reduced expressiveness. automating tasks, learning about the user preferences, and It is important to investigate how each machine learning continuously evolving the adaptation engine itself. Other algorithm support specific phases of context-aware ML algorithms also have been successfully applied to adaptation in depth; but so far there is no unified view that support CAA, for example: supports the application of ML for CAA in a broad manner, • Bayesian Networks: as directed acyclic graphs, covering general-purpose implementations. To contribute in composed by nodes and arcs representing respectively this sense, we abstracted potential applications, defined variables and their relationships, Bayesian networks have common requirements, and highlighted the main trade-offs. been applied to model users and to predict their needs. As LEARNING GOALS such, the system observes the user events, and responds to Based on the analysis of the related works presented above, them by triggering new events [Horvitz et al., 1998]. we notice that the learning mechanism can be employed to • Clustering: consists in associating information based on achieve specific goals. These goals are achieved either the similarities found in its properties. Information that based on patterns that are identified from the user shares the same characteristics is located together interaction or based on the user evaluation of the adaptation composing a group called cluster. E.g. contents in results provide by the system. Generally these goals belong webpages can be clustered according to their users’ access, to 5 main classes: as performed in MLTutor, distinguishing trends in user Associating: for instance when a navigational pattern is interaction [Smith and Blandford, 2002]. identified in the user interaction history (recorded by means • Decision Trees (DT): are composed by set of rules of log files), the tasks that are executed together can be strategically and hierarchically organized. DT aid the associated in chunks, facilitating their access and making representation, the selection and the classification of data. the interaction more efficient. In this way groups of tasks In [Vanderdonckt, 1999] and in [Eisenstein and Puerta, are created. One possible category is favourite 2000] decision trees were applied to support the decision corresponding to the most accessed contents or tasks. The and selection of the most appropriate widget to build a GUI. same approach is also valid for groups of users, i.e. users with same interests and profiles can share their favourite • Fuzzy Logic: recommended to deal with fuzzy data, the contents. fuzzy logic was applied to adapt services in a mobile context [Cao et al., 2005], and to evaluate design decisions Sorting: consists in abstracting the sequence of interaction of adaptive composition of mobile applications [Desruelle of the user in order to re-define the access order of the et al., 2011]. tasks, for instance with a given user always access the menu items in a given order (this pattern must be also identified • Genetic Algorithms: Acay (2004) recommends genetic by analysing log files of the interaction history), this algorithms in adaptation for complex systems, since they information must be used as the criterion to re-arrange the are capable of handling highly constraint problems, they are order of the menu items, links, buttons of the UI. scalable for higher dimensions, and also because in complex task the feature space can grow exponentially with Suggesting/Recommending: based on expected interaction, the number of features. the system can suggest to the user contents or tasks that are more likely to be accessed by them. A certain user profile • Markov Models: In a simple markov model each node can help to decide the contents of interest. lists possible states of the system, and the transitions reflect probabilities to change from one state to another. Only the Hiding/Deleting: the contents or tasks that have the least last state (user action) is considered, however in second- frequent access must also have less priority of access in the order models the last two states (user actions) also count, UI, i.e. while the access to the most common tasks and [Deshpande and Karypis, 2004]. In CAA, markov models contents must be facilitated the least common tasks and have been used to predict user actions based on their contents can be grouped and accessed by means of an extra interaction history. resource (link, button or icon). Thus, optimizing the space usage in the UI. Deleting the shortcuts to least accessed • Neural Networks: as an interconnected group of artificial tasks can also be an option to optimize the space usage in neurons, by using processing elements to connect input the UI, by giving priorities to the most used contents and nodes with output nodes, neural networks were applied to accessed tasks. compile a user profile and guide web browsing according to the relevance of the web content [Seo and Zhang, 2000]. Creating: some interactions that are always repeated by the user while accessing and interacting with the application • Rule Induction: consists in extracting formal rules based can be automated. For instance if the user always re-size on observation of data, or local patterns. It was specifically (maximize) the window of the UI, the system can change applied in MLTutor to suggest users relevant hypertext the default settings to automatically maximize the window microphone, speakers, headset), battery level (in case of when the application is launched. mobile devices). As possible classes for the DT, two modality types are envisaged: graphic and audio. It is out of Aiming to fulfill the goals abovementioned, different ML the scope of this paper to provide further implementation can be employed. The next section illustrates how details about modeling each scenario suggested, however algorithms can be applied to perform CAA. we do provide in the next section further refinements about A ROADMAP OF MACHINE LEARNING APPLICATIONS the design decisions. Although ML has already been applied to support CAA, the DESIGN DECISIONS works so far are sparse and thus not integrated yet, given Potential trade-offs. Although ML algorithms provide a that there is no agreed framework capable of guiding set of benefits for CAA, care must be taken to not disturb stakeholders in the selection and in the implementation of the user interaction. Clearly, there are several trade-offs that ML algorithms to provide efficient CAA to the end users, in can be expected by automating UI changes by means of this section we provide a unified view of how ML adaptation. Therefore if the user interaction is not very well algorithms can be employed for applications in the different understood, the adaptation results can: bother, annoy, phases of a CAA process. confuse or even prevent users from achieving their actual Table 1. Guidance Roadmap defining ML algorithms goals. That is why users must be always able to evaluate the according to the CAA goals adaptation, to confirm that the results achieved are ! "#$%&'%(! )*!+&$&,-%.#$(! convenient according to their goals and interests. Moreover, by changing the UI layout that the users are familiarized /&0.1.#$! !"#$%&&'()"*&+,&"%##-,.'/0" "!"9'.0+1&:"&+$+#1'-/" with, can make them lost, thus all the changes must be 2,&&1! 1-"12+',"3,-('$+&"4+505" clearly indicated. The common requirements presented !";+&1"<-.%$'1)".+#'&'-/" 3,-('#'+/#)"$+6+$7"'/1+,+&1&8" below aim at fulfilling CAA goals with ML, but also "341%&,.$5! !"0,-*3"*&+,&"9'12"&'<'$%," !"12+"<+/*"'1+<&"#%/";+" avoiding potential trade-offs. #2%,%#1+,'&1'#&"4+5057"9'12" #-<3-&+."%##-,.'/0"1-" Common Requirements. Technically, the learning process 3,-('$+"#$*&1+,&8" 12+"(,+>*+/#)"-("%##+&&" by means of ML algorithms occurs when: (i) the priorities !"('/."#$*&1+,&"%##-,.'/0"1-" !"12+"#-/1+/1&"#%/";+" (or weights) of the adaptation rules are modified in a 12+".+6'#+"1)3+"4+505"1-" #$*&1+,+.",+0%,.'/0"12+" decision tree (the conditions in each node), providing 1%#=$+"12+"<-;'$+" 3,-('$+&"-("12+"*&+,&" adaptation results that are more suitable for the context of (,%0<+/1%1'-/"3,-;$+<8" '/1+,+&1+." the user, (ii) the records of the previous interaction behavior 4,+#-<<+/.%1'-/"?" are analyzed and patterns are identified helping the &*00+&1'-/&8" Adaptation Engine to automatically perform new adaptations, (iii) a negative feedback is given by the user, 6-,7#8! !"12+"#-/1+@1"#2%/0+&"#%/" !"+&1%;$'&2+&"%"1,%/&'1'-/" "9-.$! &+,6+"%&"%"1,'00+,"(-,"12+" <%1,'@"9'12"3,-;%;'$'1'+&" and then an adaptation technique has its priority or weight %.%31%1'-/"1+#2/'>*+&7"'5+5" 12%1"#%/";+"%33$'+."(-,D" decreased in the adaptation engine. 0'6+/"%"#2%/0+"'/"12+" +%#2"%.%31%1'-/" All these approaches rely on the end user behavior and #-/1+@1"-/+"1+#2/'>*+" 1+#2/'>*+"4%/." feedback. For instance, if a certain pattern is identified in <*&1";+"%33$'+.A"+505"92+/" #-/&+>*+/1$)"+%#2" the log of the user interaction, it is likely that the navigation -,'+/1%1'-/"'&"#2%/0+.7"12+" %.%31+."BC8"0'6+/"%" in the application can be optimized accordingly. And if the BC"'&",-1%1+." &3+#'('#"#-/1+@1A"-,"+%#2" user accepts the adaptation proposed by the system, this 1%&=7"#-/1+/1"9'12'/"%"BC" feedback is used by the engine to set the adaptation as :&5,&11.#$! !"'.+/1'()"12+"+@'&1+/#+"-(" !"%&&-#'%1'-/"-("9'.0+1&" successful. 3%11+,/&";+19++/"*&+," 3,-3+,1'+&"4+5057" Machine learning algorithms aim at optimizing the 3,+(+,+/#+&"4%&"#-<<-/" .'<+/&'-/&7"6'&';'$'1)7" adaptation process. Then, mainly the learning infrastructure '/1+,%#1'-/"2'&1-,'+&8"%/." #-/1,%&1"$+6+$&8"E" %.%31%1'-/"1+#2/'>*+&" +6%$*%1'-/"<+1,'#&"4+5057" involves two general requirements. First gathering the 4+5057"#2%/0+"12+" +,0-/-<'#&7"%+&12+1'#&8" preferences of the user (with feedback or history of %33$'#%1'-/"<-.+&D"&'$+/17" 3+,"#-/1+@1"-("*&+" interaction for instance) and possibly other context $-*.7"+1#58" elements and then adapting the engine (for instance by adjusting parameters related with priorities or preferences). For each scenario listed in Table 1, specific refinements are These requirements are cyclic, i.e. once the adaptation is needed to concretely model and implement the solutions. performed and present, the user intervene again providing For instance, regarding the application of decision trees for his or her feedback, which, if positive, concludes the cycle, UI generation, in the definition of the best modality, and if negative, adjust the engine. Such general attributes gathered from the context information can be requirements can be refined in four specific requirements: considered, as: occurrence of visual impairments of the user, availability of input and output devices (e.g. • First, the user interaction needs to be recorded (with log files, or history features). 4.Bishop, C. M. Pattern Recognition and Machine • Then, the adaptation process may request the Learning, Springer, 2006, ISBN: 978-0-387-31073-2 authorization of the user to proceed, by announcing 5.Cao, J.; Xing N.; Chan, A.T.S.; Feng Y.; Jin B.; "Service what will be done, and proceeding in case of adaptation using fuzzy theory in context-aware mobile agreement. computing middleware," Embedded and Real-Time • The adaptation must be presented in a progressive Computing Systems and Applications, 2005. Proceedings. fashion, and clearly stating to the end user that there is 11th IEEE International Conference on , vol., no., pp. a change, and explaining what is happening, and also 496- 501, 17-19 Aug. 2005 doi: 10.1109/RTCSA.2005.93 why it is happening. • And finally, after the adaptation was performed, the 6.Deshpande, M. and Karypis, G. 2004. Selective Markov user must have the option to: models for predicting Web page accesses. ACM Trans. o Accept or reject it Internet Technol. 4, 2 (May 2004), 163-184. o Undo it DOI=http://doi.acm.org/10.1145/990301.990304 o Evaluate or classify it, providing a feedback 7.Desruelle, H., Blomme, D., and Gielen, F.. Adaptive that will be used to adjust the parameters of Mobile Web Applications: A Quantitative Evaluation the techniques implemented. Approach. In Proceedings of ICWE. 2011, 375-378. The user feedback must be applied to improve and evolve 8.Eisenstein, J. and Puerta, A.. 2000. Adaptation in the adaptation engine, making its results more precise and automated user-interface design. In Proceedings of the 5th accurate for next applications. international conference on Intelligent user interfaces (IUI CONTRIBUTIONS '00). ACM, New York, NY, USA, 74-81. The main contributions of this paper include: motivating the DOI=10.1145/325737.325787 use of ML algorithms for CAA; contextualizing application http://doi.acm.org/10.1145/325737.325787 scenarios; and providing a roadmap to guide stakeholders in their design decisions regarding CAA supported by ML. 9.Horvitz, E., Breese, J., Heckerman, D., Hovel, D., and The specific contributions include: the literature review of Rommelse, K. (1998). The Lumiere project: Bayesian current related works; the description of ML algorithms user modeling for inferring the goals and needs of applied for CAA scenarios; and the presentation of main software users. Proceedings of the Fourteenth Conference trade-offs and common requirements in this domain. on Uncertainty in AI FINAL REMARKS 10.Jennings, A. and Higuchi, H. (1993). A User Model Due to the large availability of technological devices, it is Neural Network for a Personal News Service. User each day more important to implement and provide users Modeling and User-Adapted Interaction 3, S. 1–25. applications able to effectively adapt themselves according 11.Mitrovic, N., Royo, J. A. and Mena, E. (2007). to the context. Such implementation is a complex task, Performance analysis of an adaptive user interface system since much context information is involved and several based on mobile agents. In Proc. DSV-IS '07. New York: application resources can be subject to context-aware ACM Press, 29--44. adaptation. Therefore, ML algorithms can efficiently support this task. ML has as main advantages the ability to 12.Mitrovic, N. Royo, J. A. and Mena, E. (2009) Adaptive sense the context in a dynamic manner to evolve the Graphical User Interfaces: An Approach Based on Mobile adaptation engines, providing users more suitable Agents, Networks, pp. 1-36, 2009. adaptations. This paper presents a roadmap that can 13.Seo, Y.-W., & Zhang, B.-T. (2000). Learning user’s effectively guide stakeholders in the application of ML preferences by analyzing web-browsing behaviors. In algorithms for CAA. Proc. of int’l conf. on autonomous agents 2000 (aa ’2000) ACKNOWLEDGMENTS (pp. 381– 387). This work received funding from the European 14.Smith, A. S. G. and Blandford, A. N. N. (2002) Commission’s Seventh Framework Program under grant MLTutor : An Application of Machine Learning agreement number 258030 (FP7-ICT-2009-5). Algorithms for an Adaptive Web-based Information REFERENCES System, International Journal of Artificial Intelligence in 1. Acay, L. (2004) Adaptive User Interfaces in Complex Education, pp. 1-22, 2002. Supervisory Tasks. PhD Thesis. 15.Vanderdonckt, J. (1999). Advice-giving systems for 2.Alpaydin, E. Introduction to Machine Learning, The MIT selecting interaction objects. Proceedings User Interfaces Press, October 2004, ISBN 0-262-01211-1 to Data Intensive Systems, 152-157. IEEE. At: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arn 3.Barber, D. Bayesian Reasoning and Machine Learning, umber=79147 Cambridge University Press , 2010