Enhancing Collaborative Sketching with Adaptation Guidelines Vivian Genaro Motti, Ugo Braga Sangiorgi, Jean Vanderdonckt Louvain Interaction Laboratory, Université catholique de Louvain Place des Doyens, 1 - B-1348 Louvain-la-Neuve (Belgium) first.last@uclouvain.be ABSTRACT stakeholders often adopt a “one-size-fits-all” approach, pro- Designing interactive systems for multiple contexts of use viding UIs that are suitable for one single conventional con- becomes a burden when several interaction scenarios, their text of use, with such an approach, accessibility issues often characteristics and constraints, must be considered. Stake- raise, hindering or even preventing the user interaction. holders face then two main challenges: they are not aware Given that it is a challenge for developers to be aware of pos- of what is significantly relevant to consider from contextual sible adaptation issues and that adaptation must be considered information, or how to appropriately adapt the user interfaces since the initial stages of the development process, this work based on this information considered. Moreover, stakeholders proposes the support to context-aware adaptation within user cannot count on existing UI editors once they usually do not interface editors, aiding stakeholders during all the develop- provide support for context-aware adaptation. Thus, adapta- ment phases, but specially starting from low fidelity proto- tion is often ignored, resulting in user interfaces that are only types. This work also aims at raising awareness for con- suitable for static and conventional contexts of use. To sup- text peculiarities and to the importance of adopting adapta- port the design of user interfaces that are properly adapted to tion patterns and guidelines. To do so, a novel methodology their target context, this paper proposes a novel methodology for electronic sketching is proposed, i.e. electronic sketching to enhance sketching activities by suggesting to the end user activities must be enhanced based on the target context of use. adaptation guidelines. This work aims at raising awareness about context-aware adaptation since the early stages of the In previous works, multi-platform sketching has been defined UI design. as the activity of drawing with an electronic stylus in differ- ent devices while still having the same system running on Author Keywords those different devices [12]. G AMBIT has been developed as a multi-platform collaborative tool for user interface de- Collaborative sketching; Context-aware adaptation; sign. G AMBIT enables to sketch UI’s and to simulate them Collaborative design; Prototyping. on different devices. Now we propose to augment sketching activities by guiding stakeholders through the development of ACM Classification Keywords adapted UI’s based on the target context. Three main dimen- H.5.2 User Interfaces: Prototyping sions of context have been defined: • Multi-platform: sketches can be done on the target device INTRODUCTION itself, enabling prototyping and testing the UI’s on the very The contexts of use in which users interact nowadays signifi- device it is intended to run. G AMBIT has been built with cantly vary in terms of user’s profiles, platforms and environ- HTML5 and Javascript being capable of running on any de- ments. Due to such variety, it is challenging for stakeholders vice with browsing capabilities, through a browser or em- to properly consider the specificities of dynamic contexts of bedded into a native application; use while developing user interfaces. As such, usually adap- • Multi-user: since the user’s profiles vary in different as- tation is ignored, either because stakeholders are not aware of pects, as expertise level (novice, intermediate, advanced) relevant characteristics and constraints of the context, or be- and impairments (visual, cognitive, motor), also different cause they are aware but unable to correctly address related user interface requirements and needs must be considered; issues and thus to provide efficient adaptations. As a result, • Multi-environment: for varied constraints and characteris- tics, different adaptation techniques must be applied. For instance taking into account light and stability levels in or- der to provide users appropriate interactors. This paper is organized as follows: the next section motivates Permission to make digital or hard copies of all or part of this work for this work, presents and discusses related works; then Gambit personal or classroom use is granted without fee provided that copies are is presented, followed by the novel proposed methodology to not made or distributed for profit or commercial advantage and that copies support context-aware adaptation. bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MOTIVATION AND RELATED WORK The main shortcomings of such tools regarding context-aware The fact that the contexts of use significantly vary challenges adaptation of user interfaces, can be summarized as: stakeholders while implementing interactive systems. The Firstly, such tools are already not adapted, being implemented profiles of users vary according to their age, expertise levels, to run in a conventional context of use of an able-bodied user preferences and needs. The platforms have different screen in a Desktop PC in a stable environment (e.g. office). Al- dimensions, interaction modalities, performances and capa- though practitioners may be more familiar working in their bilities. The environments have different light, noise and sta- traditional workspace, because the design occurs outside the bility levels. Such heterogeneity impacts the requirements target context, the results may be inaccurate. For instance, by for defining GUI’s for interactive systems, and the interaction designing mobile applications in a smartphone, a more real- may be even prevented if certain characteristics are not appro- istic result may be achieved, specially concerning constraints priately taken into account. Although some UI’s are already as screen dimensions and performance. adapted to a given context, it is not scalable for designers to take each contextual detail into account in a way that several Second, contextual information is not supported, or it is but different versions of the same system are implemented. limited e.g. to screen sizes. Stakeholders are not aware of which information to consider and how to consider. No sup- To support context-awareness during sketching activities we port is provided concerning usability guidelines and patterns. propose the adaptation guidance to be incorporated into the UI design, first by identifying the context of use, then by sug- Third, by being inflexible such tools force users to sketch and gesting related guidelines that can be adopted by the end user. to prototype one version of the same application dedicated to each context. When various contexts must be covered it The existing tools that support electronic sketching are usu- becomes not scalable to properly ensure adaptation. ally not focused on context-aware adaptation. As such, de- signers either ignore contextual information or explore differ- Finally, although users work usually in ’large’ teams, with ent alternatives in a very limited fashion. Nowadays, to gen- different profiles, such tools not always support collabora- erate UIs that are usable in different contexts, designers must tion, thus either all stakeholders must be co-located or they be aware of the target context, and then properly know how cannot simultaneously access and edit the project, resulting to apply adaptation, by following standard guidelines or well- in a pipeline of tasks. By matching different skills and ex- known patterns. There exist already many tools available for pertise simultaneously collaborating within a project, better UI design and electronic sketching, however they currently results can be obtained. do not support context-awareness and guidelines. As such, the resulting UI’s are not appropriately adapted to their con- Besides this, when the context is considered just during late text, or several versions of the same UI must be implemented, development phases, if the resulting UI’s are not satisfactory, causing delay in the time to market, among other problems. significant re-work may be needed. When usability tests and experiments are conducted during later stages, results may be Concerning the UI design, sketching and prototyping, there incorrect once a high fidelity prototype is already available, are several tools available in the market. MAQETTA, for ex- discouraging users to negatively evaluate the resulting UI’s. ample, is a visual authoring tool for designing HTML5-based user interfaces, it adopts a WYSIWYG approach. Because MAQETTA is also a web-based application, no plugins or ENHANCING SKETCHING WITH ADAPTATION downloads are required. Although it permits the design of Sketching is considered to be a powerful tool for designing UI mockups, context-aware adaptation is not integrated as UI’s. As the findings of [8] point out, the presence of ambi- a feature [4]. Besides MAQETTA, a series of alternative guity in early stages of design broads the spectrum of consid- graphical editors have been published to support activities of ered solutions and tends to deliver a design of higher quality. sketching and prototyping UI’s in varied fidelity levels. Four Fostering creativity is considered to be important since de- popular tools were selected as references; they are listed and sign is essentially a problem of wicked nature, i.e. the process briefly described below. of solving it is identical with the process of understanding it [11]. In wicked problems, in initial phases the designer does The most popular tools that are currently used for supporting not have a clear understanding of what to produce and has UI Sketching and prototyping are: only a vague goal in mind, therefore in order to support the • BALSAMIQ [1] for sketching interfaces rapidly, and com- design activity with an interactive system, it is important that municate design ideas (www.balsamiq.com); the system does not get “in the way” between the designer • J UST I N M IND [3] a platform for defining prototypes for and the solution to be developed. web and mobile applications (www.justinmind.com); G AMBIT, as Figure 1 illustrates, offers users a flexible setup • MAQETTA [4] a visual authoring of HTML5 user where they can choose their preferred devices for interacting interfaces in the browser, open source, WYSIWYG – i.e. devices that are more suitable for sketching input can be (maqetta.org); used for pen interaction, while large displays can be used for • S KETCH F LOW [5] a UI prototyping tool to create interac- visualization. By observing UI design sessions on belgian IT tive prototypes1 . companies, three main sequential phases have been identified: 1. Sketching: one or more participants produce sketches to 1 http://www.microsoft.com/silverlight/sketchflow/ express ideas. Figure 1. G AMBIT, a distributed sketching system. Left: users A, B and C collaborate in sketching activities, using respectively a laptop, a tablet and an interactive table. Right: 3 synchronized platforms in use for sketching. 2. Sharing: the participants normally share the drawings us- (based on the target context and the UI elements). ing a big sheet of paper and use post-its. The sheets are For the UI guidelines, three distinct dimensions of (graphical) arranged as a storyboard on a wall for discussion. user interfaces can be considered, namely: 3. Discussing: the participants refined the sketches based on what was discussed and learned on the discussion. • Navigation: an interactive system is usually structured by G AMBIT provides a large (infinite) workspace where users means of a task tree (being CTT [2] and HTA often em- can add pictures and sketch. Parts of the workspace can be ployed). Although one single task tree can enable the same shared among different platforms, composing a virtual meet- goals to be achieved in different contexts of use, there are ing room. The tool is a fundamental part of a research on several groupings and hierarchies that can be adopted to better accommodate tasks according to different scenarios. sketching user interfaces, its usage in current design practices For example concerning the screen dimension of the user, must also take into account the diversity of contexts of use. if it is a small screen (smartphone) a reduced number of In this sense, a designer can test an interface for a system on the same device the system is intended to be used. G AMBIT’s tasks must be presented at a time. basic requirements list was presented in [12]. • Presentation: the layout of graphical user interfaces can be presented in different ways, the distribution of the contents Aiming to enhance the sketching activity currently supported can follow a certain alignment, balance, or specific certain towards effective consideration of context-awareness, we pro- re-molding actions. pose to augment it by integrating adaptation guidelines. To do • Contents: several formats of contents can be included in so, first the end user must provide the description of the target the UI composition. Usually images, texts, videos and context, then while sketching the tool will recognize the lay- UI interactors combined provide a task for the user. De- out of the UI elements and also propose relevant adaptation pending on the context, such contents can be: removed, techniques according to the UI definitions. re-sized, replaced, etc. Such dimensions provide an overview about possible adapta- Methodology and Design Decisions tions, for a more complete list of techniques other references For the context of use, it can be provided to the system by can be accessed (e.g. [10]). means of an XML-based document, containing data about According to the elements recognized in the UI, and the con- the: user, platform and environment. Once parsered such in- text provided, the system can then automatically search for formation will be used later on to search for corresponding and present relevant guidelines for the end user. A semi- guidelines. For the sketch recognition, some algorithms have automatic approach can be applied to suggest changes for the already been investigated in previous works. UsiSketch [7], sketched UI, i.e. once the guidelines’ list is presented, the for instance, presents an algorithm that automatically recog- end user is responsible for analyzing it, judging whether it is nizes sketches for graphical user interfaces. And concern- valid, modifying the UI, and changing the guideline’s status ing the guidelines, an online repository as W3C guidelines (e.g. as checked). [6] can be accessed, a local repository integrating different sources, or third-party web services on the cloud could be connected to the system to provide the corresponding list Figure 3. Sketching GUI’s for elderly users (storyboard of 3 UI’s). CONCLUSION Many editors support UI design, but they often exclude UI guidelines. Due to the technological and contextual evolu- tion, designers are unaware of which and how context infor- mation to consider. Thus, most existing UI’s ignore adapta- tion and target one context of use, while contexts vary and evolve. We claim that by incorporating UI guidelines in UI Figure 2. Sequence Diagram editors, stakeholders can enhance sketching, by being aware of relevant principles, verifying and improving UI’s. This paper proposes enhanced sketching activities, by means of a Given that design sessions often concern a specific applica- novel approach, and illustrates the proposal with a case study. tion domain, the suggested guidelines must cover a generic purpose, following specific constraints of context of use and ACKNOWLEGDEMENTS UI elements, but still being domain-independent. Specific re- Projects: QualIHM (Région Wallonne, Dir. générale quirements of the given methodology include: opérationnelle de l’Economie, de l’Emploi et de la Recherche • Context as an input: users must provide information about DGO6) and Serenoa (E.C. FP7 (FP7-ICT-2009-5)). the target context of use REFERENCES • UI Sketching: users design the UI defining its layout 1. Balsamiq Mockups: Rapid Wireframing Tool, 2012. • Pattern Recognition: the system recognizes UI elements www.balsamiq.com. • Guidelines Suggestions: based on the element previously 2. Concur Task Trees (CTT), 2012. recognized the system search for, retrieve and present http://www.w3.org/2012/02/ctt/. guidelines that match both context of use and UI element 3. JustInMind: a platform to define web and mobile apps., 2012. • User Feedback: the user provides his/her feedback about http://www.justinmind.com/. the system suggestion, accepting, rejecting, evaluating 4. Maqetta: Visual Authoring of HTML5 User Interfaces in the browser., 2012. http://dojofoundation.org/projects/maqetta. Figure 2 structures such steps as a sequence diagram. The 5. SketchFlow: Expression Studio Ultimate, 2012. http: target context (message 1) is provided once, in the start of the //www.microsoft.com/silverlight/sketchflow/. design session, sketching (2), recognition (2.1) and guidelines 6. Web Content Accessibility Guidelines (WCAG) 2.0, 2018. (3 and 4) are repeated until the sketch is concluded. http://www.w3.org/TR/WCAG/. 7. Coyette, A., Schimke, S., Vanderdonckt, J., and Vielhauer, C. Trainable Scenario sketch recognizer for graphical user interface design. In INTERACT (1) The case study consists in designing graphical user interfaces (2007), 124–135. for elderly users. It has already been proved that older adults 8. Goel, V. ”Ill-Structured Representations” for Ill-Structured Problems. In Proceedings of the Fourteenth Annual Conference of the Cognitive perform better and prefer touch screen interfaces. Actually, Science Society, vol. 14, Lawrence Erlbaum (1992), 130–135. many studies conducted in this domain, provided valuable in- 9. Kobayashi, M., Hiyama, A., Miura, T., Asakawa, C., Hirose, M., and sight concerning guidelines for interaction, e.g. [9]. In our Ifukube, T. Elderly user evaluation of mobile touchscreen interactions. study 3 GUI’s are needed to accomplish a user task: the user In Human-Computer Interaction–INTERACT 2011. Springer, 2011, is searching for a movie. First 3 main categories are presented 83–99. (e.g. drama, sci-fi, comedy), when one of those is selected a 10. Motti, V. G., and Vanderdonckt, J. A Computational Framework for list of movies is presented. Each movie contains an image and Context-aware Adaptation of User Interfaces. In Proceedings of the Research Challenges in Information Sciences Conference on - RCIS ’13 a short description. For further information, the user needs to (2013), 12. click in a link available below the description. To navigate 11. Rittel, H. Dilemmas in a general theory of planning. Policy sciences 4, through the list of movies, the GUI can be scrolled. Figure 3 2 (1973), 155–169. illustrates this scenario. Observe that the scroll bar has been 12. Sangiorgi, U. B., Beuvens, F., and Vanderdonckt, J. User Interface detected as a potential issue for the UI. So the guideline points Design by Collaborative Sketching. In Proceedings of the Designing it to the user, and suggests a new dimension or a new element Interactive Systems Conference - DIS ’12, ACM Press (Newcastle for the user, improving accessibility. Upon Tyne, United Kingdom, 2012), 378.