Designing interactive systems for multiple contexts of use becomes a burden when several interaction scenarios, their characteristics and constraints, must be considered. Stakeholders face then two main challenges: they are not aware of what is significantly relevant to consider from contextual information, or how to appropriately adapt the user interfaces based on this information considered. Moreover, stakeholders cannot count on existing UI editors once they usually do not provide support for context-aware adaptation. Thus, adaptation is often ignored, resulting in user interfaces that are only suitable for static and conventional contexts of use. To support the design of user interfaces that are properly adapted to their target context, this paper proposes a novel methodology to enhance sketching activities by suggesting to the end user adaptation guidelines. This work aims at raising awareness about context-aware adaptation since the early stages of the UI design.
INTRODUCTION
The contexts of use in which users interact nowadays
significantly vary in terms of user’s profiles, platforms and
environments. Due to such variety, it is challenging for stakeholders
to properly consider the specificities of dynamic contexts of
use while developing user interfaces. As such, usually
adaptation is ignored, either because stakeholders are not aware of
relevant characteristics and constraints of the context, or
because they are aware but unable to correctly address related
issues and thus to provide efficient adaptations. As a result,
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
stakeholders often adopt a “one-size-fits-all” approach,
providing UIs that are suitable for one single conventional
context of use, with such an approach, accessibility issues often
raise, hindering or even preventing the user interaction.
Given that it is a challenge for developers to be aware of
possible adaptation issues and that adaptation must be considered
since the initial stages of the development process, this work
proposes the support to context-aware adaptation within user
interface editors, aiding stakeholders during all the
development phases, but specially starting from low fidelity
prototypes. This work also aims at raising awareness for
context peculiarities and to the importance of adopting
adaptation patterns and guidelines. To do so, a novel methodology
for electronic sketching is proposed, i.e. electronic sketching
activities must be enhanced based on the target context of use.
In previous works, multi-platform sketching has been defined
as the activity of drawing with an electronic stylus in
different devices while still having the same system running on
those different devices [
Multi-platform: sketches can be done on the target device itself, enabling prototyping and testing the UI’s on the very device it is intended to run. GAMBIT has been built with HTML5 and Javascript being capable of running on any device with browsing capabilities, through a browser or embedded into a native application; Multi-user: since the user’s profiles vary in different aspects, as expertise level (novice, intermediate, advanced) and impairments (visual, cognitive, motor), also different user interface requirements and needs must be considered; Multi-environment: for varied constraints and characteristics, different adaptation techniques must be applied. For instance taking into account light and stability levels in order to provide users appropriate interactors.
This paper is organized as follows: the next section motivates this work, presents and discusses related works; then Gambit is presented, followed by the novel proposed methodology to support context-aware adaptation.
MOTIVATION AND RELATED WORK
The fact that the contexts of use significantly vary challenges
stakeholders while implementing interactive systems. The
profiles of users vary according to their age, expertise levels,
preferences and needs. The platforms have different screen
dimensions, interaction modalities, performances and
capabilities. The environments have different light, noise and
stability levels. Such heterogeneity impacts the requirements
for defining GUI’s for interactive systems, and the interaction
may be even prevented if certain characteristics are not
appropriately taken into account. Although some UI’s are already
adapted to a given context, it is not scalable for designers to
take each contextual detail into account in a way that several
different versions of the same system are implemented.
To support context-awareness during sketching activities we
propose the adaptation guidance to be incorporated into the
UI design, first by identifying the context of use, then by
suggesting related guidelines that can be adopted by the end user.
The existing tools that support electronic sketching are
usually not focused on context-aware adaptation. As such,
designers either ignore contextual information or explore
different alternatives in a very limited fashion. Nowadays, to
generate UIs that are usable in different contexts, designers must
be aware of the target context, and then properly know how
to apply adaptation, by following standard guidelines or
wellknown patterns. There exist already many tools available for
UI design and electronic sketching, however they currently
do not support context-awareness and guidelines. As such,
the resulting UI’s are not appropriately adapted to their
context, or several versions of the same UI must be implemented,
causing delay in the time to market, among other problems.
Concerning the UI design, sketching and prototyping, there
are several tools available in the market. MAQETTA, for
example, is a visual authoring tool for designing HTML5-based
user interfaces, it adopts a WYSIWYG approach. Because
MAQETTA is also a web-based application, no plugins or
downloads are required. Although it permits the design of
UI mockups, context-aware adaptation is not integrated as
a feature [
The most popular tools that are currently used for supporting UI Sketching and prototyping are:
BALSAMIQ [
Second, contextual information is not supported, or it is but limited e.g. to screen sizes. Stakeholders are not aware of which information to consider and how to consider. No support is provided concerning usability guidelines and patterns. Third, by being inflexible such tools force users to sketch and to prototype one version of the same application dedicated to each context. When various contexts must be covered it becomes not scalable to properly ensure adaptation. Finally, although users work usually in ’large’ teams, with different profiles, such tools not always support collaboration, thus either all stakeholders must be co-located or they cannot simultaneously access and edit the project, resulting in a pipeline of tasks. By matching different skills and expertise simultaneously collaborating within a project, better results can be obtained.
Besides this, when the context is considered just during late
development phases, if the resulting UI’s are not satisfactory,
significant re-work may be needed. When usability tests and
experiments are conducted during later stages, results may be
incorrect once a high fidelity prototype is already available,
discouraging users to negatively evaluate the resulting UI’s.
ENHANCING SKETCHING WITH ADAPTATION
Sketching is considered to be a powerful tool for designing
UI’s. As the findings of [
GAMBIT, as Figure 1 illustrates, offers users a flexible setup
where they can choose their preferred devices for interacting
– i.e. devices that are more suitable for sketching input can be
used for pen interaction, while large displays can be used for
visualization. By observing UI design sessions on belgian IT
companies, three main sequential phases have been identified:
1. Sketching: one or more participants produce sketches to
express ideas.
2. Sharing: the participants normally share the drawings
using a big sheet of paper and use post-its. The sheets are
arranged as a storyboard on a wall for discussion.
3. Discussing: the participants refined the sketches based on
what was discussed and learned on the discussion.
GAMBIT provides a large (infinite) workspace where users
can add pictures and sketch. Parts of the workspace can be
shared among different platforms, composing a virtual
meeting room. The tool is a fundamental part of a research on
sketching user interfaces, its usage in current design practices
must also take into account the diversity of contexts of use.
In this sense, a designer can test an interface for a system on
the same device the system is intended to be used. GAMBIT’s
basic requirements list was presented in [
Aiming to enhance the sketching activity currently supported towards effective consideration of context-awareness, we propose to augment it by integrating adaptation guidelines. To do so, first the end user must provide the description of the target context, then while sketching the tool will recognize the layout of the UI elements and also propose relevant adaptation techniques according to the UI definitions.
Methodology and Design Decisions
For the context of use, it can be provided to the system by
means of an XML-based document, containing data about
the: user, platform and environment. Once parsered such
information will be used later on to search for corresponding
guidelines. For the sketch recognition, some algorithms have
already been investigated in previous works. UsiSketch [
Navigation: an interactive system is usually structured by
means of a task tree (being CTT [
Presentation: the layout of graphical user interfaces can be presented in different ways, the distribution of the contents can follow a certain alignment, balance, or specific certain re-molding actions.
Contents: several formats of contents can be included in the UI composition. Usually images, texts, videos and UI interactors combined provide a task for the user. Depending on the context, such contents can be: removed, re-sized, replaced, etc.
Such dimensions provide an overview about possible
adaptations, for a more complete list of techniques other references
can be accessed (e.g. [
According to the elements recognized in the UI, and the context provided, the system can then automatically search for and present relevant guidelines for the end user. A semiautomatic approach can be applied to suggest changes for the sketched UI, i.e. once the guidelines’ list is presented, the end user is responsible for analyzing it, judging whether it is valid, modifying the UI, and changing the guideline’s status (e.g. as checked). Given that design sessions often concern a specific application domain, the suggested guidelines must cover a generic purpose, following specific constraints of context of use and UI elements, but still being domain-independent. Specific requirements of the given methodology include:
Context as an input: users must provide information about
the target context of use
UI Sketching: users design the UI defining its layout
Pattern Recognition: the system recognizes UI elements
Guidelines Suggestions: based on the element previously
recognized the system search for, retrieve and present
guidelines that match both context of use and UI element
User Feedback: the user provides his/her feedback about
the system suggestion, accepting, rejecting, evaluating
Figure 2 structures such steps as a sequence diagram. The
target context (message 1) is provided once, in the start of the
design session, sketching (2), recognition (2.1) and guidelines
(3 and 4) are repeated until the sketch is concluded.
Scenario
The case study consists in designing graphical user interfaces
for elderly users. It has already been proved that older adults
perform better and prefer touch screen interfaces. Actually,
many studies conducted in this domain, provided valuable
insight concerning guidelines for interaction, e.g. [