=Paper=
{{Paper
|id=None
|storemode=property
|title=End-User Customization of Multi-Device Ubiquitous User Interfaces
|pdfUrl=https://ceur-ws.org/Vol-617/MDDAUI2010_Paper11.pdf
|volume=Vol-617
}}
==End-User Customization of Multi-Device Ubiquitous User Interfaces==
https://ceur-ws.org/Vol-617/MDDAUI2010_Paper11.pdf
End-User Customization of Multi-Device Ubiquitous User
Interfaces
Fabio Paternò, Giuseppe Zichitella
HIIS Laboratory – CNR-ISTI
Via Moruzzi 1, 56124 Pisa, Italy
{fabio.paterno, giuseppe.zichitella}@isti.cnr.it
ABSTRACT End-User Development [3] (EUD) can be defined as a set
In this paper we discuss an approach to supporting end- of methods, techniques, and tools that allow users of
users in customizing multi-device ubiquitous user software systems, who are acting as non-professional
interfaces. In particular, we show a tool allowing end-users software developers, at some point to create, modify or
to customize desktop-to-mobile adaptation by exploiting extend a software artefact. End-users have already
model-based descriptions in the MARIA language. Some difficulties with single device applications, thus it is easy to
results are presented along with indications for future work. understand how such difficulties increase when considering
Author Keywords applications for multi-device environments. This is one
End-user Development, Ubiquitous Applications, Multi- further reason for providing better support for EUD in
Device Environments, Adaptation. ubiquitous applications.
ACM Classification Keywords The vision of ubiquitous computing [9] is that the users
H.5.2 User Interfaces. operate in intelligent environments, which are aware of
users’ needs and able to assist, even proactively, the users
INTRODUCTION in performing their activities and reaching their goals. To
One of the main issues in current technological settings is this end, one important aspect is the possibility for a user
how to design and develop interactive applications that can surrounded by multiple devices to freely move about and
be accessed through a wide variety of devices (ranging continue the interaction with the available applications
from small watches to very large screens, including various through a variety of interactive devices. Indeed, in such
types of smartphones, PDAs and Digital TVs). This is environments one big potential source of frustration is that
particularly important in Web application, which are the people have to start their session over again from the
most common applications. beginning at each interaction device change. Continuous
One important research area in this context is the model- task performance implies that interactive applications be
based approach, in which declarative descriptions of the able to follow users and adapt to the changing context of
user interface are used in order to avoid dealing with a use while preserving their state. Thus, migratory user
plethora of low-level implementation details associated interfaces require integrated solutions able to address state
with the wide number of available devices and persistence and user interface adaptation when the user
implementation languages. Despite such potential benefits, changes the device.
its adoption has mainly been limited to professional Model-based languages are utilized at design time to help
designers, but new solutions are recently emerging that are the user interface designer cope with the increasing
able to extend such approaches in order to achieve natural complexity of today’s applications and contexts. The
development by enabling end-users to develop or modify underlying user interface models are mostly used to
interactive applications still using conceptual models, but generate a final user interface code, which is then executed
with continuous support that facilitates their development, at run time. Nevertheless, approaches utilizing the models
analysis, and use [1]. at run time are receiving increasing attention. We agree
with Sottet et al. [8], who call for keeping the models alive
at run time to make the design rationale available and show
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
a solution for this purpose.
not made or distributed
Pre-proceedings for profit or
of the 5th International commercial
Workshop advantage
on Model Driven and that copies
Development of In the following we present some research work that
bear thisUser
Advanced notice and the
Interfaces full citation
(MDDAUI 2010):onBridging
the firstbetween
page. User
To copy otherwise,
Experience and
UIor
Engineering,
republish,organized
to post onat the 28th ACM
servers or toConference
redistributeon Human
to lists,Factors in prior
requires exploits model-based approaches for multi-device
Computing Systems (CHI
specific permission 2010),aAtlanta,
and/or fee. Georgia, USA, April 10, 2010. ubiquitous applications. We show how we have enriched a
software model-based platform for migratory user
Copyright © 2010 for the individual papers by the papers' authors. Copying permitted
for private and academic purposes. Re-publication of material from this volume interfaces with a new tool for desktop-to-mobile adaptation,
requires permission by the copyright owners. This volume is published by its editors. called parametric bidimensional semantic redesign. One of
1
41
its features is that it allows the end-users to customize the selected, data entered, …) and identifies the last element
adaptation process. We present some initial results and then accessed in the source device. Thus, when a logical version
discuss how we plan to extend them. of the interface for the target device is generated, it also
contains the state detected in the source device version so
that the user inputs (selections performed, data entered, …)
MIGRATORY USER INTERFACES are not lost. In the last phase, the user interface
Migration is the result of two main features: state implementation for the target device is generated and
persistence across multiple devices and adaptation to the activated remotely at the point corresponding to the last
device interaction resources. They have to be supported basic task performed in the initial device.
while users interact with the applications made available by
the intelligent environment. For this purpose, we have
designed and developed a migration architecture [5], which
supports a number of reverse and forward transformations
that are able to transform existing desktop Web applications
for various interaction platforms and support task
continuity. The basic assumption is that there exists a huge
amount of easily accessible content for desktop Web Figure 1. The main phases of the adaptation process.
applications, which can be processed and transformed to
support migratory interfaces, even across non-Web In the process of creating an interface version suitable for a
implementation languages. The advantage of this solution platform different from the desktop, we use a semantic
with respect to others (e.g. [4]) is that it does not require redesign module. This part of the migration environment
that the applications be implemented using a particular automatically transforms the logical description of the
toolkit in order to make them able to migrate. desktop version into the logical description for the new
platform. Therefore, the goal of this transformation is to
In this environment the client devices subscribe to the provide a description of the user interface suitable for the
migration service by running a migration client that new platform. This means that intelligent rules are used for
provides the environment with information regarding the adapting the description of the user interface to the new
device characteristics. The devices access Web applications platform taking into account its capabilities (e.g.: using
through the migration server, which includes proxy interface elements that are more suitable for the new
functionalities. Migration can be triggered either by the user platform) but ensuring at the same time that the support for
or it can be automatically triggered by the smart the original set of tasks is maintained. This solution allows
environment when some specific event (such as very low the environment to exploit the semantic information
battery or connectivity) is detected, or in a mixed solution contained in the logical description. In this case the
in which the environment suggests possible migrations and semantic information is related to the basic tasks that the
the user decides whether or not to accept them. user interface elements are expected to support.
When the user accesses the application through an This software architecture for migratory user interfaces
interaction platform other than the desktop, the server currently uses MARIA [7], a recent model-based language,
transforms its user interface by building the corresponding which allows designers to specify abstract and concrete user
logical description and using it as a starting point for interface languages according to the CAMELEON
creating the implementation adapted to the accessing Reference framework [2]. This language represents a step
device. Figure 1 shows how adaptation is obtained. There forward in this area because it provides abstractions also for
are three main phases: reverse engineering, semantics-based describing modern Web 2.0 dynamic user interfaces and
adaptation, and generation. In the first phase, the tool Web service accesses. In its first version it provides an
automatically builds the logical description of the accessed abstract language independent of the interaction modalities
page. It has rules able to handle HTML and CSS tags and and concrete languages for graphical desktop and mobile
associate them with the corresponding logical elements. For platforms. In general, concrete languages are dependent on
example, if DIV, or FIELDSET or IFRAME tags are used the typical interaction resources of the target platform but
then it recognises that there is a group of logically independent of the implementation languages.
connected elements in the page. We call the adaptation
module semantic redesign since its purpose is to change the In MARIA an abstract user interface is composed of one or
design still considering the interaction semantics of the multiple presentations, a data model, and a set of external
implementation elements that are specified in the functions. Each presentation contains a number of user
corresponding logical description. In addition to interface interface elements (interactors) and interactor compositions
adaptation, the environment supports task continuity. To (indicating how to group or relate a set of interactors), a
this aim, when a request for migration to another device is dialogue model describing the dynamic behaviour of such
triggered, the environment also takes the state of the user elements, and connections indicating when a change of
interface, which depends on the user input (elements presentation should occur. The interactors are classified in
abstract terms: edit, selection, only_output, control,
42
interactive description, etc.. Each interactor can be The elements that determine the cost of the interactors are:
associated with a number of event handlers, which can the font attributes (size, style, type), the vertical and
change properties of other interactors or activate external horizontal space required by a text, image dimensions,
functions. interline value, interactor type, and so on
END-USER ADAPTATION CUSTOMIZATION Figure 2 shows the user interface that allows end users to
In the research on migratory user interfaces, one issue that configure the adaptation process. The various parameters
we are considering is how to provide users with more are grouped according to the related user interface aspect
control on the migration process in order to improve its considered. For the fonts, it is possible to specify the
usability. In this context more control can mean various minimum and maximum font in the target device, and the
things. One important aspect is control on the rules that associated measure unit. For the radio buttons it is possible
drive adaptation to the various platforms (the most common to indicate whether they should be transformed into an
case is desktop-to-mobile adaptation). For example, the interactor that supports the same semantics but with using
adaptation engine is able to split the desktop pages when less space screen. In this case, it is possible to specify the
they require considerable amount of interaction resources threshold, in terms of number of choice options, which
but some users may like to have more control on the should trigger the transformation and the type of interactor
splitting algorithm. to use for its replacement. Similar parameters are available
for the list boxes. Other parameters concern the maximum
In particular, we have designed a new tool for adaptation:
number of characters for a text, maximum and minimum
Parametric Bidimensional Semantic Redesign. It supports
dimensions for images. These parameters determine the
adaptation from desktop-to-mobile devices and overcomes
cost of rendering a presentation. This cost is compared with
limitations of previous approaches in the area [6] because it
the overall sustainable cost in the target device, which is
allows users to configure the adaptation process and
provides more control on costs calculation and the given by the screen resolution multiplied by horizontal and
vertical tolerance. The higher the tolerance coefficient
adaptation results. For example, while previous solutions
values are, the more scrollable the generated user interface
calculated the screen space requested by the user interface
elements mainly in terms of its vertical use, the new will be. This means that end users have the possibility to
algorithm calculates both the horizontal and the vertical specify to what extent the adapted content will be scrollable
in the target device. The table tolerance provides an
consumption of screen space.
additional factor to consider when calculating the
The adaptation tool takes as input the concrete description sustainable cost. In practise, this means that when there are
of a desktop user interface in the MARIA language and tables, more scrolling will be acceptable before deciding to
goes through a number of steps. For each step a number of split the presentation.
specific rules are applied. First, it performs some basic
The customization interface also allows the user to indicate
transformations: if the user provides preferences regarding
two additional parameters: what type of scrolling
the minimum and maximum fonts for the target device then
it transforms all the textual content in order to fit in the (horizontal or vertical) to avoid has the priority, and the
splitting algorithm version to apply. Indeed, the tool
given range. Next, it calculates the cost of all the interactors
supports two ways to determine how splitting should be
and composition operators in the provided specification. If
performed. In both cases it analyses the cost of the
the resulting total cost is sustainable for the target device
then the corresponding logical description is generated composition operators (grouping or relation), which
includes those of the composed interactors, and the cost of
otherwise it starts the process to reduce the cost in order to
the tables (both data and layout tables). Then, the decision
make it sustainable. First, basic elements are adapted for the
of the set of elements to allocate to the newly generated
target device: the images are reduced in space while
mobile presentation is given in one case by the most
preserving their aspect ratio, some interactors are replaced
expensive element. In the other case the algorithm first
with others that are semantically equivalent but needs less
calculates what elements are able to make the current
screen space, long texts are reduced in such a way that the
part exceeding a limit is shown only on request, image and presentation sustainable by the target device if removed,
and then selects among them the one that has the lowest
text in tables are reduced in size. After these basic
cost. The rationale for this second option is that it allows
transformations the overall cost is calculated again and if it
is not yet sustainable by the target device then the part users to obtain a sustainable presentation by removing the
related to page splitting is activated. The purpose of this least amount of information possible, thus preserving as
much as possible the original design.
phase is to split the original desktop presentation into two
or more presentations, which are sustainable for the target In terms of results of the adaptation process we have
mobile device. For this purpose the algorithm considers the conducted a study comparing our tool with two publicly
interactor compositions, and associates some of them to available tools for desktop-to-mobile adaptation: Mowser
newly generated mobile presentations, removing them from (http://mowser.com) and Skweezer
the current presentation in order to decrease its overall cost. (http://www.skweezer.com). The results were encouraging
because our tool has shown to be more flexible since it
3
43
allows end users to customize the adaptation parameters In this paper we have presented first results that allow end-
and is able to adapt a higher number of types of interface users to customize desktop-to-mobile adaptation in order to
elements than the other two tools (e.g. tables and long texts change the results that can be obtained by automatic user
do not receive specific adaptation transformations with the interface generation.
other two tools).
We plan to further extend this work in various directions.
The customization user interface can be improved in order
to make the effects of the various customization parameters
more understandable. In addition, in this work we have
considered only desktop-to-mobile adaptation but other
types of transformations can benefit from the approach
proposed, e.g. graphical-to-vocal adaptation.
ACKNOWLEDGMENTS
This work has been supported by the EU project OPEN
(http://www.ict-open.eu/)
REFERENCES
1. Berti, S., Paternò, F., Santoro C., “Natural Development
of Ubiquitous Interfaces”, Communications of the
ACM, September 2004, pp.63-64, ACM Press.
2. Calvary, G., Coutaz, J., Bouillon, L., Florins, M.,
Limbourg, Q., Marucci, L., Paternò, F., Santoro, C.,
Souchon, N., Thevenin, D., and Vanderdonckt, J. 2002.
The CAMELEON reference framework. CAMELEON
Project. Deliverable 1.1
3. Lieberman, H., Paternò, F., Wulf W. (eds), End-User
Development, Springer Verlag, ISBN-10 1-4020-4220-
5, 2006.
4. Melchior, J., Grolaux, D.,Vanderdonckt, J.,Van Roy, P.,
A Toolkit for Peer-to-Peer Distributed User Interfaces:
Concepts, Implementation, and Applications, pp. 69.78,
EICS’09, July 15–17, 2009, Pittsburgh, Pennsylvania,
USA.
5. Paternò, F., Santoro, C., Scorcia, A., Ambient
Intelligence for Supporting Task Continuity across
Multiple Devices and Implementation Languages, the
Computer Journal, the British Computer Society, 2009.
6. Paternò, F., Santoro, C., Scorcia A Automatically
Adapting Web Sites for Mobile Access through Logical
Descriptions and Dynamic Analysis of Interaction
Resources. AVI 2008, Naples, May 2008, ACM Press,
pp. 260-267.
7. Paternò F., Santoro C., Spano L.D., "MARIA: A
Universal Language for Service-Oriented Applications
Figure 2. The customization user interface.
in Ubiquitous Environment", ACM Transactions on
Computer-Human Interaction, Vol.16, N.4, November
2009, pp.19:1-19:30.
CONCLUSIONS
Ubiquitous environments call for adaptive systems in order 8. Sottet J., Ganneau V., Calvary G., Coutaz J., Demeure
to adapt to the varying interaction resources. Model-based A., Favre J., Demumieux R.: Model-Driven Adaptation
approaches can provide useful support in this context. for Plastic User Interfaces. INTERACT (1) 2007: 397-
However, there is a need for providing users with more 410.
control on ubiquitous interfaces, according to the end-user 9. Weiser M., "The Computer for the 21st Century" -
development paradigm. Scientific American Special Issue on Communications,
Computers, and Networks, September, 1991.
44