=Paper=
{{Paper
|id=None
|storemode=property
|title=Evaluation of User Interface Adaptation Strategies in the Process of Model-Driven User Interface Development
|pdfUrl=https://ceur-ws.org/Vol-617/MDDAUI2010_Paper05.pdf
|volume=Vol-617
}}
==Evaluation of User Interface Adaptation Strategies in the Process of Model-Driven User Interface Development==
https://ceur-ws.org/Vol-617/MDDAUI2010_Paper05.pdf
Evaluation of user interface adaptation strategies in the
process of model-driven user interface development
Kai Breiner, Volkmar Gauckler Marc Seissler Gerrit Meixner
University of Kaiserslautern University of Kaiserslautern German Research Center for
Software Engineering Research Center for Human-Machine- Artificial Intelligence (DFKI)
Group Interaction Trippstadter Str. 122
Gottlieb-Daimler Str. Gottlieb-Daimler Str. 67663, Kaiserslautern,
67663, Kaiserslautern, Germany 67663, Kaiserslautern, Germany Germany
breiner@cs.uni-kl.de, Marc.Seissler@mv.uni-kl.de Gerrit.Meixner@dfki.de
volkmar.gauckler@gmx.net
ABSTRACT concerning ubiquitous computing – also in production
In this paper, we describe the evaluation of our prototype environments – is becoming a reality [10].
Universal Control Device (UCD), which enables the control
of various devices in modern dynamic production In today’s production environments, technical devices often
environments, while being able to adapt itself to the current stem from multiple vendors using heterogeneous user
configuration of the environment. While it is hard to apply interfaces that differ in terms of complexity, look&feel, and
traditional user interface design heuristics in recent interaction styles. Such highly complex and networked
paradigms – such as Ambient Intelligence – there is a technical devices or systems can provide any information at
demand for suitable compensation strategies addressing any time and in any place. This advantage can turn out to be
usage errors, which can be met by applying an adequate a disadvantage when information is not presented properly
adaptation strategy. In a pilot study, we gained experience according to the users’ needs. This leads to problems,
regarding differences in the performance of selected especially concerning the usability of the user interface.
adaptation strategies in the case of our demonstrator. The level of acceptance of a user interface largely depends
on its ease and convenience of use. A user can work with a
Author Keywords technical device more efficiently if the user interface is
Universal Control Device, Adaptation Strategies, tailored to the users’ needs, on the one hand, and to their
SmartFactory. abilities on the other hand. Therefore, providing
ACM Classification Keywords information in a context- and location-sensitive manner
H5.2. Information interfaces and presentation (e.g., HCI): (depending on user, situation, machine, environmental
Evaluation/methodologie, Prototyping, User-centered conditions, etc.) has to be ensured.
design. Hence, in the following we will give a short introduction to
INTRODUCTION the SmartFactoryKL, which serves as a demonstration
The ongoing technological development of environment for future intelligent production environments,
microelectronics and communication technology is leading and a Universal Control Device (UCD), which provides
to more pervasive communication between single devices holistic control to various devices in such environments.
or entire pervasive networks of intelligent devices (smart Further, we give a brief introduction to user interface
phone, PDA, netbook, etc.). Especially industrial devices adaptation, usage errors, as well as to their compensation.
and applications can take advantage of modern smart After presenting our idea of how to approach compensation
technologies, e.g., based on ad-hoc networks, dynamic by using adaptation strategies, we describe the set-up of the
system collaboration, and context-adaptive human-machine corresponding controlled experiment and the preliminary
interaction systems. The vision of Mark Weiser [1] results of the pilot study conducted. We conclude with the
interpretation of how the results contribute towards our
hypothesis.
Permission to make digital or hard copies of all or part of this work for KL
personal or classroom use is granted without fee provided that copies are SmartFactory
Pre-proceedings of the 5th International
not made or distributed for profit orWorkshop on Model
commercial Drivenand
advantage Development
that copies Besides these aspects, modern production environments are
ofbear
Advanced User Interfaces
this notice (MDDAUI
and the full citation2010): Bridging
on the between
first page. To User
copyExperience
otherwise, characterized by a modular layout. Entire modules can be
and
or UI Engineering,
republish, organized
to post at the 28th
on servers ACM
or to Conferencetoonlists,
redistribute Human Factorsprior
requires in
Computing Systems (CHI 2010), Atlanta, Georgia, USA, April 10, 2010. replaced or reorganized. Furthermore, these environments
specific permission and/or a fee.
CHI 2009, April 4–9, 2009, Boston, Massachusetts, USA.
are able to react to errors occurring in the production
Copyright © 2010 for the individual papers by the papers' authors. Copying process (e.g., device malfunction) and to dynamically
Copyright 2009 ACM 978-1-60558-246-7/09/04...$5.00.
permitted for private and academic purposes. Re-publication of material from this
volume requires permission by the copyright owners. This volume is published by reorganize parts of the process in order to ensure the
its editors. production process. Thus, this also affects the user who
17
interacts with the individual devices – the user’s workflow Basically, there are two kinds of operating errors that may
will change, or devices will not be available anymore. lead to a failure of the system.
Serving as a demonstration environment, the In the first case, there are slips. Here, the user had the
SmartFactoryKL in Kaiserslautern, Germany is able to correct intention but executed the task in the wrong way.
simulate such a process. In previous work, we developed a Most often this is caused by poor physical skills, or by the
UCD, which is able to provide access to various devices of user interface just being just inadequate for use (e.g.,
the SmartFactoryKL [3][4]. buttons too small). In the second case – which is more
interesting in our example – there are errors. Errors are
Universal Control Device (UCD)
As a result of a model-driven process, the user interface of characterized by the user having the wrong intention while
the UCD is being generated at run-time [2]. Starting with a executing the task. This is caused by a misunderstanding of
topological description of the environment, enriched with the user interface (e.g., if the user interface is offering
user tasks on the single devices and information about how control over devices that are not available physically).
to address the single devices, this information is sufficient
for generating a functional user interface [2][4]. In order to Usage Error Cause
remain functional to the user, this user interface has to
correspond to the current configuration of the environment
and is additionally restricted to the functionality as
specified in the underlying model. During field studies,
faced by the need to adapt the user interface, we Compensation
encountered the demand for a systematic strategy that
would support the user as much as possible [3]. This was Adapt to Misunder-
Error
the trigger for a study on adaptation, which we will describe Environment standing
in the following. Slips Adapt to User Skills
ADAPTATION
Figure 1. Relationship between usage errors, their causes, and
After giving a brief overview of different types of compensation.
adaptation – their properties as well as their impact on the
users’ workflow – we will elaborate types of usage errors Compensation
resulting from static user interfaces and how adaptive user Depending on the type of usage error, there are different
interfaces contribute to the compensation of such errors. ways to compensate in order to minimize the effect. Slips,
Static versus Dynamic User Interfaces which are caused by poor skills of the user, can be
On the one hand, one important usability quality attribute is prevented by adapting the user interface to the user. In case
memorizability. The ease to remember helps users to speed of our application domain – intelligent production
up the process of interacting with the user interface [5][6]. environments – we are dealing with predefined user roles
Since humans have a very visual memory, the way a certain and user groups and are therefore able to tailor the user
user interface is structured is essential for finding items interface to the needs of these user groups.
faster. After a while, users form a coherent model of the In dynamic environments, errors can be prevented by
user interface and are able to recall how to execute their adhering to design heuristics as mentioned earlier. The
workflow. If the system (and therefore the user interface) system has to represent the current configuration of the user
changes frequently, the user will not be able to form such a interface, while supporting the development of a mental
model [7][8]. On the other hand, there are user interface model. Hence, a method of adaptation needs to be chosen
heuristics demanding that the user interface matches the that contributes to these heuristics.
real world [6]. In order to provide control over devices in
dynamic environments – such as the SmartFactoryKL – it is Types of Adaptation
extremely vital to match the user interface to the real world There are different methods of how an adaptation of the
in order to remain functional. user interface can be executed. These methods differ in
their way of execution as well as in their degree of intrusion
These simple rules about how to develop a user interface into the users’ workflow. In case of the UCD, a simple
appear to be contradictory for future information systems in adaptation scenario would be the appearance or
our application domain. Hence, there is a need for an disappearance of devices in the device selection list. In the
adaptation strategy that does not violate usability quality following, we will refer to this example while giving details
attributes leading to usage errors. about the individual methods.
Usage Errors The first method – which was implemented initially and led
In general, for each kind of usage error, there is a basic to the investigation described in this paper – is ad-hoc
cause (see Figure 1). Using the right compensation adaptation. Here, devices are added or removed from a
technique – such as adaptation of the user interface – the device list according to the physical status of the respective
users can be actively supported in preventing usage errors.
18
devices. Unless a regular user permanently observes the hypotheses are tailored to the specific set-up of the
device list, he or she will not notice any change. controlled experiment.
Furthermore, this will be distracting for the user, because
H1. Effectiveness – completion rate
the structure of the user interface changes without We assume that, on average, at least 85,6% of the test tasks
notification. can be completed without help.
Providing more information about the system state leads to Explanation: This hypothesis assumes that test persons can
the second adaptation method – notification. Now, the user deal with the user interfaces and have understood their tasks
interface provides information about the change and and therefore can complete at least 6 out of the 7 tasks.
therefore supports the user in adapting his or her mental
model. In case of the device list example, we implemented H2. Effectiveness – assistance
this method by applying the so-called instant-messenger 100% of the given tasks can be completed (with help – if
metaphor [9]. This means that new or defective devices will needed).
be emphasized graphically, which provides information for Explanation: The user interface ensures consistency
the user to understand the current state of the system. between environment and visualization independent of the
However, the user may just overlook the notifications if he adaptation strategy. Therefore, it should be possible to
or she is distracted, and then the system cannot provide complete each task.
further support.
H3. Efficiency – strategy performance
Thus, we implemented a third adaptation strategy, which Confirmation outperforms notification, which outperforms
aims at confirmation by the user. Here, the user has to ad-hoc adaptation.
actively confirm the change to the user interface. Referring
to the device list of the example above, this strategy was Explanation: On the basis of the different attributes
implemented in terms of a dialog box asking for elaborated earlier, we assume this ranking according to the
confirmation from the user that he or she has actually performance of the different strategies.
noticed the adaptation of the system. Thus, the system can EVALUATION – PILOT STUDY
be almost sure that the user is aware of the adaptation and is To evaluate these hypotheses, we decided to conduct a
supported in the presumably most adequate way. A controlled experiment. We implemented three instances of
negative side-effect of this strategy is that the confirmation our model-driven process [2][3][4], including the
dialog may distract the user during his/her regular corresponding adaptation strategies. The test persons had to
workflow. complete seven tasks on the user interface, while the
Compensation simulated production environment always reconfigured
Each of the adaptation strategies differs as to how much it between task 2 and task 3. The idea was that the adaptation
contributes to the compensation of possible usage errors. should only affect task 4, which had to be executed in a
different way (as a result of the adaptation) than indicated at
Ad-hoc adaptation ensures consistency between the first. The other tasks served as an indication that the test
controllable environment and the corresponding user persons perform in a similar way. We conducted a pilot
interface in order to prevent usage errors with respect to study with several computer science students. All six test
outdated user interfaces. But this approach provides no persons were between 21 and 34 years old.
active support to the user at all.
After being asked for personal statistical information, the
Besides consistency with the current configuration of the test persons got a thorough introduction to the production
environment, notification provides limited feedback to the industry domain. They were provided with a detailed
user (e.g., by visually emphasizing new devices). Due to the explanation of the simulated production environment as
fact that this communication with the user is unidirectional, well as of our Universal Control Device. After execution of
such notifications can be easily overlooked by the user. task 2, we initiated the reconfiguration of the simulation
Confirmation provides all the functionality of the first two and, depending on the strategy used (of which the test
approaches on the one hand and demands confirmation by persons were not aware) the user interface adapted itself,
the user on the other hand. Thus, the user interface is aware notified the user, or demanded confirmation. The strategies
of the fact that the user has recognized the new were distributed equally between the tests. Due to the
configuration and can therefore proceed with the regular adaptation of the environment, the user should not be able
functionality. to complete task 4 in the way the task description called for.
One of three redundant pumps was removed from the
Hypothesis system. Here, the user should conclude that (as displayed in
According to the diverse properties of these strategies, we the documentation of the simulated environment) there are
wanted to investigate which is the most adequate one in the various ways to complete this task and ask the moderator if
case of our model-driven approach and therefore we this is possible. The test was completed with a survey about
formulated hypotheses that we are going to verify or falsify the subjective properties of the user interface.
initially in a preliminary study described below. The
19
216 there are special requirements. According to a shown
dissonance in user interface design, when being applied in
173
these environments, we have shown there is a special need
for compensating usage errors. This can be achieved by
130
systematically integrating adaptation strategies into the
time [s]
model-driven development process. Since multiple
86
strategies exist, which provide different user experience, the
performance with respect to our demonstrator was
43
evaluated in a pilot study of a controlled experiment. First
results show that there are differences in the performance
00
1 2 3
Tasks
4 5 6 7 and therefore some of our hypotheses could be verified.
Figure 2. The performance of the 7 tasks (median, max/min ACKNOWLEDGMENTS
and quantile). Our work as well as the GaBi project is funded in part by
the German Research Foundation (DFG).
Results
Figure 2 shows the results of the performance of the single REFERENCES
tasks. As we assumed before conducting the experiment, all 1. Weiser, M. The computer for the 21st century. Scientific
tasks (the control tasks) except task 4 were performed American, 265, 3 (1991), 94-104.
similarly. Referring to the standard deviation, which is an 2. Breiner, K., Görlich, D., Maschino, O., and Meixner, G.
average of 16 seconds in case of the control tasks and 38 Towards automatically interfacing application services
seconds in case of task 4, the difference in performance can integrated in an automated model based user interface
be easily identified. Thus, we conclude that the results of generation process, Proc. of the 4th International
this preliminary experiment are representative. Workshop on Model-Driven Development of Advanced
When executing task 4, the test persons recognized the User Interfaces (MDDAUI), CEUR Workshop
redundancy of the three pumps and asked if they could use Proceedings Vol-439 (2009).
another pump, which was intended. 3. Breiner, K., Görlich, D., Maschino, O., Meixner, G., and
Zühlke, D. Run-Time Adaptation of a Universal User
Discussion of the Results
Interface for Ambient Intelligent Production
All test persons were able to complete all tasks (with help
Environments. Proc. of the 13th International
in case of task 4), which confirms hypothesis H1. Since all
Conference on Human-Computer Interaction (HCII-09),
of them needed help when executing task 4 and only during
LNCS Vol. 5613 (2009), 663-672.
task 4, the estimated 85,6% of hypothesis H2 was almost
exactly confirmed. 4. Breiner, K., Görlich, D., Maschino, O., and Meixner, G.
Automatische Generierung voll funktionsfähiger
For ah-hoc adaptation, the execution of task 4 took an mobiler Bediensoftware aus Benutzungs- und
average of 117 seconds, for notification 125 seconds, and Funktionsmodellen. Proc. of Informatik, LNI Vol. P-154
for confirmation 89 seconds. Hence, hypothesis H3 cannot (2009), 2210-2215.
be entirely confirmed, as ad-hoc adaptation outperformed
notification, but still confirmation outperformed both of the 5. Gould, J. D. and Lewis, C. 1985. Designing for
other strategies. usability: key principles and what designers think.
Communications of the ACM, 28, 3 (1985), 300-311.
Threats to validity
Because all the test persons were only computer science 6. Nielsen, J. Ten Usability Heuristics.
students, this may have had an effect on the result. But on www.useit.com/papers/heuristic/heuristic_list.html.
the other hand, those students were already familiar with 7. Norcio, A. F., and Stanley, J. Adaptive Human -
our previous work (without knowing the content of the Computer Interfaces. NRL Report 9148, Naval Research
experiment), which could also be a good simulation of Laboratory (1988).
domain knowledge. Nevertheless, the experiment needs to 8. Mitchell, J. and Shneiderman, B. Dynamic versus static
be conducted (and will be) using production industry menus: an exploratory comparison. SIGCHI Bulletin,
domain experts. The most important threat to validity to 20, 4 (1989), 33-37.
mention is the fact that this was only the pilot study for the
described experiment. This means that the sample was too 9. Lee, L. and Johnson T. URCousin: Universal Remote
small and therefore has no real statistical evidence, but it Control User Interface. Proc. of the Human Interface
serves as a first indication as to whether an investigation Technologies Conference (2006).
according our idea would make sense. 10. Zuehlke, D.: SmartFactory – From Vision to Reality in
CONCLUSION
Factory Technologies. Proc. of the 17th International
When dealing with user interfaces in highly dynamic Federation of Automatic Control (IFAC) World
environments, such as intelligent production environments, Congress (2008), 82-89.
20