=Paper=
{{Paper
|id=Vol-214/paper-7
|storemode=property
|title=GuiBuilder - A Tool for Model-Driven Development of Multimedia User Interfaces
|pdfUrl=https://ceur-ws.org/Vol-214/paper7.pdf
|volume=Vol-214
|dblpUrl=https://dblp.org/rec/conf/models/SauerDGH06
}}
==GuiBuilder - A Tool for Model-Driven Development of Multimedia User Interfaces==
https://ceur-ws.org/Vol-214/paper7.pdf
GuiBuilder—A Tool for Model-Driven Development of
Multimedia User Interfaces
Stefan Sauer Marcus Dürksen Alexander Gebel Dennis Hannwacker
s-lab – Software Quality Lab
University of Paderborn
Warburger Str. 100
D-33095 Paderborn, Germany
Phone: +49 (0) 52 51 – 60 53 90
{sauer | mduerksen | agebel | dhannwacker @s-lab.upb.de}
ABSTRACT Model-based development of user interfaces promotes structuring
GUI builder tools are widely used in practice to develop the user of the resulting implementation and allows developers to prevent
interface of software systems. Typically they are visual errors or to detect errors earlier and more easily by already
programming tools that support direct-manipulative assembling of analysing the model of the user interface. The models can also be
the user interface components. We have developed the tool used as documentation and for guiding the maintenance of the
GuiBuilder to facilitate model-driven development of graphical software system. Model-driven development even goes a step
and multimedia user interfaces. User interface developers can further by automatically generating the executable user interface
model the structure of the user interface with compositional from the model.
presentation diagrams and its behaviour with hierarchical The objective of this work is to develop a model-driven and tool-
statechart diagrams. GuiBuilder then supports the transformation based development technique for graphical and multimedia user
of the model to Java, i.e., the generation of a working user interfaces (UI). The model of the UI combines structural and
interface and the simulation of the modelled behaviour. behavioural aspects. The model-driven development of the UI is
Interactive sessions with the user interface can be recorded and then supported by a tool called GuiBuilder. GuiBuilder provides
replayed. developers with an editor for UI modelling and an execution
environment for UI simulation. A prototype user interface can be
Categories and Subject Descriptors generated from the model, executed and tested. External tools can
D.2.2 [Software Engineering]: Design Tools and Techniques – also connect to the simulation and are notified about the
Computer-aided software engineering (CASE), Object-oriented simulation progress. Simulation runs can be recorded and
design methods, User interfaces; D.2.6 [Software Engineering]: replayed. The simulation logs can also be used to support
Programming Environments – Graphical environments, regression testing based on the capture-replay paradigm.
Interactive environments A number of model-based approaches have been proposed in past
years to deal with user interface modelling at different levels of
General Terms abstraction (see e.g. [6]). GuiBuilder is targeted towards concrete
Design, Human Factors, Languages user interface modelling. The idea of combining statechart and
presentation diagrams originally stems from the OMMMA
approach [5]. Statechats have also been used in [1] for describing
Keywords UI behaviour. UsiXML (e.g. [7]) uses graph transformations
Model-driven development, user interface, prototype generation, instead. It provides a variety of GUI-Elements which are currently
capture-replay not completely supported by GuiBuilder due to its early
development state. In MOBI-D [4] the process of constructing a
1. INTRODUCTION GUI is guided and restricted by domain and task definitions,
Graphical user interfaces of (multimedia) software applications which are the building blocks of user interfaces in MOBI-D. A
provide users with the presentation of information and interaction UML-based approach towards model-driven development of
capabilities with (media) content and functionality. The user multimedia user interfaces is described [2]. Recently, model-
interface is a complex part of the overall system and often driven development of user interfaces has attracted wider interest
requires development effort comparable to building the in the research community [3].
application functionality itself. Thus, user interface development
In the next section, we will introduce the different models that are
should follow a model-based software development process and
implemented in GuiBuilder and their interplay. Section 3 presents
be supported by adequate tools in order to master its complexity.
the tool GuiBuilder. We draw conclusions and outline future
GUI builder tools that merely support visual programming of the
perspectives in Section 4.
user interface are overstrained with this task.
2. MODELS OF GuiBuilder also refined by the presentation diagrams that are assigned to the
The model of the multimedia user interface in GuiBuilder consists respective substates.
of two parts: the presentation model and the dynamics model. The
State1
presentation model captures the structure and layout of the user
interface, the dynamics model uses UML statecharts to specify
the behaviour of the UI. Dynamic behaviour is enacted by user State2
interaction or other events that cause a change of state in the user
interface (and the application). Events that are caused by user
interaction are modelled as signals which can be handled by the
presentation elements. Signals can also be sent as the actions of
triggered state transitions.
The basic concept of the compound model is to assign a Figure 2. Presentation diagrams can be assigned to
presentation design to a state, which describes the structure and hierarchical states, new presentation elements can be added
layout of the user interface while in that state. At any point in for substates or properties of existing elements be modified.
time, the GUI of an application is in a specific, possibly complex
state. An event occurrence causes a state change and thus a The composition of the presentation diagrams according to the
change of the presentation. state hierarchy works as follows:
The presentation model consists of presentation elements (see First, presentation diagrams are stacked on top of each other. The
Figure 1). Typically they are graphical elements that are part of order is determined by the state hierarchy: presentation diagrams
the application’s presentation. Such elements can e.g. be of substates are put on top of presentation diagrams of their
geometric shapes, widgets, or graphics elements for rendering superstates. The former are intended to be the more specific.
images or video. In addition to graphical elements, audio elements Their presentation elements override (cover) the presentation
can be included for playing music or sound effects. The elements of the latter. For concurrent states, an order is not
presentation elements have properties which can be assigned with defined.
values. The properties depend on the type of presentation element
Secondly, since presentation diagrams can not only contain new
and determine the presentation of the element. The types of
presentation elements, but also property changes (i.e., modify the
presentation elements are organized in a class hierarchy.
properties of presentation elements contained in presentation
diagrams that are assigned to superstates), the modified value also
overrides the ‘inherited’ value. All presentation elements that are
introduced in presentation diagrams of the superstates of a state
can be altered by modifying their property values. A property
change thus specifies the modification of a property value of an
inherited presentation element in a substate (see Figure 3).
Figure 1. A presentation consists of presentation elements Consequently, a hierarchical presentation can be interpreted as a
(left), where each presentation element is characterized by its list of modifications where the instantiation of a presentation
property values (right). element is a specific case. This list can then be processed to
construct and compose the actual presentation for a particular
The presentation elements within one presentation diagram are
state: for each presentation diagram, the list of modifications is
ordered. The topmost element is upfront and possibly covers parts
processed, whereby the order of the lists of different presentation
of other elements if they overlap.
diagrams is determined by the hierarchical state structure from
If the GUI is in a simple state, the presentation is a composition of superstates to substates.
presentation elements with their property values. The presentation
With respect to execution semantics, this means that when a state
is completely described by the presentation diagram that is
is left, the modifications of its presentation diagram to the user
assigned to this state.
interface become ineffective and are replaced by the
However, it is also possible to assign presentation diagrams to modifications of the presentation diagram of the successively
complex states in our model. Complex states allow us to entered state. Modifications of the presentation diagram of the
hierarchically structure the state of the user interface. The actual possibly still active superstates remain unaffected, yet may be
presentation is then composed from the presentation diagrams that overridden.
are assigned to the current simple state and all its parent states,
Structured specification of a user interface is facilitated by this
where the complex states can even be concurrent (i.e., AND-
composition mechanism. GUIs typically contain a limited number
superstates). Figure 2 shows an example, where the presentation
of fundamentally different views which are then subject to a
diagrams Layout1 and Layout2 are assigned to State1 and its
larger number of smaller (local) modifications for representing the
substate State2, respectively.
particularities of different states within the overreaching context.
The actual composition of the presentation is determined by the Our incremental composition mechanism eases the specification
hierarchical structure of the statechart diagram. If the behaviour of such modifications and prevents the developer from having to
of a superstate is refined by substates, the assigned presentation is specify the complete presentation design for each, even simple
modification. The UI design thus requires less effort and the UI
models become easier to extend and modify, especially since 3.2 Model Validation
redundancy is limited and controlled. The GUI editor calls the validator to validate the correctness of
the edited model. The diagrams of the dynamics model have to be
valid UML statechart diagrams. In addition, we require that the
specified behaviour is deterministic. Thus, the statechart diagrams
are validated before code is generated from them by the generator
function and the simulation can be started. To effectively support
the developer, we provide syntax-directed editing to prevent from
fundamental syntactic errors and static model analysis to detect
more complex and context-sensitive problems. For example,
missing start events or non-deterministic transitions are identified
by our model analysis. Two categories of problems, errors and
warnings are recorded and presented to the developer in the
central problems view, in the outline view (see Figure 4), and
directly at the relevant modelling elements in the editor view. The
identified problems are accompanied by correction procedures
(i.e., quick fixes). Only after all errors have been resolved, the
Figure 3. Stepwise modification of a presentation element generation can be enacted. Warnings need not to be resolved;
through hierarchical presentation diagrams. however, they should not be ignored since they mark weaknesses
In addition to presentation, interaction also profits from the of concept or style within the model. Thus, the static analysis
incremental specification. User interaction results in events which supports both the syntactic correctness of the model and its
are received by presentation elements as signals. Since signals are quality in accordance to modelling guidelines.
properties of the presentation elements as well, they can be
‘inherited’ and modified like presentation properties.
Functionality can thus be adapted in the same way by modifying
the signal specification.
3. GuiBuilder—THE TOOL
GuiBuilder has been developed as a plug-in of the Eclipse tool
environment and platform. We used the Plug-in Development
Toolkit PDT for its implementation and the Graphical Editor
Framework GEF for implementing the graphical editor of the
GuiBuilder plug-in.
GuiBuilder supports user interface software developers in the Figure 4. Problems are marked at the causing model elements.
development of multimedia user interfaces. Audio and video can Since the static analysis is the powerful core of the validation
be integrated in the presentation of the application that is module, the syntax-directed editing restrictions can be kept low,
developed. In the current version of GuiBuilder, executable UIs not to unnecessarily hinder the flexibility of model editing. For
are generated from the model and executed using Java SWT, and example, inconsistencies or incorrectness can be temporarily
the Java Media API is deployed for rendering of multimedia tolerated as long as the developer does not want to start the
artefacts. prototype generation process.
The main view of GuiBuilder is the GUI editor. Additional views Despite the wide range of checks in the static analysis, some
of GuiBuilder are the Eclipse standard views problems view, problems can still only be detected during dynamic analysis.
outline view, and properties view as well as the presentation view. Dynamic analysis is integrated with the simulation and executed
The problems view lists the detected errors and warnings. The at runtime. Dynamic errors that are detected then are e.g. infinite
outline view presents an outline page for each window of the GUI loops or non-deterministic behaviour. Such errors cause the
editor when it is selected. The properties view shows properties of termination of the simulation run.
a selected model element (statechart element or presentation
element). Properties can be edited directly in the properties view
or in an explicit properties dialog. The presentation preview is a 3.3 Generation and Simulation
GuiBuilder-specific view that presents a preview of the The simulator view shows the simulated GUI. The GUI editor
presentation (see Figure 6). passes the GUI model via a generator function to the simulator
view. The generator function flexibly implements the
transformation rules to build a prototype UI from the UI model. It
3.1 Editor can be replaced for generating a different target language or for
The editor of GuiBuilder is a graphical tool that supports the tailoring of the generation results.
direct-manipulative construction of dynamics and presentation
diagrams. The GUI editor is a multi-page editor that can manage The simulator view starts the simulation in the simulator and
windows of two different types for statecharts and presentation registers the GUI editor with the simulator. The simulator then
layouts, respectively. notifies the GUI editor about state changes.
Simulation of the user interface is accomplished by interpreting and specifically respond to their occurrence. With this mechanism
the model. The GUI simulator uses a statechart simulator which it is possible to actually control an application. Besides, the
interprets the statecharts of the dynamics model. Connected simulation recorder that we developed uses this mechanism for
objects are notified by the statechart simulator about state recording a simulation run. The recorder logs the simulation
transitions and triggered actions (signals). The GUI simulator execution. The recorded log can later be used to replay the
constructs the composite presentation view for the active state simulation or to do regression testing after the UI model has been
configuration and passes it to the simulator view of GuiBuilder. modified.
The simulator view renders the current GUI view. The user can
External tools can themselves send signals to the simulation and
then start the simulation in the simulation view, and the simulator
raise events to change the state of the simulated GUI. Thus, the
executes the generated GUI. Events can be raised either by
GUI can react to application or external events too.
interacting with the simulated GUI elements directly or by using
the ‘remote control’ that we implemented as an external plug-in. While a simulation is running, the editor view of GuiBuilder
It can be operated remotely to generate the required signals. highlights the current state of the dynamics model in its statechart
(green states in Figure 5). Thus, dynamic information is fed back
into the model representation and can be used e.g. for model
debugging.
4. CONCLUSIONS
We have integrated the model-driven development paradigm with
the GUI-builder tool concept. This provides user interface
developers with a tool for constructing multimedia user interfaces
in practice. The UI model consists of presentation and dynamics
models from which a prototype user interface can be generated
and simulated.
In a next step, we plan to further improve the capabilities of
multimedia processing by extending the dynamic model to deal
with timed procedural behaviour. We also want to demonstrate
the flexibility of the transformation approach by tailoring the
generator function to different target representations.
Additional information about GuiBuilder can be found at
Figure 5. Simulations can be tracked in the model. http://www.i3team.de/projects/GuiBuilder/
The interpretative approach has the advantage that the user
interface model can be altered at runtime, and these changes can 5. REFERENCES
directly influence the succeeding simulation behaviour. [1] Horrocks, I. Constructing the User Interface with
GuiBuilder provides this functionality in a separate hot-code Statecharts. Addison-Wesley, 1999.
replacement mode.
[2] Pleuß, A. Modeling the User Interface of Multimedia
Applications. In Proc. 8th International Conference on
Model Driven Engineering Languages and Systems
(MoDELS 2005). LNCS 3713, Springer 2005, 676-690.
[3] Pleuß, A., Van den Bergh, J., Hußmann, H., and Sauer, S
(Eds.). MDDAUI '05, Proc. of the MoDELS'05 Workshop on
Model Driven Development of Advanced User Interfaces,
CEUR Workshop Proc. 159. CEUR-WS.org, 2005.
[4] Puerta, A.R. A Model-Based Interface Development
Environment. IEEE Software 14, 4 (July/Aug. 1997), 41-47.
[5] Sauer, S. and Engels, G. UML-based Behavior Specification
of Interactive Multimedia Applications. In Proc. IEEE
Symposium on Human-Centric Computing Languages and
Environments (HCC‘01). IEEE Comp. Soc. 2001, 248-255.
[6] van Harmelen, M. (ed.). Object Modeling and User Interface
Design. Addison-Wesley, 2001.
[7] Vanderdonckt, J., Limbourg, Q., Michotte, B., Bouillon, L.,
Figure 6. The GUI of GuiBuilder.
Trevisan, D., and Florins, M. UsiXML: a User Interface
External tools and other Eclipse plug-ins can connect to the Description Language for Specifying Multimodal User
simulator and are thus notified about state changes in the Interfaces. In Proc. W3C Workshop on Multimodal
simulated model. They can assign specific actions to the signals Interaction WMI'2004. http://www.usixml.org