Deriving User Interfaces from Task Models Andreas Wolff, Peter Forbrig University of Rostock Albert Einstein Str. 21 D-18069 Rostock {Andreas.Wolff, Peter Forbrig}@uni-rostock.de ABSTRACT certain role using tools. Additionally, some tasks have to be This position paper presents our view on model driven user executed on devices with certain features only. In an abstract way interface development and relates it to other approaches. Our a task can be considered as specification of the following form: methodology is based on task models that are attributed and Task=(Goal, Sub-tasks, Temporal Relations, Pre-conditions, Post- merged with a navigational model to derive user interface models. conditions, Artifact, Tool(s), Role(s), Device). The notation of A toolset to support this development approach is introduced concurrent task trees (CTT) has nearly become a standard in which is well-integrated and itself based on the eclipse modeling representing tree structures of tasks. We also make use of an framework EMF. extended CTT-specification for designing applications as task models. 1. INTRODUCTION Our development strategy is focused on WIMP-style user Model-based software development is becoming more and more interfaces and possibly limited to those. Support for multi-modal popular because of its advantages in managing different versions or 3-dimensional systems is not actively researched. A core aspect of software for different platforms. The term model-based is today of window-based user interfaces to interactive systems is their often only identified as a part of the object-oriented development navigational structure. strategies related to UML. However, the origins of using model specifications during software development were task-based We specify this navigation structure by a special graph, which is ideas. Originally such task models were used to specify the called dialog graph. Dialog graphs consist of nodes, mostly behavior of people. interpreted as views, and of transitions between those nodes. There are different node types that distinguish views in For several years our group has been working on combining complexity, hierarchy and modality. Transitions are directed object-oriented and task-based methods. Our research is especially relations from an element of a view to another view or element. focused on using patterns for this purpose. Transitions reflect navigational aspects of user interfaces. Typical Model-to-Model transformations based on Eclipse Foundation’s elements of a view are tasks. EMF [5], an implementation of W3C MOF meta-model, have The concept of dialog graphs may reflect different abstraction been used in many fields of research and business. This paper levels. It is not limited to platform independent models (PIM) or illustrates another appliance. Two EMF models are used as abstract user interfaces either. Therefore we use dialog graphs as transformation source. One is describing the application as a task one transition point from PIM to platform specific models (PSM). model and another EMF model describes the possible and allowed Allowing the assignment of differing dialog graphs to one task navigation through an envisioned UI of this application. Typical model is one possibility of adaptation to context-of-use within our WIMP-style user-interfaces may, reasonably well, be described in development methodology. this manner. The tools developed for our approach use openArchitectureWare (oAW) [9] workflow templates to combine The combination of task model and navigational structure is the task model and dialog graph and transform those models into a basis to generate an abstract user interface model (AUI). AUIs working mock-up of a user interface prototype or into a abstract describe the internal layout of each view, i.e. the relative user interface. positioning of tasks within it. Furthermore the class of user interaction elements for every task is assigned, e.g. ‘1:n’ input Abstract user interfaces can still be represented as EMF models. (choose from list) or hierarchical selection (tree) or string output For concrete user interfaces another type of model is used. UIs (label). Note that explanations in parentheses are only illustrative, can be described, defined or programmed in many different ways. but not definitive assignments. Closely related to model-based development are any types of XML-based UI-languages. Their main advantage is a well-defined An abstract user interface can be transformed to a concrete user grammar and their hierarchical structure. interface (CUI). For this model-to-model transformation the abstract definitions of layout and user interaction elements are While several XML-based UI languages are wide-spread; XAML mapped to a precise layout. User interaction object classes are (Microsoft .NET), UsiXML[11] and XUL [13](Mozilla) certainly replaced by definitive implementing widgets. This model is are of major importance. Transformations of this paper produce eventually used by code generators to deliver an application or a XUL-models as target model. final user interface respectively. 2. METHODOLOGY As mentioned before, we strive to automate the whole workflow Our software development methodology is based on task models from task models to final user interfaces. While we found that of psychologists and manpower studies. According to these fully automatic generation does not produce feasible user studies, a task has to be performed on an artifact by a person in a interfaces, i.e. acceptable from a usability point of view, a CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings combination of automatisms and human designer assisted • IN – Ignore Node: Task and all its children are ignored for transformations yields sensible results. navigational model Table 1: Available adaptations during development process • PUNM – Pick Up for Navigation Model: Abstract tasks are included into the navigation model Model Adaptation to context-of-use • VL – View List: Explicit grouping and ordering of children Task model sequence, iteration, concurrency (semantic) grouping of tasks into views, Dialog graph transitions and navigational flow basic screen layout, Abstract UI widget class per task implementing widgets Concrete UI graphical design (coloring, images, labels) positioning of elements Table 1 summarizes the models used in our approach and indicates which kind of adaption is possible on each level. 3. TOOL SUPPORT To support our user interface development process a number of tools was developed. Most of them started as separate tools which communicated using proprietary XML-based languages. While the number and variety of tools evolved this became a stalling factor in development. Therefore we decided to port them into the EMF environment and use XMI for communication. For that purpose meta-models for task model and dialog graph were developed, other like those of abstract and concrete user Figure 1, Transformation of an annotated task model interface were reverse engineered from their respective XML schema definitions. For model-to-model transformations we now In Figure 1 partial screenshots from two tools can be seen. The use e.g. openArchitectureWare workflows. upper part is from our task editor and shows a very simple CTT- alike task model. In this notation the enabling node has the same 3.1 Defining task model and dialog graph meaning that separate enabling transitions between children nodes Task models in CTT notation form the basis of our modeling would have. Also the effective grouping annotations for each task approach. Besides standard CTT our graphical editor is capable of are displayed. Here all grouping is defined in the “enabling” node, handling advanced concepts as for example prioritization or via its marker VL. instance iteration. Also artifacts needed and presentation defaults may be defined with this tool. During the model-to-model transformation the abstract task is ignored and its children are grouped according to the explicit Next step in our methodology is the navigational structure. The ordering specification “{0}{1,2}{3}”. Activating e.g. ICV instead previously existing dialog graph editor was converted into an of VL would result in one view only with all tasks within it. EMF/GMF editor. It allows manual creation of views and assignment of tasks to views. Its features include hierarchical The lower part is an excerpt from our dialog graph editor. This is dialog graphs, i.e. views can be composites of sub-view[s] and how it displays the resulting model of three simple views with tasks; this was developed as adaptation concept to reduce sequential transitions between them. complexity and repetition in dialog graphs. Marker profiles are preset configurations that may be applied to Automatic generation of dialog graphs from task models is not a all or only selected nodes of task models; as such they may be trivial task. There are basic approaches as: “put every task in its platform independent. Manually attached information is own view” or “put all tasks in a single view”, which of course are considered to be platform specific as well. The concept and the not satisfying. Enabling task-sets, as known from CTTE [3], corresponding tool support are discussed in depth in Diebow [4]. might be a promising approach, but for now we attempted a semi- automatic procedure. 3.2 Generating a user interface Figure 2 shows the meta-model of dialog graphs. Stereotypes We require that the modeler annotates tasks of his task model with Model and ModelElement refer to their respective EMF meta- markers. Those marks enable an interpreter to decide which tasks model types, as well as EBoolean is EMF’s Boolean type. Classes belong into the same view, in our case the interpreter is the dialog Task, DeviceType and UserRole are defined in other packages; graph editor. Modelers can either manually attach the additional those are non-displayed as their actual declaration is of no information within the task editor or they can specify general importance for this paper. profiles for operators that are used during the transformation process. Any task may be marked as: Any view of a dialog graph is modeled as DialogView object. A view object has references to each task it contains and if needed to • ICV – Integrate Children into View: A task and its first level artifacts of those tasks. Navigation between views is controlled children are attached into the same view and specified using the port-metaphor. A port is either source or target of a transition. 3.3 CUI and patterns After developing and testing the dialog structure on the level of simple prototypes and abstract user interfaces the next step would be the concrete user interface. We mainly use XUL for this model level, for the same reason as for section’s 3.2 prototypes. Also we have generators for UsiXML and are working on XAML. But still most advanced support is available for XUL as CUI model. There is an EMF editor and the GEF XUL editing tool (XUL-E). It is a typical editor for graphical user interfaces and features drag and drop layout and WYSIWYG-layout as far as it is possible for XUL. Beside its graphical editing features its main purpose is to integrate into our development approach and to provide an engine for user interface pattern application. XUL-E may be used to refine or re-layout the prototypes generated in section 3.2 while leaving their navigational structure intact. Thus it is possible to keep the application’s prototype character while improving its graphical representation. This is Figure 2, Meta model of dialog graph models achieved by tracking UI elements that are modified or replaced. This modification information is used to attach the correct model The completion of a certain task may result in or trigger a references to new user interface elements. transition. Transitions are specified within objects of class TaskToTransitionMapEntry. Conditional transitions referring to Another source for concrete user interfaces are AUI descriptions ports of a view are stored in its DialogView.taskentry attribute. from the USG process [14], which we map in semi-automatic transformations to XUL. The meta-model provides means to define hierarchical dialog specifications. Class DialogViewGroup is used entirely for Another of our main research topics is the integration of HCI hierarchical dialog graphs. A parent-child relationship is patterns into MDD. The object-oriented patterns introduced by established between instances of Dialoggraph and DialogView, Gamma et.al. [16] are widely known as valuable aid and we navigatable as parent or rather subgraph. consider HCI patterns to be of the comparable value. Therefore we try to integrate those patterns into our model-based UI As discussed in section 2 it is possible to generate an abstract development process. interface from a dialog graph. We want the AUI to serve as mock- up of an application. Thus it can be used to discuss the application We focus our pattern integration on the CUI level, but consider workflow with any stakeholder in a very early development phase. task and navigational patterns as well. We try to translate the idea of a certain pattern into an algorithm or a layout template and User interface prototypes, for WIMP-interfaces, can e.g. be define this as a pattern instance component (PIC) [15]. A PIC is a defined in a number of XML languages. We decided on a machine readable definition and may be employed semi- or fully combination of HTML and XUL for that purpose. A suitable automatically to a user interface model. viewer would be every Gecko-based web-browser; Firefox is probably the most well-known example. Actual UI generation is PICs are stored as EMF models making our main tool chain done by openArchitecureWare workflows who transform the completely based on eclipse’s [graphical] modeling and as such EMF model of a dialog graph into XUL. well integrated. For the generation at first the application starting or entry point If a modeler is satisfied with its designed CUI, whether by pure has to be identified. Our basic WIMP-style application consists of editing or using PICs, he may use the source code generator and views and simple interaction objects arranged into those views. produce Java Swing source code of it. Each application, per definition, has a start view configured in its dialog-graph. 4. RELATED WORK A lot of tools and methodologies for model-based development of Dialog graph view-to-view transitions are defined using a port-to- interactive systems have been developed. In this section we port metaphor. Application start- and end-nodes fit into this concentrate on approaches that we consider as most typical and system as well. There is a port defined as application-start, type promising. outgoing and named ‘start’, and there is an opposite port, type These are TERESA [10] developed within the Cameleon [1] incoming and named ‘in’, denoting the end of the application. project and UsiXML [11] which is a language, a set of models and Objects of type Dialoggraph contain references to those entry and a tool set. Both approaches do not provide explicit navigation exit ports within their attribute DialoggraphPort. specifications like dialog graphs. This information is, if available For generating our application’s startup mask the transformation at all, distributed over different other models. begins in the context of the root dialog graph. Here the In general TERESA generates the user interface based on its application’s entry port is identified by OCL and an models by presenting all tasks active at the same time in one openArchitectureWare transformation workflow is invoked. window, but it is possible to use additional heuristics. TERESA and UsiXML do not really consider the abstraction level for task model to dialog graph transformations based on our newly of a dialog graph. They have the level of abstract user interface, introduced marker concept. which is a little bit more concrete than our approach. Our dialog graphs are more concrete than task models but more abstract than their AUIs. Unlike TERESA and UsiXML our dialog graph gives 6. REFERENCES the opportunity for an explicit design process of the navigation [1] Cameleon: http://giove.cnuce.cnr.it/cameleon.html (visited structure. Our approach may be offered as an alternative strategy November 29, 2008) to software developers, if the generation process does not deliver the expected results. It might depend on the application domain [2] Costa, D., Nóbrega, L., Nunes, N.: An MDA Approach for and especially on the number of tasks in the model, which Generating Web Interfaces with UML ConcurTaskTrees and approach fits best. Especially for larger task models our approach Canonical Abstract Prototypes, TAMODIA 2006, Hasselt, might fit better because not all tasks are presented that could be Belgium, October 23-24, 2006. executed in a certain moment. [3] CTTE: The ConcurTaskTree Environment. The approach proposed by Costa et. al. [2] uses a kind of task http://giove.cnuce.cnr.it/ctte.html (visited November 2008) trees represented in UML notation for dialog modelling. Each task [4] Diebow, Ch.: Entwicklung eines Konzeptes zur interaktiven element is detailed by the Dialog Model using a UML-compliant Trtansformation von Aufgabenmodellen in Navigations- adaptation of the CTT notation. Tool support is available by modelle, Master Thesis, University of Rostock, 2008. DialogSketch [8]. Canonical abstract prototypes are used to [5] EMF: http://www.eclipse.org/modeling/emf/?project=emf specify the first level of user interfaces. In this way it does not (visited November 4, 2008) have this level of abstraction we want to support too. It is focused on Web interfaces and related papers do not discuss the problem [6] Koch, N.: Transformation Techniques in the Model-Driven of different roles and a variety of platforms. Development Process of UWE, ICWE’06 Workshop, Palo Alto, 2006 Most publications related to navigation specifications seem to be related to the area of web applications. This was the result of a [7] Leung, K.R.P.H., Hui, L.C.K., Yiu, S.M., Tang, R.W.M.: current survey of the literature. Leung et. al. [7] specify the Modeling Web Navigation by Statechart, Proc. navigation for web applications by state charts. They present COMPSAC’00 specification solutions for intra-page, inter-page and frame-based [8] Nóbrega, L., Nunes, N. J. and Coelho, H.: DialogSketch: navigation by hierarchical states. They already raise the problem Dynamics of the Canonical Prototypes, TAMODIA’2005, of multiple windows that are specified in dialog graphs as Gdansk, Poland, September 26-27, 2005. multiple views. Their solution suggests separate state charts for each window. The authors identified dynamic content as a special [9] openArchitectureWare:http://www.openarchitectureware.org/ problem of their kind of specification. From our point of view the (visited November 4, 2008) readability is another problem. The notation is much more [10] TERESA: http://giove.cnuce.cnr.it/teresa.html (visited complex than our dialog graphs. It also does not allow different November 4, 2008) kinds of transitions. Roles and devices were not considered. [11] UsiXML: http://www.usixml.org/ (visited November 4, Koch [6] as well focuses her work on web modelling. She bases 2008) her specification on UML. UWE (UML-based Web Engineering) [12] Tool Support for an Evolutionary Design Process using User- is a model-driven development approach. Class diagrams with Interface Patterns special profiles specify the navigation model. More or less only the menu structure is represented by the specification. It is the [13] XUL: http://www.xul.org (visited November 4, 2008) goal of this approach to transform navigation models together [14] Müller, Andreas: Spezifikation geräteunabhängiger with business specifications in form of activity diagrams and state Benutzerschnittstellen durch Markup-Konzepte, PhD charts to service-oriented applications. dissertation, University of Rostock, 2003 5. SUMMARY [15] Wolff, Forbrig,Reichart: Tool Support for Model-Based In this paper we briefly described a set of tools for model-driven Generation of Advanced User Interfaces. MDDAUI '05, user interface development. Those tools form a tool chain which Proc. of the MoDELS'05 Workshop on Model Driven is embedded into the eclipse modelling framework and uses Development of Advanced User Interfaces, CEUR Workshop standardized XML dialects for specification and user interface Proc. 159. CEUR-WS.org, 2008. models. [16] Gamma, Helm, Johnson, Vlissides: Patterns-Elements- The presented methodology is based on task models and uses Reusable-Object-Oriented-Software, Addison-Wesley, 1994, specialized navigational models to derive abstract and concrete ISBN 0-201-63361-2 user interfaces and to eventually generate source code. It is adaptive on any model level and able to integrate HCI pattern into the development process. Future work will especially focus CUI integration of UsiXML, the model representation of pattern instance components and pattern