Modiquitous 2011 Proceedings Generating consistent universal controllers for Web-Service-enabled appliances Marius Feldmann, Thomas Springer, Alexander Schill Technische Universität Dresden Fakultät Informatik Professur Rechnernetze Germany {marius.feldmann, thomas.springer, alexander.schill}@tu-dresden.de ABSTRACT universal control application for a mobile device or a Today, an increasing number of home and office appliances Web-based universal control application. If a new device that we interact with contain embedded Web Servers. (DVB-T device) is added to the device infrastructure, a user Making available Web Services for remote access to their interface has to be made available dynamically within the functionality is just a small step. In this paper, we present a two universal control applications. Developing these user multi-platform, model-based generation approach enabling interfaces for various target platforms usually tends to efficient and low-cost development of interactive constitute a time consuming and expensive task. applications for accessing Web-Service-enabled appliances. The results are not stand-alone, monolithic elements of software but are capable of being integrated into a consistent host application ad-hoc during runtime. Thus, a heterogeneous and dynamic infrastructure of appliances from different manufacturers can be accessed via single control applications. The approach has been proven to be applicable for generating interactive applications for various target platforms including mobile devices. Keywords Web Services, Model-based User Interface Generation, Dynamic Service Infrastructures Figure 1. Sketch of the home appliance scenario In order to solve these problems, this paper presents an INTRODUCTION AND MOTIVATION approach to generate the user interfaces of the Web Services are software components offering their Web-Service-enabled appliances fully automatically from functionalities via a well-defined interface. This interface is functional interface descriptions. Due to the fact that described by a functional interface description language without further input, the generation result would be of low such as the Web Service Description Language (WSDL). quality, so called Service annotations [8] are applied as Nowadays, Web Services are an accepted and widely used platform independent modeling approach for defining user means for offering remote functionalities. Various tools are interfaces. available for managing the whole lifecycle of Web Services starting from creating, via testing, to deploying and The results of the generation chain named interactive managing them. However, the field of developing user components can be embedded in the universal control interfaces for Web Services has not been covered in a applications dynamically during runtime. Without stopping satisfactory manner yet. Though some approaches have this application, they appear in their navigation menu and been specified during the last years enabling the can directly be used to remote control the appropriate development of user interfaces for static Web Service device. Due to the possibility to parameterize the infrastructures, no approach exists that makes it possible to generation of interactive components by using layout and extend an interactive application fully automatically style descriptions, they can be visually adapted to different depending on the associated set of Web Services the appearances of the embedding universal control application interacts with. The necessity for such an applications. Thus, a consistent visual representation is interactive application becomes obvious in the case of the achieved. selected use case: Within an automated home, a set of The remainder of this paper is structured as follows. In the appliances can be controlled via a universal control second section an overview of the state of the art in the application available on different platforms. In the sketched addressed domain is provided. The third section defines the scenario in Figure 1, three appliances (Light Control concept of interactive components and points out their System, Music Control System and a Router) can be central characteristics. In the fourth section the assumed accessed via their offered Web Services by using either a development approach is described. The fifth section M. Feldmann, T. Springer, A. Schill: Generating consistent universal controllers for Web-Service-enabled 39 appliances. Proc. of 1st International Workshop on Model-based Interactive Ubiquitous System 2011, Pisa, Italy, June 13, 2011, http://ceur-ws.org/Vol-787 Modiquitous 2011 Proceedings introduces the steps for generating interactive components As soon as an interactive component is physically available from the results of the development approach. The sixth on a target device, it runs through the following steps: section describes the technical realization of the mentioned 1. Registration: The interactive component appliance control scenario and an end user study applied communicates the information needed for its execution using the prototypical implementation. The paper is to the host application. The registration information concluded by a summary and outlook on future work. covers: STATE OF THE ART a. Identifier: In order to identify a component and to Creating user interfaces for home appliances is a topic replace old versions on a target platform, a unique covered in the HCI community already for years (e.g. [3]). identifier is assigned to every interactive As well the specific goal to create consistent user interfaces component. for controlling different devices has been addressed [4]. However, these approaches do not focus in any way on b. Component Dependencies: An interactive Web-Service-enabled appliances. Thus, the possibility to component may demand functionality that has to generate user interfaces from functional interface be provided by the host application. This includes descriptions is not discussed. The only approach exploring for example functionality to determine the current Web Services in the field of universal appliance control geolocation or specific data storage facilities. applications exploits a task-driven development approach c. Input-/Output-Data: An interactive component to create consistent user interfaces [5]. Though the may interact with a host application via efficiency of this approach may be increased by using exchanging data. To identify the data needed as Service annotations [6], it still takes various manual steps input and returned as output, references to data to develop a user interface for different target platforms. described within the associated functional Furthermore, the resulting user interfaces cannot be interface description is used. embedded automatically into a host application during d. Context information: Every interactive runtime. Extending the focused research and development component may be generated for a concrete usage field to ad-hoc UI generation approaches for context such as a specific language, a specific Web-Service-based interactive applications, we discovered target platform or a selected geographical location. a set of ten approaches. They usually take either a A context description is expressed as a set of functional interface description or additionally Service context type and value information. annotations as input and create stand-alone markup-based UIs as output. An example for this category of approaches e. Information for visualization purposes: An is described in [7]. The results have a low quality and do interactive component is included dynamically not address the generation of consistent UIs enabling the into the navigation menu of a host application so interaction with various underlying Web Services as it is that a user can activate it. For visualizing the demanded within the home appliance scenario. component, data such as a label text or an icon is communicated to the host application. To summarize our efforts, we realized an intensive state-of-the-art analysis searching for approaches that make If a host application accepts the registration it possible to generate automatically user interfaces for information and fulfills the requirements (thus it can Service-based interactive applications and that create offer the necessary components (b) and the appropriate results which can be included ad-hoc into a running host input data (c)), the registration is successful. If the application. To the best of our knowledge, no comparable interactive component is not registered successfully, it approach like the one presented in this paper exists. is removed from the host application. INTERACTIVE COMPONENTS 2. Activation: After an interactive component has been Interactive components are a domain-specific solution for registered successfully and the current application user interfaces of Service-based interactive applications. context conforms to the context the component is Due to their characteristics, they solve the identified intended for, the component can be activated by a user. problems of dynamic integration of ad-hoc generated user The host application hands over the input parameters interfaces into a hosting interactive application. demanded by the interactive component during the registration procedure. Interactive components contain all information necessary for registration purposes within a host application. They 3. Usage: After a successful activation of the interactive offer a user interface for providing input values and component, a user can input data via the user interface displaying output values of Web Service operations and and invoke remote Web Service operations. The contain all information needed for enabling the interaction operation results are being visualized in the user of a human user with a remote Web Service. After an interface. interactive component has been generated, it is transmitted 4. Deactivation: The interactive component can be to a host application (such as the universal appliance deactivated at any time by the host application or the control application) via an application specific directory user. After the deactivation has been triggered, its user Service. interface is not visible anymore. During deactivation of 40 Modiquitous 2011 Proceedings an interactive component, it returns the output data 2. Bundling: Renders it possible to merge the user specified during the registration process to the host interfaces of different operations into one user application. interface. Furthermore, the view for input and output We have specified a Meta-model named Service-based of one operation might be merged in one view. Interactive Component Model (SBIC) for representing 3. Provided by Host: Marks an input parameter of a interactive components during the generation process. The Service operation as input to the interactive component structure of the model is sketched in Figure 2. The SBIC provided by the host application. has been developed as a model representing an interactive 4. Returned to Host: Marks output parameters to be component in a way close to a runtime representation. returned to the host application when the interactive Thus, a model-to-code transformation mapping a SBIC component is deactivated. instance to a platform specific interactive component can be implemented with little effort. 5. Component Dependency: Defines dependencies to functionalities the host application has to provide. Besides the registration information, the SBIC describes the structure, navigation flow and data flow of the user 6. Initial Operations: A subset of the operations defined interface. Furthermore, it contains layout and design in a functional interface description may be marked as information that is weaved into the model during the initial. These operations can be accessed directly from generation process. Thus, the layout and design of a the host application. Every initial operation is an entry generated interactive component can be adapted to the point to activate the interactive component. After layout and design of a host application. activating it, the user interface for the appropriate initial operation is visualized. DEVELOPMENT APPROACH As aforementioned in the state-of-the-art analysis, the In order to specify these annotations, an authoring tool development of Service-based interactive applications for named Interactive Component Editor (ICE) [2] has been various target platforms is currently a time-consuming task created. One of the views of the ICE is shown in Figure 3. involving huge manual effort. Our approach is After importing one or more functional interface characterized by the novel idea to only use Service descriptions into the tool, the tree structure of these annotations for modeling user interfaces. As the descriptions is visualized. The nodes on the various levels state-of-the-art does not offer any Service annotation model (Service, operation, input/output parameters, data types) of enabling the specification of application-specific aspects the tree representation can be included into the modeling of such as navigation or data flows, we introduced new the interactive component. annotation types. The screenshot in Figure 3 shows various navigation flows To not reinvent the wheel, these types were introduced into and data flows in a specific editor view during modeling an the annotation model originating from the ServFace project interactive component for a DVB-T device. The editor [8]. Besides others, the following annotation types have makes it possible to specify any of the annotations defined been specified and included into an extension of the within the ServFace project and of the abovementioned ServFace annotation model: extensions. Thus, a full featured development approach has been created enabling the model-based specification of 1. Navigation Flow: Enables the possibility to define a interactive components solely with annotated functional navigation flow between the user interfaces derived interface descriptions. from different Service operations. With every specified navigation flow, a data flow may be associated. After the modeling has been realized, the resulting annotations are serialized to a file and transferred to a remote annotation repository. GENERATION APPROACH The generation of interactive components is triggered on demand once an interactive component has to be embedded into a host application. In the case of the automated home scenario, the generation chain is activated as soon as a new device and thus a new annotated Web Service appears in the home infrastructure. Due to the usage of platform independent Meta-models and the platform specific parameterization of the generation approach, it can be reused for various target platforms and different host applications. The platform and host specific parameters have to be provided only once for every host application. The target specification can be reused during the generation of each interactive component that should be embedded Figure 2. Structure of interactive component Meta-model into this host application. The generation chain is summarized in Figure 4. 41 Modiquitous 2011 Proceedings Figure 3. Screenshot of the Interactive Component Editor In a first step, the functional interface description and the For instantiating the ISF, the ASM instance is traversed. By annotation file fetched from an annotation repository are default for every Service operation one container for all parsed and merged together into a common model. For this input parameters and one container for all output purpose the Annotated Service Model (ASM) has been parameters are derived and embedded into the ISF instance. developed. It enables the representation of all ServFace This default behavior may be modified by annotations of annotations and the mentioned extensions. An ASM the Bundling type (see previous section). instance contains, on the one hand, the tree representation In a next step, appropriate interactors are derived from all of the functional interface description with the different input and output parameters passed during traversing of the hierarchies (Service, operation, input/output parameters, ASM instance. They are embedded into the containers data types) and, on the other hand, the annotations created in the previous step. referencing a subset of the tree’s nodes. This subset has been determined during the development approach applied As the selected interactors are described within the ISF e.g. by using the ICE. using platform specific vocabulary, this step is parameterized by a set of inference rules which determine After the ASM has been instantiated, a basic structure of which interactor should be selected under which condition. the user interface with all its interactors is inferred in a first Besides the data types of an input or output parameter, the model-to-model transformation step. For representing the decision depends particularly on the set of annotations structure of the UI, a domain-specific model named referencing this parameter. Intermediate Service Frontend Model (ISF) adapted to the generation chain has been developed. Figure 4. Generating interactive components from annotated functional interface descriptions 42 Modiquitous 2011 Proceedings Figure 5. Usage of the generation approach within the home appliance scenario In a second model-to-model transformation step, an 4. Using platform-specific model-to-code transformations instance of the SBIC Meta-model mentioned above is (named platform adapters) for all desired target created from the ISF instance. Just as the transformation platforms, the SBIC instance is transformed to steps before, the implementation of this step can be reused platform specific interactive components. for various target platforms due to the parameterization of 5. These components are forwarded to a repository that is the transformation. The provided parameters contain checked in fixed intervals by the used universal control especially layout and design information for adapting the applications for updates. appearance of the interactive component to the appearance of a host application. This information can be made 6. As soon as a new interactive component is available available a-priori and updated as soon as the layout and within the repository, it is downloaded to the target design of the host application is modified. platform and registered in the universal control application. Finally, every SBIC instance is transformed to a platform specific interactive component that can be made available 7. After a successful registration of an interactive via a component repository to a host application. For every component it can be used to interact with the remote supported platform a model-to-code transformation has to Web Service thus enabling device control. be provided that takes an SBIC instance as input and REALIZATION AND EVALUATION returns a packaged and deployable interactive component The approach has been implemented and evaluated using as output. two heterogeneous target platforms as shown in Figure 5. The described generation chain builds the core of the home As proof-of-concept for mobile devices, a universal control appliance scenario introduced above. Figure 5 shows the application and a platform adapter for the smartphone different steps used to make interactive components platform Android has been realized and as proof-of-concept available in different universal control applications. for Web applications, the same parts of software have been implemented using the Apache Wicket framework. The The approach runs through the following seven steps: repository for devices has been realized as a simple 1. A device (e.g. a DVB-T device) is newly introduced relational database. As technology for the repository for into the infrastructure of Web-Service-enabled interactive components Subversion (SVN) was used. SVN appliances. As soon as it is switched on, it is registered has the central advantage that no additional mechanism for by transmitting the Uniform Resource Identifier (URI) versioning of interactive components had to be introduced. of its functional interface description plus a reference Figure 6 shows a screenshot of the prototypically to an annotation file to a device repository. implemented Web-based universal control application. As 2. The device repository is monitored by a specific it is depicted in the figure, interactive components are system component named Activator. As soon as this embedded once they are registered in this application into component detects a new device registration, it its navigation menu shown on the left side. The initial forwards the functional interface description operations (see fourth section) of each interactive referenced by the URI plus the annotation file to the component are displayed by a navigation option each. As generation chain described above. soon as a user selects one of these options, the appropriate 3. The generation chain parses the functional interface user interface is displayed in the center of the Web page. description and the annotation file and transforms it first to an ISF and then to a SBIC instance. 43 Modiquitous 2011 Proceedings Figure 6. Prototype for the Web-based universal control application The approach has been evaluated based on the prototypical REFERENCES implementation within an end user study. 25 people in the 1. Christensen, E., Curbera, F., Meredith and G., age between 19 and 31 years have used the two universal Weerawarana, S. Web Services Description Language remote controllers to interact with a set of simulated home (WSDL) 1.1. W3C Note. March 2011. appliances. After fulfilling several minor tasks, the users 2. Feldmann, M., Martens, F., Berndt, G., Spillner, J., participated either in a questionnaire or in an interview to Schill, A. Rapid Developed of Service-based Interactive get an insight into their experiences with the generated Applications using Service Annotations. In: interactive components. Summarizing the results, the Proceedings of IADIS International Conference interactive components have been reviewed as easily and WWW/Internet 2010. efficiently usable. All users stated that the user interface makes a consistent impression and looks like a monolithic 3. Nichols, J. Automatically generating high-quality user application. No user criticized any form of inconsistency of interfaces for appliances. In CHI ’03 extended abstracts the user interface. Thus, the component-based nature of the on Human factors in computing systems, pp. 624-625, user interface is not recognized by end users. ACM Press, 2003. SUMMARY AND OUTLOOK 4. Nichols, J., Myers, B. A., Rothrock, B. UNIFORM: Automatically Generating Consistent Remote Control This paper presented an approach to generate consistent User Interfaces. In Proceedings of CHI'2006, pp. universal controllers for Web-Service-enabled home 611-620, Montreal, Canada , 2006. appliances fully automatically. The approach is based on the idea to generate so-called interactive components from 5. Paternò, F., Santoro, C., Spano, L., D. Designing usable annotated functional interface descriptions. These applications based on Web services. In: 1st Workshop components can be embedded dynamically during runtime on the Interplay between Usability Evaluation and into a host application. The applicability of the approach Software Development, pp. 67 - 73. CEUR, 2008. has been demonstrated by implementing the home 6. Paternò, F., Santoro, C., Spano, L. D., Exploiting Web appliance scenario using a Web-based universal control service annotations in model-based user interface application and an appropriate application for a smartphone development. In: EICS'10 - 2nd ACM SIGCHI platform. The implementation has been evaluated in an end Symposium on Engineering Interactive Computing user study which confirmed the good quality and usability Systems, pp. 219 - 224. ACM, 2010. of the resulting user interfaces and their consistency. The 7. He, J., I. Yen, T. Peng, J. Dong und F. Bastani: An control application made the impression to be monolithic Adaptive User Interface Generation Framework for though it consists of components provided for every Web Services. Proceedings of IEEE Congress on available device. Services Part II, Seiten 175–182, 2008. Future work will focus in first place on improving the 8. ServFace-Konsortium: Deliverable 2.9 - Models for underlying development approach sketched in the third Service Annotations, User Interfaces, and Service-based section. In this area it will be analyzed, how the Interactive Applications (final version). Technical provisioning of Service annotations may be simplified, e.g. Report, SAP AG, Lyria S.A., Consiglio Nazionale by suggestion functionalities. Furthermore, implementing Delle Ricerche, The University of Manchester, support for further target platforms is intended. Technische Universität Dresden, 2010 44