Domain Specific Methods and Tools for the Design of Advanced Interactive Techniques Guillaume Gauffre, Emmanuel Dubois, Remi Bastide IRIT - LIIHS 118, route de Narbonne 31062 Toulouse Cedex 9, France +33 (0)5 61 55 74 05 {gauffre, emmanuel.dubois, bastide}@irit.fr ABSTRACT 2. MIS ENGINEERING FRAMEWORK Novel interactive systems such as Augmented Reality are promising tools considering the possibilities they offer, but no Common processes for HCI development include four steps: real development methods exist at the moment to help designers requirements gathering, design, implementation and evaluation. in their work. We present in this paper a design method for Figure 1 presents how our tools cover the first three steps. Task tightly coupling early interaction design choices and software models are one of the major tools to support the requirement design solutions. Our work is based on an existing model used step: they are used to describe the sequence of sub-tasks for abstract UI design, and introduces a second model dedicated (concerning user’s activities, system’s activities or interactive to the software UI specification and the model-based process activities) in a hierarchical form corresponding to the global used to derive one from the other. To achieve this, we present system task. The design step can be decomposed into two here a framework based on domain specific models and separate phases: UI design and the underlying software transformations to link them and thus support the development specification. The former step is concerned with user’s process. interaction aspects. It may be linked to requirements gathering by combining users’ observation, brainstorming or focus-group to collect user needs, and an interaction model to organize them Categories and Subject Descriptors according to the specificities of MIS [4]: domain objects H.5.2 [Information Interfaces and Presentations]: User description, user abilities, physical and digital artifacts, Interfaces - Theory and Methods. D.2.2 [Software interaction forms. In the latter, design aspects related to the Engineering]: Design Tools and Techniques - User Interfaces. software architecture are considered, using a specific model. The next step is the implementation of the system by using Keywords component-based platforms improving flexibility and Mixed Interactive Systems, Model-Driven Engineering, adaptability. Domain Specific Languages, Metamodeling, Model Transformations, Design Process. Task Focus-Group Modeling Mixed Interaction Modeling 1. INTRODUCTION In the past 10 years, a new HCI trend has emerged: traditional “Window, Icon, Menu, Pointing device” interfaces tend to be replaced by new forms of interaction that involve physical Software Component-based Architecture artifacts, easily manipulated by users. Augmented Reality Implementation modeling systems for example, are interactive systems where the realization of a physical task is enriched by the presence of digital information. Tangible User Interfaces and ubiquitous Figure 1: MIS domain specific process systems are other forms of interactive systems which merge physical and digital worlds. To refer to these approaches and In this context, rather than modifying the different models we because they deal with similar concepts and techniques, we use (task models, interaction models and platform models) to regroup them in one term: Mixed Interactive Systems (MIS). articulate them, we describe a DSL to provide a support to this Thereafter MIS frameworks have been developed and adopt process: bottom-up or top-down approaches. Each of them brings  Multiple models are required in each step of the consequent advances at different level of abstraction of the development process and one role of MDE is to “promote design [6] but interlacing them remains difficult to accomplish, models to primary artifacts that drive the whole thus limiting the coverage of the development process. development process” [1]. MDE will facilitate their As the use of Mixed Interactive Systems increases, elaborating articulation and permit the elicitation of coherence rules. a convenient development process becomes necessary. To cover  The MIS domain, with regards to their applications in our the different steps of such process, our approach promotes the every day life, produces emergent systems. Elaborating results gathered in the early design steps and bridges the gap methods to develop them requires to evaluate the between the abstraction level of these results and the adequacy of models and to support their evolution when implementation. To do so, we articulate models to progress required. MIS domain is in a phase of empiricism and along the development process and adopt a MDE approach, thus begins to develop theories; MDE will be a powerful introducing a Domain Specific Language [1] for MIS. support of this evolution. 3. TWO DOMAIN SPECIFIC MODELS Entities) and roles in the architecture (Devices, APIs, Models, The DSL we proposed is based on two models: Controls, and Views).  ASUR, an existing model which describes the user’s ASUR-IL Adapters in input or output, correspond to the interaction with a Mixed Interactive System. It can be adapters in the ASUR model and group devices and software used by itself or as mentioned before, in combination with libraries used to connect physical and digital worlds. Devices a focus-group. are used to capture/render data from/to the physical world. They can translate physical phenomenon into digital data. The second  ASUR-IL, a complementary model that we introduce to part of an adapter is an assembly of specific APIs which permit cover the description of the software decomposition and to combine several computing facilities to obtain required data, structure. Its aim is to prepare the implementation step by such as ARToolKit, a specific toolkit for Augmented Reality, producing a coherent architecture, promoting the which, from a captured frame, produces 3D coordinates of the interactive forms chosen in a technological perspective. recognized markers. After an overview of the ASUR metamodel in the next section, we present the ASUR-IL metamodel to enable the collaboration ASUR-IL Entities are the other concepts that make up an of our two domain specific models. ASUR-IL model. They correspond to the digital concepts involved during interaction and identified in ASUR as STool, 3.1 ASUR Overview SObject or SInfo. They are triplets of three ASUR-IL components For a given task, the role of ASUR is to support the description called Models, Views and Controls, inspired from the MVC of the physical and digital entities that make up a mixed decomposition [7]. Controls are in charge of interpreting the interactive system and the boundaries among them. ASUR physical phenomena translating data from Adapters into components include adapters (AIn, AOut) bridging the gap commands on Model parts. Models are the entry point to the between both digital and physical worlds, digital tools (Stool) or functional core. They are an abstraction of it, enabling the concepts (SInfo, SObject), user (U) and physical artifacts used as dialog with the application core. Finally, Views are in charge of tools (RTool) or object of the task (RObject). the computation required to reflect the state of each digital concept on each Adapter connected. Components can be inter-connected by several kinds of relationships. The major one, Data Exchange, is used to describe the kind of data transmitted. In the physical part, they 3.3 MIS Design Support represent the information channels between components, and in the digital part the way the system treats them. The Eclipse Representation link expresses a coupling between a physical GuideMe Plugins component and a digital one in terms of behavior and rendering. Finally Real associations express a physical proximity of two ASUR ASUR-IL physical components and Triggers represent an action of one EMF .codegen EMF .codegen component over another. On the basis of previous works in the .edit .edit domain, design-significant aspects have been identified and added to the model: ASUR characteristics improve the GMF .diagram GMF .diagram specification of components (perception/action sense, location, ASUR2IL etc.) and relationships (type of language, point of view, ATL dimension, etc.). By analyzing the characteristics of each element, the model supports the predictive analysis of two WComp properties: continuity and compatibility of interactions. Figure 2 : Tools integration 3.2 ASUR-Implementation Layer: Towards ASUR has its own editor: GuideMe. It is a graphical editor the Implementation Phase which can export diagrams as XML files. After its metamodel For each ASUR model, i.e. a given mixed interactive task, an was defined [3], a second version of the editor has been ASUR-IL model is associated. The main contribution of this developed using EMF to separate graphical editing from model model is to identify the software components and relationships manipulation. As mentioned above, ASUR and ASUR-IL are required to implement this specific task. Only the components two models required at different steps of a MIS design process. involved in the interaction part of the system are described. The Other models could also be required such as task model for description of functional parts of the application is out of requirements gathering or UML for functional core ASUR-IL scope. This model is also the frontier between specification. To support the integration of our two models and Platform Independent Model and Platform Specific Model: it further evolution, we adopt an MDE approach and choose to describes the software components involved in the task and instrument it with tools from the Eclipse Modeling Project their communications, the next step being the transfer to a PSM (EMP [5]). It enables the creation of dedicated tools for each where each ASUR-IL component will be associated to software model with EMF, GMF, and others. Therefore each model can component, existing assembly or new ones. be edited using the corresponding plug-ins in Eclipse (cf. Figure 2). To present this assembly of components, the main concepts of the ASUR-IL metamodel are Components and Data Flows. A Thanks to these tools, the designer can manipulate the two third item Port, represents the interfaces between each of them. models easily. The main challenge is now to couple them by The correctness of the data flow between two components is model transformations to rapidly observe the consequences of ensured by the value given to the attribute data type of each modifying the description of the interactive situation modeled port. There is only one kind of relationships as opposed to with ASUR on the software architecture described with ASUR- components for which the definition follows two principles: IL. The next section presents the transformation between ASUR correlation with the ASUR components (ASUR adapters  and ASUR-IL and finally introduces the transformation ASUR-IL adapters, ASUR System components  ASUR-IL between ASUR-IL and a software component model: WComp 4.2 ASUR-IL 2 WComp: Platform Specific [2]. Model Definition Assuming, that during ASUR-IL edition the designers carefully 4. DOMAIN TRANSFORMATIONS identified each component of the system, they now must be In order to implement these transformations, the Atlas transposed on the platform model. The currently chosen Transformation Language (ATL) has been chosen. One of the platform is WComp [2] which is dedicated to rapid prototyping main reasons is that ATL is now fully integrated in the Eclipse of wearable and ubiquitous interactive systems. Considering Modeling Project [5] and so ensures us a complete coherence these goals, this platform allows the creation of assemblies of between the different tools. As the targeted platform embeds its components with a small granularity and the runtime adaptation metamodel as code and thus using a Model-2-Model engine is to the platform context (i.e. low battery level, devices actually not possible, we also use a Model-2-Text engine: JET disconnected, etc.). Its flexibility and its simplicity are the major points to use it. Adapters (1) Adapters : The definition of this transformation is an on-going work using ASUR APIs + Devices ATL and JET. It will make the bridge between our PIM model SObjects (2) (ASUR-IL) and a PSM (an assembly of WComp components), Entities : with two goals: (3) M+V+C  to create a component by describing the data manipulated and the interfaces associated (Figure 3 - 4), or to identify ASUR-ILmodel a component in a repository (Figure 3 - 6) of already (4) Components defined components from older projects or standard APIs,  to manage the assembly of components (Figure 3 - 5) i.e. Components WComp (5) establishing the connections between each components in configuration Components accordance with the ASUR-IL model. (6) Once this transformation is realized, it will be possible to offer Assembly Repository designers a range of tools from interaction design to implementation. It will help to rapidly experiment with Figure 3 : Specific transformations of MIS process designed interactive situations from the ASUR results to the WComp assembly of components dedicated to MIS. To 4.1 ASUR 2 ASUR-IL: Software Modeling illustrate the kind of process it will create, we next describe our Initialization tools on a particular case study. The goal of this transformation is to prepare the construction of a component-based architecture. ASUR identifies several digital 5. TUI FOR MUSEUM EXHIBITIONS concepts considering their roles in the interaction: this is the left The goal is to design innovative interactive situations in the hand side of the transformation. On the right hand side, ASUR- context of museum exhibitions. Our work is to design solutions IL is in charge of describing the different kinds of components promoting knowledge transmission and entertainment in a involved in the interactive part of the system, with adequate science museum for particular themes: in this case the species ports and data flows between them. Major rules were already evolution. By using this approach, we can rapidly experiment defined, but not formalized. The goal was to convert them into advanced interaction and adapt them to other themes by reusing ATL rules. components. Each ATL rule follows roughly the same behavior. By identifying the type of each ASUR component plus the relationships between them, specific matched rules are involved. It consists, for example, in creating for each ASUR adapter, an ASUR-IL adapter (Figure 3 - 1) containing one default device, and a default API. Each rule contains imperative code used to interconnect components (Figure 3 - 3) and to factorize common processes. For example, when ASUR digital components are transposed in ASUR-IL (Figure 3 - 2), they trigger the creation of multiple Views and Controls, after Models have been created. Figure 4 : ASUR model for evolution-tree construction This transformation is the starting point of the software architecture design. From the characterisation of a mixed The current project aims at proposing to visitors to discover interactive situation with ASUR, it produces the base of the species evolution by elaborating an evolution tree based on software architecture. It offers to rapidly design the structure of phylogenetic criteria. Adopting MIS in that context offer the a concrete system before starting its implementation. This opportunities to manipulate physical objects and to enlarge the combination enables now to easily support the designers during experience by digital rendering (video, 3D, sound, etc.). To the crucial phase linking abstract UI design and software UI elaborate the evolution tree, the user manipulates physical specification. Following the transformation, designers can representation of species (a frog, a crocodile, etc.) to add them extend the specification by additional design decisions before to the tree which is rendered by video on the interactive space the next step which is to define a component-based model of the with related phylogenetic criteria. The first solution (Figure 4) system. uses marker-based detection to capture tangible objects (species) and video projection to report the data. finally evaluate their evolution during each cycle of the process. Further work will aim at identifying additional properties, relevant at the software design level (ASUR-IL) such as computing time or hardware constraints, and structuring their impacts on the remaining design steps of our process. It will increase the ability to evaluate the quality of each interactive situation. Another perspective is to study the feasibility of reverse transformations between each step and their impact on the higher levels of abstraction. In Figure 7, what would be the Figure 5: Asur2IL transformation result impacts of applying a reverse transformation from the modified ASUR-IL model to the ASUR model? Figure 5 shows the ASUR-IL model resulting from the asur2il transformation. The designer can now extend the model: Figure Finally, we focus here on specific models for MIS. To make 6 shows an insertion of another camera to modify the marker possible the development of concrete systems, others aspects detection and a specification of the rendering APIs. must be included: collaboration with business models for the connection with the functional core, interactive modalities ontology to support the choice of specific devices and APIs. In this way, we planned to describe, in ASUR-IL, the behaviour of the components using dialog models (State charts, Petri nets, etc.). As already mentioned, the MDE approach is very helpful to articulate and transform models. However, it appears that designing MIS may rely on a lot of models and maintaining the coherence among all of them may be difficult. The management of this combination of models and transformations need to be investigated to better assess the usability of the MDE approach Figure 6: ASUR-IL model extended for a MIS development process. To illustrate the dependencies between the two models, we can focus on the case the museum visitors wish to see the evolution 7. REFERENCES tree. It results (Figure 7) by the insertion of an AOut in the ASUR [1] Bézivin, J., Jouault, F., Kurtev, I., Valduriez, P.: Model- model and in the ASUR-IL model (only one view is used based DSL frameworks. 21st ACM SIGPLAN conference because the same interaction modality is used). on Object-oriented programming systems, languages, and applications, Portland - USA (2006) [2] Cheung, D.F.W., Tigli, J.Y., Lavirotte, S., Riveill, M.: WComp: a Multi-Design Approach for Prototyping Applications using Heterogeneous Resources. In Proceedings of the 17th IEEE International Workshop on Rapid System Prototyping, Chania - Crete (2006) [3] Dupuy-Chessa, S., Dubois, E.: Requirements and Impacts of Model Driven Engineering on Mixed Systems Design. In Proceedings of the conference IDM'05, Sébastien Gérard, Figure 7: Models evolution Jean-Marie Favre, Pierre-Alain Muller et Xavier Blanc (eds.), Paris - France (2005) 43-54 6. CONCLUSION AND FUTURE WORKS [4] Dubois, E., Gauffre, G., Bach, C., Salembier, P.: This work is a step toward the definition and instrumentation of Participatory Design Meets Mixed Reality Design Models. a design process for Mixed Interactive Systems. This process In conference Proceedings of Computer Assisted Design of will permit to increment on the designed solution until User Interface (CADUI'06), Springer-Verlag, Information obtaining a convenient degree of usability. The advances Systems Series, Bucarest - Romania (2006) 71-84 presented here, ASUR-IL model and related transformations, [5] Eclipse modeling Project - offer to rapidly navigate between the abstract design of http://www.eclipse.org/modeling/ innovative interaction techniques and their concrete realizations. The Domain Specific Language developed is an efficient tool [6] Hampshire, A., Seichter, H., Grasset, R., Bilinghurst, M.: for promoting the characteristics issued from the user-centered Augmented Reality Authoring: Generic Context from design, into the crucial phase of implementation. As this Programmer to Designer. In proceedings of the 20th approach uses models as primary artifacts, thanks to the MDE conference CHISIG of Australia, OZCHI’06, ACM Press, tools, each level of abstraction defined in the development Sydney – Australia (2006) 409-412 process embeds properties standing for the usability of the [7] Krasner, G.E., Pope, T.: A cookbook for using the Model- interactive system. View-Controller User Interface Paradigm in Smalltalk-80. In the Journal of Object Oriented Programming, (1988) ASUR model defines some properties related to the quality of 26-49 the interaction between a user and a mixed environment. Our goal is to plainly integrate them among the entire process, to