Software tools are being used by experts in a variety of domains. There are numerous software modeling editor environments (MEs) tailored to a speci c domain expertise. However, there is no consistent approach to generically synthesize a product line of such MEs that also take into account the user interaction and experience (UX) adapted to the domain. In this position paper, we propose a solution to explicitly model the UX of MEs so that di erent aspects of UX design can be speci ed by non-programming experts. Our proposal advocates the use of multi-paradigm modeling where this aspect of the design of an ME is modeled explicitly and adapted to a speci c user expert.
Software modeling refers to the use of software to model a solution for a problem
in a speci c domain (e.g., music, nance, biology). As such, there is a plethora
of software tools that enable domain experts (e.g., musicians) to represent,
manipulate, and simulate models using notations from the domain, speci ed at
a suitable level of abstraction (e.g., a music sheet) [
Model-Driven Engineering (MDE) [
In most current approaches to ME development, such as the Eclipse Modeling
Framework (EMF) [
With the popularity of new computational devices and peripherals at the
expense of traditional desktop environments, such as tablets with varying sizes,
virtual and augmented reality goggles, and collaborative interactive tables, new
human user interactions will keep on appearing. It becomes clear that the
interaction with ME, needs to be adapted in new ways. For instance, works such
as [
In this position paper, we propose the idea to promote the UX into an explicit
component of ME creation, at the same level as the AS and CS of a DSL. This
would allow us to tap into knowledge from the UX domain [
In Section 2, we brie y discuss about work done on the development of user interfaces. In Section 3, we extend the MDE approach for generating MEs by tackling the speci cation of the UX. We illustrate our approach with a simple music modeling language example and its editor. Finally, we discuss our roadmap in Section 4. 2
The closest work to addressing the interaction issue in some standard or formal
way is the recent standardization of the Interaction Flow Management
Language (IFML) [
There are nevertheless several aspects of IFML that do not concord with our approach. IFML describes user interfaces (UI) as constituents without any speci cation of visual properties or design choices. This limits the description since these properties have a large in uence on the UX and should be adapted accordingly. IFML is targeted to Software Developers instead of UX designers that could use their domain knowledge to better adapt the software to its users. It promotes the combined use with other OMG standards, such as Class Diagrams and Business Process Models, but through declaration of speci c function calls that e ectively bind the user interaction to a speci c implementation, obfuscating most of the abstraction e ort. It also relegates complex interactions to be speci ed by the implementation framework instead of being an explicit part of the interaction model. Finally, our evaluation of IFML also concluded that it has some limitations in terms of scalability, where the complexity of the speci cation increases greatly with each UI element that is introduced, and the use of modules is focused on implementation reuse rather than development simpli cation.
In our previous work [
Our rst step to tackle the speci cation of user interactions for a DSL is to break
it into di erent models, each focusing on a di erent aspect of user interaction
description. This allows us to specify each aspect in its own domain-speci c
model, with its own concepts and targeted to the appropriate experts. From the
knowledge acquired in [
To illustrate our proposal we use a case of a music modeling language akin to
MusicXML [
The language provides a form for writing music sheets. In this paper, we focus only on a small number of interactions: various ways of placing notes on the sheet, playing a note, and selecting a note length (half note, quarter note ...). The goal here is to demonstrate how we address multiple interaction requirements of a music modeling language, so that the nal ME is properly adapted to the music domain expert and the environment he uses. 3.2
In our approach, domain-speci c MEs are not only generated form the AS and
CS, speci ed by the language modeler using common MDE techniques [
Group {ordered}
group
ClipPath
Marker packagedMarker size]:]Dimension reference]:]Point *
Canvas backgroundColor]:]Color][0..1] * backgroundFil 0..1 package*dFil packagedSty*le
Fil Style depicts an excerpt of the Diagram Graphics metamodel that speci es the definition graphical elements and the segment of the metamodel that allows the speci cation of element groups and other organizational constructs, such as the canvas. However, these are not enough to de ne the topological placements of UI elements, in a variety of dimensional spaces from 0D (Numerical Display, Single speaker) to 2D (Screen, binaural Audio, Touch Tables), that we require for UI speci cation. Thus our proposal is to extend the DD metamodel with constructs that would allow for a better speci cation of UI, as shown in Fig. 2. cardinalAnchors
1 DD: GraphicalElement 5 Anchor * 1 Layer * DD: member {ordered} 0. 1 DD: group TopologicalGuard * *
DD: Group * anchors packagedAnchors packagedGuards
DD: Canvas
Widget
AudioElement audioResource: le
HapticFeedback packagedLayers * {ordered}
1 packagedCanvas 1
1 1
InteractionStream
We start by organizing these spaces as Interaction Streams (IS). These represent forms of interaction with the user that are not representable in the same space, e.g., screen elements, keyboard inputs, and sound. Fig. 3a shows an Interaction Stream, namely the representation of screen elements for our music ME. Fig. 3b shows a note input stream and a sound stream, both complementing the rst stream, but with no direct visual representation as a screen element. These abstract inputs can then be mapped onto more concrete forms of input, such as an electronic piano keyboard, but at this level, we only allow for the inclusion of platform-independent input.
These IS are in turn organized into Layers. Layers group representational characteristics of the UI elements such as shade, size, position, data, audio characteristics, representations of haptic feedback, and other perceptible characteristics, and how to group them, in the form of DD Graphic Elements (GE).
These GE are then specialized into UI speci c elements such as buttons, check boxes, drop down menus, and other UI widgets, to simplify the modeling process. Fig. 3a shows the de nition of such GE: we can have the typical menu and button widgets positioned at the top, with a representation of a selected button for illustration purposes, and the remaining space as the canvas.
Additionally all of these GE are extended with a set of cardinal anchors and a central anchor. Alongside these automatic anchors, manual unmovable anchors and guides can be placed in the model. These allow us to have topological constraints in the form of Guards between any such anchors, including di erent anchors of the same GE. Through these topological constraints, we can de ne
Edit
Help 2RSSTU-+$%&(-$%&(RSSTU-$%&(-$%&(-$%&(RSSTU-$%&(-$%&(-$%&(RSSTU-$%&(-$%44-4444=
(a) UI screen model example C1 C#1 D1 D#1 E1 F1 F#1 G1 G#1 A1 A#1 B1
C2 C#2 D2 D#2 E2 F2 F#2 G2 G#2 A2 A#2 B2 (b) UI abstract input and sound output model examples positional and scale restriction on elements, minimum and maximum distances between elements, minimum and maximum dimensions, and automatic alignments. This is achieved by adjusting the movable anchors pointed by the Guard so that its constraint expression is satis ed. The metamodel presented is further constrained by well-formlessness rules not represented here, such as: the number of Position elements of an Anchor matches the number of dimensions of that particular Interaction Stream.
For representation purposes, we also show the positioning of CS elements from the Music Modeling DSL in the canvas. These CS elements align at the top left anchor in the canvas and also conform to the Music Modeling DSL CS and AS speci cations. The two remaining elements on the canvas are the representations of a cursor as a line with a marker over the CS, and a pointing device as a large cross.
List of Essential Actions This is a documentation model, contains a listing of all Essential Actions the ME must perform. These are common actions (e.g., copy, paste, save) and actions speci c to a modeling language (e.g., instantiate elements in particular ways, perform checks at key points of the modeling process). For our example, we only consider copy and paste actions, as well as actions speci c to music modeling: placing a note, playing a note, and selecting a note type. This model is populated by a domain expert to de ne the actions speci c to a language and by a UX designer to de ne the common actions. List of System Events This documentation model lists all System Events provided from the target platform (e.g., operating system). For example, for a computer with a touch screen, mouse , and keyboard, events such as OnLeftClick, OnTouch and OnKeyPress can be expected. This list presents platform-speci c events that will trigger a particular behavior of the ME when received. The list is populated by the software developer or architect responsible for the deployment on a given implementation.
Behavior Model This model expresses the logic of actions expected to take place while interacting with the ME. It de nes how the Interface Model reacts to Essential Actions received.
We propose to de ne behavior models in a hybrid formalism composing
Statecharts (SC) [
Pre conditions
External
Internal*
Post conditions
Goal conditions
Fig. 4 illustrates the structure of a specialization model: a SC to be
embedded in another generic one for specialization purposes. There are three kinds of
states: pre-condition, post-condition, and goal condition states. States contain
CS and Interface Model elements that specify the condition for a transition to be
triggered. Transitions can be external or internal. External transitions are
triggered by Essential Actions performed by the user and/or perform actions that are
perceived by the user. These transitions are to be mapped to platform-speci c
System Events. Internal transitions are triggered by actions internal to the scope
of the system and do not result from a human interaction, such as the passage of
time and transitions that are immediately triggered as soon as the source state
is reached. The semantics of the application of a transition t is as follows: if the
pre-condition of t is satis ed and the corresponding event is received, then the
system should update itself to satisfy the post- or goal condition. A pre-condition
state represents the left-hand side pattern of a model transformation rule that
shall be matched over the current state of the ME. A post-condition state
represents the right-hand side pattern of a model transformation rule that must be
satis ed after the event is received. Post-conditions also serve as pre-conditions
of following transitions. Goal conditions special kinds of post-conditions that
must be satis ed only when an internal event is received. Specialization
models must always start with an external transition. Conditional, hierarchical, and
parallelization constructs of SC, such as branching, forking/joining, and
ORand AND-states are all supported [
This approach allows us to easily de ne concurrent interactions, through the use of AND-states in the re nement of the generic behavior SC. In Fig. 5a and 5b we have two di erent interactions sequences to place a note on the music sheet. In Fig. 5a we have the note placement with a pointing device, where a UI speci c button element that will relate to note length is selected and the corresponding CS element is placed on the pointed place of the music sheet as an external action. In Fig. 5b we again have the selected length note placement, but instead it occurs on a note activation (note B1 is activated) and the corresponding CS element is placed by the external action wherever the cursor is currently placed, followed by an internal action that advances the cursor in a linear fashion. The note placement of Fig. 5a can then be used with any pointing device, such as a mouse or a touch screen. The note placement of Fig. 5b can be used with any device that allows the direct referencing of a note, such as a musical keyboard.
Fig. 5c shows the interaction of using a pointing device when a note is already placed at that point of the music sheet. Instead of an interaction with the CS by placing a new note as an external action, we play the sound of that note. This means that the play interaction will be placed on all states that stem from the placement interaction. Once the tone for that note has nished playing an internal action triggered by the system advances to an idle state.
The Behavior Model is aimed at UX designers that tailor the user experience to the domain expert users of the ME. In addition to all the model views of the interaction, we have a mapping between the platform-independent Essential Actions (EA) and the platform-speci c System Events (SE). This mapping is achieved through the left total function deploy : EA ! SE. This relation can be directly established if EAs are uniquely de ned, so that their context information (state of the ME) and UI elements are accessible through the EA information.
SE
teo tre EA tce lS laP on e ceN ionP tceeoN rrsouC y laP on l a
P
The mapping is speci ed in the form of a table, as shown in Table 1. It is produced by the UX designer to establish the relation between platformindependent external actions that interact with the user de ned in the list of essential actions and real world actions translatable by the platform-speci c system events. For example, in Table 1, we map multiple SEs (OnTouch and OnLeftClick) onto the same EA (Select) to provide alternative interactions and to adapt it to a particular device. Because each action is tied to its own context, the mapping of the same SE (OnLeftClick) can be mapped to di erent EAs (Select, Place Note on Pointer, Place Note on Cursor) without con ict. 4
In this position paper, we motivated the need to address the issue that generated MEs currently su er from: the lack of adaptation of its behavior to improve the UX of its domain expert users. Following a multi-paradigm modeling approach, we proposed to explicitly model the UX of a DSL in order to generate a ME where the domain user will feel completely immersed with interactions and utilization of the tool adapted to what he is accustomed to. The AS and CS of the DSL specify the syntax of valid models. The Interface Model represents the representation of the user interface. The CS of the DSL, Interface Model, and Essential Actions are used to de ne the Behavior Model of the ME. System Events are mapped to Essential Actions. All of these models are necessary to the synthesize MEs where the UX is adapted to domain on a platform using specialized I/O peripherals.
Our future work is to complete the full implementation of a prototype so we can start validating and improving our solution with real-world experts from di erent domains. We also plan to investigate di erent interaction streams that go beyond visual screen rendering, such as audio, video animation, and tactile haptic sensing supported by appropriate hardware devices. Furthermore, we plan to formalize the di erent models and formalisms outlined here as to provide precise feedback to the users for inconsistencies, well-formlessness and other design aws.