Modeling and building interactive user interfaces (UI) typically requires the skills of software developers and HCI experts who cooperate with platform and marketing experts in order to arrive at solutions with the required software quality, usability, and user experience. The combination of model-driven user interface development practices with pattern-based approaches that specify HCI- and softwarepatterns in a formalized way and respect emerging standards offers potentialities to facilitate and at least partially automate the user interface development process, therefore reduce the time-to-market and development costs, and lead to solutions that can easily be adapted to varying contexts and target devices. Such pattern-aided UI adaptation is not limited to design time decisions but can also be applied during runtime. This paper highlights the architecture and capabilities of the Pattern-Based Modeling and Generation of Interactive Systems (PaMGIS) framework to broadly support the construction and adaptation of user interface models. It is discussed, how pattern descriptions that capture important parts of the design knowledge should be organized in order to be automatically processed during the modeling process.
INTRODUCTION Highly interactive software has become a crucial ingredient of modern human life. Independent of time and location, people are used to interact with products built around interactive software components, such as web applications, telecommunication devices, car navigation systems, smart home appliances, wearables, or other electronic equipment. Nowadays users expect that software products run on a Workshop on Large-scale and model-based Interactive Systems: Approaches and Challenges, June 23 2015, Duisburg, Germany. Copyright © 2015 for the individual papers by the papers’ authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors. variety of heterogeneous devices with a consistent look and feel, invariable high usability, and an extremely high degree of appealing user experience. Additionally, users tend to be impatient and want to have the software available for their different devices at the same point in time. Therefore, timeto-market is vital for software suppliers.
It is nearly impossible to meet all requirements simultaneously when exercising traditional software engineering and development processes. A promising way out of this dilemma is the application of a model-driven approach that allows for describing the particular aspects of the intended user interface by means of distinct models at different abstraction levels which can be created - at least partially by automatic transformations.
We have combined model-driven user interface
development practices with pattern-based approaches that
specify HCI-patterns in a formalized way [
RELATED WORK
Model-based user interface development environments
(MB-UIDE) introduce models to the development process
of interactive applications. A variety of existing MB-UIDE
and model-driven approaches for facilitating the
development process of interactive systems can be found in
the literature. Related recapitulations and discussions are
provided in [
The role of the various models used in MB-UID
environments varies with respect to the modeling purpose.
Typically more than one model is exploited during the
development process to construct the desired solution
interactively or (semi-) automatically. Some degree of
standardization was brought into the diversity of
MBUIDEs by the CAMELEON Reference Framework (CRF)
[
Hence, descriptions of HCI patterns are equipped with model fragments that on one hand can be used as building blocks for the diverse models and on the other hand allow influencing model transformations. In addition, usability evaluation results can be fed back in order to draw conclusions and improve the patterns, models, and the resulting user interfaces. The framework supports our research with respect to the potentials and limits of automated UI development. In order to enable automatic processing, all model entities as well as pattern specifications are expressed and stored in an XMLcompliant format.
Model-driven Aspects
The model-driven part of the framework as illustrated in
Figure 2 is designed in the style of the CAMELEON
Reference Framework. Particularly, the ontological domain
and context-of-use models are used as proposed by the CRF.
However, we decided to split the CRF platform model into a
device model and a UI implementation model. While the
former comprises all relevant characteristics of the respective
end device the latter holds information about the UI elements
that are available on the respective underlying software
platform. This avoids redundancies especially in cases where
the same software basis supports significantly different
devices, e.g. Android on smartphones and tablet computer.
The framework is organized in six abstraction levels, i.e,
domain, context of use, abstract user interface (AUI), concrete
user interface (CUI), final user interface (FUI), and runtime
level. As the most abstract representation, the domain level
embodies the domain model which in turn consists of the task
and concept sub-models. The task model provides information
of domain-specific user goals and the entirety of process steps
and actions which must be executed in order to attain these
goals. The concepts model can be understood as a type of data
model describing all UI-relevant data elements and artifacts
which are required in the course of task completion. Hence,
these two models are closely interrelated. In the context of
PaMGIS, the task model is represented in a ConcurTaskTrees
(CTT) notation [
The knowledge captured within the domain model is used to construct an abstract user interface model which is a canonical representation of the rendering of the domain concepts which is independent from the actually available UI elements as specified within the UI implementation model. At this juncture, the concepts model indicates which AUI objects are required while the task model’s hierarchical structure and inherent temporal dependencies enforce the definition of the relationships between these objects. A list of feasible AUI objects is provided Table 1.
The information contained within the context-of-use model is
used to control the subsequent transformations of the diverse
UI models and to substantiate deliberate design decisions. For
instance, some tasks or sub-tasks might be undesired,
impractical or impossible to be carried out within a certain
context of use due to user-, device-, and/or
environmentrelated restrictions. In this case, the corresponding parts of the
AUI have to be eliminated. Furthermore the design of the
dialog structure is defined in consideration of the given context
of use by means of dialog graphs [
Once the AUI model is completed, it can be transformed into a concrete user interface model. For this purpose, the abstract user interface objects are replaced by appropriate concrete ones. In this sense, the most appropriate CUI object is the one that fits best to both the requirements and restrictions which result from the various aspects of the context-of-use model. Further, a first impression of the final look-and-feel is created by roughly determining the layout, i.e., positioning, and the appearance, e.g. color, font, and size, of the CUI objects. In a last step, the final user interface can be automatically generated from the CUI model.
Figure 3 recapitulates the necessary transformation steps between the four different levels of abstraction as specified in the CAMELEON reference framework. The process starts with the domain model followed by the abstract and concrete model levels and finally arrives at the final user interface. Please note, that the framework user may perform manual adjustments at any step of the development process. From a runtime perspective, there are three general options how to deal with FUIs. Firstly, the FUI is available as source code that can be transformed into an executable format by means of a compiler. Secondly, the FUI has the format of a script that can be executed by an interpreter. Thirdly, the FUI can be executed by a runtime engine provided with the development framework. The advantage of such a runtime engine is that it is not necessarily bound to the FUI level, but can also create at least executable UI prototypes from higher abstraction levels, i.e., CUI and AUI models, and therefore enables framework users to identify design problems in early stages of the development process. The utilization of default values within the respective model allows executing UI prototypes on the basis of not yet finalized models. In addition, the use of a runtime engine also allows for implementing model-based responsive designs and runtime adaptive behavior of the user interface.
Pattern-based Aspects Within our combined development approach, patterns are used as means to alleviate the complexity of the modeldriven processing. The patterns provide pre-assembled building blocks which can be used for domain and UI model construction. In addition, certain patterns provide valuable input to the various model transformation steps shown in Figure 3.
For this purpose it is essential to specify the patterns in a uniform and machine-readable manner and equip them with the required information. Further, it must be possible to compose pattern languages, i.e., to define the interrelationships between the patterns.
Hence, we developed the PaMGIS Pattern Description Language (PPSL) which is suitable to fulfil the aforementioned requirements. We reviewed existing pattern description standardization approaches as well as pattern tools in order to define PPSL in a way that patterns which are specified in the related formats can be transformed to PPSL. Thus, the entirety of all PLML 1.1 description elements is covered in PPSL, where required in a restructured or modified form. The only exception is the PLML element Evidence which is not directly included, but whose two sub-elements Example and Rationale are part of PPSL. Further, the PLML description element Literature can be mapped to References and Related-Patterns to Relations.
In addition, we introduced new description attributes for storing the supplementary information required by PaMGIS. An overview of the description elements of PPSL is provided in Figure 4. Pattern specifications are organized in four top level elements, i.e., Head, Body, Relationships, and Deployment.
The Head element incorporates metadata such as unique pattern identification, pattern classification, pattern name and aliases, information about pattern authors, credits, pattern evolution, and references to further sources and literature. The Body element is split into the two subelements Theory and Practice. The former provides theoretical background, including – amongst others – descriptions of the underlying problem, the context in which the pattern can be applied, and the proposed solution of the given problem. The latter demonstrates how the pattern was applied in practice by means of illustrations, examples, and counter-examples. The Relationships element serves as resource for the specification of the relationships between the various patterns and therefore allows the construction of pattern languages. Finally, the Deployment element contains – amongst others – model fragments of different types and abstraction levels as usability feedback. The model fragments are used as building blocks for the domain and the diverse UI models. The model fragments are stored within the Deployment/PaMGIS/ModelFragments element and provide ready-to-use modelings of the pattern’s inherent solution. During the process of constructing the domain model, the framework user can search, select, and apply patterns, i.e., automatically insert the respective task and concept model fragments into the domain model. It is also possible to store prefabricated AUI, CUI, or FUI model fragments with the pattern which can be directly embedded into the UI models of the corresponding abstraction levels. While patterns typically contain only one task and one concept model fragment, they might possess multiple AUI, CUI, and FUI model fragments for different contexts of use. This allows both applying different UI design solutions during design time and even during runtime, i.e., substituting one model fragment by another one. This mechanism is not limited to model fragments of the selfsame pattern. In fact, it is even possible to substitute whole patterns by alternative ones.
Regarding the process of finding appropriate patterns the framework offers multiple methods: pattern browsing, keyword search, free text search, exploiting pattern relations, or evaluating formal context descriptions which are stored as logical expressions within the Body/Theory/Context/Digest element.
Usability Evaluation Aspects Running user interfaces – either on the basis of a complete FUI or in form of a prototype based on more abstract UI models – can be evaluated in terms of their usability and user experience using pertinent techniques and methods. The evaluation itself is not in the scope of PaMGIS. Hence, the framework does not offer any support for evaluation preparation, execution, and post-processing. But it is possible to document relevant insights within the system. Since the origin of model elements is captured inside the PaMGIS domain model and the various UI models, it is possible to locate the respective pattern and post the evaluation results to the pattern definition. For this purpose we introduced the pattern description element named Body/Practice/UsabilityFeedback.
A second, more automated option is to specify and utilize special usability evaluation (UE) patterns. They can be integrated in the domain model where they add some measuring instrumentation. For instance, the Textual User Usability Feedback Dialog pattern ensures, that an appropriate dialog is available allowing the user to record and send his or her opinion about certain aspects of the user interface at hand back to the PaMGIS framework. In the simplest case, this dialog is composed at least of an Output object providing some textual explanations for the user, an Editor object for the acquisition of the actual textual user feedback, and two Activators for either submitting the feedback or canceling the action. The aforementioned pattern includes the required task and concept model fragments as well as AUI and optionally less abstract UI model fragments. In this sense, the underlying domainspecific pattern language can be enriched by such UE patterns in order to capture usability feedback. At least in the case that the user interface is executed by means of the runtime environment, it is possible to automatically attach the user feedback directly to the respective pattern. Otherwise the information can be temporarily stored in a log file outside the scope of PaMGIS and fed back manually or in a semi-automatic way at a later point in time.
The collected usability feedback can be used to improve the quality of the patterns, the diverse models, and therefore of the final user interface.
Functional Framework Architecture The PaMGIS framework consists of several logical function units, each supporting the various users in different fields of activities. An overview of the functional framework architecture is provided within Figure 5.
The core components are the two repositories, the Pattern Repository for storing the pattern specifications and the Model Repository to accommodate the diverse models as shown in Figure 2.
Access control is managed by means of the User Database which is administered via the Framework User Administration component. PaMGIS distinguishes several general types of users, i.e., unregistered users, registered users, pattern authors, power users, and administrators. Unregistered users are allowed to access a restricted part of the pattern specifications solely in read-only mode. In addition, they may register themselves to the framework. Registered users gain more insight into pattern details, have very limited write permissions, and may use certain collaboration functions, such as sending messages to pattern authors. The pattern authors have full access to the pattern descriptions and may create new patterns and modify existing ones. Power users can read entire pattern specifications and are allowed to make copies to which they have full read and write access. In addition, they can use and control the model-driven part of the framework. Finally, administrators take over the responsibility of managing the framework, e.g. creating, modifying, and deleting users, granting and withdrawing access rights, and maintaining the PaMGIS meta-models via the Pattern Meta Model Administration and the Model Meta Model Administration components. On the one hand, the Pattern and Pattern Language Administration unit supports pattern authors in creating and modifying patterns. On the other hand, power users can copy particular patterns to a private workspace where they can modify them according to their needs and build up pattern languages by specifying interrelationships between patterns.
The Pattern and Pattern Language Dissemination tool can be used by unregistered users to browse, search, and display certain aspects of the pattern descriptions which are released for this purpose. Additionally, it allows registered users to view more pattern details, send feedback and comments to pattern authors, and attach information about existing implementations to the pattern specifications. For this purpose we introduced the Body/Practice/KnownUses description element.
The Domain Model Editor, Context-of-Use Model Editor, and UI Model Editor allow power users to create and modify the respective PaMGIS models manually. In contrast, the Pattern Selection and Assignment component helps power users to search and find adequate patterns which can be selected and applied, i.e., insert the attached model fragments automatically into the domain, context-ofuse, and/or different UI models. The Model Transformations unit supports the execution of the model transformations summarized in Figure 3 and can be configured to a certain extent.
The Runtime Environment is a means to execute user interfaces in the form of final UIs or prototypes as described above. The File Export component is used for exporting models in the form of text files for further external processing or documentation purposes.
Finally, the Usability Feedback unit offers support regarding the import of usability evaluation results into the framework and write it back to the respective patterns and/or models.
CONCLUSION In this paper we presented our concerted pattern-based and model-driven approach for the development of interactive ubiquitous systems and provided an overview of the functional architecture of the related PaMGIS framework. We strongly believe that the mélange of model- and pattern-related methods and techniques has the potential to alleviate weaknesses of the individual approaches and can create benefits in terms of reducing complexity and realizing reuse of already existing design knowledge. The implementation of the framework is indeed work in progress, but major components already exist at least in prototypical form. Many patterns and several pattern languages have been developed, amongst others a pattern language for the domain of public transportation ticket selling.
The framework is a cornerstone for our further research on the potentials and limits of automated UI development. Moreover, we will intensify our work on supporting wearable computers with the PaMGIS framework.