Generating user interfaces out of semantic models is still an issue because of the semantic gap between ontologies and user interfaces. We bridge this gap through semantic model-driven development. More precisely, we show how to automatically generate high-level interaction models (in the form of communication models representing discourses) out of (annotated) ontologies, using model-transformation rules. From these discourse models, user interfaces can be generated (semi-)automatically.
In a specific category of interactive systems, such as product
recommendation systems, reservation systems or shopping
applications, the underlying (semantic) model may strongly
influence the behavior of the systems and, therefore, also the
interactions to be implemented through the user interfaces.
For this category of interactive systems, we address the
semantic gap between underlying ontologies and user
interfaces. We make use of our discourse models [
IUI SEMAIS Workshop 2011.
Copyright is held by the owner/author(s)
The remainder of this paper is organized as in the following manner. First we give a brief background on our previous work relevant for this paper, and compare it with some of the related work in the field. Then we present our approach for generating interaction models out of (annotated) ontologies. This approach contains two parts: the generation of discourse models representing the flow of communication between the user and the computer, and the generation of domain-of-discourse models representing what they “talk about”. Finally we conclude and provide an outlook of our future work in this direction.
Our previous work focused on manual modeling of
interaction designs [
Instead of our discourse models, ConcurTaskTrees from
Paterno et al. [
Ontology Set of Individuals
Domain-Speci c Constraints for
Discourse
Model Rule-Based Transformation
Step 1
Discourse Model (without content description)
Domain-Speci c Constraints for
Domain-of
Discourse Model Rule-Based Transformation
Step 2
nication, and they can be used for machine-machine
communication as well [
UsiXML [
Paulheim and Probst [
Now let us present our approach to automatically transforming an ontology to a high-level interaction model in the form of a specific communication model by using model transformations. We focus on a small part of an ontology and its corresponding transformations to generate the interaction model of a Product Advisor for digital cameras as a running example. The Product Advisor is designed to ask questions about desired properties of a digital camera to be bought.
In Figure 1, we provide an overview of the transformation process, which consists of two steps for generating a communication model from an annotated ontology. We start from such an ontology represented in OWL1 (illustrated in the left part of Figure 1). While the ontology per se con1Last visited on December 10, 2010: http://www.w3.org/ TR/owl2-overview/ 2Last visited on December 10, 2010: http://www. heppnetz.de/projects/goodrelations/ Copyright is held by the owner/author(s)
COMMUNICATION MODEL
Discourse Model Domain-of-Discourse Model
Rule-Based User Transformation Interface tains the knowledge of the given domain, the annotations contain meta-knowledge, e.g., the priority of a given piece of knowledge with respect to the Product Advisor to be implemented. The result is a communication model consisting of a discourse model and a domain-of-discourse model. We use a GoodRelation2 ontology for digital cameras as a basis. In Figure 2, we depict selected parts from the Digicam GoodRelation ontology. The top concept Thing is specialized by the concepts ProductOrService and DomainSegment. ProductOrService is further specialized by the Digicam concept. DomainSegment groups together properties of a ProductOrService that have a semantic relation with each other.
Our ontology contains additional annotations that describe characteristics of certain datatype and object properties with respect to the intended Product Advisor. For example, the annotation priority specifies how important for the Product Advisor a specific object or datatype property is compared to other properties. These priorities are a distinguishing feature when the transformation process applies the transformation rules. The priority is an integer value between 0 (low priority) and 100 (high priority). The priorities allow the transformation process to decide which datatype and object properties are of interest for the discourse and the domainof-discourse.
In the first step, a set of model-transformation rules matches
parts of the ontology (including its individuals) and
transforms them automatically into corresponding parts of a
discourse model (see the middle part of Figure 1). These
transformations are subject to domain-specific constraints
explained in detail below. The discourse model generated
in this step represents only the generic communication flow
[
In the second step, our model-transformation approach transforms the individuals of the ontology and their concrete datatypes and object property values into a domain-ofdiscourse model. Here we apply domain-specific constraints for a domain-of-discourse model. In effect, this step defines the content of the communicative acts so that the discourse model refers to the domain-of-discourse model. The contents of communicative acts in the case of a Product Advisor are concrete question and answer texts that link to elements of the domain-of-discourse model.
Our transformation approach applies several rules to create a discourse and a domain-of-discourse model out of the ontology. Each rule application can be constrained by domainspecific constraints, that are externally configured. We have a logical rule chain (by using Operational Query/View/Transformation3 (QVT)) defining the application order of the rules. In principle, a rule that is applied later in the transformation process can influence the outcome of a rule that is applied sooner. In the following, however, we describe two independent rules applied in the transformation process, the DomainSegmentClusterRule and the SingleQuestionRule. The first rule explained as an example is the DomainSeg3Last visited on December 10, 2010: http://wiki. eclipse.org/M2M/Operational_QVT_Language_ %28QVTO%29 Copyright is held by the owner/author(s) mentClusterRule illustrated in Figure 3. It matches the concept DomainSegment in the ontology, which has several individuals. For example, the digital camera ontology has the domain segments EnergySupply, LensFeatures, Ports, etc. All datatype and object properties in the ontology that are related to a DomainSegment via the object property belongsToDomainSegment are of interest for our transformation process.
The rule DomainSegmentClusterRule creates a cluster of questions for all object and datatype properties that belong to the same domain segment. A cluster groups questions that hold a semantic relation (e.g., the ports USB and FireWire belong to the DomainSegment Ports). Such a definition of a cluster results in a Joint RST relation of a discourse (see the right part of Figure 3). The datatype and object properties are transformed into question/answer pairs that are branches of the Joint relation. In addition to the rule presented above, the following domain-specific constraint applies: Each property needs a minimum priority value (e.g., 20) to be included in a cluster. The minimum priority value is configured a priori in the domain-specific constraints.
After the DomainSegmentClusterRule has been applied, all properties with a minimum priority are grouped in the different clusters. For example, USB belongs to the segment Ports. Now a rule applies that combines all Boolean properties (like USB) of one domain segment into one question, for optimizing the interaction with the Product Advisor. The left part of Figure 4 shows the datatype property USB, which represents the USB port of a digital camera having the priority 85. For this USB property, the SingleQuestionRule takes Domain-Speci c Constraint Create cluster for domain segments that have properties with priority > 20 Domain-Speci c Constraint Create question/answer pair ONLY if priority of property > 70 effect now. Due to its high priority value (and importance), USB becomes a single ClosedQuestion-Answer pair again. This adjacency pair, shown in the right part of Figure 4, becomes also part of the generated discourse model. As shown with these example rules and their applications, for each selected property a corresponding question is being generated for the Product Advisor, since this property is considered important for the selection of a camera. While the ontology specifies what exists in the domain, the process being implemented in the Product Advisor contains related questions. This semantic gap is bridged by our approach in the context of the given application.
We show an excerpt of a yet incomplete discourse model in Figure 5, that is the result of the first transformation step depicted in Figure 1. The contents of the communicative acts are URIs that refer to datatype or object properties in the ontology. A Joint relation combines one Adjacency Pair and the Background relation connecting two more Adjacency Pairs. In this example, these properties have a high enough priority, so that they have to be grouped together in a special cluster at the beginning of the recommendation process of the Product Advisor. The first question gathers information on the price range, defining the minimum and maximum price that the user is potentially willing to pay. The second question elicits the interest for a USB port on the digital camera. This Boolean question is modeled as a closed question. Moreover, there is an RST relation Background intended to optionally inform the human user on additional details about the subject matter, e.g., more information on USB.
The second step in the transformation from ontologies to our communication models is to generate the domain-ofdiscourse model. This model represents the content of the communication, more precisely the content of the communicative acts within our discourse models. For example, the digital camera’s property hasCurrencyValue (representing the price of the camera) is the content of the question Copyright is held by the owner/author(s) where the Product Advisor asks the user about his or her preferences (e.g., price range) regarding the camera price (shown at the top of Figure 5). The Product Advisor should only ask for relevant product properties and their values. For example, the prices of all cameras should be within the price range offered for selection. So, the set of individuals of the products is used to generate the possible contents of the communicative acts, e.g., to determine their price range. So, for the content of each question in the discourse model, a unique datatype representing the product property is generated in the domain-of-discourse model. Figure 6 shows a small excerpt of such a generated domain-of-discourse model. For product properties representing numbers (e.g., price), only the minimum and maximum values are relevant for the Product Advisor (e.g., to generate a slider in the final UI for selecting the preferred value between the minimum and maximum). These values are stored together with the generated datatype. The left part of Figure 6 shows the datatype of the price property realized by a Float number. The minimum and maximum values are displayed as an annotation in a note below the datatype. For product properties representing Boolean values, the concrete individuals do not have to be searched for possible values, of course. As an example of such a Boolean datatype, the USB datatype is shown in the middle of Figure 6. For all other properties, an Enumeration datatype is generated for storing all possible values. The right part of Figure 6 shows the Enumeration type generated for the producer datatype. The values of the enumeration are derived from the set of all camera producer individuals in the given ontology.
The applications of these transformation rules can be influenced by domain-specific constraints specific for the domain-of-discourse model. For example, if no values for a specific property exist in the set of individuals in the ontology or if all of them are same (e.g., if all cameras have a USB interface), then the content of the question would be empty, so that the whole question is deleted from the discourse model.
To ease and speed up the development of ontology-based interactive systems, the automatic generation of their user interfaces would be advantageous. However, due to different perspectives as well as technical and conceptual foci of ontologies used in such systems, the generation of user interfaces directly from ontologies would be hard. We use a high-level interaction model in the form of a communication model based on discourses as an intermediate language. In this paper, we explain the automatic generation of such models out of (annotated) ontologies, and taking applicationspecific constraints into account.
From such communication models, user interfaces can
be generated (semi-)automatically, as we have previously
shown already [
This research has been carried out in the SOFAR project (No. 825061), funded by the Austrian FIT-IT Program of the FFG and Smart Information Systems GmbH.