=Paper=
{{Paper
|id=None
|storemode=property
|title=A Formal Ontology on User Interfaces – Yet Another User Interface Description Language?
|pdfUrl=https://ceur-ws.org/Vol-747/paper9.pdf
|volume=Vol-747
}}
==A Formal Ontology on User Interfaces – Yet Another User Interface Description Language?==
https://ceur-ws.org/Vol-747/paper9.pdf
A Formal Ontology on User Interfaces –
Yet Another User Interface Description Language?
Position Paper
Heiko Paulheim and Florian Probst
SAP Research
Bleichstrasse 8
64283 Darmstadt, Germany
{heiko.paulheim,f.probst}@sap.com
ABSTRACT ontologies are different by nature. An ontology claims
During the past years, a lot of user interface descrip- to be a generic, commonly agreed upon specification of
tion languages, most of them based on XML, have been a conceptualization of a domain [6], with a focus on pre-
introduced. At the same time, the use of formal ontolo- cisely capturing and formalizing the semantics of terms
gies for describing user interfaces has been discussed used in a domain. A software model in turn is task-
for a number of use cases. This paper discusses the specific, with the focus on an efficient implementation
differences between a formal ontologies and user inter- of an application for solving tasks in the modeled do-
face description languages and and points out how both main [2, 16, 18]. Thus, a software engineer would rather
research directions can benefit from each other. trade off precision for a simple, efficient model, with the
possibility of code generation, while an ontology engi-
Author Keywords neer would trade off simplicity for a precise representa-
User Interfaces, Ontology, UI Description Languages, tion. Another difference is that in software engineering,
Formal Models models are most often prescriptive models, which are
used to specify how a system is supposed to behave,
while ontologies are rather descriptive models, which
ACM Classification Keywords
describe how the world is [1]. Figure 1 illustrates those
D.2.2 Software Engineering: Design Tools and Tech- differences.
niques—User Interfaces; D.2.11 Software Engineering:
Software Architectures—Languages; I.2.4 Artificial In- Taking this thought to the domain of user interfaces
telligence: Knowledge Representation Formalisms and and interactions, models are used to define particular
Methods—Semantic Networks user interfaces (e.g. with the goal of generating code
implementing those interfaces), while a formal ontology
General Terms would capture the nature of things that exist in the
Design, Languages domain, e.g., which types of user interfaces exist, and
how they are related.
INTRODUCTION
Recently, a number of use cases have been proposed that Due to those differences, we argue that developing a
employ ontologies for modeling user interfaces, their formal ontology on user interfaces will not lead to yet
components and interaction capabilities. Examples are another user interface description language, but to a
automatic generation of explanations for user interfaces, formal model with different intentions and usages. In
adaptation of user interfaces for different needs and con- the next sections, we will discuss how the two worlds
texts, and integration of user interface components [14]. can benefit from each other.
Those use cases require a strongly formalized ontology
of the domain of user interfaces and interactions. HOW A FORMAL ONTOLOGY CAN BENEFIT
In parallel, various UI description languages have been FROM UI DESCRIPTION LANGUAGES
proposed, most of them XML based [7, 12]. The duality A lot of research work has gone into the development
of UI description languages and formal ontologies gives of different user interface description languages. Those
rise to the question whether an additional ontology is research efforts can be and should be taken into account
really needed, or whether it is going to be yet another when developing an ontology of the domain.
user interface description language.
Collection of Concepts
ONTOLOGIES AND MODELS Most methodologies for ontology engineering foresee the
Although ontologies and software models are related, capturing of key concepts and relationships as one of the
they are not essentially the same. Software models and first steps. This can be done by conducting interviews
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
Different goals lead to
different models Goal: „complete picture“,
Goal: efficient
semantic account of terms
programming
in a domain
- tas
- pr k-spec a ch
e if ppro
- sim scriptiv ic app n e ric a e tion
p e roac - ge criptiv resenta
repr licity o h
e s e p Ontology
Software Model e se v d r
ntat er prec shared - ci se icity
ion i se - pre r simpl
conceptualization o ve
of a domain
develops develops
software developer ontology engineer
Figure 1. Ontologies and modeling languages serve different purposes.
with domain experts, scanning books and other mate- the development and usage of new and existing user
rial, and/or reusing parts of other ontologies [5, 19]. At interface description languages as well.
this point of ontology engineering, lots of input can be
used from existing user interface description languages. Disambiguation of Terms
In an analysis of user interface description languages,
Since those languages are most often XML-based, they
we have found that terms are often used differently in
consist of a smaller or larger number of tags and at-
different standards. An example is the term dialog. In
tributes, which determine the expressivity of the lan-
XIML, for example, a dialog element is defined as be-
guage. As many of those elements define certain con-
ing “like a command that can be executed [...] It is the
cepts of the domain, such as UI components or actions
more concrete instantiation of a task.” [15]. In contrast,
that can be performed with them, they are a good start-
XUL defines a dialog as an “element [which] should be
ing point for developing a formal ontology of the do-
used in place of the window element for dialog boxes”
main.
[10]. Such ambiguities can easily lead to misinterpre-
tations, especially if users are trained on a particular
Benchmarking the Ontology’s Completeness language and switch to another one.
As discussed above, ontology engineering aims at pro-
viding a complete, comprehensive formal description of Mapping a user interface description language to a for-
a domain. However, assessing the completeness of an mal ontology capturing the semantics of those terms
ontology is not always an easy task. Here, user interface can avoid such misinterpretations. With the exam-
description languages can once again help by providing ple term dialog, a formal ontology can help resolving
a benchmark for the ontology’s completeness. the ambiguity by indicating that the languages imply
different top-level categories such as Process, Plan,
Such a benchmark can be performed in different ways. or Software Component as super-category for Dia-
On the meta-model level, the number of concepts con- log.
tained in the meta model (e.g., tags and attributes in
an XML schema) which have a counterpart in the on- Facilitating Extensibility of User Interface Description
tology can be determined. On the model level, one can
check whether given models in a user interface descrip- Languages
tion language can be expressed using only the terms XML based languages usually use a fixed set of tags.
given in an ontology, either informally, or formally, e.g., In order not to be too strictly limited for practical use,
in RDF. Thus, user interface description languages can many of those languages provide some extension mech-
provide a measure for the completeness of an ontology anisms such as universal general purpose tags that can
of the domain. be used for user-defined concepts (e.g. the ELEMENT tag
in XIML). These extension slots are then filled with ar-
bitrary strings.
HOW UI DESCRIPTION LANGUAGES CAN BENEFIT
FROM A FORMAL ONTOLOGY Arbitrary strings, however, are dangerous. They lead
Once an ontology of the domain of user interfaces and to extensions that are incompatible with each other,
interactions has been created, it can be used to improve interpreted differently by different people and systems
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
ontology
engineer To end up with a comprehensive ontology, we have an-
alyzed several user interface description languages in
order to collect a maximum set of relevant terms. We
n develops
ctio s
have used UsiXML, XIML, UIML, Maria, XUL, LZX,
olle es
p t c k for eten WAI ARIA, and XForms as a basis for identifying the
n ce a r m p l
o m
- c e n ch y co
- b tolo
g core concepts.
on
rm
s In order to build upon well-acknowledged roots, we have
f te Ontology
Modeling no chosen the top level ontology DOLCE [9] and its exten-
a tio
igu
Language
i sa
mb ibility and
n
sions as a basis for our ontology. This top level ontology
- d xtens riso
- e ompa rsion provides an embracing basic classification of things and
c
- onv e
develops c has been used as a basis for building numerous ontolo-
gies. Since the top level provides a complete classifica-
modeling language
designer and user tion, it ensures extensibility of the ontology by design,
as every new concept can be classified in some existing
Figure 2. How user interface description languages and category. Furthermore, we have reused two core ontolo-
ontologies can benefit from each other gies of software and software components [11], which
are also built upon the foundations of DOLCE.
relying on different conventions and external documen- The ontology we have developed is divided into two
tations, and, in the end, foil the overall idea of having parts: a top level which captures the semantics of the
a standardized modeling language. basic terms of the domain, such as User Interface Com-
ponent and Interaction, while the detail level classifies
A formal ontology can help here by providing a stan- the actual things that exist in the domain, such as types
dardized vocabulary which can be used to fill such ex- of user interface components and user tasks that can be
tension slots. Thus, it can be assured that there is an performed with those components. The OWL version of
unambiguous interpretation of the extensions. the top level ontology consists of 15 classes, two object
properties, and 75 axioms, while the detail level consists
Model Comparison and Conversion of 179 classes, eleven object properties, and 448 axioms.
When bringing together different development teams,
information systems, or organizations, it is likely that
models created with different user interface description CONCLUDING REMARKS
languages already exist. Using a mediating ontology for This position paper has discussed the differences be-
annotating the models is a common way of establishing tween UI description languages and a formal ontology
comparability between models, not only user interface of the domain of user interfaces and interactions. Fur-
models [4]. thermore, We have given insight into the development
of a comprehensive formal ontology of the user inter-
Once models are annotated and can be compared using faces and interactions domain. In the long run, we are
a common ontology, automatic conversion of models can confident that formal ontologies and UI definition lan-
be long-term objective. For the moment, a common on- guages will both have their places, and that both will
tology can at least support developers in understanding benefit from each other.
each other’s models and assist them in unambiguously
transferring their contents between modeling languages We have presented a number of potential improvements
manually. where developers employing user interface description
languages could benefit from those languages being map-
Fig. 2 summarizes how modeling languages and a formal ped to a formal ontology of user interfaces and interac-
ontology can benefit from each other. tions. Thus, our claim is that organizations providing
user interface description languages could improve the
usability and acceptance of those languages by provid-
TOWARDS A FORMAL ONTOLOGY OF THE DOMAIN
ing such a mapping.
OF USER INTERFACES AND INTERACTIONS
With these considerations in mind, we have started to As a long-term objective, such a mapping could even fa-
develop a formal ontology of the domain of user inter- cilitate automatic conversion between models developed
faces and interactions. The goal is to end up with an with different user interface description languages. To
ontology that is comprehensive at least with respect to that end, more sophisticated mapping approaches than
the expressivity of current user interface definition lan- simply relating elements form a modeling language to a
guages, that is universal enough to be extendable to category in an ontology are needed [13].
future user interfaces that do not exist at the moment.
Furthermore, to support valuable reasoning on user in- A formal ontology will not replace user interface de-
terfaces and provide meaningful semantics, the ontology scription languages, but be a valuable enhancement.
should be highly axiomatized. Due to the conceptual differences between software mod-
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
Description of Description of
Interactions Components
dns:Situtation dolce:Particular / owl:Thing io:Information Object
dolce:Region
plan:task-precondition plan:task-postcondition cso: data type
cso:identifies cso:Data
plans:component dns:Task
cso:Abstract Data cso:Software
(Representational Data) (Executable Data)
cso:Computational Task User Task
cso:identifies cso:User Data cosc:Software Component
Detail Level:
Display, Highlight, Detail Level:
Delete, ... Select, Organize, ...
dolce:part User Interface Component Processing Component Storage Component
dns:Plan
dns:defines dns:expresses dns:realizes
Detail Level:
Label, Image,
dolce:part Interaction Plan
Button, Text field, ...
Design Time
Run Time
dolce:Non Agentive
dolce:event dns:Agent dolce:Non-Physical Object
Physical Object
dns:sequences makes use of
≡ fp:use of
dolce:accomplishment dolce:specifically
cso:Computational Object Hardware dolce:part
constantly depends on
fp:performs
dns:action
Peripherical Hardware Non-peripherical Hardware
tr:causally follows involves as tool
dns:activity ...
≡ fp:instrument
Visual Computational
Tangible Hardware Object Monitor
Object
cso:Computational Activity User Activity dolce:has quality dolce:has quality
dolce:Physical Quality dolce:Region
provides
...
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
Detail Level:
Click, Type, Style dolce:spatial-location-quality Screen Region adjacent to
Scroll...
Copyright is held by the author/owner(s)
... ...
Position on Screen dolce:q-location
Figure 3. The top level of the ontology of the user interfaces and interactions domain. In the upper part, the design time concepts are shown, the lower
part contains the run time concepts. The left part deals with interactions, the right part with components. The white ellipses denote concepts from the
reused ontologies (with the following namespace conventions: DOLCE (dolce), Descriptions and Situations (dns), Plans (plans), Functional Participation
(fp), Temporal relations (tr), Core Ontology of Software (cos), Core Ontology of Software Components (cosc)), the grey ellipses denote concepts from the
top level ontology of the user interfaces and interactions domain. The grey triangles denote definitions carried out in the detail ontology.
els and ontologies, user interface description languages Ontology Library (final), 2003.
do a better job, e.g., when developing user interfaces http://wonderweb.semanticweb.org/
in model based approaches. Although there have been deliverables/documents/D18.pdf. Accessed
attempts for UI code generation from ontologies [8, 17], August 2nd, 2010.
the latter even claiming that ontologies should entirely
replace existing user interface description languages, we 10. Mozilla. XUL.
believe that a co-existence of both is more beneficial. https://developer.mozilla.org/en/XUL, 2010.
Accessed August 4th, 2010.
Acknowledgements 11. D. Oberle, S. Grimm, and S. Staab. An Ontology
The work presented in this paper has been partly funded for Software. In S. Staab and R. Studer, editors,
by the German Federal Ministry of Education and Re- Handbook on Ontologies, International Handbooks
search under grants no. 01IA08006 and 13N10711. on Information Systems, chapter 18, pages
383–402. Springer, 2nd edition edition, 2009.
REFERENCES
1. U. Aßmann, S. Zschaler, and G. Wagner. 12. F. Paternò, C. Santoro, and L. D. Spano. XML
Ontologies, Meta-models, and the Model-Driven Languages for User Interface Models - Deliverable
Paradigm. In Calero et al. [3], chapter 9, pages D2.1 of the ServFace Project. http://141.76.40.
249–273. 158/Servface/index.php?option=com_
docman&task=doc_download&gid=5&Itemid=61,
2. C. Atkinson, M. Gutheil, and K. Kiko. On the August 2008. Accessed August 9th, 2010.
Relationship of Ontologies and Models. In
S. Brockmans, J. Jung, and Y. Sure, editors, 13. H. Paulheim, R. Plendl, F. Probst, and D. Oberle.
WoMM, volume 96 of LNI, pages 47–60. GI, 2006. Mapping Pragmatic Class Models to Reference
Ontologies. In 2nd International Workshop on
3. C. Calero, F. Ruiz, and M. Piattini, editors. Data Engineering meets the Semantic Web
Ontologies for Software Engineering and Software (DESWeb), 2011. to appear.
Technology. Springer, 2006.
14. H. Paulheim and F. Probst. Ontology-Enhanced
4. J. Fengel and M. Rebstock. Linking Heterogeneous User Interfaces: A Survey. International Journal
Conceptual Models through a Unifying Modeling on Semantic Web and Information Systems,
Concepts Ontology. In N. Stojanovic and 6(2):36–59, 2010.
B. Norton, editors, Proceedings of the 5th
International Workshop on Semantic Business 15. RedWhale Software. The XIML Specification.
Process Management (SBPM 2010), volume 682 of Available as part of the XIML Starter Kit version
CEUR-WS, pages 1–4, 2010. 1, available at
http://www.ximl.org/download/step1.asp,
5. M. Fernández, A. Gómez-Pérez, and N. Juristo. 2000. Accessed August 3rd, 2010.
METHONTOLOGY: From Ontological Art
Towards Ontological Engineering. In Proceedings 16. F. Ruiz and J. R. Hilera. Using Ontologies in
of the AAAI97 Spring Symposium, pages 33–40, Software Engineering and Technology. In Calero
1997. et al. [3], chapter 2, pages 49–102.
6. T. R. Gruber. A translation approach to portable 17. K. A. Sergevich and G. V. Viktorovna. From an
ontology specifications. Knowledge Acquisition, Ontology-Oriented Approach Conception to User
5(2):199–220, Juni 1993. Interface Development. International Journal
”Information Theories and Applications”,
7. J. Guerrero-Garcia, J. M. Gonzalez-Calleros, 10(1):89–98, 2003.
J. Vanderdonckt, and J. Munoz-Arteaga. A
Theoretical Survey of User Interface Description 18. P. Spyns, R. Meersmanand, and M. Jarrar. Data
Languages: Preliminary Results. In LA-WEB ’09: modelling versus ontology engineering. SIGMOD
Proceedings of the 2009 Latin American Web Rec., 31(4):12–17, 2002.
Congress (la-web 2009), pages 36–43, Washington, 19. M. Uschold and M. Gruninger. Ontologies:
DC, USA, 2009. IEEE Computer Society. Principles, Methods and Applications. Knowledge
8. B. Liu, H. Chen, and W. He. Deriving User Engineering Review, 11:93–136, 1996.
Interface from Ontologies: A Model-Based
Approach. In ICTAI ’05: Proceedings of the 17th
IEEE International Conference on Tools with
Artificial Intelligence, pages 254–259, Washington,
DC, USA, 2005. IEEE Computer Society.
9. C. Masolo, S. Borgo, A. Gangemi, N. Guarino,
and A. Oltramari. WonderWeb Deliverable D18 –
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA