A Formal Ontology on User Interfaces – Yet Another User Interface Description Language? Position Paper Heiko Paulheim and Florian Probst SAP Research Bleichstrasse 8 64283 Darmstadt, Germany {heiko.paulheim,f.probst}@sap.com ABSTRACT ontologies are different by nature. An ontology claims During the past years, a lot of user interface descrip- to be a generic, commonly agreed upon specification of tion languages, most of them based on XML, have been a conceptualization of a domain [6], with a focus on pre- introduced. At the same time, the use of formal ontolo- cisely capturing and formalizing the semantics of terms gies for describing user interfaces has been discussed used in a domain. A software model in turn is task- for a number of use cases. This paper discusses the specific, with the focus on an efficient implementation differences between a formal ontologies and user inter- of an application for solving tasks in the modeled do- face description languages and and points out how both main [2, 16, 18]. Thus, a software engineer would rather research directions can benefit from each other. trade off precision for a simple, efficient model, with the possibility of code generation, while an ontology engi- Author Keywords neer would trade off simplicity for a precise representa- User Interfaces, Ontology, UI Description Languages, tion. Another difference is that in software engineering, Formal Models models are most often prescriptive models, which are used to specify how a system is supposed to behave, while ontologies are rather descriptive models, which ACM Classification Keywords describe how the world is [1]. Figure 1 illustrates those D.2.2 Software Engineering: Design Tools and Tech- differences. niques—User Interfaces; D.2.11 Software Engineering: Software Architectures—Languages; I.2.4 Artificial In- Taking this thought to the domain of user interfaces telligence: Knowledge Representation Formalisms and and interactions, models are used to define particular Methods—Semantic Networks user interfaces (e.g. with the goal of generating code implementing those interfaces), while a formal ontology General Terms would capture the nature of things that exist in the Design, Languages domain, e.g., which types of user interfaces exist, and how they are related. INTRODUCTION Recently, a number of use cases have been proposed that Due to those differences, we argue that developing a employ ontologies for modeling user interfaces, their formal ontology on user interfaces will not lead to yet components and interaction capabilities. Examples are another user interface description language, but to a automatic generation of explanations for user interfaces, formal model with different intentions and usages. In adaptation of user interfaces for different needs and con- the next sections, we will discuss how the two worlds texts, and integration of user interface components [14]. can benefit from each other. Those use cases require a strongly formalized ontology of the domain of user interfaces and interactions. HOW A FORMAL ONTOLOGY CAN BENEFIT In parallel, various UI description languages have been FROM UI DESCRIPTION LANGUAGES proposed, most of them XML based [7, 12]. The duality A lot of research work has gone into the development of UI description languages and formal ontologies gives of different user interface description languages. Those rise to the question whether an additional ontology is research efforts can be and should be taken into account really needed, or whether it is going to be yet another when developing an ontology of the domain. user interface description language. Collection of Concepts ONTOLOGIES AND MODELS Most methodologies for ontology engineering foresee the Although ontologies and software models are related, capturing of key concepts and relationships as one of the they are not essentially the same. Software models and first steps. This can be done by conducting interviews Copyright is held by the author/owner(s) SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA Different goals lead to different models Goal: „complete picture“, Goal: efficient semantic account of terms programming in a domain - tas - pr k-spec a ch e if ppro - sim scriptiv ic app n e ric a e tion p e roac - ge criptiv resenta repr licity o h e s e p Ontology Software Model e se v d r ntat er prec shared - ci se icity ion i se - pre r simpl conceptualization o ve of a domain develops develops software developer ontology engineer Figure 1. Ontologies and modeling languages serve different purposes. with domain experts, scanning books and other mate- the development and usage of new and existing user rial, and/or reusing parts of other ontologies [5, 19]. At interface description languages as well. this point of ontology engineering, lots of input can be used from existing user interface description languages. Disambiguation of Terms In an analysis of user interface description languages, Since those languages are most often XML-based, they we have found that terms are often used differently in consist of a smaller or larger number of tags and at- different standards. An example is the term dialog. In tributes, which determine the expressivity of the lan- XIML, for example, a dialog element is defined as be- guage. As many of those elements define certain con- ing “like a command that can be executed [...] It is the cepts of the domain, such as UI components or actions more concrete instantiation of a task.” [15]. In contrast, that can be performed with them, they are a good start- XUL defines a dialog as an “element [which] should be ing point for developing a formal ontology of the do- used in place of the window element for dialog boxes” main. [10]. Such ambiguities can easily lead to misinterpre- tations, especially if users are trained on a particular Benchmarking the Ontology’s Completeness language and switch to another one. As discussed above, ontology engineering aims at pro- viding a complete, comprehensive formal description of Mapping a user interface description language to a for- a domain. However, assessing the completeness of an mal ontology capturing the semantics of those terms ontology is not always an easy task. Here, user interface can avoid such misinterpretations. With the exam- description languages can once again help by providing ple term dialog, a formal ontology can help resolving a benchmark for the ontology’s completeness. the ambiguity by indicating that the languages imply different top-level categories such as Process, Plan, Such a benchmark can be performed in different ways. or Software Component as super-category for Dia- On the meta-model level, the number of concepts con- log. tained in the meta model (e.g., tags and attributes in an XML schema) which have a counterpart in the on- Facilitating Extensibility of User Interface Description tology can be determined. On the model level, one can check whether given models in a user interface descrip- Languages tion language can be expressed using only the terms XML based languages usually use a fixed set of tags. given in an ontology, either informally, or formally, e.g., In order not to be too strictly limited for practical use, in RDF. Thus, user interface description languages can many of those languages provide some extension mech- provide a measure for the completeness of an ontology anisms such as universal general purpose tags that can of the domain. be used for user-defined concepts (e.g. the ELEMENT tag in XIML). These extension slots are then filled with ar- bitrary strings. HOW UI DESCRIPTION LANGUAGES CAN BENEFIT FROM A FORMAL ONTOLOGY Arbitrary strings, however, are dangerous. They lead Once an ontology of the domain of user interfaces and to extensions that are incompatible with each other, interactions has been created, it can be used to improve interpreted differently by different people and systems Copyright is held by the author/owner(s) SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA ontology engineer To end up with a comprehensive ontology, we have an- alyzed several user interface description languages in order to collect a maximum set of relevant terms. We n develops ctio s have used UsiXML, XIML, UIML, Maria, XUL, LZX, olle es p t c k for eten WAI ARIA, and XForms as a basis for identifying the n ce a r m p l o m - c e n ch y co - b tolo g core concepts. on rm s In order to build upon well-acknowledged roots, we have f te Ontology Modeling no chosen the top level ontology DOLCE [9] and its exten- a tio igu Language i sa mb ibility and n sions as a basis for our ontology. This top level ontology - d xtens riso - e ompa rsion provides an embracing basic classification of things and c - onv e develops c has been used as a basis for building numerous ontolo- gies. Since the top level provides a complete classifica- modeling language designer and user tion, it ensures extensibility of the ontology by design, as every new concept can be classified in some existing Figure 2. How user interface description languages and category. Furthermore, we have reused two core ontolo- ontologies can benefit from each other gies of software and software components [11], which are also built upon the foundations of DOLCE. relying on different conventions and external documen- The ontology we have developed is divided into two tations, and, in the end, foil the overall idea of having parts: a top level which captures the semantics of the a standardized modeling language. basic terms of the domain, such as User Interface Com- ponent and Interaction, while the detail level classifies A formal ontology can help here by providing a stan- the actual things that exist in the domain, such as types dardized vocabulary which can be used to fill such ex- of user interface components and user tasks that can be tension slots. Thus, it can be assured that there is an performed with those components. The OWL version of unambiguous interpretation of the extensions. the top level ontology consists of 15 classes, two object properties, and 75 axioms, while the detail level consists Model Comparison and Conversion of 179 classes, eleven object properties, and 448 axioms. When bringing together different development teams, information systems, or organizations, it is likely that models created with different user interface description CONCLUDING REMARKS languages already exist. Using a mediating ontology for This position paper has discussed the differences be- annotating the models is a common way of establishing tween UI description languages and a formal ontology comparability between models, not only user interface of the domain of user interfaces and interactions. Fur- models [4]. thermore, We have given insight into the development of a comprehensive formal ontology of the user inter- Once models are annotated and can be compared using faces and interactions domain. In the long run, we are a common ontology, automatic conversion of models can confident that formal ontologies and UI definition lan- be long-term objective. For the moment, a common on- guages will both have their places, and that both will tology can at least support developers in understanding benefit from each other. each other’s models and assist them in unambiguously transferring their contents between modeling languages We have presented a number of potential improvements manually. where developers employing user interface description languages could benefit from those languages being map- Fig. 2 summarizes how modeling languages and a formal ped to a formal ontology of user interfaces and interac- ontology can benefit from each other. tions. Thus, our claim is that organizations providing user interface description languages could improve the usability and acceptance of those languages by provid- TOWARDS A FORMAL ONTOLOGY OF THE DOMAIN ing such a mapping. OF USER INTERFACES AND INTERACTIONS With these considerations in mind, we have started to As a long-term objective, such a mapping could even fa- develop a formal ontology of the domain of user inter- cilitate automatic conversion between models developed faces and interactions. The goal is to end up with an with different user interface description languages. To ontology that is comprehensive at least with respect to that end, more sophisticated mapping approaches than the expressivity of current user interface definition lan- simply relating elements form a modeling language to a guages, that is universal enough to be extendable to category in an ontology are needed [13]. future user interfaces that do not exist at the moment. Furthermore, to support valuable reasoning on user in- A formal ontology will not replace user interface de- terfaces and provide meaningful semantics, the ontology scription languages, but be a valuable enhancement. should be highly axiomatized. Due to the conceptual differences between software mod- Copyright is held by the author/owner(s) SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA Description of Description of Interactions Components dns:Situtation dolce:Particular / owl:Thing io:Information Object dolce:Region plan:task-precondition plan:task-postcondition cso: data type cso:identifies cso:Data plans:component dns:Task cso:Abstract Data cso:Software (Representational Data) (Executable Data) cso:Computational Task User Task cso:identifies cso:User Data cosc:Software Component Detail Level: Display, Highlight, Detail Level: Delete, ... Select, Organize, ... dolce:part User Interface Component Processing Component Storage Component dns:Plan dns:defines dns:expresses dns:realizes Detail Level: Label, Image, dolce:part Interaction Plan Button, Text field, ... Design Time Run Time dolce:Non Agentive dolce:event dns:Agent dolce:Non-Physical Object Physical Object dns:sequences makes use of ≡ fp:use of dolce:accomplishment dolce:specifically cso:Computational Object Hardware dolce:part constantly depends on fp:performs dns:action Peripherical Hardware Non-peripherical Hardware tr:causally follows involves as tool dns:activity ... ≡ fp:instrument Visual Computational Tangible Hardware Object Monitor Object cso:Computational Activity User Activity dolce:has quality dolce:has quality dolce:Physical Quality dolce:Region provides ... SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA Detail Level: Click, Type, Style dolce:spatial-location-quality Screen Region adjacent to Scroll... Copyright is held by the author/owner(s) ... ... Position on Screen dolce:q-location Figure 3. The top level of the ontology of the user interfaces and interactions domain. In the upper part, the design time concepts are shown, the lower part contains the run time concepts. The left part deals with interactions, the right part with components. The white ellipses denote concepts from the reused ontologies (with the following namespace conventions: DOLCE (dolce), Descriptions and Situations (dns), Plans (plans), Functional Participation (fp), Temporal relations (tr), Core Ontology of Software (cos), Core Ontology of Software Components (cosc)), the grey ellipses denote concepts from the top level ontology of the user interfaces and interactions domain. The grey triangles denote definitions carried out in the detail ontology. els and ontologies, user interface description languages Ontology Library (final), 2003. do a better job, e.g., when developing user interfaces http://wonderweb.semanticweb.org/ in model based approaches. Although there have been deliverables/documents/D18.pdf. Accessed attempts for UI code generation from ontologies [8, 17], August 2nd, 2010. the latter even claiming that ontologies should entirely replace existing user interface description languages, we 10. Mozilla. XUL. believe that a co-existence of both is more beneficial. https://developer.mozilla.org/en/XUL, 2010. Accessed August 4th, 2010. Acknowledgements 11. D. Oberle, S. Grimm, and S. Staab. An Ontology The work presented in this paper has been partly funded for Software. In S. Staab and R. Studer, editors, by the German Federal Ministry of Education and Re- Handbook on Ontologies, International Handbooks search under grants no. 01IA08006 and 13N10711. on Information Systems, chapter 18, pages 383–402. Springer, 2nd edition edition, 2009. REFERENCES 1. U. Aßmann, S. Zschaler, and G. Wagner. 12. F. Paternò, C. Santoro, and L. D. Spano. XML Ontologies, Meta-models, and the Model-Driven Languages for User Interface Models - Deliverable Paradigm. In Calero et al. [3], chapter 9, pages D2.1 of the ServFace Project. http://141.76.40. 249–273. 158/Servface/index.php?option=com_ docman&task=doc_download&gid=5&Itemid=61, 2. C. Atkinson, M. Gutheil, and K. Kiko. On the August 2008. Accessed August 9th, 2010. Relationship of Ontologies and Models. In S. Brockmans, J. Jung, and Y. Sure, editors, 13. H. Paulheim, R. Plendl, F. Probst, and D. Oberle. WoMM, volume 96 of LNI, pages 47–60. GI, 2006. Mapping Pragmatic Class Models to Reference Ontologies. In 2nd International Workshop on 3. C. Calero, F. Ruiz, and M. Piattini, editors. Data Engineering meets the Semantic Web Ontologies for Software Engineering and Software (DESWeb), 2011. to appear. Technology. Springer, 2006. 14. H. Paulheim and F. Probst. Ontology-Enhanced 4. J. Fengel and M. Rebstock. Linking Heterogeneous User Interfaces: A Survey. International Journal Conceptual Models through a Unifying Modeling on Semantic Web and Information Systems, Concepts Ontology. In N. Stojanovic and 6(2):36–59, 2010. B. Norton, editors, Proceedings of the 5th International Workshop on Semantic Business 15. RedWhale Software. The XIML Specification. Process Management (SBPM 2010), volume 682 of Available as part of the XIML Starter Kit version CEUR-WS, pages 1–4, 2010. 1, available at http://www.ximl.org/download/step1.asp, 5. M. Fernández, A. Gómez-Pérez, and N. Juristo. 2000. Accessed August 3rd, 2010. METHONTOLOGY: From Ontological Art Towards Ontological Engineering. In Proceedings 16. F. Ruiz and J. R. Hilera. Using Ontologies in of the AAAI97 Spring Symposium, pages 33–40, Software Engineering and Technology. In Calero 1997. et al. [3], chapter 2, pages 49–102. 6. T. R. Gruber. A translation approach to portable 17. K. A. Sergevich and G. V. Viktorovna. From an ontology specifications. Knowledge Acquisition, Ontology-Oriented Approach Conception to User 5(2):199–220, Juni 1993. Interface Development. International Journal ”Information Theories and Applications”, 7. J. Guerrero-Garcia, J. M. Gonzalez-Calleros, 10(1):89–98, 2003. J. Vanderdonckt, and J. Munoz-Arteaga. A Theoretical Survey of User Interface Description 18. P. Spyns, R. Meersmanand, and M. Jarrar. Data Languages: Preliminary Results. In LA-WEB ’09: modelling versus ontology engineering. SIGMOD Proceedings of the 2009 Latin American Web Rec., 31(4):12–17, 2002. Congress (la-web 2009), pages 36–43, Washington, 19. M. Uschold and M. Gruninger. Ontologies: DC, USA, 2009. IEEE Computer Society. Principles, Methods and Applications. Knowledge 8. B. Liu, H. Chen, and W. He. Deriving User Engineering Review, 11:93–136, 1996. Interface from Ontologies: A Model-Based Approach. In ICTAI ’05: Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence, pages 254–259, Washington, DC, USA, 2005. IEEE Computer Society. 9. C. Masolo, S. Borgo, A. Gangemi, N. Guarino, and A. Oltramari. WonderWeb Deliverable D18 – Copyright is held by the author/owner(s) SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA