Integrating Multiple Contexts and Ontologies in a Pervasive Computing Framework 1 Adrian K. Clear and Stephen Knox and Juan Ye and Lorcan Coyle and Simon Dobson and Paddy Nixon 2 Abstract. There is a commonly accepted need for contexts and Contextual data can be viewed as being part of a spectrum where ontologies to describe the vast amounts of data that are available to data modelled by ontologies lie at one extreme, data modelled by pervasive computing applications. Existing contexts and ontologies contexts lie somewhere in the middle, and data without an explicit are either much generalised, very application specific, or inflexible. model lie at the other extreme. In an effort to clearly illustrate this An integrated approach is required in which new concepts can be spectrum, we propose the concept of a semantic sphere of pervasive added and related to existing ones transparently. This paper describes system data (see figure 1). In the semantic sphere we define a set of a novel approach to the design of a set of contexts and ontologies fundamental ontologies for pervasive systems. We call this set the for context-aware pervasive computing systems. It describes a Query core ontology. The core ontology describes the principle concepts in Service, that lies between applications and contextual information, a pervasive computing environment – who, where and when. More which complemented by the contexts and ontologies, offers a more precisely, these are: the entities that are in the environment (people, powerful query answering service to application developers than is sensors, etc.), the locations of interest, and the times of interest, re- currently available. spectively. All remaining data are viewed as being somewhat less general and are modelled using application contexts (for example, weather and music), or not modelled explicitly. Within the semantic 1 Introduction sphere, a class definition in a context or ontology can be viewed as a hook. By creating an instance of one of these classes, contextual Pervasive systems are interactive systems, whose behaviour must information is effectively hooked onto the context or ontology. Our adapt to the user’s changing tasks and environment using different contexts and ontologies are designed in such a way that semantically interface modalities and devices [8]. In order to be able to adapt to equivalent contextual data can be found regardless of their syntax, its environment, the pervasive system’s applications and the environ- and coarser levels of abstraction can be inferred from finer ones. mental sensors must have a common understanding of the contextual Consequently, the scope of an information search is broadened us- information. For this purpose, contexts and ontologies are vital. We ing simple relations. In this paper we also present the Query Service view an ontology as an explicit modelling of the fundamental con- (QS) that has been developed so that high-level application queries cepts of a domain that may be shared and reused. A context is an can be handled transparently, and results of the appropriate level of explicit model of the secondary concepts in a domain. It is more spe- abstraction and representation are returned to the application. cialised than an ontology but can still be shared and reused. Our design approach delivers contexts and ontologies that are To date, most ontologies for pervasive systems have been devel- well-defined and flexible. Sensor developers can hook contextual oped in a top-down manner in which the main focus is on application data onto, or extend, an existing context or ontology. They can be eas- semantics. This leads to ad-hoc models which are neither extensi- ily adapted to different applications and environments. Once hooked, ble nor support interoperability [12]. On the contrary, they should the contextual data is put into a distributed store, and applications can be flexible in their design to support a wider range of applications access it independently of the sensors. The novelty of this approach and environments. Moreover, the current approach to modelling con- is the organisation of the ontologies. This, along with a powerful QS, textual data is to give it a single representation in contexts and on- will be very useful for the building and supporting of a large number tologies. However, it is evident that sensors acquiring conceptually of context-aware applications. equivalent data provide different representations of such because of Our work is built within a framework called Construct, a fully- their nature; issues such as accuracy and heterogeneity necessitates distributed and decentralised context aggregation infrastructure for that the data provided by these sensors are represented differently. pervasive computing environments [15]. Construct consists of a num- At a common level of abstraction these representations are concep- ber of nodes that aggregate contextual data. Each node has its own tually equivalent. The need to incorporate such relationships into the data-store, and sensors register themselves with a node and inject design of contexts and ontologies should be recognised and is thus data into it. Construct nodes gossip [6] amongst themselves to main- the primary focus of this paper. Dealing with this issue at design time tain a global model of the system as a whole. All information is can be instrumental in the run-time inference of unknown facts from represented using RDF as the underlying data model. Applications known contextual data. therefore see a soup of contextual data derived from sensors and can 1 This work is partially sponsored by Science Foundation Ireland under grant access it through the QS. High-level queries are passed to the QS numbers 05/RFP/CMS0062 and 04/RP1/I544. Adrian Clear is funded by a using RDQL (RDF Data Query Language) [14]. The low level infer- joint IBM/IRCSET EMBARK scholarship. ring is handled transparently and application-interpretable results are 2 The authors are with the Systems Research Group, School of Computer returned. A model of this process can be seen in figure 1. Science and Informatics, UCD Dublin IE (email adrian.clear@ucd.ie) erences which “support the declarative specification of interest in a set of CTK components through a general query package”. Refer- ences process queries to discoverers and automatically subscribe to any components that match. Like our approach, the low level queries are handled transparently. Khedr et al [12] introduces context-level agreements into a mul- tiagent pervasive computing environment. They allow user agents to specify context that is relevant to them so that the context manage- ment agent can subscribe to the appropriate context providing agents in order to have the appropriate context delivered. All three systems support a high-level query language that decom- poses requests into satisfiable responses and then returns a response to an application’s request without the application needing to know the details of how the infrastructure is satisfying the response. How- ever, there is no effort to structure the semantics of the context data to provide a more powerful query service. Similar to the work from Chen et al on SOUPA (Standard Ontol- ogy for Ubiquitous and Pervasive Applications) [4], we use the Web Figure 1. Contexts and Ontologies within Construct Ontology Language (OWL) [3] to model our ontologies. The distinc- tion that we make between the application contexts and the ontolo- gies is closely based on the divide that exists in SOUPA between The rest of this paper is organised as follows: Section 2 briefly SOUPA Core and SOUPA Extension. Although the models that they illustrates the related work in the area along with the semantic web define are quite extensive, we take the approach of organising our technologies that our approach depends on; in Section 3 we introduce ontologies more effectively while keeping them simple. We also use the ontologies that are defined for the pervasive computing domain Jena [1] which is a semantic web framework for java. and describe some specific application contexts; Section 4 describes how the data represented in these ontologies and contexts are con- 3 Contexts and Ontologies for Pervasive verted into information suitable for consumption by pervasive ap- Computing plications. This process is demonstrated with an example location- based application in Section 5. Finally, in Section 6 we conclude the Numerous ad hoc ontologies have been created for pervasive com- paper and give some directions for future work. puting systems to date. They have been designed with the primary goal of providing a semantics for contextual data so that a common understanding can be given to data from heterogeneous sensors along 2 Related Work with entities in the pervasive environment. The goal of this work is This paper addresses the issues of context modelling and context ac- to not only develop such a semantics for contextual data, but also to cessibility in context-aware pervasive computing systems. The area develop our ontologies in a way in which they can be efficiently rea- has attracted attention recently and some seminal approaches that fo- soned about. The hypothesis is that different applications may require cus on the same issues have emerged: Firstly, the work carried out the same contextual data, but in different representations or levels of by Heer et al on the liquid extension to the Context Fabric [10, 11] abstraction. By adding a structure to our ontologies, using relations consists of a query service which supports distributed, continuous between their contents, this reasoning over data can be done at a query processing for context data. They introduce the notion of an lower level and will thus be transparent to the application developer. infospace which is a logical storage unit that may be centralised or The three core ontologies of where, when and who are described decentralised. Once context is sensed, it is added to the appropriate in this section along with their general properties and relationships. infospace. Context is stored in infospaces using tuples consisting of These ontologies form a base model that is general enough to be types and values. The value can be a basic value or another infos- used in a range of pervasive computing applications. An overview pace allowing queries to be structured as a concatenation of different of the application contexts along with a description of the relations types. Thus, to resolve a query involves the traversal of a string of used in the contexts and ontologies to achieve equality and levels of tuples. There are drawbacks to this, however. The user is required abstraction are also given. to know the structure of the infospaces and the types of their tuples in order to make a query. Also, there is no mention of a common 3.1 The Where Ontology semantics for types that tuples may contain making interoperability difficult. The where ontology describes the concept of location in a pervasive Another related concept is that of the Enactor extension to the computing environment. A location may be defined as a point (Co- Context Toolkit (CTK) [13, 9]. The CTK introduces Widget com- ordinate) or as a region (Space). Figure 2 shows the hierarchy of ponents which are structures that encapsulate a particular type of location types that are possible. Locations may be either physical, context acquiring sensor, for example, a location sensor. Each lo- e.g. a set of GPS coordinates; or symbolic, e.g. “RivadelGarda”. cation sensor will have the same interface, be they an internal RF Locations may be related to each other in ways that declare equiva- location system or GPS. This, however, allows only one level of ab- lence, e.g. RivadelGarda=GPS (45.88,10.82) and contain- straction per interface. The Enactor, which encapsulates some ap- ment, e.g. RoomA007 isContainedIn CS-Building. plication logic, obviates the application developer from having to Section 5 gives an example of how these mappings are used to subscribe to each widget manually. It consists of a number of Ref- transform location data from multiple contexts into a single result at specific to be modelled in an ontology. We have defined application contexts for data from a number of diverse applications that we are working on. These include weather data, flight data and music data. These contexts are stored in a catalogue of data models and are avail- able as hooks for application developers who wish to access the data of that type that are in the data store. Figure 2. The where Ontology 3.5 Transitivity and Equivalence Contextual data can be modelled using set-theory. Referring to our core ontologies and application contexts, two relations in particu- the correct level of abstraction in response to a query from an appli- lar are critical to their structure; transitivity and equivalence. Conse- cation. quently, there exists the notion of transitive and equivalence relations on elements of sets. 3.2 The When Ontology In mathematics, a binary relation R over a set X is transitive if it holds for all a, b and c in X, that if a is related to b and b is related to The when ontology defines the concept of time in pervasive com- c, then a is related to c. Transitive relations strengthen the reasoning puting environment. Figure 3 illustrates this hierarchy. Time may capabilities and are invaluable for certain ontology structures. For be declared as an instant (InstantTime) or as a range of time (e.g. example, the where ontology is naturally modelled using transitive TimeRegion). Time may be declared as being either symbolic, e.g. relations between different levels of abstraction of contextual data. Yesterday; or physical, e.g. 00:28, Friday 7th April Rooms are contained in floors which are contained in buildings, so 2006. Again, there is an equivalence relationship. TimeRegion ex- that a result for an application query for a high level of abstraction presses a period of time in a tuple of . We define three such as “What building...” can be inferred from lower level represen- relationships for time: equals; before; and after. tations of the same content. The equivalence relation is a little simpler. An equivalence rela- tion on a set X is a binary relation on X that is reflexive, symmetric, and transitive and it is used to group objects that are similar in some sense. Taking the where ontology as an example, symbolic names for locations are equivalent to their corresponding physical represen- tations. Furthermore, in the who ontology, the Identity instances of an Entity are equivalent representations of the Entity. To demonstrate the usefulness of these relations alone, a general Figure 3. The when Ontology query for a building name can be derived from hx, y, zi coordinates sensed by a tag-based location system by inferring the physical loca- tion that contains the coordinate (at a building level of abstraction) and finding the equivalent symbolic name. 3.3 The Who Ontology 4 Query Service The who ontology is different from the when and where ontologies. It The Query Service (QS) is a layer that sits between the application describes an agent that inhabits a pervasive computing environment, layer and the data-store. It provides an interface to the application e.g. a human user, intelligent agent or sensor. The who ontology has to make high-level queries on the store, and returns the results to only one hook: an Entity. Every Entity in the system will be attached the application in the correct level of abstraction. In order for sensor to this hook and will be uniquely identified. Each Entity must con- developers to take advantage of the QS functionality, they can make tain one or more Identity classes which are represented as attributes use of the existing ontologies and contexts so that their contextual with values. Any piece of contextual data that identifies an agent de- data can be represented with the appropriate semantics and relations clares itself to be representative of this token, e.g. in a tag-based lo- between levels of abstraction. The ontologies and contexts mentioned cation application the tag ID is mapped to an Identity attribute of in the previous section make such a tool possible. the corresponding user Entity. Instances of the who ontology can be The QS consists of three main components; the Query Handler, mapped to further, less general, contextual information such as a per- the Query Executer and the Query Service Reasoner: sonal profile. Thus, by traversing the equivalence relations between The Query Handler is the query interface that the QS provides Identity classes, any representation can gain access to contextual in- to the applications of the system. Any application can use the QS formation regarding the agent in question. by sending a high-level query to the Query Handler. When an ap- In Section 5 we demonstrate how three representations (Identities) plication makes a query, the Query Handler must first determine the of a user from three different sensors are mapped together allowing known and unknown facts of the query. The unknown facts are those an application to benefit from access to each of the contexts. that the application is requesting and the known facts are those that the unknown facts are requested in relation to. For example, take the query “What room is Bob in now?”. In this situation, the unknown 3.4 Application Contexts fact is the room and the known facts are Bob (the subject) and Tues- Besides these core ontologies, there exist many other types of data day, 11th April, 10:03am (the time that the query is made at). The that are reusable in a pervasive environment. However, they are too next step is to find the different representations of the known and unknown facts, and query for each representation of the unknown’s • Ubisense [2] sensors generate coordinate location data for each using the known ones as filters on the results. The Query Executer is tagged user with a peak level of abstraction of 30cm in 3D space. handed all of the derived queries and the results are returned to the • Bluetooth location sensors which can track location to approxi- QS Reasoner. mately ten metres. This provides a room-level abstraction to the The Query Executer The purpose of the Query Executer is to data-store. execute all of the low-level queries that are passed to it from the • Activity sensors determine whether an individual is located at a Query Handler. The Executer queries the data-store and passes the computer by checking whether they are logged in and active at the results on to the Query Service Reasoner so that it can then infer terminal. This sensor also provides a room-level abstraction. further information that the application requires. Virtual sensors may Each of these sensors insert data into the data-store which have be used to derive properties that are not explicit in the contexts and the properties: hasLocation, hasTime and hasIdentity. The Ubisense ontologies. For example, a hasLocation property can be derived from sensor generates raw data that looks as follows: (tagID=tag184, a higher level notion of a sighting. A sighting might introduce three time=30/03/2006 13:22:13, x=13.28, y=11.82, predicate triples to the data-store; one stating the person, one stating z=0.35). These data are hooked to the core who, when, and where the time, and another stating the location. ontologies as follows: tagID is hooked onto the Identity class of The Query Service Reasoner The basic results returned by the the who ontology; time is hooked onto the InstantTime class of Query Executer may not be of the level of abstraction required by the when ontology; and x=13.28, y=11.82, z=0.35 are collectively the application. The job of the Reasoner is to reason about the results hooked onto the Coordinate class of the where ontology. so that, if possible, they can be moulded into the representation re- When an application asks a question relating to a person’s loca- quired by the application. Currently, only bottom-up inferencing is tion, e.g. “What room is Bob in now?”, the Query Handler takes the supported as top-down inferencing would produce ambiguous or su- known and unknown terms and generates a suitable query in RDQL: perfluous results (e.g. a building reasoning about what is contained in it could return numerous rooms). Finer levels of abstraction can SELECT ? l o c a t i o n WHERE be generalized to coarser ones using the relations from Section 3.5. ? p e r s o n alsoKnownAs Bob From the query, the Reasoner knows the type, level of abstraction ? time after ( currentTime − 5) and representation that it must match. Using the ontologies and con- ? x hasTime ? t i m e texts as a reference, this match can be inferred from different repre- ?x hasIdentity ? person sentations and finer levels of abstraction. In the above example, the ?x hasLocation ? l o c a t i o n tag-based location system may return a coordinate which, using the The Query Executer executes this broad query. At least three where ontology as a reference, can infer that the coordinates are in results will be found (one for each active sensor). The data a particular room which, in turn, is in a building as these physical that came from the Ubisense sensor might come out in the fol- spaces are defined by a set of coordinates. lowing format: (Bob, 30/03/2006 13:22:13, (13.28, Currently, a simple custom-inferencer has been implemented to 11.82, 0.35). These results are passed to the Query Service reason over equivalence and transitive relations in order to seek out Reasoner. the required levels of abstraction. Referring to the where ontology, The required level of abstraction for the location data response is one of the transitive relations is isContainedIn. Equivalence relations at the room level. In this example, two different levels of abstraction can be defined over multiple types also. For example, a symbolic are returned; data at the room granularity (the data that originated name of a location might be equal to a physical representation of a at the activity sensor and Bluetooth sensor); and at the coordinate location. They are semantically equal but they are syntactically dif- level (from the Ubisense sensor). The former results are at the correct ferent. Consequently, a query returned for one representation can be level, and can be returned unaltered. However, the coordinate data converted to another to be of use to the application. Using these rela- must be raised from the coordinate level to the room level. tions, the Reasoner references the appropriate ontology to locate the Figure 4 illustrates a series of steps that the Query Service Rea- level of abstraction and representation that the application is looking soner goes through to process this inconsistent data in order to return for. If the values returned by the Query Handler are not syntacti- the correct level of abstraction to the application. The Query Ser- cally correct the Reasoner searches for an equivalence relationship vice Reasoner starts at the level of abstraction of the available data; between the syntactic form that is required and the form that is re- in this case coordinate data, and follows the transitive isContainedIn turned by the Query Handler. If one exists, the Reasoner then ab- relation, defined in the where ontology, to discover whether it is con- stracts the value to the correct level of abstraction using the transitiv- tained within a defined space. The equals relation is also used to ity relation. Once the level of abstraction is met, the representation move between physical and symbolic locations. These relationships can be mapped to the required syntactic form using the equivalence are followed until the resulting location maps to a level of abstraction relation. that matches the original query (or it fails if there is no valid mapping — i.e. the coordinate does not match a known room). In this example, the Query Service Reasoner returns the room called “RoomA007”. 5 Application This is done for all available data and the results are returned to the application that made the original query. To demonstrate the exchange of context data and ontology data in a pervasive system, we introduce a location-tracking application that 6 Conclusions and Future Work queries the data-store for the location of a user. It has a semantic map defining the locations in its realm, and a list of the users of the This paper describes a novel design approach to a set of core per- system. It is capable of making queries for the location of a user at the vasive computing ontologies describing the concepts of who, where level of abstraction of a room, floor or building. We use the following and when. These ontologies are used to ensure interoperability be- sensors to provide location data at different levels of abstraction. tween data from different application contexts. Data is accessed us- Figure 4. The process by which location data is abstracted to a higher level. ing a specialised query service that searches for and translates ap- data-store. In this way, the result to every satisfiable query is explicit propriate data to the required level of abstraction for the query. We in the data-store. This has its own problems, but would improve la- demonstrate this interoperability with an application that queries for tency, which is paramount in pervasive systems. Truth maintenance location data. This data has been collected from a variety of sensors is also a factor in pervasive systems, whereby information is inferred at different levels of abstraction. By using the tools in this paper the from lower level data. If this data is deleted or changes, it is important application developer does not need to be concerned with translating that this inference is still valid. this data. We describe the core ontologies. However, developers may ex- tend from the core by implementing their own contextual models REFERENCES and adding them to the semantic sphere. When creating new sensors, [1] Jena semantic web framework. http://jena.sourceforge.net/. the developer should use the preexisting contexts and ontologies but [2] Ubisense. http://www.ubisense.net/. [3] Sean Bechhofer, Frank van Harmelen, Jim Hendler, Ian Harrocks, Deb- this is not required. If they enter data without an explicit model, it is orah L. McGuinness, Peter F. Patel-Schneider, and Lynn Andrea Stein, available to application queries but only if they query directly against ‘Owl web ontology language guide’, (2004). the data. [4] Harry Chen, Filip Perich, Tim Finin, and Anupam Joshi, ‘SOUPA: When a query is made, multiple sensors may have sensed a con- Standard Ontology for Ubiquitous and Pervasive Applications’, in In- ternational Conference on Mobile and Ubiquitous Systems: Networking text which matches the query constraints. Each of these results will and Services, Boston, MA, (August 2004). be returned to the application level. It is up to the application devel- [5] Michael Collins, Simon Dobson, and Paddy Nixon, ‘Security issues oper to process this data. One characteristic of pervasive computing with pervasive computing frameworks’, in Workshop on Privacy, Trust environments is that sensors cannot be relied upon to always give and Identity Issues for Ambient Intelligence at Pervasive 2006, pp. 1–7, accurate readings. Work is being done to associate a quotient of ac- (2006). [6] Alan Demers, Dan Greene, Carl Hauser, Wes Irish, John Larson, Scott curacy with each piece of contextual data provided by a sensor [7]. Shenker, Howard Sturgis, Dan Swinehart, and Doug Terry, ‘Epidemic This will be available to applications and will improve the overall algorithms for replicated database maintenance’, SIGOPS Oper. Syst. accuracy of an application by allowing sensor data to be fused based Rev., 22(1), 8–32, (January 1988). on the individual accuracies of the available data. We will annotate [7] Simon Dobson, Lorcan Coyle, and Paddy Nixon, ‘Hybridising events and knowledge as a basis for building autonomic systems’, Journal of data with notions of trust [5] in the same way. Trusted and Autonomic Computing, (2006). To Appear. As part of our ongoing development, we intend to explore seman- [8] Simon Dobson and Paddy Nixon, ‘More principled design of pervasive tic translation (e.g. adjacent in the where ontology) with richer rela- computing systems.’, in EHCI/DS-VIS, eds., Rémi Bastide, Philippe A. tionships in basic structures. Such semantic translation will help to Palanque, and Jörg Roth, volume 3425 of Lecture Notes in Computer support more reasoning capabilities. We intend to further develop our Science, pp. 292–305. Springer, (2004). [9] A. K. Dey et al, ‘A conceptual framework and a toolkit for supporting location-tracking algorithm to query against other types of data. We the rapid prototyping of context-aware applications’, in Human Com- also intend to develop another application for making generic queries puter Interaction, pp. 97–166, (2001). for pieces of data in the data-store that will assist in self-diagnosis, [10] J. Heer, A. Newberger, C. Beckmann, and J. Hong, ‘liquid: Context- e.g. “tell me everything you know about x”, where x is a single piece aware distributed queries’, in Fifth International Conference on Ubiq- uitous Computing, pp. 140–148, (2003). of data. [11] Jason I. Hong and James A. Landay, ‘An infrastructure approach Additionally, as a consequence of Construct’s use of gossiping to to context-aware computing’, in Human-Computer Interaction, vol- spread information between its distributed nodes, latency is a con- ume 16, (2001). cern which must be more fully investigated. Some safeguards have [12] M. Khedr and A. Karmouch, ‘Negotiating context information in con- to be put in place so that the occurrence of redundant data is min- text aware systems’, IEEE Intelligent Systems magazine, 19(6), 21–29, (November/December 2004). imised. Due to our use of backward chaining in our virtual sensor [13] A. Newberger and A. Dey, ‘Designer Support for Context Monitoring for inferencing, every query is independently dealt with by the QS. and Control’, (2003). The disadvantages to this are latency and the computational cost of [14] Andy Seaborne. RDQL - a query language for RDF. W3C member the same query being inferred multiple times for different applica- submission, Hewlett Packard, January 2004. [15] Graeme Stevenson, Lorcan Coyle, Steve Neely, Simon Dobson, and tions. We will therefore be investigating the use of forward chaining, Paddy Nixon, ‘Construct — a decentralised context infrastructure for where all inferencing is done on all data when it is inserted into the ubiquitous computing environments’, in IT&T Annual Conference, Cork Institute of Technology, Ireland, (2005).