Uncertainty Extensions to Ontologies as a Tool for Semantic Interpretation in Audiovisual Systems Pavel Smrž, Miroslav Vacura and Ondřej Šváb Abstract— This paper deals with semantic interpretation of instance [1]. We deal with a particular situation of the subway audiovisual data. It investigates how explicit interpretation of monitoring system in this paper and show how the selected uncertainty can help to bridge the semantic gap. We present a scenario can be modelled in the given framework. case study of FuzzyOWL — a current uncertainty representation framework — applied in the context of the recent European To demonstrate the main features of the formalism, we project CARETAKER focusing on automated situation aware- defined the following task connected to subway monitoring: ness, diagnosis and decision support. A particular user-oriented There are 4 cameras installed in a station – 2 in the corridor task is modelled in FuzzyOWL. The paper briefly summarizes (different directions), and one for each platform. There is a basic features of the formalism, discusses pros and cons and microphone array in the main corridor. The cameras report points out difficulties and problems connected to its employment. an unusual (for non-peak hours) crowd of people in the main corridor. One of the cameras there shows that most of people Index Terms— uncertainty representation, FuzzyOWL, audio- are not standing in a reading distance from the travel info visual system. sign. Although quite a long time elapsed from the departure of the last train, there is a fuss (strong noise) detected by the I. I NTRODUCTION microphones. The system should fire alarm for the operator. Uncertainty representation in audiovisual domain is one of the topics explored within a recent European project – II. F UZZY OWL FOR U NCERTAINTY M ODELLING IN the Network of Excellence K-Space (Knowledge Space of CARETAKER semantic inference for automatic annotation and retrieval of As the name suggests, Fuzzy OWL combines OWL with multimedia content). We participate in the preparation of a the fuzzy logic [7]. Two different approaches have been survey [2] that summarizes advanced features of the current proposed: the first one [5] permits only A-box fuzzy axioms representation and reasoning frameworks dealing with various in the form a : C  n and (a, b) : R  n, where  is one kinds of imperfect knowledge. The survey discusses pros and of {≤, <, ≥, >}, C is a concept, R is a role, and a, b are cons of particular formalisms and points out the differences. individuals. A new reasoning tool based on the first approach However, the systems are compared from a general point of is currently under development.1 view, we do not expose the systems to real conditions and do The second approach [6] permits T-box fuzzy concept not show their qualities in a real domain. Therefore, there is a inclusion axioms in the form α  n, where  is one danger of missing important aspects needed for their practical of {≤, <, ≥, >} and α is a non-fuzzy SHOIN concept application. inclusion axiom and R-box fuzzy role inclusion axiom in the That is why we decided to demonstrate one of the for- form α  n, where  is one of {≤, <, ≥, >} and α is a malisms – Fuzzy OWL – in real conditions and to show non-fuzzy SHOIN role inclusion axiom. its features in a case study from a real project. The se- As the non-fuzzy T-box and R-box of ontology can be lected testbed is provided by the current European project developed by standard techniques, we decided to use just the CARETAKER (Content Analysis and REtrieval Technologies first mentioned approach for demonstration purposes (although to Apply Knowledge Extraction to massive Recording) in the second approach is more expressive). Our use-case does which the first author participates. CARETAKER focuses on not need the representation of uncertainty on T-box or R-box the extraction of a structured knowledge from large multimedia levels. collections recorded over networks of camera and microphones To evaluate Fuzzy OWL representation of uncertainty, we deployed in real sites. The produced audio-visual streams, developed simple domain ontology in Protégé.2 It is designed in addition to surveillance and safety issues, could represent as a spatio-temporal ontology based on DOLCE [4]. The a useful source of information if stored and automatically spatial part consists of a system of space regions. The temporal analyzed, in urban planning and resource optimization, envi- part complies with the OWL-Time specification [3]. ronment planning and disabled/elderly person monitoring for The described ontology (see Figure 1) consists of a T-box terminology and a partial A-box containing information about P. Smrž (smrz@fit.vutbr.cz) is with the Faculty of Information “static” individuals like microphones or physical sectors of Technology, Brno University of Technology, Božetěchova 2, 612 66 Brno, Czech Republic subway stations. The second part of the A-box is defined M. Vacura (vacuram@vse.cz) and O. Šváb (svabo@vse.cz) are 1 http://www.image.ece.ntua.gr/ with the Faculty of Informatics and Statistics, University of Economics, ˜nsimou/ W. Churchill Sq. 4, 130 67 Prague 3, Czech Republic 2 http://protege.stanford.edu Fig. 1. A simplified OWL ontology for the use-case as a result of real world analysis of sensory data. Resulting background knowledge for the subway monitoring task. The features contain typically vague or uncertain information. scenario-based approach will allow us to create user-friendly Microphone can report noise in the corridor at level 0.8 interfaces, to enable end-users to introduce context information between No_noise and a chosen top level High_noise. about a new scene, add new scenarios adapted to a specific One can append this information in Fuzzy OWL form to the A- environment, and define the specification compliant with the box (as a fuzzy instance of the relation has_noise_level). given ontology. Similarly, we could include fuzzy A-box axiom of relation is_crowded_in_time between a given sector and the ACKNOWLEDGEMENTS current time instant which can be computed from the number This work was supported by the European Commission of people in particular physical sectors. under the 6th Framework Programme, projects CARETAKER (contract no.: FP6-027231) and K-Space (contract no.: FP6- III. D ISCUSSION AND F UTURE D IRECTIONS 027026). Various aspects of Fuzzy OWL have been taken into account in our work. We paid attention especially to the complexity R EFERENCES of reasoning, the way of dealing with imperfect knowledge, [1] C. Carincotte, R. Ravera, F. Bremond, J. Orwell, J.M. Odobez, B. Cor- availability of tools for reasoning, the support for ontology bucci, J. Palo, J. Cernocky, and X. Desurmont. Toward generic intelligent development, the complexity of the modelling from the user knowledge extraction from video and audio: The EU-funded CARE- point of view, and the possibility to present frequent patterns TAKER project. In Proceedings of ICTE, 2006. [2] K-Space Consortium. Modeling the uncertainty of semantic descriptors in an intuitive form. of multimedia. (under preparation), 2006. In summary, reasoning in Fuzzy OWL is decidable. The [3] Jerry R. Hobbs and Feng Pan. An Ontology of Time for the Semantic formalism handles imperfect knowledge in both – ABox and Web. In ACM Transactions on Asian Language Processing (TALIP): Special issue on Temporal Information Processing, number 1, pages 66– TBox. There is a prototype of a reasoning tool for Fuzzy 85. March 2004. OWL available for download. It does not use the standard [4] C. Masolo, S. Borgo, A. Gangemi, N. Guarino, A. Oltramari, and XML-based OWL syntax. Ontologies (T-boxes) can be de- L. Schneider. WonderWeb Deliverable D17. The WonderWeb, Library of Foundational Ontologies and the DOLCE ontology. Technical report, veloped using standard tools (e. g., Protégé) if one decides 2002. to include fuzzy information on the A-box level only. A [5] Giorgos Stoilos, Giorgos Stamou, Vassilis Tzouvaras, Jeff Z. Pan, and Ian general conclusion of the work can be that the fuzzy approach Horrocks. Fuzzy OWL: Uncertainty and the Semantic Web. In Proc. of the International workshop on OWL: Experience and Directions (OWL- is appropriate for the textual specification of the vague and ED2005), 2005. possibly ambiguous description of the use-cases given in the [6] U. Straccia. A fuzzy description logic for the semantic web, 2005. natural language. On the other hand, it is rather tricky to model [7] L. A. Zadeh. Fuzzy sets. Information and Control, 8(3):338–353, 1965. the dependencies expressed by conditional probabilities. Our future work will focus on the development of an intuitive user environment enabling application-specific vi- sualisation of the domain knowledge. Future directions of our research will also lead to a detailed specification of the