Mind the Gap - Requirements for the Combination of Content and Knowledge Tobias Bürger and Rupert Westenthaler Abstract— In this short paper we report on a semantic model there were metadata models for content like MPEG-7 [3], and for content and knowledge which distinguishes between three on the other hand there were domain ontologies developed by descriptive levels: information relating directly to the resource, the Semantic Web community. However, in the past few years to the meta data of the resource, and to the subject matter addressed by the content. This model addresses five fundamental much work has been done on the specification of ontologies requirements for automation: formality, interoperability, multiple that aim to combine traditional multimedia description models interpretations, contextualisation, and independence of knowl- [4], [1]. Related approaches include work from two different edge items from the resource’s content. research communities: First there are traditional content mod- Index Terms— knowledge content objects, intelligent content els like MPEG-7 or MPEG-21 [5] which are coming from the models, rich media content. multimedia community. Besides these traditional approaches some efforts in modeling of intelligent content objects exist: I. I NTRODUCTION More recent efforts include amongst others the ACEMEDIA3 ACE-objects [6] or the knowledge content objects (KCOs) of Semantics – i.e. the interpretation of the content – is the METOKIS project4 . important to make content machine-processable and to enable the definition of tasks in workflow-environments for knowl- III. R EQUIREMENTS FOR K NOWLEDGE C ONTENT edge workers in the content industries. Some of the recent research projects in the area of semantic (or symbolic) video Possible relations between knowledge and content are man- annotation try to derive the semantics from the videos’ low ifold. Based on observed applications and requirements of level features or from other available basic metadata. Most current projects (see [7] for details) we have derived the of these approaches are – as also pointed out in [1] – not following requirements for knowledge content: capable of fully exploiting the semantics of multimedia content 1) Knowledge must be encoded using a formal language because the meaning of the content is not localized just in the 2) Interoperability especially for cross domain aspects media that is being analysed. The construction of meaning is 3) Different interpretations of content objects – for humans – an act of interpretation that has much more 4) Link content with knowledge that cannot be directly to do with pre-existing knowledge and the context of the user derived from the content and/or the media than with recognition of the contents’ low- 5) Make knowledge independent of content level-features. This is known as the semantic gap [2]. Popular examples on the Web show that there are currently many IV. KCO – A M ODEL FOR K NOWLEDGE C ONTENT service-based platforms (like Flickr1 or LastFM2 ) that make use of their users’ knowledge to understand the meaning of KCOs are based on the DOLCE foundational ontology5 and multimedia content. have so-called semantic facets that form modular entities to We suggest that representing richer semantics for multime- describe the properties of KCOs, including the raw content dia content requires more expressive and more sophisticated object or media file, metadata and knowledge specific to the knowledge content models than those currently used. We content object and knowledge about the topics of the content therefore introduce a model for representing knowledge and (its meaning). content alongside each other, with clear separation of the Knowledge is represented by the structure of the KCO in content and the knowledge items, so as to obtain optimal con- three different levels: ditions for content and knowledge reuse, and for subsequent 1) Resource Level: This level refers to the actual content re-contextualization of content. object (File, stream, image, etc). 2) Meta Level: This level refers to knowledge describing II. R ELATED W ORK features of the content object, eg. frame rate, compres- sion type or colour coding scheme. For a long time, combining knowledge and content did not 3) Subject Matter Level: This level comprises knowledge play a great role in the research communities: On the one hand about the topic (subject) of the content as interpreted by Tobias Bürger and Rupert Westenthaler are with Salzburg Research an actor. The content object realizes this interpretation. Forschungsgesellschaft mbH, Salzburg, Austria. Contact: {tobias.buerger, rupert.westenthaler}@salzburgresearch.at 3 http://www.acemedia.org 1 http://www.flickr.com 4 http://metokis.salzburgresearch.at 2 http://www.last.fm 5 http://www.loa-cnr.it/DOLCE.html In addition to this knowledge structure the KCO also defines profiles containing additional knowledge about preferences of a structure based on the different domains of the knowledge the users. This information can be used to further contextualize objects. This structure is divided into six so-called facets, each queries by combining the context specified by the query, with of them optimized for a specific usage. Facets include for the characteristics of the user profile. Based on the active example a content- or community description facet [8]. concepts and relations of the contextualized query, the system Relating the above description to the requirements from can find similar interpretations. Such a system is sensitive to section III on knowledge content, we suggest that KCOs different interpretations of one and the same content object, provide a good foundation for modeling combinations of because it handles different interpretations of different users content and knowledge. separately. 1) Knowledge must be encoded using a formal lan- c) Context-based content classification to minimise the guage: KCOs are based on the information objects semantic gap: This scenario refers to the problem of how design pattern, which is an extension of the DOLCE to overcome the gap between low level features and higher foundational ontology. The main concepts and relations level semantics. It assumes that new content objects typically used in the description of the KCO are well grounded have to be analysed at the expense of some time and effort. on this foundational framework. The current definition The complextity of this operation can be reduced based on of KCOs is based on OWL-DL6 . background information about the content or some predefined 2) Interoperability especially for cross domain aspects: knowledge of parts of the content as knowledge about one part The facet based structure of the KCO serves as a good can help to understand other parts of the content. starting point for the alignment of standards to the KCO structure. Some of the facets and elements there use parts VI. C ONCLUSIONS AND F UTURE W ORK of different standards like NewsML7 or MPEG-7. More details about the reported work can be found in [7]. 3) Different interpretations of content objects: The pos- We are currently trying to apply KCOs in several national and sibility of different interpretations of content objects is international projects. Amongst them is the recently started the main reason for distinguishing between the mesta IST project LIVE8 , in which we are responsible for the defini- level and the subject matter level. Thus the KCO model tion of an intelligent media framework to support broadcasters support multiple interpretations of one content object. in the live staging of media events. 4) Clear definition of possible relations between content and knowledge: The KCO defines two different inter- VII. ACKNOWLEDGEMENTS relations between content and knowledge: First, knowl- edge objects can be about a content object, meaning The work reported here was part-funded by the EU projects that the subject matter of the knowledge object is the METOKIS (contract number IST-FP6-507164) and LIVE content object itself. Second, knowledge objects can be (contract number IST-FP6-27312). realized by content objects. This relation is used for all knowledge objects which are about the subject matter of R EFERENCES the content object. [1] Bloehdorn, S. et al.: Semantic Annotation of Images and Videos for 5) Make knowledge independent of content: This re- Multimedia Analysis In: Proceedings of the 2nd European Semantic Web Conference, ESWC 2005, Heraklion, Greece, May 2005. quirement is modeled by the fact that knowledge objects [2] Smeulders, A. W. M. et al.: Content-Based Image Retrieval at the End of which belong to the subject matter level are about an the Early Years In: IEEE Transactions on Pattern Analysis and Machine arbitrary topic and only realized by the actual content. Intelligence Vol. 22 No. 12, December 2000. [3] Martinez-Sanchez, J. M., Koenen, R., and Pereira, F.: Mpeg-7: The generic multimedia content description standard, part 1. IEEE Multi- V. B RIDGING THE G AP Media, 9(2): pp. 78-87, 2002. [4] Hunter, J.: Adding Multimedia to the Semantic Web - Building an In this section we will shortly sketch how the ideas of KCOs MPEG-7 Ontology, Proc. of the 1st International Semantic Web Working can be applied to address the various conceptual relationships Symposium (SWWS 2001), Stanford, USA, 2001. [5] Bormans, J. and Hill, K.: Mpeg-21 overview v.5. between content and knowledge in media-rich systems. http://www.chiariglione.org/mpeg/standards/mpeg-21/mpeg-21.htm, a) Search based on meta data: KCOs can be used to 2002. ISO/IEC JTC1/SC29/WG11. (last visited: 06.10.2006) model complex queries: In mental models that are representing [6] Kompatsiaris, I. et al.: Integrating knowledge, semantics and content for user-centred intelligent media services: the acemedia project. In: Proc. users intentions, subject matter level information about the of WIAMIS 2004, Lisboa, Portugal, 23 April 2004. topic of the content is often mixed up with meta level [7] Bürger, T. and Westenthaler, R.: Why the combination of content and information about the content objects: By posing queries to knowledge matters, Salzburg Research Technical Report, 2006. [8] Behrendt, W., Gangemi, A., Maass, W., and Westenthaler, R.: Towards the system, users create descriptions about topics or subjects an Ontology-Based Distributed Architecture for Paid Content In: A. that they are interested in. Gomez-Perez and J. Euzenat (Eds.) : ESWC 2005, LNCS 3532, pp. b) Collaborative filtering: A query in that setting spec- 257-271, 2005. ifies the actual context of the actor by considering some of the concepts and relations within the knowledge space of the actor, which are typically stored in the form of user 6 http://www.w3.org/TR/owl-guide/ 7 http://www.newsml.org/pages/index.php 8 http://www.ist-live.org