Knowledge Space of Semantic Inference for Automatic Annotation and retrieval of Multimedia Content – K-Space Craig Stewart and Ebroul Izquierdo Horizontal integration is achieved by integrating, for a Abstract—K-Space is a network of leading research teams purpose, what can be disparate disciplines including image from academia and industry conducting integrative research and processing, information retrieval, human computer interaction dissemination activities in semantic inference for automatic and and industrial activities and inclusion of the wider community semi-automatic annotation and retrieval of multimedia content. K-Space exploits the complementary expertise of project is achieved through joint training schemes, joint workshops partners, enables resource optimization and fosters innovative and joint education tools and courses as well as access to research in the field. much of the resources such as platforms and data. Index Terms— Semantic Inference, digital media processing. To achieve these ambitious objectives K-Space brings together a critical mass of 14 leading research institutions I. INTRODUCTION from 9 different countries across Europe. Each partner brings unique expertise and skills which overlap or complement to K SPACE is a network of leading research teams from academia and industry conducting integrative research and dissemination activities in semantic inference for semi- each other. The main strength of the consortium is the unification of the partners to build a Network focusing on automatic annotation and retrieval of multimedia content. K- research areas that partners will benefit from dedicated and Space exploits the complementary expertise of project strong integration efforts. The consortium expertise represents partners, enables resource optimization and fosters innovative the know-how, commitment and competence that is absolutely research in the field. necessary to undertake the technical, dissemination and further integration aspects of the Network of Excellence The aim of K-Space research is to narrow the semantic gap (NoE). between low-level human content descriptions that can be computed automatically by an machine and the richness and The K-Space consortium consists of Queen Mary subjectivity of semantics in high-level human interpretations University of London (Coordinator, UK), Universitaet of audiovisual media: The Semantic Gap. Specifically, K- Koblenz-Landau (Germany); Joanneum Research Space integrative research focus on three areas: Content based Forschungsgesellschaft mbH ( Austria); Centre for Research multimedia analysis; Knowledge extraction and Semantic and Technology Hellas (Greece); Dublin City University multimedia. (Ireland); Centrum voor Wiskunde en Informatica (The Netherland); Groupe des Ecoles des Telecommunications An objective of the Network is to implement an open and (France); Institut National de l’Audiovisuel (France); Institut expandable framework for collaborative research based on a Eurecom (France); Univeristy of Glasgow (UK); German common reference system. K-Space research activities are Research center for Artificial Intelligence (Germany); designed to integrate basic research in semantic web Technische Universitat Berlin (Germany); Ecole technologies, multimedia processing, information retrieval, Polytechnique Federele de Lausanne (Switzerland); resource management in heterogeneous networks, semantic University of Prague (Czech Republic).. inference, user modelling and interaction with technological development by producing software platforms and open II. OVERVIEW OF JOINT RESEARCH ACTIVITIES libraries that will be available to the cross and multimedia The joint research activities of the network are aimed at scientific forum, thereby achieving vertical integration. convergence and resources optimization by exploiting important multidisciplinary aspects of multimedia knowledge Craig Stewart is with the Multimedia and Vision Research Group, extraction. This will be achieved by linking research efforts Electronic Engineering of Queen Mary University of London, e-mail: over the following three research clusters underpinning the K- craig.stewart@elec.qmul.ac.uk. Prof. Ebroul Izquierdo is with the Multimedia and Vision Research Group, Space framework. Electronic Engineering of Queen Mary University of London. e-mail: ebroul.izquierdo@elec.qmul.ac.uk A. Content – based multimedia analysis: data) in order to derive additional semantic Research into tools and methodologies for low- and medium- characterizations. level multimedia content analysis will be conducted. The • Provide a mechanism to handle such additional context target is to tackle the semantic gap in a bottom up approach, based characterizations, i.e. constructing, representing, i.e. adapting low-level techniques for knowledge based querying and using them. multimedia analysis. For the results of these analysis algorithms suitable descriptor schemes for MPEG – 7 or other III. INTERMAG CONFERENCE-RELATED PAPERS appropriate description standards will be developed. The The networking and distributed interaction between research activities in this area include: Content Structuring; partners modules is supported by suitable interfaces, database Moving 2D and 3D Object segmentation and Indexing; management and networking tools. The integration of these Audio/Speech Processing and Text Analysis; and Content tools builds the K-Space communication infrastructure. The Description. The outcome of this research activity will be project will put in place the necessary software and network tools and methodologies for low-level signal processing, interfaces required to ensure that every partner has access to object segmentation audio processing, text analysis, and the research resources of other partners. In order to achieve audiovisual content structuring and description. this, the project will seek to leverage the outputs from other B. Knowledge Extraction EU projects namely AIM@SHAPE and 3DTV. The output of this activity can be thought of as the physical instantiation of This research activity aims at studying techniques for the the K-Space. It will consist of distributed repositories of semantic analysis, annotation and retrieval of multimedia partners software, test data and results accessed via a central content The overall approach will be based on knowledge portal that supports remote use of these resources. The design, assisted content analysis and annotation using a multimedia instantiation and maintenance of the distributed environment ontology infrastructure. More specifically, semantic and low- will be the full-time responsibility of a K-Space system level attributes of the objects to be objects to be detected in administrator who will lead these activities supported by combination with appropriately defined rules. Semantic dedicated effort from all partners. Some of the technologies concepts within the context of specific domains will be that will be considered in this activity are XML-based defined in Ontologies, extended with qualitative attributes of technologies, such as those recommended by the Web the semantic objects (e.g. colour homogeneity), multimedia Services Activity of the W3C, Web services, JBoss etc. processing methods (e.g. colour clustering), and numerical data or low-level features (e.g. colour models, also defined in IV. K-SPACE DISSEMINATION ACTIVITIES the ontology). Semantic Web technologies will be used for knowledge representation including rules to describe how Dissemination will be achieved by the usual mechanisms of tools for multimedia analysis should be applied according to research papers but also by issuing training material and different object attributes and low-level features. Areas of tutorials on key topics. A thematic book will be published at study include: Specification of a Multimedia Ontology the end of the funding period. Several dissemination forums, Infrastructure; Knowledge assisted Multimedia Analysis; standardization bodies, international conferences and Multimedia reasoning and annotation; Context based exhibitions have been identified and are targeted as part of the Multimedia Mining; Intelligent User Relevance Feedback. dissemination activities of the network. The International Conference on Semantic and Digital Media Technologies C. Semantic Multimedia (SAMT) is the main yearly event of the Network. The main This research activity will focus on using and extending goal of this activity is to consolidate a key event in the area at semantic web technologies in order to respond to the needs of European level and beyond. One important objective of this multimedia analysis and knowledge extraction and thus, to activity is to broaden participation to the point where it facilitate the bridging the semantic gap. The rationale behind becomes a main conference with wide participation. The the approach is that a purely bottom up will not be sufficient vision for SAMT is to become “The” conference in the area to close the semantic gap, because a lot of knowledge around attracting most relevant technical papers and presentation the use and context of multimedia sources is needed around world-wide, which is one of the important long-term legacies the use and the context of multimedia sources is needed in of K-Space. order to bridge between low-level multimedia features and the pragmatics of its use. Research is structured along four tasks: ACKNOWLEDGMENT • Represent semantic metadata, user experience and needs The project overview given in this paper was taken from the to obtain bottom – up results and user context. program of work of the K-Space NoE. All members of the • Investigate how people interact with multimedia in order network contributed to the proposal, project definition and to derive new semantic characterizations, which may be program of work. The effort of all project partners and their exploited in the life cycle of the multimedia data. valuable contributions are acknowledged. K-Space is funded • Extract knowledge from complementary sources (such as by the EC commission under the IST 6th Framework subtitles of movies or web-sites of similar multimedia Programme.