- The Integrated Project SALERO aims to advance the state of the art in digital media to the point where it becomes possible to create audiovisual content for cross-platform delivery using intelligent content tools, with greater quality at lower cost, to provide audiences with more engaging entertainment and information at home or on the move. SALERO will build on and extend research in media technologies, web semantics and context based image retrieval, to reverse the trend toward everincreasing cost of creating media.
Index Terms— audiovisual intelligent objects, content creation, context aware behaviour
SALERO’s [
Complete realisation of SALERO’s vision is a long-term goal. This gives rise to three overarching R&D objectives: Address characters, objects, sounds, language sets and behaviours, Research into methodologies for creating and finding intelligent content, Develop toolsets to create, manage, edit, retrieve and deliver content objects.
The first goal is to obtain a better understanding of the relations between media types, genres, workflows and styles as a pre-requisite to the adaptation and transfer of content elements across productions and platforms. To this end, metadata, media semantics and ontologies need to be analysed, researched and developed that define the parameters necessary for the creation and manipulation of semantically aware media objects of various types. Practical methods of context-based information retrieval will be researched that simplify the location and retrieval of characters, sounds, images, movements or behaviours from very large datasets and media storage systems. Improved methods and tools for language processing and speech synthesis, as a means of supporting the generation of multilingual media content, need to be developed.
The objective of this research strand is twofold: the main objective is to devise a machine process able description for the semantic features of a multimedia object and the context it should be used in. This will be tackled by building up a set of ontologies taking into account a layered approach – using an appropriate representation technique for every level of metainformation – that relies as much as possible on current description standards for multimedia. The second objective is to design and implement necessary tools and applications to build up, maintain and query ontologies for multimedia objects.
Media objects have different specification needs on
different platforms from game consoles and online services to
DVD, television and cinema, related to the overall expressive
and stylistic objectives of the production. Audience
expectations are related to the production genre, be it a
western, soap opera, comedy, tragedy, thriller,
actionadventurer, a medieval sword & magic MMPORG (Massively
Multi-Player Online Role-Playing Game). The genre
expectations (whether the engagers are passive watchers of
Jerry Seinfeld on television, or active console game players
represented on the screen by Lara Croft) are elegantly
expressed by the active questions of Philip Parker [
We expect that the intelligent content elements developed by SALERO will adapt themselves to the context of the production. We will therefore need to research ways of defining, creating (or locating), managing and delivering content objects of different kinds in a range of contexts. We address the context-based retrieval of media objects within a media production environment. The idea is to re-use objects, motion data and other production related data for the creation of new production materials. It is not only about finding and re-using elements at production time, but also using retrieval technologies for the creation of interactive media productions. That means for example that a character may react or adapt to a scene in a way that is based on the users input. A number of factors affect the use of retrieval techniques within media production environments. The most important one is that they should be integrated into the production environment: retrieval should happen as part of the programme development and not as a cumbersome, extra activity. We also need intelligent context sensitive retrieval mechanisms that identify both user context and task context.
The aim of this activity is to enable programmes created in
one language to be re-purposed and/or synthesised in another
language or dialect by researching and developing a ‘speech
corpus/concordancer’. A speech corpus[
This requires a framework for defining voice stereotypes for age, genre, emotional dimension etc and a suitable tagging system for corpus transcripts-initially for Catalan, Spanish and English. In the case of English, tagging for speech rhythms and other acoustic features within the recorded speech clips is seen to play a crucial role in developing a more natural corpus. The tagging of the resultant speech corpus will be applied to rule-based analysis, synthesis, lip-synching and character animation.
The research in this activity deals with both visual objects (such as characters) and audio objects (such as effects, speech). It provides grounding work for the linking of visual, audio, and behavioural objects, whose initial intelligence is expected to be increased along the lifetime of the project. It develops along different levels, from the low level provision of basic affordable rendering engines for media; through intermediate level, such as the modelling and animation of characters; to high level aspects, e.g. programme generators.
Software toolkits, software systems, plug-ins and interfaces will be developed that allow the control of appearances, sounds, semantic behaviour and properties of intelligent content objects for media production and post-production, and can be used in conjunction with existing industry programs. They will be validated and evaluated through a series of experimental productions, based on scenarios defined by artists and creative media professionals.
Results will be promoted by a broad initiative, developing demonstration test beds and training structures for professionals and researchers, as well as by addressing the relevant standardisation bodies.
A number of research groups are dealing with ontology
based description of multimedia items often by applying
reasoning to low level features extracted, e.g. [
The SMaRT networking cluster [