The Design of “Intelligence” for the Management of Personal Multimedia Content Patricia Charlton and Jonathan Teh Abstract— aceMedia has taken the challenge of organising II. ONTOLOGIES, METADATA AND SEMANTICS personal content by creating tools to automatically generate Knowledge-assisted applications can enhance a user metadata descriptors and search content intuitively. In this paper we review part of the approach taken by aceMedia to create experience because, for example, the user decides what they semantic metadata (ontologies) and use of this to enable more want to access and can then set up an automatic feature that appropriate search and matching and managing of content. the user controls. This is based on the same features that are However, there is further benefit the end user can gain from this available to the application developer, because it is about the semantic metadata and this is from adding intelligence in the same domain model; the only difference is in the access software. The benefits and how intelligence can be added is perspective. The application developer can make available to described with a particular focus on assisting the user in the creation of privacy preference rules when sharing content: the the user the same application configuration features that are creation of self-governing inferencing. available to the application itself because of the formal representation of the model. A granularity of control is made Index Terms— semantic metadata, ontologies, intelligent possible because the formal model explicitly states what is software, inferencing, self-governance. Topic “Integration of available. Further re-use is gained as the user profiles, models, multimedia processing and Semantic Web technologies (SS3)” and data can be easily shared across many applications III. ACEMEDIA PROJECT I. INTRODUCTION aceMedia uses the metadata and ontologies to provide T HE vision of the aceMedia project is to provide the tools to assist in advanced content management. This is to deal with the classic information overload, as users have not just context to assist in content management on behalf of the user. The metadata and ontologies are used to: 1. Enable the user to search for content: the user can access to content in many forms but also many tools and enter natural language sentences that are parsed to devices to create all types of content themselves. provide a structured query for the knowledge base. Management of content becomes increasingly difficult such as 2. Enable personalisation methods to create and manage finding the right content, creating collections, annotating personal metadata to be applied to the search and content etc. retrieval of appropriate content, in particular ranking aceMedia is researching methods to assist in information content, which is based on using machine learning and content management via knowledge technologies and techniques to weight preferences and content. developments in the semantic web. The aceMedia approach 3. Enable visual methods, which use intelligent involves creating and using metadata to enable intelligent multimedia algorithms, through the low level applications such as advanced search and retrieval, descriptors, to assist the user in matching similar personalisation, self-organisation of content, and autonomous visual properties of particular content with other content actions such as self-determined privacy. The use of content. metadata does not come without some key challenges itself. As well as the above functionality to assist the user in Many terms used within the metadata may refer to an implicit content management, aceMedia has researched into the informal semantics and do not necessarily provide essential requirement and application of the intelligent layer, as part of properties or relationships between terms to assist in any the Autonomous Content Entity (ACE). An ACE is a concept automated approach to be applied. However, the move which captures the content, metadata and the intelligence layer towards the development of ontologies that model domains, as a type of intelligent media object. There are two specific preferences, policies and profiles provide an approach to assist drivers that require the intelligence layer: in automating the matching and filtering of content searches. 1. Digital Content is very nomadic; in that context it is better to have content management attached to the Manuscript received June x, 2006. This work was supported in part by the content such that the user can always optimally deal European Commission under contract FP6-001765 aceMedia. with the content, wherever it resides P. Charlton and J. Teh are with Motorola Labs, Jays Close, Basingstoke, 2. Digital content can easily flow to other places where RG22 4PD, United Kingdom. (e-mail: patricia.charlton@motorola.com). the owner of the content does not have control over that have not yet been considered. This approach to self- this content; by carrying the rules of management with governing inferencing system is further enabled because of the the content then the owner’s rights and privacy structured semantic metadata used as grounding knowledge preferences are enabled. within the overall aceMedia system. The concept of the The Intelligence Layer is defined as code executing in an intelligence layer means that the rules can be executed when aceMedia system that provides “intelligence” to autonomously the content is being accessed, used or modified in some way. support content management. This for example means how to We capture the user’s privacy preference in a way that does present itself, maintain and enhance the metadata, handle not demand constant attention from the user but does capture privacy, self-adapt and self-organize etc. The intelligence the intended behaviour from the user by using a policy model layer can be transported together with the content and [2] to convert the user’s preferences into a rights context. This metadata as one object and the execution of the intelligence rights context plus a priority system converts this into rules layer is done in a secure environment. that is carried with the context. There are default policies and There are two specific applications which make use of the priority settings but these can be changed by the user and a intelligence layer: personal content ownership rights to particular context. support personal preferences about privacy of content through Some degree of self-governance of a computational system self-governance and self-organising content to assist in the is required if there is a need to support privacy preference- automation of content collections dependent on the devices based access to content when content leaves the control of the and environments. The self-governance context is built on owner. Here we express a degree of self-governance and the assisting the user with a means to declare their access self-governance of a system is required to be grounded in preferences to their content. This in terms of concepts is close some knowledge and facts about itself. Although theoretically to digital access rights, so access preferences available to the computational self-governance could be infinite, for example, user are captured in terms of digital rights access attributes. if the architecture incorporates a reflective tower principle [3] e.g. of self-governing of the self-governing rules etc. but this A. aceMedia framework only indicates to the designer the flexibility potential of such The aceMedia system facilitates digital multimedia content an architecture to deal with dynamics of a system itself. management through its software framework that enables the execution of application modules and ACEs. The aceMedia IV. CONCLUSION framework [1] enables ACEs to run, and reuse base content We are now in the first stages of including into the analysis functionality, shared by all running ACEs. The aceMedia framework computational intelligence. The use of framework further enables users to control and restrain the semantic metadata has provided some of the key cues for behaviour of ACEs. An application interface allows users to enabling context-aware computations that provide interact with individual ACEs and manage ACE collections. computational intelligence functionality. It has provided the B. Intelligence layer: Requirements and Design semantic structure and grounding knowledge that is required The intelligence layer provides a framework where code when performing intelligent behaviours. (rules, methods and inferencing techniques) can exploit the The use of autonomy was drawn from the autonomic semantic structure of the explicit knowledge about the content computing of defining a notion of “self”. This is supported by (multimedia content plus metadata descriptors, users, devices the aceMedia system: 1) the aceMedia framework enables a and domains). concept of “code” within an ACE itself, 2) the semantic The original concept of the ACE was about creating a metadata provide some context that can benefit and be mechanism that would assist in the users in managing their benefited by some form of computational intelligence, and 3) content and enable designers and developers methods to the inclusion of the self-governing inference rules. support this managing of content while creating flexibility for: In providing the self-governing inference rules we have a) tailoring solutions to the trends (ease of configuration) and added to the semantic metadata and have incorporated more b) extensibility for future unknown development. context cues through the use of policies that capture a user’s privacy preferences. The semantic metadata has meant we can C. Design of the self-governance define an inferencing system to build both the appropriate In the user studies done by the aceMedia project there were privacy preference context and rules that can be attached to indications that the users had concerns and wanted to have the content creating an intelligence layer as part of the content. some control over their content and have certain checks about how their personal content is shared and used [4]. The REFERENCES autonomous nature of content requires that the user’s intent is [1] A. Matellanes, A. May, P. Villegas, F. Snijder, A. Kobzhev, E. O. Dijk, carried with the content and this is captured and generated as “An architecture for multimedia content management”, EWIMT 2005. self-governing rules. The rules are generated dynamically [2] http://www.cs.umbc.edu/~lkagal1/rei [3] Smith B, “Reflection and Semantics in Lisp” ACM, pages 23-35, 1983. because it is impossible to define before hand all the possible [4] P. Charlton, J. Teh, A Self-governance Approach to Supporting Privacy rules that user would consider when applying to their content Preference-based Content Sharing, International Transactions on plus this gives the possibility to be extended to new contexts Systems Science and Applications, Special issue 2006.