Combining traditional personalization techniques with novel knowledge representation paradigms, such as the ontologybased approach proposed in the Semantic Web field, is a challenging task. Personalization is a difficult problem when dealing with multimedia content and information retrieval, where context is increasingly acknowledged to be a key notion in order to make proper sense of user needs. This work focuses on contextualization within personalization in a multimedia environment. Towards that scope, we propose a novel contextual knowledge modeling scheme, and an approach for the dynamic, contextual activation of semantic user preferences to better represent user interests in coherence with ongoing user activities, e.g. in an interactive retrieval process. The application of this methodology is demonstrated using two user scenarios, and the performance results of a preliminary experiment are shown.
Over the last decades, the task of personalization is related to various
scientific and applied fields, with applications of techniques ranging
from artificial intelligence and pattern recognition to traditional or
multimedia databases and information retrieval applications [
In order to provide effective personalization techniques and develop intelligent personalization algorithms, it is appropriate not only to consider each user’s queries/searches in an isolated manner, but also to take into account the surrounding contextual information available from prior sets of user actions. As an example, consider having some irregularities occurring in random places within a user’s preferences, due to spontaneous changes of user’s attention and focus. Taking into account further contextual information, the system can provide an undisturbed, clear view of the actual user’s preferences, cleaned from extraordinary - according to each user’s profiling information - anomalies, distractions or ”noise” preferences. We refer to this surrounding information as contextual knowledge or just context.
Since several forms of context exist in the area [
The rest of the paper is organized as follows: in section 2, we present the main components of the underlying knowledge infrastructure, introducing the notion of fuzzified context, as well as the use of fuzzy relations within ontologies. Section 3 deals with the problems of runtime context determination and context usage in order to influence activation of user preferences, ”contextualize” them and predict or take into account the drift of preferences over time. As will be described a runtime context is represented as a set of weighted concepts from the domain ontology. How this set is determined, updated, and interpreted, will also be explained. In section 4 we provide early experimental results in the form of two user case-study examples and some conclusions are drawn in section 5. 2
Knowledge representation is one of the central and in some ways
most fundamental notions in fields like information retrieval.
Different views have been proposed and studied, and attempts have been
made at determining what representation properties are important
for knowledge representation in multimedia applications. However,
most proposed solutions are not sufficient due to performance
reasons, as well as due to the lack of accompanying contextual
information. The latter forms a major limitation and it lies within the
intensions of current work to manipulate and improve this kind of
information in an efficient manner. Unquestionably, design and analysis of
such a task is not straight-forward and many approaches are
acceptable. The term context can take many interpretations and definitions
when dealing with specific application-domains [
An ontology is a formal specification of a shared understanding of
a domain [
Although in general any type of relations may be taken into
consideration, in order to extract and use the desired ontological context,
we define it in the means of fuzzy ontological relations. Fuzziness
is an intrinsic property of knowledge representation, since accurate
representation of real-life information is only achieved through the
use of fuzzy relations. In [
The presented relations are based on the set of semantic
relations defined by the MPEG-7 standard [
Finally, when dealing with implementation issues of the proposed
context knowledge representation, we propose a specific way of
representing context, following a standardized language like OWL or
RDF. We smoothly integrate context’s functionalities in the
ontology infrastructure, i.e. we adopt enhanced characteristics available
in the area of the Semantic Web, like the reification technique [
Having fulfilled the first step towards contextual personalization in
the form of contextual knowledge representation, the next basic step
to consider is the definition of a strategy on dynamic
contextualization of user preferences. Three basic principles dominate the latter:
1. representation of context as a set of domain ontology concepts that
a user has ”touched” or followed in some manner,
2. extension of this representation of context by using explicit
semantic relations among concepts represented in the ontology
3. extension of user preferences by a similar principle
Roughly speaking, the ”intersection” of the above two sets of
concepts, with combined weights, are taken as the user preferences.
In the following, an approximation to conditional probabilities will
be utilized as an ontology-based extension mechanism. The latter is
based on the existence of relations between concepts. More formally,
given a finite set Ω, and α ∈ Ω, let P (α) be the probability that α
holds some condition. We shall use this form of estimating ”the
probability that α holds some condition” with the purpose of extending
user preferences for ontology concepts. The condition will be ”the
user is interested in concept α”, that is, P (α) will be interpreted as
the probability that the user is interested in concept α of the ontology.
Universe Ω will correspond to a domain ontology O (the universe of
all concepts). In the process of preferences and context expansion,
a variation of constrained spreading activation (CSA) strategy is
utilized [
Our model for context-based personalization can be formalized as follows: let U be the set of all users, let C be the set of all contexts, and P the set of all possible user preferences. Since each user will have different preferences, let P : U → P map each user to his/her preference. Similarly, each user is related to a different context at any given time, which we represent by a mapping C : U × N → C, since we assume that context evolves over time. Thus we shall often refer to the elements from P and C as in the form P (u) and C(u, t) respectively, where u ∈ U and t ∈ N. We define the contextualization of preferences as a mapping Φ : P × C → P so that for all p ∈ P and c ∈ C, p| = Φ(p, c).
In this context the entailment p| = q means that any consequence
that could be inferred from q could also be inferred from p. For
instance, given a user u ∈ U , if P (u) = q implies that u ”likes x”
(whatever this means), then u would also ”like x” if her/his
preference was p. Now we can particularize the above definition for a
specific representation of preference and context. In our model, we
consider user preferences as the weighted set of domain ontology
concepts for which the user has an interest, where the intensity of
interest can range from 0 to 1. Given the domain ontology O, we
define the set of all preferences over O as PO = [
As already mentioned, the selective activation of user preferences is based on an approximation to conditional probabilities: given x ∈ O with Px(u) > 0 for some u ∈ U , i.e. a concept on which a user u has some interest, the probability that x is relevant for the context can be expressed in terms of the probability that x and each concept y directly related to x in the ontology belong to the same topic, and the probability that y is relevant for the context. With this formulation, the relevance of x for the context can be computed by a constrained spreading activation algorithm, starting with the initial set of context concepts defined by C.
Our strategy is based on weighting each semantic relation r in the
ontology with a measure w(r) that represents the probability that
given the fact that r(x, y), x and y belong to the same topic. We will
use this as a criteria for estimating the certainty that y is relevant for
the context if x is relevant for the context, i.e. w(r) will be interpreted
as the probability that a concept y is relevant for the current context
if we know that a concept x is in the context, and r(x, y) holds.
Based on this measure, we use a constrained spreading activation
strategy over the semantic network defined by semantic relations in
the ontology, to expand the set of context concepts. As a result of
this strategy, the initial context C(t) is expanded to a larger context
vector EC(t), where of course ECx(t) ≥ Cx(t)Sfor all x ∈ O.
Since R is the set of all relations in O, let Rb = R {r−1|r ∈ R},
and w : Rb → [
Cy (t) if Cy (t) > 0 R
{ECx (t) · w (r) · power (x)}x∈O,r∈Rb,r(x,y) ECy (t) =
( where R is defined as:
X
S⊂Nn R(X) = (−1)|S|+1
Y
i∈S
xi
and X = {xi}in=0, where xi ∈ [
In information retrieval two major issues need to be considered towards the efficient manipulation and exploitation of user preferences. The first thing to consider is their ability to adapt to the contextual environment, i.e. their context adaptiveness, and the second thing is the special care that needs to be taken for an overall profile consistency after application of user preferences contextualizing methodologies. Under these circumstances, a novel approach is followed: extension of preferences through ontology relations, following in general the same approach, that is used to expand the runtime context.
The idea behind this methodology is to follow the principles used for the extension of the semantic context in the previous subsection 3.2. The main difference is that here relations are assigned different weights w0(r) for propagation, since the inferences one can make on user preferences, based on the semantic relations between concepts, are not necessarily the same as one would make for the contextual relevance. In general, it is expected that w0(r) ≤ w(r), i.e. user preferences are expected to have a shorter expansion than context has. Given an initial user preference P , the extended preference vector EP is defined by:
( EPy =
Py, if Py > 0 R
{EPx · w0 (r) · power (x)}x∈O,r∈Rb,r(x,y) otherwise which is equivalent to the previous formula for ECy(t), where EC, C and w have been replaced by EP , P and w0, and variable t has been removed, since long-term preferences are taken to be stable along small amounts of time. 3.4
After expanding of context, only preferred concepts with a context value different from zero will count for personalization. This is done by computing a contextual preference vector CP , as defined by CPx = EPx · Cx for each x ∈ O, where EP is the vector of extended user preferences. Now CPx can be interpreted as a combined measure of the likelihood that concept x is preferred and how relevant the concept is to the current context. Note that this vector is in fact dependent on user and time, i.e. CP (u, t). At this point we have achieved a contextual preference mapping as defined in subsection 3.1, namely Φ(P (u), C(u, t)) = CP (u, t), where P (u)| = Φ(P (u), C(u, t)), since CPx(u, t) > Px(u, t) only when EPx(u) has been derived from P (u) through the constrained spreading expansion mechanism, and CPx(u, t) < EPx(u). 4
The contextualization techniques described in this work have been implemented within an experimental prototype. We have tested the prototype on a medium-scale corpus in order to put to trial the validity and soundness of the proposed model, tune parameters, observe the behavior of the contextualization system, and draw some empirical results. In subsection 4.1 we present an example scenario of detecting a user’s preferences and in subsection 4.2 we provide some early evaluation results of our methodology. 4.1
An example scenario is provided in this section as an illustration of the application of the contextual personalization techniques. For the sake of clarity and space, full account of example details, such as the entire set of ontology concepts and relations involved, are omitted. Let us consider that Clio’s family and friends have set up a common repository where they upload and share their pictures. Clio has not checked out the collection for quite a while, and she is willing to take a look of what images her relatives have brought from their last summer vacations. Let us also assume that the proposed framework has learned some of Clio’s preferences over time, i.e. Clio’s profile includes the weighted semantic preferences for domain concepts of the ontology, shown in Table 2, where Tobby is her brother’s pet and an instance of Dog:
This would define the P vector. Now suppose that Clio selects two images shown in Figure 1. As a consequence, a runtime context is built including the elements shown in Table 3, which corresponds to the C vector.
Now, Clio wants to see some picture of her family members, and issues the query ”my family”. The contextualization mechanism comes into place and the context set is expanded through semantic relations from the initial context, adding two more weighted concepts, shown in bold in Table 4.
This makes up the EC vector. Similarly, Clio’s preferences are extended through semantic relations from her initial ones. The expanded preferences stored in the EP vector are the following, where we show the new concepts in bold (Table 5).
The contextual preferences are computed by multiplying the coordinates of the EC and the EP vectors one-on-one, yielding the CP vector depicted in Table 6 (concepts with weight 0 are omitted).
Comparing this to the initial preferences in Clio’s profile, we can see that Car, Sea and Tobby are disregarded as out of context preferences, whereas Construction and Flower have been added because they are strongly semantically related both to the initial Clio’s preferences and to the current context. 4.2
In general, evaluating personalization tasks is known to be a difficult
and expensive task [
Since the contextualization techniques are applied in the course of a user session, a sequence of steps needs to be defined in order to put them to work. According to this, we use again a short scenario, as follows: Clio is fond of all kinds of luxurious and stylish articles. Her preferences include fancy brands such as Rolex, Maybach, Lexus, Hilton, Aston Martin, Bentley, Louis Vuitton, Sony, Apple, RollsRoyce, Mercedes, Ferrari, Prada, and BMW, among others. Clio starts a search session with a query for news about Daimler-Chrysler and the different brands the company owns. Daimler-Chrysler owns both luxury brands as Mercedes or Maybach, and other more ordinary ones like Dodge or Setra that are not of interest to Clio. Personalization reorders the results according to Clio’s preferences, showing first the documents related to Daimler-Chrysler and its brands Mercedes or Mayback, and pushing down other documents related to the lower-end brands of the company. In consequence, personalized search performs significantly better from the user’s point of view. Since this is the first query of the session, no context exists yet and at this point there is no measure of the performance of the contextualization techniques.
In order to obtain this kind of information, Clio opens some documents in the search result, about the Mercedes brand and how Daimler-Chrysler is going to commercialize a new car model. She also opens a document about the new model Maybach 62. The context monitor extracts the concept of Mercedes from the documents opened by the user, along with the concept Maybach, since the selected documents were mainly about these two brands. Next, Clio makes a new query: ”companies that trade on the New York Stock Exchange and have brands in the USA”. The context is expanded to new concepts such as Daimler-Chrysler, owner of Mercedes and Maybach, along with all its brands. The query results are resorted according to the contextualized preferences of Clio. The documents that mention Daimler-Crhysler and Mercedes are pushed up in the result set. Clio still encounters other companies and brands that trade in the New York stock exchange and match her preferences, like the Sony Corporation, but these are not found semantically close to the brands in the context, and therefore get a lower sorting that other contents more in context with the previous user actions, which explains the improvement shown in Figure 2.
Reliability is a well-known concern in the area of personalization, and one important source of inaccuracy of automatic personalization techniques is that they are typically applied out of context. I.e. although users may have stable and recurrent overall preferences, not all of their interests are relevant all the time. Instead, only a subset is usually active for them at a given time. What are the driving factors that determine this subset in a given situation is a hard question in general, all the more difficult to grasp and formalize in a computer system. Indeed, a wide range of procedural, cognitive, and environmental factors intervene in the dynamic orientation of user focus while s/he interacts with a system. The notion of context becomes elusive if one aims at a holistic approach. In this paper we propose an approximation to this problem on a specific perspective, namely based on a model of semantic runtime context of user actions, with the aim to achieve a perceivable improvement in the combination of personalization and content retrieval techniques.
As widely acknowledged, context is an increasingly common notion in information retrieval. In our approach, we combined traditional personalization techniques with novel knowledge representations, such as ontologies and reification. The use of semantic concepts, rather than plain terms, for the representation of contextual meanings, and exploitation of explicit ontology-based information attached to the concepts forms a significant novelty in the area. We also combined implicit context meanings collected at runtime, with a persistent and more general representation of user preferences. Benefit from the overall methodology is twofold: personalization techniques gain accuracy and reliability by avoiding the risk of having locally irrelevant user preferences getting in the way of a specific and focused user retrieval activity. Inversely, the pieces of meaning extracted from the context are filtered, directed, enriched, and made more coherent and meaningful by relating them to user preferences. Further insights to be drawn upon theoretic analysis, as well as the observations concerning performance and trade-offs from the experimental results of future implementation work and testing, shall provide further grounds for the analysis and evaluation of this approach.