=Paper=
{{Paper
|id=None
|storemode=property
|title=Interactive News Video Recommendation: An Example System
|pdfUrl=https://ceur-ws.org/Vol-747/paper5.pdf
|volume=Vol-747
}}
==Interactive News Video Recommendation: An Example System==
https://ceur-ws.org/Vol-747/paper5.pdf
Interactive News Video Recommendation: An Example
System
Frank Hopfgartner
International Computer Science Institute
1947 Center Street, Suite 600
Berkeley, CA, 94704
fh@icsi.berkeley.edu
ABSTRACT signals and accompanying metadata. The audio-visual fea-
This position paper introduces a recommender system which tures can be described by low-level feature descriptors, the
has been developed to study research questions in the field main description standard being MPEG-7.
of news video recommendation and personalization. The
system is based on semantically enriched video data and Retrieving videos using low-level features is, due to the Se-
can be seen as an example system that allows research on mantic Gap [18], a challenging approach. An analysis of
semantic models for adaptive interactive systems. state-of-the-art research on video retrieval indicates that
content-based video retrieval performance is still far away
from their textual counterparts [7]. An interesting approach
1. INTRODUCTION to narrow this performance gap is to further enrich video
In recent years, the amount of multimedia content available documents using external data sources, called metadata.
to users has increased exponentially. This phenomenon has Blanken et al. [4] list three types of metadata: (1) Descrip-
come along with (and to much an extent is the consequence tive Data, (2) Text Annotations and (3) Semantic Annota-
of) a rapid development of tools, devices, and social services tion. All approaches aim to provide annotations in textual
which facilitate the creation, storage and sharing of personal form that allow to bridge the Semantic Gap. Fernández et
multimedia content. A new landscape for business and in- al. [9], for instance, have shown that ontology-based search
novation opportunities in multimedia content and technolo- models that exploit semantic annotations can outperform
gies has naturally emerged from this evolution, at the same classical information retrieval models at a web scale. The
time that new problems and challenges arise. In particular, advantage of these models is that external knowledge is used
the hype around social services dealing with visual content, to set the content into their semantic context.
such as YouTube or Dailymotion has led to a rather scat-
tered publishing of video data by users worldwide [8]. Due In [10], we introduced a news video recommender system
to the sheer amount of large data collections, there is a grow- which relies on such semantic annotations. The system cap-
ing need to develop new methods that support the users in tures daily broadcasting news, and segments the bulletins
searching and finding videos they are interested in. into semantically related news stories. DBpedia is exploited
to set these stories into context. DBpedia is a structured
Video retrieval is a specialization of information retrieval representation of Wikipedia [2]. This semantic augmenta-
(IR), a research domain that focuses on the effective stor- tion of news stories is used as the backbone of our news video
age and access of data. In a classical information retrieval recommendation. Our first hypothesis was that implicit rel-
scenario, a user aims to satisfy their information need by evance feedback can be used to create appropriate long-term
formulating a search query. This action triggers a retrieval user profiles. Implicit relevance feedback refers to user in-
process which results in a list of ranked documents, usually teractions that are performed implicitly during a search ses-
presented in decreasing order of relevance. The activity of sion, such as clicking a search result or spending time to
performing a search is called the information seeking pro- read/view a document. We introduced an implicit user mod-
cess. A document can be any type of data accessible by a eling approach which automatically captured users’ evolving
retrieval system. In the text retrieval domain, documents information needs, representing interests in a dynamic user
can be textual documents such as emails or websites. Image profile. Another research question was to study whether the
documents can be photos, graphics or other types of visual il- selection of concepts in a generic ontology can be used for
lustrations. Video documents consist of a set of audio-visual accurate news video recommendations. Therefore, we intro-
duced our approach of exploiting DBpedia to set concepts
of news stories into their semantic context. As our evalu-
ation indicates, semantic recommendations can successfully
be employed to improve the recommendation quality.
While we evaluated within this work the underlying person-
alization technique, which takes advantage of an ontology,
the impact of the adaptive presentation of the recommen-
dations and search results, i.e. the interface design, has not
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
been evaluated yet. Given a well-evaluated backend which of this approach is that it simplifies the information seeking
relies on Semantic Web technologies, we argue in this posi- process, e.g. by releasing the user from manually reformulat-
tion paper that the introduced personalization system can ing the search query, which might be problematic especially
be seen as an exemplar system which allows for studying the when the user is not exactly sure what they are looking for or
research questions that are within the scope of this work- does not know how to formulate their information need. Two
shop. After introducing the research domain in Section 2, types of relevance feedback exist: explicit and implicit feed-
we illustrate in Section 3 how users can use the system to re- back. While explicit RF models rely on users permanently
ceive frequent news video recommendations that match their providing relevance information about documents they re-
personal interests. In Section 4, we introduce the interface trieved, implicit RF models rely on automatically mining
of prior mentioned system, which is required to visualize se- user interaction data. The main advantage is that this ap-
mantically enriched video data. Section 5 discusses how this proach delivers the user from providing explicit feedback.
system can be used as an example to study semantic models
for adaptive interactive systems. Most personalization services rely on users explicitly specify-
ing preferences. However, users tend not to provide constant
2. SEMANTIC NEWS VIDEO RECOMMEN- explicit feedback on what they are interested in. In a long-
DATION term user profiling scenario, this lack of feedback is critical,
since feedback is essential for the creation of such profiles.
When interacting with a video retrieval system, users ex-
Considering that each interface feature is designed to allow
press their information need in search queries. The under-
users to either retrieve or explore document collections, we
lying retrieval engine then retrieves relevant results to the
hypothesized in [10] that the users’ interactions with these
given queries. A necessary requisite for this IR scenario is to
features can be exploited as implicit relevance feedback. We
correctly interpret the users’ information need. As Spink et
introduced a news video recommender system which auto-
al. [19] indicate though, users very often are not sure about
matically generates personalized multimedia news that cover
their information need. One problem they face is that they
topics of the users’ long-term interests.
are often unfamiliar with the data collection, thus they do
not exactly know what information they can expect from
Defining the technical conditions for such recommender sys-
the corpus [17]. Further, Jansen et al. [12] have shown that
tems, we argued that the creation of a private news video
video search queries are rather short, usually consisting of
collection is required, consisting of up-to-date news bulletins
approximately three terms. Considering these observations,
from different broadcasting stations. Further, we argued
it is hence challenging to satisfy users’ information needs,
that semantic web technology can be exploited to link con-
especially when dealing with ambiguous queries. Triggering
cepts in the news broadcasts and suggested a categorization
the short search query “Victoria”, for example, a user might
of stories into broad news categories. From a user profiling
be interested in videos about cities called Victoria (e.g. in
point of view, these links and categories can be of high value
Canada, United States or Malta), landmarks (e.g. Victoria
to recommend semantically related transcripts, hence creat-
Park in Glasgow or London), famous persons (e.g. Queen
ing a semantic-based user profile. For example, a user could
Victoria or Victoria Beckham) or other entities called Vic-
show interest in a story about the sunset at the Greek island
toria. Without further knowledge, it is a demanding task
Santorini. The story transcript might contain the following
to understand the users’ intentions. Interactive information
sentence:
retrieval aims at improving the classic information retrieval
model by studying how to further engage users in the re-
trieval process, in a way that the system can have a more “This is Peter Miller, reporting live from San-
complete understanding of their information need. Thus, torini, Greece, where we are just about to wit-
aiming to minimize the users’ efforts to fulfill their informa- ness one of the most magnificent sunsets of the
tion seeking task, there is a need to personalize search. In decade. [...]”.
a web search scenario, Mobasher et al. [14] define personal-
ization as “any action that tailors the Web experience to a
particular user, or a set of users”. Another popular name is If the same user enjoys travel with emphasis on warm Mediter-
adaptive information retrieval, which was coined by Belew ranean sites, he/she might also be interested in a report
[3] to describe the approach of adapting, over time, retrieval about the Spanish island Majorca. For example, imagine
results based on users’ interests. the following story:
Most of the approaches that follow the interactive informa-
tion retrieval model are based on relevance feedback tech- “Just as every year, thousands of tourists enjoy
niques [17]. Relevance feedback (RF) is one of the most im- their annual sun bath here in Majorca. [...]”.
portant techniques within the IR community. An overview
of the large amount of research focusing on exploiting rele-
vance feedback is given by Ruthven and Lalmas [16]. The An interesting research question is how to identify whether
principle of relevance feedback is to identify the user’s infor- this story matches the user’s interests. Lioma and Ounis
mation need and then, exploiting this knowledge, adapting [13] argue that the semantic meaning of a text is mostly ex-
search results. Rocchio [15] defines relevance feedback as pressed by nouns and foreign names, since they carry the
follows: The retrieval system displays search results, users highest content load. Indeed, most adaptation approaches
provide feedback by specifying keywords or judging the rel- rely on these terms to personalize retrieval results, e.g. by
evance of retrieved documents and the system updates the performing a simple query expansion. The two example sto-
results by incorporating this feedback. The main benefit ries, however, do not share similar terms. A personalization
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
technique exploiting the terms only would hence not be able in the search panel on top, results are listed on the right
to recommend the second story. However, linking the con- side and a navigation panel is placed on the left side of the
cepts of the transcripts using DBpedia reveals the semantic interface. When logging in, the latest news will be listed in
context of both stories. It becomes evident that both sto- the results panel. Search results are listed based on their
ries are about two islands in the Mediterranean Sea. Ex- relevance to the query. Since we are using a news corpus,
ploiting this link could hence satisfy the user’s interest in however, users can re-arrange the results in chronological
warm Mediterranean Sites. We therefore proposed to set order with latest news listed first. Each entry in the result
news broadcasts into their semantic context by exploiting list is visualized by an example key frame and a text snippet
the large pool of linked concepts provided by DBpedia. of the story’s transcript. Keywords from the search query
are highlighted to ease the access to the results. Moving
Having established a semantically annotated data collection, the mouse over one of the key frames shows a tool tip pro-
the recommender system can be operated on a regular basis viding additional information about the story. A user can
to retrieve news stories that match the user’s interests. In get additional information about the result by clicking on
the next section, we illustrate a typical use-case that illus- either the text or the key frame. This will expand the result
trates the use of the exemplar system. and present additional information including the full text
transcript, broadcasting date, time and channel and a list
3. USE-CASE SCENARIO of extracted named entities. In the example screenshot, the
In the previous section, we provided a brief summary of the third search result has been expanded. The shots forming
research challenges that have been tackled in [10]. Users the news story are represented by animated key frames of
can interact with this system on a regular basis, e.g. over each shot. Users can browse through these animations either
several weeks, to satisfy their information need, allowing for by clicking on the key frame or by using the mouse wheel.
longitudinal user studies where the system can be evaluated. This action will center the selected key frame and surround
The following example depicts a typical use-case scenario: it by its neighboring key frames. The user’s interactions
with the interface are exploited to identify multiple topics
of interests. On the left hand side of the interface, these in-
“Imagine a user who is interested in multiple news terests are presented by different categories, i.e. those news
topics. They registered with a news recommender categories that the user showed interest in during previous
system with a unique identifier. For a period of search sessions.
several months, they log into the system, which
provides them access to the latest news video sto- Summarizing, the interface provides access to different news
ries of the day. On the system’s graphical inter- categories in which the user showed interest in. These inter-
face, they have a list of the latest stories which ests can adapt over time, i.e. when a user shows interest in a
have been broadcast on two national television certain news aspect right now, this aspect might already be
channels. They now interact with the presented irrelevant in a few days. Imagine, for example, a user who
results and logs off again. On each subsequent has shown high interest in any news regarding the FIFA
day, they log in again and continue the above Soccer World Cup. Just a few days after the end of the
process.” tournament, the user’s interest might drop to a minimum
again. Our interface serves this evolving need by automati-
cally updating the categories in which the user showed the
In this scenario, a user frequently uses the system to gather most interest in during the last sessions. The evolving inter-
latest news. The interface has been designed to adapt its est is modeled by applying the Ostensive Model [6], which
content based on users’ personal interests by employing the provides a decay function that aligns a higher weighting to
semantic context of the data collection. Each time, he/she more recent user interests.
interacts with the video documents which have been dis-
played by the graphical user interface, he/she leaves a “se- 5. DISCUSSION AND CONCLUSION
mantic fingerprint” of their interests. Based on this finger- Above description reveals that the interface has been de-
print, more video documents are identified by exploiting the signed to visualize news videos that match users’ interests.
semantic link between the video documents in the collection. The categorization of these interests is highly user-centric.
Hence, each time the user interacts with retrieval results, The interface adapts its content, i.e. both categories on the
other related videos are identified and displayed. A long- left hand side and news videos on the right hand side based
term user study focusing on evaluating the performance of on the users’ previous interactions. Even though the recom-
different recommendation techniques has been introduced in mendation technique relies on interlinked data, the interface
[11]. itself does not support filtering or browsing the data accord-
ingly.
While this evaluation is focused on the recommendation
techniques, a thorough evaluation of the interface has not As mentioned before, this constraint is due to the different
been done yet. An overview over the interface is given in focus of the research, which was aiming at studying rec-
the next section. ommendation techniques rather than adaptive interface de-
signs. Nevertheless, given the support of semantically en-
4. INTERFACE DESIGN riched video data, we argue that the system can be seen
Figure 1 shows a screenshot of the adaptive news video re- as an example framework which enables to study such in-
trieval interface which was used within the study. It can be terface features. Example improvements include visualizing
split into three main areas: Search queries can be entered story interlinking by using a hyperbolic tree, as has been
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
Figure 1: News Video Recommender Interface
introduced by Bürger et al. [5]. In their Smart Content [3] R. K. Belew. Adaptive information retrieval: using a
Factory, each document in the index has been enriched with connectionist representation to retrieve and learn
semantic information, i.e. places mentioned in the transcript about documents. SIGIR Forum, 23(SI):11–20, 1989.
are matched with a generic geography thesaurus. Such tree [4] H. M. Blanken, A. P. de Vries, H. E. Bok, and
would allow users to browse the video collection based on the L. Feng. Multimedia Retrieval. Springer Verlag,
semantic content of each video. Another improvement could Heidelberg, Germany, 1 edition, 2007.
be to provide thesaurus supported query auto-completion [5] T. Bürger, E. Gams, and G. Güntner. Smart content
features as shown by Amin et al. [1]. This would allow users factory: assisting search for digital objects by generic
to get an idea about the collection based on the query sug- linking concepts to multimedia content. In Proc. HT,
gestions. pages 286–287. ACM, 2005.
[6] I. Campbell and C. J. van Rijsbergen. The ostensive
Acknowledgment model of developing information needs. In Proc.
The author was supported by a fellowship within the Postdoc- Library Science, pages 251–268, 1996.
Program of the German Academic Exchange Service (DAAD). [7] M. G. Christel. Establishing the utility of non-text
search for news video retrieval with real world users.
6. REFERENCES In MULTIMEDIA ’07: Proceedings of the 15th
[1] A. Amin, M. Hildebrand, J. van Ossenbruggen, international conference on Multimedia, pages
V. Evers, and L. Hardman. Organizing suggestions in 707–716, New York, NY, USA, 2007. ACM.
autocompletion interfaces. In ECIR’09: Proceedings of [8] S. J. Cunningham and D. M. Nichols. How people find
the 31st European Conference on IR Research, ECIR videos. In Proc. 8th ACM/IEEE-CS Joint Conference
2009, Toulouse, France, pages 521–529, 2009. on Digital libraries, pages 201–210, New York, NY,
[2] S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, USA, 2008. ACM.
R. Cyganiak, and Z. G. Ives. DBpedia: A Nucleus for [9] M. Fernández, V. López, M. Sabou, V. Uren,
a Web of Open Data. In Proc. 6th Int. Semantic Web D. Vallet, E. Motta, and P. Castells. Using TREC for
Conf., pages 722–735. Springer Berlin / Heidelberg, 11 cross-comparison between classic IR and
2007. ontology-based search models at a Web scale. In
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA
SemSearch’09, 4 2009. [15] J. J. Rocchio. Relevance feedback in information
[10] F. Hopfgartner and J. M. Jose. Semantic user retrieval. In G. Salton, editor, The SMART retrieval
modelling for personal news video retrieval. In MMM, system: experiments in automatic document
pages 336–346, 2010. processing, pages 313–323, Englewood Cliffs, USA,
[11] F. Hopfgartner and J. M. Jose. Semantic user profiling 1971. Prentice-Hall.
techniques for personalised multimedia [16] I. Ruthven and M. Lalmas. A survey on the use of
recommendation. Multimedia Systems, 16(4):255–274, relevance feedback for information access systems. The
2010. Knowledge Engineering Review, 18(2):95–145, 2003.
[12] B. J. Jansen, A. Goodrum, and A. Spink. Searching [17] G. Salton and C. Buckley. Improving retrieval
for multimedia: analysis of audio, video and image performance by relevance feedback. Readings in
web queries. World Wide Web, 3(4):249–254, 2000. information retrieval, pages 355–364, 1997.
[13] C. Lioma and I. Ounis. Examining the Content Load [18] A. W. M. Smeulders, M. Worring, S. Santini,
of Part of Speech Blocks for Information Retrieval. In A. Gupta, and R. Jain. Content-Based Image
ACL’06: Proceedings of the 21st International Retrieval at the End of the Early Years. IEEE Trans.
Conference on Computational Linguistics and 44th on Pattern Analysis and Machine Intelligence,
Annual Meeting of the Association for Computational 22(12):1349–1380, 2000.
Linguistics, Sydney, Australia, 2006. [19] A. Spink, H. Greisdorf, and J. Bateman. From highly
[14] B. Mobasher, R. Cooley, and J. Srivastava. Automatic relevant to not relevant: examining different regions of
personalization based on web usage mining. relevance. Inf. Process. Manage., 34(5):599–621, 1998.
Communications of the ACM, 43(8):142–151, 2000.
Copyright is held by the author/owner(s)
SEMAIS'11, Feb 13 2011, Palo Alto, CA, USA