=Paper=
{{Paper
|id=None
|storemode=property
|title=SOI Based Video Recommender Systems - Interaction Design Issues and Collective Intelligence
|pdfUrl=https://ceur-ws.org/Vol-1051/paper1.pdf
|volume=Vol-1051
|dblpUrl=https://dblp.org/rec/conf/ihc/DiasWG13
}}
==SOI Based Video Recommender Systems - Interaction Design Issues and Collective Intelligence==
https://ceur-ws.org/Vol-1051/paper1.pdf
SOI Based Video Recommender Systems:
Interaction Design Issues and Collective Intelligence
Alessandro Dias Leandro Krug Wives Isabela Gasparini
PPGC - Instituto de Informática PPGC - Instituto de Informática Departamento de Ciência da
Universidade Federal do Rio Universidade Federal do Rio Computação – Universidade do
Grande do Sul Grande do Sul Estado de Santa Catarina
Porto Alegre, RS, Brazil Porto Alegre, RS, Brazil Joinville, SC, Brazil
asdias@inf.ufrgs.br wives@inf.ufrgs.br isabela.gasparini@udesc.br
ABSTRACT No domínio de vídeos, uma dessas extensões é baseada em
Recommender systems help users to cope with information Segmentos de Interesse, i.e., trechos de vídeo que o usuário
overload and have become one of the most powerful and mais gostou ou está interessado. Neste trabalho, nossa
popular tools in electronic commerce. In order to provide intenção consiste em analisar e discutir questões de design
better recommendations and to be able to use recommender de interação em sistemas de recomendação de vídeo
systems in arguably more complex types of applications, apoiados por segmentos de interesse, além de discutir a sua
most of the typical used approaches need significant relação com inteligência coletiva. Para tanto, apresentamos
extensions. On the video recommendation domain, one of duas abordagens para realizar a marcação de segmentos em
these extensions is based in Segments of Interest (SOI), i.e., um ambiente social Web e discutimos suas vantagens e
video segments that the user liked more or is interested. For desvantagens. Além disso, mostramos porque os segmentos
this work, our intention is to stress and discuss interaction podem ser vistos como uma fonte de inteligência coletiva e
design issues about SOI based video recommender systems que a informação e o conhecimento que emerge de uma
and discuss the relation between SOI and collective comunidade que marcou seguimentos pode ser usada na
intelligence. We present two approaches to marking SOI on tomada de decisões consensuais e trazer melhorias para a
a Web social environment and discuss their advantages and sociedade.
disadvantages, and we show why SOI can be seen as a
Palavras-chave
source of collective intelligence and that information and
Sistemas de Recomendação; vídeo; segmento de interesse;
knowledge emerged from a community that had marked
inteligência coletiva.
SOIs can be used on consensus decision-making and to
bring improvements to society. INTRODUCTION
Recommender Systems provide recommendations for items
Author Keywords
to be of use to a user. They help users to cope with
Recommender systems; video; segment of interest;
information overload and have become one of the most
collective intelligence.
powerful and popular tools in electronic commerce.
ACM Classification Keywords
H.3.3 Information Search and Retrieval. In these systems, recommendations are generally made by
two types of filtering: collaborative and content-based.
RESUMO Many systems use these types combined in a hybrid
Sistemas de Recomendação auxiliam usuários a lidar com o approach. Furthermore, in order to provide better
problema da sobrecarga de informação e se tornaram uma recommendations and to be able to use recommender
das ferramentas mais populares e poderosas do comércio systems in arguably more complex types of applications,
eletrônico. De modo a prover melhores recomendações e most of the typical approaches need significant extensions.
para que possam ser utilizados em tipos cada vez mais On video recommendation domain, one of these extensions
complexos de aplicação, grande parte das abordagens is based on Segments of Interest (SOI), i.e., video segments
tipicamente utilizadas precisam de extensões significativas. that the user liked more or is interested.
In a previews work [7] we developed a website with a video
Copyright © by the paper's authors. Copying permitted only for private recommender engine based on collaborative filtering, and it
and academic purposes. In: Proceedings of the V Workshop sobre
Aspectos da Interação Humano-Computador na Web Social
presented an approach that uses SOI to enhance the
(WAIHCWS'13), Manaus, Brazil, 2013, published at http://ceur-ws.org. accuracy of rating predictions of video recommender
systems.
For this work, our intention is to stress and discuss
interaction design issues and the relation between SOI and
collective intelligence. We present two approaches to
7
marking SOI and discuss their advantages and
disadvantages, and we show why SOI can be seen as a
source of collective intelligence and discuss that
information and knowledge emerged from a community
that had marked SOIs can be used on consensus decision- Figure 1. SOIs marked by a user in a video.
making and to bring improvements to society.
RECOMMENDER SYSTEMS AND SEGMENTS OF In our previous work, we show that SOIs from a
INTEREST community of users can be used to find similar people, i.e.
The recommendations provided by recommender systems people who have similar interests about videos. When a pair
are aimed at supporting users in various decision-making of users has a quantity of intersections of SOIs above a
processes, such as what items to buy, what music to listen, threshold (and with a minimum acceptable size) in one
or what news to read. Recommender systems have proven video and this pattern occurs in a certain quantity above a
to be valuable means for online users to cope with the threshold in a set of videos, they have similar interests
information overload problem [17]. about video. Figure 2, for instance, illustrates that users u2
As already stated, recommender systems are generally and u4 will have similar interests about video if it is
made by two types: collaborative filtering (CF) and content- considered that similar users have at least 2 intersections of
based filtering (CBF). For them, various algorithms and SOIs on a set of 3 or more videos.
techniques have been proposed and successfully evaluated. We also show, in our previous work, that this similarity
CBF takes the descriptions of the previously evaluated or between pairs of users based on SOI can be used to enhance
currently accessed items by the user to calculate the the accuracy of ratings predictions of video recommender
similarity between items, and, then, recommend items of systems with user based nearest neighbor collaborative
interest to the user. This type of filtering enables recommendation.
personalized recommendations for users. CF calculates the
similarity between users and recommends items that are Marking SOI
liked by similar users. This uses ratings given by users in One important aspect is how users can mark SOIs in video.
the past to find the best item to a similar user. Another The way users mark SOIs is directly related to the user
approach is to calculate the similarity between items to interface of the system. This interface must provide specific
produce recommendations [10]. components for marking SOIs. We have proposed two
approaches for this purpose:
These two approaches present some issues, if used alone.
For CBF, for instance, the two main disadvantages are: (i) it • Buttons with predefined time slices: in this case, different
depends on one objective description of the items; and (ii) it buttons, with different time slices, are presented to the
tends to overspecialize recommendations [14]. For CF, the user. Each time the users want to indicate a SOI, they
two main disadvantages are: (i) the early-rater problem that click on the button that corresponds better to the size of
occurs when a user is the first from his/her neighborhood to SOI they want (whose the end coincides with the current
rate an item; and (ii) the sparsity problem that is caused instant of time in the video). An example of this approach
when there are few ratings for the items [17]. On the other is presented in Figure 4.
hand, these types recommendation can be combined in a • Sliders that allow users to mark the beginning and the end
hybrid recommender system that takes the advantages of of each SOI. In this case, users can mark videos while
both in order to overcome their disadvantages alone. they are watching the video (like the previous approach)
Furthermore, according to [1], in order to provide better or after watching it. An example of this approach is
recommendations and to be able to use recommender presented in Figure 3.
systems in arguably more complex types of applications, The approach for marking SOI must be chosen accordingly
most of the typical approaches need significant extensions. to the context and to the user tasks, because both have
On the video recommendation domain, one of these advantages and disadvantages. Marking SOI with the use of
extensions is based on Segments of Interest. buttons on the user interface (or buttons in a remote control)
Segments of Interest (SOI)
is indicated for video websites, video/movie on demand
A SOI is a segment on video that the user liked more or is services and personal video recorders, where the user's goal
interested. Users tend to like particular segments of the is to enjoy entertainment or get information. The advantage
video more than the rest [3] and, therefore, they can mark is that users do not lose so much attention when they want
their segments of interest on video. Figure 1 illustrates SOIs to mark SOIs on the video, and this marking occurs quite
marked by a user in a video. The first one was marked from effortlessly, quickly and easily. The drawback is that the
t1 to t2 seconds related to video timeline, the second one beginning and the end of a segment may not be as precise
from t3 to t4 seconds, and the third one from t5 to t6 seconds. as intended (since the intervals are predefined). If SOI are
used to enhance the accuracy of the ratings predictions of
8
recommenders systems, this precision is relevant for the reluctant to give explicit feedback (in this case, marking
accuracy [7]. SOIs), that not everyone likes to participate more actively
and more interactively, and that, in our approach, there is a
significant level of user involvement [7]. As we discussed
before, the context and user tasks impose the choice of the
approach to be implemented and it is related to the success
of the system and their recommendations.
THE DEVELOPED SYSTEM
In our previews work we developed a video website with a
recommender engine as a web application. The server hosts
the application, and its interface is similar to many current
video websites, i.e., it has a screen containing a video
gallery where the user can browse videos, and a screen to
watch videos. The client can be any device that contains a
web browser, such as a desktop computer, a notebook, a
smartphone, a tablet or a smart TV. This system only stores
metadata to catalog videos and data generated during the
use of the system. Videos are loaded directly from clients
on demand from a video content provider (YouTube, in the
specific case).
For performing an experimental evaluation of our
recommender system approach, we have used educational
videos of a given subject, but the system can deal with
several domains, such as news, sports and movies. For this
purpose, we have created a catalog containing 50
educational videos up 20 minutes of duration from
YouTube.
Figure 3 shows part of the screen of the system where users
watch videos. Through it, users can rate videos (rating
options are, "Very bad", "Bad", "Ok", "Good" and "Very
Figure 2. Intersections of SOIs in a set of videos.
good"), and the users can also mark their SOIs. This is
The approach for marking SOIs delimiting the beginning performed through a double slider below the progress bar of
and the end of the segment can be provided, for example, the video player. Users drag and drop the sliders to mark
through widgets as a pair of sliders bellow the progress bar the beginning and the ending of each SOI. This approach to
(video timeline) of the video player. It is suitable for marking segments was used previously in video remix tasks
environments where users have more freedom (since they [18]. Additionally, the screen presents a sorted list of SOIs
can fast forward and rewind the video, or watch it as many already marked and the user can delete incorrect or
times as they want), such as websites and video on demand unnecessary SOIs, if necessary. SOIs from this list could be
services with more specific goals, such as websites with used after by the users to directly access their segments of
educational videos. The advantage is that users can define interest into the video as a shot index [13]. This approach
the exact beginning and end of each SOI. If SOIs are used for marking SOIs is suitable for educational environments
to enhance the accuracy of recommender systems ratings where users usually are near the screen, and move forward
predictions, this approach is better suited. The disadvantage and rewind the video several times during their learning
is that the marking of SOIs requires a greater effort from activities.
the user, as it needs to locate and define exactly each
segment of interest on the video timeline. There may be
users who prefer to watch the whole video and then rewind
it to mark a specific SOI, but others may want to mark the
segment while they are watching the video. In the first case,
users may forget to mark some SOI, and, in the second
case, users can lose attention while watching the video and
marking SOI at the same time.
Figure 3. Partial view of the system's screen
The approach to mark SOI, that is part of the interaction containing sliders.
design, is extremely important. The designer of SOI based
recommender systems must be aware that users are
9
If the context was other, like watching movies in a smart we have asked the subjects of the previous experiment to
TV, the segments of interest could be marked in another answer a questionnaire (anonymously) available on the
way, using specific buttons on remote control or on a Web. Results are described below. It is important to state
smartphone application, for example, while the user that only 18 of 88 users have answered the questionnaire.
watches the video. Figure 4 illustrates the approach based
on buttons on the user interface of the video website. There, Question #1: Did you mark segments of interest in what
moments?
three buttons can be seen: ‘10 seconds’, ‘20 seconds’, and
‘30 seconds’. If the user is watching a video and click on
the "20 seconds" button, a SOI is marked from the current 40%
position on the video timeline until 20 seconds back. It is
easy and fast, and could be even done remotely, if the 30%
buttons are in a remote device (remote control or
smartphone). 20%
10%
0%
after while most while most after
In this question, "after" means "I marked after I watched
the video", "while" means "I marked while I was watching
the video", "most while" means "I marked after and while I
Figure 4. Partial view of the system's screen
was watching the video, but in the most of times after", and,
containing buttons with predefined time slices.
finally, "most after" means "I marked "after" and "while" I
was watching the video, but in the most of times while".
EXPERIMENTAL EVALUATION
As already stated, the system was described in a previous
Question #2: How often do you watch videos on Web?
work [7]. There, we also described an experiment in which
the system was used by three different groups of students.
We will not detail the experiments and its results here, but it I do not watch I rarely watch I watch frequently
is important to state that the subjects were free to choose 0%
videos among the catalog created, and also were free to
evaluate them and to mark SOIs. Based on the interaction
17%
of the users with the system, a historical dataset was built.
This dataset contains 88 user profiles (students of computer
science, at the age of 20 years), 764 video ratings and 269 83%
SOIs.
The focus of our previous work was to compare different
recommender system strategies. In this sense, we have
conducted an offline experimental evaluation based on the
leave-one-out strategy [17]. Based on it, we compared a Question #3: Do you often collaborate with some information
traditional collaborative strategy against the same approach, on the Web (such as 'add a comment', 'submit a link to a
friend' or give a "like"')?
but boosted by SOI. The results showed that it is possible to
find similar people based on SOI and that the system´s
accuracy improvement is directly related to the level of no yes, rarely yes, frequently
participation of people marking SOI, so, as more people
collaborate and interact, better is the result of the system.
In this work, our intention is to stress and discuss
interaction design issues related to our SOI video 28% 33%
recommender system. Therefore, we have used a
questionnaire that is described below. 39%
Questionnaire
We have built a questionnaire to understand the user´s
experience related to marking SOI on our system, to know
about their collaboration habits on the Web, and about their
habits related to watching video on the Web. In this sense,
10
Question #4: Do you usually use somehow the information potential of social networks as a mean to exercise the
collaboratively added by others on websites? citizenship. It assumes that individual intelligences are
summed and shared across society.
no yes, sometimes yes, frequently Collective intelligence led to the rise of a new business
model known as crowdsourcing. This term describes a
model that takes advantage of several creative solutions that
people can propose [9]. The initial idea of crowdsourcing
22% 33% was to send a task to the crowd instead of running it using
its own resources. This approach is known as explicit
45% crowdsourcing. For instance, on Internet users can evaluate
particular items like books or movies, or share by posting
products or digital content. Users can also build artifacts by
providing information and editing other people's work.
Linux operating system and Wikipedia are examples of
Question #5: Would you like to share your segments of explicit crowdsourcing results.
interest with other people?
The other approach is known as implicit crowdsourcing,
which can take two forms: standalone and piggyback [8].
yes, always yes, but with my permission no The standalone allows people to solve problems as a side
effect of the task they are actually doing, whereas
piggyback takes users' information from a third-party
website to gather information. Implicit crowdsourcing is
22% 33% less obvious because users do not necessarily know with
whom they are contributing, yet it can still be very effective
45% in completing certain tasks [2]. For instance, piggyback
crowdsourcing can be seen most frequently in websites
such as Google, which mine users' search history and
websites in order to discover keywords for ads, spelling
corrections, and finding synonyms [11]. Google's PageRank
Analysis of the questionnaire´s results
algorithm harness collective intelligence. Every time users
The results showed that in the context of educational write a link, or even click on one, they are feeding their
videos, considering the approach used to mark SOIs, most intelligence into Google’s system [15]. On video domain,
of users prefer to mark SOIs after watching videos. In [5] proposes to leverage implicit user activity on video
another question, not viewed by graphs, 100% of the player (e.g., pause/play, seek/scrub) in order to dynamically
subjects answered that they enjoyed using the system to identify segments of interest on video, and presents an
watch the educational videos offered. implicit user-based key-frame detection system.
About the habits of the subjects, they frequently watch In this sense, SOI, which is collected in an explicit way and
videos on the Web and use collaboration mechanisms to is used to enhance the accuracy of rating predictions of
obtain information. However, very few subjects effectively video recommender systems, can be used implicitly as
collaborated with others (to provide information). Finally, source of information and knowledge about a user
most of them stated that they would like to share their SOIs. community. For instance, depending of the context, a
It could be done, for instance, in a social network among cluster of SOIs can sign something relevant in one video.
their friends. For instance, in the entertainment video domain, a cluster of
Segments of Interest on Video and Collective
SOIs can represent the most popular segments of a
Intelligence particular movie for the community; similarly, in the
In this work, another relevant aspect to discuss is the educational environment, a cluster of SOIs can be the
relation between SOI and collective intelligence. starting point for a teacher to discover the more relevant
segments for a community of students and, based on this
Collaboration among people has been gaining attention due information, figure out why that point is important for
to the popularity of social media tools such as forums, learning. In this sense, SOI can be seen as a source of
blogs, wikis and social networks. These tools are collective intelligence.
collaborative systems that enable users to share their
knowledge, skills and other information. As a result, they SOI Data Visualization
are becoming important sources of collective intelligence To discover clusters of SOIs we have implemented a data
[4]. The science of collective intelligence, proposed from visualization display in the developed system. This display
the discussions of Pierre Lévy [12], tries to harness the presents the set of SOIs marked by each user in a video. As
11
we had only two dimensions (SOI vs. users), we have used CONCLUSION AND FUTURE WORK
a simple data visualization technique that uses rectangles on Segments of interest, which can be used to enhance the
orthogonal axes to represent intervals [6]. accuracy of rating predictions of video recommender
system, can be marked by different ways. Each way is
The display illustrated in the Figure 5 corresponds to the directly related to the user interface of the system that
aforementioned visualization technique. There, users are contains the recommender system. In this work, we showed
represented on the vertical axis and the horizontal axis is that each way have advantages and disadvantages.
the length of the video. For instance, in this figure, most Therefore, the designer of SOI based recommender systems
users marked just one SOI, one marked two and another must be aware that the approach chosen to allow users to
marked three. mark SOIs may be directly related of the success or failure
of the system and its recommendations.
Furthermore, we have evaluated the use of SOI concerning
collaboration and user's habits about consuming video on
the Web. The results showed that, in the context of
educational videos and considering the approach used to
mark SOIs, most of the subjects prefer to mark SOI after
watching videos, and all subjects enjoyed to use the
developed system to watch videos. About their habits, they
stated that the frequently watch videos on the Web and use
advices obtained through collaboration. However, very few
of them admitted to collaborate with others on the Web. In
relation to sharing their SOIs, most of them stated that they
would like to share them with friends, for instance, in a
social network.
Figure 5. Data visualization of SOI by users on a video.
We also discussed why SOI could be seen as a source of
In the display, each rectangle represent a SOI. The size of
collective intelligence. Information and knowledge emerged
each rectangle indicates the size of the interval in relation to
from a community that had marked SOIs can be used on
the video timeline. The position of the rectangle (and it
consensus decision-making and to bring improvements to
size) corresponds to where it was marked in the video
society. We implement data visualization over the
timeline.
developed system to illustrate how SOI can be such a
To aid the analysis, we included a combo box to allow source of collective intelligence.
browsing videos. Once a video is selected, one key frame is
In future works, we will make more experiments to
shown to illustrate the video selected. With this tool, we
compare even further the presented approaches. The goal is
were able to discover the parts that were most marked by
to verify which one is better suited for different contexts
the community of users. For instance, Figure 5 shows the
and scenarios.
data visualization display for video "Visualization of
Quicksort", with the 13 users who had marked SOIs in it. ACKNOWLEDGMENTS
The video is an animation about sorting algorithms. As we This work was partially supported by CNPq and CAPES.
can see, many users had marked SOI in a special part of the REFERENCES
video (on the right). Such part is the moment where the 1. Adomavicius G., Tuzhilin, A. Toward The Next
animation starts to compare the Quicksort and the Generation of Recommender Systems: A Survey of the
Bubblesort algorithms, running side-by-side in a dispute. State-of-The-Art and Possible Extensions. IEEE
When we pass the mouse pointer over a rectangle Transactions On Knowledge And Data Engineering, 17,
representing a SOI, a tooltip presents its exact beginning 6 (2005), 734–749.
and end, in seconds.
2. Brabham, D. Crowdsourcing as a Model for Problem
It was another example that shows why SOI can be seen as Solving: An Introduction and Cases, In Convergence:
a source of collective intelligence. Information and The International Journal of Research into New Media
knowledge that emerged from a community that had Technologies 14, 1(2008), 75–90.
marked SOI can be used on consensus decision-making and
to bring improvements to society. For instance, using 3. Chakoo, N., Gupta, R., Hiremath, J. Towards Better
information and knowledge originated from SOI clusters a Content Visibility in Video Recommender Systems, In
video on demand provider can improve its recommender Proc. IEEE FCST, 2008.
engine, or a TV channel can discover groups, segment 4. Chaves, A. P., Steinmacher, I., Vieira, V. Social
viewers and send personalized ads. Networks and Collective Intelligence Applied to Public
12
Transportation Systems: A Survey. In Proc. SBSC 12. Levy, P., Bonommo, R. Collective Intelligence:
2011. Mankind’s Emerging World in Cyberspace. Perseus
5. Chorianopoulos, K. Collective intelligence within web Books, USA, MA, 1999.
video. Human-centric Computing and Information 13. Li, F. C., Gupta, A., Sanocki, E., He, L., Rui, Y.
Sciences, 3,10 (2013). Browsing digital video. In Proc. CHI 2000. ACM
6. D’Enza, A. I. Interval Data Visualization - An advanced (2000), 169-176.
course on Knowledge Extraction by Interval Data 14. Nguyen, N. T., Rakowski, M., Rusin, M., Sobecki, J.,
Analysis. http://www.novauniversitas.it/System/2458/ Jain, L. C. Hybrid filtering methods applied in web-
Alfonso%20Iodice.pdf based movie recommendation system. In Proc. KES
7. Dias, A., Wives, L., Roesler, V. Enhancing the 2007/ WIRN 2007, Springer-Verlag Press (2007), 206-
Accuracy of Ratings Predictions of Video 213.
Recommender System by Segments of Interest, In Proc. 15. Pasquinelli, M. Google’s PageRank algorithm: A
WebMedia 2013 (2013) (to appear) diagram of cognitive capitalism and the rentier of the
8. Doan, A; Ramarkrishnan, R; Halevy, A. Crowdsourcing common intellect. Deep Search (2009), 152-162.
Systems on the World Wide Web, Communications of 16. Ricci, F., Rokach, L., Shapira, B., Kantor, P. B.
the ACM 54, 4 (2011), 86–96. Recommender Systems Handbook. Springer Press. NY,
9. J. Howe. The rise of crowdsourcing. Wired Magazine, USA, 2011.
14, 6 (2006). 17. Sarwar, B., Konstan, J., Borchers, A., Herlocker, J.,
10. Jannach, D., Zanker, M., Felfernig, A., Friedrich, G. Miller, B., Riedl, J. Using Filtering Agents to Improve
Recommender Systems: An Introduction. Cambridge Prediction Quality in the GroupLens Research
University Press, NY, USA, 2011. Collaborative Filtering System. In Proc. CSCW 1998
(1998), 1–10.
11. Kittur, A, Chi, E.H., Sun, B. Crowdsourcing user
studies with Mechanical Turk, In Proc. CHI 2008, ACM 18. Shaw, R., Schmitz, P. Community annotation and
Press (2008), 453-456. remix: a research platform and pilot deployment, In
Proc. HCM 2006. ACM (2006), 89-98.
13