This paper presents a system that detects and enables the exploration of relevant fragments (called Hot Spots) inside educational online videos. Our approach combines visual analysis techniques and background knowledge from the web of data in order to quickly get an overview about the video content and therefore promote media consumption at the fragment level. First, we perform a chapter segmentation by combining visual features and semantic units (paragraphs) available in transcripts. Second, we semantically annotate those segments via Named Entity Extraction and topic detection. We then identify consecutive segments talking about similar topics and entities that we merge into bigger and semantic independent media units. Finally, we rank those segments and lter out the lowest scored candidates, in order to propose a summary that illustrates the Hot Spots in a dedicated media player. An online demo is available at http://linkedtv.eurecom.fr/ mediafragmentplayer.
Nowadays, people consume all kind of audiovisual content on a daily basis. From
breaking news to satiric videos, personal recordings or cooking tutorials, we are
constantly feed by video content to watch. A common practice by viewers
consists in fast browsing through the video, using sometimes the key frames provided
by the video sharing platform, with the risk of missing the essence of the video.
This phenomena is even more obvious when it comes to educational web
content. A study made over media entertainment streaming services reveals that
the majority of partial content views (52.55%) are ended by the user within the
rst 10 minutes, and about 37% of these sessions do not last past the rst ve
minutes [
The challenge of video segmentation has been addressed by numerous
previous research. Some of them rely exclusively on low-level visual features such
as color histograms or visual concept detection clustering operations [
Generating and Exploring Hot Spots in Web Videos This demo implements a multimodal algorithm for detecting and annotating the key fragments of a video in order to propose a quick overview about what are the main topics being discussed. We conduct an experiment over a corpora of 1681 TED talks 1, a global set of conferences owned by the private non-pro t Sapling Foundation under the slogan: "Ideas Worth Spreading" 2.1
First, we perform shot segmentation for each video using the algorithm described
in [
We rely on the subtitles available for the 1681 TED talks for annotating the
media fragments which have been generated. More precisely, we detect topics
and named entities. For the former, we have used the dedicated TextRazor topic
detection method2, while for the latter, we used the NERD framework [
Once all chapters are delimited and annotated, we iteratively cluster them, in particular, when temporally close segments are similar enough in terms of topics named entities. More precisely, we compute a similarity function between 1 http://www.ted.com/ 2 https://www.textrazor.com/documentation consecutive pairs of segments S1 and S2 until no new merges are possible. This comparison leverages on the annotations attached to each segment by analyzing the number of coincidences between topics T = max3 nPtopici Relio and entities E = max5W 0s nPentityi Relio, where Reli is the TexRazor's relevance: d (S1; S2) = wtopic
jT1 T T2j max fjT1j ; jT2jg + wentity
jE1 T E2j max fjE1j ; jE2jg (1)
After this clustering process, the video is decomposed into less but longer chapters. However, there are still too many candidates to be proposed as Hot Spots. Therefore, we lter out those fragments which contain potentially less interesting topics. We de ne a function for measuring the interestingness of a video segment, which directly depends on the relevance and frequency of the annotations and which is inversely proportional to its length. In our current approach, the Hot Spots are those fragments whose relative relevance falls under the rst quarter of the nal score distribution.
In a last step, for each Hot Spot, we also generate a summarization to be shown in a dedicated media player where we highlight the main topics T and entities E which have been discovered. 2.4
The Hot Spots and their summaries are visualized in a user friendly Media Fragment URI compliant media player. The procedure to get the Hot Spots for a particular Ted talk is the following: the user enters a valid TED Talk URL to get a landing page (Figure 1a). When the results are available, the hot spots are highlighted on the timeline together with the label of the most relevant chapter annotation (Figure 1b). This label can be extended to a broader set of entities and topics (Figure 1c). Finally, the user can always share those hot spots segments using media fragment URIs (Figure 1d). 3
Discussion We have presented a demo for automatically discovering Hot Spots in online and educational videos. We leverage on visual analysis and background knowledge available in the web of data for detecting what fragments illustrate the best the main topics discussed in the video. Those Hot Spots allow the viewer to quickly decide if a video is worth watching and will provide incentive for consuming videos at the fragment level. In addition, Hot Spots can be explored in a dedicated media fragment player which also display the attached semantic annotations.
We plan to carry out an exhaustive evaluation of our approach involving real users feedback, in order to optimize the results of our Hot Spot detection algorithm and to improve the usability and e ciency of the developed interface. We also plan to further exploit the segmentation results and their corresponding annotations for establishing links between fragments belonging to di erent videos in order to generate true hyperlinks within a closed collection such as TED talks and make results available following Linked Data principles.