This demo enables the automatic creation of semantically annotated YouTube media fragments. A video is rst ingested in the Synote system and a new method enables to retrieve its associated subtitles or closed captions. Next, NERD is used to extract named entities from the transcripts which are then temporally aligned with the video. The entities are disambiguated in the LOD cloud and a user interface enables to browse through the entities detected in a video or get more information. We evaluated our application with 60 videos from 3 YouTube channels.
New W3C standards such as HTML5, Media Fragment URI and the Ontology for Media Resources have nally made videos a rst class citizen on the Web. Indexing a video at a ne grained level such as the scene is, however, not yet a common practice on popular video sharing platform. In this demo, we propose to use NERD for extracting named entities from timed text associated to videos in order to generate media fragments annotated with resources from the LOD cloud. Our contributions include a new combined strategy for extracting named entities, temporal alignment of the named entities with the video and a user interface for browsing the enriched videos.
The LEMO multimedia annotation framework provides a uni ed model to
annotate media fragments while the annotations are enriched with contextually
relevant information from the LOD cloud [
This demo is powered by the integration and extension of two systems: Synote [
A live demo can be found at http://linkeddata.synote.org7. A user rst logged in on Synote. When going to the recording creation page, a user can start the ingestion of a YouTube video. The recording is then available in the recording list. The \NERD Subtitle" button enables to launch the named extraction process. When completed, a \Preview Named Entities" button enables to go to the player page where named entities can be used to seek in particular video fragments.
Figure 1b shows the screenshot of a preview page. The right column displays the named entities found grouped according to the 8 main NERD categories. The YouTube video is included in the left column together with the interactive subtitles. The named entities are highlighted in di erent colours according to their categories. If a media fragment is used in the preview page URI, the video starts playing from the media fragment start time and stops playing when the end time is reached. When clicking on a named entity, the video jumps to the media fragment that corresponds to the subtitle block where the named entity has been extracted. If a named entity has been disambiguated with a DBpedia URI, the entity is underlined. In addition, when the entity is hover, a pop-up window shows additional information such as the generic label, abstract and depiction properties. For named entities of type Person, the birth date is displayed while latitude and longitude information are given for Location. 4
We ltered the videos which have subtitles for 3 di erent channels: People and Blogs, Sports and Science and Technology and collected 60 videos in total (the top 20 for each category). Videos have di erent duration ranging from from 32 to 4505 seconds and di erent popularity ranging from 18 to 2,836,535 views (on July 30th, 2012). The corpora is available at http://goo.gl/YhchP and can be visually explored in Synote at http://goo.gl/XmMqp after being logged in with the iswc2012 account. The video #16 is the only one discarded because its subtitles are written in Romanian. The evaluation consisted in two steps: i) be able to get all subtitles and ii) perform entity recognition using NERD. We combined all extractors supported by NERD and we aligned the classi cation results to 8 main types (Event is only supported by OpenCalais in beta) plus the general type Thing used as fallback in the case NERD cannot nd a more speci c type. We de ne the following variables: number of documents per category nd; total number of words nw; number of words per document ratio rw; total number of entities ne; number of entities per document re (Table 1). We observe that Science and Technology videos tend to be more about people and organizations while Sports videos mention more often locations, time and amount. People and Blogs videos have less useful information although it is interesting to see that this type of video can be used to train event detection. 5
This demo paper presents a system that creates media fragments from YouTube videos and annotates their subtitles using NERD. The process includes named entities extraction in timed-text documents. Those entities annotate and enrich media fragments with 7 As credentials, please insert for both user and password: \iswc2012". pointers to the LOD cloud. We provide a lightweight evaluation of the system in order to show that we are e ectively able to retrieve the subtitles of YouTube videos and to run named entities extractions. Although a more thorough analysis will be needed, we already show that videos exhibit a very di erent behavior in terms of named entities depending on their genre.
nd nw rw ne re
People and Blogs Sports Science and Technology
19 20 20 7,187 21,944 39,661 378.26 1,097.20 1,983.05 610 897 1,303 32.11 44.85 65.15
The research leading to this paper was partially supported by the French National Agency under contracts ANR.11.EITS.006.01, \Open Innovation Platform for Semantic Media" (OpenSEM) and the European Union's 7th Framework Programme via the projects LinkedTV (GA 287911).