The search and the browsing of picture collections from sharing platforms requires automatic processing of both content and metadata that are provided by users or owners. Social event detection consists in finding, in a large collection of photos, the ones that are related to a specific event. Our system performs two steps that consist first in extracting all the pictures related to the event category (i.e. soccer, Barcelona and Roma), and then to select pictures related to specific events. In the following, the category extraction step is named subset extraction, the by-event clustering being named subset clustering.

We tested a method based on a semantic representation of pictures by LDA, that is compared to a simple clustering method. These 2 methods are described respectivelly in the Section 2 and 3 of this paper.

This method relies on an intermediate representation of pictures in a semantic space obtained by LDA. In order to estimate the LDA model, we collect text materials from the Web by using queries related to the event category, represented by a set of keywords. 2.1

The proposed extraction method relies first on creating a corpus from the web. This corpus is obtained by querying google with the keywords of the challenge query. When we collect this corpus, we create a query-dependent feature vector to evaluate picture/event category similarites. Then, we select the nearest pictures according to a fixed threshold.

! Query representation The event category is represented by a feature vector obtained by analysing the related web pages. Web pages are collected by sending query 1 to google and to select the 100 best documents. The query depedent feature vector v is composed by the relative frequency (p(wjv)) of each words w of the corpus, divided by the average position of the first occurence of the word positionw in the returned documents. A stop-list based filtering process takes off the meaningless words. The last feature of the vector is the number of seconds since the first january 1970 until the picture taken date. ! Distance between pictures and query We want to select the pictures that are related to the query. This is achieved by calculating the distance between the picture and the feature vector by:

Dist(pictk; v) =

v(w) ; v(w) =

X w2pictk

T Fw positionw (1)

We create a subset with pictures those distance to the features vector exceeds a fixed threshold. 2.2

We have to cluster this subset into parts that are supposed to be related to social events belonging to the same category. This clustering step is achieved into the LDA space.

! Semantic Space with LDA

For each challenge, we just have the words in the query to find the events on the pictures set. Here, we locate the query in a topic space estimated by LDA. The 50-topic LDA model is estimated on the dataset obtained for the Subset Extraction step.

! Vector of distance We calculate, for each picture, a vector of distance with all topics. We add to this vector another feature: the number of seconds between the date of the picture (dateTaken field) and the 01 jan. 1970. 1C1:soccer barcelona rome C2:may 2009 parc del forum barcelona paradiso amsterdam

We use the prior probability p(tj jC) of a topic tj in the corpus C to weight the distance of a picture with a (un)relevant topic. ! Clustering Selected picture set is clustered by using the Expectation-Maximisation, gaussian-based clustering algorithm [ 4 ].

In this run, we use only the information of pictures and the query for the two challenges. These information are constituted by all textual metadata available. The global processing scheme is similar to the one we used for LDA based approach: a first step select a subset of relevant pictures that are clustered in a second step.

Here, similarities are only based on text-level comparison, the query being represented by it’s keywords and the pictures being represented by title, description and taggs (if available). Section 3.1 presents the method to extract a subset of pictures related to the query and Section 3.2 describes the clustering method. 3.1

This first selection step relies on an estimation of proximity of a picture to the targeted query. We count the number of occurences of each word of the query in the picture text materials. A specific weighting is applied according to the field in which a word occurs. If the word appears in the title, the weight is 1:0, in the description, 0:75 and 0:25 for a tag.

For each element of the picture, we calculate the f-score[ 5 ] between words in query and words in pictures features. If the f-score of a section exceeds a threshold (0:75 seems to give the best result), we apply another boosting rule by multiplying the score of the element by 100. We add to the subset the pictures with a score over 40% of the highest score. 3.2 We determine the similarity between each pair of pictures: Sim(pictj ; pictk) =

Nj;k Nj + Nk Where j 6= k and Nj is the total number of words in pictj . Nj;k is the number of words that belongs to a element of pictj AND a element of pictk. The system puts each picture with the cluster that contains the picture of highest similarity.

For this task, we used 73269 pictures from Flickr [ 1 ]. Each picture is associated to a title, a description, owner nickname and tags.

! Results We present in the Table 1 the results in each challenge where E is the number of events detected for this challenge, PA is the number of pictures accepted for the challenge, PR the number of rejected pictures for the challenge and % show the percentage of accepted pictures. In Table 2 we present our evaluation (as evaluated by MediaEval Benchmark organisers). Measures are Normalized Mutual Information (NMI) and F-Score.

We proposed two methods to cluster a set of pictures from Flickr. In the first run, we use LDA and the Web pages to cluster pictures with topics. We also propose a second method that use the query of the challenge to estimate picture/cluster similarities. Evaluation probably presents technical problem that remains to be clearly understood.

Nevertheless, the results show that high level approach such representation in a semantic space doesn’t perform well, probably due to it’s complexity and the various possiblity of adding noise at different level of the processing chain (in data collecting, topic modeling, document representation in the topic space, etc.). 6.

E 10 3