INTRODUCTION

The Social Event Detection task at MediaEval 2011 aims at detecting social events that occurred during May 2009 from a dataset composed of images shared on Flickr [2]. The strategy we investigate is to find the event instances that occurred during this period of time and then try to match these event instances with photos from the Flickr dataset. We also study how to employ the visual features and "owner" metadata from the photos to improve the performance. We first detail our approach (Section 2) before presenting and discussing our results (Section 3). Finally, we conclude the paper in Section 4.

APPROACH DESCRIPTION

The challenge of the social event detection task is to find the photo clusters that are relevant to events held on a given location during a particular period of time. We tackle this problem in two steps: first, we attempt to retrieve all of the events that occurred at a given place and time; second, we use the extracted information about these events and attempt to match them to the photos metadata in the dataset. All of the photos that are matched to the same event can be grouped in one cluster. Besides these two main steps, we also improve the detection results with visual feature and "owner" metadata.

Prior knowledge acquisition

We known that it is easier and more accurate for the computer to identify specific pattern compared with abstract concept. To find concert or soccer events that may be hidden in the dataset, we first look for all instances of these two kinds of events held in a given place and time.

Soccer games and concerts are types of favorite activities in people's daily life and one can find substantial infor-

Copyright is held by the author/owner(s).

MediaEval 2010 Workshop, September 1-2, 2011, Pisa, Italy mation online about such scheduled events. For example, FBLeague1 provides the official football games that registered in FIFA2 and UEFA3 . From this web site, we obtained 461 football games that occurred in May 2009, among which 6 took place in Roma and Barcelona. These 6 soccer events are our prior knowledge for the challenge 1.

For challenge 2, we extract concerts information from event directories such as Last.fm4 , Eventful5 , and Upcoming6 . After manual check, only Last.fm contains descriptions of events held on the given conditions. Last.fm is a popular music web site that records concert events held in more than 190 countries. In addition, Last.fm provides an API for the developer to build their algorithm based on its data. Using its public API, we found 68 events that took place in the Paradiso and 3 events in Parc del Forum in May 2009.

Event Identification Model

With the prior knowledge of scheduled events description, the event detection task changes to a matching problem where a model can be used to measure the relationship between events and photos. Here, we consider events as something happening in some place during sometime. Therefore, the title, time and location are three key factors that identify an event. The corresponding photo metadata are text description, taken time and place. Since these three factors are independent, we can measure the probability of a given photo P to be relevant to an event E by p(P |E) = p(P.text|E.title)p(P.time|E.time)p(P.geo|E.geo) (1) where: The first item measures the similarity of a photo text description with an event title. Since both of them are short and sparse, the most straightforward way to measure them is:

p(T ext1|T ext2) = |T ext1 ∩ T ext2| |T ext2|(2)

Where the function | • | is the total number of words in a text vector.

The second item in Equation 1 measures the difference between photo taken time and event held time. Here, we measure the difference using the Dirac function.

p(T ime1|T ime2) = δ( date(T ime2 − T ime1) N )(3)

Where the function date(•) calculates the number of days for a time span, δ is the Dirac delta function that takes the value 1 when and only when the input parameter is zero, and N is used for scaling (its value will be discussed in the Section 3). The third item in Equation 1 measures the distance between photo geo tags and event location. The best distance measure to use seems the L2 distance between the two locations. However, an important amount of photos do not have geo tags and when provided, GPS data in the Flickr dataset can be inaccurate. Consequently, we just use the city/venue name to measure the location feature and we use the textual metric formalized in the Equation 2.

This method finds many photos with a clear description and association to events. However, text-based matching brings also noise and it can not deal with photos without any text description. We employ visual features to remove the noisy photos and "owner" metadata to find out relevant photos without text description.

Visual Pruning

Visual pruning is employed to remove the noisy photos from the results of the Event Identification Model [1]. We assume that the photos that are corresponding to the same event should be similar visually. The method used here is quite straightforward. Given a set of the photo feature {fi, i ∈ [1, N ]}, the distance between each feature fi and its mean vector m is measure by the L1 distance.

di = sum(|fi − m|)(4)

Photos are then sorted according to the distance di. The bigger the distance and the less similar the photo is with the photo cluster, so we prune the photos with such a large distance. Experimentally, we remove the 5% photos that are far from the center in the visual feature space.

Owner Refinement

Owner refinement is another way to improve the detection results [1]. We assume that a person can not attend more than one event simultaneously. Therefore, all the photos that have been taken by the same owner during the event duration should be assigned to the same cluster. Using this heuristic, it is possible to retrieve photos which do not have any textual description.

EXPERIMENTS AND RESULTS

Based on the proposed approach and the events instances obtained previously, we design our runs as follows:

Challenge 1 :

• run1 The parameter N in Equation 3 is set to 3, and the basic Event Identification Model is run.

• run2 Owner Refinement is performed on the results of run1.

Challenge 2 :

• run1 the parameter N in Equation 3 is set to 1, and the basic Event Identification Model is run.

• run2 Owner Refinement is performed on the results of run1.

• run3 the parameter N in Equation 3 is set to 3 to reduce the impact from erroneous taken time, and the basic Event Identification Model is run.

• run4 Owner Refinement is performed on the results of run3.

• run5 Visual Pruning and Owner Refinement are performed on the results of run3.

A summary of the results is detailed in the Table 1. As For the challenge 2, there are mainly two groups of runs. The first group (run1,run2) used the parameter N=1, and 18 events are found from the 69 events set previous obtained. In the second group (run3, run4, run5), 24 events are found with the parameter N=3. In general, the results for the challenge 1 are just average since only 6 football games were found as prior knowledge and we suppose that several other games have been missed. For the challenge 2, the results are more promising and competitive.

CONCLUSION

In this paper, we propose a framework to detect social events within a media dataset. In our approach, the events instances are retrieved first as prior knowledge, and then, an Event Identification Model is used to measure the similarity of event and photos. In the solution, multi-modality feature such as text, time, visual feature and "owner" metadata are used.