INTRODUCTION

The success of a product or a service essentially depends of the way to present it. Thus, companies pay much attention to choose the most appropriate advertisement that will make a difference in the customer choice. The advertisers have different media possibilities, such as journal paper, radio, TV or Internet. In this context, they can exploit the audio media using a song related to the commercial which attracts listeners. Therefore, the choice of an appropriate song is crucial and can determine the success of a product [5,2].

For these reasons, the MediaEval 2013 Soundtrack task for commercials becomes a challenging and helpful task [3]. Indeed, the MusiClef task seeks to make this process automated by taking into account both context-and contentbased information about the video, the brand, and the music. The main difficulty of this task is to find the set of relevant features that best describes the most appropriate song for a video. We propose a hybrid approach that uses a set of features from textual and audio media.

PROPOSED APPROACH

The proposed hybrid system is composed of two processes. The first one projects a TV commercial into a topic space to find a set of other commercials sharing close topics. A TV commercial from the test set is thus linked to the TV commercial from the development set sharing the closest topics.

As a result, each TV commercial from the test set will be associated with a song extracted from the development data.

The second step has the responsibility to find, using audio features, the most similar songs to the one associated during the first step from a list of candidate songs (see figure 1). In details, the development set D is composed of TV commercials C d , with for each, a soundtrack S d and a vector representation V d related to the d th TV commercial. In the same manner, the test set T is composed of TV commercials C t , with, for the t th one, a vector representation V t and a soundtrack S t to predict. Then a similarity score {α d,t } t=1,...,T d=1,...,D is computed for each commercial C d i of the development set given one from the test set C t :

D = {C d , V D , S d } d=1,...,D(1)

T = {C t , V T , S t k } k=1,...,5000 t=1,...,T .

In the next sections, the topic space representation and the mapping of a commercial in this topic representation are described. Then, the computed similarity score is detailed. Finally, the soundtrack prediction process from a TV commercial is explained.

Topic representation of a TV Commercial

. . . Vd[1]

Vd [2] Vd [3] Vd size N . This corpus contains 10, 724 Web pages related to brands of the commercials contained in D. This corpus is composed of 44, 229, 747 words for a vocabulary of 4, 476, 153 unique words. The topic representation is performed using a Latent Dirichlet Allocation (LDA) [1] approach. At the final LDA analysis, a topic space m of n topics is obtained with, for each theme z, the probability of each word w of v knowing z and for the entire model m, the probability of each theme z knowing the model m. Each TV commercial from both development and test set is mapped into the topic space (see figure 2).

[4] Vd[n]

Similarity measure

Each commercial have been mapped into the topic space to produce its vector representation. Then, commercials from the test set T that deal with the same subjects of commercials from the development set D are clustered. The cosine is used as a similarity measure:

cosine(V d , V t ) = α d,t = n i=1 V d [i] × V t [i] n i=1 V d [i] 2 n i=1 V t [i] 2 (2)

Rhythm pattern

The cosine measure, presented in the previous section, is also used to evaluate the similarity between a mean rhythm pattern vector S t of a song and all the candidate songs S t k of the test set.

In details, each commercial from D, is related with a soundtrack that is represented with a rhythm pattern vector. In our experiments, the 10 rhythm features of the song are used (speed, percussion, periodicity, rhythm pattern. . . ). As a result, each commercial is represented by a rhythm pattern vector of size 58. From the subset of soundtracks of the l nearest commercials from D, a mean rhythm vector S is performed as:

S = 1 l d∈l S d .

Finally, the cosine measure between this mean rhythm S of the l nearest commercials from D and each commercial (cosine(S, S t )t∈T ) is used to find, from the soundtrack S t of the test set T , the 5 songs from all the candidates having the closest rhythm pattern.

EXPERIMENTS AND RESULTS

The proposed system is evaluated in the MediaEval 2013 MusiClef benchmark [4]. The aim of this task is to predict for each video in the test set, the most suitable soundtrack from 5,000 candidate songs. The dataset is split into 3 sets. The development set contains multimodal information on 392 commercials (various metadata, Youtube uploader comments, various audio features, video features, web pages and text features). The test set is a set of 55 videos where a song should be associated using the recommandation set of 5,000 soundtracks (30 seconds long excerpts).

For each video in the test set, a ranked list of 5 candidate songs is proposed. The song prediction evaluation is manually performed using the Amazon Mechanical Turk platform. Three scores have been computed from our system output [4]:

• First rank average score: 2.16

• Top 5 average score (arithmetic mean): 2.24

• Top 5 average score (harmonic mean, taking rank into account): 2.22

Considering that human judges rate the predicted songs from 1 (very poor) to 4 (very well), we can consider that our system is slightly better than the mean evaluation score (2) no matter the metric considered.

CONCLUSION

In this paper, an automatic system to assign a soundtrack to a TV commercial has been proposed. This system combines two media: textual commercial content and audio rhythm pattern.