In this paper we describe our work for the MC2 CLEF 2017 lab. We participated in the content analysis task that involves filtering, language recognition and summarization. We combine Information Retrieval with Multi-Sentence Compression methods to contextualize microblogs using Wikipedia's pages.
Many newspapers use microblogs (Twitter, Facebook, Instagram, etc.) to disseminate news quickly. These microblogs have a limited length (e.g. a tweet is limited to 140 characters) and contain few information about an event. Therefore, it is complicated to describe an event completely in a single microblog. A way to overcome this problem is to get more information from another source to better explain the microblog.
Several studies on tweet contextualization have been done on this topic. To just name a few, Liu et al. introduced a graph-based multi-tweet summarization system [8]. This graph integrates the functionalities of social networks, solving partially the lack of information contained in tweets. Chakrabarti and Punera used a Hidden Markov Model in order to model the temporal events of sets of tweets [4]. Linhares Pontes et al. used Word Embedding to reduce the vocabulary size and to improve the results of Automatic Text Summarization (ATS) systems [7]. MC2 CLEF 2017 [1] lab analyzes the context and the social impact of of a microblog at large. This lab is composed of three main tasks: Content Analysis, Microblog Search and Time Line Illustration. We participated in the Content Analysis task that involves classification, filtering, language recognition, localization, entity extraction, linking open data, and summarization of Wikipedia's pages and microblogs. Specifically, we worked on the following subtasks: filtering, language recognition and automatic summarization.
The filtering subtask analyzes whether a tweet describes an existing festival or not (values are between 0 and 1, 1 for the positive case and 0 otherwise). The language recognition subtask consists in identifying the language of a microblog. Finally, the summarization task is to generate a summary (maximum of 120 words) in four languages (English, French, Portuguese and Spanish) of Wikipedia's pages describing a microblog.
This paper is organized as follows. In Section 2 we describe the architecture of our system to solve the tasks of MC2 CLEF 2017 lab. Then, we present the process of document retrieval on Wikipedia and the summarization system in Sections 3 and 4, respectively. Finally, we conclude in Section 5.
The CLEF's organizers selected a set of microblogs (tweets) with the keyword "festival" to be contextualized by the participants using four versions of Wikipedia (English, French, Portuguese, and Spanish).
For the language identification task, we pre-processed microblogs to remove all punctuation and emoticons. Then, we use the library langdetect [9] to detect the languages of microblogs.
For filtering and summarization tasks, we divided our system in two parts (Fig. 1). The first part aims at retrieving the Wikipedia's pages that best describes the festival mentioned in a microblog (Section 3). We scored the Wikipedia's pages according to their relevance with respect to a microblog, which corresponds to the filtering subtask.
The second part of our system analyzes the 3 best scored pages and creates clusters of similar sentences with relevant information. Then, we use an Automatic Text Compression (ATC) system (Section 4) to compress the clusters and to generate summaries in four languages describing the festival mentioned in a microblog (summarization subtask). The algorithm 1 describes how our method analyzes the microblog, selects the 3 best Wikipedia's pages and generates the summaries.
The set of CLEF's microblogs is composed of tweets in different languages related to festivals in all the world. Wikipedia provides a more thorough description of a given festival according to the selected language (e.g. The festival of Avignon is better described in the French Wikipedia). We independently analyze the four versions of Wikipedia (en, es, fr, and pt) for each microblog, repeating the whole process to first retrieve the best Wikipedia's pages and then to summarize the pages for the four versions of Wikipedia.
Our system retrieves the most related Wikipedia's pages to a microblog using a method similar to our previous work [7]. We assume that the hashtags and usernames represent the keywords of a tweet, and are independent of the language 4 . From hashtags, usernames, and the plain text (i.e. the tweet without hashtags, usernames and punctuation), we create Indry queries to retrieve 50 Wikipedia's documents per each tweet. For each of these documents, we analyze the title and the summary in relation to the tweet's elements (hashtag, username and word). Normally, the title of the Wikipedia's document has few words and contains the core information, while the summary of the document, which is made of the first paragraphs of the article before the start of the first section, is larger and provide more information 5 . Therefore, we consider Equation 4to compute the relevance score of the Wikipedia's document D with respect to the microblog T .
score doc = score title + score summary (4) where ht are the hashtags of the tweet T , un the usernames of T , nw the normal words of T , and sum the summary of D. occur(x, y) represents the number of occurrences of x in y, while cosine(x, y) is the cosine similarity between x and y using Continuous Space Vectors6 [3]. We set up empirically the parameters as follows: α 1 = α 2 = 0.1, α 3 = 0.01, β 1 = β 2 = 0.05, β 3 = 0.005, γ 1 = 1 and γ 2 = 0.5 . These coefficients give more weights to hashtags than usernames and the tweet text and compensate the shorter length of Wikipedia's article titles with respect to their summary. For each tweet, we finally keep in each language the 3 Wikipedia's documents with the highest scores to be analyzed by the ATC system.
The summary provided at the start of Wikipedia's pages is assumed to be good enough to be coherent and to provide the basic information. However, relying only of this part of the article may lead to miss relevant information about the festival that could be obtained from other sections or even pages in Wikipedia. For this reason, we preferred to use the summary of the top article as a basic abstract and to improve its quality with relevant information using Multi-Sentences Compression (MSC) (i.e. generate sentences that are shorter and more informative than the original sentences of the summary). Therefore, we consider the sentences of the summary of the best scored page as key sentences. Then for each of these sentences, we create a cluster made of the sentences of the complete 3 retrieved Wikipedia's pages which are similar; to do this, the cosine similarity is used as metrics and we empirically set up a threshold of 0.4 to consider two sentences as similar.
Then, for each cluster MSC generates a shorter and hopefully more informative compression (Section 4.1). Next, we generate the summary concatenating the most similar compression to the microblog 7 .
Some language versions of Wikipedia do not have a page or they have a small description describing an specific festival. Therefore, we analyzed the summaries of each microblog obtained in the four studied languages and only retain the abstract with contains the best description of the microblog, which is estimated through the similarity between each summary and the microblog. So, we used the Yandex library8 to translate the kept summary to others languages (en, es, fr, and pt).
Our MSC system adopts the approach proposed by Filippova [5] to model a document D as a Word Graph (WG), where the vertices represent the words and arcs represent the cohesion of the words (more details in [6]). The weights of the arcs represent the level of cohesion between the words of two vertices based on the frequency and the position of these words in the sentences (Equation 5).
w(e i,j ) = cohesion(e i,j ) f req(i) × f req(j) ,
cohesion(e i,j ) = f req(i) + f req(j)
In a previous study, we proposed to extend this approach with the analysis of the keywords and the 3-grams of the document to generate a more informative MSC. Since each cluster to compress is composed of similar sentences, we consider that there is only one topic; the Latent Dirichlet Allocation (LDA) method is used to identify the keywords of this topic [2].
From the weight of 2-grams (Equation 5), the relevance of a 3-gram is based on the relevance of the two 2-grams, as described in Equation 8:
In order to generate a better compression, the objective function expressed in Equation 9 is minimized in order to improve the informativeness and the grammaticality.
where x ij indicates the existence of the arc (i, j) in the solution, w(i, j) is the cohesion of the words i and j (Equation 5), z t indicates the existence of the 3-gram t in the solution, d t is the relevance of the 3-gram t (Equation 8), c k indicates the existence of a word with color (keyword) k in the solution and β is the geometric average of the arc weights in the graph (more details in [6]). Finally, they calculate the 50 best solutions according to the objective (9) and we select the sentence with the lowest final score (Equation 10) as the best compression.
where score opt (f ) is the value of the path to generate the compression f from Equation 9. As Linhares Pontes et al. [6], we set up the parameters to α = 1.0, β = 0.9 and γ = 0.1.
In this paper, we presented our contributions to the MC2 CLEF 2017 lab in the Content Analysis task. We considered different scores for each microblog element (hashtags, arrobases, and text) to retrieve in four languages (en, es, fr, and pt) the Wikipedia's pages most related to a microblog. Then, we generated summaries using MSC from clusters initially made of the abstract of the top retrieved article and extended with similar sentences from the 3 top retrieved articles per language. Finally, we analyzed summaries of each microblog obtained in the four languages to select the one most similar to the microblog; the kept summary is translated to other languages (en, es, fr and pt).