The microblogs search task at CLEF 2017 consists of developing a system to search the most relevant microblogs for cultural query in a collection about festivals in all languages. Our general approach to get this objective is the following: we propose to generate from the initial tweet queries, provided for the task, extended queries able to get an answer-rich set of microblogs. This is achieved using a thematic representation of tweet query extracted from microblog corpus. We investigate in this paper a novel method to improve topics learned from Twitter content without modifying the basic machinery of LDA. This latter is based on Information Retrieval (IR) process to generate a query-speci c set of similar tweets. The result then represent the input of a basic LDA topic modeling process. Finally, the output thematic cluster serves as our source of expansion for the initial queries.
The microblog search is the second task 1 at CLEF 2017 [
We consider that using topic models such as Latent Dirichlet Allocation (LDA)
could be useful in this microblog search task. Taking into account that \topic
model is often employed to mine \latent topics" from high dimensionality of
terms in text"[
The contribution of this work is to identifying the \topic" or topics being discussed in a query, and then using this knowledge of topics to include semantically related words. To better match with ambiguous nature of tweet query, topics are extracted from a microblog corpus. A novel method to improve topics learned from Twitter content without modifying the basic machinery of LDA is investigated. Following this introduction, the paper is organized as follows. In Section 2, a state of the art is shown. In Section 3, the methodology used is presented. The Conclusion section wraps up the paper. 2 2.1
Topic models are used to uncover the latent semantic structure from text
corpus. A topic consists of a cluster of words that frequently occur together. Using
contextual clues, topic models can connect words with similar meanings and
distinguish between uses of words with multiple meanings. Traditional topic models,
like LDA, rely on co-occurrence patterns of words in documents to learn latent
topics [
There are two obvious approaches to including topic models in IR. In the rst,
a document is represented by itself and the topics to which it belongs. A second
approach is to calculate a query related topic by using topic models and use it for
query expansion. In this case, queries are reformulated (i.e. usually expanded)
to improve the retrieval e ectiveness. Authors in [
With the rapid development of microblogs, microblog search has become one
of the most trendy research areas in recent years. In contrast to traditional
text retrieval, microblog search signi cantly di ers. In fact, microblogs users
often issue short queries to nd relevant information. Moreover, the restriction
of messages length lead to a problem in discriminating terms within a given
item. To improve retrieval e ectiveness in microblogs researches tend to use
query expansion techniques. The goal is reducing the usual query/document
mismatch. In this area, using topic model like, LDA, could be useful. However,
unlike regular text, Topic modeling is not good at processing short text. Rare
are works which try to enhance microblog search using topic modeling such as
LDA. We cite [
To build a robust LDA model, a large amount of data is needed. From this
perspective, two tweet corpora are used in di erent runs. We note that we have
choose to use explicitly microblog corpus to respect the noisy nature of tweet
query. The rt corpus is a comparable tweet corpus about Arab spring collected
through Twitter's API3 in Arabic and French languages. Basically, accessing
Twitter data is done by collecting tweets that contain speci c keywords. More
information about this corpus could be found here [
In this work, an unsupervised topic model based on aggregating tweets that are
thematically closed is presented. The goal is to adapt the LDA basic process to
short tweet text. This will lead to improve the quality of topics latent
discovered. To perform tweet pooling, we propose to use information retrieval strategy
and hierarchical classi cation in order to avoid data sparsity in short texts as
illustrated in Figure 1. Our approach represents an alternative of state of the
art methods based on tweet pooling via meta data (hashtags, user information,
etc). Indeed, such methods are highly dependent on the meta data content of
the tweet corpus. Our approach relies on three main steps, namely:
{ Step 1: Preliminary set generation: For each tweeti, we propose to
retrieve a set i of matched tweets out of a large tweet collection C of n tweets,
using an information retrieval system. Thus, for a tweet tweeti, performed as
a query, several tweets in C may match it with di erent similarity degrees.
{ Step 2: Pooled set construction: The idea is to aggregate a set of tweets
in partitions by gathering preliminary sets, resulting from the IR process,
according to a combination criterion. If same search results are assigned to
di erent tweets then the tweets are considered thematically close.
{ Step 3: Topic extraction: It consists on learning latent topics from
aggregating tweets via LDA.
{ Extract Topics using the combined method of IR and aggregation strategy
as described above section 3.2
{ Project the resulted Topic on the tweet text to detect the subset of thematic
relevant terms
{ Reformulate the initial query using the subset of thematic terms as enriched
features in form of indri query
{ RUN 1: Extend the initial query using the latent topic extracted from
comparable tweet corpus[
A large multilingual collection of posts have become publicly available due to the phenomenal growth of using social networks and microblogs all over the world. This makes from Microblogs valuable information sources. However little is known about how search socially-generated content in e ective way. In this paper, we present a method to expand short messages using a thematic representation. A novel method for aggregating tweets in order to improve the quality of LDA-based topic modeling for short text is used to improve the quality of latent topics.