<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Patterns Based Query Expansion for Enhanced Search on Twitter Data</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Meryem Bendella</string-name>
          <email>meryem.bendella@etu.univ-amu.fr</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mohamed Quafafou</string-name>
          <email>mohamed.quafafou@univ-amu.fr</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Aix Marseille Univ, Université de Toulon</institution>
          ,
          <addr-line>CNRS, LIS, Marseille</addr-line>
          ,
          <country country="FR">France</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Aix Marseille Univ, Université de Toulon</institution>
          ,
          <addr-line>CNRS, LIS, Marseille</addr-line>
          ,
          <country country="FR">France</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Social microblogging services have an especially significant role in our society. Twitter is one of the most popular microblogging sites used by people to find relevant information (e.g., breaking news, popular trends, information about people of interest, etc). In this context, retrieving information from such data has recently gained growing attention and opened new challenges. However, the size of such data and queries is usually short and may impact the search result. Query Expansion (QE) has a main task in this issue. In fact, words can have different meanings where only one is used for a given context. In this paper, we propose a QE method by considering the meaning of the context. Thus, we use patterns and Word Embeddings to expand users' queries. We experiment and evaluate the proposed method on the TREC 2011 dataset containing approximately 16 million tweets and 49 queries. Results revealed the effectiveness of the proposed approach and show the interest of combining patterns and word embedding for enhanced microblog retrieval.</p>
      </abstract>
      <kwd-group>
        <kwd>Query expansion</kwd>
        <kwd>Patterns</kwd>
        <kwd>Word Embeddings</kwd>
        <kwd>Microblog retrieval</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Twitter is one of the most popular social media platforms that enable users to
post short texts (up to 280 characters for one text) called tweets. Nowadays,
users can share ideas, opinions, emotions, suggestions, daily stories and events
through this platform. Many of these online services start to be part of daily life
of millions of people around the world. However, a huge quantity of information
is created in these platforms, hence, finding recent and relevant information is
challenging.</p>
      <p>
        There are many users who are interested in collecting recent information
from such platforms. This information can be related to a particular event, a
Copyright c 2019 for this paper by its authors. Copying permitted for private and
academic purposes.
specific topic or popular trends. Users express their need through a query to
search posts (tweets). According to [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ], most people use few search terms and
few modified queries in Web searching. However, query formulation becomes
difficult for the user in order to express appropriately what he is looking for. Query
Expansion (QE) plays a considerable contribution towards fetching relevant
results in this case. An expanded query will contain more related terms (called
candidate terms) to increase the chances of rendering the maximum number of
relevant documents. The objective is to find the microblogs answering to a need
for information specified by a user.
      </p>
      <p>
        In this paper, we propose a new method based on formal concept analysis
and word embeddings to expand user queries. In order to achieve this goal, we
prepare our dataset collection by preprocessing the tweet text which is a
critical step in information retrieval (IR) and Natural Language Processing (NLP).
Then, each tweet is represented as a set of words and will be indexed by the
Terrier system3. Next, we use this system to retrieve tweets according to the
TREC2011 query set [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. We used the BM25 [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] model to retrieve relevant
tweets answering original queries. After that, these retrieved tweets are used to
extract frequent closed patterns. This can be defined as the frequent closed
patterns of words contained in tweets dataset. Furthermore, word embeddings are
trained on our textual dataset by using Word2Vec model [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. Indeed, we
expand original queries by combining patterns and word embeddings approaches.
This combination consists in enriching the query by finding most closely related
words to patterns’ words. The proposed model extends the semantics used in
the original query and improves the results search of microblogs.
      </p>
      <p>The rest of the paper is organized as follows: We cover related work on
query expansion for microblogs retrieval in Section 2. Section 3 describes all
steps required in our proposed approach to expanding queries for microblogs
retrieval. The proposed approach is given in Section 4. The Section 5 is dedicated
to experiments and evaluations. Finally, the conclusion and future work are
presented in Section 6.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Related Work</title>
      <p>Query expansion (QE) has recently gained growing attention in IR domain and
there is considerable research addressing the short query problem. Web queries
posted by users can be too short and that makes the search results not focused
on the topic of interest.</p>
      <p>
        Much effort has been made to improve microblog retrieval. [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] explored use
of three different IR query expansion techniques in order to enhance the search
results. In [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] authors propose a retrieval model for searching microblog posts
for a given topic of interest. This model is based on a combination of quality
indicators and the query expansion model.
3 Terrier is an effective open source search engine (Information Retrieval system),
readily deployable on large-scale collections of documents
      </p>
      <p>
        Query expansion approaches for microblog retrieval can be divided into three
groups which are local, global and external [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]:
– Local: Local QE techniques select candidate expansion terms from a set
of documents retrieved in response to the original (unexpanded) query. This
kind of approach is known as Pseudo-Relevance Feedback (PRF). It is widely
used for query expansion in research of microblog search [
        <xref ref-type="bibr" rid="ref15 ref28 ref6">6,15,28</xref>
        ]. This
approach consists in using terms derived from the top N retrieved documents
(relevant documents) to retrieve other similar documents which are also
likely to be relevant. In [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], authors propose an algorithm of extracting
features from tweets using frequent patterns. Their QE method is based on
PRF approach by applying weights for different features.
– Global: Global QE approaches select the expansion terms from the entire
database of documents. These techniques select candidate terms by mining
term-term relationships from the target corpus [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. This type of techniques
has been used in many applications [
        <xref ref-type="bibr" rid="ref17 ref9">9,17</xref>
        ], and was one of the first techniques
to produce consistent effectiveness improvements through automatic
expansion [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. In our work, we use, in a certain way, the global analysis by training
word embeddings on the entire dataset in order to extract terms that are
most similar to the patterns. In [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ], authors propose an QE method using
word embeddings and some external ressources.
– External: External QE techniques comprise methods that obtain expansion
terms from other resources besides the target corpus [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. Several approaches
have been proposed to use external resources such as Wikipedia, WordNet
and DBpedia to improve query expansion [
        <xref ref-type="bibr" rid="ref1 ref12 ref29">12,1,29</xref>
        ].
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Pattern Mining-Based Query Expansion</title>
      <p>In this section, we describe all steps required in our proposed approach to
expanding queries for microblogs retrieval. As a first step, we have been
concentrating on data preprocessing. Then, we describe pattern extraction task and define
notations and basic notions necessary for understanding the proposed approach.
3.1</p>
      <sec id="sec-3-1">
        <title>Preprocessing</title>
        <p>This process comprise preprocessing of tweet’s text i.e., dealing with stop words,
emoticons, punctuation, stemming, etc. Microblogs data such as tweets are too
short, generally not well written and do not respect the grammar. However,
this preliminary step is very crucial to eliminating the noise and cleaning data.
We, therefore, prepare the dataset for indexing by filtering tweets as follows:
(1) Removing null tweets and short tweets which contain less than two words;
(2) Removing Retweets (tweets starting with RT followed by username) as they
would be judged as non-relevant; (3) Removing non-English tweets; (4)
Eliminating the non-ASCII content found in any of the English tweets; (5)Removing
link and mentions from the tweet.</p>
        <p>Furthermore, we perform tokenization, tweet normalization, text stemming,
and stopwords removal, as part of the preprocessing phase. Tokenization is the
process of breaking each tweet up into words or other meaningful elements called
tokens. Then, we stem all tokens present in tweets except hashtags. We have
used the standard Porter stemmer of Stanford NLP tool4. After that, we remove
English stopwords5 that are present in tweets. We also perform normalization
of the tweet content, by resolving words containing many repeated letters, such
as the word "yes" or "happy", they may appear as "yeeees" or "happyyy" on
Twitter. The output of preprocessing task is used for indexing.
3.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Patterns extraction and Formal concept analysis</title>
        <p>
          Frequent pattern has an important and active role in many data mining tasks
[
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. This comprises to find interesting patterns (sets of items) called frequent
itemsets from databases. It was initiated by Agrawal et al. [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] and it corresponds
to finding the sets of attributes (or items) that appear simultaneously in at least
a certain number of objects (or transactions) defined in an extraction context
(see definition 1).
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>Construct Transactional Dataset</title>
        <p>The transactional dataset is
represented by a set of preprocessed tweets.</p>
        <p>Each tweet is represented as a set of
words which are considered as itemsets.</p>
        <p>We provide an example of transactional
dataset in Table 1.
Definition 1. (Data mining context). A data mining context is a triple D = (T ; I; R)
composed of a set of transactions T , a set of items I and a binary relation
between transactions and items R T I. Each couple (t; i) 2 R denotes the fact
that the transaction t is related to the item i.</p>
        <p>Definition 2. (Pattern, cover and support). A pattern X is a subset of items
X I. Its cover and its support are defined by:
cover(X) = ft 2 T j8i 2 X; (t; i) 2 Rg</p>
        <p>support(X) = jcover(X)j
Definition 3. (Frequent patterns). Given a context of data mining D and
minsup the minimum support, the set of frequent itemsets in D is:</p>
        <p>F I = fX</p>
        <p>Ijsupport(X)
minsupg:
4
https://nlp.stanford.edu/IR-book/html/htmledition/stemming-and-lemmatization1.html
5 https://github.com/ravikiranj/twitter-sentiment-analyzer/blob/master/data/feature_list/stopwords.txt</p>
        <p>
          In this work, we are interested in frequent closed patterns (denoted F CI),
which has been proposed by [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ]. The pattern X is called closed if none of its
supersets have the same support as X. In other words, 8Y; X Y , support(Y ) &lt;
support(X). Here, Y is a superset of X.
        </p>
        <p>
          Formal Concept Analysis (FCA) Formal concept analysis (FCA) is a
theory of data analysis identifying the conceptual structures within data sets. It
also presents an interesting unified framework to identify dependencies among
data, by understanding and computing them in a formal way [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. This gives it
the advantage to be an effective technique to analyze the different pattern
relationships such as in Social Network [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ], and Information Retrieval [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. Given
a formal context D, there is a unique ordered set which describes the inherent
lattice structure defining natural groupings and relationships among the
transactions and their related items. This structure is known as a concept lattice or
Galois lattice [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]. Each element of the lattice is a couple (T; I) which consists of
a set of transactions (i.e., the extent ) and a set of items (i.e., the intent ).
        </p>
        <p>Let X be a closed pattern of items (words), a formal concept is composed of
X and of the set of tweets containing this closed pattern.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Frequent closed patterns and expand the query</title>
    </sec>
    <sec id="sec-5">
      <title>Word Embeddings to</title>
      <p>In this section, we describe our proposed method to expand queries for the
microblog retrieval task. The proposed method is based on frequent concept
extraction and Word Embeddings. It is composed of three main steps: (1) Generate
frequent closed itemsets and select patterns, (2) Extend patterns using Word
Embeddings, and (3) expand the query (see Figure 1).</p>
      <p>Original
query
Extended
query</p>
      <p>Expanded terms
selection</p>
      <p>Retrieving
tweets using
IR model</p>
      <p>Extend
patterns using</p>
      <p>Word2Vec</p>
      <p>Top-N
tweets
Patterns
extraction
&amp; FCA
Patterns
selection</p>
      <p>Fig. 1. Overview of the proposed query expansion approach.</p>
      <sec id="sec-5-1">
        <title>Frequent concept extraction</title>
        <p>
          We perform patterns extraction on top-N tweets returned by Terrier system in
the initial search. The topmost relevant tweets are retrieved by using Terrier
with original queries. After that, we discover the closed frequent itemsets in
this large database of transactions (retrieved tweets). This process is performed
by using Charm-L algorithm [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ], where the discovery of patterns computes
the closed sets of items (i.e., words) that appear together in at least a certain
number of transactions (i.e., tweets) recorded in a database. This number is
called threshold and it is defined empirically. We compute the frequent concept
lattice L according to the minimal support threshold value minsup. Each node of
the formal concept lattice L represents the correspondence between a pattern and
the set of tweets that contain the words of this pattern. The parameters N (for
N-top tweets) and minsup are chosen according to experiments we conducted
(described in section 5.4).
        </p>
        <p>The algorithm for generating the complete set of interesting frequent closed
patterns and selection of candidate terms for the query expansion is shown in
Algorithm 1, with LK denoting the set of K closed frequent itemsets, T the set
of n transactions which represent tweets, minsup the minimal support threshold
value, and QP selected patterns for the expanded query. The list of K patterns
contains all interesting patterns found in top-N retrieved documents in the initial
query. The top-3 patterns with a high support value and with common tweets
are selected to represent candidate terms for the expanded query.
Algorithm 1 Patterns extraction
Require:</p>
        <p>T : a set of n tweets
W: Vocabulary of all words contained in tweets dataset
minsup: Minimum support threshold value</p>
        <p>R: a binary relation where R T W
Ensure:</p>
        <p>LK : List of K patterns</p>
        <p>QP : selected patterns for the expanded query
1: Creation of formal context D =(T ; W; R);
2: Computation of frequent concept lattice L for D according to minsup;
3: LK = CharmL(T , minsup);
4: QP =Top-3fargmax(Support(LKi )g
5: return LK ; QP ;
4.2</p>
      </sec>
      <sec id="sec-5-2">
        <title>Word Embeddings: Word2Vec model</title>
        <p>
          Word Embeddings has been used in query expansion task to enhance search [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ].
In this paper, we combine them with frequent closed patterns to expand queries
for microblogs retrieval. We train word embeddings on the entire dataset in order
to extract terms that are most similar to the selected patterns computed in the
previous step. For each query, we extend terms of the selected patterns by adding
the most similar terms contained in the dataset that are likely to be relevant to
the query but do not appear in the patterns. For the training of embeddings, we
used a project alternative Word2Vec 6 implemented in programming language
java by Medallia team7 to integrate it into our main program of query
expansion implemented in java. The training of the neural network is carried out on
the preprocessed corpus TREC 2011 on which search is performed. This model
estimates the probability that a term will appear in a position in a text based on
terms that appear in a window around this position. Each term in the dataset is
represented by a vector embedded in a vector space. Similarities between these
vectors were shown to correspond to semantic similarities between terms [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ].
        </p>
        <p>For the setting of the neural network, we have set a window with a size of 7
words (the appearance frequency of the words is of a minimum equal to 5, the
dimensions of the vectors are 200, the number negative examples is 7 with a use
of the hierarchical alternative softmax. Specifically, the continuous bag of words
model (CBOW) is used.
4.3</p>
      </sec>
      <sec id="sec-5-3">
        <title>Query expansion</title>
        <p>Given an original query q = fwq1; :::; wqng, the process of expanding q is
threefold: (1) retrieving relevant tweets answering q using a retrieval model, (2)
selecting a set of candidate terms (CT) for q by extracting patterns from the top-N
ranked tweets, and using formal concept analysis, (3) selecting the most related
terms to the CT set using Word2Vec model, to add only terms that are
semantically related to q. These terms are then selected to obtain the set of terms which
represent the expanded query denoted eq, with eq = q S fweq1 ; :::; weqm g.</p>
        <p>The process of obtaining candidate terms consists in selecting terms from
the patterns set as detailed in section 4.1. For extending these terms, we used
Word2Vec model in order to select terms that are semantically related to the
obtained patterns. The process for selecting these terms computes the cosine
similarity between the corresponding pattern-term-vector and each
tweet-termvector in the corpus, and rank the words in decreasing order of the cosine
similarity.
5</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Experimental Evaluation</title>
      <p>In this section, we conduct experiments to evaluate the effectiveness of the
proposed query expansion method. To demonstrate the performance of our proposed
method, we compare our patterns-based query expansion method with several
methods. Our experiments are conducted on the TREC 2011 collection.
6 https://github.com/medallia/Word2VecJava
7 http://engineering.medallia.com
5.1</p>
      <sec id="sec-6-1">
        <title>Dataset description</title>
        <p>
          In order to evaluate the proposed approach, we use the Twitter data collection
(TREC 2011 Microblog Track Data). This dataset contains approximately 16
million tweets collected over a period of 2 weeks (24th January 2011 until 8th
February) [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. Since the provided dataset contains only tweet ids, on the whole,
we have gathered around 12 million tweets with content. The rest of the tweets,
either was removed by their editors or we have no access to them. After
performing the data filtering and processing task explained in section 2, we obtained a
dataset of around 3.5 million tweets on which our experiments were conducted.
We have used 49 queries defined by TREC track 2011 [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ].
5.2
        </p>
      </sec>
      <sec id="sec-6-2">
        <title>Retrieval model</title>
        <p>
          We use the well-known Terrier IR system8 to index our data collection (TREC
2011). Terrier, an open source software, offers a range of document weighting
and query expansion models. It has been successfully used for ad-hoc retrieval,
cross-language retrieval, Web IR and intranet search [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. All the tweets (after
the preprocessing task) were indexed using Terrier, and the original queries were
used to retrieve and rank tweets using the standard BM25 retrieval model [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]
of Terrier. BM25 model has been used extensively within the TREC community
on a variety of corpora.
5.3
        </p>
      </sec>
      <sec id="sec-6-3">
        <title>Evaluation metrics</title>
        <p>
          We use Precision, MAP and nDCG metrics, which are widely used in information
retrieval [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ], to evaluate the proposed method for query expansion. Moreover,
we evaluated performance of the proposed method according to its Precision,
Recall, F-measure, and R-PREC metrics in order to compare the performance
and effectiveness of the proposed approach with other approaches. The MAP
(Mean Average Precision) for a set of queries is the mean of the average precision
scores for each query.
        </p>
        <p>
          Normalized discounted cumulative gain (nDCG) is a measure of retrieval
quality for ranked documents that, in contrast to precision, makes use of graded
relevance assessments [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. nDCG is computed as follows:
nDCG = Zi XR 2r(j) 1
j=1 log(1 + j)0
(1)
        </p>
        <p>Here, Zi is a constant to normalize the result to the value of 1. r(j) is an
integer representing the relevance level of the result returned at rank j where
R is the last possible ranking position. In our case, the relevance levels are 0
(irrelevant), 1 (relevant), and 2 (high relevant). nDCG@n is a variation of nDCG
where only the top-n results are considered.</p>
        <sec id="sec-6-3-1">
          <title>8 http://terrier.org/</title>
          <p>5.4</p>
        </sec>
      </sec>
      <sec id="sec-6-4">
        <title>Experimental results</title>
        <p>We evaluate the effectiveness of our proposed approach by using evaluation
metrics detailed in section 5.2. We conducted two different runs in our experiments,
the first one is based on patterns, and the second one represents the combination
of patterns and Word Embeddings. Table 2 reports these runs performance
compared with the baseline run. In this work, we define our baseline as a single run
which was generated using Terrier system, selecting the most recent 1000 tweets
that contain any of the query terms. In other words, without any query
expansion process. We also carried out a test on a method based on Pseudo-Relevance
Feedback (PRF) in order to compare our results. This expansion method is
applied using the Bo1 term weighting model implemented in terrier.</p>
        <p>Pattern mining utilized for short query expansion was effective for most
queries. Due to the shortness of queries and content of tweets, some queries give
a poor performance in evaluations. That’s why we used Word Embeddings to
extend patterns and enrich the query. This additional method added to the query
expansion model leads to a significant improvement compared to the pattern
and baseline runs as shown in figure 2. Compared to the baseline, we have
obtained significant improvements over the four measures : +191,30%, +218,01%,
+134,43% and +168,5% respectively on P@10, P@30, MAP and nDCG@10.</p>
        <sec id="sec-6-4-1">
          <title>9 https://trec.nist.gov/trec_eval/</title>
          <p>0,4
0,35
0,3
0,25
0,2
0,15
0,1
0,05
0</p>
          <p>Run-P-WE</p>
          <p>
            Table 4 shows MAP, Normalized discounted cumulative gain (NDCG) and
F-measure for the proposed approach compared to the query expansion method
introduced by authors in [
            <xref ref-type="bibr" rid="ref26">26</xref>
            ], where they proposed a new framework for query
expansion based on multiple sources of external information. We compared our
run with their different runs according to the MAP, NDCG, and F-measure
scores. We observe that our proposed model gives a good performance compared
to their runs.
          </p>
          <p>Metric/RunID NMF+Query Word2Vec NMF+W2V Run-P-WE</p>
          <p>In overall, the experimental results show that the approach leveraging
patterns and Word Embeddings outperforms the baseline and some methods of the
literature. The formal concept lattice we used to extend queries selects the
related terms which appear together in documents (tweets). Also, the extraction
of interesting frequent patterns allows us to select the most important terms
related to the initial query by considering the top N retrieved documents.</p>
          <p>In our empirical study, we fixed the number of retrieved documents in the
initial query to 500 (N). When this number increases to 1000, there is no
significant improvement. We also have varied the minsup value for extracting patterns
and have chosen 10 (i.e. 2%). All experiments we report on evaluation metrics
are performed using TREC Eval.
6</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>Conclusion</title>
      <p>In this paper, we proposed a query expansion method to enhance microblogs
search. The shortness of the microblogs and the queries may impact the
quality of search. Our proposed method is based on frequent closed patterns and
formal concept analysis. The frequent closed patterns are combined with word
embedding for finding the words that are most similar to the original query. The
results revealed the effectiveness of the proposed approach and show the interest
of combining patterns and word embedding to enhance microblog search.</p>
      <p>In our future work, it will be interesting to investigate temporal
information presented in tweets. We also propose to integrate the location information
while searching within the tweets where the query can be composed of
region-ofinterest (ROI) and text. We will further investigate external resources to compare
our proposed method with external QE approaches.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Aggarwal</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Buitelaar</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Query expansion using Wikipedia and DBpedia</article-title>
          . In: CLEF (Online Working Notes/Labs/Workshop) (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Agrawal</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Imieliński</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Swami</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Mining association rules between sets of items in large databases</article-title>
          .
          <source>SIGMOD Rec</source>
          .
          <volume>22</volume>
          (
          <issue>2</issue>
          ),
          <fpage>207</fpage>
          -
          <lpage>216</lpage>
          (
          <year>Jun 1993</year>
          ). https://doi.org/10.1145/170036.170072
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Carpineto</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Romano</surname>
          </string-name>
          , G.:
          <article-title>A survey of automatic query expansion in information retrieval</article-title>
          .
          <source>ACM Comput. Surv</source>
          .
          <volume>44</volume>
          (
          <issue>1</issue>
          ) (
          <year>Jan 2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Codocedo</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baixeries</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kaytoue</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Napoli</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Contributions to the Formalization of Order-like Dependencies using FCA. In: What can FCA do for Artificial Intelligence? The Hague</article-title>
          ,
          <source>Netherlands (Aug</source>
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Codocedo</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Napoli</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Formal Concept Analysis and Information Retrieval - A Survey</article-title>
          . In: International Conference in Formal Concept Analysis - ICFCA
          <year>2015</year>
          . vol.
          <volume>9113</volume>
          , pp.
          <fpage>61</fpage>
          -
          <lpage>77</lpage>
          . Springer, Nerja,
          <source>Spain (Jun</source>
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Diaz</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mitra</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Craswell</surname>
          </string-name>
          , N.:
          <article-title>Query expansion with locally-trained word embeddings</article-title>
          .
          <source>CoRR abs/1605</source>
          .07891 (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Ganter</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wille</surname>
          </string-name>
          , R.:
          <source>Formal concept analysis: mathematical foundations</source>
          . Springer Science (
          <year>1999</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8. Han,
          <string-name>
            <given-names>J</given-names>
            ., Cheng, H.,
            <surname>Xin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Yan</surname>
          </string-name>
          ,
          <string-name>
            <surname>X.</surname>
          </string-name>
          :
          <article-title>Frequent pattern mining: current status and future directions</article-title>
          .
          <source>Data Min. Knowl. Discov</source>
          .
          <volume>15</volume>
          (
          <issue>1</issue>
          ),
          <fpage>55</fpage>
          -
          <lpage>86</lpage>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Hu</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Deng</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guo</surname>
          </string-name>
          , J.:
          <article-title>Improving retrieval performance by global analysis</article-title>
          .
          <source>In: 18th International Conference on Pattern Recognition</source>
          . vol.
          <volume>2</volume>
          , pp.
          <fpage>703</fpage>
          -
          <lpage>706</lpage>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Järvelin</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kekäläinen</surname>
          </string-name>
          , J.:
          <article-title>Cumulated gain-based evaluation of IR techniques</article-title>
          .
          <source>ACM Trans. Inf. Syst</source>
          .
          <volume>20</volume>
          (
          <issue>4</issue>
          ),
          <fpage>422</fpage>
          -
          <lpage>446</lpage>
          (
          <year>Oct 2002</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Jones</surname>
            ,
            <given-names>K.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Walker</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Robertson</surname>
            ,
            <given-names>S.E.</given-names>
          </string-name>
          :
          <article-title>A probabilistic model of information retrieval: Development and comparative experiments</article-title>
          .
          <source>Inf. Process. Manage</source>
          .
          <volume>36</volume>
          (
          <issue>6</issue>
          ) (
          <year>Nov 2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Kotov</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhai</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Tapping into knowledge base for concept feedback: Leveraging ConceptNet to improve search results for difficult queries</article-title>
          .
          <source>In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining</source>
          . pp.
          <fpage>403</fpage>
          -
          <lpage>412</lpage>
          . WSDM '12,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Lau</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tjondronegoro</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Microblog retrieval using topical features and query expansion</article-title>
          .
          <source>Proceedings of The Twentieth Text REtrieval Conference (November 15-18</source>
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jones</surname>
            ,
            <given-names>G.J.F.</given-names>
          </string-name>
          :
          <article-title>Comparative evaluation of query expansion methods for enhanced search on microblog data: DCU ADAPT @ SMERP 2017 workshop data challenge</article-title>
          .
          <source>In: Proceedings of the First International Workshop on Exploitation of Social Media for Emergency Relief and Preparedness co-located with European Conference on Information Retrieval</source>
          . pp.
          <fpage>61</fpage>
          -
          <lpage>72</lpage>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Massoudi</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsagkias</surname>
          </string-name>
          , M.,
          <string-name>
            <surname>de Rijke</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Weerkamp</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          :
          <article-title>Incorporating query expansion and quality indicators in searching microblog posts</article-title>
          .
          <source>In: Proceedings of the 33rd European Conference on Advances in Information Retrieval</source>
          . pp.
          <fpage>362</fpage>
          -
          <lpage>367</lpage>
          . ECIR'
          <volume>11</volume>
          , Springer-Verlag, Berlin, Heidelberg (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Mikolov</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Corrado</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dean</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          :
          <article-title>Efficient Estimation of Word Representations in Vector Space</article-title>
          .
          <source>In: Proceedings of the International Conference on Learning Representations</source>
          . pp.
          <fpage>1</fpage>
          -
          <lpage>12</lpage>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Mittal</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nayak</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Govil</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            ,
            <given-names>K.C.</given-names>
          </string-name>
          :
          <article-title>Dynamic query expansion for efficient information retrieval</article-title>
          .
          <source>In: 2010 International Conference on Web Information Systems and Mining</source>
          . vol.
          <volume>1</volume>
          , pp.
          <fpage>211</fpage>
          -
          <lpage>215</lpage>
          (
          <year>Oct 2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Ounis</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Amati</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Plachouras</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>He</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Macdonald</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Johnson</surname>
          </string-name>
          , D.:
          <article-title>Terrier information retrieval platform</article-title>
          .
          <source>In: Proceedings of the 27th European Conference on Advances in Information Retrieval Research</source>
          . pp.
          <fpage>517</fpage>
          -
          <lpage>519</lpage>
          . Springer-Verlag, Berlin, Heidelberg (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Ounis</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Macdonald</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lin</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Soboroff</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Overview of the trec-2011 microblog track</article-title>
          .
          <source>In: In Proceedings of TREC</source>
          <year>2011</year>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Pal</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mitra</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bhattacharya</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Exploring query categorisation for query expansion: A study</article-title>
          .
          <source>CoRR abs/1509</source>
          .05567 (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Pasquier</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bastide</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Taouil</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lakhal</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          :
          <article-title>Efficient mining of association rules using closed itemset lattices</article-title>
          .
          <source>Inf. Syst</source>
          .
          <volume>24</volume>
          (
          <issue>1</issue>
          ),
          <fpage>25</fpage>
          -
          <lpage>46</lpage>
          (
          <year>Mar 1999</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Roy</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paul</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mitra</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Garain</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          :
          <article-title>Using word embeddings for automatic query expansion</article-title>
          .
          <source>CoRR abs/1606</source>
          .07608 (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Sanderson</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Test collection based evaluation of information retrieval systems</article-title>
          .
          <source>Foundations and Trends R in Information Retrieval</source>
          <volume>4</volume>
          (
          <issue>4</issue>
          ),
          <fpage>247</fpage>
          -
          <lpage>375</lpage>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Silva</surname>
            ,
            <given-names>P.R.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dias</surname>
            ,
            <given-names>S.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brandão</surname>
            ,
            <given-names>W.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Song</surname>
            ,
            <given-names>M.A.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zárate</surname>
            ,
            <given-names>L.E.</given-names>
          </string-name>
          :
          <article-title>Formal concept analysis applied to professional social networks analysis</article-title>
          .
          <source>In: Proceedings of the 19th International Conference on Enterprise Information Systems</source>
          , Volume
          <volume>1</volume>
          ,
          <string-name>
            <surname>Porto</surname>
          </string-name>
          , Portugal, April,
          <year>2017</year>
          . pp.
          <fpage>123</fpage>
          -
          <lpage>134</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Spink</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wolfram</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jansen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saracevic</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Searching the web: The public and their queries</article-title>
          .
          <source>Journal of the American Society for Information Science and Technology</source>
          <volume>52</volume>
          ,
          <fpage>226</fpage>
          -
          <lpage>234</lpage>
          (02
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Yang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fan</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huang</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          :
          <article-title>Exploiting multi-sources query expansion in microblogging filtering</article-title>
          .
          <source>Neural Network World</source>
          <volume>27</volume>
          ,
          <fpage>59</fpage>
          -
          <lpage>76</lpage>
          (01
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Zaki</surname>
            ,
            <given-names>M.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hsiao</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Efficient algorithms for mining closed itemsets and their lattice structure</article-title>
          .
          <source>IEEE Trans. Knowl. Data Eng</source>
          .
          <volume>17</volume>
          (
          <issue>4</issue>
          ),
          <fpage>462</fpage>
          -
          <lpage>478</lpage>
          (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Zhai</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lafferty</surname>
          </string-name>
          , J.:
          <article-title>Model-based feedback in the language modeling approach to information retrieval</article-title>
          .
          <source>In: Proceedings of the Tenth International Conference on Information and Knowledge Management</source>
          . pp.
          <fpage>403</fpage>
          -
          <lpage>410</lpage>
          . ACM, NY,USA (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <string-name>
            <surname>Zingla</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chiraz</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Slimani</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          :
          <article-title>Short query expansion for microblog retrieval</article-title>
          .
          <source>Knowledge-Based and Intelligent Information &amp; Engineering Systems: Proceedings of the 20th International Conference KES-2016</source>
          96(C) (
          <year>Oct 2016</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>