Microblogging platforms make it easy for users to share information through the publication of short personal messages. However, users are not only interested in sharing, but even more so in consuming information. As a result, they are confronted with new challenges when it comes to retrieving information on microblogging platforms. In this paper we present a query expansion method based on latent topics to support users interested in topical information. Similar to news aggregator sites, our approach identifies subtopics to a given query and provides the user with a quick overview of discussed topics within the microblogging platform. Using a document collection of microblog posts from Twitter, we compare the quality of search results returned by our algorithm with a baseline approach and a state-of-the-art microblog-specific query expansion method. We introduce a novel, innovative semi-supervised evaluation strategy based on expert Twitter users. In contrast to existing query expansion methods, our approach can be used to aggregate and visualize topical query results based on the calculated topic models, while achieving competitive results for traditional keyword-based search.
Along with the development of Web 2.0, users have increasingly become content
providers. A good example of this trend are microblogging platforms. These
platforms allow users to share short text messages, images, or links with interested
observers (followers) [
In 2011, Twitter’s search engine processed about 1.6 billion search queries
daily. An analysis of the search behavior [
In contrast to Web search, searching microblogs displays some
characteristic challenges [
While many researchers propose query expansion algorithms for
microblogging platforms [
There are many approaches that use topic models for query expansion in classic
information retrieval [13], not so many for microblog posts. Yan et al. [12] present
an alternative to LDA specially for short texts: the biterm topic model (BTM).
Instead of generating documents, BTM models the generation of biterms
(unordered word-pairs that co-occur in short texts) and assumes that each biterm is
drawn from one topic. One work similar to ours describes the automatic
topicfocused monitor (ATM) [
Several approaches for query expansion and document expansion have been
proposed in the context of the Microblog Track at TREC. For example, Wang
et al. [
The presented approaches mostly aim to expand the given query to match
the language which is used in the short microblog posts [
Collected Tweets Preprocessing Processed tweets
LDA Topic Model Preprocessing Processed query Query Expansion Expanded Queries Search Engine
Relevant Tweets We want to support users searching for general topics, such as “politics” or “Ukraine”. To this end, we propose a query expansion approach based on topic modelling. These models are learned on a daily basis from a data set of crawled and preprocessed tweets and are later used to expand user-specified queries. Figure 1 displays our system’s architecture. The crawling of tweets and topic model construction is handled offline, while the topic model is being used to expand queries in an online fashion at query time. If a new, unkown query term is used which is not present in our offline-computed topic model, we fall back to standard keyword search. However, this essentially does not happen for our targeted general queries. Furthermore, we address recency and popularity in Twitter indirectly via computing new topic models daily so our model reflects trends accordingly.
Topic Model Construction We used latent Dirichlet allocation (LDA) [
Using LDA, the number of topics K has to be specified in advance. A larger K leads to splitting of topics, allowing for the separation of ambiguous topics. However, if no ambiguous topics are left, homogeneous ones are split up. For the purpose of query expansion, it is important that different topics can be found for a term, and that the topics found are not ambiguous, as this could lead to topic shifts. We evaluated different values for K on a validation set, which is described in Section 4.
Query Expansion We are interested in the most probable topics for all words of a query q, i.e., we search for topics i where p(z = i|q) (in the following p(i|q)) is maximal. During Gibbs sampling, we sample values for z for each word w in the vocabulary w. We use these samples of z to estimate pˆ(i|q) with n(i,w) . In n(w) other words, pˆ(i|q) is estimated by the number of times the query words q were assigned to topic i divided by the total number of occurrences of words q in the corpus. Note that, although our test queries only contain a single term, this formulation also holds for queries with multiple words. For the query expansion, we then use the topics’ best representatives, i.e., for a topic i the most probable words based on p(w|i) = φiw. The quality of the query expansion is heavily influenced by the number of topics the query is expanded with, as well as the number of words chosen from each topic for expansion. We optimized these model parameters on a validation set (see Section 4). Best results were achieved setting K, the number of topics, to 200; the number of terms to use for query expansion to 10, and the threshold to include a topic for an expansion to pˆ(i|q) > 0.05. For our example in Table 1 the top 7 topics would be used for query expansion, while the rest are disregarded. 4
To assess the ability of our algorithm to retrieve topically relevant tweets, we propose a novel, semi-automatic evaluation strategy that produces high-quality labeled data by utilizing expert Twitter users. In addition, we present some example queries together with the expanded queries based on our topic model as anecdotal evidence demonstrating how our algorithm can help users to get a topical overview of subtopics for a given general query.
Data Set Most existing annotated data sets are focused on detailed information needs, such as the Tweets2011 corpus used for the TREC Microblog Track6. General topical queries are not included. Therefore, we created our own data set with 6 TREC microblog data http://trec.nist.gov/data/tweets semi-automatic annotations. We chose 2 general topical queries:“sports”, “politics” and for each general query 2 more specific ones: “baseball”, “basketball”, “Ebola”, “Ukraine”. To find relevant tweets for each of the queries, we handpicked 10 expert twitter users who primarily tweet on the topic corresponding to the query. Together with the relevance of tweets we used popularity and the number of tweets to select these users. For “politics”, e.g., these users were: @BBCPolitics, @CNNPolitics, @NicRobertsonCNN, @KevinBohnCNN, @TheWhiteHouse, @politico, @thehill, @HuffPostPol, @CBSPolitics, @BarackObama. We then crawled these users’ tweets together with the 1% of general tweets available through the Twitter API. We annotated only our expert users’ tweets as relevant for the respective queries, leading to small values in precision, because some tweets marked as non-relevant are actually relevant. Yet, tweets marked as relevant are in large part actually relevant. Thus, we estimate a method’s tendency for the actual precisions. We constructed two data sets, one for validation and one for testing. Each set includes a training set of one day of twitter data to learn the topic model and the subsequent day to validate or test (Oct. 21st and Dec. 4th 2014, each 1.4m tweets (1% of all tweets)). On average, our expert users published 196 tweets per query per day.
Baseline Approach As baseline approach BL, we search for the given queries without query expansion. Similar to Twitter’s search engine, we search for the query terms in tweets as well as in linked content using BM25. In contrast to Twitter’s search, our ranking is not incorporating recency or popularity.
Next to the baseline approach, we compare our search results with a
competing query expansion algorithm that is designed for microblogging platforms
and based on word co-occurance [
Topic-Based Approach Our topic-based approach results in a set of expanded queries for each initial query according to our topic model. We set α asymmetric and choose the initial value αi = K · 0.01 for all i ∈ {1, 2, . . . , K}. In contrast to α, we set β symmetric with initial value βi = 0.05. We run Gibbs sampling for 500 iterations. Each topic i in our model that contains the query term q (i.e. pˆ(i|q) > 0.05) forms the basis for one query. To compare our search results with other search algorithms and the baseline, we merge the tweets resulting from each expanded query q into one ranking. We calculate a ranking score scq(i, d) for each tweet d that was found for a query q. The score depends on the topic (=expanded query) i for which the tweet was found and the tweet d itself. The score combines the probability pˆ(i|q) of the query term q belonging ttwoetehte BtoMpi2c5i(,dt)h:sectqo(ip,idc)’s=prpˆo(pio|qr)ti·oθndiθ+di BfoMrtw25e(edt)dT,hanisdstchoereBaMllo2w5ssctoorecofmorbtinhee the results of all expanded queries for a query term into one ranking, which is needed to compare the precision with other approaches.
Results The results differ from query to query. Mean average precision (MAP) is 0.101 for the baseline approach (BL), 0.152 for the co-occurance-based approach (CB), and 0.152 for the topic-based approach7. The co-occurrence-based query expansion and our topic-based approach improve the results decidedly over the baseline. CB outperforms the topic-based approach only for the query “Ukraine”, which results in similar MAP scores, see Table 2. Less general queries, such as Ebola, are less likely to benefit from query expansion since most tweets contain the keyword itself, whereas tweets about baseball are much more likely to contain words such as “MLB” instead of the word “baseball”.
The expanded queries give an overview of the topic. The co-occurance-based approach only produces one expanded query, whereas our topic-based approach finds multiple topics for a given keyword and thus can create multiple expanded queries representing subtopics. Table 3 shows how our approach identifies different subtopics related to sports: English soccer, injuries, and American sports, while the co-occurance based approach fails to give a good overview and mixes various sports-related terms. The results are similar for the query Ebola. Here our approach identifies a topic related to Ebola in the U.S. vs. Africa. Discussion The co-occurance-based expansion is calculated specifically for each query, therefore it benefits from the expansion terms being well suited. Yet, especially for the more general queries, the expanded queries can become ambiguous, i.e., contain more than one specific topic with considerable topic shifts. In contrast to the co-occurance approach, our topic-based approach discovers more 7 To create comparable MAP scores, each ranking is restricted to 500 tweets relevant terms for a given query. Thus, the focus of the search can transform to a broader topic than the original one. A strength of our topic-based approach is also the flexibility allowing to expand the query with a variable number of topics and visualize the inherent subtopics. 5
We have analyzed the usage of topic models to support general keyword queries in microblog search. We proposed a query expansion method using latent Dirichlet allocation to find relevant tweets and to group them based on latent topic information. Our experiments have shown that our approach outperforms standard keyword-based search and further demonstrated competitive results compared to a state-of-the-art microblog-specific query expansion algorithm. While standard search algorithms do not by default cluster search results, our approach returns tweets from various subtopics and the topics itself can be inspected to get a quick overview of what is currently discussed in Twitter related to general keywords. Besides a further, large-scale evaluation, for future work we are interested in the development of topics over time. Since Twitter is a highly dynamic platform, we hope to capture trending subtopics for general keywords by substituting LDA with a dynamic topic model. 12. Yan, X., Guo, J., Lan, Y., Cheng, X.: A biterm topic model for short texts. In:
WWW. pp. 1445–1456. ACM (2013) 13. Yi, X., Allan, J.: A comparative study of utilizing topic models for information retrieval. In: ECIR. pp. 29–41. Springer (2009)