This paper describes some preliminary results obtained by treating the tweet contextualization task as a passage retrieval task. Each tweet was submitted as a query to the Indri 5.2 search engine after some preprocessing. Either paragraphs or sentences were retrieved in response to a query. Passages retrieved from the same document were concatenated. This approach does not work very well in terms of informativeness: the best of our runs was ranked 23rd out of 33 runs. Further exploration of ways to improve e ectiveness is needed.
The INEX tweet contextualization task at CLEF 2012 is a new task. The aim of this task is to provide some context for a given topic tweet 1. For this task, the context consists of a passage of at most 500 words extracted from a cleaned dump of the English Wikipedia. It is intended to provide some background information that will help a user to better understand the tweet.
In this report, we describe our very preliminary attempts at tweet contextualization. To begin with, we have simply treated contextualization as a passage retrieval task. After some preprocessing, the textual content of a tweet is used as a query to retrieve paragraphs or sentences from the Wikipedia corpus. If multiple passages are retrieved from the same article, they are merged together.
Related work is discussed in the next section (Section 2). Our approach is
described in Section 3. Section 4 presents our results and discusses some obvious
limitations of our approach. Our plans for further experimentation are outlined
in Section 5.
The tweet contextualization task is introduced by INEX at CLEF 2012. Bellot
et al. [
We divided each page in the corpus into separate paragraphs using the <p> and </p> tags. All text contained between these tags was indexed. Each paragraph was also split further into sentences using periods (.), question marks (?) and exclamation marks (!) as sentence delimiters. Stopwords were removed, and Porter's stemmer was used. Some statistics about the processed corpus are given below. Since any period (.) was regarded as an end-of-sentence marker, abbreviations were also split up when the text was indexed at the sentence level. This is why the number of terms (total and distinct) is somewhat di erent when the same text is indexed at two levels of granularity.
The topic tweets (1142 in all) were provided in two formats: JSON and simple text. We used the simple text format. Stopwords, URLs, the name of the tweeting authority, and the text \RT" were removed. The remaining words were stemmed using Porter's stemmer. Using these preprocessed tweets as queries, and Indri 5.2 as the search engine, we retrieved in turn paragraphs and sentences for each query tweet. A total of three runs were submitted. Details about these runs are given below.
Run1 | Top 50 returned paragraphs were submitted. If multiple paragraphs were retrieved from a document, then those paragraphs were concatenated. The similarity scores of individual paragraphs were simply added together to obtain the score of the concatenated result. Any paragraph longer than 500 words (including those obtained by concatenation) was truncated to the rst 500 words.
Run2 | Same as the Run1, except that we started with the top 100 sentences for each query.
Run3 | Same as the Run1, except that the top 100 paragraphs were used. 4
Submitted summaries were evaluated according to their informativeness and readability. Table 2 compares the performance of our submitted runs (Run1, Run2, Run3) with the best run at INEX 2012.
It is clear that the overly simplistic approach that we tried did not perform well with regard to informativeness (they did obtain good readability, however). Out of these runs, the sentence-level run performs best. A number of obvious drawbacks need to be recti ed.
{ When multiple paragraphs / sentences from a single document are
concatenated, their similarity scores are simply added together. This may lead to
poor ranking [
As mentioned in Section 2, a number of query-oriented summarisation approaches have been proposed in earlier work. In future work, we intend to explore how these may be applied to the contextualization task. Also, given that the \topics" or tweets are short to start with (at most 140 characters, many of which are taken up by URLs), query expansion is likely to be bene cial. We also hope to investigate query expansion / reformulation techniques as ways to improve informativeness of the generated summaries.