This short report describes the participation of the Universita della Svizzera italiana (USI) at the SMERP Workshop Data Challenge Track for the task text summarization of Level 1. Our participation is based on a linear interpolation for combining relevance and novelty scores of the retrieved tweets. Our method is fully automatic. For the relevance score we used the results from our runs at the text retrieval task whereas for the novelty we used a method based on Word2Vec. In total, we submitted four di erent runs and we used two di erent weight parameters. The results showed that when relevance and novelty have an equal contribution in selecting the tweets to use for the summary, the performance is better compared to favoring only the novelty. Additionally, information from POS tags improves the performance of the summarization task.
Recent years have seen the rapid growth of social media platforms (e.g., Facebook, Twitter, Google+) that enable people to share information on the web with a simple way. People use social media platforms for a number of di erent reasons that range from writing their opinions on products to sharing information on emergency situations.
Twitter1, one of the most popular microblogs, is a good source of information
and mining it can be very useful to assist relief operations in emergency
situations. However, a large number of data is posted online, hence it is very di cult
to extract and summarize useful information from tweet. Tweet
summarization aims to automatically generate a condensed version of the most important
content from the tweets that are relevant to a speci c information need. Past
research work on tweet summarization focused on topic-level summarization.
Shari et al. [
Other researchers have analyzed Twitter data for nding newsworthy
stories [
In this short report, we present our methodology for the text summarization task at the Exploitation of Social Media for Emergency Relief and Preparedness (SMERP) data challenge. Our participation is based on a linear interpolation which combines relevance and novelty scores of the retrieved tweets.
For computing the relevance scores we used the same techniques used for the
runs we submitted to the SMERP Data Challange Track of the text retrieval
task. Our rst submitted run for this task was based on plain query expansion
whereas the second one used additional information from POS tags. A detailed
description of the methodology we proposed for the task of text retrieval is
provided in [
Our summarization methods are fully automatic. We submitted four di erent runs for the summarization task (i.e., two for each of the two runs used in the text retrieval task). For each of them we assigned a di erent weight parameter which represents the importance of relevance and novelty of tweets and allows to produce a list of relevant and at the same time diverse tweets which can be used in the summary.
To compute the novelty of each tweet, we decided to use a metric that is
based on text similarity. For computing this similarity we used a methodology
based on word embeddings. More speci cally, we used Word2Vec [
The results showed that setting the weight parameter to 0.5 (i.e., relevance and novelty have an equal contribution) performs better compared to favoring only the diversity. In addition, we could observe that information from POS tags improves the performance in the summarization task.
This report is organized as follows. Section 2 describes the methodology we adopted for the task of text summarization. In Section 3 we present the results of our experiments, and Section 4 concludes the report. 2
For this task, we used a fully automatic method to extract summaries based on the linear interpolation of relevance and novelty scores. The novelty is quanti ed as the diversity of the current tweet with respect to the other tweets in the relevance ranking that can be selected for the text summary.
For the summarization we used the tweets retrieved in the two runs of the
text retrieval task [
The diversity score refers to the novelty of each tweets that is in the result list and is calculated as: div(ti) = 1
maxSim(ti) where maxSim(ti) is the maximum similarity between the tweet ti and each of the tweets that were retrieved before it:
maxSim(ti) = maxj2f1;:::;i 1gsim(ti; tj )
Such similarity is computed by using a methodology based on Word2Vec2.
We use Word2Vec [
The collection Ce consists of the tweets posted during Nepal earthquake that occurred on the 25th of April 2015. To be more speci c, the original collection contains 90,000 tweets posted from the 1st to the 5th of May 2015. To use the collection for the training, we rst removed the URLs, some speci c characters (e.g., @, #), and the retweets. Then, we ltered out terms that are speci c to Nepal earthquake by extracting the entities related to geographical names or people (e.g., Kathmandu, Mahadevstan, Rahul Gandhi) and removing all of them. At the end of this cleaning process we had 22,017 tweets, 198,280 tokens, and 12,379 unique tokens.
After having computed the summary scores, we ranked the tweets based on their decreasing values and took the rst tweets in the summary-score ranking in order to have a summary up to 300 words.
Table 1 shows the summary of the submitted runs for the task of text summarization for Level 1. 2 The library used for Word2Vec: https://radimrehurek.com/gensim/models/word2vec.html
Finally we should note that our runs were the only fully automatic methods submitted for text summarization at Level 1 and therefore we can not directly compare the performance of our methods to the one achieved by the approaches submitted by the other groups. 4
In this short report we presented the participation of the Universita della Svizzera italiana (USI) at the SMERP Workshop Data Challenge Track for the task text summarization at Level 1. Our participation was based on a linear interpolation for combining relevance and novelty scores of the retrieved tweets. We submitted four di erent runs. The results showed that setting the weight parameter to 0.5 performs better compared to favoring diversity. In addition, the results showed that using information from POS tags yields better performance in the summarization task.
Acknowledgments. This research was partially funded by the Swiss National Science Foundation (SNSF) under the project OpiTrack.