Food communities in Twitter are growing every year, and food-related content permeates everyday conversations. Users meet on Twitter to share recipes, give cooking advices or simply inform others about what they are eating. While some of these food-related conversations are not associated with any special occurrence, many conversations take place instead during speci c events. The detection of food-related events gives interesting insights: people do not talk only about Halloween and Easter, but they also create their own food-related events, such as the promotion of products (e.g., an online petition to propose the production of bacon- avored chips) or themed home-made recipes (e.g., a day of recipes dedicated to chocolate). In this paper, we propose an approach that accurately captures food-related content from the tweet live stream, and analyze the detected conversations to identify food-related events. The proposed technique is general as it can be applied to the identi cation of other thematic events in digital streams.
Recently, Twitter has received much attention from the research community.
It is reported1 that 500 million tweets are published on a daily basis. Tweets
cover a variety of topics, ranging from personal status updates (e.g., \going to
the gym") to local and global news (e.g., \FBI investigating possible corruption
at New York prison"). Tweets may contain hashtags, i.e., words pre xed with
the hash symbol #, which allow tweets with similar topics to be identi ed. Users
interested in speci c topics can search for relevant tweets by hashtags, which
make it particularly easy for users to create conversations about speci c events.
In the following, we denote by event a recognizable happening of limited
duration [
Regione Lombardia's \Proactive" project
1 https://about.twitter.com/company
parties. Food conversations, as for other topics with a wide coverage in social
media, permeate several events, which originate either within the boundary of
the digital community (e.g., #TacoTuesday) or in the real world (e.g., #easter).
Despite the huge adoption of Twitter as a platform for publishing and talking
about events, their automatic detection still remains an open problem [
In this paper, we propose a technique for the automatic detection of topicrelated events, i.e., events pertaining to a given topic of interest. More precisely, we devise a two-step detection procedure: we rst identify hashtags related to a given topic of interest, and then analyze them in order to extract the associated topic-related events. We show that, when applied to food-related events, our method is able to successfully identify relevant events among the top-1000 hashtags, attaining 100% Precision@10, and 80% Precision@172. Moreover, in addition to common food-related celebrations such as #easter, the proposed technique also manages to identify more Twitter-speci c initiatives, such as #MeatlessMonday. Nevertheless, note that our technique is applicable to several other contexts, including disaster management, breaking news and political events.
The remainder of this paper is organized as follows. We formally introduce the topic-related event detection problem in Section 2. In Section 3, we introduce our process for the retrieval and subsequent identi cation of topic-related tweets. In Section 4, our approach for the detection of events is presented. In Section 5, we demonstrate the e ectiveness of our method on a real-world scenario. In Section 6, we discuss about the related works in the literature, just before our nal conclusions and discussion of future works in Section 7. 2
Let T = f 1; : : : ; N g denote the tweet set obtained from observing the tweet live stream for a certain amount of time. Each tweet j = h!j ; Ij ; Hj i is composed of a textual component !j , a (possibly empty) image component Ij , and a set of related hashtags Hj . Moreover, let denote a topic of interest. If we indicate with Y = fY; Ng the set of relevance classes for the topic , we can associate each tweet j with a label yj 2 Y, such that yj = Y if tweet j is related to topic , and yj = N otherwise. By considering the set of the sole relevant tweets T R = f j : yj = Yg T , and de ning HR = Sj: j2T R Hj as the set of hashtags extracted from T R, we can therefore formulate the topic-related event detection problem as that of nding a set of topic-related hashtags F HR that are also associated with an event.
In order to solve the event detection problem, we devise the following twostep procedure. 1. Topic-related tweet retrieval. Each tweet entering our system is classi ed as relevant/non-relevant for the topic . Speci cally, to determine the releSeed users Seed terms Keywords Hashtags
Real-time tweets crawling
Filtering Multimodal classification
Text Image classification classification
Classifier aggregation Seed terms update topic-related tweets temporal distributions
Event classification
non-periodic Periodic classification periodic
Spike classification spike
non-event
V
event
(a) Topic-related tweet retrieval pipeline
(b) Event detection pipeline
vance to topic we adopt a multimodal classi cation approach [
These two phases are implemented as independent processes, discussed in Section 3 and Section 4, respectively. 3
The process for the retrieval and classi cation of topic-related tweets is illustrated in Figure 1(a). The system identi es topic-related tweets from the live stream in three phases: crawling, ltering and classi cation. Thanks to the presence of a feedback loop, the system automatically follows the topics users are currently discussing, thus adapting the crawling step to the emerging trends in conversations. Let us comment in greater detail on each such step. 3.1
Let a seed user denote a user which was identi ed by a domain expert as relevant to topic . Moreover, let S be a set of tweets manually labeled as relevant/nonrelevant to topic . A seed term (either keyword or hashtag) is a term that appears frequently in positively labeled tweets and rarely in negatively labeled tweets in S. The crawling module monitors the tweet live stream2, and retains tweets meeting at least one of the following selection criteria: i) authored by a seed user; or ii) containing at least one relevant seed term.
3.2
The collected tweets proceed in input to the ltering module, which discards a tweet if at least one of the following conditions holds: i) the tweet content is not written in English, ii) the tweet contains inappropriate words, or iii) the tweet contains words belonging to a topic-dependent set of stop words (e.g., \apple" in the case of food). 3.3
This step consists of a classi cation phase, at the end of which each tweet is labeled as relevant/non-relevant to the topic . We rst disaggregate each tweet j 2 T in its constituting components !j and Ij , and then use a textual and an image classi er to obtain two independent opinions on the relevance of !j and Ij to the topic . Finally, we merge these opinions to obtain a unique relevance label yj for the tweet j .
Text classi cation. We collected a dataset of tweets T ! (such that T !\T =
;) and manually annotated their textual components !j with a label y!
j 2 Y,
which speci es the relevance of !j w.r.t. topic . Each textual component !j
is subdivided in terms. User mentions (written as @username) and stop words
are deleted from the list of extracted terms, since they are not attributable to
a speci c topic. On the contrary, hashtags (after trimming the # symbol o )
are kept as discriminative features. Finally, terms are normalized by lowercasing
letters and applying Porter stemming [
Image classi cation. We collected a dataset of tweets T I (such that T \
T I = ;) and manually annotated their image component Ij with a label yjI 2 Y,
which speci es the relevance of Ij w.r.t. topic . An equal (and small) amount of
positive and negative samples is extracted from fIj gj: j2T I , and their key-points
together with the related SIFT descriptors [
I I is subdivided in training set Ttrain (60%), cross validation set TCV (20%) and test set TtIest (20%). An SVM classi er with RBF kernel is nally trained on the available training set f(xjI ; yjI )gj: j2TtIrain , and performance is computed on TtIest.
Classi er aggregation. In case tweet j is made of a single component (i.e., either !j or Ij ) the aggregation is not necessary. When both text and image (a) #NationalCheesecakeDay ]120 ts100 e tew80 [#60 )k40 ( f 20 00 23 46 69 92 115 138 161
k [day]
(b) #MeatlessMonday
content exist, we aggregate the classi ers opinions, with the method proposed
in [
Twitter users track content related to speci c topics using hashtags. Some tags are just used to describe content, so that it can be easily classi ed and retrieved in the future. Other hashtags are meant to track real-world events (e.g., earthquakes, holidays, elections) and social events (e.g., birthday of a social community).
When an event occurs and users start talking about it, the rate of usage of the related hashtag(s) increases rapidly, and it stays o -the-scale with respect to other common hashtags until either the event ends or the community loses interest in it. To study the rate of usage of hashtags, one can analyze their temporal distributions. A temporal distribution is a K-dimensional histogram associated with hashtag H, where the k-th component indicates the number f (k) of tweets produced during day k that contain H. Two examples of temporal distribution are shown in Figure 2.
In this paper, we identify topic-related events by tracking temporal variations in the usage of hashtags. We start from a collection of tweets related to topic downloaded as described in Section 3. For each hashtag in the collection, we extract its temporal distribution, and use a supervised approach to decide if the hashtag is related to an event. 4.1
Events discussed on Twitter have di erent natures. Some events happen once, and generate a large interest (although limited in time). For these events, which we call Spike Events, there is a single (and strong) perturbation in the usage of related hashtags. An example of spike event is shown in Figure 2(a). Here, a single activity peak on the hashtag #NationalCheesecakeDay was detected, as the Twitter food-related community joined the event by massively publishing cheesecake recipes in a limited amount of time. 14 12 10 )j8 (uFj64 2 0 0 0.1 0.2 0.3 u [1/day] 0.4
0.5 (a) #MeatlessMonday 0.1
On the other hand, some events are recurring periodically. For these events, which we call Periodic Events, there are multiple perturbations in the usage of related hashtags, such that the interest in the event raises periodically, and is null (or low) during the other days. An example of periodic event is shown in Figure 2(b). Here, an activity peak on the hashtag #MeatlessMonday can be detected on each Monday, since the event is joined by people that meet virtually every Monday to discuss about meatless recipes.
Figure 1(b) depicts our event classi cation process. The temporal distributions associated with the hashtags we want to classify as event-related/eventunrelated are fed as an input to a chain of two binary classi ers, the rst dedicated to spike event detection, while the second dedicated to periodic event detection. A hashtag (or equivalently its temporal distribution) is labeled as event-related if at least one the classi ers recognizes it.
Feature set. Spike and periodic events have a peculiar temporal distribution which is common for all the events of the same class. However, when it comes to training a classi er for the recognition of event classes, temporal distributions cannot be used as feature vectors: they su er from temporal dependence of subsequent components, and consequently events that clearly belong to the same class but happened in di erent periods of time would have completely di erent feature vectors and thus would not help the classi er learn the underlying model. For this reason, we used as feature vector the spectrum of the Fourier transforms jF (u)j of the normalized temporal distribution, which describe the frequency components of the signal and are agnostic with respect to the actual time of the events. As an example, Figure 3 shows the Fourier transforms of two periodic events, which happen in di erent periods but have similar spectrum.
Event classi ers. When it comes to building an annotated dataset to train
the classi er, we come up with an unbalanced training set, since events are rare
if compared to the total number of produced hashtags. Due to the lack of
positive samples (i.e., temporal distributions corresponding to events), the classi ers
could easily fall into the problem of over tting the data. Thus, we applied the
EasyEnsemble algorithm [
(a) Topic classi er sTTae!mxtples PNDTooeicsgttiaatiotilvinvesaearsmysaamspmiplzpeleelsess 11124248292438154882 IsTmaImagpeles PNDTooeicsgttiaatiotilvinvesaearsmysaamspmiplzpeleelsess 1512101370755040965 (b) Spike and periodic event classier Samples Positive samples 2030
in Hs Total samples 6900 Samples Positive samples 5000
in Hp Total samples 10890 In this section we assess the performance of the proposed topic-related event detection approach. We rst show how we can correctly identify topic-related tweets captured from the tweet live stream. Then, we apply event detection to the resulting tweet set, showing that our approach is capable of attaining good performance (measured as Precision@K). In the following, we illustrate the characteristics of the datasets we used to assess the multimodal classi er performance and report classi cation performance.
Dataset description. We trained the text and image classi ers on, respectively, the textual and image datasets T ! and T I , whose cardinalities are reported in Table 1(a). To test our classi cation approach, we randomly extracted and manually annotated the following sets of samples: i) T~ !, composed of 1900 tweets containing only text; ii) T~ !+I , composed of 1900 tweets containing both text and images, where T !, T I , T~ !, T~ !+I are all disjoint. Note that some tweets are characterized by ambiguous content, and thus annotating them as relevant or not relevant is di cult for a human annotator too. On our dataset, the inter-annotator agreement is 93:86%.
Classi ers performance. Multimodal classi cation improves performance with respect to text classi cation on T~ ! and T~ !+I . Table 2 shows how accuracy, precision, recall and F 1-measure increase in this scenario.
Text classi cation performance is insu cient when images are involved, because it is not able to interpret visual content and may misinterpret the text associated with images. In this section, we assess the performance of the proposed event detection technique on the food-related tweets.
Dataset description. We ran our topic-related tweet retrieval process from
June 1, 2014 to June 10, 2015. During that period, the system processed more
than 15 million tweets, 9 millions of which were labeled as food-related. The
corresponding number of relevant hashtags was 171451. However, only 21451
were associated with a temporal distribution comprising more than 5 tweets
and were included in the nal set of topic-related hashtags HR. In order to
train the spike classi er, we took a random sample HsR of size 6900 from HR.
Then, we performed a data annotation campaign on the crowdsourcing
platform Champagne [
R as Hp . The cardinalities of the two datasets are reported in Table 1(b).
Classi ers performance. The performance of the spike and periodic event classi ers are reported in Table 3. As shown, both classi ers attain high values of F 1-measure and accuracy, on both the training and test set. In order to further evaluate the e ectiveness of our approach, we also tested the proposed event deR tection technique against a gold standard dataset Hg , which we obtained by rst ordering hashtags in HR by total number of tweets, and then providing a gold label for the rst 1000 hashtags. In particular, each hashtag has been assigned a gold label by analyzing di erent factors, such as the name of the hashtag, its current use on Twitter, the shape of its temporal distribution, and the content of tweets collected by the process. Since the total number of tweets might be intended as a proxy for the success of an event, we believe that testing the proposed technique against the top-1000 hashtags can provide a meaningful insight on its e ectiveness in detecting successful events. Since the test was performed against a top-K ranked list, we measured performance by means of a Precision-Recall curve, which depicts the attained precision-recall values as K increases. Figure 4 R reports the performance of our technique on Hg . As shown, our method correctly identi es the rst 14 food-related events. Overall, our method labels 172 events as food-related, which leads to a nal precision-recall value of (0:80; 0:67).
Discussion. Table 4 shows the top-10 food-related hashtags retrieved by our event detection pipeline, together with tweet samples showcasing their usage. The list reports: i) food-centered social events that are con ned in the Twittersphere (#foodiechats, #MeatlessMonday, #bandwiches); ii) holidays (#halloween, #easter) and periodic calendar-based events (#sunday, #tbt [that is: `Throwback Thursday'], #tgif [that is: `Thank God it's Friday ']) during which users share themed recipes; iii) media events (#espys), during which people have dinner in front of the TV and share comments about the show and their food; iv) food-centered advertising campaign (#TeamWalmartProduce). Finally, Figure 5 shows a set of images retrieved by our pipeline and related to the periodic calendar-based events #TacoTuesday and #NationalCheesecakeDay. This sample shows how our pipeline is able to retrieve high quality multimedia content (thanks to multimodal classi cation), which could be used, e.g., to summarize the contents shared by Twitter users during the detected events. 6
A number of recent works in the literature cover the problem of event detection
on Twitter. The work in [
Several works use supervised classi cation methods to state if content is
related to an event. The work in [
In this paper, we investigated the problem of topic-related event detection on Twitter, which we cast as a supervised learning problem. We focused on the concrete use case of identifying events that include a food-related component, such as holidays or commercial initiatives. We rst induced a multimodal classi er capable of identifying tweets related to the topic of interest, which we used to isolate relevant tweets from the global tweet stream. Events were therefore identi ed by applying a chain of two classi ers, one for the identi cation of periodic events and one for the identi cation of spike events.
The experimental evaluation showed that our approach attains a Precision@10 value of 100%, and a Precision@172 value of 80%, proving therefore competitive with other state-of-the-art approaches available in the literature. Future work will focus on enriching the event classi er feature vector to capture social components, such as user pro le characteristics (e.g., authority) and network characteristics (e.g., centrality) and on spatial distribution analysis.