The 2017 Microblog Cultural Contextualization task consists in three challenges: (1) Content Analysis, (2) Microblog search, and (3) TimeLine illustration. This paper describes the use of cosine similarity, which is characterized by the comparison of similarity between two vectors of an inner product space. This research used two approaches: (1) word2vec and (2) Bag-of-Words (BoW) for extracting all relevant tweets to each event related to the four festivals: Charrues, Transmusicales, Avignon and Edinburgh.
Opinion mining is dened as "the task of classifying texts into categories depending on whether they express positive or negative sentiment, or whether they enclose no emotion at all" [1].
The growth of social media provides a domain of great interest for several studies related on opinion mining (also known as sentiment analysis) [2] such as opinion analysis related to topics or problems of political preferences [3], opinion about a specic product [4], news [5], and others. According to [6], Twitter is the most popular microblogging network in the world; this microblogging service has more than 300 million users, which represents a competitive advantage for many organizations.
In this project we propose the usage of cosine similarity with two features:
Bag-of-Words and word2vec using a dataset of workshop Microblog Cultural Contextualization 2017 to determine the tweets relevance according to each event from four European festivals. The remainder of this paper is structured as follows. In section 2, related work on timeline illustration, cosine similarity and opinion mining are presented. Section 3 is focused on showing the materials and methods. Section 4 describes the experimental results where two features: Bag-of-Words and word2vec are used with cosine similarity. Finally, section 5 gives a summary of this work.
2 Related Work
The CLEF 2017 TimeLine illustration shared task [7] was dedicated to retrieve all relevant tweets based on festival events. A variety of analysis, such as descriptive (duplicate and web addresses removal, abbreviations, etc.) [8,9], correspondence and interactive (FactoMineR package) [8] were used.
When looking at the State-of-the-Art on opinion mining for TimeLine illustration, we distinguish that Dogra et al. [10] used several approaches, such as (i) content-based retrieval, where a query represents a topic for matching in the documents. A common method used in this task is BM25 (Best matching), which is a probabilistic information retrieval function for documents features such as term and document frequencies, and document length [11,12], (ii) diversication is used when a retrieval function does not take into account the relations among returned documents; they may have relevant and redundant information [13], and (iii) re-ranking for improving the retrieval results through a baseline system [14].
In related research, Murtagh [8] used correspondence analysis to map the dual spaces of days and hashtags to a latent semantic space with social narratives information, and related work on Pierre Bourdieu news.
According to Hoan [15], the model usage according to base knowledge, makes use of an ontology to identify the relation among tweets, festivals and locations. They used a combination of Stanford NER, festival location and user prole. The tweets comparison related to each festival consists of festivals and properties lists with the tweet content.
In this paper, we select a cosine similarity approach for unsupervised learning, and now, we will present some works related to this method with similar objectives.
Shi and Macy [16] compared a standardized Co-incident Radio (SCR) with Jaccard index and cosine similarity. They chose SCR to map sport league studies, music artists, and congress members that can obtain more followers on Twitter.
AL-Smadi et al. [17] provided a semantic text similarity approach using text overlap, word alignment and semantic features to identify the paraphrasing in Arabic news tweets. This semantic text similarity is based on Support Vector Regression, which is used for regression analysis. The method achieved good accuracy according to State-of-the-Art.
Tajbakhsh and Bagherzadeh [18] used cosine similarity to nd coincidences among tweets. Also, the results were compared with several semantic-based algorithms such as Shortest path, Wu & Palmer, Lin, JiangConrath, Resnik, Lesk, LeacockChodorow, and Hirst-STOnge.
Other related works that use cosine similarity are presented in [1921].
In this section, we overview several researches based on tweets classication. Looking at particular systems, Dela Rosa, et al. [22] presented a study related on clustering and classifying tweets into dierent categories such as highly relevant, somewhat relevant, not relevant and spam using hash-tags as indicators. The aim of this approach is to nd the most representative tweets for some story such as video's Lady Gaga, Obama's help to Japan, disturbing video of Charlie Sheen, and others. The results of this technique performed well based on Stateof-the-Art. Nádia et al. [23] besought a combination of classier ensembles (Random Forest, Suppport Vector Machines, Multinomial Naive Bayes and Logistic Regression) and lexicon for predicting whether a tweet is positive or negative concerning a query term. This ensemble method used two feature representations (Bag-of-Words and hashing) and the results provided an improvement in the State-of-the-Art
Another approach was presented in [24], where a combination of classier ensembles (Multinomial Naive Bayes, SVM, Random Forest, and Logistic Regression) and lexicons were trained for identication of tweets polarity.
In related research, Paltoglou and Thelwall [25] proposed an unsupervised approach based on lexicon-classier using Bag-of-Words as features. This method estimates the polarity and subjectivity of informal texts on the web, such as tweets, social network and online discussions. The results show that the proposed algorithm presents a trustworthy solution for analyzing feelings of informal communication on internet.
With regard to sentiment analysis of tweets posted on Twitter during a disaster, Venkata et al. [26] provided an approach with Bag-of-Words, polarity clues, emoticons, internet acronyms sentistrength and puntuaction as features for identifying and categorizing the sentiments.
As we can see, many researchers have focused on the information that twitter can express, due to the opinions of consumers concerning brands and products [24], political and social events [27], health care [28,29], higher education [30], microblogging and social network services [31], foresight [32] and others.
In this work, we selected an approach for unsupervised learning called cosine similarity method using two types features: word2vec and Bag-of-Words. Each experiment was evaluated individually.
Tweets gathered from Twitter without a preprocessing stage have noise, confusing words, URLs, stop words and so on. Therefore, this study used the following preprocessing:
1. Consider tweets, which are written in latin alphabet such as English, Spanish, German, Italian, French, and others. 2. Eliminate all retweets. 3. Remove special characters, accents and lingüistic inections. 4. Convert the dates mm/dd/yy into an only format dd/mm/yy. 5. Serialize and reduce the dataset into 10,000 tweets due to the large amount of tweets. 6. Tokenize all words via NLTK toolkit [34]. 7. Replace several white spaces into a white space.
This study uses the following features after preprocessing the tweets.
1. Bag-of-Words. This is a basic representation of words as features. The aim is to compute the word frequencies in a document to nd the similarities and dierences among the documents [35]. Figure 1 shows Bag-of-Words approach applied in this study.
Fig. 1: An example of Bag-of-Words approach 2. Word2Vec. This is a useful technique used to create word embedding [36]. This means, representing a word as a vector [37]. We use Word2Vec to create an embedding vector for each word in the tweet, compare all vectors and take the maximum value to obtain sentence representation.
Fig. 2: An example of Bag-of-Words approach
Cosine similarity is a measure that calculates the angle cosine between two vectors [16]. This technique indicates the degree of similarity between documents which are represented by vectors; when the two vectors are equal then the similarity is high and we obtain a value of 1.
Fig. 3: An example of cosine similarity
In this context, each topic and tweet are represented as vectors, where each vector has the word frequencies (See Figure 1), and then, the cosine formula is applied as follows.
where the topic document is represented by A, and the tweet document is represented by B.
In these experiments, we used the format .res for representing the results as shown in Table 1. We conducted two experiments. The rst set of experiments consisted in using Bag-of-Words. Based on this approach, Table 2 shows the ten tweets most relevant according to the Charrues event.
In the next stage of experiments, we used word2vec as features to perform the task. The results of this experiment are shown in Table 3. The motivation of this work is focused on retrieval of the most relevant tweets for each of the four festivals. We concentrated on two types of features: Bag-of-Words and word2vec using the cosine similarity approach. Our results show that this technique is capable of detecting the more popular opinion gathered from Twitter based on similarity between tweets and topics.
There is much future work to be done in this study such as the analysis of hashtags, URL, emoticons and tweets characteristics. Furthermore, we will conduct experiments using Bag-of-Words and word2vec features with other measures of structural similarity such as Jaccard index and Phi coecient.