Posts published on the Internet could serve as a valuable source of information regarding emotion. Recommendation systems, stock market forecast and other areas are likely to benefit from the advancement in mood classification. To deal with this task, researchers commonly rely on preassembled lexicons of emotional words. In this paper we discuss the possibility of extracting emotion-specific words from userannotated blog entries. The study is based on analysis of the collection from 14800 Live Journal posts containing the “Current mood” tag, specified by the author. The analysis findings and possible applications are discussed.
Over the last few years, a considerable amount of work has been done to reduce
the overload of user-generated web content [
Online diaries provide researchers with extremely diverse and manifold data. Blog entries are rich in deeply personal and subjective content. Unlike other corpora used in sentiment analysis, "Current Mood" is a text attribute directly specified by the author at the time of writing, rather than by some independent annotator. We expect that analysis of user-annotated data could provide new information about words people use to express their emotional states.
Over the past few decades there have been several projects devoted to analysis
of emotions in the Internet posts. For example, a research project for measuring
emotions is “Pulse of a nation” is based on analysis of Twitter messages from
September 2006 to August 2009 [
To measure emotions in each tweet, Mislove and coauthors used the ANEW
word list [
The project focused on the expression of happiness in social media was
developed by a group of researchers from the University of Vermont. They tried to
measure happiness in Twitter posts [
First of all, Dodds and his coauthors conducted a survey using Amazon Mechanical Turks to obtain happiness evaluations of over 10,000 individual words, representing a tenfold size improvement over similar existing word sets (chosen by frequency of usage in collected samples of nearly 4.6 billion expressions posted over a 33 month span). The created words list contains ranks of their relation to happiness. For example, the top happiness words in their rank are laughter (rank=1) and happiness (rank=2). Next, they created on-line service hedonometer.org, which provides real time happiness analytics based on analysis of frequencies of the words from the list. It is worth mentioning that this service also has a rank for the word “birthday”, so the expression “Happy Birthday” is not excluded from analysis.
Lansdall-Welfare, Lampos, and Cristianini tried to measure several emotions
in twitter posts by counting the frequency of emotion-related words in each text
published on a given day [
After this pre-processing Lansdall-Welfare, Lampos, & Cristianini compiled four word lists containind 146 anger words, 92 fear words, 224 joy words and 115 sadness words. The evaluation of emotions in tweets was based on counting the amount of tweets containing each word from the compiled list. Lansdall-Welfare, Lampos, & Cristianini say they do not expect the high frequency of the word ‘happy’ to necessarily signify a happier mood in the population, as this can be due to expressions of greeting, like “Happy Birthday”. Although they do not filter this and similar expressions in their analysis.
We can conclude that the projects running analysis of emotions and moods in social networks usually use the lexicon methodology based on expert-annotated words lists.
Application of the lexicon approach based on expert or naïve rating of
emotions in the Internet posts can be supported by the findings made by Gill, Gergle,
French and Oberlander. They examined the ability of naive raters of emotion to
detect one of the eight emotional categories by asking participants to read 50 and
200 word samples of a real blog text and evaluate whether this message expresses
one of the eight emotions: anticipation, acceptance, sadness, disgust, anger, fear,
surprise, joy or being neutral [
Although raters show agreement in annotation of emotions, we can raise a question about its validity from the psychological perspective. We are not sure that all people express their emotions in a straightforward way, using words closest to the chosen mood category.
An interesting study of emotion in the context of a computer-mediated
environment was conducted by Hancock, Landrigan, & Silver [
In this study the people understand emotions of their partner even if these emotions were not explicitly expressed. This raises a question: could we extend the lists of emotional words by analyzing data annotated with the current mood of an author?
Analysis of text semantics, therefore, can provide information about user emotions and we expect analysis of user-annotated data from LiveJournal to help extend the existing words lists related to emotions.
We used DuckDuckGo1 search engine in conjunction with "GoogleScraper"2 Python module to make a list of English-speaking LiveJournal users who have at least once used the "Current Mood" functionality. The list of obtained URLs is passed down to the web crawler hosted on "import.io"3 platform. Each visited page is parsed to extract user messages and links to other LiveJournal blogs to be added to crawling query (e.g. from the comment section). Data collecting 1 https://duckduckgo.com/ 2 https://github.com/NikolaiT/GoogleScraper 3 https://import.io/ continues until the specified maximum page depth is reached.
For each message in visited blogs we extract a web address, title, text content and mood tag (usually accompanied by "Current mood:", "Feeling rather:" or just "Mood:"). Although the presence of subjective content in a title or web address is questionable, they are needed to identify continuous entries (eg. stories divided into series of posts). Blog posts, especially the ones with a fair amount of text, are not as frequent as, say, twitter posts. For that reason we do not use time stamps and rarely present geolocation data. The acquired dataset contains 14,800 documents tagged with 800 unique mood labels. 6% of the labels were responsible for 60% of data entries (Figure 2). Average text length is 420 words. An approximate post count for the average author is 5 messages (Figure 3). The most popular mood tags are: "accomplished", "cheerful", "tired" and "amused".
The initial step is to clean data from invalid entries (non-Latin or comprising only media content). The dictionary is then reduced by transforming everything to lowercase, stemming words, removing punctuation, stopwords and numbers. If we find negations, like “don’t”, “didn’t” or “not”, the subsequent token is replaced by not_token. For example, “they didn’t come” includes three tokens: “they”, “didn’t”, “come”. We also keep negations as we can expect negative moods negotiations to carry some additional information. URLs are shortened to their respective domains and repeating letters (more than three) are reduced to three. Words and numbers representing time or date are replaced with "time_date". blah good merry complaccent
full ecstatic apathetic
peaceful grateful csonfused thankfulaggravated sick jubilant eumbarrassed optimistic relieved
t determined angry bored horny worried ixno sseed tnebitcrehmsyteldlelosrwsalgiosnlcvoeaeormddedy ivcsoehuchreunagtirvyedeviopwuisessirdeladzyirrgitiadteddy a trs nnoerdydisappointed tismhirsty ditzy giggly lethargincervous
crazy silly crushed accomplished
distressed grumpy
satisfied frustrated hot sore
location pensive ddorky relaxed curiouscrappy ehgroggyproductive cranky suthoughtfulbusy derpressed geeky blank impressed melancholy cheerful sad energetic
hyper indescribable uncomfortable chipper tired amused awake calm ing sed k a ro l
e w p annoyed exhaustedhopeful
bouncy contemplative cold okay happy
excited sleepy
artistic said get around
way something can even know time now eyes just back one like tired happy really now face even can head time get said eyes know back one just like something can head way get now around know even time eyes one back just like After pre-processing the portion of non-sparse terms doubled. Overall dimensionality of feature space was reduced by more than 3 times.
Fifteen highest word frequencies for 8 most used mood labels are very similar
and do not provide any evidence that people use emotional words to mark their
emotions (Figure 4). We can see that words highly associated with a mood are
not included in the list with top 20 frequencies. For example, it is not often
that messages tagged “Happy” contain “happy” in their body. The list of top 20
words does not contain many words from emotional lists. Words “like”, “one”,
“back” are not put on the list of 10,000 words related to “happy” according to a
Hedonometrics survey [
The TF-IDF coefficient frequently used for document classification can provide more focused information about semantics of each emotion categories. To calculate TF-IDF, we joined all documents of the category into one document. First, the calculated TF-IDF allowed us to find most of the names used in posts. The words with the highest TF-IDF scores were “leo", "maes", "vampir", "jare","sandi","gaara", and "roger". After including the names in a list of stopwords, we received almost the same situation as with calculation of term frequencies (see Table 1).
accomplished hand 505.7 said 447.1 eye 416.0 back 389.8 head 375.5 smile 368.7 look 360.8 say 355.2 robot 353.5 mule 338.5
tired cheerful sleepy hand 222.4 artwork 173.3 ghost 342.0 rift 216.7 hand 167.7 hand 170.2 say 191.8 said 154.5 margin 153.3 f*ck 170.9 head 151.1 head 151.1 eye 156.9 eye 142.8 back 144.5 back 155.6 smile 135.7 color 142.3 look 149.5 face 135.2 say 140.1 head 148.6 back 119.3 eye 130.8 doesnt 145.5 look 117.9 superhero 130.0
said 145.2 lip 110.8 said 125.4 amused trade 692.7 prize 379.7 ward 143.9 claim 135.6 vote 128.4 said 91.6 hand 86.6
bill 86.0 materia 79.7 back 69.4
happy busy bouncy array 279.9 dev 263.9 sampl 120.7 hand 164.9 alt 178.5 hand 115.1
eye 164.7 hand 148.6 introspect 106.3 back 137.1 said 147.0 head 91.3
lip 127.4 border 133.4 back 89.4 smile 125.5 back 118.3 said 84.8 head 122.0 head 114.8 charact 80.6 knew 122.0 eye 114.7 lip 76.5 realis 121.2 knew 101.3 look 74.6
face 119.1 multi 101.1 kiss 74.4
Next, we introduced the TF-ICF coefficient. In order to identify important group-specific words, the term frequencies T F ij for word i in group j are multiplied by: log
||D|| P||D|| j=1 maxt∈Tj T Ftj
T Fij where ||D|| is the number of document groups and Tj - unique words in document group j.
The results produced by this transformation are listed in Figure 5 (apart from persons, locations and brands on top of the list) and provide more information regarding sentiment. For example, the word "finally" has a high value in documents tagged with "accomplished" or "tired". Although, some of these results are relatively counter-intuitive or even contradictory (e.g. "bed" is present in "accomplished", "bouncy", "cheerful", "busy", but absent in "sleepy").
This suggests that the distance between documents written in different emotional states could be shorter than that between documents written in the same emotional state by different authors. To test this hypothesis, we filtered documents by author and then, using vector representation of documents, we calcu(1) turned part bed maybe yeah mouth okay says looks finally help fingers words keep moment part better mouth looks says knows fingers arm kind okay help behind shoulder color ghost without okay mouth yeah almost f*ck says might looks work knows open already three f*cking maybe side body without might tried almost words mind came place keep behind work world tired lated cosine similarity between every pair of documents. The same procedure was carried out for documents filtered by current mood tag.
The only pair of tags "nervous" and "accomplished" has the distance between mood labels shorter than the average distance between different authors. This is probably because they carry a lot of objective content, which should have been filtered at earlier stages. The previously mentioned self-containing states of mind fall within the same group of labels, whose distances do not exceed the global average.
The vector model, therefore, contained enough information to distinguish
emotions and what we needed was to find an approach to extracting words
with maximum information. To solve this task, we used the Mutual Information
feature selection algorithm [
Application of the mutual information feature selection algorithm showed that the word “happy” provided relevant information about the mood of an author. However, the top twelve terms for the “happy” category only contained two emotional words included in the Hedonometics or ANEW list (“happy” and “wonder”).
We saw that, according to mutual information feature selection, many of
the categories were determined by the terms not included in emotional words
lists. Then we checked whether or not category name synonyms obtained from
WordNet were frequently encountered in the documents labeled with the same
mood [
eye 0.0084 head 0.0080 pull 0.0079 turn 0.0074 smile 0.0072 arm 0.0071 first 0.0070 pair 0.0069 behind 0.0069 hand 0.0068 away 0.0067 side 0.0063
tired three 0.0014
got 0.0011 home 0.0010 day 0.0010 lot 0.0009 tell 0.0009 realli 0.0009
far 0.0008 think 0.0008
let 0.0008 time 0.0008 part 0.0008 cheerful sleepy yes 0.0005 found 0.0009 feel 0.0004 name 0.0007 still 0.0004 probabl 0.0007 like 0.0004 knew 0.0007 way 0.0003 side 0.0006 right 0.0003 bad 0.0006 see 0.0003 tri 0.0006 guy 0.0003 rate 0.0006 think 0.0003 time 0.0005 girl 0.0003 walk 0.0005 hold 0.0002 mayb 0.0005 just 0.0002 find 0.0005 way. Only 14 of 50 popular moods or their synonyms are frequently encountered in a text tagged with the same mood: crazy (stressed, crazy, sick), curious (good, curious, sore), depressed (depressed, hopeful, artistic), ecstatic (hopeful, ecstatic, productive), hopeful (hopeful, crazy, sad), pissed off (pissed off, nervous, okay), sad (sad, frustrated, pissed), sick (confused, crazy, sick), sleepy (stressed, sleepy, sick), sore (sad, ecstatic, sore), stressed (stressed, depressed, curious).
Surprised by such results, we tried to analyze the document using words from the Hedonometrics list. Analysis of frequencies of top twelve words from the Hedonometrics list in texts written in different moods showed that these words have the most common usage in emotional states different from “happy” (Table 3). Only one word “successful” is used more frequently by authors who tagged their message with the current mood “happy”. laughter love happy laugh excellent joy successful win rainbow smile won pleasure celebration Analysis of user-annotated blog messages showed that connections between emotions and their linguistic expression could not necessarily be straightforward as is usually expected by compilers of emotional words lists. The most frequent words in each mood category are not included in the list of emotional terms. Application of TF-IDF and the calculated TF-ICF coefficient did not change the situation. Words with the highest scores continue not to be included in popular lists used for mood analysis. Application of the Mutual Information feature selection algorithm allowed us to find the most important words in each category, but only few of them are included in popular lists of emotional words. We can confirm that. according to the mutual information coefficient, the word “happy” has high discriminative power, while other words from the Hedonometrics list were not as successful.
People show a high ability to evaluate emotions of other persons even in a computer-mediated environment, although the way we can understand other people’s emotions still raises questions. On the one hand, the ability to understand emotions also exists in situations where emotions are not explicitly expressed; on the other hand, our analysis showed a paradoxical situation when the terms used for evaluation of emotions are not among the top 20 frequent or discriminative words for each of mood categories. These facts raise a question about psychological validity of straightforward techniques for measuring emotions.
In our further research we plan to move in two different directions. One is to compare results of emotion analysis by applying the classical lexical approach with two dictionaries (ANEW and Hedonometrcs) and Naïve Bayes algorithm using the probabilities calculated in the current research. The other direction is to test agreement between naïve or expert annotators and authors of mood labels. We also intend to develop more sophisticated procedures to filter objective content and detect invalid entries, establish a meaningful connection between content and label and further extend our database to improve validity of our study.