<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>“Buon appetito!” - Analyzing Happiness in Italian Tweets</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Pierpaolo Basile</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nicole Novielli</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Science University of Bari Aldo Moro Via</institution>
          ,
          <addr-line>E. Orabona, 4 - 70125 Bari</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>English. We report the results of an exploratory study aimed at investigating the language of happiness in Italian tweets. Specifically, we conduct a time-wise analysis of the happiness load of tweets by leveraging a lexicon of happiness extracted from 8.6M tweets. Furthermore, we report the results of a statistical linguistic analysis aimed at extracting the most frequent concepts associated with the happy and sad words in our lexicon.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1 Introduction</title>
      <p>
        The widespread diffusion of social media has
reshaped the way we interact and communicate.
Among others, microblogging platforms as
Twitter are becoming extremely popular and people
constantly use them for sharing opinions about
facts of public interest. Furthermore, its
worldwide adoption and the fact that tweets are publicly
available, makes Twitter an extremely appealing
virtual place for researchers interested in language
analysis as a mean to investigate social
phenomena
        <xref ref-type="bibr" rid="ref5 ref6 ref7">(Bollen et al., 2009; Garimella et al., 2016)</xref>
        .
      </p>
      <p>
        In addition, recent research showed how
microblogging is also used for self-disclosure of
individual feelings
        <xref ref-type="bibr" rid="ref14 ref2 ref8">(Roberts et al., 2012; Andalibi
et al., 2017)</xref>
        . As such, microblogs constitute an
invaluable wealth of data ready to be mined for
discovering affective stereotypes
        <xref ref-type="bibr" rid="ref2 ref8">(Joseph et al.,
2017)</xref>
        using corpus-based approaches to linguistic
ethnography
        <xref ref-type="bibr" rid="ref9">(Mihalcea and Liu, 2006)</xref>
        . Such
analyses, can further enhance our understanding on
how people conceptualize the experience of
emotions and what are their more common triggers.
Recent studies even envisaged the emergence of
tools for monitoring the public mood 1 and health
through the analysis of Twitter users’ reaction to
major social, political, economics events
        <xref ref-type="bibr" rid="ref5 ref7">(Bollen
et al., 2009)</xref>
        .
      </p>
      <p>
        In this study we report the results of an
exploratory analysis of the language of happiness in
Twitter. In particular, we perform a partial
replication of the approach proposed by
        <xref ref-type="bibr" rid="ref9">(Mihalcea and
Liu, 2006)</xref>
        for mining sources of happiness in blog
posts. The contributions of this paper are as
follows. First, we extract a happiness dictionary from
a sample of about 8.6M tweets from the TWITA
corpus of Italian tweets
        <xref ref-type="bibr" rid="ref3">(Basile and Nissim, 2013)</xref>
        .
For each word in the dictionary, we compute a
happiness factor by adapting the approach
proposed in the original study. Furthermore, we
perform a qualitative investigation of the 100
happiest and saddest words by mapping them into
psycholinguistic word categories (see Section 2). As
a second step, we use our dictionary to perform a
time-wise analysis of happiness as shared in
different hours and days of the week (see Section 3).
Third, we extract concepts most frequently
associated with happy words in our dictionary, which
we map into WordNet super-senses (see Section
4). We discuss limitations and provide suggestions
for future work in Section 5.
      </p>
    </sec>
    <sec id="sec-2">
      <title>The Happiness Dictionary 2</title>
      <p>2.1</p>
      <sec id="sec-2-1">
        <title>A Dataset of Happy and Sad Tweets</title>
        <p>
          Our study is based on TWITA
          <xref ref-type="bibr" rid="ref3">(Basile and
Nissim, 2013)</xref>
          , the largest available corpus of
Ital1’What Twitter tells us about our happiness’ https://
goo.gl/fmYBP3 - Last accessed: Oct. 2018
ian tweets. In particular, we analyze a subset of
400M tweets obtained by filtering-out re-tweets
from all the 500M tweets collected from February
2012 to September 2015. Following the idea
proposed in
          <xref ref-type="bibr" rid="ref13 ref5 ref7">(Read, 2005; Go et al., 2009)</xref>
          , we select
positive and negative tweets based on the presence
of positive or negative emoticons2. Since a tweet
can contain multiple emoticons, we selected only
tweets that contain a single emoticon appearing at
the end of the tweet. Using this procedure we
obtain a corpus Chappy of 8,648,476 tweets.
2.2
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Happy/Sad Word Extraction and Scoring</title>
        <p>From the Chappy corpus, we extract a subset of
words and we assign them an happiness factor
(hf ) computed according to the log of the odds
ratio between the probability that the word occurs
in positive tweets phappy(wi) and the probability
that it occurs in negative tweets psad(wi) as in Eq.
1.</p>
        <p>hf (wi) = log phappy(wi)
psad(wi)
(1)
We adopt additive smoothing (Laplace smoothing)
for computing both phappy and psad probabilities.
In our lexicon, we include and compute the
happiness factor only for words that occur at least
10,000 times, for a total of 718 words. We call
this list “the happiness dictionary” (Dh)3. Table 1
reports the most happy/sad words with the
corresponding happiness factor (score(hf)).</p>
        <p>We observe that some happy words (fback,
ricambi, benvenuta) are due to several positive
tweets that users post when they establish new
connections, i.e. when they start following a
2We use :-) and :) for happy and :-( and :( for sad.
3The dictionary is available on github https://
github.com/pippokill/happyFactor
new user or when they ask sombebody to follow
them back (fback) as in: @usermention ciao sono
nuova, fback? Grazie mille :) Sad words refer to
negative emotions or evaluations, such as triste,
dispiace, brutto, peccato. Interestingly, several
negative words emerge from the school domain
(compiti, studiare) and the word scuola has a
negative score of -0.93 itself.
2.3</p>
      </sec>
      <sec id="sec-2-3">
        <title>Happiness by Psycholinguistic Categories</title>
        <p>
          We are interested in understanding how happiness
words map into psycholinguistic word classes.
Hence, we check their distribution along the word
categories in the Linguistic Inquiry and Word
Count (LIWC) taxonomy
          <xref ref-type="bibr" rid="ref11">(Pennebaker and
Francis, 2001)</xref>
          . To this aim, we perform a qualitative
investigation on the 100 most happy and 100 most
sad words, that are the words with the highest and
lowest happiness scores, respectively. We map
each word into LIWC word categories. LIWC
organizes words into psychologically meaningful
categories, based on the assumption that the
language reflects the cognitive and emotional
phenomena involved in communication. It has been
used for a wide range of psycholinguistics
experimental settings, including investigation on
emotions, social relationships, and thinking styles
          <xref ref-type="bibr" rid="ref15">(Tausczik and Pennebaker, 2010)</xref>
          .
        </p>
        <p>We perform a coding of the English
translation of the happy/sad words into LIWC categories.
When translating, we keep the information about
the subject conveyed by the Italian verbs (e.g.,
’penso’ is translated as ’I think’). The coding
is performed manually by the authors: in a first
round, one rater associates each word with the
corresponding LIWC category; then, the other
revises the annotation, checking for consistency and
verifying also the correctness of the translation.
22 words are discarded and replaced with others
from the dictionary because we could not find a
mapping with any of the categories. Furthermore,
we add an ad hoc category to enable modeling of
words from the social media domain (retweet,
follow).</p>
        <p>Figure 1 shows how the happy and sad words
distribute along the dimensions associated with the
most frequent categories. Sample words for each
word category are reported in Table 2. We observe
that happy words in the dictionary mainly refer to
positive emotions as well as to the social and social
media dimensions. Conversely, sad words mainly
describe negative emotions with focus on the
author. Words describing cognitive mechanisms are
also associated with sadness.
As observed in the original study, happiness is not
constant in our life and different degrees of
happiness might be observed at different moments in
time. As such, we analyze how happiness changes
over time. In particular we take into account the
days of the week and the different hours in a day.
For this analysis, we exploit the whole corpus
of 400M tweets and we compute the distribution
(a) Happiness load by day of the week
(b) Happiness load for a 24-hour day
of words occurring in the happiness dictionary in
each different time period. Using this strategy, in
each time period the word has an happiness load
obtained by multiplying its frequency in that
period by its happiness factor. The happiness load
of each time period is the average of all the
happiness load in that period. The obtained values are
mapped in the interval [-1, 1] and plotted in Figure
2a (for days) and in Figure 2b (for hours).</p>
        <p>
          Our time-wise analysis reveals a drop in
happiness on Thurdsay, with a subsequent twist towards
positive mood on Friday, before the weekend that
is the happiest moment in the week. This is
consistent with the findings of the original study
reporting mid-week blues around Wednesday and a
happiness peak on Saturday
          <xref ref-type="bibr" rid="ref9">(Mihalcea and Liu, 2006)</xref>
          .
Regarding the hours, we observe the highest
happiness load in the morning, with a peak around 6
AM, and it constantly decreases over the day, with
the lowest value observed around 11 PM.
4
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Concept analysis</title>
      <p>We are interested in concepts related to words in
the happiness dictionary. In the original study, the
authors extract the ’ingredients’ for their recipe of
happiness by ranking the most relevant 2- and
3grams from their corpus according to their
happiness load. Such an approach is not easy to
replicate as the number of 2- and 3-grams extracted
from 400M tweets is potentially huge. Hence,
starting from the words in our happiness
dictionary, we extract the most 50 co-occurring words
in a window of two words. Then we rank all the
word pairs (dictionary word, co-occurring word)4
according to the Pointwise Mutual Information
(PMI) multiplied by the happiness factor. Table
3 reports some of the most happy and sad pairs.</p>
      <p>
        Starting from word pairs, we perform another
kind of analysis aiming at mapping the words
occurring in each pair with super-senses in WordNet.
A super-sense is a general semantic taxonomy
defined by the WordNet lexicographer classes as a
way for defining logical aggregation of senses in
each syntactic category. We assign a happiness
score to each super-sense by averaging the
happiness factor associated with the dictionary word
in the pair. Since each pair contains a dictionary
word and a co-occurring word, we map the
cooccurring word to its super-sense and increment
the score of the super-sense by summing the
happiness factor associated with the dictionary word.
Finally, the score of each super-sense is divided
by the number of the co-occurring words
belonging to the super-sense. For ambiguous words, we
select the super-sense associated with the most
frequent sense. In this study, we do not rely on
a Word Sense Disambiguation (WSD) algorithm
since WSD is a critical task. We need to test
the WSD performance on tweets before to use
it. Generally, WSD algorithms give performance
slightly above the most frequent sense. We plan
to test WSD in a further study. As super-senses
are defined in the English version of WordNet, we
4We do not take into account the word order in the pairs.
performed a mapping of Italian words to the
English WordNet through the use of both Morph-it!
        <xref ref-type="bibr" rid="ref16 ref4">(Zanchetta and Baroni, 2005)</xref>
        and MultiWordNet
        <xref ref-type="bibr" rid="ref12">(Pianta et al., 2002)</xref>
        , while sense occurrences are
extracted from MultiSemCor
        <xref ref-type="bibr" rid="ref16 ref4">(Bentivogli and
Pianta, 2005)</xref>
        .
      </p>
      <p>
        In Table 4 we report the most happy and
sad super-senses with the most frequent words
extracted by our corpus. Consistently with
the evidence provided by the analysis of the
psycholinguistic word categories (see Section
2.3), we observe that socialness is
associated with positive feelings, with concepts
referring to people (noun.person) and communication
(verb.communication, noun.communication)
scoring high in happiness. Food (noun.food) also
seems to be a major cause of positive mood, as
well as money and gifts (noun.possession), sport
achievements (’vittoria and ’gol’ in noun.act),
and mundane locations and events (’centro’,
’piazza’, ’concerto’, ’viaggio’ in noun.location and
noun.act). This is consistent with suggestion by
        <xref ref-type="bibr" rid="ref9">(Mihalcea and Liu, 2006)</xref>
        to enjoy food and drinks
in an ’interesting social place’ as a recipe for
happiness. People also report their desires and
preferences (voglio, amo, spero in verb.emotion).
      </p>
      <p>Also for sadness, results confirm findings
emerging from the analysis of
psycholinguistic categories in LIWC. In fact, we
observe that people tend to report their own
individual negative feelings (rido, piango in
verb.body), thoughts (verb.cognition),
perceptions (e.g., ’vedo’, ’sento’), and personal needs
(’bisogno’ and ’sonno’ in noun.state). We observe
also stereotypical complaints about weather
(piove) as well as swear words (noun.body).
5</p>
    </sec>
    <sec id="sec-4">
      <title>Discussion and Conclusions</title>
      <p>We performed an exploratory analysis of the
lexicon and concepts associated with happiness in
Italian tweets. We leveraged a corpus of happy
and sad tweets to extract a ”happiness dictionary’,
which we use to perform a time-wise analysis of
happiness on Twitter and to extract the most
frequent concepts and psycholinguistic categories
associated to positive and negative emotions.</p>
      <p>
        This study is a partial replication of the
previous one by
        <xref ref-type="bibr" rid="ref9">(Mihalcea and Liu, 2006)</xref>
        on blog
posts. The main differences with respect to the
original study are in the size, language and source
of the corpus used for extracting the happiness
most frequent concepts
resto, ricambio
cena, pranzo, colazione, caffe´
coraggio, voce, numero, bellezza, splendore, silenzio
mamma, ragazz*, amic*, dio, tesoro, donna
dico(no), parlare, prego, profilo, parla, chiedere
film, scusa, merda, musica, buongiorno, canzone, concerto
trov*, dare, perdere, perso, averti, comprato
voglio/vorrei, amo, piace, vuoi, spero, odio, auguri
sito, centro, post, piazza, scena, sud, nord, regione
soldi, regalo, fondo
vittoria, gara, onda, campagna, scarica, fuoco, episodio, meraviglia
cose, partita, gol, colpa, ricerca, viaggio, tour, bacio, corso, sesso
bisogna, mangiare, usare, mangio/mangiato, usa/o, usato, mangio
piangere, dormire, ridere, sveglia, sorridere, piango, rido
swear words, testa, occhi, mano/i, capelli
inizio/inizia(re), cambiare, finito, morire/morte, successo, finisce
vedere, vedo, sento, sentire, guarda, guardare, ascoltare, pare
so, sai, penso, letto, credo, sa, leggere, sapere, pensare, studiare
bisogno, punto, problemi/a, accordo, pace, crisi, situazione, sonno
aria, acqua
piove
lexicon. Specifically,
        <xref ref-type="bibr" rid="ref9">(Mihalcea and Liu, 2006)</xref>
        rely on a collection of 10,000 blog posts in
English from LiveJournal.com to extract a list of
happy/sad words with their associated happiness
scores, while we leverage a bigger corpus
consisting of 8.6M Italian tweets. Furthermore, the blog
posts were labeled as happy or sad by their
authors. Conversely, for tweets we relied on silver
labeling based on the presence of emoticons as a
proxy the author self-reporting of her own positive
or negative emotions.
      </p>
      <p>Our analysis of psycholinguistic categories and
the extraction of concepts and WordNet
supersenses associated with them reveals interesting
findings. Happiness appears related to the
social aspects of life while sad tweets mainly
revolves around self-centered negative feelings and
thoughts. In addition, our-time wise analysis
reveals a mid-week drop in happiness also observed
in the original study. We also observe that
happiness is high in the morning and decreases over
the day. As a future work, it would be interesting
to investigate if time-wise analysis based on hours
produces consistent results if a weekday or the
weekend is considered and if emotion-triggering
concepts associated with happiness also vary over
time.</p>
      <p>
        We are aware of the main limitations of this
study. First of all, by relying on microblogs we
are probably able to mine emotion triggers that
do not necessarily coincide with those shared in
daily face-to-face conversations or reported in
private logs. Furthermore, we do not attempt to make
any categorization of the authors of tweets.
Indeed, different target user groups could be studied
to fulfill specific research goals and enable
perspective applications, i.e. for supporting creative
writing or for providing personalized
recommendations based on moods. Finally, we consider only
Twitter as a source of data. The same methodology
could produce different results if applied to other
social media. Indeed, recent research
        <xref ref-type="bibr" rid="ref2 ref8">(Andalibi et
al., 2017)</xref>
        showed that other media, such as
Instagram, are also used for sharing extremely private
emotions, such as feelings linked to depression.
      </p>
      <p>
        Based on these observations, further replications
could focus on finer-grained emotions, also
leveraging corpora from different platforms and
including consideration of demographics and
geographical information
        <xref ref-type="bibr" rid="ref1 ref1 ref10 ref10">(Mitchell et al., 2013; Allisio et
al., 2013)</xref>
        as additional dimensions of analysis.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [Allisio et al.2013]
          <string-name>
            <given-names>Leonardo</given-names>
            <surname>Allisio</surname>
          </string-name>
          , Valeria Mussa, Cristina Bosco, Viviana Patti, and
          <string-name>
            <given-names>Giancarlo</given-names>
            <surname>Ruffo</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Felicitta`: Visualizing and estimating happiness in italian cities from geotagged tweets</article-title>
          .
          <source>In Proceedings of the First International Workshop on Emotion and Sentiment in Social and Expressive Media: approaches and perspectives from AI</source>
          (ESSEM
          <year>2013</year>
          )
          <article-title>A workshop of the XIII International Conference of the Italian Association for Artificial Intelligence (AI*IA</article-title>
          <year>2013</year>
          ), Turin, Italy, December 3,
          <year>2013</year>
          ., pages
          <fpage>95</fpage>
          -
          <lpage>106</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [Andalibi et al.2017]
          <string-name>
            <given-names>Nazanin</given-names>
            <surname>Andalibi</surname>
          </string-name>
          , Pinar Ozturk, and
          <string-name>
            <given-names>Andrea</given-names>
            <surname>Forte</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Sensitive self-disclosures, responses, and social support on instagram: The case of #depression</article-title>
          .
          <source>In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, CSCW '17</source>
          , pages
          <fpage>1485</fpage>
          -
          <lpage>1500</lpage>
          , New York, NY, USA. ACM.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <source>[Basile and Nissim2013] Valerio Basile and Malvina Nissim</source>
          .
          <year>2013</year>
          .
          <article-title>Sentiment analysis on Italian tweets</article-title>
          .
          <source>In Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis</source>
          , pages
          <fpage>100</fpage>
          -
          <lpage>107</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <source>[Bentivogli and Pianta2005] Luisa Bentivogli and Emanuele Pianta</source>
          .
          <year>2005</year>
          .
          <article-title>Exploiting parallel texts in the creation of multilingual semantically annotated resources: the multisemcor corpus</article-title>
          .
          <source>Natural Language Engineering</source>
          ,
          <volume>11</volume>
          (
          <issue>3</issue>
          ):
          <fpage>247</fpage>
          -
          <lpage>261</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [Bollen et al.2009]
          <string-name>
            <given-names>Johan</given-names>
            <surname>Bollen</surname>
          </string-name>
          , Alberto Pepe, and
          <string-name>
            <given-names>Huina</given-names>
            <surname>Mao</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena</article-title>
          .
          <source>CoRR, abs/0911</source>
          .1583.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [Garimella et al.2016]
          <string-name>
            <given-names>Kiran</given-names>
            <surname>Garimella</surname>
          </string-name>
          , Michael Mathioudakis,
          <string-name>
            <surname>Gianmarco De Francisci Morales</surname>
            , and
            <given-names>Aristides</given-names>
          </string-name>
          <string-name>
            <surname>Gionis</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Exploring controversy in twitter</article-title>
          .
          <source>In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion, CSCW '16 Companion</source>
          , pages
          <fpage>33</fpage>
          -
          <lpage>36</lpage>
          , New York, NY, USA. ACM.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [Go et al.2009]
          <string-name>
            <given-names>Alec</given-names>
            <surname>Go</surname>
          </string-name>
          , Lei Huang, and
          <string-name>
            <given-names>Richa</given-names>
            <surname>Bhayani</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>Twitter sentiment analysis</article-title>
          .
          <source>Entropy</source>
          ,
          <volume>17</volume>
          :
          <fpage>252</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [Joseph et al.2017]
          <string-name>
            <given-names>Kenneth</given-names>
            <surname>Joseph</surname>
          </string-name>
          , Wei Wei, and
          <string-name>
            <surname>Kathleen</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Carley</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Girls rule, boys drool: Extracting semantic and affective stereotypes from twitter</article-title>
          .
          <source>In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing</source>
          ,
          <string-name>
            <surname>CSCW</surname>
          </string-name>
          <year>2017</year>
          ,
          <article-title>Portland</article-title>
          ,
          <string-name>
            <surname>OR</surname>
          </string-name>
          , USA,
          <source>February 25 - March 1</source>
          ,
          <year>2017</year>
          , pages
          <fpage>1362</fpage>
          -
          <lpage>1374</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <source>[Mihalcea and Liu2006] Rada Mihalcea and Hugo Liu</source>
          .
          <year>2006</year>
          .
          <article-title>A corpus-based approach to finding happiness</article-title>
          .
          <source>In Proc. AAAI Spring Symposium and Computational</source>
          Approaches to Weblogs, page 6 pages.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [Mitchell et al.
          <year>2013</year>
          ] Lewis Mitchell, Morgan R. Frank, Kameron Decker Harris, Peter Sheridan Dodds, and
          <string-name>
            <surname>Christopher</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Danforth</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>The geography of happiness: Connecting twitter sentiment and expression, demographics, and objective characteristics of place</article-title>
          .
          <source>PLOS ONE</source>
          ,
          <volume>8</volume>
          (
          <issue>5</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>15</lpage>
          ,
          <fpage>05</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [Pennebaker and Francis2001]
          <string-name>
            <given-names>J.</given-names>
            <surname>Pennebaker</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Francis</surname>
          </string-name>
          .
          <year>2001</year>
          .
          <article-title>Linguistic inquiry and word count: Liwc</article-title>
          . Mahway: Lawrence Erlbaum Associates,
          <fpage>71</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [Pianta et al.2002]
          <string-name>
            <given-names>Emanuele</given-names>
            <surname>Pianta</surname>
          </string-name>
          , Luisa Bentivogli, and
          <string-name>
            <given-names>Christian</given-names>
            <surname>Girardi</surname>
          </string-name>
          .
          <year>2002</year>
          .
          <article-title>Multiwordnet: developing an aligned multilingual database. 1st gwc</article-title>
          .
          <source>In Proceedings of the First International Conference on Global WordNet.</source>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [Read2005]
          <string-name>
            <given-names>Jonathon</given-names>
            <surname>Read</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>Using emoticons to reduce dependency in machine learning techniques for sentiment classification</article-title>
          .
          <source>In Proceedings of the ACL student research workshop</source>
          , pages
          <fpage>43</fpage>
          -
          <lpage>48</lpage>
          . Association for Computational Linguistics.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [Roberts et al.2012]
          <article-title>Kirk Roberts, Michael A</article-title>
          .
          <string-name>
            <surname>Roach</surname>
          </string-name>
          , Joseph Johnson, Josh Guthrie, and
          <string-name>
            <surname>Sanda</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Harabagiu</surname>
          </string-name>
          .
          <year>2012</year>
          .
          <article-title>EmpaTweet: Annotating and Detecting Emotions on Twitter</article-title>
          . In Nicoletta C. Chair, Khalid Choukri, Thierry Declerck, Mehmet U. Dou gan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis, editors,
          <source>Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12)</source>
          , Istanbul, Turkey, May.
          <source>European Language Resources Association (ELRA).</source>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [Tausczik and Pennebaker2010]
          <string-name>
            <surname>Yla</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Tausczik</surname>
            and
            <given-names>James W.</given-names>
          </string-name>
          <string-name>
            <surname>Pennebaker</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>The psychological meaning of words: Liwc and computerized text analysis methods</article-title>
          .
          <source>Journal of Language and Social Psychology</source>
          ,
          <volume>29</volume>
          (
          <issue>1</issue>
          ):
          <fpage>24</fpage>
          -
          <lpage>54</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>[Zanchetta and Baroni2005] Eros Zanchetta and Marco Baroni</source>
          .
          <year>2005</year>
          .
          <article-title>Morph-it!: a free corpus-based morphological resource for the italian language</article-title>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>