<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Exploring Sentiment in Social Media and O Statistics: a General Framework cial</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Q.R.S. soc. coop. V.le C. Battisti 15</institution>
          ,
          <addr-line>13900 Biella</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Universita degli Studi di Torino, Dipartimento di Informatica c.</institution>
          <addr-line>so Svizzera 185, I-10149 Torino</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The integration between o cial statistics and social media data is a challenging topic. This contribution aims to present a recentlydesigned framework to compare sentiment analysis on social media content with social and economic data. Such framework - which has already been applied, in a preliminary fashion, to the Felicitta project - is meant to integrate o cial statistics and correlate it with online social media data. Its ultimate goal, in fact, namely consists in giving a contribution to the de nition of a measure of subjective well-being that could fully bene t from both traditional, well-established social indicators and dynamic data obtained from the web.</p>
      </abstract>
      <kwd-group>
        <kwd>Subjective Well-Being</kwd>
        <kwd>Sentiment Analysis</kwd>
        <kwd>O cial Statistics</kwd>
        <kwd>Social Media</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>The signi cant growth of user-generated content on the web, and in
particular the increased availability of data from online social media, has fostered the
development of automatic techniques for the extraction and processing of such
content for di erent purposes. This development is re ected, among other things,
by the spread of scienti c contests whose main track is the Sentiment Analysis
(henceforth SA) of texts in di erent languages (see eg. SemEval1 for English,
and SENTIPOLC@EVALITA20142 for Italian).</p>
      <p>In turn, these achievements are encompassed into a broader and
interdisciplinary debate related to the study and de nition of measures that could be
considered as reliable indicators of the well-being of a community. In fact, a
growing debate has recently involved the measurement of social and individual
well-being. New statistical measures have been proposed besides the bare Gross
Domestic Product (GDP), traditionally seen as the best way to measure national</p>
    </sec>
    <sec id="sec-2">
      <title>1 http://alt.qcri.org/semeval2014/task9/</title>
      <p>2 http://www.di.unito.it/~tutreeb/sentipolc-evalita14
economic results. Among such measures are a large amount of indicators that,
in several ways and from di erent points of view, attempt to assess the degree of
\happiness" and life satisfaction, also designated with the expression Subjective
Well-Being, or simply SWB (see Section 2). Such measures are usually provided
by governmental institutions or entitled research organizations, and they
generally include social indicators measuring life quality and concerning all major
areas of citizens' lives. However, such data are static and their recovery may
require much e orts in terms of time and resources. Moreover, the increasing
success of Sentiment Analysis (SA) techniques on social media has made it
possible to develop alternative tools and measures, with respect to the latter, to
assess the degree of happiness and well-being. Social media and their content
can thus be used to complement and corroborate the information gathered from
traditional data sources as regards SWB detection.</p>
      <p>
        The work presented here is just part of this research context. In particular,
the purpose of this paper is to describe a framework whose entire de nition and
completion is still in progress, for the analysis and assessment of the degree of
\happiness" of a given community in Italy, taking into account and combining
together the information gathered from two main data sources: a) social media
content, and Twitter in particular; b) the socio-demographic information made
available by the main suppliers of o cial statistical data, such as the Italian
National Institute of Statistics (ISTAT)3. The rst point in particular has
actually been explored and developed under a recent project called Felicitta4 [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ],
i.e an online platform for estimating happiness in Italian cities that daily
analyzes Twitter posts and exploits temporal and geo-spatial information related to
tweets, in order to enable the summarization of SA outcomes.
      </p>
      <p>The present work is both an extension and a comprehensive reference
framework of that project. As a matter of fact, its aim is manifold and includes: 1) the
use and further development of techniques for the visualization of SA outcomes
in Italian texts; 2) the study of the correlations between o cial statistics and
user-generated media content; 3) providing a contribution to the debate on what
can be considered e ective and reliable indicators of social well-being.</p>
      <p>The remainder of the paper is structured as follows: Section 2 provides a brief
introduction to the notion of subjective well-being, summing up the more recent
work carried out on this matter while Section 3 describes the whole architecture
of the system, as currently conceived. Final remarks in Section 4 close the paper.
2</p>
      <sec id="sec-2-1">
        <title>Background and Related Work</title>
        <p>The present contribution covers the debate on the Subjective Well-Being as a
social indicator and sheds some light on happiness studies based on the
sentiment analysis of social media.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3 http://www.istat.it/it/ 4 http://www.felicitta.net/</title>
      <p>
        Subjective Well-Being. As stated in Diener [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], SWB includes re ective
cognitive evaluations about the quality of life, such as life and work satisfaction,
interest and engagement, and a ective reactions to life events, such as joy and
sadness. The common measurements of SWB are self-report methods and
surveys with questionnaires. Social indicators and life quality research is a speci c
eld of study grown over the years as witnessed, for instance, by the birth of
the review \Social Indicators Research" and underlined by the initiatives of the
Organization for Economic Co-operation and Development (OECD) since the
Nineties5. Namely the OECD recently proposed a survey-based measurement of
SWB at national level [16], as alternative to purely economic measures.
      </p>
      <p>
        The well-being of the population: from Easterlin to GNH. The Gross
Domestic Product (GDP) is today the main measure of the nation's economic activity.
However, since late 70's, a huge debate has grown over this measure [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
Easterlin [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] rst identi ed the paradox for which the increase of economic well-being in
wealthier countries has no further increases in subjective well-being [13]. As
alternative to GDP, new concepts have arisen as sustainable socio-economic
development, governance, environmental conservation and so on. Besides the OECD,
several organizations and countries take into account new measurements,
basically focused on the concept of happiness : see, for instance, the World Happiness
Report [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] in a recent United Nations initiative, or the Gross National Happiness
(G.N.H.) index developed by Bhuthan. In Italy, an inter-institutional initiative
proposes a set of indicators on \Equitable and Sustainable Well-Being(BES)"6.
      </p>
      <p>
        Social Media and Well-Being. The analysis of textual expressions in social
media contents on a Big-Data scale would o ers an opportunity to economists
and sociologists in the measurement of social well-being. There's an open debate
on the topic and several works already investigated this subject with contrasting
results. Wang et al. [20] examine Facebook's Gross National Happiness (FGNH)
indexes and Diener's Satisfaction with Life Scale (SWLS), and nally criticize
the idea that a well-being index can be based on the contents of a speci c online
social networks. Quercia et al.[17] explore the relationship between sentiment
expressed in Twitter messages and community socio-economic well-being and,
on the contrary, they found interesting correlations between sentiment and
general well-being. Kramer [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] proposed a metric to represent the overall emotional
health of the nation as a model of \Gross National Happiness". Our work aims to
improve these studies by the analysis of a set of more extensive o cial statistics,
better detailed in 3.2. Social media analysis also suggests the prediction of stock
market [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], and of collective mood state [15]. Emotions have been considered
with respect to social media and their dynamics [14] [12], also with geographical
concerns [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. We attempt to enrich and extend these studies by focusing on a
5 See e.g. the Better Life Index http://www.oecdbetterlifeindex.org/
6 The national statistical institute ISTAT and the National Council for Economics
and Labour (CNEL) propose the BES Index which analyzes the changes in quality
of life in Italy focusing on 12 di erent areas http://www.misuredelbenessere.it/
ner-grained administrative territorial division; as a matter of fact, our data
describe the situation not only at a national level, but also with respect to regions,
provinces and municipalities.
      </p>
      <p>
        The challenge of visualization. The lecture of patterns and trends from
spreadsheets or lists of numbers is a di cult task when we have to deal with large
amounts of data. An improvement is often obtained by the use of graphs. Shapes
and lines immediately create meanings and signi cance from data. In this way,
data visualization allows us to present trends, to discover what is often hidden [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]
and simplify the identi cation of patterns not easily detectable [21]. Several
different tasks can be spotted in the design of a visualization system [18]. Some
interesting works already dealed with social media data, highlighting aspects
of public sentiment in the web [19] or public interest information7. Such works
inspired, in their main principles, the design of our visualization module within
the framework.
3
      </p>
      <sec id="sec-3-1">
        <title>Framework Description</title>
        <p>The present framework aims to include several approaches and techniques in
order to detect the well-being of a community under a broader perspective. The
steps entailed in the design phase was: a) the de nition of the whole pipeline;
b) the selection of data from o cial statistics to be correlated with the analysis
performed by the SA module; c) the presentation of the most promising patterns
emerging from the comparison between social media data and o cial statistics.
In this section, we describe the general framework architecture with an overview
of its modules.
3.1</p>
        <sec id="sec-3-1-1">
          <title>Architecture</title>
          <p>The whole framework architecture, as shown in Figure 1, consists of 5 main
parts: Providers, Data Gathering, Data Analysis, Data Exposure and Data
Visualization.</p>
          <p>
            Providers. Providers are the data sources: i.e Twitter, from which we retrieve
the geolocated8 Italian tweets using the Stream API, and the various
sociodemographic data sources (detailed in Table 1), that return demographic and
socio-economic variables of di erent Italian administrative divisions.
Data gathering. This module is further divided in submodules, each one tackling
one particular task:
{ the Collect submodule collects data from di erent providers;
7 http://twitter.github.io/interactive/sotu2015/
8 For details on the geolocalization methods used, see [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]
{ the Filter submodule lters the collected data in order to remove all the
possible noisy data, such as duplicate records, empty voices, characters
instead of numbers and other formatting errors; as possible correlations have
been observed between sentiment and time of the day or day of the week
(weekdays or holidays), or between sentiment and geographical areas in a
given time frame due to the occurrence of some special event, during this
step, we also intend to add a further lter that leaves out all the tweets
that bear such temporal or geographical bias9, as already made in [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ], in the
creation of the validation corpus.
{ the Homologate submodule is devoted to the proper organization of collected
and ltered data into a uni ed format. For example, 1420070400, 01/01/2015
and Thu, 01 Jan 2015 00:00:00 GMT indicate the same date, and 058091,
[41.53,12.28] and Roma indicate the same city. The Homologate submodule
converts dates in YYYY/MM/DD format, and administrative divisions in
the ISTAT code10.
          </p>
          <p>
            Data Analysis. First, the Sentiment Analysis submodule returns for each
tweet a mood value (positive, negative, neutral); the SA engine is the one
developed in Felicitta, as described in [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ]. Then, the Mash-Up submodule aggregates
Italian geolocated tweets by regions, provinces and municipalities. In this way,
data about moods and social indicators can be grouped on the basis of the same
9 Indeed, conventional expressions such as \Happy New Year ", \Merry Christmas",
and others, should not be considered as equally representative of, for example, joy.
10 http://www.istat.it/it/archivio/6789
period and the same administrative level. The aggregate data are nally stored
in a database. A correlation analysis across moods and, in turn, each statistic
is performed, in order to quantify the strength of the relationship between the
variables. As further detailed in 3.2, this is the most recent part of the project,
that extends the one implemented in Felicitta.
          </p>
          <p>Data Exposure. A web server exposes elaborated data by REST API. When a
client runs a query, the server queries the database and returns the response.</p>
          <p>Data Visualization. Finally, a web client presents the data obtained as response
to the queries. For the time being, the visualization module allows to browse
either the sentiment data (as in the example in Figure 2), or the sentiment data
combined with demographic data, as shown in Figure 3. The part that shows
socio-demographic statistics and correlations is yet to be completed.
3.2</p>
        </sec>
        <sec id="sec-3-1-2">
          <title>Statistics</title>
          <p>As a measure of the mood related to an area in a given period, we consider
the percentage of positive tweets. In order to relate moods and numeric social
indicators in di erent administration degrees, in Table 1 we summarized some
social indicators that could provide an overview of the social well-being of a
given community.</p>
          <p>As data collection is not always an easy task and the Open Data is not
yet widespread in Italian public administration, we realistically decided to focus
our attention on data that could be easily accessed and retrieved from public
administration web-sites. In order to detail di erent aspects of the society, we
resort to di erent sources. In this way, we consider data from di erent elds and
viewpoints, mainly demographic and economic.</p>
          <p>As regards the demographic eld, the main aspects considered are nationality,
gender, age and marital status, since they are closely related to the perception
of social well-being. We are interested, for example, in understanding whether
and to what extent nationality may in uence the sentiment expressed through
social media, or whether married men are happier than singles.</p>
          <p>Concerning the economic eld, we consider both jobs data (e.g. the
unemployement rate) and data about companies (e.g. enterprises demography). Our
hypothesis is that people express negative sentiments more likely if they live in an
area with signi cant unemployment rate or with a greater amount of cessations
of business.</p>
          <p>We also collected data about the real estate market, that we consider a typical
indicator of the wealth of a territory. A correlation, in fact, is expected between
this aspect and the overall mood detected in social media: the higher the prices
(then the wealthier the area considered) and the greater the happiness may be.
Similarly, we consider the amount of deposit and loan from the Bank of Italy
as a measure of both individuals and public wealth. We selected this set of data
as they are representative of di erent relevant social needs, with di erent time
and granularity. A rst integration between social media data and demographic
data is shown in Figure 3.</p>
          <p>Our current work then namely consists in exploring all the possible
correlations between the indicators mentioned above and the output of the SA engine,
and in improving the visualization module so as to better highlight such
correlations and emerging patterns.
4</p>
        </sec>
      </sec>
      <sec id="sec-3-2">
        <title>Conclusions and future work</title>
        <p>
          In this paper we introduced an ongoing project on a framework for the analysis
and assessment of the degree of \happiness" of a given Italian community,
taking into account data from o cial statistics and SA data obtained from social
media. We noticed at least two main problems: the representativeness of data
and the role of ironic sentences. First, the di usion of internet and the use of
online social networks is not widespread in the same way over all kinds of
population. Therefore, for instance, the sentiment of poorest people and elderly can
be not represented or largely underrepresented. This is a classical problem of
quite every sociological inquiries, mainly solved by representative sampling and
qualitative research. A second issue is the presence of irony where the unintended
meaning of words can often reverse the polarity of the message. We well know
this problem and we state how exists a growing interest in this research subject,
as we already investigate the role and the detection of irony and sarcasm[
          <xref ref-type="bibr" rid="ref9">9</xref>
          ]. As
mentioned above, the work is still in progress and some issues limit the results,
but there are also several expected positive impacts of the proposed approach.
First, we focus on the selection of data from o cial statistics that better
correlate with social media data. An hypothesis is that a variation in the data on
the labor market and, most of all, the youth employment situation in a given
region entail a variation in the mood of the public opinion as expressed in online
social media. Detecting the strength of the statistical relation between di erent
variables could help in using social media as a tool for detection of social and
economic trends. Another relevant concrete application of the present framework
is the inclusion in the platform of Felicitta of the selected statistical data with
the output emerging from the correlation analysis.
        </p>
        <p>Acknowledgments. Part of the present contribution has been awarded in the
2014 Istat-Google Contest on \Producing o cial statistics with Big Data": http:
//www.istat.it/it/archivio/144042
12. A. Kramer. The spread of emotion via facebook. In Proceedings of the SIGCHI</p>
        <p>Conference on Human Factors in Computing Systems, pages 767{770. ACM, 2012.
13. E.W.Dunn L.B.Aknin, M.Norton. From wealth to well-being? money matters, but
less than people think. The Journal of positive psychology, 4(6):523{527, 2009.
14. L. Mitchell, M. Frank, K. Harris, P. Dodds, and M. Danforth. The geography of
happiness: Connecting twitter sentiment and expression, demographics, and
objective characteristics of place. PLoS ONE, 2013.
15. T. Nguyen, B. Dao, D. Phung, S. Venkatesh, and M. Berk. Online social capital:
Mood, topical and psycholinguistic analysis. Proceedings of the Seventh
International AAAI Conference on Weblogs and Social Media, 2013.
16. OECD. OECD Guidelines on Measuring Subjective Well-being. OECD Publishing,</p>
        <p>Paris, 2013.
17. D. Quercia, J. Ellis, L. Capra, and J. Crowcroft. Tracking gross community
happiness from tweets. Proceedings of the ACM 2012 conference on Computer Supported
Cooperative Work (CSCW '12), 2012.
18. Ben Shneiderman. The eyes have it: A task by data type taxonomy for information
visualizations. In Visual Languages, 1996. Proceedings., IEEE Symposium on,
pages 336{343. IEEE, 1996.
19. C.o Wang, Z. Xiao, Y. Liu, Y. Xu, A. Zhou, and K. Zhang. Sentiview: Sentiment
analysis and visualization for internet popular topics. Human-Machine Systems,
IEEE Transactions on, 43(6):620{630, 2013.
20. Kosinski M. Stillwell D.J. Wang, N. and J. Rust. Can well-being be measured
using facebook status updates? validation of facebook's gross national happiness
index. Social indicators research, pages 483{491, 2014.
21. N. Yau. Visualize this: the FlowingData guide to design, visualization, and
statistics. Wiley Publishing, 2011.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>L.</given-names>
            <surname>Allisio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Mussa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bosco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Patti</surname>
          </string-name>
          , and
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Ru o. Felicitta: Visualizing and estimating happiness in italian cities from geotagged tweets</article-title>
          .
          <source>In Proceedings of the First International Workshop on Emotion and Sentiment in Social and Expressive Media: approaches and perspectives from AI</source>
          (ESSEM
          <year>2013</year>
          ), pages
          <fpage>95</fpage>
          {
          <fpage>106</fpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>J.</given-names>
            <surname>Bollen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Mao</surname>
          </string-name>
          , and
          <string-name>
            <given-names>X.J.</given-names>
            <surname>Zeng</surname>
          </string-name>
          .
          <article-title>Twitter mood predicts the stock market</article-title>
          .
          <source>Journal of Computational Science</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>C.</given-names>
            <surname>Bosco</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Allisio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Mussa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Patti</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Ru o, M. Sanguinetti, and</article-title>
          <string-name>
            <given-names>E.</given-names>
            <surname>Sulis</surname>
          </string-name>
          .
          <article-title>Detecting happiness in Italian Tweets: Towards an evaluation dataset for sentiment analysis in Felicitta</article-title>
          .
          <source>In Proceedings of the 5th International Workshop on EMOTION</source>
          ,
          <string-name>
            <surname>SOCIAL</surname>
            <given-names>SIGNALS</given-names>
          </string-name>
          ,
          <article-title>SENTIMENT &amp; LINKED OPEN DATA, (ESLOD</article-title>
          <year>2014</year>
          ), pages
          <fpage>56</fpage>
          {
          <fpage>63</fpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>A.</given-names>
            <surname>Cairo</surname>
          </string-name>
          .
          <article-title>The Functional Art: An introduction to information graphics and visualization</article-title>
          . New Riders,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <given-names>E.</given-names>
            <surname>Diener</surname>
          </string-name>
          .
          <article-title>Subjective well-being: The science of happiness and a proposal for a national index</article-title>
          .
          <source>American psychologist</source>
          ,
          <volume>55</volume>
          (
          <issue>1</issue>
          ):
          <fpage>34</fpage>
          ,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>P.S.</given-names>
            <surname>Dodds</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.D.</given-names>
            <surname>Harris</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.A.</given-names>
            <surname>Kloumann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.M.</given-names>
            and
            <surname>Bliss</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.M.</given-names>
            <surname>Danforth</surname>
          </string-name>
          .
          <article-title>Temporal patterns of happiness and information in a global-scale social network: Hedonometrics and twitter</article-title>
          .
          <source>PLoS ONE</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>R.</given-names>
            <surname>Easterlin</surname>
          </string-name>
          .
          <article-title>Does money buy happiness? The Public Interest</article-title>
          ,
          <year>1973</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <given-names>M.</given-names>
            <surname>Forgeard</surname>
          </string-name>
          , E. Jayawickreme,
          <string-name>
            <given-names>M.</given-names>
            <surname>Kern</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Seligman</surname>
          </string-name>
          .
          <article-title>Doing the right thing: Measuring wellbeing for public policy</article-title>
          .
          <source>International Journal of Wellbeing</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <given-names>D.</given-names>
            <surname>Hernandez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Sulis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Patti</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Ru o, and</article-title>
          <string-name>
            <given-names>C.</given-names>
            <surname>Bosco</surname>
          </string-name>
          . Valento:
          <article-title>Sentiment analysis of gurative language tweets with irony and sarcasm</article-title>
          .
          <source>Proc. Int. Workshop on Semantic Evaluation (SemEval-2015)</source>
          ,
          <article-title>Co-located with NAACL and *SEM</article-title>
          . To appear.,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>J. Sachs J. Helliwell</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Layard</surname>
          </string-name>
          and Emirates Competitiveness Council.
          <source>World happiness report 2013. Sustainable Development Solutions Network</source>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>A.</given-names>
            <surname>Kramer</surname>
          </string-name>
          .
          <article-title>An unobtrusive behavioral model of gross national happiness</article-title>
          .
          <source>In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems</source>
          , pages
          <fpage>287</fpage>
          {
          <fpage>290</fpage>
          . ACM,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>