English. This article describes a Twitter corpus of social media contents in the Subjective Well-Being domain. A multilayered manual annotation for exploring attitudes on fertility and parenthood has been applied. The corpus was further analysed by using sentiment and emotion lexicons in order to highlight relationships between the use of affective language and specific sub-topics in the domain. This analysis is useful to identify features for the development of an automatic tool for sentiment-related classification tasks in this domain. The gold standard is available to the community.
Italiano. L’articolo descrive la creazione
di un corpus tratto da Twitter sui temi del
Subjective Well-Being, fertilita` e
genitorialita`. Un’analisi lessicale ha mostrato il
legame tra l’uso di linguaggio affettivo e
specifiche categorie di messaggi. Questo
esame e` utile per se e per l’addestramento
di sistemi di classificazione automatica sul
dominio. Il gold standard e` disponibile su
richiesta.
are the target of sentiment in the fertility-SWB
domain. The relationship between big data and
official statistics is increasingly a subject of
attention
A number of filtering steps have been applied for selecting from TWITA14 a corpus of tweets where users talk about fertility and parenthood (TW-SWELLFER corpus, henceforth). We could not rely on the exploitation of one or few hashtags or other elements that allow identifying posts on fertility and parenthood. In fact, these topics are somehow spread in the dataset and messages may contain relevant information on such subjects even if the main topic of the post is different. Therefore, we are facing a situation where, on the one hand, the set of the data that are potentially relevant for our specific analysis is wider than usual; on the other hand, it is more difficult to identify the presence of information related to the topics we are interested in. This leaded us to adopt a multi-step thematic filtering approach. In a first step (Keyword-based filtering step), eleven hashtags1 and other 19 keywords have been chosen for selecting tweets of interest, including 8 roots to consider diminutives, singulars and plurals. This list is the result of a combination of a manual content analysis on 2,500 tweets sampled at completely random (taken as a starting point) and a linguistic analysis on synonyms. We obtain a total amount of 3.9 million tweets. A second filtering step consisted in removing noisy tweets from corpus (User-based filtering step), as the off-topic ones (messages not concerning individual expression on fertility and parenthood topics). Tweets posted by company/institutions/newspapers accounts have been deleted. Finally, duplicated tweets not marked as RT were deleted (Duplicate-based filtering step). The resulting TW-SWELLFER corpus consists of 2,760,416 tweets.
1#papa`, #mamma, #babbo, #incinta, #primofiglio, #secondofiglio, #futuremamme, #maternita`, #paternita`, #allattamento, #gravidanza 2.2
Given the TW-SWELLFER dataset, we developed
and applied an annotation model aimed at studying
not only the sentiment expressed in the tweets, but
also specific parenthood-related topics discussed
in Twitter that are the target of the sentiment.
To build our annotation model, we relied on a
standard annotation scheme on sentiment
polarity (POLARITY), by exploiting the same labels
POS, NEG, NONE and MIXED provided the
organizers of the shared task for sentiment analysis
in Twitter for Italian
TOBEPA - To be parents. This tag is introduced to mark when the user generically comments about his status of parent.
TOBESO - To be sons. This tag marks the sons point of view, i.e. on when the user is a son that comments on the parent-son relationship.
DAILYLIFE - Daily life. This tag marks messages commenting on recurring situation in everyday life for what concerns the relationship between parents and children.
JUDGOTHERPA - Judgment over other parents behaviour. The tag allows to mark comments on educations of children, for instance comments of behaviours which does not seems to be appropri - ated for the parent role.
FUTURE - Children’ future. This tag is used for tweets where parents do express sentiments about the future of children.
BECOMPA - To become parents. This tag is introduced to mark tweets where users speak about the prospect or fear of being parents. POL - Political side. This tag is introduced to mark tweets talking about laws having impact on being parents.
Finally, two additional tags (IN-TOPIC/OFFTOPIC) have been added to allow annotators to mark if the tweet is relevant. The addition of this tag was necessary because of the noise still present in the dataset. Moreover, in this way, the manual annotation will produce also data to be used in order to create a supervised topic classifier from the whole TW-SWELLFER corpus. This opens the way to the exploitation of the corpus for a finegrained sentiment analysis, by identifying different aspects and topics of the Twitter debate on parenthood and the sentiment expressed towards each aspect/topic. 2.3
A random sample of 5,566 tweets from
TWSWELLFER has been collected. On this sample
we applied crowdsourcing for manual annotation
via the Crowdflower platform already used in
literature
3We selected the CrowdFlower’s dynamic judgment
option: having the goal of collecting at least 3 reliable
annotations for each tweet, the system was collecting up to a
maximum of 5 annotations (to deal with cases when row’s
conselect the true label we used majority voting.
In-topic vs off-topic: manual annotation on this
aspect resulted in 2,355 in-topic tweets (42.3%)
and 3,136 off-topic (56.3%); the remaining 75
tweets were unknown or null (cases of
disagreement). Thanks to the preliminary filtering steps,
the proportion of in-topic tweets is pretty high
compared to common results from different
Twitter based content and opinion analysis
the end of the manual annotation process we collected 1,545 labeled with the same tags for all the layers.
Notice that in the analysis in the next section will report results also on tweets labeled as IN-TOPIC after the manual annotation (2,355), but where annotators did not agree on the polarity, irony and subtopics labels. We refer to those tweets as NULL messages.
Summarizing, the TWSWELLFER-GOLD corpus includes 1,545 IN-TOPIC tweets labeled with the same tags for all the layers (POLARITY, IRONY and SUBTOPICS). 3
Regarding IN-TOPIC tweets, the 26.4% has been labeled as positive and 22.3% as negative (See Fig.1), giving us a guidance on what might be the general feeling in Twitter about the research topics on happiness and parenthood. The irony issue is limited to a 15.7% of all the messages and negative irony prevails (10.1% of negative ironic tweets and 5.6% of positive ironic tweets), while neutral tweets are just the 8.3%. fidence score is low). In our jobs we set 0.7 as minimum accuracy threshold.
The amount of mixed tweets is limited to 1.2% (remaining 26% are labelled as NULL because of annotators disagreement). Regarding these results, it appears that positive and negative feelings towards family, parenthood and fertility appear more or less equally spread through Twitter Italy. Even if the positive posts are a little bit more than the negative ones, ironic tweets must be considered: most of them are negative ironic posts (i.e., insulting/damaging the target) balancing the slight difference between pure positive and negative tweets. Furthermore, this particular topic, combined with the Twitter nature which provides short direct message, discourages people to stand in the grey (neutral) area, as could happens in other cases: about the 90% of the tweets shows an explicit polarity, meaning people take a side and express their opinions.
Which are these opinions and about what? Going further with the analysis and looking also at the contents, so taking into consideration the “topic specification attribute and its values (Fig. 2), the largest category refers to sons tweets (TOBESO) (40.3%), in which children are discussing and posting about being children and/or about relating themselves with parents. Parents tag (TOBEPA) settles on 15% and becoming tag (BECOMEPA) on 10%. Remaining categories have minor impact, all being in between 1% and 6%. 3.1
The exam of the corpus includes a lexical analysis on different aspects of affect: sentiment and emotions. The distribution of terms in each group of messages reveals interesting patterns. Adopting sentiment lexical resources4 the whole polarity of messages is computed summing positive and negative terms. A normalization is finally performed, i.e. dividing the polarity value by the number of terms in each group. In particular, the four lexica considered count more positive terms in positive messages. Similarly, negative terms are more frequent in negative messages. Ironic messages reveal a similar pattern, even if smoothed. Table 1 presents some of these results.
In addition, the emotion lexicon indicates a
larger frequency of terms related to anger, sadness,
fear and disgust in negative messages than in
positive ones (See Fig. 3). On the contrary, positive
messages contain more terms related to joy,
anticipation and surprise. Some suggestions can be
derived in the comparison of polarity categories and
the corresponding ironic ones. For instance, terms
related to joy are more frequent in ironic negative
messages than in negative ones. It is an insight of
the polarity reversal phenomena, where a shift is
produced by the adoption of a seemingly positive
statement, to reflect a negative one
The analysis of topic specification messages reveals a positive polarity for messages concerning TOBEPA (to be parents), while BECOMEPA (to become parents) has a more negative polarity (See Table 1). Focusing on emotion lexicon, TOBEPA has an higher incidence of Joy words (Fig. 4).
Messages concerning educations of children
(JUDGOTHERPA) contain a high frequency of
anger and disgust term. The category TOBESO (to
4EmoLex
The contribution of this paper is the exploration of opinions and semantic orientation about fertility and parenthood by scrutinizing about 3 million Italian tweets. This analysis is useful to identify features for the development of an automatic system to address automatic classification tasks in this domain. The corpus is available to the community. Its development constitutes a first step and a precondition to a further analysis that can be applied on such contents in order to extract, from semantically enriched data, measures of SWB constructed in an indirect way. This will hopefully improve our understanding of attitudes on fertility and parenthood.
We are currently extending the corpus by exploring the very interesting debate around the “Fertility Day’s initiative” from the Italy’s Minister of Health Beatrice Lorenzin, which had a remarkable echo on social media such as Twitter, with a substantial number of (also sarcastic) messages with hashtag #fertilityday posted.
The authors gratefully acknowledge financial support from the European Research Council under the European ERC Grant Agreement n. StG313617 (SWELL-FER: Subjective Well-being and Fertility, P.I. Letizia Mencarini).