I Stand With You: Using Emojis to Study Solidarity in Crisis Events Sashank Santhanam1* , Vidhushini Srinivasan1 , Shaina Glass2 , Samira Shaikh1 1 Department of Computer Science, 2 Department of Psychology University of North Carolina at Charlotte * ssantha1@uncc.edu iors like solidarity then become necessary and essen- tial in overcoming ideological differences and finding Abstract common ground [Bau13], especially in the aftermath of crisis events (e.g. natural disasters). Recent so- We study how emojis are used to express sol- cial movements with a strong sense of online solidar- idarity in social media in the context of two ity have had tangible offline (real-world) consequences, major crisis events - a natural disaster, Hur- exemplified by movements related to #BlackLivesMat- ricane Irma in 2017 and terrorist attacks that ter, #MeToo and #NeverAgain [DCJSW16, Bur18]. occurred in November 2015 in Paris. Using There is thus a pressing need to understand how soli- annotated corpora, we first train a recurrent darity is expressed online and more importantly, how neural network model to classify expressions of it drives the convergence of a global public in OSNs. solidarity in text. Next, we use these expres- Furthermore, research has shown that emoticons sions of solidarity to characterize human be- and emojis are more likely to be used in socio- havior in online social networks, through the emotional contexts [DBVG07] and that they may serve temporal and geospatial diffusion of emojis. to clarify the message structure or reinforce the mes- Our analysis reveals that emojis are a power- sage content [MO17, DP17]. Riordan [Rio17] found ful indicator of sociolinguistic behaviors (sol- that emojis, especially non-face emojis, can alter the idarity) that are exhibited on social media as reader’s perceived affect of messages. While research the crisis events unfold. has investigated the use of emojis over communities and cultures [BKRS16, LF16] as well as how emoji use 1 Introduction mediates close personal relationships [KW15], the sys- tematic study of emojis as indicators of human behav- The collective enactment of online behaviors, includ- iors in the context of social movements has not been ing prosocial behaviors such as solidarity, has been undertaken. We thus seek to understand how emojis known to directly affect political mobilization and so- are used when people express behaviors online on a cial movements [Tuf14, Fen08]. Social media, due to global scale and what insights can be gleaned through its increasingly pervasive nature, permits a sense of the use of emojis during crisis events. Our work makes immediacy [Gid13] - a notion that produces high de- two salient contributions: gree of identification among politicized citizens of the web, especially in response to crisis events [Fen08]. • We make available two large-scale corpora1 , an- Furthermore, the multiplicity of views and ideologies notated for expressions of solidarity using muti- that proliferate on Online Social Networks (OSNs) ple annotators and containing a large number of has created a society that is increasingly fragmented emojis, surrounding two distinct crisis events that and polarized [DVVB+ 16, Sun18]. Prosocial behav- vary in time-scales and type of crisis event. • A framework and software for analyzing of how Copyright c 2018 held by the author(s). Copying permitted for emojis are used to express prosocial behaviors private and academic purposes. such as solidarity in the online context, through In: S. Wijeratne, E. Kiciman, H. Saggion, A. Sheth (eds.): Pro- the study of temporal and geospatial diffusion of ceedings of the 1st International Workshop on Emoji Under- emojis in online social networks. standing and Applications in Social Media (Emoji2018), Stan- ford, CA, USA, 25-JUN-2018, published at http://ceur-ws.org 1 https://github.com/sashank06/ICWSM_Emoji We anticipate that our approach and findings would and terrorist attacks in Paris, November 2015. We be- help advance research in the study of online human gin this section by briefly describing the two corpora. behaviors and in the dynamics of online mobilization. Irma Corpus: Hurricane Irma was a catastrophic Category 5 hurricane and was one of the strongest hur- 2 Related Work ricanes ever to be formed in the Atlantic2 . The storm Defining Solidarity: We start by defining what we caused massive destruction over the Caribbean islands mean by solidarity. The concept of solidarity has been and Cuba before turning north towards the United defined by scholars in relation to complementary terms States. People took to social media to express their such as “community spirit or mutual attachment, so- thoughts along with tracking the progress of the storm. cial cooperation or charity” [Bay99]. In our work, we To create our Irma corpus, we used Twitter streaming use the definition of expressional solidarity [Tay15], API to collect tweets with mentions of the keyword characterized as individuals expressing empathy and “irma” starting from the time Irma became an intense support for a group they are not directly involved in storm (September 6th , 2017) and until the storm weak- (for example, expressing solidarity for victims of nat- ened over Mississippi on September 12th , 2017 result- ural disasters or terrorist attacks). ing in a corpus of >16MM tweets. Using Emojis to Understand Human Behav- Paris Corpus: Attackers carried out suicide ior: With respect to research on expressional soli- bombings and multiple shootings near cafes and the darity, Herrera et al. found that individuals were Bataclan theatre in Paris on November 13th , 2015. more outspoken on social media after a tragic event More than 400 people were injured and over a hun- [HVBMGMS15]. They studied solidarity in tweets dred people died in this event3 . As a reaction to this spanning geographical areas and several languages re- incident, people all over the world took to social media lating to a terrorist attack, and found that hashtags to express their reactions. To create our Paris corpus, evolved over time correlating with a need of individu- we collected >2MM tweets from 13th November, 2015 als to speak out about the event. However, they did to 17th November, 2015 containing the word “paris” not investigate the use of emojis in their analysis. using the Twitter GNIP service4 . Extant research on emojis usage has designated three categories, that are 1) function: when emo- Annotation Procedure We performed distance jis replace a conjunction or prepositional word; 2) labeling [MBSJ09] by having two trained annota- content: when emojis replace a noun, verb, or ad- tors assign the most frequent hashtags in our corpus jective; and 3) multimodal: when emojis are used with one of three labels (“Solidarity” (e.g. #soli- to express an attitude, the topic of the message or daritywithparis, #westandwithparis, #prayersforpuer- communicate a gesture [NPM17]. [NPM17] found torico), “Not Solidarity” (e.g. #breakingnews, #face- that the multimodal category is the most frequently book ) and “Unrelated/Cannot Determine” (e.g. #re- used; and we contend that emojis used in the mul- bootliberty, #syrianrefugees). Using the hashtags that timodal function may also be most likely to demon- both annotators agreed upon (κ > 0.65, which is re- strate solidarity. Emojis are also widely used to con- garded as an acceptable agreement level) [SW05], we vey sentiment [HGS+ 17], including strengthening ex- filtered tweets that were annotated with conflicting pression, adjusting tone, expressing humor, irony, or hashtags from both corpora, as well as retweets and intimacy, and to describe content, which makes emojis duplicate tweets. Table 1 provides the descriptive (and emoticons) viable resources for sentiment analysis statistics of the original (not retweets), non-duplicate [JA13, NSSM15, PE15]. We use sentiment of emojis tweets, that were annotated as expressing solidarity to study the diffusion of emojis across time and region. and not solidarity based on their hashtags that we used To the best of our knowledge, no research to date for further analysis. has described automated models of detecting and clas- sifying solidarity expressions in social media. In ad- dition, research on using such models to further in- Table 1: Descriptive statistics for crisis event corpora vestigate how human behavior, especially a prosocial # of Tweets Solidarity Not Solidarity Total behavior like solidarity, is communicated through the Irma 12000 81697 93697 use of emojis in social media is still nascent. Our work Paris 20465 29874 50339 seeks to fill this important research gap. 3 Data Collection 2 https://tinyurl.com/y843u5kh Our analysis is based on social media text surround- 3 https://tinyurl.com/pb2bohv ing two different crisis attacks: Hurricane Irma in 2017 4 https://tinyurl.com/y8amahe6 4 The Emojis of Solidarity RQ2: Which emojis are used in expressions of solidarity during crisis events and how do The main goal of this article is to investigate how in- they compare to emojis used in other tweets? dividuals use emojis to express a prosocial behavior, To start delving into the data, in Table 4 we show in this case, solidarity, during crisis events. Accord- the total number of emojis in each dataset. We ob- ingly, we outline our analyses in the form of research serve that the total number of emojis in the tweets questions (RQs) and the resulting observations in the that express solidarity (using ground-truth human an- sections below. notation) is greater than the emojis in not solidar- RQ1: How useful are emojis as features in ity tweets, even though the number of not solidarity classifying expressions of solidarity? tweets is greater than the solidarity tweets in both cri- After performing manual annotation of the two cor- sis events (c.f. Table 1). We also observe that count of pora, we trained two classifiers for detecting solidarity emojis is greater in the Irma corpus than in the Paris in text. We applied standard NLP pre-processing tech- corpus, even though the number of solidarity tweets niques of tokenization, removing stopwords and lower- is smaller in the Irma corpus. One reason for this casing the tweets. We also removed hashtags that were could be that the Hurricane Irma event happened in annotated from the tweets. Class balancing was used 2017 when predictive emoji was a feature on platforms, in all models to address the issue of majority class im- while the Paris event occurred in 2015 when such func- balance (count of Solidarity vs. Not Solidarity tweets). tionality was not operational. Baseline Models: We used Support Vector Ma- chine (SVM) with a linear kernel and 10 fold cross validation to classify tweets containing solidarity ex- Table 4: Total number of emojis in each dataset pressions. For the baseline models, we experimented # of Emojis Solidarity Not Solidarity Total with three variants of features including (a) word bi- Irma 26197 25904 52101 grams, (b) TF-IDF [MPH08], (c) TF-IDF+Bigrams. Paris 24801 12373 37174 RNN+LSTM Model: We built a Recurrent Neural Network(RNN) model with Long Short-Term Memory (LSTM) [HS97] to classify social media posts Table 5: Top ten emojis by frequency and their counts into Solidarity and Not Solidarity categories. The in Irma and Paris corpora embedding layer of the RNN is initialized with pre- Rank Irma Irma Paris Paris trained GloVe embeddings [PSM14] and the network Sol. Not Sol. Sol. Not Sol. consists of a single LSTM layer. All inputs to the net- work are padded to uniform length of 100. Table 2 1 6105 2098 5376 2878 shows the hyperparameters of the RNN model. 2 2336 1827 2826 1033 Table 3 shows the accuracy of the baseline and 3 1977 1474 2649 909 RNN+LSTM models in classifying expressions of soli- 4 1643 1193 2622 779 darity from text, where the RNN+LSTM model with 5 1530 823 2581 760 emojis significantly outperforms the Linear SVM mod- 6 1034 794 2225 616 els in both Irma and Paris corpora. 7 934 726 1702 513 Table 2: RNN+LSTM model hyperparameters 8 820 725 386 510 Hyperparameters Value 9 625 724 340 433 Batch Size 25 10 367 683 259 361 Learning Rate 0.001 Epochs 20 To address RQ2, we show in Table 5 the top ten Dropout 0.5 most frequently used emojis across both crisis events in the tweets that express solidarity and those that do not. We observe that is used more frequently Table 3: Accuracy of the baseline SVM models and in the Irma solidarity tweets (Rank 3) but not in the RNN+LSTM model Irma tweets that do not express solidarity. In the top Accuracy Irma Paris 10 Irma emojis used in tweets not expressing solidar- RNN+LSTM (with emojis) 93.5% 86.7% ity, we also observe more negatively valenced emojis, RNN+LSTM (without emojis) 89.8% 86.1% including and . The emoji is interesting, since TF-IDF 85.71% 75.72% the prevailing meaning is “face with tears of joy”, how- TF-IDF + Bigrams 82.62% 76.98% ever this emoji can sometimes be used to express sad- Bigrams only 79.86% 75.24% ness [WBSD16]. In addition, is used across all four Figure 1: Cooccurrence network for emojis expressing Figure 2: Cooccurrence network for emojis expressing solidarity from regions affected by Hurricane Irma solidarity from regions not affected by Hurricane Irma sets, albeit at different ranks (e.g. Rank 1 in Irma solidarity and Rank 6 in Paris solidarity tweets). Table 6: Total count and proportion of emojis in geo- When comparing the two crisis events, we make tagged tweets from affected vs. other regions the observation that the top 5 ranked Paris solidar- Irma Paris ity emojis are flags of different countries, related to Affected Regions 10048 (67.81%) 925 (6.52%) expressions of solidarity from these countries, includ- Other Regions 4770 (32.19%) 13267 (93.48%) ing France ( ) at Rank 1, while appears at Rank 9 in the Irma solidarity set. We can thus observe that Figure 1 represents the co-occurrence network of even though the underlying behavior we study in these emojis within the regions affected by the Hurricane two events is solidarity, the top emojis used to express Irma (United States, Antigua and Barbuda, Saint such behavior are different in the two events. Dur- Martin, Saint Barthelemy, Anguilla, Saint Kitts and ing the Paris event, solidarity is signaled through the Nevis. Birgin Islands, Dominican Republic, Puerto use of flag emojis from different countries, while in the Rico, Haiti, Turks and Caicos and Cuba)7 . Irma corpus flag emojis do not play a prominent role. We find the pair – occurs most frequently RQ3: Which emojis coocur in tweets that are in solidarity tweets collected within the Irma affected posted within areas directly affected by crisis regions. The other top co-occurring pairs following the events as compared to those tweets that are sequence include – , – , – and – posted from other areas? ; these pairs might convey the concerns expressed in This research question and the two following RQs the tweets that originate within affected areas. The are driven by the hypothesis that solidarity would be emoji appears at the centre of the network denoting expressed differently by people that are directly af- the impact of the Irma event. The – , – , fected by the crisis than those who are not [Bue16]. To – are the emojis that appear in isolation from the address RQ3, we first geotagged tweets using geopy network. The and emojis can serve as indicators Python geocoding library5 to map the users’ locations to stay strong during this hurricane calamity. to their corresponding country. Table 6 shows the to- Figure 2 represents the co-occurrence network of tal number of emojis in solidarity tweets that were emojis in tweets posted outside the regions affected by geotagged and categorized as posted within regions Hurricane Irma. We find that the pair – tops the affected by the event vs. other regions. We then co-occurrence list. Next, we have other co-occurring built co-occurrence networks of emojis in both Irma pairs like – and – following the top most and Paris corpora using the R ggnetwork package6 frequent pair in sequence. We see the three disjoint with the force-directed layout to compare these emoji networks apart from the main co-occurrence network. co-occurrence networks in solidarity tweets that were The emoji appears at the centre of the network posted within areas directly affected by the crisis and expressing sorrow and the concern of the people during the areas that were not (shown in Figures 1-4). the event. The disjoint networks also contain flags and 5 https://github.com/geopy/geopy other emojis that express sadness and sorrow. 6 https://tinyurl.com/y7xnw9lr 7 http://www.bbc.com/news/world-us-canada-41175312 Figure 3: Cooccurrence network for emojis express- Figure 4: Cooccurrence network for emojis expressing ing solidarity from regions affected by November 2015 solidarity from regions not affected by November 2015 terrorist attacks in France terrorist attacks in France Figure 3 shows the co-occurrence network of emo- Figures 5 and 6 the diffusion of emojis across time jis for November terrorist attacks in France. Within (filtering emojis that occur fewer than 50 times and France, the pair – tops all the co-occurrence 25 times per day resp. for the 26197 emojis in Irma pairs. Co-occurrence pairs like – and – fol- and 24801 emojis in the Paris solidarity corpus (c.f. low the top co-occurring sequence, strongly conveying Table 4). The emojis are arranged on y-axis based on the solidarity of people who tweeted during Novem- their sentiment score based on the publicly available ber terrorist attacks. We also have co-occurring pairs work done by Novak et al. [NSSM15]. containing flags of other countries following the top- In the Irma corpus, the temporal diffusion of emo- tweeted list that shows uniform feeling among the peo- jis is quite interesting (Figure 5). Hurricane Irma ple by trying to express their sorrow and prayers. The grabbed attention of the world on September 6th when emoji appears in the centre of the large network it turned into a massive storm and the reaction on so- as an expression of danger during terrorist attacks. cial media expressing solidarity for Puerto Rico was We can also see that the network contains many flags through and . During the following days, the that indicates the concern and worries of people from United States is in the path of the storm, and there is many different countries. There are five disjoint net- an increased presence of and the presence of other works that again contain emojis that express the sor- countries flags. As the storm lashes out on the is- row, prayers and discontent. lands on September 7th , people express their feelings Figure 4 represents the co-occurrence network of through and emojis and also warn people about emojis for November terrorist attacks in Paris outside caring for the pets. As the storm moves through the France. We find that the pair – tops the co- Atlantic, more prayers with and emojis emerge occurrence list as within France, which is followed by on social media for people affected and on the path of the co-occurring pairs – and – . We can this storm. The storm strikes Cuba and part of Ba- infer that the people within or outside France shared hamas on September 9th before heading towards the common emotions that includes a mixture of prayers, Florida coast. As the storm moves towards the US on support and concern towards Paris and its people. We September 10th , people express their thoughts through find the , and appear at the centre of the net- , and causing tornadoes. When images of mas- work to convey solidarity. The emoji also appears sive flooding emerge on social media, people respond at the center, similar to Figure 1. One important in- with pet emojis like , , and to save them. ference is that the emoji appears at the network The emoji may also serve as an indicator of high- center Irma affected regions whereas it appears at the pitched crying [WBSD16]. network center for unaffected regions in Paris event. In the Paris attacks, the first 24 hours from 13th RQ4: How can emojis be used to under- night to 14th were the days on which most number of stand the diffusion of solidarity expressions emojis were used. When news of this horrible attack over time? spreads on social media, the immediate reaction of the For addressing this research question, we plot in people was to express solidarity through hashtags at- Figure 5: Diffusion of emojis across time for the Hur- Figure 6: Diffusion of emojis across time for the ricane Irma disaster (N=26197 emojis) November Paris attacks (N=24801 emojis) tached with emoji. As a result, was the most Figures 7 and 8 allow us to contrast how the Hur- frequently used emoji across all days. Emojis of other ricane Irma event is viewed within and outside the country flags such as , emerge to indicate solidar- affected regions. On September 6th , emerged when ity of people from these countries with France. Even the hurricane started battering the islands along with after the end of the attack on Nov 13th , people express a lot of heart emojis. On the other hand, more heart prayers for the people of France through emoji. Im- emojis emerged expressing solidarity from outside the ages and videos of the attacks emerge on social media affected regions when the people realized the effect of on Nov 14th leading to the use of the emoji. The the hurricane (Sep 7th . As the storm moved forward, emoji occurs across all days for the Paris event. people in the affected regions express a lot of prayers. Across both events, we also observe a steady pres- On September 9th , as the storm moves towards the ence over all days of positively valenced emojis in the United States after striking Cuba, there seems to be tweets expressing solidarity (the top parts of the diffu- more prayers amongst affected as well as outside sion graphs), while negatively valenced emojis are less communities. The emoji is constant in the affected prevalent over time (e.g. appears in the first two regions. In both Figures 8 and 7, we see that variety of and three days in the Irma and Paris events resp.). emojis appear in the latter days of the event (starting RQ5: How can emojis be used to study the Sep 9th ), including the , , and as well as the temporal and geographical difussion of solidar- animal/pet emojis, likely indicating the emergence of ity expressions during crisis events? different topics of discourse related to the event. Our primary aim with this research question was to look at how the emojis of solidarity diffuse over time 5 Conclusion and Future Work within the affected community and compare commu- nities not affected by the same event. Using the geo- We described our data, algorithm and method to ana- tagged tweets described in RQ2, we created Figures lyze corpora related to two major crisis events, specif- 7 and 8 to represent the diffusion of emojis over time ically investigating how emojis are used to express within the affected regions on the path of Hurricane solidarity on social media. Using manual annota- Irma and the non-affected regions respectively. Since tion based on hashtags, which is a typical approach the distribution of emojis in geotagged tweets for the taken to distance label social media text from Twit- Paris attacks is skewed (6.52% emojis expressing sol- ter [MBSJ09], we categorized tweets into those that idarity from affected regions, c.f. Table 6), we have express solidarity and tweets that do not express sol- excluded the Paris event for analysis in this RQ. idarity. We then analyzed how these tweets and the Figure 7: Diffusion of emojis across time for the Hur- Figure 8: Diffusion of emojis across time for the Hur- ricane Irma within affected regions (N=10048 emojis) ricane Irma outside affected regions (N=4770 emojis) emojis within them diffused in social media over time future. Second, we analyzed solidarity during crisis and geographical locations to gain insights into how events including a terrorist attack and a hurricane, people reacted globally as the crisis events unfolded. whereas solidarity can be triggered without an overt We make the following overall observations: shocking event, for example the #MeToo movement. In future work, there is great potential for further in- • Emojis are a reliable feature to use in classifica- vestigation of emoji diffusion across cultures. In ad- tion algorithms to recognize expressions of soli- dition to categorizing tweets as being posted from af- darity (RQ1). fected regions and outside of affected regions, we wish • The top emojis for the two crisis event reveal the to analyze the geographical diffusion with more granu- differences in how people perceive these events; in larity, using countries and regions as our units of anal- the Paris attack tweets we find a notable presence ysis to better understand the cultural diffusion of sol- of flag emojis, likely signaling nationalism but not idarity emojis. An additional future goal is to analyze in the Irma event (RQ2). the interaction of sentiment of emojis and solidarity as well as the text that cooccurs with these emojis in • Through the cooccurence networks, we observe further detail. We anticipate our approach and find- that the emoji pairs in tweets that express sol- ings will help foster research in the dynamics of on- idarity include anthropomorphic emojis ( , , line mobilization, especially in the event-specific and ) with other categories of emojis such as and behavior-specific usage of emojis on social media. (RQ3). Acknowledgements • Through analyzing the temporal and geospatial diffusion of emojis in solidarity tweets, we observe We are grateful for the extremely helpful and con- a steady presence over all days of positively va- structive feedback given by anonymous reviewers. We lenced emojis, while negatively valenced emojis thank the two trained annotators for their help in cre- become less prevalent over time (RQ4, RQ5). ating our dataset. This research was supported, in part, by a fellowship from the National Consortium of Future Work: While this paper addressed five Data Science to the last author. salient research questions related to solidarity and emojis, there are a few limitations. First, our dataset References contains emojis that number in the few thousand, which is relatively small when compared to extant re- [Bau13] Zygmunt Bauman. Postmodernity search in emoji usage [LF16]. However, we aim to and its Discontents. John Wiley & reproduce our findings on larger scale corpora in the Sons, 2013. [Bay99] Kurt Bayertz. Four uses of solidarity. [Gid13] Anthony Giddens. The consequences In Solidarity, pages 3–28. Springer, of modernity. John Wiley & Sons, 1999. 2013. [BKRS16] Francesco Barbieri, German [HGS+ 17] Tianran Hu, Han Guo, Hao Sun, Kruszewski, Francesco Ronzano, Thuy-vy Thi Nguyen, and Jiebo Luo. and Horacio Saggion. How cos- Spice up your chat: The intentions mopolitan are emojis?: Exploring and sentiment effects of using emoji. emojis usage and meaning over dif- arXiv preprint arXiv:1703.02860, ferent languages with distributional 2017. semantics. In Proceedings of the 2016 [HS97] Sepp Hochreiter and Jürgen Schmid- ACM on Multimedia Conference, huber. Long short-term memory. pages 531–535. ACM, 2016. Neural computation, 9(8):1735–1780, 1997. [Bue16] Steven M Buechler. Understanding social movements: Theories from the [HVBMGMS15] Enrique Herrera-Viedma, Juan classical era to the present. Rout- Bernabe-Moreno, Carlos Porcel ledge, 2016. Gallego, and Maria de los Angeles Martinez Sanchez. Solidarity in so- [Bur18] INQUIRER.net US Bureau. never- cial media: when users abandon their again walkouts spark half a million comfort zone-the charlie hebdo case. social media posts in 24 hours, Mar REVISTA ICONO 14-REVISTA CI- 2018. ENTIFICA DE COMUNICACION Y TECNOLOGIAS, 13(2):6–22, [DBVG07] Daantje Derks, Arjan ER Bos, and 2015. Jasper Von Grumbkow. Emoticons and social interaction on the inter- [JA13] Tanimu Ahmed Jibril and net: the importance of social con- Mardziah Hayati Abdullah. Rel- text. Computers in human behavior, evance of emoticons in computer- 23(1):842–849, 2007. mediated communication contexts: An overview. Asian Social Science, [DCJSW16] Munmun De Choudhury, Shagun 9(4):201, 2013. Jhaver, Benjamin Sugar, and Ingmar [KW15] Ryan Kelly and Leon Watts. Char- Weber. Social media participation in acterising the inventive appropria- an activist movement for racial equal- tion of emoji as relationally meaning- ity. In ICWSM, pages 92–101, 2016. ful in mediated close personal rela- [DP17] Giulia Donato and Patrizia Paggio. tionships. Experiences of Technology Investigating redundancy in emoji Appropriation: Unanticipated Users, use: Study on a twitter based cor- Usage, Circumstances, and Design, pus. In Proceedings of the 8th Work- 2015. shop on Computational Approaches [LF16] Nikola Ljubešić and Darja Fišer. A to Subjectivity, Sentiment and Social global analysis of emoji usage. In Media Analysis, pages 118–126, 2017. Proceedings of the 10th Web as Cor- pus Workshop, pages 82–89, 2016. [DVVB+ 16] Michela Del Vicario, Gianna Vivaldo, Alessandro Bessi, Fabiana Zollo, An- [MBSJ09] Mike Mintz, Steven Bills, Rion Snow, tonio Scala, Guido Caldarelli, and and Dan Jurafsky. Distant super- Walter Quattrociocchi. Echo cham- vision for relation extraction with- bers: Emotional contagion and group out labeled data. In Proceedings of polarization on facebook. Scientific the Joint Conference of the 47th An- reports, 6:37825, 2016. nual Meeting of the ACL and the 4th International Joint Conference [Fen08] Natalie Fenton. Mediating solidar- on Natural Language Processing of ity. Global Media and Communica- the AFNLP: Volume 2-Volume 2, tion, 4(1):37–57, 2008. pages 1003–1011. ACL, 2009. [MO17] Kris M Markman and Sae Os- [Tuf14] Zeynep Tufekci. Social move- hima. Pragmatic play? some pos- ments and governments in the digi- sible functions of english emoticons tal age: Evaluating a complex land- and japanese kaomoji in computer- scape. Journal of International Af- mediated discourse. 2017. fairs, pages 1–18, 2014. [MPH08] Christopher Manning, Raghavan [WBSD16] Sanjaya Wijeratne, Lakshika Bala- Prabhakar, and Schütze Hinrich. In- suriya, Amit Sheth, and Derek Do- troduction to information retrieval. ran. Emojinet: Building a ma- An Introduction To Information chine readable sense inventory for Retrieval, 151:177, 2008. emoji. In International Conference on Social Informatics, pages 527– [NPM17] Noa Na’aman, Hannah Provenza, 541. Springer, 2016. and Orion Montoya. Varying linguis- tic purposes of emoji in (twitter) con- text. In Proceedings of ACL 2017, Student Research Workshop, pages 136–141, 2017. [NSSM15] Petra Kralj Novak, Jasmina Smailović, Borut Sluban, and Igor Mozetič. Sentiment of emojis. PloS one, 10(12):e0144296, 2015. [PE15] Umashanthi Pavalanathan and Ja- cob Eisenstein. Emoticons vs. emojis on twitter: A causal in- ference approach. arXiv preprint arXiv:1510.08480, 2015. [PSM14] Jeffrey Pennington, Richard Socher, and Christopher Manning. Glove: Global vectors for word representa- tion. In Proceedings of the 2014 con- ference on empirical methods in nat- ural language processing (EMNLP), pages 1532–1543, 2014. [Rio17] Monica A Riordan. The communica- tive role of non-face emojis: Affect and disambiguation. Computers in Human Behavior, 76:75–86, 2017. [Sun18] Cass R Sunstein. # Republic: Di- vided democracy in the age of social media. Princeton University Press, 2018. [SW05] Julius Sim and Chris C Wright. The kappa statistic in reliability stud- ies: use, interpretation, and sample size requirements. Physical therapy, 85(3):257–268, 2005. [Tay15] Ashley E Taylor. Solidarity: Obli- gations and expressions. Journal of Political Philosophy, 23(2):128–145, 2015.