Lexicon-based approaches rooted in Natural Language Processing (NLP) were tested to explore collective identity, collective eficacy beliefs, group sentiment, and group emotions within the framework of the German environmental movement. A dataset comprising 5607 social media posts from six prominent environmental groups in Germany spanning the period from 2022 to 2024 was gathered and analyzed using both Valence Aware Dictionary and sEntiment Reasoner (VADER) and Text-Based Emotion Detection (TBED) with the ed8 dictionary. Additionally, collective identity and collective eficacy beliefs were assessed based on the prevalence of specific representative terms within the texts. To validate the sentiment and emotion scores obtained, a random subset of documents was manually reviewed for comparison. The validation revealed limitations in the reliability of sentiment analysis and TBED methodologies with lexicon-based approaches, potentially stemming from the utilization of German language and climate change-specific content, which may not align optimally with existing lexicons. To enhance the applicability of lexicon-based approaches in such contexts, the development and application of climate change domain-specific lexicons tailored for the German language are recommended for future research endeavors.
CEUR ceur-ws.org
In light of the pressing environmental challenges of our era, an increasing number of
individuals are joining activist groups and participating in street demonstrations on a weekly
basis, advocating for efective policy measures and climate and environmental justice [
This project aims to explore the potential of lexicon-based sentiment and emotion detection methods to investigate collective identity, collective eficacy beliefs, group sentiment, and group emotions as social psychological drivers among diferent EMOs in Germany.
Natural Language Processing (NLP) has emerged as a valuable tool in psychological and
sociological research, ofering innovative approaches to understanding human and group behavior,
cognition, and emotion through the analysis of language. In psychology, NLP techniques are
applied across various domains, including clinical psychology, social psychology, cognitive
psychology, and beyond [16, 17, 18]. Especially, social media allows access to digital footprints
constructed by diverse communities. Therefore, a major challenge in the social sciences gets
addressed, namely the extensive reliance on non-representative samples that are small, consist
of (predominantly female) students, and are disproportionately WEIRD (i.e., Western, educated,
industrialized, rich, and democratic; [19]). For example, social media data has been used to
analyze how digital media influence collective action dynamics [ 20]. Phan and Airoldi discovered
that online social activity on Twitter is positively correlated with proximity to natural disasters,
and that individuals who experience natural disasters are more likely to reinforce this form of
social interaction by, for example, forming groups on Twitter [21]. Another study examined the
usage of eficacy related terminology on the websites of environmental organizations [ 22]. Their
analysis indicates that environmental groups frequently employ linguistic cues pertaining to
eficacy and identity on their websites, ultimately increasing the mobilization potential of their
message recipients. Moreover, better-funded groups seem to utilize these cues more frequently
than groups with fewer resources [22].
1.1.1. Sentiment Analysis and Text-based Emotion Detection
NLP tools like sentiment analysis have been used frequently to examine the nature of climate
change discussion among diferent countries and over time [ 23, 24, 25]. A sentiment can be
defined as “an attitude, thought, or judgment prompted by feeling” [
This study leveraged NLP methodologies to explore collective identity, collective eficacy beliefs, group sentiment, and group emotions within the context of the German environmental movement. Specifically, sentiment analysis and Text-Based Emotion Detection (TBED) were conducted utilizing a lexicon-based approach. Data from the Facebook pages of six prominent environmental groups in Germany spanning from 2022 to 2024 were collected via web scraping. The organizations under study include Fridays for Future, Letzte Generation, Extinction Rebellion, Ende Gelände, Greenpeace, and Bund für Umwelt und Naturschutz (BUND). These entities represent diverse environmental concerns and engagement strategies within Germany. A total of 6371 posts were gathered, providing comprehensive insights into the discourse surrounding climate change and environmental activism in the region. Following data preprocessing, which involved eliminating duplicates and documents lacking substantial text (e.g., containing only images or URLs), 5653 documents remained. These comprised 1625 posts from Extinction Rebellion, 1314 from Letzte Generation, 1030 from Greenpeace, 737 from Bund für Umwelt und Naturschutz, 615 from Fridays for Future, and 332 from Ende Gelände. Given that organizations often share the same content across all their social media platforms, a single platform (Facebook) was selected for analysis. Further preprocessing steps involved text standardization by converting it to lowercase, lemmatization, and removal of URLs, stop words, English terms, punctuation, and numerical characters.
Valence Aware Dictionary and sEntiment Reasoner (VADER) was employed for sentiment
analysis, utilizing a lexicon and rule-based methodology tailored for assessing sentiments
prevalent in social media content. This tool considers not only word usage but also sentence
structure and modifiers that influence sentiment intensity [ 30]. Sentiment scores range from
+1 (indicating a predominantly positive text) to -1 (indicating a predominantly negative text),
with a score of 0 indicating neutrality. For TBED, the ed8 dictionary, specifically designed for
political texts, was utilized [
To ensure the accuracy of the compound sentiment scores, 50 documents were randomly selected and manually reviewed. Each document was read and labeled for sentiment, and these annotated labels were compared with the VADER compound value. Only positive, negative, or neutral sentiment labels were distinguished during the evaluation. Similarly, another random sample of 50 documents was reviewed to assess emotion scores. This evaluation focused on whether a document was assigned an emotion, without considering specific normalized emotional values. Therefore, precision, recall and F1 were calculated. These metrics are commonly used in machine learning classification tasks. Precision represents the ratio of accurately predicted instances to the total predicted instances, revealing the count of false positives. Conversely, recall quantifies the proportion of correctly predicted instances relative to the total true instances, highlighting the number of false negatives. The F1 score, a composite measure, is calculated as the harmonic mean of precision and recall (1).
The sentiment trend over the entire period measured with VADER is illustrated in Figure 1. Documents labeled with positive sentiment primarily included calls to strike, reports on previous strikes, and other related achievements. Conversely, documents expressing negative sentiment addressed issues such as the climate crisis, political matters, and instances of police violence. However, a notable number of discrepancies were identified during the manual evaluation process. Out of the sampled documents, 33 exhibited consistent sentiment with the VADER compound score, while 17 did not align. It is important to acknowledge that a majority 1(”we,” ”us,” ”our,” and ”together”) 2(”goal,” ”achieve,” ”accomplish,” ”succeed,” and ”persevere”) of manually labeled documents were categorized as having neutral sentiment, which may introduce inaccuracies. Precision, recall and F1 are presented in Table 1.
Based on the sample drawn and the accuracy of the sentiment assignments, it can be concluded that positive sentiment was recognized far better than negative sentiment. In general, the manual evaluation found that the text data is mostly neutral in sentiment, as the EMOs’ posts often lack subjectivity, which is a challenge for VADER. Despite these challenges, there remain opportunities for enhancing sentiment analysis accuracy. Examples of documents along with their respective VADER compound scores and manually assigned sentiment labels are provided below. ’Das EU-Parlament hat’s verkackt und alle wissen’s! Nach der katastrophalen Entscheidung des EU-Parlaments muss sich die Bundesregierung den anderen europäischen Ländern anschließen, die gegen das Greenwashing von Atom & Gas klagen! #NotMyTaxonomy [...]’3 [VADER compound score: neutral (0.000); manually assigned: negative] ’Yeah! Mit 35.000 Menschen inkl. Greta Thunberg haben wir bei Lützerath im rheinländischen Kohlerevier klar gemacht: Lützi bleibt! [smiley]’4 [VADER compound score: positive (0.296); manually assigned: positive]
By analyzing the language employed in the documents and comparing it with the ed8 emotion lexicon, the emotions depicted in Figure 2 were identified. Utilizing the ed8 dictionary, all emotions (anger, fear, disgust, sadness, pride, enthusiasm, hope, and joy) were detectable in 3’The EU Parliament screwed up and everyone knows it! After the EU Parliament’s disastrous decision, the German government must join the other European countries that are taking legal action against the greenwashing of nuclear & gas! #NotMyTaxonomy [...]’ (translated with deepl.com) 4’Yeah! With 35,000 people including Greta Thunberg, we made it clear at Lützerath in the Rhineland coalfield: Lützi stays! [smiley]’(translated with deepl.com) certain documents. Table 1 presents the findings of the evaluation process. The evaluation suggests that TBED did not consistently and reliably discern emotions from the text data. The accuracy of the classification partly difers greatly between the diferent emotions. Thereby, the emotion classification worked best for the emotion anger. However, the manual evaluation indicated that EMOs frequently employ emotional language. For instance, the following emotion scores were lexicon-based and manually assigned to these documents: ’220.000 Menschen für die Verkehrswende! [smileys] Wir haben mit @fridaysforfuture.de für den Klimastreik aufgerufen - und ihr wart dabei. Wow! Danke, dass ihr so zahlreich erschienen seid. Nur gemeinsam können wir etwas bewegen. Wir bleiben laut und führen den Protest weiter, @VolkerWissing! Was brauchst du für deine gerechte Verkehrswende von der Politik? Schreib es uns in die Kommentare. [...] ’5 [ed8 normalized emotions: joy (0.023809524), pride (0.023809524); manually assigned emotions: enthusiasm, pride, joy] 5’220,000 people for the transport turnaround! [smileys] We called for the climate strike with @fridaysforfuture.de - and you were there. Wow! Thank you for showing up in such large numbers. Only together can we make a diference. We’ll stay loud and continue the protest, @VolkerWissing! What do you need from politicians for a fair transport transition? Let us know in the comments. [...]’ (translated with deepl.com) ’Ist es nicht beängstigend, über die Welt nachzudenken, die wir künftigen Generationen hinterlassen könnten, wenn wir jetzt nicht gemeinsam handeln? Greenpeace hat Lösungen. Aber um diese umzusetzen, brauchen wir Menschen wie dich. Zusammen können wir es schafen. [smileys] Wirst du uns unterstützen? Spende noch heute an Greenpeace. Wir werden zu 100 % von Einzelpersonen wie Dir finanziert. Nur mit Deiner Unterstützung können wir dafür sorgen, dass heute geborene Kinder saubere Luft zum Atmen, plastikfreie Meere und eine grünere, gerechtere Welt erleben werden. [URL]’ 6 [ed8 normalized emotions: enthusiasm (0.073170732), pride (0.048780488), hope (0.07317073); manually assigned emotions: fear, hope] 6’Isn’t it scary to think about the world we could leave for future generations if we don’t act together now? Greenpeace has solutions. But to implement them, we need people like you. Together we can do it. [smileys] Will you support us? Donate to Greenpeace today. We are 100 % funded by individuals like you. Only with your support can we ensure that children born today will have clean air to breathe, plastic-free oceans and a greener, fairer world. [URL]’ (translated with deepl.com)
The appendix presents the detailed results for collective identity and collective eficacy beliefs. Through the applied word search method, indications of both collective factors were identified in the documents. Notably, the organizations exhibited significant variations in their language usage, interpreted here as expressions of collective identity or collective eficacy beliefs.
While this study utilized NLP methods to investigate collective identity, collective eficacy beliefs, group sentiment, and group emotions within German environmental movements, several limitations merit consideration. Firstly, it is important to emphasize that the manual evaluation in the study was based on a person’s subjective assessment, which may vary from person to person and therefore has limited reproducibility. Further, existing sentiment and emotion lexicons lack domain specificity, providing only general labels for each term despite the highly context-dependent nature of sentiment and emotions. Moreover, it is crucial to acknowledge that all analyses and conclusions are based solely on linguistic cues. Thus, while it can be inferred that environmental groups’ Facebook posts may contain sentiment, emotional language, or expressions of collective identity and collective eficacy beliefs, definitive statements about the actual presence of these factors are constrained. Additionally, relying on publicly available social media data may not fully capture the diverse voices within environmental movements.
Future endeavors could involve comparing alternative lexicons to assess their accuracy and
exploring the annotation of more data specific to the climate change context in German. This
could enhance the performance of both lexicon-based and machine learning-based approaches
[
Thanks to the valuable support of Judith Maier, Margot Madina, Damian Bednarz, Nora Cremille, L. Antoinette Engelbrecht-Schnür and Patricia Stehl. [15] G. Albrecht, Psychoterratic conditions in a scientific and technological world, Ecopsychology: Science, totems, and the technological species (2012) 241–264. [16] B. Desmet, V. Hoste, Emotion detection in suicide notes, Expert Systems with Applications 40 (2013) 6351–6358. doi:10.1016/j.eswa.2013.05.050. [17] M. J. Tanana, C. S. Soma, P. B. Kuo, N. M. Bertagnolli, A. Dembe, B. T. Pace, V. Srikumar, D. C.
Atkins, Z. E. Imel, How do you feel? using natural language processing to automatically rate emotion in psychotherapy, Behavior research methods (2021) 1–14. doi:10.3758/ s13428-020-01531-z. [18] T. Zhang, A. M. Schoene, S. Ji, S. Ananiadou, Natural language processing applied to mental illness detection: a narrative review, NPJ digital medicine 5 (2022) 1–13. doi:10. 1038/s41746-022-00589-7. [19] J. Henrich, S. J. Heine, A. Norenzayan, Most people are not weird, Nature 466 (2010) 29–29.
doi:10.1038/466029a. [20] Y. Theocharis, S. Vitoratou, J. Sajuria, Civil society in times of crisis: Understanding collective action dynamics in digitally-enabled volunteer networks, Journal of ComputerMediated Communication 22 (2017) 248–265. doi:10.1111/jcc4.12194. [21] T. Q. Phan, E. M. Airoldi, A natural experiment of social network formation and dynamics, Proceedings of the National Academy of Sciences 112 (2015) 6595–6600. doi:10.1073/ pnas.1404770112. [22] R. Gulliver, K. S. Fielding, W. R. Louis, Assessing the mobilization potential of environmental advocacy communication, Journal of Environmental Psychology 74 (2021) 101563. doi:10.1016/j.jenvp.2021.101563. [23] E. M. Cody, A. J. Reagan, L. Mitchell, P. S. Dodds, C. M. Danforth, Climate change sentiment on twitter: An unsolicited public opinion poll, PloS one 10 (2015) e0136092. doi:10.1371/journal.pone.0136092. [24] B. Dahal, S. A. Kumar, Z. Li, Topic modeling and sentiment analysis of global climate change tweets, Social network analysis and mining 9 (2019) 1–20. doi:10.1007/ s13278-019-0568-8. [25] Y. Kirelli, S. Arslankaya, et al., Sentiment analysis of shared tweets on global warming on twitter with data mining methods: a case study on turkish language, Computational Intelligence and Neuroscience 2020 (2020). doi:10.1155/2020/1904172. [26] B. Liu, Sentiment analysis: Mining opinions, sentiments, and emotions, Cambridge university press, 2020. doi:10.1017/9781108639286. [27] M. L. Loureiro, M. Alló, Sensing climate change and energy issues: Sentiment and emotion analysis with social media in the uk and spain, Energy Policy 143 (2020) 111490. doi:10. 1016/j.enpol.2020.111490. [28] S. M. Mohammad, P. D. Turney, Nrc emotion lexicon, National Research Council, Canada 2 (2013) 234. [29] A. H. Safar, T. K. Mann, B. Ofoghi, Textual emotion detection in health: Advances and applications, Journal of Biomedical Informatics 137 (2023) 104258. doi:10.1016/j.jbi. 2022.104258. [30] C. Hutto, E. Gilbert, Vader: A parsimonious rule-based model for sentiment analysis of social media text, Proceedings of the international AAAI conference on web and social media 8 (2014) 216–225. doi:10.1609/icwsm.v8i1.14550.
The R script is available via • osf