=Paper=
{{Paper
|id=Vol-2769/54
|storemode=property
|title=ItaGLAM: A corpus of Cultural Communication on Twitter during the Pandemic
|pdfUrl=https://ceur-ws.org/Vol-2769/paper_54.pdf
|volume=Vol-2769
|authors=Gennaro Nolano,Carola Carlino,Maria Pia di Buono,Johanna Monti
|dblpUrl=https://dblp.org/rec/conf/clic-it/NolanoCBM20
}}
==ItaGLAM: A corpus of Cultural Communication on Twitter during the Pandemic==
https://ceur-ws.org/Vol-2769/paper_54.pdf
ItaGLAM: A corpus of Cultural Communication on Twitter during the
Pandemic
Gennaro Nolano, Carola Carlino, Maria Pia di Buono, Johanna Monti
UniOr NLP Research Group
”L’Orientale” University of Naples
Italy
{gnolano, ccarlino, mpdibuono, jmonti}@unior.it
Abstract the web as a way to create and foster an online com-
munity (Langa, 2014; Allen-Greil and MacArthur,
This paper describes the compilation and 2010).
annotation of ItaGLAM, a corpus of While during the first decade of this century mu-
tweets written by Italian Galleries, Li- seum professionals considered the exhibition of
breries, Archives and Museums (GLAMs) collections on social networks (Laws, 2015) as ‘ex-
during the lockdown period in Italy due to cessive’, nowadays the use of these platforms has
the COVID-19 pandemic. ItaGLAM has become the norm. As Amanatidis et al. (2020)
been annotated with a set of labels which pointed out in their study about the use of social
may be useful to identify different types networks (and in particular Instagram) by museums
of communication. Furthermore, the col- in the Greek culture scene: ‘social media has be-
lected data have been used to train a set of come a key factor in the way that cultural organiza-
classifiers. tions communicate with their public in supporting
The results are analyzed to evaluate the in- the marketing of performing art organizations’.
formation flow between GLAM and users Such centrality makes the Social network a po-
and to analyze cultural communication on tentially effective means that allows GLAMs to
the Web. reach a wide and heterogeneous audience and to
adapt to it. Therefore, we believe that the analysis
1 Introduction of the cultural communication implies an analysis
Over the last years, Social Networks have be- of how cultural corporations interact with the audi-
come one of the most popular platforms for sharing ence through social networks.
experiences and opinions through the use of simple After considering the most used social networks
strings of text (Zhao and Rosson, 2009). Indeed, (namely Facebook, Instagram, Twitter) in the cul-
this way of communicating has become an essen- tural sector, we have decided to focus our research
tial interaction tool, not only among private users, on the use of Twitter, which has already been
but also among companies to engage with their au- proven to be a solid basis to analyze institutional
dience and to promote their brands (Alturas and communication, as Preoţiuc-Pietro et al. (2015)
Oliveira, 2016). have highlighted.
The use of social networks has also been adopted Therefore, the main aim of our research is to
by museums, that, over time, have changed their investigate how Italian GLAMS have extraordinar-
way of communicating with their audience2 . In ily (Giraud, 2020) interacted with their audience
particular, in regards to the GLAM sector, a new during the lockdown in Italy due to the COVID-19
trend has been observed in recent years: the use of pandemic (NEMO - Network of European Museum
Organisations, 2020), i.e. in the period from the
Copyright c 2020 for this paper by its authors. Use 8th of March to the 5th of May 2020 (as per DPCM
permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0). March 11 2020).
2
https://www.osservatori.net/it/ Over this time many cultural initiatives have
ricerche/comunicati\-stampa/laumento\- been launched with the aim of strengthening the
del\-livello\-di\-interesse\-per\-
le\-attivita\-online\-dei-musei\- dialogue with the audience and make sure that,
incentivato\-dal\-covid\-19\-e\-gli\- despite the impossibility of any kind of physical
investimenti\-per\-migliorare\-i\- access, the connection between GLAMs and their
servizi\-offerti
visitors would not be interrupted3 . Nissim, 2013; Cimino et al., 2014) and automatic
It has been observed that during the aforemen- misogyny identification (Anzovino et al., 2018). To
tioned period, while GLAMs institutions have dras- the best of our knowledge, no work has been done
tically increased their use of Facebook, Instagram so far on communicative intent classification for
and Twitter, the latter one was the only Social Italian tweets.
Network for which an increase in the interaction 3 Methodology
audience-institution has been registered (Politec-
nico di Milano, 2020). The task of tweet classification has turned out to
The study of communicative intents by GLAMs be rather challenging for various reasons, many in-
through Social Networks in the Italian language is herent to the platform itself. First and foremost,
still novel and, as such, best practices and tools to tweets are very short texts (with the maximum
use still need to be tested and honed. In particular, length of 280 characters), and with an average to-
there is still the need for an annotated corpus and a ken count of 16.80 in our corpus.
classifier that can be used on large amounts of data. Secondly, it is not unusual to find tweets composed
Despite the time frame taken into account is rel- only of hashtags, or URLs. While URLs by them-
atively short (covering 58 days in total), we think selves are rarely if ever useful in a classification
that investigating how Italian GLAMs used the web task, hashtags could represent a source of informa-
when it was the only form of communication at tion only if they are used according to their original
their disposal, represents a good training ground to communicative intent or to the initiative to which
test our practices and to train and evaluate different they are related.
kinds of classifiers useful also in future works. In the following subsections we describe how:
The paper is organized as follows: Section
• the corpus was created;
2 describes the related works in the analysis of
communication on Twitter by cultural institutions. • the annotation set has been chosen and then
Section 3 introduces the methodology used in this applied;
analysis: namely, it describes the creation of the
corpus, the creation and use of the annotation • the classifiers have been trained and tested.
set, and the training and evaluation of different
classifiers. Finally, in Section 4 we explain the 3.1 Dataset
results of the research. Because of the COVID-19 outbreak, the Italian
Government (as many others around the world)
imposed a lockdown policy, which lasted from the
2 Related Work
8th March to the 5th May 2020 (58 days in total as
The large amount of data available on Twitter per DPCM March 11 2020).
makes this platform ideal for several studies. As During this period of time, museums and art
such, during the years tweets have been used in sev- galleries adopted several strategies to continue
eral research projects regarding disaster response engaging with their audience in order to maintain
(Zahra et al., 2019), content classification (Dann, the communication alive, and to grant access
2010; Stvilia and Gibradze, 2014) and, in partic- to digital cultural heritage media. As already
ular, sentiment analysis (O’Connor et al., 2010; mentioned in Section 1, they increased the scope of
Gamallo and Garcia, 2014; Talbot et al., 2015). De- their communication on the main social platforms,
spite these efforts, only a few studies have focused i.e. Facebook, Twitter and Instagram.
on the classification of communicative intents of In this context, the focus of our analysis is the
organizations and institutions on Twitter, like Love- use of Twitter. The communication on Twitter is
joy and Saxton (2012) and Foucault and Courtin characterised by the use of certain hashtags,
(2016), who focused on French tweets written dur- which have been used by GLAMs to propose
ing the MuseumWeek event. several types of initiatives to their audience.
Similar kinds of study can be found in researches Initially, the set of hashtags we used was made
dealing with Italian tweets, with several contribu- up of 33 hashtags promoted and used by Italian
tions dealing with sentiment analysis (Basile and GLAMs and Italy’s Ministry of Cultural Heritage
3
https://icom.museum/en/news/how-to- and Activities (Italian: Ministero per i Bene e le
reach-and-engage-your-public-remotely/ Attività Culturali e per il Turismo - MiBACT), and
selected on the basis of their popularity according to annotate the information flow on a social net-
to the Twitter trend topics (TT)4 . work during a cultural event.
Among these hashtags, #museitaliani (and The annotation has been done at tweet level, using
its graphic variation #museiitaliani) is the a set of labels composed as follows:
only one already existing before the pandemic, and • Sharing Experience - SE: tweets that share
subsequently adapted by museums for the initia- an experience, an opinion or one’s feeling
tives proposed during the pandemic; while others, Example: Eccoci qui oggi a ricordare e a rac-
such as #artyouready and #emptymuseum contare come i musei chiusi non siano chiusi
have been created ad hoc during the lockdown e i musei vuoti non siano vuoti. Forza!
period to describe specific initiatives. (Here we are today, reminding and telling how
By using these hashtags as a queue in the public closed museums are not actually closed and
Twitter API5 we have created a corpus with a total empty museums are not actually empty. Come
of 23,716 tweets. on!);
To better focus on the tweets and their intents con- • Promoting Participation - PP: tweets that
cerning cultural communication, we have decided require some kind of activity from the users,
to filter out of the corpus any hashtag with less either in real life or on-line
than 1,000 occurrences. We have thus obtained Example: Art you ready? Domani partecipa
a queue of six hashtags (#artyouready, anche tu al contest di @ museitaliani con-
#emptymuseum, #museitaliani, dividendo con noi le tue foto dei musei privi
#museichiusimuseiaperti, di persone. Cerca fra i ricordi, seleziona la
#laculturanonsiferma, foto, e condividi con # artyouready # Muse-
#laculturaincasa) and a corpus of 15,988 umFromHome # iorestoacasa. Ti aspettiamo!
tweets. (Art you ready? Take part in tomorrow’s @
This corpus has been filtered once again so that museitaliani contest by sharing with us your
only unique tweets (i.e. no retweets) written in photos of empty museums. Search through
Italian have been kept. By using a list of GLAMs your memories, choose the photo and share
manually extracted from the corpus, we have then it with # artyouready # MuseumFromHome #
extracted out of the remaining 8,038 tweets those iorestoacasa. We are waiting for you!);
written by a GLAM institution, thus ending up • Interacting with the Community - ItC:
with our final corpus of 3,429 tweets published by tweets through which Institutions create and
213 Italian cultural institutions. Table 1 shows the foster their communities by directly interact-
occurrences of the hashtags in the final corpus. ing with the users
Example: Siete stati davvero tanti ad ac-
cogliere l’invito a partecipare al flashmob #
Hashtag # Occ.
#artyouready 367
artyouready e tutti avete postato foto merav-
#emptymuseum 373 igliose! Ecco i tre scatti selezionati tra i più
#museitaliani 906 belli
#museichiusimuseiaperti 1560
#laculturanonsiferma 668 (So many of you accepted to take part in the
#laculturaincasa 283 # artyouready flashmob, and you all posted
Total 4,157 great photos! Here are the three shots selected
among the most beautiful ones);
Table 1: Number of occurrences for each hashtag. • Promoting-Informing - PI: tweets that pro-
mote or inform other users about activities,
3.2 Annotation Process exhibitions, or about any sort of information
In order to define the intents of GLAMs to- on the museum.
wards the users, the corpus has been annotated Example: Il castello di Fénis si trova in
with four communication categories first presented Valle d’Aosta circondato da una doppia cinta
by Courtin et al. (2015), and then used by Foucault di mura merlate è caratterizzato da torri
and Courtin (2016), and Juanals and Minel (2018) quadrate e cilindriche con feritoie e caditoie.
(Fénis Castle is located in Aosta Valley, with
4
This process is described in details in Carlino et al. (2020) its double crenellated surrounding walls, it is
5
https://developer.twitter.com/en
characterized by square and cylindrical towers Label # Occ. % corpus
SE 843 25.24%
with loopholes and storm drains). PP 165 4.94%
A fifth category N/A has been included in order ItC 1110 33.23%
to classify tweets that do not fit in any of the afore- PI 1222 36.59%
mentioned categories, like the ones composed of
only hashtags. Table 2: Number of occurrences and percentage in
Following this set of categories and our guidelines, the corpus for each label.
the tweets have been annotated using the open
source platform INCEpTION6 , and a first round of stopwords7 removed. The cleaning process has
annotation has been carried on 400 tweets, double been done via the NLTK package for Python8 ,
annotated by a domain expert and a non-expert in which has also been used for tokenization.
order to calculate the Inter-Annotator Agreement The experiments have been conducted on six
(IAA). classifiers: five more traditional classifiers trained
The use of a non-expert was necessary so that the on a TF-IDF vectorized text (created using the
annotation would not have been influenced by any machine learning library for Python Scikit-learn9 ),
external knowledge (for example the original mean- and a Feed Forward Neural Network10 created
ing behind the various hashtags). with Keras11 and trained on a 100-dimensions
The resulting Fleiss’ Kappa has revealed to be mod- GlOve12 embedded text.
erately good at 0.629, which is considered suffi- The set of classifiers is thus the following: a
cient for the task at hand. As it can be seen from Naive Bayes (NB, also used as baseline); a
the confusion matrix in Figure 1, the agreement is Support Vector Classifier (SVC); a the K-Nearest
very strong on PI and ItC, moderately strong on Neighbors classifier (KNN); a Decision Tree (DT);
SE, and very weak on PP. a Multilayer Perceptron (MLP) and a Neural
Furthermore, 89 tweets have been deemed unus- Network classifier (NN).
The dataset was split using the train test split tool
found in the sklearn library for Python, which
splits the data into random train and test subsets
given a test set size. With test size set at 0.3, the
training set is composed of 2,338 tweets, and the
testing set is composed of the remaining 1,002
tweets.
In order to evaluate the classification task, the
Figure 1: Confusion matrix for the agreement on values of precision, recall and F1 have been all
every label. weighted by the number of samples of each label.
The final results are shown in Table 3.
able as they have been tagged with the label N/A,
Classifier P R F1
therefore, they have been removed from the cor- NB 0.69 0.66 0.64
pus. SVC 0.70 0.68 0.67
Table 2 presents the number of occurrences for KNN 0.70 0.39 0.35
DT 0.56 0.55 0.55
each label for the remaining 3,340 tweets. These MLP 0.66 0.66 0.66
results show an issue regarding the label PP, that NN 0.64 0.63 0.63
is severely underrepresented in the corpus. The ef-
fects of this underrepresentation on our classifiers Table 3: System results.
will be explained in detail in Section 4, and the
analysis of possible solutions will be the focus of 7
The list of stopwords used is the default one for Italian
future work. found in the NLTK package. Furthermore, the term ’Twitter’
has been added to this list after the first experiments.
3.3 Intent classification 8
https://www.nltk.org/
In order to train the classifiers, the corpus has 9
https://scikit-learn.org/stable/
been preprocessed so that all tweets are lowercase, 10
Parameters: 4 layers, dropout=0.7, Adam Optimizer
11
and all punctuation marks, URLs, numbers and https://keras.io/
12
https://nlp.stanford.edu/projects/
6
https://inception-project.github.io/ glove/
Token SE PP ItC PI
#museichiusimuseiaperti 1.55 2.47 2.12 1.99
#iorestoacasa 1.66 1.92 1.51 1.71
4 Evaluation and Result Analysis #museitaliani 2.43 2.35 2.05 2.33
#laculturanonsiferma 2.88 2.47 2.83 2.35
The results show that the methodology adopted #emptymuseum 3.02 1.97 3.29 3.4
in this work can be useful in better understanding #artyouready 2.9 1.71 3.04 3.23
#laculturaincasa 4.1 - 3.47 2.96
how cultural institutions communicate on the Web. flashmob - 2.1 - -
The tools used in this specific task are adequate in mibact 3.34 2.4 2.7 3.09
oggi 3.03 - - 2.78
annotating and automatically classifying the way youtube - - 3.16 -
cultural institutions communicate on the Twitter cultura 4.18 - 3.12 4.18
platform.
Table 5: Top 5 word by their tf-idf score on each
That being said, the results shown in Section 3
label.
demonstrate that our experiments can still be im-
proved.
Firstly, the increase in the size of the dataset would
surely enhance the performances of the classifiers. to the classification process, as shown by
In particular, this should be done focusing on the #museichiusimuseiaperti values which
label PP, that, as it can be observed in Table 4, is are seemingly strong enough as a feature to
the less frequent among the four. differentiate PP against the others, but does not do
Furthermore, while the precision for the label PP is a good job differentiating the other labels against
usually higher than the average (note how it reaches each other.
1.00 in our baseline), its recall is very low, even for Those data could give us some insight on how
our SVM classifier, which shows the best results museums communicate through the Twitter
overall. The intuition here is that, while it is usually platform. Indeed, usually, GLAMs tend to use
easy for the classifiers to understand which tweet the same hashtags regardless of their commu-
has the PP label, they are also very “picky”, and nicative intents (even when the hashtag used was
cannot really learn all the features needed in order initially linked to certain initatives), which was
to classify this label against the others. already expected with some general hashtags, like
#iorestoacasa.
Classifier SE PP ItC PI The effects of possible removal or reweighting of
NB P 0.66 1.00 0.58 0.61 these hashtags needs to be further explored.
R 0.37 0.02 0.74 0.76
SVC P 0.66 0.88 0.72 0.60
R 0.48 0.52 0.65 0.81
KNC P 0.36 0.69 0.74 0.68
5 Conclusion and Future Work
R 0.79 0.32 0.43 0.39 In this work, we have described our project
DT P 0.47 0.43 0.51 0.50
R 0.51 0.50 0.53 0.48 for classifying communicative intents in tweets
MLP P 0.59 0.71 0.67 0.68 written by Italian GLAMs during the COVID-19
R 0.62 0.54 0.68 0.67
NN P 0.61 0.77 0.60 0.71
lockdown. Through the experiments and the
R 0.64 0.47 0.71 0.61 following analysis we have shown how this task
can be challenging.
Table 4: Precision (P) and Recall (R) for each label. As future work we will focus on: increasing the
size of the corpus, integrating statistical techniques
Other possible solutions to this issue can be to help dealing with imbalanced labels, and finally
the use of techniques such as resampling and improving the selection and reweighting of the
cost-based methods. features (in particular concerning the hashtags).
Secondly, by focusing on the textual features Another topic which needs further investigation
of the tweets, we can further investigate where concerns the use of different kinds of textual
improvements can be made. embeddings, which might improve the result.
In particular, looking at the top 5 tf-idf scores Once honed, the metholody and the tools we have
for each label (Table 5), we notice that the used in this research could become an important
selected hashtags may occur in all types of tweets asset in better understanding and analyzing cultural
with a low difference among their scores. Such communication on the Web.
a low deviation does not contribute enough
Acknowledgments Antoine Courtin, Brigitte Juanals, Jean-Luc Minel, and
Mathilde de saint leger. 2015. A tool-based method-
This work has been partially supported by ology to analyze social network interactions in cul-
Programma Operativo Nazionale Ricerca e In- tural fields: The use case “museumweek”. In Pro-
novazione 2014-2020 - Fondo Sociale Europeo, ceedings of the Tenth International Conference on
Azione I.2 “Attrazione e Mobilità Internazionale Language Resources and Evaluation (LREC’16).
dei Ricercatori” Avviso D.D. n 407 del 27/02/2018 Stephen Dann. 2010. Twitter content classifcation. In
and by PON Ricerca e Innovazione 2014-2020 First Monday, volume 15.
“Dottorati innovativi con caratterizzazione industri- Nicolas Foucault and Antoine Courtin. 2016. Au-
ale”. tomatic classification of tweets for analyzing com-
Authorship Attribution is as follows: Gennaro munication behavior of museums. In Proceedings
Nolano is author of Section 3.3 and 4, Carola Car- of the Tenth International Conference on Language
Resources and Evaluation (LREC’16), pages 3006–
lino is author of Section 3, 3.1 and 5, Maria Pia di
3013, Portorož, Slovenia, May. European Language
Buono is author of Section 2 and 3.2, and Johanna Resources Association (ELRA).
Monti is author of Section 1.
Pablo Gamallo and Marcos Garcia. 2014. Citius: A
naive-Bayes strategy for sentiment analysis on En-
References glish tweets. In Proceedings of the 8th Interna-
tional Workshop on Semantic Evaluation (SemEval
Dana Allen-Greil and Matthew MacArthur. 2010. 2014), pages 171–175, Dublin, Ireland. Association
Small towns and big cities: How museums foster for Computational Linguistics.
community on-line. Museums and the Web 2010:
Proceedings. Toronto: Archives & Museum Infor- Claudia Giraud. 2020. Il Ministero della Cultura pre-
matics. Published March, 31(2010):84–96. senta un report sulla reputazione online dei musei
durante il Covid, 07. Available online at https:
Bráulio Alturas and Liliana Oliveira. 2016. Con- //www.artribune.com/professioni-
sumers using social media: impact on companies’ e-professionisti/2020/07/il-
reputation. In Proceedings of the Academy of ministero-della-cultura-presenta-
Marketing Conference 2016: Radical Marketing. un-report-sulla-reputazione-
Academy of Marketing. online-dei-musei-durante-il-covid/,
last accessed 10/09/2020.
Dimitrios Amanatidis, Ifigeneia Mylona, Spyridon Ma-
malis, and Irene Eirini Kamenidou. 2020. Social Brigitte Juanals and Jean-Luc Minel. 2018. Analysing
media for cultural communication: A critical inves- cultural events on twitter. In Irene Garrigós and
tigation of museums’ instagram practices. Journal Manuel Wimmer, editors, Current Trends in Web En-
of Tourism, Heritage & Services Marketing, 6(2):38– gineering, pages 213–222, Cham. Springer Interna-
44. tional Publishing.
Lesley A Langa. 2014. Does Twitter help museums
Maria Anzovino, Elisabetta Fersini, and Paolo Rosso. engage with visitors? In iConference 2014 Proceed-
2018. Automatic identification and classification of ings.
misogynistic language on twitter. In Natural Lan-
guage Processing and Information Systems, pages Ana Sánchez Laws. 2015. Museum websites and so-
57–64. cial media: issues of participation, sustainability,
trust and diversity, volume 8. Berghahn Books.
Valerio Basile and Malvina Nissim. 2013. Sentiment
analysis on italian tweets. In Proceedings of the 4th K. Lovejoy and Gregory D. Saxton. 2012. Informa-
Workshop on Computational Approaches to Subjec- tion, community, and action: How nonprofit orga-
tivity, Sentiment and Social Media Analysis. nizations use social media. In Economic & Social
Impacts of Innovation eJournal.
Carola Carlino, Gennaro Nolano, Maria Pia di Buono, NEMO - Network of European Museum Organ-
and Johanna Monti. 2020. Laculturanonsiferma– isations. 2020. Survey on the impact of
report su uso e la diffusione degli hashtag delle is- the COVID-19 situation on museums in Eu-
tituzioni culturali italiane durante il periodo di lock- rope: Final Report, 12. Available online at
down. arXiv preprint arXiv:2005.10527. https://www.ne-mo.org/fileadmin/
Dateien/public/NEMO documents/
Andrea Cimino, Stefano Cresci, Felice Dell’Orletta, NEMO COVID19 Report 12.05.2020.pdf,
and Maurizio Tesconi. 2014. Linguisti- last accessed 10/09/2020.
cally–motivated and lexicon features for sentiment
analysis of italian tweets. In The 4th Conference Brendan O’Connor, Ramnath Balasubramanyan,
for Evaluation of NLP and Speech Tools for Italian Bryan R. Routledge, and Noah A. Smith. 2010.
(EVALITA). From tweets to polls: Linking text sentiment to
public opinion time series. In ICWSM, volume 11,
pages 1–2.
Politecnico di Milano. 2020. La reputazione
online di musei, parchi archeologici, istituti e
luoghi della cultura italiani - Report relativo
al mese di maggio 2020., 5. Available online
at http://musei.beniculturali.it/
notizie/notifiche/report-maggio-
2020-sulla-reputazione-online-dei-
musei.
Daniel Preoţiuc-Pietro, Vasileios Lampos, and Niko-
laos Aletras. 2015. An analysis of the user occupa-
tional class through Twitter content. In Proceedings
of the 53rd Annual Meeting of the Association for
Computational Linguistics and the 7th International
Joint Conference on Natural Language Processing
(Volume 1: Long Papers), pages 1754–1764, Beijing,
China. Association for Computational Linguistics.
Besiki Stvilia and Leila Gibradze. 2014. What do
academic libraries tweet about, and what makes a
library tweet useful? In Library & Information Sci-
ence Research.
Ruth Talbot, Chloe Acheampong, and Richard H
Wicentowski. 2015. Swash: A naive bayes
classifier for tweet sentiment identification. In
SemEval@NAACL-HLT.
Kiran Zahra, Muhammad Imran, and Frank Ostermann.
2019. Automatic identification of eyewitness mes-
sages on twitter during disasters. In Information
Processing and Management, volume 57.
Dejin Zhao and Mary Beth Rosson. 2009. How and
why people twitter: the role that micro-blogging
plays in informal communication at work. In Pro-
ceedings of the ACM 2009 international conference
on Supporting group work, pages 243–252.