In this paper, we focus on applying sentiment analysis to resources from online collections, by exploiting, as information source, tags intended as textual traces that visitors leave for commenting artworks on social platforms. Our aim is to create a semantic social space where artworks can be dynamically organized according to an ontology of emotions. We propose to tackle this issue in a semantic web setting, through the development of an ontology of emotional categories based on Plutchik's circumplex model, a well-founded psychological model of human emotions. The ontology has been conceived for categorizing emotiondenoting words and has been populated with Italian terms. The capability of detecting emotions in artworks can foster the development of emotion-aware search engines, emotional tag clouds or interactive map of emotions, which could enable new ways of accessing and exploring art collections. First experiments on tags and artworks from the ArsMeteo Italian web portal are discussed.
The development on the web and the advent of social media has brought about
new paradigms of interactions that foster rst-person engagement and
crowdsourcing content creation. In this context the subjective and expressive
dimensions move to the foreground, opening the way to the emergence of an a ective
component within a dynamic corpus of digitized contents, which advocate new
techniques for automatic processing, indexing and retrieval of the a ective
information present. Therefore, recently a high interest raised among researchers in
developing approaches and tools for sentiment analysis and emotion detection,
aimed at automatic analyzing and processing the a ective information conveyed
by social media [
In this paper we address the above issues in the context of the
ArsEmotica project [2{4]1. ArsEmotica is an application software that detects emotions
evoked by resources (artworks) from online collections, by exploiting, as
information source, tags intended as textual traces that visitors leave for commenting
artworks on social platforms. The nal aim is to create a semantic social space
where artworks can be dynamically organized according to an ontology of
emotions. Detected emotions are meant to be the ones which better capture the
a ective meaning that visitors, collectively, give to the artworks. We propose
to tackle this issue in a semantic web setting, through the development of an
ontology of emotional categories based on Plutchik's circumplex model [
In the last years, many cultural heritage institutions opened their collections
for access on the web (think for instance to the Google Art project2). User
data collected by art social platforms are a precious information source about
trends and emotions. Therefore, a growing interest in monitoring the sentiment
of the visitors in virtual environments can be observed among art practitioners,
curators and cultural heritage stakeholders, as discussed in [
In the following, we will describe the most recent achievements within the
ArsEmotica project, with a special focus on the development of the ontology of
emotions. The ontology inspired an interactive user interface for visualizing and
summarizing the results of our emotion detection algorithm; detected emotional
responses to artworks are represented by means of a graphical representation
inspired to the Plutchik's emotion wheel. Moreover, we will present rst results of
the ongoing experiments of running the ArsEmotica emotion detection engine on
a real dataset of artworks and tags from the ArsMeteo web portal [
In this section, we brie y recall the characteristics and the main components
of ArsEmotica 2.0, the application software that we developed for testing our
ideas. Details can be found in [
engine", which can be interfaced with any resource sharing and tagging system which provides the data to be processed, i.e. digital artworks and their tags. Social tagging platforms for art collections, having active communities that visit and comment online the collections, would be ideal data sources.
The pipeline of ArsEmotica includes four main steps.
1. Pre-processing: Lemmatization and String sanitizing. In this step
tags associated with a given artworks are ltered so as to eliminate aws
like spelling mistakes, badly accented characters, and so forth. Then, tags
are converted into lemmas by applying a lemmatization algorithm, which
builds upon Morph-It!, a corpus-based morphological resource for the Italian
language [
The resulting output can be produced in di erent modalities. Emotions
evoked by artworks are visualized by a sort of emotion wheel, graphically
inspired to the color wheel used by Plutchik for o ering a bi-dimensional
representation of his circumplex model of emotions [
In this section we describe the ontology, which plays a key role in all steps
of ArsEmotica computation. It is an ontology of emotional categories based on
Plutchik's circumplex model [
The ontology structures emotional categories in a taxonomy, which includes 32 emotional concepts. Due to its role within the ArsEmotica architecture, the ontology has been conceived for categorizing emotion-denoting words, as the one used in the previous version of the application. It, includes two root concepts: Emotion and Word.
Class Emotion For what concerns the class Emotion, the design of the emotional categories taxonomic structure, of the disjunction axioms and of the object and data properties mirrors the main features of Plutchik's circumplex model, (see Fig 1, left side). Such model can be represented as a wheel of emotions and encodes the following elements and concepts: { Basic or primary emotions: joy, trust, fear, surprise, sadness, disgust, anger, anticipation (i.e. expectancy ); in the color wheel this is represented by di erently colored sectors. { Opposites: basic emotions can be conceptualized in terms of polar opposites: joy versus sadness, anger versus fear, trust versus disgust, surprise versus anticipation. { Intensity: each emotion can exist in varying degrees of intensity; in the wheel this is represented by the vertical dimension. { Similarity: emotions vary in their degree of similarity to one another; in the wheel this is represented by the radial dimension. { Complex emotions: complex emotions are a mixtures of the primary emotions; in the model in Fig 1 emotions in the blank spaces are compositions of basic emotions called primary dyads.
Emotion is the root for all the emotional concepts. The Emotion's hierarchy includes all the 32 emotional categories presented as distinguished labels in the model. In particular, the Emotion class has two sub-classes: BasicEmotion and ComplexEmotion. BasicEmotion and CompositeEmotion are disjoint classes.
Basic emotions of the Plutchik's model (Disgust, Trust, Sadness, Joy, Anticipation, Surprise, Anger and Fear) are direct subclasses of BasicEmotion. Each of them is specialized again into two subclasses representing the same emotion with weaker or the stronger intensity (e.g. the basic emotion J oy has Ecstasy and Serenity as subclasses). Therefore, we have 24 emotional concepts subsumed by the BasicEmotion concept. Instead, the class CompositeEmotion has 8 subclasses, corresponding to the primary dyads in the Plutchik's model.
Other relations proposed in the Plutchik's model have been expressed in the ontology by means of the following object properties, where Arch is the set of the basic emotions and Comp the set of the complex emotions: { hasOpposite: (f : Arch ! Arch), encodes the notion of polar opposites ; { hasSibling: (f : Arch ! Arch) encodes the notion of similarity ; { isComposedOf : (f : Comp ! Arch) encodes the notion of composition of basic emotions.
The data type property hasScore:(f : Arch ! R) was introduced to link
each emotion with an intensity value mapped into the hourglass model.
Class Word Word is the root for the emotion-denoting words, i.e. those words
which each language provides for denoting emotions, in line with related and
previous work [
We semi-automatically populated the ontology with Italian words by following
the same methodology described in [
Our population process started by manually selecting a set of
representative Italian emotional words, at least one word for each concept. This initial set
was including less than 90 words classi ed under our 32 emotional concepts, but
they were only nouns. In order to expand with adjectives the set of Italian words
representative of emotional concepts, we included and classi ed according to the
ontology6 the list of 32 emotion terms in [
In a second phase we automatically expanded the set of individuals (emotion
denoting words) belonging to the emotional concepts by exploiting
MultiWordNet and the WordNet-A ect. All manually classi ed words and adjectives were
used as entry lemmas for querying the lexical database. The result for each word
was a synset, representing the \senses" of that word, labeled by MultiWordNet
unique synset identi ers. Each synset was then processed by using
WordNetA ect [
WordNet relation derived-from, for which can be assumed that the a ective meaning is preserved. Therefore, all synsets obtained by an application of the derived-from relation (and not yet classi ed in our ontology) were included as individuals of the proper emotional concept. At the end of the process a human expert checked the identi ed terms. The resulting ontology contains about 700 Italian words referring to the 32 emotional categories of the ontology. 4
We are currently testing the version 2.0 of the ArsEmotica prototype against a
dataset of tagged multimedia artworks from the ArsMeteo art portal (http://
www.arsmeteo.org [
Our dataset ArsM is a signi cant set of tagged artworks from the ArsMeteo web portal. It consists of 9171 artworks with the associated tags7. The ArsMeteo web platform combines social tagging and tag-based browsing technology with functionalities for collecting, accessing and presenting works of art together with their meanings. It enables the collection of digital (or digitalized) artworks and performances, belonging to a variety of artistic forms including poems, videos, 7 Speci cally, ArsM includes comments associated to the artworks from Arsmeteo users until December 2010. pictures and musical compositions. Meanings are given by the tagging activity of the community. Currently, the portal collected over 10,000 artworks created by about 300 di erent artists; his community has produced over 37,000 tags (an average of 10 tags per artwork). 4.2
Emotions belonging to the ontology are detected in about 20 percent of our dataset8. Let us denote with Af f ectiveArsM the set of artworks classi ed according to some emotions of our ontology after the emotional analysis performed by ArsEmotica.
In ArsMeteo, artworks usually have many tags, expressing a variety of meanings, thus supporting the emergence of di erent emotional potentials. This is consistent with the idea that art can emotionally a ect people in di erent ways. When this happens, the analysis performed by ArsEmotica provides multiple emotional classi cations. Fig 2 shows results on number of emotions detected for each artworks in Af f ectiveArsM . About 40% of the artworks received multiple classi cation, i.e. ArsEmotica detected more than one emotion associated to the artwork.
For what concerns the emotion distribution in Af f ectiveArsM , when we consider basic emotions in their varying degree of intensities, the most common emotions were the ones belonging to the sadness (457 artworks) and joy family (405 artworks), followed by anticipation, fear, disgust and surprise. Anger was 8 Notice that the tagging activity, monitored in ArsMeteo since 2006, was not performed with the aim of later applying some kind of Sentiment Analysis, but as a form of spontaneous annotation produced by the members of the community. rarer, and trust was almost nonexistent (see Fig. 3); when we consider complex emotions, results are summarized in Fig. 4: love is very common (424 artworks), optimism and awe are rare, and the other complex emotions are almost nonexistent. 5
We have developed an interface linked to our ontology of emotions, which have as main aims: a) to present the outcomes of the emotional analysis for tagged artworks, b) to propose to the user intuitive metaphors to browse the emotional space, and c) to ease the task of emotionally classifying tags having indirect emotional meanings, by means of emotional concepts from the ontology. On this perspective, the Plutchik's model is very attractive for three main reasons. First, the reference to a graphical wheel is very intuitive and o ers a spacial representation of emotions and their di erent relations (similarities, intensities, polar oppositions). Such kind of representation allows to convey to the user a rich information on the emotional model, without referring to tree-like visualization of the ontology hierarchy. Second, the use of colors for denoting di erent emotions provides a very intuitive communication code. Di erent color nuances for di erent emotions, transmit naturally the idea that primary emotions can blend to form a variety of compound emotions, analogous to the way colors combine to generate di erent color graduations. Third, the number of emotional categories distinguished in the wheel is limited. This aspect facilitates the user that is involved in an emotional evaluation.
The Interface The sequence of interactions o ered to the user follows the ux of computation sketched in Section 2. After the user selects an artwork from the collection (Fig. 5, top-left window), the application applies the emotional analysis on the artwork tags. The result of this computation, i.e. the evoked emotions, is summarized to the user by a graphical representation (obtained by adapting the RGraph tool9) called \La rosa delle emozioni" which strongly recalls the Plutchik's color wheel. Let us consider, for instance, to run the emotional analysis to the artwork\Dove la Ra nata Ragazza Bionda guarda il Grosso Toro Malmorto" by Filippo Valente, belonging to our ArsM dataset. The resulting window (Fig 5, top-left window), includes a preview of the artwork and a summary of related metadata (e.g. title and author of the selected artwork); below, the four red colored tags are identi ed as emotional according to the emotional ontology: `orrore', `infamia', `cattiveria', `tristezza'; the presence of emotional responses related to Sadness and a strong disgust (Loathing ) is highlighted by coloring the sectors of the emotion wheel corresponding to those emotions. Internal sectors of the ArsEmotica's wheel are intended as representing light intensity of emotions, while the external ones as representing high intensity. Underlined blue colored tags denotes tags that have been recognized by the sentiment analysis stage as possibly conveying some a ective meaning. Then, they appear as active links for the user's emotional feedback: see e.g. `sangue', `scon ggere', and so on.
Indeed, a user which is not satis ed with the outcome can select a tag to evaluate among the active ones. Then, the application activates a pop-up window, where an uncolored emotional wheel is shown. Users can express the emotional evaluation in terms of basic emotions with di erent intensities, and color the wheel accordingly, by clicking on one of the 24 sectors of the wheel; otherwise 9 http://www.rgraph.net/ they can select compound emotions, by selecting the wedge-shaped triangles inserted between the basic emotions. In our example (Figure 5, bottom-center) the user associated to the tag `sangue' (blood) the emotions Fear and Disgust (with high intensity, which corresponds to Loathing ). Notice that the tag evaluation is contextual to the vision of the artwork, which indeed remains visible in the background. After user expressed her feedback, detected and collected emotions are combined and the resulting emotional evaluation is again presented to the user by using the ArsEmotica's wheel. 6
In this paper we have described the OWL ontology of emotions used in the ArsEmotica 2.0 prototype, which refers to a state-of-the-art cognitive model of emotions and inspired an interactive user interface for visualizing and summarizing the results of the emotion detection algorithm.
Recently, many researchers are devoting e orts in developing ontology of
emotions in the Semantic Web context [
The Hourglass Model we refer to in order to combine detected and collected emotions in ArsEmotica allows us to design a uid and continuous emotional space, where artworks (but also possibly user's tag) can be positioned. The actual ArsEmotica interface provide our users with the possibility to access the outcomes of a the emotional analysis. On this line, the next step is to study innovative strategies to browse the artworks, by relying on their semantic organization in the ArsEmotica emotional space. The aim is to provide users with the possibility to explore the resources by exploiting the various dimensions suggested by the ontological model. Possible queries to deal with could be: \show me sadder artworks" (intensity relation); \show me something emotionally completely di erent" (polar opposites); \show me artworks conveying similar emotions" (similarity relation).
For what concerns sentiment visualization, designing engaging interfaces that
allow an appropriate granularity of expression is not an easy task. We plan to
evaluate soon the new prototype and its interface, by carrying on a user test
where users of the ArsMeteo community, which in the past have already actively
participated to a user study on the rst version of our prototype [