Nowadays people share everything on online social networks, from daily life stories to the latest local and global news and events. Many researchers have exploited this as a source for understanding the user behaviour and profile in various settings. In this paper, we propose two quantitative methods that investigate the relevance of the published photos about a cultural event in terms of knowledge that can be extracted, user behaviour and relation to the context of the event. We show our approach at work for the monitoring of participation to a large-scale artistic installation that collected more than 1.5 million visitors in just two weeks (namely The Floating Piers, by Christo and Jeanne-Claude). We report our findings and discuss the pros and cons of the analysis.
Today social networks are the most popular communication channels for users
looking to share their experiences and interests. They host considerable amounts
of user-generated materials for a wide variety of real-world events of di↵erent type
and scale [
Despite the importance of social media, the number of studies and analyses
on the impact of cultural and art events in social networks is rather limited,
and focused on English-only content or are tailored to only one specific site,
with addressing one type of document e.g., textual messages, photos or videos.
Moreover, due to the noisy nature of the data extracted from social media,
especially ungrammatical and ambiguous textual features, previous works [
In this paper we aim to analyze visual social media content specifically photos related to a cultural or art event on Instagram. We capture the visual features of photos (namely colors, concepts, and demographics of people), we extract contextual and behaviour knowledge about what and how users share about the event, and then based on this we can tackle our main research questions: (1) finding the relevance between the shared photos about an event and the event itself, and (2) extract a summary of the statistics of the event and its attendees. Our findings can help marketing and event organizers in creating engaging content that communicates more e↵ectively with their audiences and their future customers.
The paper is organized as follows: Section 2 discusses the related work; Section 3 describes our methods and data; Section 4 reports the outcomes of the analysis. Finally, Section 5 concludes and outlines the future work. 2
Several recent researches proposed techniques for identifying social media
content for planned events. Many of these approaches like [
In recent years, creating e↵ective content for social media marketing
campaigns has become a challenge to understand what drives user engagement. While
researchers have applied various methods to study how users engage with textual
[
In contrast with these e↵orts, we focus on analyzing the di↵erent aspects of event-related visual content on Instagram and show it at work on a real case study.
Our main objective in this work is to exploit the knowledge that can be extracted via low-level and high-level features of shared images for finding the relevance between the shared photos about an event and the event itself. We follow two quantitative approaches to investigate the relationship between content features of Instagram photos and a cultural or art event.
The first approach employs the concepts (i.e., objects or entities detected in the image) that can be extracted from photos to find the level of relevance of the image; based on this, we classify the images into two classes, as relevant and irrelevant.
The second method finds relevant images by analyzing the color schema of each photo and specifying the relevance based on existence of the main color pattern(s) related to the event.
In this section, we describe how we collected and analyzed the data, and present a statistical overview of our case study. 3.1
This study exploits Instagram and Twitter datasets from a famous artwork called ”The Floating Piers” that was created by the world-renowned artists Christo and Jeanne-Claude 1 and exposed to the public view at the Lake Iseo in Italy, from June 18 through July 3 2016 (see Figure 12).
We use this artistic event as a use case for our methods. We extracted the social media content relevant to the event, during a time period from June 10th 1 http://christojeanneclaude.net/projects/the-floating-piers 2 Photo Credits:Sailko, Monte Isola. License: Creative Commons Attribution-Share
Alike 3.0 Unported. to July 30th 2016, that contains 30,256 Instagram posts and 14,062 tweets, using Twitter and Instagram APIs.
Figure 2 illustrates the total numbers of Instagram posts vs. tweets within a timeline. One could conclude that Twitter users have a tendency to tweet about the news at the moment when an event starts, whereas Instagram users usually share their experiences when an event ends. To have a clear intuition of the level of user engagement in Instagram, the volume of likes and comments received by uploaded posts are depicted in Figure 3. As demonstrated, Instagram users are more interested in liking the posts rather than commenting, that is why the number of comments is much less than likes count and remains on a constant rate during the time interval.
According to the statistics, unlike Instagram users, most Twitter users are not willing to specify the location of their published tweets. We displayed the density of Instagram posts on geographical plots in Figure 4. As one can see the density of posts has a direct relationship with their locality which means most Instagram posts have been published near the main venue of the event. 3.3
Our research process continued with collecting a random sample of Instagram posts (3000) because of the limitation of requests in Clarifai API. Then we captured and stored available visual features namely concepts, colors schema (a) Italy (b) Lombardy Region (c) Iseo Lake
Fig. 4. Density of Instagram posts in di↵erent coordinates and demographic features of people (faces) in photos including age, gender and race, using Clarifai.
In order to evaluate our proposed methods, we designed a web-based survey3 consisting of two questions about each Instagram photo: 1- Is this photo related to the Floating Piers event? 2- Does this photo contain the Piers? We asked three people to answer these questions for all 3000 photos that we had in the dataset.
In the first approach, we try to find the relationship between the event and the concepts in the photos that are captured by Clarifai. Theoretically speaking, if the concepts found in the photos are similar to the real concepts of the event, we can conclude that those photos are related to the event and thus are not spams. To make this method quantitative, we assign a numerical weight to each concept which is its normalized frequency (number of repetitions) in the set of photos. This way the most frequent concepts (e.g., travel, water, sea, outdoors) gain higher weights than other words. Subsequently, we sum all the weights corresponding to a photo to calculate the final score of that photo. After finding the right threshold for this score, we determine which photos belong to the event. In the end, we compare the results of the survey and this method by computing performance measures that will be explained in section 4.2.
In the second approach, we try to find the relationship between the event, in particular the piers’ structure, and the top colors in the shared photos that can be extracted by Clarifai. To recognize the presence of the Floating Piers artifacts in the photos, we search through all extracted colors of each photo and check if there are any colors in a specific shade (the piers’ color shade). Then we compare the results of the survey and this method by computing performance measures that will be explained in section 4.2. 4
In this section, the most significant results of the experiment over the case study are shown and discussed. 4.1
Using Clarifai API, we can exract the number of faces (people) in each photo and each person’s dempgraphic features such as gender, age and race. As presented in Figure 5, nearly 75 percent of shared photos do not include a face (person) while 12 and 14 percent of photos include one person and a group (two or more persons) respectively. However, the avergae number of likes and comments that photos containing one person gained is almost equal to the avergae number of likes and comments of the majority of photos (with no face). One can conclude that portraits (and selfies) receive more attention from users in Instagram.
According to the data extracted from Clarifai, approximately both female and male equally participated in the event (50.4%, 49.6%). Moreover, as shown 3 https://goo.gl/etvZqM in Figure 6 and Figure 7, three quarter of attendees were between 25 and 45 years old and 67 percent of them were white.
One of the most popular features of Instagram is that it allows its users to capture and customize their photos and videos with several filter e↵ects. Considering that, we extracted the filters applied on photos to see if the users were interested in using filters for their photos taken from The Floating Piers or not. The results are indicated in Figure 8 and shows that more than half of the photos were uploaded on Instagram with no filter. 4.2
As explained in section 3.3, in the first method we extracted the concepts of each photo using Clarifai API and then computed the relevance scores. Figure 9.a shows the most frequent concepts (words) appeared in all photos. Besides,
Fig. 8. Top filters vs. the number of photos in favour of comparision between these concepts and user generated content, we extracted the hashtags of each photo using Instagram API (Figure 9.b). As it can be seen Instagram users, in this event, do not usually tend to use hashtags to describe their shared photos using existing concepts in the photos. (a) Concepts
(b) Hashtags
Subsequently, in order to find the right threshold for the calculated relevance scores, we use discrete derivative which is an analogue of derivative for a function (here the descending order of scores) whose domain is discrete. As can be seen in Figure 10, the value of the discrete derivative is maximum when the relevance score is 2.4. So we set the threshold to this number and consider all the photos with scores lower that this threshold as irrelevant.
As mentioned earlier, in the secound method we extracted top colors of each photo and then we used a specific color shades to distinguish between photos comprising the piers and the rest. As shown in Figure 11, the shades of orange are the biggest portions among the four main ranges of the colors, which makes sense because the color of the fabric used to make the piers is also in this color spectrum.
Once we have built our methods or models, the most important question that arises is how good they are. Therefore, to evaluate our methods we use Confusion Matrix in which true condition corresponds to the survey results and predicted condition corresponds to the outcomes of our proposed methods. Considering this matrix that is often used to describe the performance of a classification model, we calculate precision, recall and accuracy measures for each method separately and indicate their values in Table 1.
As one can see in this table, the accuracy of the first method is higher than the second one. Since our datasets are symmetric, which means that the values of false positive and false negative are almost the same, we can conclude that model with higher accuracy is a better model in terms of performance. Besides, the higher values of precision and recall for the first method are approved seals on the preference of this method. 5
In this study, we proposed two quantitative methods to probe the relationship between features of Instagram photos and a cultural or art event and then employed an online survey to evaluate these methods. We used The Floating Piers event as a case study to show how the proposed approachs work with the real life scenarios.
Based on the outcomes of these two approaches we can conclude that employing concepts of photos (first method) eventuates more accurate results rather than using the extracted colors (second method). The reason behind that can be the high diversity of images in terms of angle of photography, time of the day, usage of Instagram filters etc., which can led to less precise analysis over colors. Furthermore, the resemblance of piers’ color and other objects namely faces, foods, etc. in a picture can be another reason for the lack of accuracy in the second approach.
The current study can go further with considering other social media platforms such as Facebook, Google+, Flickr, etc. that might result in a clearer and wider picture of the characteristics of the event.