The study of social systems shows that there is a relationship of mutual influence between social connections and individual behavior, known as homophily. In this work, we developed a methodology to allow the analysis of interests of groups of users in Twitter network, based on automatic community detection and tweets ranking. The techniques presented reveal evidences that the presence of communities is related to topic specialization, and allow the characterization of elaborate profiles of groups of users based only on their location on the network.
The experience of the last decade has shown that online
social networks (OSNs) are not only useful for the
amusement of Internet users, but can also be a valuable source of
data for the study of social systems. The millions to billions
of users who access OSNs services everyday are providing
researchers an unprecedented possibility to gather
information from human activity and social behavior, enabling the
investigation of complex issues [
One of the interesting topics that can be explored in this
context is the creation and diffusion of content. OSNs users
are constantly involved in complex dynamics of message
sharing, which may result in the emergence of new trends,
collective mobilization and opinion formation.
Understanding the essential mechanisms of this process can be useful to
areas such as politics [
This paper explores the interplay between social connections and content shared by individuals in an online social network, which is one aspect of social communication. We investigated here how it is possible to understand behavior and interests of a group of individuals based on their social connections, paying special attention to the role of homophily – the tendency for someone to establish connections to similar peers. Analytical tools were developed – and applied to real data extracted from OSNs – to pinpoint content relevant to the understanding of interests of communities and users composing them. The techniques used are based solely on the knowledge of the users that shared each message, not resorting to their contents. We believe that, in addition to indicating the relevance of sharing information for the classification of messages, such tools can be useful to the practical study of social systems and to the development of new applications, such as recommendation systems.
The paper is structured as follows. In Section 2 we introduce relevant literature on OSN and homophily. In Section 3, we present the methods used to acquire and analyze data. Then, in Section 4, we report and analyze the results obtained using our methodology. Finally, in Section 5, we present final remarks, discussing implications of this work and possible future directions. 2.
An increasing number of studies have been conducted over
the last years focusing on online social networks [
An important aspect investigated is the presence of
selforganized processes. Despite not having central controllers
that rule on how content is disseminated or connections
between users are created, OSNs display many organized
behaviors. Common examples of such process are the collective
curation of contents [
Self-organizing processes also take part on how
connections are formed in an OSN. Usually, social connections are
not created uniformly between all individuals, but are
concentrated in few hubs [
Considering the flow of information that travels over the
network, groups with highly connected individuals may
imply a redundancy of communication channels, possibly
enhancing information that pass through or are generated by
these groups. In the case of online social networks, it can
imply that a content may be more easily proliferated and
reinforced inside a community after it is shared by a member
of such group. While communities are expected to influence
on how their members receive and process information, it
is also believed that community formation is influenced by
pre-existing affinities between members [
Researchers identified in social networks outside the
virtual world a tendency (not only within communities) of
individuals with common interests to be usually connected to
each other [
Kwak et al. [
Bollen et al. [
Some of the most common approach to community
detection are modularity-based algorithms [
In order to address this issue, Rosvall and Bergstrom
[
As Twitter has plenty of public data available online, it is a good source for creating a database of social events. However, the rate limits imposed by its API1 hamper the download of large volumes of data from events that took place in the past. An alternative is to use Twitter’s Streaming APIs, which allows the download of messages in real time, as they are posted.
Therefore, in order to collect a satisfactory amount of data to be analyzed, we decided to track the interactions between a popular user account producer of original content and the users that share (i.e. retweet) these messages. As popular Twitter accounts interact with many users daily, this approach revealed to be an effective way for collecting message diffusion processes and user interactions as they happen.
We chose the Brazilian largest newspaper Twitter account, Folha de Sa˜o Paulo2, and collected: original messages posted, retweets of those messages posted by other users, account details of those users and the relationships (followers and followees) of all of them. 3.2
As with many other newspapers, almost all messages published by the chosen source are headlines, followed by a link to the newspaper’s website with the news’ full content. As the news articles on the website belong to thematic categories (newspaper’s sections), it was possible to automatically attribute a class to each tweet, based on these categories. This procedure was carried out to all tweets and six most common topics were verified, namely: “daily life news”, “sports”, “world”, “politics”, “entertainment” and “market”. 3.3
After collecting and classifying all data, we then
evaluated whether retweeting behaviors of communities’
members are coherent among themselves regarding the subjects
of the shared messages. In order to detect groups of tightly
connected users from the social connections observed, we
executed Rosvall and Bergstrom’s community detection
algorithm [
Understanding the interests of a community is a hard task.
To address such issue, we propose here an adaptation of a
well-known statistics from text mining, the term
frequencyinverse document frequency (tf-idf ) [
The tf-idf method usually considers a corpus of documents, each composed of different terms. By comparing the frequency of specific terms inside and outside documents, values are assigned to each term, pondering its importance 1https://dev.twitter.com/rest/public/rate-limiting 2https://twitter.com/folha for each document. The tf-idf of a term t in a specific document d is calculated as follows:
tf idfd(t) = tfd(t) × idf (t), where tfd(t) is a value that is higher the higher the frequency of the term t in d while idf (t) is inversely proportional to the frequency of t in all documents of a corpus.
In the traditional use of the tf-idf algorithm in text-mining, a term is a word and a set of terms is a textual document (e.g. a book, or a webpage). This creates a document-term matrix, where each row represents a document, each column a word and each cell, indexed by (i, j), the frequency of the term j in document i. For the case of our application, we adapted this approach by considering each tweet as a term and each community as a document. Therefore, we built a community-tweet matrix where each cell (i, j) represents the number of retweets of tweet j in community i.
Thus, when the same operations of tf-idf are applied to a community-tweet matrix, tweets that are both highly shared by members of a community and more akin to such community’s behavior are highlighted with higher scores than the others.
From March 19, 2014 to September 21, 2014, all messages (tweets) posted by the source account, as well as any share of this content by other users (retweets) were collected.
It was possible to track, from the data collected, a large amount of information diffusion processes triggered by the observed source. During the observed period, 13463 distinct and original messages posted by the source account were collected. From this set, a series of filters were sequentially applied, forming a more appropriate dataset for the work, as described below: • Filter 1 - Only messages which had received at least 20 retweets were selected, resulting in a group of 4671 distinct messages; • Filter 2 - Messages not belonging to one of the six main categories (“everyday news”, “sports”, “world”, “politics”, “entertainment” and “market”) were removed from the above set, resulting in a group of 3185 messages; • Filter 3 - Users who retweeted more than 60% of the publications were considered automated scripts (bots) and therefore removed from the database and from the retweets count (just one user was identified as such); • Filter 4 - As in 2014 Brazil held the Football World Cup and presidential elections, there were a high number of messages in categories “politics” and “sports”. In order to balance the proportion of messages in each topic, a maximum limit of 450 messages was set for each category.
The filtered data resulted in a collection of 2444 messages, of which 44320 distinct users retweeted one or more messages at some moment. From the users lists of followers and followees, it was possible to characterize a network, registering the social connections between them. The source user, Folha de Sa˜o Paulo, was removed from the network, so that the interactions among its followers became the focus of the analysis. Table 1 presents a more complete characterization of the collected data and the network formed between users. It is worth pointing, also, that most users do not participate actively on the diffusions, implying in high diversity of users participating in the processes, but low recurrence: during the observation period, each user retweeted in average only two messages from the source. 4.2
After running the community detection algorithm, 4278 communities were detected, many with 2 elements (2 connected users, isolated from the rest of the network), but 26 larger groups with 200 users or more have also been identified.
A first experiment involving communities consisted of checking how coherent communities behaviors were, by comparing the frequency at which specific messages were shared inside communities and in the complete network. For each community with a relevant number of users (200 or more), it was computed the number of retweets for each Folha de Sa˜o Paulo’s original tweet. A high coherence was verified in the behaviors of individuals belonging to the same community. Table 2 compares the sharing rates inside and outside the four largest communities, showing the five most distinctive cases where the community has singular sharing behavior, differentiating from the global behavior.
The cases shown on Table 2 indicate that there is a certain level of coordination in the selection of shared messages by each community. It is interesting to see how some messages are much more emphasized inside a community, than it was in the general network. Community 2, for example, present higher sharing rates for its messages, compared to the rest of the network. Equally interesting are cases where the community seems to suppress the spread of a message, as seen in community 8, where highly retweeted messages are less emphasized by the community’s members.
For comparison purposes, an attempt to break the relationship between user connections and postings was made, by randomly swapping the retweet pattern of different users on the database, while keeping their social connections. Thus, Community 1, members: 2621, total tweets: 12292 1st 2nd 3rd 4th 5th Inside 2.5% 1.9% 2.1% 1.7% 1.4% Outside 0.6% 0.3% 0.6% 0.2% 0.0% under this new experiment, the communities remain defined as they were originally, but their retweets have patterns from users of different communities, in order to eliminate any homophily related to retweeting behavior but keeping intact other characteristics of our data (as global retweet counts and correlations between sharings by individual users). The same analysis made before is now performed on the randomized dataset, resulting on Table 3. It becomes evident that when homophily is suppressed, communities lose their particular behavior and present sharing rates closer to the whole network rates, indicating that differences in retweeting patterns are not only artifact of communities’ finite sizes. dex3 for their number of retweets within each community in the real data with the same index for the data in the randomized dataset. Our results indicate that the real communities have stronger preferences for specific messages, with satisfactory statistical significance when considering all communities (Wilcoxon test, p = 1.23 ∗ 10−15, Gini index difference of 1.5 ∗ 10−4) and even more pronounced when restricting the comparison only to larger communities, with at least 50 individuals (Wilcoxon test, p = 2.13 ∗ 10−4, Gini index difference of 8.26 ∗ 10−3). 4.3
A second experiment consisted in the analysis of how topics are distributed among communities. For this, the six standard categories were considered and the number of messages of each category shared by each community was computed. First, Figure 1a shows the general distribution of messages per topic for the whole network. Then, Figures 1b1h show the distribution for seven distinctive communities chosen among those with more than 200 members. The figures demonstrate how communities’ topic distribution may have different profiles compared to the rest of the network.
From the graphs presented, communities 1 and 16 (Figures 1b and 1f) seems to have a stronger interest in political issues, with community 1 showing an interest in daily life news slightly above the average. Community 12 (Figure 1e) also shows a higher interest in politics, but divide it with a focus on sports news. Community 21 graph (Figure 1g) shows a preference for daily life news, but not in a very distinctively way. Community 10 (Figure 1d) does not have a distribution very different from the whole network (Figure 1a), being a representative of the average preferences. Community 25 (Figure 1h), in turn, is remarkably different, with over 50% of its retweets being about entertainment and practically all the rest regarding sports news, almost ignoring the other topics. 4.4
By applying the tf-idf normalization on the retweet counts of the communities we identified the most characteristic tweets for each community. Table 4 presents the top five messages for the largest communities in the network. The messages content were translated from Portuguese to English, with some translation notes (in brackets), where necessary.
It was possible to deepen the analysis of each community profile beyond what would be possible by simply looking to the distribution of general topics in each community. Analysing each group of messages individually, it is possible to notice specific and subjective categories. For example, although communities 1 and 16 both have an emphasis in politics, community 1 seems to be supportive to the Brazilian government – focusing on good results of politics made by the government and scandals of the opposition – while 16 appears more involved in topics related to (at the time) election’s opposition candidates.
A very interesting conclusion comes from the analysis of the relevant messages from community 12, as we discover that the tweets are not connected by the newspaper sections, 3The Gini index is a measure of how unevenly a value is distributed among elements of a group – in our case, if the Gini index is close to 1 then most of the retweets seen in a community were associate with few messages, if its value is near 0, then the distribution of retweets is closer to uniform. (a) Whole network (b) Community 1, members: 1022, retweets: 3160 (c) Community 9, members: 272, retweets: 636 (d) Community 10, members: 283, retweets: 861 (e) Community 12, members: 316, retweets: 820 (f) Community 16, members: 215, retweets: 527 (g) Community 21, members: 268, retweets: 745 (h) Community 25, members: 217, retweets: 254 but by subjects regarding the Brazilian state of Pernambuco and its capital, Recife. All the messages presented were related to events taking place in the state, involving both politics matters (police strike) and sports (2014 FIFA World Cup events). This analysis shows both the limitations of the standard categorization of topics (the six classes defined by the newspaper) and the potential of the tf-idf technique on revealing new subjective connections among messages.
Another strong topic specialization is noticed in community 9, where all the top five messages are related to the football team Palmeiras, giving evidence that the community consists mainly of the team’s supporters. Community 25 is specialized in entertainment topics, showing an apparent tendency to emphasize messages related to international pop culture. Interestingly, although the topic distribution in this community was predominantly on entertainment and sports, the tf-idf normalization reveals the relative relevance of messages in other categories, such as world and market (market is the least shared topic among the six categories). Sports tweets were not present among the top five messages, which can probably mean that the sports messages shared by the community followed the general distribution, not revealing a distinctive behavior of the community.
This kind of qualitative analysis of communities behavior could be made with most communities detected in the network, but are not presented here, due to space constraints. Other examples of topic specialization present in communities include: regional news (from diverse Brazilian states), international politics, economy, football discussions (in general and regarding specific teams) and corruption.
This research presents a computational framework for
general investigations on collective behavior. When applied to
a large dataset, the method presents new evidences of
homophily in Twitter’s network. Despite the existence of
homophily in Twitter was already found in different studies [
The use of tf-idf jointly with community detection was able to group and order messages according to their relevance to a social community, enabling the characterization of complex behavior profiles inside communities. The presented method was able to reveal more nuanced classes of contents, such as political positions, regional matters, fan clubs, that were not covered in the original six categories, defined by the newspaper’s staff with the specific purposes of organization and classification. It is relevant to notice that, beyond Twitter, the same technique can be applied to the analysis of different sets and databases from social networks, enabling similar studies for different services and contexts.
The empirical results of this study show new concrete evidences of how individual behavior and social connections are closely related, as expected in theory. In fact, so much information is present in social connections that it was even possible to find groups of similar messages without any knowledge of their content, using techniques that only consider network properties and sharing behaviour. This is reflected on the list of the most representative messages for each community (Table 4), which are usually related to few subjects. Accordingly, by aggregating information about which communities interacted with some object (a tweet, for instance) to other techniques (e.g., natural language processing), it may be possible to gather new knowledge about such object that are not evident in it (as social or geographic contexts). In an extended perspective of this result, one can wonder if it might be possible to infer significant elements about the nature of a process happening on a social network even without access to the content traveling through such network, if there is information available about its structure and dynamics.
Ethical implications of the power of such methods for detecting users specific interests, without the direct access to their personal information, should be considered. If, on one hand, this knowledge can be used in order to improve the performance of useful systems, as machine learning algorithms, on the other hand it may also incur in risks to privacy and security. This discussion is not conducted in details here, but should not be forgotten.
The communities observed had elements of cohesion on their general behaviors, emphasizing or even repressing the spread of certain types of content. An interesting conflict between individual autonomy and collective behavior seems to be part of information diffusion processes that take place on OSNs self-organized in communities. The community specialization in topics of interest is also evidenced. On a context of proliferation of many different subjects, the limitation of the scope of themes discussed within a community can be an efficient strategy for individuals to deal with information overload.
Further steps of the research include a deeper analysis of a database including more messages from multiple sources, using text-mining to define messages subjects and comparing such classification to results obtained by tf-idf. In future studies, the homophily of sharing behavior in online social networks can be subject of a deeper analysis, developing new methods to try to determine how much of it is due to (1) preference of individuals to establish new social connections to similar peers; (2) social influence; (3) indirect homophily, which occurs due to the existence of homophily of another trait (e.g., if two individuals usually access Twitter in the same hours of the day, despite of their connections they will be more likely to read the same news and, thus, retweet it). An even deeper analysis can be made about social influence, in order to be able to divide it into its reactive part – an individual exhibits a sharing behavior in favor of some subject because his/her community publishes more about such subject, not because of an inner preference – and its cognitive part – by observing his/her peers’ behaviors, an individual shapes his/her preferences according to those practiced by his/her peers.