As social media continues to grow, the connectivity between individuals and organisations becomes tighter, and the availability of data becomes more immediate, constant and abundant. Aside from conversational chat, social media platforms are being used to share relevant information and to report news. As information credibility becomes an increasing concern, there rises an important question regarding the impact of a rumour. We address this challenge, focusing on rumour impact on social media. In this paper, we measure the impact of a given rumour, impact that will be calculated by a formula we suggest, representing social media user engagement measures. Our results indicate that rumours do differ in terms of impact, with some rumours representing higher impact. Analysis is then conducted in an attempt to understand why some rumours are more impactful than others.
1
The explosion of social media has characterised Internet growth in recent years. Some
original social networks included AOL, chat rooms and Live Journal. While many have
come and gone, some more notable than others, social networking is no passing trend,
with market leaders, such as Facebook, Twitter and WhatsApp, boasting billions of
users. Social networks provide an online voice to just about anyone. Users can publish
their thoughts, opinions and ideas, through online communities. Today's Internet is
flooded with such user-generated content, and opinionated material in particular [
Rumours are prevalent in our society. From the home to the once, they influence our
beliefs and behaviours toward others and generally affect the way we see the world [
these media. As a result, news organisations are losing their audiences to lies and unverified stories, costing them both money and reputation. Misinformation can also endanger life if adopted by individuals during times of crisis.
There is an increasing need to interpret and act upon rumours spreading quickly
through social media, especially in circumstances where their veracity is hard to
establish [
In our study, we consider an important property of rumours, the temporal
characteristic that exists, related to the rumour's lifetime. During this lifetime, the rumour spreads
and is received by individuals that come across it. Many studies have been done
focusing on the long term spreading of rumours, and the speed of spread [
We are interested in understanding a speci_c topic related to the behaviour of rumours.
Psych logical research has been cyclical for many years, while technological research
is of very recent interest, following the birth and success of the Internet and social
media, giving rise to new ways of communicating rumours to large audiences. The ability
to understand and control the type of information that propagates social networks has
become ever more important [
Rumour research is a topic of historical interest, as it is a problem central to human
psychology. [
The problem of modelling frequency profiles of rumours in social media was
introduced in [
Studies within rumour impact have largely focused on the ways in which rumours affect
people, their beliefs, and various aspects within their lives. In [
We adopt the approach as implemented by [
In this example, messages related to the rumour, "The movie 'The Notebook 2' has
started filming.", are expected to be collected. To collect messages related to another
rumour, the same process is followed - choosing keywords and crafting a regular
expression. This creates unique groups of rumorous tweets relating to specific rumours.
We favour this detection approach, over others such as [
Rumours are detected and gathered from the Twitter API, necessary filtering is performed, and metadata related to these tweets is parsed. 4.1
Gathering The Gathering process submits query strings to Twitter's Search API, queries that reflect regular expressions, constructed of keywords specific to the desired rumour. There are pre-processing requirements that are met prior to rumour search. Firstly, the tool must be capable of handling many different search requests, and keeping rumour sets organised, i.e. separated in their applicable sets. Retweets are excluded to allow for a diverse set of tweets relating to the rumour in question. Finally, we avoid searching in the same time range more than once, i.e. searching only for tweets older than the last set received, to avoid receiving the same set time and time again, and to gather the most advantageous snapshot, avoiding duplicates.
Rumours were chosen and collected at one point in time between February and April
2016, by the detection approach discussed in the previous section (IV). Each query
represents a known rumour, fitting to our rumour definition, introduced in section II.
We are interested in rumours of the day, rumours that were circulating stories at the
time of search. To avoid rumours and build a snapshot, circulating rumour stories fitting
our rumour definition were found using online resources, similar to the approach
adopted in previous work [
We selected Rumour Targets based on present regular expression queries used for detection and the number of tweet collected in each rumour set. Each set corresponds to a rumorous story chosen from an online source. For example, 'germany pork' relates to the rumour, "Germany bans pork under Sharia law"4. The size of the rumour sets varies greatly. This has added complexity to the research, where the temporal characteristics of rumours, and the decision to collect in a snapshot method disallowing a large 1 http://www.snopes.com/ 2 http://starcasm.net/ 3 http://www.celebuzz.com/ 4 http://www.snopes.com/germany-bans-pork-undersharia-law/ dataset built over time, has affected set sizes. The mean size of rumour sets is noted below, along with the standard deviation. The large standard deviation reflects the wide spread of set population sizes.
The mean size of the rumour sets is 139, and the standard deviation in the size of the rumour sets is 156.
The next step is to collect the metadata required, properties used for impact measurement, and other metadata properties that are investigated as being influential to such measurements. 4.2
Feature Retrieval The Feature Retrieval process takes each Tweet object received from the Search API, funnelled through the Gathering process, and parses the metadata required for impact scoring, namely retweet count and favourite count, as required by our formula, (1), section II, page 2. It also obtains those features required for subsequent impact investigation, the list of other features for analysis as potential influencers to our impact measurement. After preparation of impact measurement properties and impact analysis features, the data are organised together, along with the associated tweet ID and text.
These features are collated, parsing properties of the tweet itself, and are used to determine if composition features inuence the impact of a rumour on social media. • HT: #tweets in the set containing Hashtags • M: #tweets in the set containing Media • URL: #tweets in the set containing URLs • UM: #tweets in the set containing User Mentions • R: #tweets in the set containing the word 'retweet' • AC: #tweets in the set entirely All Caps • ?: #tweets in the set containing a Question Mark • !: #tweets in the set containing an Exclamation
Mark • Q: #tweets in the set containing a Quote • Av .L: Average Length of the tweets in the set
These features correspond to account properties of the tweet's composing author, and are used to determine if characteristics of the author and their account, influence the impact of a rumour on social media.
• Y: Average Creation Year of the accounts associated
with the set • FO: Accumulative #Followers • FR: Accumulative #Friends • S: Accumulative #Statuses • DP: Accumulative #Default Pro_les • DA: Accumulative #Default Avatars • V: #Veri_ed accounts associated with the set
The rumour sets gathered through the Gathering process, section V(A), have allowed the detection and collection of rumorous tweets, supplying a rumorous corpus. The Feature Retrieval process, section V(B), carried out the task of building the data required for impact measurement, and obtained a selection of tweet property data which would allow for further analysis.
It is worth noting the observations that are apparent from this raw data. The largest rumour set collected is germany pork, the set related to the rumour, "Germany bans pork under Sharia law". This set has obtained the highest impact score of 1098. However, the mean impact of the rumour tweets within the set is only 1.84, which is not one of the highest mean impact scores obtained. Therefore, it can be argued that this high impact score is largely depending of the set size, with more tweets lending to a higher accumulative score. Thus, we cannot simply analyse impact scores on their own. The mean impact scores are important, as they supply an indication of the respective impact of each rumour tweet within each set. The mean scores allow us data that is comparable.
As the snapshot method was followed in rumour gathering, all tweets related to a rumour were collected at one time only, i.e. we did not conduct numerous searches over a number of days, for example. A direct result of this is that the timing of a rumour has had a great effect on the amount of tweets that were available for retrieval. To put this in simple terms, if rumour A became topical only today, and rumour B became topical 3 days ago, it is very likely that there will be a lot more rumour tweets to collect for rumour B, if we perform our search today. Interestingly, khloe_parentage is one of the smallest sets of rumours collected, but represents the highest mean impact score, reflective of the high impact of the rumour tweets within the set. 5.1
Statistical Significance in Impact A t-test is a statistical hypothesis test that can be used to determine if the variances of two sets of data are significantly different from each other. Rumour sets that are statistically different from each other can be taken as statistically significant, rejecting the null hypothesis5. Therefore, the means of the rumour populations are not equal, with one representing high impact, in contrast to the other set.
T-tests were conducted on impact scores (Table IV). A random sample, of size 40, was chosen for all tests. The decision regarding 40 as the size of the random sample was made in an attempt to detect the most meaningful difference as possible, with consideration of the varying set (total population) sizes.
By performing t-test analyses, we endeavored to and statistical significance in the data, allowing us to ag rumours that were higher in impact compared to others. Therefore, the null and alternative hypotheses were as follows:
H0: There is no significant difference between specific populations, or no difference among rumour sets, regarding their impact.
H1: There is significant difference between specific populations. The rumour sets are different in terms of impact.
As is to be expected, with data as flimsy as that associated with rumours, and under the limits of our snapshot method, statistical significance was not found in a large proportion of cases. However, the study was successful in obtaining statistical significance in some cases, by t-test analyses, highlighting those rumours that were statistically different in term of impact, granting us the ability to perform further analysis.
Once again related to the changeable nature of rumours, we were cautious in immediately accepting results obtained through t-test analyses. The t-test calculation is reflective of the sample populations it is presented with. Being aware of how flimsy rumour data is, and how varying individual impact scores can be, many t-tests on pairs of
samples were performed, which at first appeared to be statistically different, but for which assurance was needed.
As the variances differed, H0 was rejected for certain rumours, those detailed in table IV, after receiving similar t-value results with many t-test calculations, within the same rumour populations, selecting different random sample sets. Table IV presents examples of those rumours where statistical signifcance was found, with a t-value and pvalue, representative of one of the t-test calculations associated with the pair. Rumour A is of higher impact than Rumour B.
Degrees of Freedom: (sample size * 2) - 2 = (40 * 2) - 2 = 78 Significance Level a: 0.05, the most widely used significance level.
Given a t-value and the degrees of freedom, a p-value is obtained. The p-value is compared to a. A small p-value (0.05) indicates strong evidence against the null hypothesis, so it is rejected. Following our comprehensive study, and with the data we have presented, we conclude that there exists statistical difference between the impact of respective rumours. We must now make attempts to understand why this difference exists, highlighting potential factors, which supply / inuence this difference. 5.2
Features Influential to Rumour Impact on Social Media Recall in section V(B), we gathered Message-Based Features and Account-Based Features, through the feature retrieval process. After observing the raw data collected, it was decided to reduce the number of features that would be taken any further through the process of impact analysis. The key objective to the following analyses is to those features that are influential to rumour impact on social media, i.e. contributing to a higher impact score, obtained through the formula we suggest, formula (1). The features that will not be considered are as follows: Message-Based Features Word 'retweet', All caps, Question mark, Exclamation mark, Quote - Eliminated due to minimal existence in the dataset.
Account-Based Features Year, Default Proble, Default Avatar - Eliminated due to minimal / insuficient existence in the dataset.
The approach taken in feature analysis is to take each feature that will be investigated individually. We present those rumours once again, those incurring statistical difference related to impact, and compare each feature's existence in the higher impact rumour compared to that of lower impact. Those features that are more substantial in higher impact rumours can then be concluded as contributing / in to impact. 5.2.1
Features Influential to Rumour Impact on Social Media We investigate whether account properties of the composing author influence rumour impact on social media. The features that are analysed are followers, friends, and statuses, representing how popular and active the authors are. The inuence of verified accounts is also investigated, those accounts belonging to key individuals that Twitter have signified by placement of the verified badge. These accounts, belonging to politicians, celebrities, journalists tend to have many followers, and attract significant user attention. We investigate whether the number of followers associated with the composing authors involved in a rumour, has an effect on the impact of the rumour. Followers are those people who have connected with a Twitter account. Someone who thinks you're interesting can follow you. Following is not mutual, you don't have to follow back.
Out of the seven experiments where statistical difference was found, highlighting the higher impact of one rumour compared to another, six cases have higher follower counts in the higher impact rumour than for the lower impact rumour, see table V, column 'FO'. Experiment E is the one exception where the lower impact rumour has more followers in the set than the higher impact rumour. Six cases out of seven represents 86%. Under the specific conditions of our experiments, it can be concluded that followers in the impact of a rumour.
We investigate whether the number of friends (followees) associated with the composing authors involved in a rumour, has an effect on the impact of the rumour. Friends are those people you have connected with, the people you follow, who do not necessarily follow you back. A result similar to Followers was obtained. Six cases out of seven, 86%, have higher friends counts in the higher impact rumour than for the lower impact rumour, with Experiment E. Under the specific conditions of our experiments, it can be concluded that friends influence the impact of a rumour.
We investigate whether the total number of statuses (messages / tweets) that the authors have composed in the lifetime of their accounts, has an effect on the impact of the rumour. In all seven experiments where statistical difference was found, highlighting the higher impact of one rumour compared to another, total statuses counts associated with the authors are higher in the higher impact rumour compared to the lower impact rumour. This result represents 100%. Under the specific conditions of our experiments, it can be concluded that the total number of statuses associated with author accounts influences the impact of a rumour.
Findings of our comprehensive study suggest: • The number of followers has a significant in on the impact of a rumour. • The number of friends has a significant influence on the impact of a rumour. • The total statuses related to the author, has a significant influence on the impact of a rumour. • Verified accounts do not significantly influence the impact of a rumour. • hashtags do not significantly influence the impact of a rumour. • media is likely to be influential to rumour impact. • user mentions are likely to be influential to rumour impact. • URLs do not significantly influence the impact of a rumour.
• length 120-130 chars does not significantly in the impact of a rumour. 5.2.2
Impactful Rumours compared to Non-Rumours The final stage of impact evaluation is inspired by a question of the specific nature of rumours, compared to all other messages. Following our investigation and findings regarding influential features lending to the impact of rumours, analyses is now extended as we ask if there is a measurable difference between rumours and non-rumours - not related to the text but related to impact and features.
For choosing non-rumours, news stories from the credible source - BBC News6 were selected. These stories were chosen on the 26th April 2016, and involve factual events, i.e. no question regarding veracity. Table VII, presents non-rumour sets collected - set names, associated news stories, and the number of tweets collected in each non-rumour set.
• •
T-test analysis between sample size (40) messages of impactful rumour and sample size (40) messages of non-rumour.
Where statistical difference is not found, i.e. the rumour and non-rumour represent the same impact, the presence of features investigated previously is compared.
By performing this investigation, we are ultimately asking, "are rumours and nonrumours essentially the same or is there something that can be measured that makes them different".
A summary of findings related to rumours vs non-rumours is presented in the coming discussions. The rumours compared to the five non-rumours listed in table VII, are kim_divorce and obama pay increase, the three higher impact rumours of the data, noted in table VI. This gives a total of fifteen experiments: (3 rumours) * (5 non-rumours).
The five features are presented individually, the five deemed to be influential and likely influential. In each case, we assess whether the feature is more prominent in the rumour sets compared to the non-rumour sets. A feature more prominent in the rumour sets allows something that can be measured specific to rumours and their impact.
Following feature analysis, our study suggested that followers are influential to rumour impact. By comparing the numbers of followers associated with impactful rumours against non-rumours, we now determine whether there are more followers associated with rumours compared with non-rumours. Out of fifteen experiments, ten cases had more followers in the rumour set compared to the non-rumour set. This result represents 67%, see table V, column 'FO'. This result is not quite conclusive but we believe that with more data and more analyses, this percentage is likely to increase, and become more conclusive.
Under the specific conditions of the experiments of our study, it can be concluded that the number of followers is likely to be an acceptable measure for rumour impact, a measure unique to messages deemed rumorous, highlighting the different nature of rumours compared to other messages.
Friends were suggested as being inuential to rumour impact by our study, and we now determine whether there are more friends associated with rumours compared with nonrumours. Out of fifteen experiments, 12 cases had more friends in the rumour set compared to the non-rumour set. This result represents 80%, see table V, column 'FR'. Under the specific conditions of the experiments of our study, it can be concluded that the number of friends is an acceptable measure for rumour impact, a measure unique to messages deemed rumorous, highlighting the different nature of rumours compared to other messages.
Our feature study suggested that the total number of statuses associated with the accounts of the rumour authors, i.e. total number of messages composed in the lifetime of the accounts, is influential to rumour impact. We now ask whether there are more statuses associated with the authors of rumours compared with non-rumours. Out of fifteen experiments, only three cases had more total statuses associated with the authors in the rumour set compared to the non-rumour set. This result represents 20%, see table V, column 'S'. In other words, the authors posting non-rumours, related to credible news, tend to post more often in general, compared to those who post rumorous messages. Under the specific conditions of the experiments of our study, it can be concluded that the total number of statuses associated with the author accounts is a likely measure for the impact of all types of messages and is not unique to rumours.
The inclusion of media items (images) is likely to be iential to rumour impact, according to our feature study. We now investigate whether there are more tweets containing media items associated with rumours compared with non-rumours. Out of _fteen experiments, ten cases had more tweets containing media items in the rumour set compared to the non-rumour set. This result represents 67%, see table V, column 'M' This result is not quite conclusive but it is believed that with more data and more analyses, this percentage is likely to increase, and become more conclusive. Under the specific conditions of the experiments of this study, it can be concluded that the inclusion of media items is likely to be an acceptable measure for rumour impact, a measure unique to messages deemed rumorous, highlighting the different nature of rumours compared to other messages.
Our feature study suggested that the inclusion of user mentions (tagging another user, @user) is likely to be intential to rumour impact. User mentions are the last feature we analyse in consideration of non-rumours. We investigate whether there are more tweets containing user mentions associated with rumours compared with non-rumours. Out of fifteen experiments, eight cases had more tweets containing user mentions in the rumour set compared to the non-rumour set. This result represents 53%, see table VI, column 'UM'. In other words, the authors posting non-rumours, related to credible news, tend to include user mentions as commonly (slightly more) as authors posting rumorous messages.
Under the specific conditions of the experiments of this study, it can be concluded that the inclusion of user mentions is a likely measure for the impact of all types of messages and is not unique to rumours. 6
We have collected a dataset of rumour messages, associated with known rumours sourced online. This dataset represents a rumorous corpus su_cient for experimental analyses within rumour behaviour, such as those associated with rumour impact. Feature data associated with every rumour message in the dataset has been collected and stored with the rumorous text, enriching the resultant dataset of rumours. A formula has been suggested for calculating the impact of rumours on social media itself. The formula presented by the work appears Twitter specific in its variables; retweets and favourites. However, this may be customized / altered / extended, to reflect user engagements or other application specific properties on other social media.
The significant findings of our study are, a) statistical significance has been highlighted in our rumour data, whereby we have found statistical difference between some rumours, in terms of social media impact, b) important findings related to rumour author features and rumour composition features that we suggest are inuential (and not in to the impact of rumours on social media, c) behavioural differences in rumours compared to all other messages, properties impactful to rumours specifcally, suggesting that rumours act differently to other messages, and that there exists measurable differences in terms of influential features unique to rumour impact.
In the context of the greater research area, the work bridges the void that exists in the State of the Art, and provides study, measurement, and analyses of rumour impact on social media itself, a gap created as rumours and new environments to thrive, resulting from the success and growth of social media. Our study acts as an encouraging step towards building a customisable model for measuring impact, that can be applied over the various social media platforms that exist. This work introduces an exciting opportunity for extensive research in rumour impact on social media itself, where research should be continued, possibly according to some of the suggestions to follow.