The many-to-many social communication activity on the popular technology-news website Slashdot has been studied. We have concentrated on the dynamics of message production without considering semantic relations and have found regular temporal patterns in the reaction time of the community to a news-post as well as in single user behavior. The statistics of these activities follow log-normal distributions. Daily and weekly oscillatory cycles, which cause slight variations of this simple behavior, are identified. The findings are remarkable since the distribution of the number of comments per users, which is also analyzed, indicates a great amount of heterogeneity in the community. The reader may find surprising that only two parameters, those of the log-normal law, allow a detailed description, or even prediction, of social many-to-many information exchange in this kind of popular public spaces.
Nowadays, an important part of human activity leaves electronic traces in form of server logs,
e-mails, loan registers, credit card transactions, blogs, etc. This huge amount of generated data
allows to observe human behavior and communication patterns at nearly no cost on a scale and
dimension which would have been impossible some decades ago. A considerable number of
studies have emerged in recent years using some part of these data to investigate the time
patterns of human activity. The studied temporal events are rather diverse and reach from directory
listings and file transfers (FTP requests)
To the authors’ knowledge no study of this type has been performed on systems where
social interaction occurs in a more complex manner than just person to person (one-to-one)
communication. We think it is valuable to analyze the temporal patterns of the many-to-many
social interaction on a technology-related news-website which supports user participation. We
have chosen Slashdot1, a popular website for people interested in reading and discussing about
technology and its ramifications. It gave name to the “Slashdot effect”
Slashdot was created at the end of 1997 and has ever since metamorphosed into a website that hosts a large interactive community capable of influencing public perceptions and awareness on the topics addressed. Its role can be metaphorically compared to that of commercial malls in developed markets, or hubs in intricate large networks. The site’s interaction consists of shortstory posts that often carry fresh news and links to sources of information with more details. These posts incite many readers to comment on them and provoke discussions that may trail for hours or even days. Most of the commentators register and comment under their nicknames, although a considerable amount participates anonymously.
Although Slashdot allows users to express their opinion freely, moderation and
meta-moderation mechanisms are employed to judge comments and enable readers to filter them by quality.
The moderation system was analyzed by
Given that a great amount of users with different interests and motivations participate in the discussions, one would expect to observe a high degree of heterogeneity on a site like Slashdot. However, what if the posts and comments were analyzed just as imprints of an occurring information exchange, with no regard to semantic aspects? Is there a homogeneous behavior pattern underlying heterogeneity? To answer these and related questions we collected and studied one year’s worth of interchanged messages along with the associated metadata from Slashdot. We show here that the temporal patterns of the comments provoked by a post are very similar, indicating that homogeneity is the rule not the exception. The temporal patterns of the social activity fit accurately log-normal distributions, thus giving empirical evidence of our hypothesis and establishing a link with previous studies where social interaction occurs in a simpler way.
Finally, our analysis allows more insight into questions such as: is there a time-scale common to all discussions, or are they scale-free? What does incite a user to write a comment, is it the relevance of the topic, or maybe just the hour of the day? Can we predict the amount of activity triggered by a post already some minutes after it has been written? Which type of applications can we devise on the basis of using these conclusions?
The rest of the article is organized as follows: In section 2 we briefly explain the process of data acquisition. We then present the results in section 3 providing first an overview of the global activity and then explaining our analysis in detail. We finish the paper with section 4 where we discuss the results. 2.
In this section we explain the methods used to crawl and analyze Slashdot. The crawled2 data correspond to posts and comments published between August 26th, 2005 and August 31th, 2006. We divided the crawling process into two stages. The first stage included crawling the main HTML (posts) and first level comments and the second stage covered all additional comment pages. Crawling all the data took 4.5 days and produced approximately 4.54 GB of data. Post-processing caused by the presence of duplicated comments was necessary (due to an error of representation on the website). Although a high amount of information was extracted from the raw HTML (sub-domains, title, topics, hierarchical relations between comments) we concentrated only on a minimal amount of information: type of contribution (either post or comment), its identifier, author’s identifier and time-stamp or date of publishing. The selected information was extracted to XML-files and imported into Matlab where the statistical analysis was performed. Table 1 shows the main quantities of the crawling and the extracted data. 2Software used: wget, Perl scripts, and Tidy on a GNU/Linux, Ubuntu 6.0.6 OS. ing two quantities:
The Post-Comment-Interval (PCI) stands for the difference between the time-stamps of a comment and its corresponding post.
The Inter-Comment-Interval (ICI) refers to the difference between the time-stamps of two consecutive comments of the same user (no matter what post he/she comments on). 3.
In this section we first give an overview of the global activity looking at the data on different temporal scales and analyzing some relations between variables of interest. We then focus on the activity provoked by single posts and analyze the behavior of single users, concentrating on the most active ones. 3.1
As previously explained, comments can be considered as reactions triggered by the publishing of posts. This difference in nature between both types of contributions justifies a separate analysis of their dynamics.
Figure 1 shows (normalized) mean activity and standard deviations of both posts and comments. It illustrates patterns in agreement with the social activity outside the public sphere. Figure 1a shows regular, steady activity during working days which slows down during weekends. This weekly cycle is interleaved by daily oscillations illustrated in Figure 1b. The daily activity cycle reaches its maximum at 1pm approximately and its minimum during the night between 3am and 4am. Although Slashdot is open to public access around the world, we see that its activity profile is clearly biased towards the American time-schedule.
Interestingly, although post activity shows more fluctuations and higher standard deviations than comment activity, there is little discrepancy between their mean temporal profiles. This difference in the deviations is not surprising given the greater number of comments (see Table 1). We notice that the standard deviations of the daily post- and commenting activities also show similar cyclic behavior (Figure 1b). 3.2
In this section we analyze the activity (comments) a post induces on the site. The histogram of Figure 2 gives an idea of the number of comments the posts receive. Note that half of the posts provoke more than 160 comments and some of them even trigger more than 1000. To analyze the time-distribution of these comments we study their post-comment intervals (PCIs). 3.2.1
We are especially interested in the resulting probability distribution of all the PCIs of a certain post. This distribution reveals us the probability for a post to receive a comment t minutes after it has been published. Figures 3a and 3b show this distribution for a post which provoked 1341 comments. Although there are some important fluctuations, the characteristic shape of the probability density function (pdf) resembles a log-normal distribution. This becomes even clearer if the cumulative probability distribution (cdf) is observed, since there the fluctuations of the pdf are averaged out. Figures 3c and 3d show a good fit of the PCI-cdf of the data with the cdf of the log-normal distribution.
To classify the quality of the fit we have used a normalized error measure ε based on the `1-norm (see Appendix A). For the post shown in Figure 3 we obtain ε = 0.007, meaning that the average error is below 1%.
The PCI-cdf of three more posts can be observed in Figure 4. The top two sub-figures show good fits, indicating that the PCI is well approximated even for a small number of comments. However, the fit is not that accurate for all posts. For example, the comments of the post shown in Figure 4 (bottom) start to show considerable different behavior from the expected log-normal approximation about 3 hours after its publication. The activity is lower than the predicted one, but starts to increase again at about 6am in the morning the next day. At around at 8:30pm it increases further to recover the lost activity during the night. More such increases and decreases of activity can be observed during the following days. The time-spans of variations in activity coincide quite exactly with the average daily activity cycle shown in Figure 1b. We analyze this coincidence further in the next section. With the log-normal shape of the PCI-distribution identified, we focus on the quality of this approximation in general. We therefore calculate the error measure ε of the fit for all posts which received comments. The resulting distribution of ε can be seen in Figure 5a. For 87% of the posts the approximation error ε is lower than 0.05, and for 29% lower than 0.02.
If we take a closer look at the data, we notice a dependence of ε on the publishing-hour of a post (Figure 5b). The best fit is reached when the post is published between 6am and 11am. Then the mean error increases successively until 11pm to stay high during the night and recover again in the early morning.
This behavior can be understood looking at the daily activity cycle (Figure 1b). The less time the community has to comment on a post during the time-window of high activity, the greater is the need to comment on it the next time the high activity phase is reached, and hence the expected log-normal behavior is altered. Figure 4 (bottom) gives examples of such a late post (published at 10:35pm).
The good quality of the approximation allows us to describe the activity triggered by a post
with only two parameters, the median3 and the geometric standard deviation σg of the PCI-pdf,
commonly used to compare log-normally distributed quantities
In this section we analyze the activity on Slashdot taking the authorship of the comments into account. We first study the distribution of activity among all the users participating in the debates and then focus on the temporal activity patterns of single users.
3Note that the median coincides with the geometric mean for a log-normally distributed random variable.
The activity of all users is best illustrated by the distribution of the number of comments per user.
It is shown in double-logarithmic scale in Figure 7a. The obtained distribution follows quite
closely a straight line, suggesting a power-law probability distribution governing this relation.
We note that 53% of the users write 3 or less comments whereas only 93 users (0.1%) write more
than 1000 comments. Indeed, after applying linear regression as in other studies
However, if we apply rigorous statistical analysis as proposed in
As an alternative hypothesis to describe the data we propose a (truncated) log-normal probability distribution, shown in Figure 7 as grey-solid-line. Its parameters are found using the MLE. Clearly, the fit is better using this hypothesis. We remark that in many studies some data points (considered outliers) are discarded to improve the power-law fit. Here, in contrast, the truncated log-normal approximation can characterize the entire data-set. 3.3.2
After characterizing the user activity at a general level, we investigate the temporal behavior patterns of single users . The analysis concentrates on the two most active users (to protect their privacy we call them user1 and user2). Table 2 shows the number of commented posts and the total number of comments these two users published during the time-span covered by our data. We focus on the distribution of the PCIs of all of their comments as well as on their intercomment-interval (ICI) distribution, i.e. the time-difference between two comments of the same user.
The PCI-cdf (see Figure 8a) of the two users can also be approximated by a log-normal distribution, although the fit is worse than in the case of the post-induced comment activity. Again we notice a clear dependence of the quality of the fit on the activity cycle (shown in the insets of Figure 8a). The approximation is much better for user1, whose daily and especially weekly activity cycles are much more balanced than those of user2. The activity of the latter user concentrates almost exclusively on the working hours from Monday to Friday. Hence his PCI-distribution shows a clear decrease after 8 but increases again after 16 hours. This increase is less pronounced if only the first comment to a post is considered (data not shown), indicating that the user frequently rechecks the posts he commented the day before to participate again in an ongoing discussion.
The same effect can be observed in their ICIs, which are illustrated in Figure 8b. There
the cdf (inset of Figure 8b) of user1 shows an even more pronounced increase around an ICI
of 16 hours. We further observe that the ICI-pdf peaks for both users as well as for the whole
population at 3 minutes. This is probably caused by an anti-troll filter
The special architecture of the technology-related news website Slashdot allowed us to analyze the temporal communication patterns of an online society without considering semantic aspects. The site activity is driven by news-posts which provoke communication activity in the form of comments.
Despite the great amount of users participating in the discussions, close to 105 in the data we have studied, and the diversity of themes (games, politics, science, books,etc.) some simple patterns can be identified, which repeat themselves over and over again. One of these patterns appears in the shape of the distribution of time differences between a post and its comments (the PCIs). It can be well approximated by a log-normal distribution (Figures 3 and 4) for most of the posts. The only remarkable deviations from these approximations are caused by oscillatory daily and weekly activity patterns (Figure 1), which become less noticeable if a post is published early in the morning (Figure 5a).
In single user behavior an akin pattern appears in the PCI-distribution of all of the comments a user writes to several posts (Figure 8a). Again deviations are caused by the circadian cycle. Another interesting pattern can be observed analyzing the ICI of single-users, i.e. the time-span between two consecutive comments of a certain user. In the case of the two most active users (Figure 8b) the ICI-distributions are very similar, which further supports our hypothesis of the existence of homogeneous temporal patterns on Slashdot.
We would expect that the time-spans between publishing and reading of a post also follow a
log-normal pattern. This could be easily verified checking the server logs of Slashdot or
accesstimes of an external homepage linked by a Slashdot post. Such a study has been performed to
show the Slashdot effect
Log-normal temporal patterns similar to those described above were found in
person-toperson communication by
Our results indicate that communication activity on Slashdot can be described using only two parameters, i.e. the median and the geometric standard deviation (Figure 6). The medians are very low compared to the overall duration of the activity provoked by a post. Although the posts might be available for commenting during more than 10 days, the first few hours decide whether they will become highly debated or just receive some sporadic comments. We would therefore expect that the simplicity of the approximation together with the high initial activity should make an accurate prediction of the expected user behavior feasible at an early phase after a post has been put online. The accuracy of such forecasting is subject of current research and will be published elsewhere.
An early characterization of the activity triggered by a post could be applied, for instance, on
dynamic pricing or placing of online advertisements or on the improvement of online marketing.
The success of a campaign might be predicted already after a short time-period, thus allowing
an early adaptation of the strategy of information diffusion. In this context the viral marketing
concept
In our opinion, the regular communication activity patterns described in this work may be relevant in two aspects. The first, simpler one, is related to applications where a better understanding of information trade in the web translates easily into a better description, and even quantification, of Internet audience. But a second, more complex, aspect is related to the human “communicative” behavior uncovered at present time: Internet based communication capabilities. We face a new, large scale, all-to-all public space in which a novel kind of social behavior arises, a scenario that we do not yet fully understand. However, we should not forget that the new activity is being largely recorded and the data can be available for research. The work presented in this contribution is a good example of how those data can be collected and analyzed to give, at least, a quantitative description of the behavior. This is a first step towards a more ambitious target: to develop “ab initio” models for the population dynamics of message interchange, which is also the goal of our current research.
This work has been partially funded by Càtedra Telefónica de Producció Multimédia de la Universitat Pompeu Fabra.
We use the following distance measure to calculate the error of log-normal approximation of the data. The distance between approximation and data is only calculated for the time-bins (i.e. minutes) where a post actually receives a comment to avoid a distortion of the error measure by the periods with low comment activity.
Definition 1. Let T be the set of time-bins where a post receives at least one comment and T its cardinality. We define then the approximation error ε of a function f (t) approximating g(t) (both defined for all t ∈ T) as the normalized `1-norm of f (t) − g(t): ε = ∑ | f (t) − g(t)|
T (1)
If f (t) and g(t) are cumulative probability density functions (i.e. 0 ≤ f (t) ≤ 1 and 0 ≤ g(t) ≤ 1), it follows that 0 ≤ ε ≤ 1.