Reddit is one of the few social platforms that handles NSFW (Not Safe For Work) content in an explicit and well-structured way. Despite this fact, such an issue has been very neglected in the past by researchers who have studied this social network. In this paper, we aim at providing a contribution in this setting by proposing an approach to extract and analyze text patterns from NSFW content in Reddit. An important peculiarity of our approach is that patterns are extracted not only based on their frequency (as it generally happens in the past literature), but also, and especially, on one or more utility measures.
Data Cleaning and Annotation
Pattern Extraction and
Enrichment Network-based Pattern Analysis Removing bot's content, cleaning text Sentiment annotation Lexical annotation Frequent pattern mining Filter patterns by utility Enrich patterns with related data information Building of pattern-based networks and user-based networks Network analysis and (iii) Network-based Pattern Analysis. Applying our approach on Reddit allowed us to make several contributions to this research scenario. They involve: (i) discovering that traditional approaches to sentiment computation are unreliable in the context of NSFW adult content; (ii) defining and finding opinion leaders in real communities sharing NSFW adult content; (iii) discovering text patterns representing the building blocks of NSFW posts and comments on Reddit; (iv) determining new virtual communities of users sharing NSFW adult content; (v) identifying opinion leaders who could influence such communities.
The rest of this paper is structured as follows: Section 2 provides a general description of our approach and the dataset used for our experiments. Section 3 illustrates the Pattern Extraction and Enrichment step. Section 4 describes the Network-based Pattern Analysis step. Finally, Section 5 presents our conclusions and possible future developments of our research eforts.
The general workflow of our approach is shown in Figure 1, which highlights the three steps composing it (i.e., Data Cleaning and Annotation, Pattern Extraction and Enrichment, and Network-based Pattern Analysis).
The Data Cleaning and Annotation step removes irrelevant content and standardizes text
representations. It also performs lexical (e.g., part-of-speech and named entities) and sentiment
annotations. These latter highlight the polarity of sentiments expressed in the texts, represented
in terms of a compound score, computed by applying Vader [
The Pattern Extraction and Enrichment step extracts a set of text patterns from the posts and comments identified in the previous step; they are the basis for the next Network-based Pattern Analysis step. To this end, it first extracts frequent patterns. Then, it associates each pattern with a rich set of features regarding the posts and comments it derives from, as well as the users who published them. Afterwards, it defines some utility measures and associates the corresponding values with each pattern. Finally, it selects only those patterns with high frequency and high utility. Our approach allows the definition of diferent concepts and utility measures and, consequently, the selection of diferent sets of useful patterns based on them. This allows us to analyze the available NSFW content from very diferent perspectives, yet adopting a uniform methodology.
The Network-based Pattern Analysis step applies the concepts and approaches of Social Network Analysis to the patterns obtained during the previous step with the goal of extracting information and knowledge from them. Specifically, it constructs and uses three social networks, namely: (i) User Interaction Network, in which a node represents a user , who published at least one post or comment. An arc (, , ) denotes that commented a post of ; indicates how many times this happened. (ii) Pattern Network, in which a node represents a pattern extracted in the previous step. An arc (, , ) indicates that and were adopted by at least one user in common; indicates the number of users who adopted both and . (iii) User Content Network, in which a node represents a user , who published at least one post or comment. An arc (, , ) indicates that there is at least one comment posted by and at least one comment posted by containing the same pattern; denotes the number of times this happened. Once these networks are built, this step proceeds by applying Social Network Analysis concepts and approaches to them for extracting information and knowledge on Reddit users publishing, commenting and reading NSFW adult posts and on the content these users exchange. Due to space limitations, in this paper, we focus only on the User Interaction Network.
To perform our experiments for evaluating our approach, we downloaded a dataset from
the pushshift.io [
This step extracts text patterns from posts and comments in the dataset and, then, enriches them with additional information concerning their frequency and utility. Pattern mining plays a key role in this activity. It is a well known task in the literature, which extracts from a dataset some (hopefully interesting and/or unexpected) information that can be understood by humans.
Many pattern mining approaches are based on the concept of pattern frequency and aim at
identifying the most frequent patterns in the texts received as input. They are based on the
assumption that frequent patterns are interesting [
After having introduced the concepts of frequency and utility of a pattern, we can illustrate our approach to pattern extraction and the model which it operates on. Let = {1, 2, · · · , } be a set of lemmatized comments, obtained at the end of the Data Cleaning and Annotation step. Each comment ∈ corresponds to a post and is written by a Reddit user. We can represent as a set of lemmas = {1, 2, · · · , }. Therefore, if we denote by ℒ = {1, 2, · · · , } the set of all possible lemmas, then is a subset of ℒ. From the HUPM perspective, each lemma is an item. A pattern is a set of items and, therefore, ⊆ ℒ . can occur in zero, one or more comments in our dataset. We denote by ⊆ the subset of the comments of in which is present, and by frequency of the cardinality of . inherits the set of features characterizing the comments of , and the utility of can be defined as an appropriate function of all or some of these features. The choice of the features and the utility function to adopt determine the perspective one whishes to consider in the analysis of patterns.
For example, consider the features score_comm (denoting the score of a comment) and
compound (indicating the sentiment value extracted from the text of a comment). Suppose that
the utility function is the Pearson’s correlation [
Figure 2 shows the trend of the number of extracted patterns as ℎ− (resp., ℎ+) decreases
(resp., increases). Patterns are also grouped based on their length. This figure provides us with
non-obvious and extremely interesting knowledge. In fact, the number of patterns extracted by
− is much greater than the one extracted by +. This allows us to say that, given a pattern, a
positive (resp., negative) sentiment of it is not necessarily accompanied by a high (resp., low)
score of the comments where it is present. This phenomenon is very evident for moderately
positive or negative values of , while it reduces strongly for extreme values. It can be explained
by considering that, given the nature of the reported texts, NSFW posts and comments tend
to be associated with a negative sentiment by any sentiment analysis tool. This happens even
when such terms are used in goliardic comments, which are actually appreciated by this type
of audience. For instance, consider the text pattern {ℎ, }, possibly accompanied by an
emoticon with two little hearts instead of eyes. Applying the sentiment analysis tools available
in the literature to our dataset [
We end this discussion by pointing out that many other utility functions could be defined on many diferent features concerning posts and users. This important peculiarity of our approach allows us to analyze the phenomenon of NSFW content from many diferent perspectives.
In this section, we formally introduce the User Interaction Network and show how information can be derived from it. Let be the set of patterns obtained by applying the utility function , and let be the set of users who have written at least one comment or post containing at least one of the patterns of . A User Interaction Network can be defined as: = ⟨ , ⟩. is the set of nodes of . There exists a node ∈ for each user ∈ . is the set of arcs of . An arc (, , ) ∈ denotes that made comments on a post published by ; measures how many times this task happened.
allows us to characterize the behavior of users, who interact with each other by posting and commenting NSFW adult content.
Table 1 lists some parameters of the User Interaction Networks built exploiting the dataset outlined in Section 2, the utility functions + and − , and the thresholds ℎ+ and ℎ− . This contains a larger number table confirms the information derived in Section 3. For instance, − of nodes and arcs than + . Note that the cardinalities of − and + follow the same trend as the number of nodes (and users) in the User Interaction Networks, even though they represent patterns instead of users. It is worth observing the high density coupled with a high clustering coeficient characterizing + . This tells us that, in this network, users tend to form very tight has a high density but communities, whose structure resemble that of a clique. Instead, − a low clustering coeficient; this suggests the presence of very strong power users, i.e., users receiving comments from many other ones, who do not actually interact with each other. In both networks, the number of nodes composing the maximum connected component is very high, i.e., about 60%. parameters and of these power law distributions; they are: = 1.371, = 0.062 for − and = 1.507, = 0.063 for + . The results of these analyses show that there is a low number of pairs of users in which one of them comments a post of the other at least twice (we call them interacting users in the following). This can be considered a minimum condition to detect non-random relationships between pairs of users. We compared the values of the average indegree, outdegree and clustering coeficient of the interacting users, on one hand, and all the users, on the other hand. The bottom of Table 1 shows this comparison for the two considered networks. It is easy to observe that, in both networks, interacting users have indegrees and outdegrees much higher than the other users. Therefore, they can be considered power users. Also, their clustering coeficient is very high, indicating that they are able to promote the generation of communities. Therefore, it can be said that they are community leaders in the distribution of NSFW adult content in Reddit.
At this point, we found it interesting to investigate the possible existence of mutual relationship between interacting users. For this purpose, we determined the fraction of interacting users such that a user comments the posts of a user , and vice versa. We found that this fraction , whereas it is higher (i.e., 0.433) for + . Furthermore, although is low (i.e., 0.141) for − the number of nodes of + is considerably lower than that of − , the two networks have similar average indegree and outdegree for both normal and interacting users. Moreover, + has a much higher clustering coeficient and a much higher fraction of interacting users than . Based on all these outcomes, we can state that, although the number of users of − is − much greater than that of + , these last ones have a higher attitude to be opinion leaders. Taking also the utility function underlying + into account, we can deduce that the users of this network are particularly able to maintain a positive correlation between the sentiment of their comments and the associated scores. Finally, the results obtained so far allow us to state that the users of + are the most dynamic ones, as they publish posts attracting interest (since they are commented by other users), and comment the posts of the other users. This feature makes them particularly important, as they are not only content producers, but also dynamic participants who contribute to maintain their communities active, and act as opinion leaders for these communities. We call them proactive users.
This paper presented an approach to analyze NSFW users, comments and posts on Reddit, taking into account and exploiting the knowledge extracted by investigating text patterns. The methods and results considered in this paper can represent the basis for many new applications. In fact, posts and subreddits of other target categories (e.g., vegetarian or vegan users) could be examined by means of the same methodology. Moreover, the analysis of text patterns could represent the engine of an automatic classifier aiming at tagging posts and comments containing unsuitable content. Furthermore, our approach can be adapted to other social networks managing NSFW content less explicitly than Reddit. Finally, we plan to design an automatic tool that exploits a knowledge base built by integrating utility patterns and semantic analysis tools to automatically classify new contents, and, then, suggest the most pertinent communities which they should be directed to.