Synthetic datasets used in benchmarking need to mimic all characteristics of real-world datasets, in order to provide realistic benchmarking results. Synthetic RDF datasets usually show a signi cant discrepancy in the level of structuredness compared to real-world RDF datasets. This structural di erence is important as it directly a ects storage, indexing and querying. In this paper, we show that the synthetic RDF dataset used in the Social Network Benchmark is characterized with high-structuredness and therefore introduce modi cations to the data generator so that it produces an RDF dataset with a real-world structuredness.
Linked Data and RDF benchmarking require the use of real-world or synthetic
RDF datasets [
In order to create the basis for more realistic RDF benchmarking, we modify the existing RDF data generator for the Social Network Benchmark so that the resulting synthetic dataset follows the structuredness observed in real-world RDF datasets. Additionally, we implement the coherence measurement for RDF datasets as a Virtuoso procedure, to simplify the process of measuring the structuredness of any given RDF dataset by using the RDF graph stored in the quad store, instead of using RDF les.
* The presented work was funded by the H2020 project HOBBIT (#688227).
2.1
The Social Network Benchmark (SNB) [
Attribute correlations. Real data is correlated by nature. For instance, given names to persons are correlated with their country. Similarly, the textual content of a message is correlated with the interests of its creator. For the purpose of benchmarking and performance evaluation, reproducing these correlations is essential since their understanding can be used to generate optimal execution plans or to properly lay out the data in memory. Datagen uses dictionaries extracted from DBpedia to create entities with correlated attribute values.
Degree distributions. The underlying graph structure of OSNs exhibits
skewed degree distributions: most of people have between 20 to 200 friends,
while a few of them have over 5000 friends [
Structure-Attribute correlations. The homophily principle states that in
a real social network, similar people have a larger probability to be connected,
which leads to the formation of many transitive relations between connected
people with similar characteristics. As a consequence, even though it is commonly
accepted that graph data access patterns are usually random, in practice there is
some degree of locality that can be exploited. For instance, people from a given
country are more likely to be connected among them than to people from other
countries. Thus, this information can be used to lay out data in memory wisely
to improve graph traversal's performance, for instance, by putting all people in a
country closer in memory. Datagen generates person relationships based on
different correlation dimensions (i.e. the interests of a person, the place that person
studied, etc.). These correlation dimensions are used to sort persons in such a
way that those that are more similar, are placed close. Then, edges are created
between close by persons, with a probability that decreases geometrically with
their distance. As shown in [
Spiky activity volume. In a real social network, the amount of activity is not uniform but reactive to real-world events. If a natural disaster occurs, we 1 https://github:com/ldbc/ldbc snb datagen will observe people talking about it mostly after the time of the disaster, and the associated activity volume will decay as the hours pass. This translates to an spiky volume activity along the like of the social network, mixing moments with a high load with situations where the load level is small. Also, this means that the amount of messages produced by people and their topics are correlated with given points in time, which complicates the work of a query optimizer when estimating cardinalities for their query plans. This can also make a system unable to cope with the load if it has not been properly overprovisioned to handle these spiky situations. Instead of generating posts and comments uniformly distributed along time, Datagen creates virtual events of di erent degrees of relevance, which are used to drive the generation of the user activity, producing a spiky distribution. 2.2
The comparison of data generated with existing RDF benchmarks (TPC-H,
BSBM, LUBM, SP2Bench, etc.) and data found in widely used real RDF datasets
(DBpedia, UniProt, etc.) shows that these two have signi cant structural di
erences [
CH(T ; D) = X W T (CV (T; D)) CV (T; D)
T 2T
This is the weighted sum of the coverage CV (T; D) of individual types T 2 T ,
where the weight coe cient W T (CV (T; D)) depends on the number of
properties for a type T , the number of entities in dataset D of type T , and their share
in the totality of the dataset D among the other types. Its rationale is to give
higher impact to types with more instances and properties. CV (T; D) represents
the coverage of type T on the dataset D. It depends on whether the instances
of the type T set a value for all its properties. If that is the case for all the
instances, the coverage will be 1 (perfect structuredness), otherwise it will take
a value from [0; 1). The conclusion of [
In order to make an RDF benchmark more realistic, the dataset should follow the nature of real data. Since the SNB dataset is developed to test not only RDF stores, but also graph database systems, graph programming frameworks, relational and noSQL database systems, we wanted to measure how this dataset is suitable for RDF benchmarks. 4
The authors of [
Here, we propose a new approach to compute coherence of any dataset,
using an e cient RDF store, such as Virtuoso [
and P = iri_to_id('rdf:type') and O = iri_to_id(t) ; return cast (a as real) / (b * c);
Using the original SNB Datagen, we prepared the datasets whose sizes and coherence metrics are presented in the Section 5. Since their coherence varies from 0.86 to 0.89, we can conclude that these datasets are much more structured than the real-world RDF datasets, thus they are not suitable for benchmarking RDF stores. Our intention is to make them mimic real-world Linked Data datasets, with a structuredness level of around 0.6.
The original SNB Datagen (Section 2.1) reproduces the important structural
characteristics observed in real online social networks. However, the authors of
[
The authors of [
The weight of a type mostly depends on the number of its instances in the dataset. In the SNB dataset the comments are the most numerous, followed by posts, which is visible in the results shown in Table 1 where we can see their dominance in this regard: they hold 96% of the dataset weight. In order to decrease the coherence metric of the dataset, CH(T ; D), we should decrease the coverage CV (T; D) of each type T from the dataset. But, if we, for example, decrease the coverage of type Person from 0.95 to 0 (which is not realistic), that will result in a drop of the coherence measure for less than 0.3%. Bearing in mind that we have to decrease it much more than that, the only reasonable choice for modi cations are the Comment and Post types.
The mutual predicates of these two types are browserUsed, content,
creationDate, hasCreator, hasTag, id, isLocatedIn, length and locationIP, while Comment
instances additionally have the replyOf predicate, and Post instances can have
language and imageFile properties if the Post instance is a photo. One way of
decreasing the coverage of speci c types is the removal of a high number of triplets
related to a speci c property. But, taking into account the speci c domain, we
conclude that the only property that can be removed in part of the posts and
comments is isLocatedIn. The initial purpose of this property was to specify a
country from which the message was issued, and it was determined by the IP
address of the location where the message had been created. However, since a
lot of users access social networks using their smartphones equipped with GPS
receivers, social networks o er the possibility of adding locations to the
messages. If we consider this property in that manner, we can remove the location
from some messages, as not all messages contain location information. Various
research in the domain show that users rarely share their location in the posts:
the authors of [
Since it does not make sense to remove any other property, in order to achieve
our goal, we decided to introduce new ones. In the initial dataset, all of the
comments are textual, while recently social networks added a prede ned set of
GIFs which can be used as comments [
In the next step, we add one more property to posts and comments: mentions.
Its purpose is to mention a person in a post or a comment. This modi cation
is also in line with what we have on social networks such as Facebook, Twitter,
Instagram, etc., where a user can mention another user in a post or a comment,
usually to make the other person aware of it. An analysis we performed over the
Twitter7 dataset [
A signi cant issue in operating a social network is privacy. Facebook
introduced the possibility for each author of a post/comment to determine its level of
privacy: if you want to share it publicly, to your friends only, or to speci c group
of people [
The nal change we added to the data generator is the addition of the link
property, which both textual posts and comment can have. This corresponds to
the real-world activity of sharing a link in a post, in addition to the text. Based
on the analysis of user behavior on social media [
The new SNB Datagen which generates RDF datasets with real-world coherence is publicly available on GitHub3. 5
To assess the structuredness of the RDF datasets generated by the original data generator and our modi ed version of it, we made measurements of the datasets in di erent sizes: 1, 3, 10, 30, and 100GB. Table 2 and Table 3 depict the results of the measurements: they show the number of triplets (in millions), and the coherence metric for all versions of the datasets. The tables provide a good overview of the dependence of the structuredness measure on the dataset size and the number of instances, in both the original and the modi ed dataset. In this paper, we introduced modi cation to the SNB data generator, to lower the generated RDF dataset coherence to a value of around 0.6, which corresponds better with real RDF datasets. We removed the location value in most posts and comments, and introduced new properties in the dataset: a GIF-type comment, user mentions and links in posts/comments, as well a level of visibility of a post. We used general characteristics of real social networks, such as Twitter, Facebook and Instagram, to generate a dataset which mimics real social networks. With all changes combined, we manage to get an RDF dataset for the SNB with the desired structuredness, i.e. coherence value. Additionally, we introduce a set of Virtuoso procedures which can be used for calculating the dataset coherence of
any RDF dataset stored in the quad store. With this, we simplify the process of
coherence calculation introduced by the authors of [
As future work, we plan to reduce the coverage of other types besides posts and comments. We will also address the correlations in the newly added parts of the dataset. This will not change the overall structuredness, but the dataset will further correspond to real-world RDF data. The changes and additions introduced in the dataset will be implemented in the corresponding SNB queries.