Personal data is being centralized at an unprecedented scale, and this comes with widely known and far-reaching consequences, considering the recent data scandals with companies such as Equifax and Facebook. Decentralizing personal data storage allows people to take back control of their data, and Semantic Web technologies can facilitate data integration at runtime. However, such data processing over decentralized data requires far more expensive algorithms, while at the same time, less processing power is available in individual stores compared to large-scale data centers. This article presents a vision in which nodes in decentralized networks are incentivized to collaborate on data processing using a distributed ledger. By leveraging the collective processing capacity of all nodes, we can provide a sustainable alternative to the current generation of centralized solutions, and thereby put people back in control without compromising on functionality.
agenda pictures my personal data storage contacts photo gallery social feed meeting scheduler applications photo editor document editor
Ben’s storage
Alice’s storage Carol’s storage
Such a wide cross-application interoperability without strong prior agreements can
be achieved by encoding semantics along with data and queries, as is possible with
Semantic Web technologies like rdf and sparql. Data can be represented through
a choice of widely used and custom ontologies. Every person is free to pick their
ontologies and, because of semantics, reasoning can bridge ontological differences.
In other words, the decentralized aspects of Linked Data and the uncoordinated nature
of rdfs and owl ontologies are a good fit for such scenarios [
Compared to centralized systems, decentralized systems are facing a double disadvantage: individual nodes are not only solving a harder problem, they are doing so with far fewer resources. On the one hand, algorithms for decentralized data processing require significantly more processing power and network bandwidth than their centralized counterparts, because of heterogeneity and distribution. On the other hand, each individual node in the network—be it a data store or a client running an application—possesses far less computational power and bandwidth than large centralized data centers.
Furthermore, many of our data processing algorithms are not prepared for the scale of decentralization entailed by full data ownership. As a simple but realistic example, building the social media feed of a person with 500 friends requires executing a query over 500 different data sources in the worst case, where each of those friends store their data at a different location. State-of-the-art federated sparql query engines consider use cases of a dozen of large datasets with entirely different data shapes. In contrast, decentralized data storage will require federated queries over hundreds of small datasets with highly similar shapes. Current summarization and source selection strategies, crucial to federated performance, are not designed to function under such conditions.
Finally, exposing personal data storage through query endpoints comes with challenges
of its own. Federated sparql query engines are usually benchmarked in private networks.
On the public Web, sparql endpoints have long suffered from availability problems [
Decentralizing the Semantic Web through Incentivized Collaboration
Leveraging strength in numbers through collaboration Decentralized networks have a particular asset: even though individual nodes have limited resources compared to large-scale server clusters, collectively, these nodes possess a far larger amount of computational power and bandwidth. Every single personal data store, as well as every client (computers, smartphones, tablets, . . . ), brings their own cpus— which, in a centralized environment, are typically underused. If we find a way for these nodes to collaborate, we solve the resources problem in decentralized networks. If we take optimization measures, such as performing preparatory work on the nodes closest to the data, we can counter the increased complexity of decentralized algorithms.
Let us apply this insight to the data gathering phase of applications, which in
a decentralized network amounts to federated query evaluation. A straightforward query
to collect the recent activity of one’s contacts would involve the application sending
subqueries to each of those contacts’ data stores. However, social media networks typically
contain overlapping clusters of people, so any person on a contact list is likely to have
a subset of that list as contacts too. Therefore, we can set up agreements along the lines
of “I will help you execute your query if you help me execute mine”. Then instead of
sending subqueries to, for instance, 500 contact nodes, we can delegate larger subqueries
to 10 or 20 hubs in parallel. Instead of executing data gathering entirely at the server or
the client [
Providing incentivization and trust through distributed ledgers
In order to reach sustainable collaborations, nodes need to be incentivized to act as
a contributor to the network. Otherwise, a node cannot be sure that, if it helps other nodes
while idle, the others will return the favor when needed. However, when incentives are
created, nodes also gain a reason for dishonest behavior, so we will need a trust mechanism
to verify whether the work was completed correctly. For lack of a centralized entity
in the network, such incentives and trust need to be established through decentralized
consensus. This is possible through distributed ledgers [
One category of distributed ledgers are blockchains [
Figure 2 shows the architectural components of an individual node in the network. When a query arrives, the node determines what incentive it is willing to accept, and what incentives it is wiling to pay others for subquery delegation. After possibly delegating some parts, and performing the remaining work itself, it maintains provenance of the data and generates a correctness proof of the results. Transactions are registered on the blockchain, such that all participants receive their reward. Some nodes might start performing preparatory work, such as precomputing partial results of common queries in the network, or locally caching other stores’ data to speed up querying. Provenance & Proof
Query Processor spelalencntiiongn discovery collaborative ecosystem of clients and stores blockchain … tasks tasks tasks …
This idea goes beyond data marketplaces [
This proposal can have a strong impact on the scale at which we apply Semantic Web technologies, especially in absence of clear business models. It opens up new directions in decentralized algorithms, and creates a connection between the Semantic Web and agent theory, as well as economic models for incentives. We also must pay attention to challenges such as privacy, perhaps through encryption. Most importantly, this vision sketches a Web-oriented future path to a Semantic Web for large and small players.