Although Solid provides semantic data through a defined interface, the development of applications that are able to discover and consume decentralized data is still a complex task. This paper presents the concept of a rule-based client for discovering, collecting and processing diferently structured Linked Data on Solid storages. Declarative rules for link traversal and RDF querying are expressed in the logic language Notation3. These rule-based programs are executed in an N3 Solid crawler that combines Notation3 reasoning and an HTTP client for fetching public and private RDF resources from Solid storages and the Web.
[
Section 2 briefly introduces Notation3 and its implementations related to the N3 Solid crawler and describes the relation to Link Traversal Query Processing. The concept of the N3 Solid crawler and the components of a possible implementation are presented in Section 3. The application scenarios are explained in Section 4 while the conclusion chapter gives an outlook on the expansion of the concept.
Notation3 Logic [
The application of rule-languages such as N3 for the programming of user agents for Linked
Data is not new [13]. Linked Data-Fu [14] is a "data processing system for data integration and
system interoperation with Linked Data" that uses a rule language based on N3 and executes
HTTP requests for RDF crawling or updating the state of resources in LDP [
Although the N3 Solid crawler is focused on practical applications for the Solid ecosystem,
the underlying concept has overlaps with existing research about querying the Web of Linked
Data, in particular Link Traversal Query Processing (LTQP). LTQP adopts the follow-your-nose
principle of Linked Data during query execution, in which new RDF documents are continuously
crawled by following links between documents. The declarative query language LDQL [16]
separates the selection of the query-relevant region of the Web and the evaluation of a query
over the collected documents. Link path expressions, so-called link patterns, specify the range
of link traversals while the final query is defined with SPARQL 1.1 graph patterns [ 16]. In
the N3 Solid crawler concept, link following is defined by N3 rules that generate RDF triples
representing HTTP requests while the separate query is expressed with graph patterns wrapped
inside an N3 rule. The application of LTQP for decentralized Solid storages can be optimized by
exploiting the existing structural information of the Solid environment such as the Solid type
index [17] or LDP [
The N3 Solid crawler combines N3 reasoning and the loading of additional RDF resources in a
loop. The activity sequence of the reasoning-crawling loop is shown in Figure 1. The process
starts with reasoning over the initial N3 program, a document which includes RDF statements
and N3 rules for logical processing of RDF data. In contrast to the link patterns of LDQL [16],
the crawler does not match patterns to decide which links to follow. Instead, a link extractor
component collects RDF nodes of type http:Request [20] which may already exist in the initial
program as so-called "seed requests" or which are inferred through reasoning. For example, the
N3 rule in Listing 1 creates an instance of http:Request which describes an HTTP GET request for
fetching the public type index document linked from a Solid profile document. A more detailed
example for discovering Solid profile information is presented in Section 4. The N3 built-in
functions log:semantics and log:content already provide a way to import public RDF resources
from the Web [
_:d log:notIncludes { ?ptindex a solid:TypeIndex }. } => {
[] a http:Request; http:mthd httpm:GET; http:requestURI ?ptindex . }.
Listing 1: N3 rule for the definition of an HTTP requests for fetching a type index document track of the URIs that have been fetched to prevent requesting the same URI in the next iteration of the loop. An HTTP client then executes each of the HTTP request objects while an RDF parser processes the results. The HTTP client uses Solid authentication to fetch private RDF resources from Solid Pods. The new RDF content is then merged with the initial N3 program N3 program/
dataset
N3 reasoning to create the input for the next iteration of the reasoning-crawling loop. If every http:Request instance has been processed and there is no new RDF content, a final reasoning step applies an N3 query rule for generating the RDF results of the process. Without a query, the result of the reasoning-crawling loop is the aggregated RDF dataset with inferred triples. In this simple configuration, results are not available until the end of the process. However, the process could return incomplete intermediate results by applying the N3 query reasoning step in every iteration of the reasoning-crawling loop. The evaluation of the N3-based query after every batch of HTTP requests is a compromise between early feedback and performance.
Solid auth
N3 query collect RDF HTTP instances
execute
HTTP request reasoning-crawling loop
RDF parsing no
yes all instances processed ? construct new
N3 rule-data-set yes
no new content ?
N3 query reasoning RDF results The N3 Solid crawler concept can be implemented as a library for client-side or server-side Solid applications. The necessary components of the N3-based crawler are illustrated in Figure 2. The reasoning-crawling loop is implemented in a so-called program executor component that uses the EYE JS, an RDF/JS and a Solid library. The crawler should provide predefined N3 rule sets for RDFS and OWL entailment rules [11], e.g. for the OWL-LD subset [21].
By embedding the rule-based crawler, Solid apps benefit from the advantages of declarative programming with N3 rules instead of traditional imperative and procedural programming. Instead of hard-coding decisions and control flows, the N3 language allows to define logical rules for RDF processing and for when and how to follow links. Furthermore, the reasoning component executes formal logic, e.g. mapping expressions for ontology alignment [15]. This simplifies gathering RDF that is modeled with similar, but diferent ontologies. In comparison to bloated programming code with many if-clauses, the declarative N3 rules improve the reuse and maintenance of discovery and crawling operations for scattered data on Solid storages. In summary, the application of reasoning during the discovery and processing of RDF contributes to application interoperability. If a Solid app is more flexible in handling data that is modeled with similar vocabularies or data that is partitioned in diferent ways, the app is more likely to work with data sets created by other apps.
«subsystem»
Solid application «subsystem»
N3 Solid crawler N3 reasoner
EYE JS
RDF lib.
N3.js
Solid lib.
solid-client-authn reasoning rulesets RDFS/OWL
RDF parser RDF store program executor
Solid authentication
user interface business logic other component N3 rule execution,
RDF crawling
Application scenarios for the N3-rule-based Solid crawling range from the discovery to the collection and processing of partitioned information on Solid storage and the Web.
An example for the discovery of information is the crawling of profile information. A Solid profile is a description of a social agent’s identity that may include links to the agent’s storage, extended profile or preferences documents [ 22]. The starting point for this scenario is the WebID of an agent. Thus, the N3 example program appended in Listing 2 includes RDF statements that describe an initial HTTP GET request to a WebID. A first N3 rule is used to fetch any linked extended profile document. The rule body specifies a condition, an RDF triple pattern that checks if a variable object is connected to the agent’s URI via the property rdfs:seeAlso and an N3 formula that verifies that the current dataset not already includes information about the extended profile. If this condition is true, the rule implies the creation of triples that describe an HTTP request to the variable that matched the URI of the extended profile. Another rule is added to import any solid:TypeIndex documents linked in the profile. Solid type indexes [ 17] are registries for discovering specific data on a Solid storage. Both link traversal rules are applied independently during N3 reasoning so that it is irrelevant if the initial or extended profile document includes the link to the type index. Instead of a given WebID, the N3 program could also start with an RDF document which describes a vcard:Group and fetch profile information for each member of this group. Furthermore, the N3 program can provide context for the reasoning, for example in the form of equivalence and type relations. If the program defines the subclass relation between foaf:Person and foaf:Agent and imports RDFS entailments, the first traversal rule also matches Solid profiles in which the WebID is defined as foaf:Person.
An example for the collection and processing of RDF data is the calculation of an average
value of sensor measurements, shown in Listing 3 in the appendix. The necessary collection,
string and math operations are expressed by means of the N3 built-in functions [
This poster paper presents the concept of an N3 Solid crawler that combines N3 reasoning and link following for the discovery, collection and simple processing of RDF resources from Solid storage and the Web. The author plans to implement the concept as a reusable, isomorphic Typescript library for Solid applications. With the implementation, the performance can be set in relation to existing querying techniques for Solid such as LTQP [18, 19].
Future research includes the modeling of more extensive programs with N3. The concept could be extended with write capabilities and an abstract state machine to allow CRUD operations on Solid resources [13]. Notation3 logic has also been used to define a finite state machine for task network models [23].
Resource Availability Statement: A demo implementation2 of the N3 Solid crawler concept and the poster3 which was presented at the 2nd Solid Symposium 2024 are available on the Solid server of Fraunhofer IIS.
This work has been partially funded by the German Federal Ministry for Economic Afairs and Climate Action (BMWK) through the "Antrieb 4.0" project (Grant No. 13IK015B). 2https://solid.iis.fraunhofer.de/SCS-DS-IoT/public/2024/n3solidcrawler/ 3https://solid.iis.fraunhofer.de/SCS-DS-IoT/public/2024/n3solidcrawler/Poster_N3SoC.pdf [10] R. Verborgh, J. De Roo, Drawing conclusions from Linked Data on the Web: The EYE reasoner, IEEE Software 32 (2015) 23–27. URL: https://josd.github.io/Papers/EYE.pdf. [11] J. De Roo, Euler Yet another proof Engine, 2023. URL: https://eyereasoner.github.io/eye/. [12] J. Wright, J. De Roo, EYE JS - a distribution of EYE reasoner in the JavaScript ecosystem using WebAssembly, 2024. URL: https://github.com/eyereasoner/eye-js. [13] T. Käfer, A. Harth, Rule-based programming of user agents for Linked Data, in: T. BernersLee, S. Capadisli, S. Dietze, A. Hogan, K. Janowicz, J. Lehmann (Eds.), Workshop on Linked Data on the Web co-located with The Web Conference 2018, LDOW@WWW 2018, Lyon, France April 23rd, 2018, volume 2073 of CEUR Workshop Proceedings, CEUR-WS.org, 2018. [14] A. Harth, et al., Linked Data-Fu, 2017. URL: https://linked-data-fu.github.io/. [15] D. Arndt, W. Van Woensel, Decentralization rules: Linking Solid Pods in diferent vocabularies using Notation3, in: DKG-22: 1st Workshop on Consumers of Distributed Knowledge Graphs in Digital, Industry and Space, 2022. URL: https://github.com/doerthe/ oslo-to-fhir/blob/main/paper/Decentralisation_rules.pdf. [16] O. Hartig, J. Pérez, Ldql: A query language for the web of linked data, Journal of Web
Semantics 41 (2016) 9–29. doi:https://doi.org/10.1016/j.websem.2016.10.001. [17] J. Zucker, V. Balseiro, S. Capadisli, T. Berners-Lee, T. Turdean, Type Indexes, Editor’s Draft,
Solid working group, 2023. https://solid.github.io/type-indexes/. [18] R. Taelman, R. Verborgh, Link traversal query processing over decentralized environments with structural assumptions, in: T. R. Payne, et al. (Eds.), Proceedings of the 22nd International Semantic Web Conference, Springer, 2023, pp. 3–22. doi:10.1007/ 978-3-031-47240-4_1. [19] R. Taelman, et al., Link traversal for Comunica, 2024. URL: https://github.com/comunica/ comunica-feature-link-traversal. [20] J. Koch, C. A. Velasco, P. Ackermann, HTTP Vocabulary in RDF 1.0, W3C Working Group
Note, W3C, 2017. https://www.w3.org/TR/HTTP-in-RDF10/. [21] B. Glimm, A. Hogan, M. Krötzsch, A. Polleres, Owl: Yet to arrive on the web of data?,
ArXiv abs/1202.0984 (2012). URL: https://api.semanticscholar.org/CorpusID:585128. [22] T. Berners-Lee, S. Capadisli, V. Balseiro, T. Turdean, J. Zucker, Solid WebID Profile, Editor’s
Draft, Solid working group, 2024. https://solid.github.io/webid-profile/. [23] W. Van Woensel, S. Abidi, K. Tennankore, G. Worthen, S. S. R. Abidi, Explainable decision support using task network models in Notation3: Computerizing lipid management clinical guidelines as interactive task networks, in: Artificial Intelligence in Medicine, Springer International Publishing, Cham, 2022, pp. 3–13. @prefix rdfs: <http://www.w3.org/2000/01/rdf −schema#>. @prefix log: <http://www.w3.org/2000/10/swap/log#>. @prefix http: <http://www.w3.org/2011/http#>. @prefix httpm: <http://www.w3.org/2011/http −methods#>. @prefix foaf: <http://xmlns.com/foaf/0.1/>. @prefix solid: <http://www.w3.org/ns/solid/terms#>. # seed request [] a http:Request; http:mthd httpm:GET;
http:requestURI <https://id.inrupt.com/myusername>. # rule for fetching extended profile { ?webid a foaf:Agent; rdfs:seeAlso ?extprofile.
_:b log:notIncludes { ?extprofile a ?type }. } => {
[] a http:Request; http:mthd httpm:GET; http:requestURI ?extprofile. }. # rule for fetching public type index { ?webid solid:publicTypeIndex ?ptindex.
_:d log:notIncludes { ?ptindex a solid:TypeIndex }. } => {
[] a http:Request; http:mthd httpm:GET; http:requestURI ?ptindex . }. # context for reasoning foaf:Person rdfs:subClassOf foaf:Agent.
Listing 2: Section of an N3 program for crawling Solid profile information ?observation a sosa:Observation; sosa:madeBySensor ?sensor; sosa:hasSimpleResult ?tempvalue. } ?tempvalues) log:collectAllIn []. ?tempvalues math:sum ?sum; math:memberCount ?count. ?count math:greaterThan 0.
(?sum ?count) math:quotient ?avg. } => {
?sensor <http://example.org/averageValue> ?avg. }.
Listing 3: Section of an N3 program for calculating the average value of temperature measurements of diferent sensors