This research focuses on the problem of helping clients nd relevant SPARQL endpoints. To the best of author's knowledge, there are two online services that clients could use to try to nd SPARQL endpoints based on their content: DATAHUB and/or VoID STORE. However, both services rely on static content descriptions provided by publishers themselves. Many of the endpoints listed in the DATAHUB have been o ine for years; also, of the endpoints surveyed, VoID descriptions are only available in the DATAHUB for 33.3% and in the VoID STORE for 22.4%. Authors instead propose to compute extended VoID descriptions for public endpoints directly through their SPARQL interface. The SPORTAL service is available online at http://www.sportalproject.org/ originally introduced in journal publication[3]
Rather than relying on publishers to compute and keep content descriptions
up to date, we propose to compute content descriptions directly from the
endpoints themselves. In particular, it is proposed to design a set of queries that
can be issued to endpoints, where the results of these queries can then be used
to build a catalogue that enables clients to nd endpoints with relevant
content with general descriptions. Such descriptions are normally not provided by
the publishers of SPARQL endpoints independently of the endpoint itself. This
work explores the feasibility of computing content descriptions directly from
SPARQL endpoints. More concretely, SPORTAL (SPARQL PORTAL) is
proposed which is a centralised catalogue indexing content descriptions computed
from individual SPARQL endpoints [
This section lists the set of SPARQL 1.1 queries that are used to compute a VoID-like description from the content indexed by an endpoint. 2.1
We test the availability and SPARQL 1.1 compliance of an endpoint using two queries with features: sub-queries and the count aggregate function. 2.2
A set of \dataset-level" queries is listed in Table 1 that form a core part of VoID to ascertain the number of triples (QB1), and the number of distinct classes (QB2), properties (QB3), subjects (QB4), and objects (QB5). These queries require support for SPARQL 1.1 COUNT and sub-query features. Using these agents can use them to nd endpoints indexing datasets that fall within a given range of triples in terms of overall size, or, for example, to nd the endpoints with the largest datasets. Counts may be particularly useful - in combination with later categories - to order the endpoints; for example, to nd the endpoints with a given class and order them by the total number of triples they index. Third similar statistics about the instances of each class following the notion of class partitions in VoID were ascertained: a subset of the data considering only triples where instances of that class are in the subject position. Table 2 lists the six queries used. The rst query QC1 merely lists all class partitions. The other ve queries (QC2 6) count the triples and distinct classes, predicates, subjects and objects for each class partition; e.g., QC2 retrieves the number of triples where instances of that class are in the subject position. Queries QC2 6 introduce COUNT, sub-queries and also GROUPBY features from SPARQL 1.1. No Query QC1 CONSTRUCT <D> v:classPartition [ v:class ?c ] WHERE ?s a ?c QC2 CONSTRUCT v:classPartition [ v:class ?c ; v:triples ?x ]
WHERE SELECT (COUNT(?p) AS ?x) ?c WHERE ?s a ?c ; ?p ?o GROUP BY ?c QC3 CONSTRUCT v:classPartition [ v:class ?c ; v:classes ?x ] WHERE
SELECT (COUNT(DISTINCT ?d) AS ?x) ?c WHERE ?s a ?c , ?d GROUP BY ?c QC4 CONSTRUCT v:classPartition [ v:class ?c ; v:properties ?x ] WHERE
SELECT (COUNT(DISTINCT ?p) AS ?x) ?c WHERE ?s a ?c ; ?p ?o GROUP BY ?c QC5 CONSTRUCT v:classPartition [ v:class ?c ; v:distinctSubjects ?x ] WHERE
SELECT (COUNT(DISTINCT ?s) AS ?x) ?c WHERE ?s a ?c GROUP BY ?c QC6 CONSTRUCT v:classPartition [ v:class ?c ; v:distinctObjects ?x ] WHERE
SELECT (COUNT(DISTINCT ?o) AS ?x) ?c WHERE ?s a ?c ; ?p ?o GROUP BY ?c Once catalogued, agents can use statistics describing class partitions of the datasets to nd endpoints mentioning a given class, where they can additionally (for example) sort results in descending order according to the number of unique instances of that class, or triples used to de ne such instances, and so forth. Hence the counts computed by (QC2 6) help agents to distinguish endpoints that may only have one or two instances of a class to those with thousands or millions. Likewise, criteria can be combined arbitrarily for multiple classes, or with the overall statistics computed previously.
This section looks at property partitions in the dataset, where a property partition refers to the set of triples with that property term in the predicate position. Queries are listed in Table 3. As before, QD1 lists the property partitions. QD2 4, count the number of triples, distinct subjects and distinct objects. Using these statistics about property partitions in the catalogue, agents can, for example, retrieve a list of public endpoints using a given property ordered by the number of triples using that speci c property. Likewise criteria can be combined arbitrarily for multiple properties, or with the dataset- or class-level metadata previously collected; for example, an agent may wish to order endpoints by the ratio of triples using a given property (where the count from QD2 for the property in question can be divided by the total triple count from QB1), or to nd endpoints where all subjects have an rdfs:label value (where the count computed from QD3 for that property should match the count for QB4). Fifth, the focus is to look at how classes and properties are used together in a dataset, gathering statistics on property partitions nested within class partitions: these statistics detail how properties are used for instances of speci c classes. Table 4 lists the four queries used. QE1 lists the property partitions nested inside the class partitions, and QE2{4 count the number of triples using a given predicate for instances of that class, as well as the number of distinct subjects and objects those triples have.An agent could use the resulting metadata to nd endpoints describing instances of speci c classes with speci c properties, with ltering or sorting criteria based on, e.g., the number of triples. For example, an agent might be speci cally interested in images of people, where they would be looking for the class-partition foaf:Person with the nested property-partition foaf:depicts. It would have been possible to nd endpoints that have data for the class foaf:Person and triples with the property foaf:depicts, but not that the images were de ned for people. 3.3
In the nal set of experiments, the focus is to look at queries that yield statistics
not supported by VoID as listed in Table 5. In particular, experiments were
designed to see if endpoints can return a subset of statistics from the VoID
Extension Vocabulary , which include counts of di erent types of unique RDF
terms in di erent positions: subjects IRIs (QF1), subject blank nodes (QF2),
objects IRIs (QF3), literals (QF4), object blank nodes (QF5), all blank nodes
(QF6), all IRIs (QF7), and all terms (QF8). Inspired by the notion of \schema
maps" as proposed by Kinsella et al. [
This section looks at how public SPARQL endpoints themselves perform for the list of previously enumerated self-descriptive queries. A list of 540 SPARQL endpoints registered in the DATAHUB along with a list of 137 endpoints from Bio2RDF releases 1-3 was collected resulting in total of 618 unique endpoints (59 endpoints were present in both lists) were considered.
Although there is no generic exact method of determining the engine powering a SPARQL endpoint, the HTTP header may contain some clues in the Server eld. Hence the rst step was to perform a lookup on the endpoint URLs. The response codes of this step, where quite a large number of endpoints return error codes 4xx, 5xx, or some other exception. This indicates that a non-trivial fraction of the endpoints from the list are o ine. With respect to the server names returned by those URLs that returned a HTTP response, although some of the server names denote generic HTTP servers - more speci cally Apache, nginx, Jetty, GlassFish, Restlet and lighttpd - some names that indicate SPARQL implementations namely Virtuoso, Fuseki and 4s-httpd (4store).
Hence, the next step was to look at how many endpoints respond to the basic availability query QA1. Given that the queries were run in an uncontrolled environment, multiple runs were performed to help mitigate temporary errors and remote server loads: the core idea is that if an endpoint fails at a given moment of time, a catalogue could simply reuse the most recent successful result. Along these lines, three weekly experiments were run. In total, 306 endpoints (49.5%) responded to QA1 at least once in the three weeks; these endpoints were considered to be operational and others to be o ine. Of the operational endpoints, 7 (1.1%) responded successfully exactly once to QA1, 28 (4.5%) responded successfully exactly twice, and 272 (44.1%) responded successfully thrice. In the most recent run, 298 endpoints responded to QA1. Of these, 168 (56.4%) also responded with a single result for QA2, indicating some support for SPARQL 1.1 in about half of the operational endpoints.
The focus is on the overall success rates for each query, looking at the ratio of the 307 endpoints that return non-empty results. Figure 1, shows the success rates varying from 25% for QE3 on the lower end, to 94% for QC1 on the higher end. The three queries with the highest success rates require only SPARQL 1.0 features to run: list all class partitions (QC1), all property partitions (QD1), and all nested partitions (QE1). Hence, only 49% could respond to the SPARQL 1.1 test query QA1 - more endpoints can answer queries not requiring novel SPARQL 1.1 features such as counts or sub-queries. The query with the highest success rate 1 g iren 0.8 w s n tsa 0.6 n i o ndp 0.4 e f o itao 0.2 R
0 105 104 lts 103 u s freo 102 e izS 101 100 that involved SPARQL 1.1 features was QB1, where 51% of endpoints responded with a count of triples. In general, queries deriving counts within partitions had the lowest success rates
B1 B2 B3 B4 B5 C1 C2 C3 C4 C5 C6 D1 D2 D3 D4 E1 E2 E3 E4 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10
Query
Finally, the focus was on runtimes for successfully executed queries, incorporating the total response time for issuing the query and streaming all results. Figure 3, presents the runtimes for each query considering di erent percentiles across all endpoints returning non-empty results in log scale. A large variance in runtimes is noticed, which is to be expected given that endpoints host datasets of a variety of sizes and schemata on servers with a variety of computational capacity. In general, the 25th percentile roughly corresponds with the one second line, but that slower endpoints may take tens or hundreds of seconds. The at max trend seems to be the e ect of remote timeout policies, where query runtimes often maxed out at between 100 - 120 seconds, likely returning partial results.
Our primary motivation is to investigate a method for cataloguing the content of public SPARQL endpoints without requiring them to publish separate, static descriptions of their content-or indeed, for publishers to o er any additional infrastructure other than the query interface itself. This section, describes the SPORTAL catalogue itself, including its interfaces, capabilities and limitations.
103 )s 102 ( s e m i tun 101 r y r e u Q 100 B1 B2 B3 B4 B5 C1 C2 C3 C4 C5 C6 D1 D2 D3 D4 E1 E2 E3 E4 F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 Query
SPORTAL, itself provides a public SPARQL endpoint, where the RDF triples produced by the CONSTRUCT clauses of the self-descriptive queries issued against public endpoints can themselves be queried. This allows users with speci c requirements in mind to interrogate the catalogue in a exible manner. Let us consider an agent looking for SPARQL endpoints with at least 50 unique images of people, where this agent may ask the following query with results:
Query for 50 images Results with 50 images Query SELECT DISTINCT ?endpoint ?imgs WHERE f ?dataset v:classPartition [ v:class f:Person ; v:propertyPartition [ v:property f:depiction; v:distinctObjects ?imgs ] ] ; v:sparqlEndpoint ?endpoint . FILTER(?imgs > 50) g ORDER BY DESC (?imgs) ?endpoint ?imgs http://eu.dbpedia.org/sparql 4,517 http://eudbpedia.deusto.es/sparql 4,517 http://data.open.ac.uk/query 311
To help non-expect users, SPORTAL also provides an online user interface with a number of functionalities. First, users can search for speci c endpoints by their URL, by the classes in their datasets, and/or by the properties in their datasets. These features are o ered by means of auto-completion on keywords, meaning that the agent need not know the speci c IRIs they are searching for. If a user clicks on or searches for an endpoint, they can retrieve all the information available about that endpoint as extracted by the queries previously described, providing an overview of how many triples it contains, how many subjects, how many classes, etc. (as available). The SPORTAL user interface also includes some graphical visualisations of some of the high-level features of the catalogue, such as the most popular classes and properties based on the number of endpoints in which they are found, the most common server headers, and so forth. While this may not be of use to a user with a speci c search in mind, it o ers a useful overview of the content available across all endpoints on the Web, and the schema-level terms that are most often instantiated.