Building Web APIs on top of SPARQL endpoints is becoming a common practice to enable universal access to the integration favorable dataspace of Linked Data. However, the Linked Data community cannot expect users to learn SPARQL to query this dataspace, and Web APIs are the most common way of enabling programmatic access to data on the Web. However, the implementation of Web APIs around Linked Data is often a tedious and repetitive process. Recent work speeds up this Linked Data API construction by wrapping it around SPARQL queries, which carry out the API functionality under the hood. Inspired by this, in this paper we present grlc, a lightweight server that translates SPARQL queries curated in GitHub repositories to Linked Data APIs on the fly.
queries remains. As shown in this paper, users require organised APIs that adapt to their existing query curation workflows. This requires a paradigm shift where not just the data, but the queries themselves also become first class citizens.
In recent Linked Data projects such as CEDAR [
The organization and management of SPARQL queries is central to the study of
their efficiency, nature, and use at improving Linked Data applications. SPARQL
query logs have been used to study differences between queries by humans and
machines [
The Semantic Web has developed significant work on the relationship
between the Linked Data and Web Services [
From GitHub Repos to Linked Data APIs
It is becoming increasingly popular to maintain a great variety of data projects
beyond software code using git repositories, especially those in GitHub8 [
This section investigates how the organisational characteristics of GitHub repositories can be used to build, manage and maintain a Swagger-spec compliant API. First, in Section 3.1 we study the requirements of Swagger-compliant APIs and map them to elements of the GitHub API. Since these elements are insufficient for a complete API spec, in Section 3.2 we propose to complete it with non-intrusive SPARQL decorators. 3.1
We propose to align the metaphor of the repository with that of the API, since both share abstract notions of organizing files and operations in a way that is meaningful for their users. For this reason, in this section we study a possible mapping between the two. Table 1 shows the mapping between the attribute requirements of the Swagger RESTful API specification, and how these correspond with either attributes of the GitHub API (repository organisation elements) or attributes of the SPARQL usage decorator (usage metadata elements). The latter are discussed in Section 3.2. 3.2
To complete the mapping of the GitHub API to the Swagger RESTful API specification shown in Table 1, we propose SPARQL decorators to add metadata in queries as comments. We assume SPARQL queries organised as .rq files in git repositories. Each of these files will translate into an API operation. We propose to comment them in the first file lines, with the syntax depicted in the following example9:
9Additional examples can be found at https://github.com/CEDAR-project/ Queries and https://github.com/CLARIAH/wp4-queries Swagger attribute Swagger version
Scope
API API version Title Contact name Contact URL License Host Base path Schemes Path name Path method Path tags
API API API API API API API API Operation Operation
Operation Path description Operation Path parameters Operation Path responses
Operation
Description
Mapping Version number of com- Static: independent of the LDA. pliant Swagger RESTful Currently version 2.0 of the SwagAPI specification ger RESTful API spec is sup
ported Version number of the GitHub API: last repo release API from the release API through
GET /repos/:owner/:repo/releases/ latest Title of the API GitHub API: name of the repository through GET /repos/:owner/: repo Author and contact in- GitHub API: login name of formation the repository owner through GET
/repos/:owner/:repo URL to be followed for GitHub API: link to the HTML additional information page of the repository owner
through GET /repos/:owner/:repo License under which the Repository file: a link to the raw API is released LICENSE file of the repo if it exists;
empty otherwise Host name to compose grlc parameter: supplied host the API calls name in grlc’s configuration;
localhost by default Base path to compose GitHub API: the string /: the API calls owner/:repo in GET /repos/:owner/:
repo Supported schemes to Static: http is supported compose the API calls Name of the API opera- GitHub API: the file name, withtion out the extension, of any .rq file
found in the repository HTTP method for the Static: GET is supported operation (GET, POST) Tags under which the SPARQL decorator: the parsed operation will be classi- list of the decorator tags in .rq fied files Description of the API SPARQL decorator: the parsed operation description decorator in .rq files Parameters of the oper- SPARQL decorator: all paramation eter placeholders parsed in the
query (see Section 4) Responses of the opera- SPARQL decorator: response tion codes on success, datatypes of parameters (see Section 4) #+ summary: A brief summary of what the query does #+ endpoint: http://example.org/sparql #+ tags: #+ - UseCase1 #+ - Awesomeness
This indicates the summary of the query (which will document the API operation), the endpoint to send the query, and the tags under which the operation falls in. The latter helps to keep operations organized within the API. In addition, we suggest to include two special files in the repository. The first is a LICENSE file containing the license for the SPARQL queries and the API. The second is the endpoint.txt file, with the URI of a default endpoint to direct all queries of the repository. When parsing the repository (see Section 4) the target endpoint will be the one indicated by the #+ endpoint decorator, the endpoint.txt file, or http://dbpedia.org/sparql, in this order of preference.
grlc10 is a thin gateway that automatically builds complete, well documented, and neatly organized Linked Data APIs on the fly, with no input required from users beyond a GitHub user and repository name. To do so, it implements the GitHub API mappings proposed in Section 3.1, and uses the SPARQL decorators described in Section 3.2. It provides three basic operations: (1) generates the Swagger spec of a specified GitHub repository; (2) generates the Swagger UI to provide an interactive user-facing frontend of the API contents; and (3) translates http requests to call the operations of the API against a SPARQL endpoint with several parameters. If the GitHub repository at https://github.com/: owner/:repo contains decorated SPARQL queries, grlc uses these, together with organisational repo information from the GitHub API, to build the API interface automatically. Assuming that grlc is running in :host, these operations are available at the following routes: – http://:host/:owner/:repo/spec: JSON Swagger-compliant specification, using the mappings of Section 3 – http://:host/:owner/:repo/api-docs: Swagger-UI, rendered using such mappings, as shown in Figure 1. – http://:host/:owner/:repo/:operation?p_1=v_1...p_n=v_n: http GET request to :operation with parameters p1; :::; pn taking values v1; :::; vn. grlc composes the Swagger spec as follows: (1) the user requests the URI http://:host/:owner/:repo/spec11 to a host running grlc; (2) grlc issues the http GET request to the GitHub API at https://api.github.com/repos/: owner/:repo, using the owner and repo names indicated in the previous step; (3) for each .rq file described in the response, grlc derefences https://raw. githubusercontent.com/:owner/:repo/master/file.rq to get the SPARQL file contents; (3) grlc parses these file contents to extract: (a) the values of 10Source code at https://github.com/CLARIAH/grlc; demo instance at http:// grlc.clariah-sdh.eculture.labs.vu.nl/ 11Requested from any http compliant client: a Web browser, curl, etc. the decorators (if any), and (b) any parameter placeholders in the query; (4) grlc uses all the gathered data to compose the Swagger spec, and returns it to the client as JSON. The composition of the Swagger UI is analogous: first the JSON spec is composed and, after, it is used to render the Swagger UI template12.
Operations of the form http://:host/:owner/:repo/:operation?p_1=v_1. ..p_n=v_n, are executed by grlc by first retrieving the raw query at https:// raw.githubusercontent.com/:owner/:repo/master/:operation.rq, and then rewriting it using the parameter values supplied in the corresponding placeholders.13. After this, the query is submitted to the endpoint indicated by the methods described in Section 3.2, using supplied http headers (e.g. content-type). The endpoint results are forwarded to the client application. 5
In this section we evaluate requirements satisfied by grlc in two use cases.
Dutch Historical Census Data. The CEDAR project14 has published the
Dutch historical censuses (1795–1971) as 5-star Linked Data [
Born Under a Bad Sign. In CLARIAH19, querying structured humanities
data from combined sources is central. This particular use case focuses on
validating the hypothesis that prenatal and early-life conditions have a strong impact
on socioeconomic and health outcomes later in life, by using 1891 census records
of Canada and Sweden. These were converted to Linked Data with QBer [
In this paper we have presented grlc, a novel approach to automatically build Linked Data APIs by using SPARQL queries stored and documented in git repositories. Our approach addresses two pitfalls of current practice in constructing Linked Data APIs: (1) the coupling of SPARQL curation workflows and the API infrastructure, which hampers query reuse and forces query retyping and ramifications; and (2) the common lack of organisation in Linked Data APIs. grlc maps the Swagger specification with GitHub API features and a proposed SPARQL decorator notation, and builds and maintains Linked Data APIs automatically with minimal effort. We argue that this approach enables a better coexistence of SPARQL and non-SPARQL savvy applications, and allows developers to switch their efforts from API infrastructure to applications.
We plan to extend this work in several ways. First, we will support additional repository elements and SPARQL decorators. Second, we will add compatibility with other collaborative coding platforms, like Bitbucket and GitLab, enabling private APIs and authentication. Finally, we plan to create a grlc companion to facilitate the curation of SPARQL queries in git repositories. 18https://github.com/the-open-university/BASIL 19http://clariah.nl/