The European Data Portal fosters the adoption and distribution of Linked Open Data by o ering more than 130 million RDF triples. It applies DCAT-AP, the RDF vocabulary for public sector datasets in Europe. Many Open Data solutions do not satisfactorily support this metadata speci cation. To address this problem, we designed and implemented a novel platform for harvesting and managing native DCAT-APcompliant RDF. Our approach uses a triplestore as main database and applies state-of-the-art development and deployment patterns to ensure performance and scalability.
F. Kirstein et al.
RDF and SPARQL). If necessary, the acquired metadata is transformed into DCAT-AP-compliant RDF. The Registry supports the resource-based management of the major DCAT-AP entities via a RESTful API (Catalogs, Datasets and Distributions). It receives data from the Harvester, preprocesses it and stores the RDF in the triplestore. The preprocessing includes generation of consistent URIs, validation based on SHACL and creation of a full-text search index. The results of the validation are represented by the Data Quality Vocabulary (DQV)9. The Quality Service fetches the metadata and validation results, performs availability checks for external links and creates detailed quality reports. Fig. 1 illustrates the highlevel architecture, including the most essential services. The system makes extensive use of an asynchronous and reactive programming paradigm, applies the container technology Docker10 for deployment and supports container-orchestration, like Kubernetes11. Therefore, we were able to decrease the hardware workload and increase the overall performance, in comparison to the CKAN-based solution. A full harvesting run was accelerated by a factor 30, where the major blocking factor is the limited response time of the data sources. We have shown that our solution and Linked Data can be applied successfully in a real world scenario with signi cant advantages over established Open Data solutions, e.g., a higher scalability, a more exible data schema and rich queries with SPARQL. Since the European Commission (EC) establishes DCAT-AP as a standard, our work acts as an enabler for its broad adoption. Open Data from public administrations is an essential source for Linked Open Data. Hence, our solution will have a direct impact on its dissemination. The entire platform is available as Open Source.12 References 9 https://www.w3.org/TR/vocab-dqv/ 10 https://www.docker.com/ 11 https://kubernetes.io/ 12 https://gitlab.com/european-data-portal