The Data Market Austria (DMA) is an ecosystem of federated data and service infrastructures. It aims at making data from various data providers accessible and interoperable by allowing the submission, storage, management and dissemination of static datasets or streaming data services. By creating a metadata vocabulary, standardizing the ingest of data and ensuring the quality and completeness of metadata, it lays the ground to enable participants to share or consume datasets residing in di erent infrastructures. This demo focuses on the mapping services used in the DMA to standardize data from di erent sources using a modi ed version of the DCAT metadata schema. We present tools that enable inter organizational integration of datasets, in a manner that is both user-friendly and powerful enough to handle vast amounts of data.
The amount of data produced every day is growing at breathtaking speed { data
has become an important asset that is of high importance in nearly every
industry sector worldwide [
The Data Market Austria (DMA)5 project addresses these problems by developing the technological, infrastructural, regulatory, and economic foundations for a comprehensive, innovation-supporting, sustainable Austrian dataservices ecosystem. The technological foundation includes Blockchain technology
for provenance, smart contracts and security, interconnected clouds, data access, constraint-preserving processing and analysis algorithms, semi-automated data quality improvement, and recommender-based brokerage technology. Additionally, two pilots in the areas of ICT for Mobility and ICT for Earth Observation are being developed to demonstrate the rst usage scenarios of DMA.
The DMA is a network of participating (or member) organizations that contribute to the data market by o ering their products in form of datasets or services to customers of the DMA. Each participating node must implement a de ned set of services and mandatory standard interfaces. These are, for example instances of a Data Crawler a Metadata Mapper, a Blockchain peer, and Data Management and Storage components. Together with a common conceptual model, these standard interfaces represent the basis of interoperability for the use of datasets in the DMA.
The gateway to this network of nodes containing data and providing services is the DMA portal which, while not hosting any data or providing major services, collects information from all nodes to keep an up to date catalogue of available datasets. The focus of this demo is the design and implementation of this uni ed catalogue. 2
A Semantic Catalogue for a Data Market Since the data in the DMA lies in a set of distributed repositories, it is necessary to build a uni ed catalogue to enable end users to search all available data sets and services. Furthermore, a single catalogue can be exploited for recommendation, deduplication, and various metadata quality measures. In the DMA, the creation of this uni ed catalogue is approached by creating i) a single metadata standard for uni ed representation of data sets, including standardized vocabularies for describing resources, ii) tools for facilitating the compliance of existing metadata with the previous points and iii) the technological foundation for the building and maintenance of the catalogue itself.
The DMA metadata catalogue is based on DCAT-AP, the DCAT application pro le for data portals in Europe6 and extends the schema for DMA use cases. This standardization enables future cooperation with international data portals and ensures that the DMA is easily accessible for cooperating companies with a certain data quality standard. The DMA extension of the DCAT-AP, the Data Maket Core Vocabulary (DMAV), provides more classes and properties for describing datasets and services that are accessible on the DMA. The extension focuses on the business use case of the DMA and adds predicates covering topics like price modeling and dataset exchange, not present in the original DCATAP catalogue. The dmav:priceModel predicate, for example, allows us to handle the transaction fees for commercial datasets that are being made available in
Semantic catalogue for the Data Market Austria the DMA. The dmav:SLA (Service Level Agreement) class allows to model the condition of a service contract in more details.
In the DMA metadata catalogue, every dataset constitutes an RDF7 resource. There is a set of predicates that link every resource to di erent literals, which constitute the values of the metadata elds. These values can be of two types: i) literals, as in the case of dcat:description or owl:versionInfo, or ii) elements of a controlled vocabulary, as in the case of Language or License. These controlled vocabularies, which are managed by PoolParty Semantic Suite8, enable accurate search, ltering and linking of di erent datasets. Additionally, the DMA includes a series of semantic enrichment services which automatically annotate free-text elds (such as dcat:description or dcat:title) with elements of controlled vocabularies.
Since the DMA aims at making available data which was not originally produced for commercialization, we must assume that the metadata describing it does not comply to any particular standard. This is specially true because the data in each node is managed by a di erent organization. Therefore, the conversion to the uni ed metadata standard described above must be treated in a case by case basis.
The DMA provides two tools to facilitate this. The rst is a UI component
in which a node's administrator can upload a sample (in XML or JSON) of the
metadata they wish to make abailable in the DMA. They are then prompted to
select, for each of the metadata elds required by the DMA, which elds of their
metadata schema should be used. This UI tool, called the Metadata Mapping
Builder is, in essence, a user-friendly way to generate XPath and JSONPath
expressions. Once these expressions have been generated, they are arranged into
an RML[
Each node in the DMA that wishes to make a series of datasets available, must implement the following work ow. First, the Data Harvesting Component, which must be con gured by the node's administrator to nd the di erent datasets within the node, sends the corresponding metadata les to the Metadata Mapping Service, which uses the mapping le created as described above to generate, for each dataset, a set of RDF triples (serialized in Turtle format).
Afterwards, the dataset, its original metadata, and the corresponding RDF are ingested into the Data Management component which takes care of the packaging, versioning and assignment of unique identi ers to all datasets, whose
Ivanschitz et al. hashes are furthermore registered in the Blockchain. Next The node's Data Management component publishes, through a ResourceSync9 interface, links to metadata les in RDF format of recently added or updated datasets. This way, the node's metadata management is decoupled from the process of incorporating metadata into the DMA catalogue.
In the DMA's central node, the Metadata Ingestion component constantly
polls the ResourceSync interfaces of all registered nodes, and when new datasets
are reported, harvests their RDF metadata which, let us recall, already complies
with the DMA metadata vocabulary. This metadata is then enriched
semantically. The enrichment is based on EuroVoc10, which is used in DMA as the main
thesaurus. The NLP interchange format [
With small variations, the processes described above are also used for ingesting publicly available data from goverment portals as well as ingesting small amounts of data that an individual would like to make available in the DMA. Acknowledgements The Data Market Austria project is funded by the \ICT of the Future" program of the Austrian Research Promotion Agency (FFG) and the Federal Ministry of Transport, Innovation and Technology (BMVIT) under grant no. 855404
10 http://eurovoc.europa.eu/