A growing amount of public procurement data is being made available in the EU for the purpose of improving the e ectiveness, e ciency, transparency, and accountability of government spending. However, there is a large heterogeneity, due to the lack of common data formats and models. To this end, we developed an ontology network for representing and linking tender and company data and ingested relevant data from two prominent data providers into a knowledge graph, called TBFY. In this poster paper, we present an overview of our knowledge graph.
In the EU, public authorities spend around 14% of GDP on the purchase of services, works, and supplies every year7. Therefore, a growing amount of public procurement data is being made available in the EU through public portals for the purpose of improving the e ectiveness, e ciency, transparency, and accountability of government spending. However, there is a large heterogeneity, due to the lack of common data formats and models for exposing such data.
There are various standardization initiatives for electronic procurement, such
as Open Contracting Data Standard (OCDS)8 and TED eSenders 9. However,
these are mostly oriented to achieve interoperability, document-oriented, and
provide no standardised practices to refer to third parties, companies participating
in the process, etc. This again generates a lot of heterogeneity. The Semantic Web
approach has been proposed as a response [
To this end, we developed an ontology network for representing and linking tender and company data and ingested relevant data from two prominent data providers into a knowledge graph, called TBFY. In this poster paper, we present an overview of our knowledge graph for public procurement. 2
We integrated two datasets according to an ontology network: tender data provided by OpenOpps10 in the OCDS format and company data provided by OpenCorporates11. OpenOpps has gathered over 2M tender documents from more than 300 publishers through Web scrapping and by using open APIs, while OpenCorporates currently has 140M entities collected from national registers. 2.1
We are currently using two main ontologies. First, an ontology for tender data (see Figure 1) that we developed using the OCDS' data model12.
Second, we reused the euBG ontology for company data13. Both ontologies reuse other ontologies and vocabularies (FOAF, Dublin Core, etc.). 10 https://openopps.com 11 https://opencorporates.com 12 https://github.com/TBFY/ocds-ontology 13 https://github.com/euBusinessGraph/eubg-data 2.2
The data ingestion process is composed of several steps using data APIs of both providers (see Figure 2). Initially, company data is extracted from OpenOpps for a given period of time and preprocessed primarily to handle null values. Suppliers appearing in tender data are matched against company data provided by OpenCorporates by using the reconciliation service of OpenCorporates. The matched company data is extracted and then supplier data is annotated with the corresponding company identi ers.
OpenOpps
API
1.
Fetch tender data
2.
Preprocess
Annotate Reconciliation
API Reconcile 3. 4.
OpenCorporates
API
5.
Fetch company data company data tender data
Ontologies
6.
RML Mapping
Knowledge
Graph
7.
Upload to
KG
Finally, the extracted datasets are translated to RDF using the RDF Mapping Language (RML)14 according to our ontology network. The supplier data and company data is linked through the owl:sameAs property. Thereafter data is uploaded to a graph database, namely GraphDB15. The current release of the knowledge graph includes 23M triples originating from tender data collected initially for the rst quarter of 2019. The knowledge graph is available online16. An example query and its results are depicted in Figure 3. The example query lists top ten companies in the Norwegian jurisdiction that have the highest number of supplier role, where the jurisdiction data comes from OpenCorporates and contract data comes from OpenOpps. 3
Currently, one of the main issues concerns data quality including missing, duplicate, poorly formed, and erroneous data. For example, the missing address 14 http://rml.io/ 15 http://graphdb.ontotext.com/ 16 http://data.tbfy.eu information, variations on address format, etc. hinder the quality of reconciliation process. Currently, we are working on various approaches to improve data quality ranging from machine learning to crowd-sourcing.
Acknowledgements. This work has been partly funded by the EU H2020 projects TheyBuyForYou (780247) and euBusinessGraph (732003).