Scientific observations and other open data are usually made available online in a tabular manner as CSVs and spreadsheets. However, users of these data face three main challenges when attempting to use these products: finding which datasets are related to a topic of interest; determining which existing information can be used to extend a given dataset; and how to share their integrated dataset results with the rest of the community. In this paper we present WDPlus, a framework designed to address these challenges by leveraging Wikidata. WDPlus allows searching for heterogeneous datasets, facilitates completing tabular data using Wikidata and proposes a mechanism to extend Wikidata in a decentralized manner.
Today, data about any domain can be found on the web in data repositories, web APIs and millions of spreadsheets and CSV files. These data comes in a myriad of formats, layouts, terminology and cleanliness that make them dificult to integrate together.
Users of these data face three main challenges. The first one is ifnding datasets related to a feature or topic of interest. For example, climate scientists often look for years of observational data from authoritative sources when estimating the climate of a region. The second challenge is how to complete a given dataset with existing knowledge: machine learning applications are data hungry and require as many data points and features as possible to improve their predictions, which often requires integrating data from diferent sources. The final challenge is sharing integrated results: once several datasets have been merged together, how to make them available to the rest of the community?
Knowledge graphs have become the preferred technology to address these challenges. Large organizations, including search engine providers, shopping giants and finance institutions are investing in large knowledge graphs to integrate and retrieve heterogeneous data. However, data integration pipelines are usually created manually, require significant expertise, and are seldom available to the general public. Similarly, linking to existing datasets in the the Linked Open Data Cloud1 usually requires the expertise of a knowledge engineer to properly identify the appropriate target instances to link to in other datasets. 2
WDPlus is a framework designed to explore and build large multidomain knowledge graphs using the wealth of structured data available on the web. Our approach shifts the burden of semantic linking away from data publishers to communities that have an incentive to do so in a collaborative manner.
An overview of WDPlus can be seen in Figure 2. At its core,
WDPlus relies on Wikidata [
New data from the web may be processed through our proposed WDPlus toolkit, designed to automatically create prototype table models that can be extended with existing Wikidata knowledge and be refined into full Wikidata extensions. We provide more information about our toolkit, table metadata index and satellites below. 2.1
In order to process and link tables and spreadsheets, we created a
toolkit with the following capabilities:
• Entity Linking: One of the main challenges when linking
a dataset to existing knowledge graphs is to find whether
the entities described in the dataset are already defined in
a target knowledge graph. In order to facilitate this task,
we have created a CSV Wikifier (inspired by [
The WDPlus toolkit is available online with a MIT License.6 2.2
We have created an index to store table metadata, where each record
is an instance of the Wikidata Dataset class (Q1172284). The
rationale for the metadata index is to contain datasets that may not be
necessarily materialized as a knowledge graph, but that would be
interesting resources to link and extend other datasets. Our metadata
schema relies on Wikidata and Schema.org [
The metadata index connects heterogeneous tables to Wikidata, and therefore we use it for data augmentation. For example, a user with a table containing demographic information by country may be interested in adding climate observations to find correlations between population and temperature. Thus, given a dataset to augment (e.g., CSV with city name and population) and a search term (e.g., temperature), the metadata index returns a ranked selection of datasets that may complement the target dataset with additional metadata. Once a user selects a result dataset, WDPlus automatically adds a new column in the dataset to complete, filling in information for every row from Wikidata or materializing the search results. Missing values are currently not imputed. The API for the metadata index can be found online.7 2.3
Users may want to contribute tabular datasets to WDPlus as an extension of Wikidata entities. This process can be accomplished in two steps. The first one consists on an interactive table understanding step, where the user is presented the GUI shown in Figure 2 with entity linking candidates for all cells included in the target table. The GUI then helps users define a table model, indicating how to map a target table to the Wikidata data model. We collect all qualifiers for each assertion (e.g., source, point in time and space, units, etc.) as these are crucial for ensuring data quality and trust. 6https://github.com/usc-isi-i2/t2wml
The second step generates RDF triples from the table, storing them in a Wikidata satellite using the WDPlus toolkit. WDPlus defines a special range of URIs for each satellite, which are separated in graphs. These graphs may be stored in separate triplestores, allowing to grow Wikidata in a decentralized manner while keeping all related pointers in the same metadata index. Wikidata satellites may define new entities and properties that are not part of Wikidata. However, our early entity linking process minimizes the creation of duplicate entities, ensuring that all satellites are linked together.
A key aspect of WDPlus is that we store all curated table models in our metadata index for each transformed dataset. This helps keeping the provenance of all results in a satellite and may inform the transformation of other tabular data with very similar structure. For example, in the US, demographic data is usually provided for each county as a CSV file. A single table model for one county can be used to transform the county CSVs of the whole country.
This paper introduces WDPlus, a framework that leverages Wikidata to link, search and augment tabular data. While the framework is still under development, our WDPlus prototype integrates heterogeneous data from crime, economic and education domains, illustrating the feasibility of our approach.