The vast amount of data on the web presents a growing need to advance data search. Rich and meaningful metadata can enhance the discovery of datasets and establish connections between them. Where metadata is not comprehensive, it can be expanded through dataset proling. The relative importance of di erent types of pro les varies depending on the user's context and the objective of the task. We discuss an approach to nd un-Linked datasets and increase result relevance by o ering related information. We propose generating rich pro les for datasets; counting the number and strength of relations between them and showing a graph of pro les that represents connections between different datasets. We can thereby capture correlations between datasets that can then improve the e ciency and e ectiveness of data search. If developed further this would improve discoverability and reusability of datasets.
The availability of data is growing rapidly and with it, big data technologies and
services. The vast amount of information available presents unique challenges
around data integration, data ownership or assuring data quality to users [
Value derived from data is generated to a great extent through
understanding relations between di erent datasets and between the entities they describe.
Most techniques for extracting information from datasets rely on the
statistical characteristics of the data or do not scale well [
The remainder of this paper is structured as follows. Section 2 describes related work that in uenced the concept presented in section 4. Section 3 gives a rationale for the focus on un-Linked data and presents use scenarios. 2 2.1
Data pro ling comprises a broad range of methods to e ciently describe datasets
[
Naumann proposes joining two datasets based on their pro les, similar to
a reverse engineering process which reveals possible relations between datasets
based on constraints identi ed in the data [
These pro ling techniques are strongly focused on numerical data and
creating associations between datasets columns. On the other side of the spectrum,
Linked Data is used to improve pro ling processes. Fetahu et al. propose
building a graph of linked datasets including the relative importance of each node,
each of which is an openly available dataset. This is done with the objective of
improving pro ling accuracy [
Users should be involved in the process of searching for data and richer
proles can support that search process. Ele et al. emphasise the usefulness of
explicit semantic information for the dataset discovery task to support the
interlinking of datasets [
The most basic type of pro les are structural pro les for which no external
resource analysis is needed. These are created by reporting the number of columns,
rows and their data types, along with information about uniqueness and
completeness of the data [
Although these di erent types of pro les exist, they were created to solve particular problems and are rarely used together. If combined meaningfully, these pro les could enhance discoverability of datasets even further. Section 4 describes how the availability of richer pro les could be used to understand connections between datasets. 3 3.1
Existing research on techniques for understanding the relationships between
datasets, such as [
We are focused on three use scenarios that address di erent user needs. These serve as high-level exempli cation of the variety of issues users can encounter when attempting to nd datasets. Apart from the type of data, we note that the context and the individual objective when searching for datasets in uences the relevance of a speci c type of pro le for a speci c task.
Scenario A Posing a question that can be answered by consulting a single data set. This scenario implies limited time and there should be no, or very limited, cost associated. Accuracy and licensing can be of varying importance. An example would be looking for the closest free wi to the users current location. The pro le should give information about whether or not this dataset contains data about my location and how up to date it is.
Scenario B Looking for datasets to compare for relevance and quality to be integrated into an application. Time and cost can vary in importance. Here information about granularity and reusability (licensing, format) of the data can be more important. For example, if a user wants to evaluate the air quality in London in 1995 in comparison to today and the metadata mentions the years 2000 to 2005, a naive search for the year 2004 might not provide results. Scenario C Comparison of as many as possible available datasets on a topic for research purposes. Here, more time is available and cost is less of an issue. Archived data can be important; coverage and granularity are selection criteria. For example, a user may want to compare all datasets available on a speci c city to evaluate which areas to support nancially.
What is evident from these scenarios is that considering the context of the user and the objective of the task, search process and source selection may vary. Furthermore the presentation of results and the way di erent users might want to interact with those results is also dependent on these factors. The proposed approach in section 4 serves as an initial step to resolve the issues described in the above scenarios, by allowing user interaction in the process of source selection. 4
As we have discussed, structural and statistical pro les can enhance the discovery
of relations between datasets. Knowing the general structure of the dataset with
its data types and value ranges can be the rst step in narrowing down the
number of columns that can relate to each other. This can be used to create
actual links between unlinked datasets, e.g. by creating foreign keys between
them [
Other types of pro les can be used for linking datasets in a broader, less strict, sense. Topical and spatial summaries are descriptions of datasets on a semantic level. Similar topics, coverage or data patterns can indicate relationships between datasets that could inform search.
We propose to build a weighted graph based on the pro les of un-Linked data. By generating rich pro les for datasets and counting the number and strength of relations between them,correlations between datasets could be captured which can then improve the e ciency and e ectiveness of search within open data portals, such as data.gov.uk.
In accordance with Wagner et al. [
In the rst step, based on a query, keyword extraction can be used to generate ranked keywords which, as well as informing an initial search, are presented to the user in order of importance. The user can interact with the list of keywords, by swapping them with each other, to achieve more relevant results.
The relevance of the keywords to the user in uences the subset of datasets that appear within this graph - not all keywords are equally important to the user. Results that match with all keywords are of higher importance; datasets that match with only a few would be included if there were not enough results. Fig. 1(a) shows an example query about the population of London in 1995 and the datasets that matched the query the most. It also illustrates the strength of the connections between keywords and the pro les of the suggested datasets. This increases the end users awareness of the dataset content before opening it. As a second step, a graph of connections between related datasets is presented, which is based on one chosen dataset from the results in g 1(a). The weighting of connections between datasets are based on the similarity of the their pro les and the chosen datasets that matched the keywords from the original query, taking into account the users priorities. These initially identi ed datasets serve as recommendations and aid exploration of related information.
Fig. 1(b) presents results after the user chose the dataset D1 to be the most relevant for their information need. To improve the search process, other pro les which have strong connections to the pro le of D1 are presented along with information about the type of the connections between them.
The graph should present an adjustable number of results in a graphical
visualisation. The argument for the graph is to deepen the user's understanding
and the mental model of the search process. The availability of relevant data
depends on the topic as well as the speci city of the search. Hence the number,
the availability and the similarity of the datasets presented as results can vary
signi cantly. Data pro ling is naturally a user-oriented task [
Pro le features vary in their importance dependent on the users context and objective. When looking at the use scenarios, section 3, di erent factors can be important for users. Concerning the interaction with the system these concern time, cost, computational power, trust. Beyond that users have varying requirements on the type of information covered in metadata, depending on their task and context.
The approach described here would improve discoverability and aids all of the mentioned use scenarios as the user can be involved in the process of weighting the results. This can decrease the importance of the accuracy of ranking algorithms and enhance the quality of results according to ones individual needs. The possibility to detect similarities between pro les provides the basis for explorative search, engaging the user in source selection. To further develop this approach and validate its applicability we aim to develop pro les that support the identi cation of connections between datasets and topics, and between datasets and other datasets. Following from that we aim to develop interfaces, such as user-dependent result presentations, that support explorative data search. As the proposed graph is created on an abstract level and the connections are formed based on the datasets pro les the approach is not limited to a type of data, but can be used for Linked Data and un-Linked data at the same time; as the focus is to improve user experience.
Copyright held by the authors