In this paper we introduce ODMiner, an automatic tool that enhances open datasets provided in heterogenous structured formats (e.g. JSON, CSV, XML, etc.) to Linked Open Data. ODMiner mines OD by recognising well known data types and formats (e.g., dates, emails, currencies, etc.) and by exploiting well known open linked datasets and vocabularies (e.g. DBpedia, WordNet, etc.) in order to extract named entities and relations between the open dataset elements. ODMiner is designed as modular and extensible software architecture and its process can be customised in order to address specific needs of final data representation and modelling. Finally, an evaluation of ODMiner with heterogenous multi-language OD datasets is provided in order to give evidence of its practical effectiveness.
Over the last years, the volume of Open Data (OD) published on the Web has grown hugely, raising the joining interest of public institutions, private companies and citizens. Unfortunately, OD consumers have still to face a variety of challenges when trying to access, understand or use OD in order to develop innovative services for solving real world problems on top of them. In particular, a manual effort is still required for analysing and mining open datasets. For example, in most of the cases it is complex to understand their content and context, to extract information and transform it into an machine-understandable format, to filter the elements of interest, etc. This limits the exploitation of OD, posing serious threats to their real reuse. In this paper we introduce ODMiner, an automatic tool that enhances open datasets provided in heterogenous structured formats (e.g. JSON, CSV, XML, etc.) to Linked Open Data. ODMiner mines OD by recognising well known data types and formats (e.g., dates, emails, currencies, etc.) and by exploiting well known open linked datasets and vocabularies (e.g. DBpedia, WordNet, etc.) in order to extract named entities and relations between the open dataset elements.
We mention only two of the possible applications of these results. First of all, the linking of OD datasets to popular open linked datasets and vocabularies can be used to automatize the process of OD triplification and semantic enrichment, and their consequent use in Semantic Web-based applications. Another useful application we envision is using this information for automatic OD visualization. By leveraging rich OD description we could be able to automatically generate, for instance, visual analytics tools and panels that can be used by users to explore, analyse and make sense of OD datasets.
ODMiner is designed as modular and extensible software architecture and its process can be customised in order to address specific needs of final data representation and modelling. Finally, an evaluation of ODMiner with heterogenous multi-language OD datasets is provided in order to give evidence of its practical effectiveness. The structure of the paper is the following: (i) Section 2 reports the related work; (ii) Section 3 describe our system, i.e. ODMiner; (iii) Section 4 describes the evaluation we carried on to assess the effectiveness of ODMiner in the task of data type recognition; (iv) finally, Section 5 provides the conclusions and possible future work.
In recent years a lot of research has been carried on to populate the
Semantic Web with Linked Open Data. Most of the approaches rely on fixed
or mostly limited customisable solutions to convert structured sources to
Linked Open Data. Worth mentioning tools are the D2R Server [
On the other hand, mining meaningful information from structured as
well as semi-structured data sources (e.g. CSV, XML or JSON) remains a
challenging task. In addition to OD quality [
Another approach is using a tool such as Wrangler [
To the best of our knowledge ODMiner is a novel and extensible solution to support the generation of Linked Open Data. It provides an HTTP REST interface that allows to automatically mining the data types from Open Data available in heterogeneous formats such as XML, JSON and CSV in a nearly one-click process.
ODMiner4 is a web-based tool aimed at analysing and making sense of heterogeneous OD datasets. In particular, ODMiner takes as input the URL of an OD dataset (JSON, XML, CSV and TSV are the data formats currently supported), analyses its content, and infers which kind of data each field of the dataset is an instance of. For example, given the tabular dataset in Figure 1, ODMiner map each column to a meaningful class (the red text at the top of the table) of a well-known ontology.
ODMiner has a modular and extensible architecture. It provides some software modules, each of which is responsible of recognising if the data fields are instances of a specific class. The list of some of the basic classes that are identified by ODMiner is shown in Table 1, and has been derived by the analysis of the set of italian datasets described in Table 2. Other software modules can be implemented and loaded into the ODMiner engine, extending its analysis capabilities (e.g. extending the set of managed ontologies to which input data can be mapped).
ODMiner main engine is responsible of coordinating the analyses
performed by its software modules. The overall process can be summarised as
4 http://eelst.cs.unibo.it:8099/detector/api/url
follows. First, ODMiner splits the dataset fields into homogeneous partitions.
For example, the fields of tabular datasets are grouped by columns. Then,
each module analyses the fields of each group separately. The kind of
analysis differs from module to module: for example, the module responsible of
recognising dates performs a simple pattern matching test, while a more
sophisticated one such the place analyser computes Levenshtein distance [
ODMiner also provides users a mean to configure the analysis modules to define custom data analysers. For example, a user may be interested in recognising some specific types of time-related data (e.g. dates, times, durations, intervals, etc.). To do so, the user has to invoke ODMiner providing a configuration file5 defining that he/she wants include an analysis based on a regular expression search, together with the regular expression to test and the ontological class that should be associated to the matching data.
Another configurable ODMiner module allows to identify data values that are instances of specific DBpedia classes. The basic idea is that ODMiner compares each field of the dataset (e.g. the cells of a column in a table) to the labels associated to DBpedia entities. When a match is found, ODMiner collects all the classes those entities are instances of. Each match is considered by ODMiner as a vote for that class. At the end of the analysis, all the classes that have not obtained a minimum number of votes (i.e. a config5 The description of the format of the configuration, together with all the other information needed to download, compile and run ODMiner, is available at the address https://bitbucket.org/Cigna/semanticdetector urable quorum threshold set by the user) are discarded. Among the remaining, the majority principle is applied, and the class with the higher number of matches is chosen. The DBpedia classes users are interested in can be specified by users, together with the number of records to analyse and the confidence threshold (i.e. the quorum). As discussed in Section 4, this principle has proven to be effective for the datasets and the classes taken into account in our evaluation.
We designed an experiment to assess the capability of ODMiner to enhance Open Datasets by effectively recognising the correct types associated with data. We randomly selected 16 datasets from Italian Open Data archives6 and from some US data portals. Table 2 reports the full list of datasets used for the evaluation of ODMiner. Each row of the table identifies an open dataset and specifies the topic of the dataset and the URL that can be used to retrieve the dataset. The datasets were selected according to the following rationale: (i) homogeneous distribution of heterogeneous formats (e.g., CSV, XML, etc.); (ii) multilinguality of datasets, i.e. we did not focus on a single language (e.g. Italian); (iii) not negligible size of source data; (iv) heterogeneity of topics.
LPPPSAAETCCDDDNMMoiluhebbmohrroubauu.rupabssncirnrssplvWeergltistdaiitiliocmassccescsocrhrtthcarieyaooshoetyelecssspeaibeusinarlleloacsniitoflsccfnhhotiovveehdoRrifpennmdososElrsseluemsoaseemrrseftddseviiRpooliocpicnrfoaforeomdonoCsRgoftfeareoorcatlsamftihmsfocteoaihtrorogseeDnn,rnneasiCaaraplocios(tttlfiy.lootmIoafrolsflleIfii,lanRnlCeinotenhtihocoasicf.aidsabNmgielioinwtaistYtioorarnktiovSfefitacrteeegoifonC)alifornia htthptsthh:hp/hthhth/t:ththdtttt/thpphht/ttptttap8tspt:ptpttstt/ptp:hts5:ss/ap:/pp/g::://t:.:/.:////:h1//t:gca////d////p/gdge8/cic/abgagtg:aaaahh.bry/ta1aa/btlhbihahorbgtaeb7brtir.shsfra.leir3ticri..iel.eipeibddllil.dhee.eleil1l:raacinealiel/.ln1eicei/tcte.to.eg8aaoac.7ca.i.iclcgn5c.i.igise/.cgicsginoin.a.go.gg.aag1ncnpyga..nn..nw.o8ioaae.ggoggaawavr...nevooo1w.nw.g/.wdebw/vavwv7a/aeeba.//ap/e.3eecpaababwt.pibb.sbcppiap/..1i/cv.e.c.s./i/ivicuc1cs/iE/vb/s.isevvvusn.7s.ie.lu.uieiw.ic.nei/uweeueubnmwnsnisnwwwnnbos.ii/csubaibi/5.iosssbob/2ibpnoo//c./vtoBojegiio.pi/.jhwtzi.bdig.xz.a/tietid3idtoafx/r8t/nt//dvxd-/-./Batu-ddwd9id--iadatt-gei/aeaaavi/tSbnftbr/ttdiMatiwtfflakc/ios/ifiaiar/dp//gd9okaypuumdtsroet//////ilsmorrrrrrrr/teaoaiwactnooooooccnpacosewwwwwwgahhtilcrieesslleoiissssssUii................xxxxxxccccccccccRmmmmmmssssssssssLvvvvvvvvvvllllll 4881221139273214S40830292402425iz44077e1(KBs) ForXXXXXXCCCCCCCCCCmMMMMMMSSSSSSSSSSaVVVVVVVVVVLLLLLLt Table 2. Topics, source URLs, sizes and formats of the open datasets used for the evaluation of ODMiner.
We constructed a ground truth by manually annotating the correct data type for each field (e.g. a column of a CSV table) of the datasets presented in Table 2. The ground truth is available on-line as CSV for consultation7. Hence, we used ODMiner to mine data types against the same datasets in order to compare the output of ODMiner with respect to the ground truth in 6 http://www.dati.gov.it/ and http://85.18.173.117/ 7 http://eelst.cs.unibo.it/ld4ie/goldenstandard terms of precision, recall and F-measure. We set up ODMiner with the configuration provided in Table 3. This configuration was provided to ODMiner as a JSON file8 and allowed us to specify the confidence threshold and the matching criteria for data type recognition. The confidence thresholds reported in Table 3 are expressed in a range between 0 and 1, and have been selected through an empirical analysis, trying to find a good trade-off between the accuracy of the results and the analysis duration. The meaning associated with matching criteria is the following: (i) RegEx is a regular expression that is used to perform the matching; (ii) pattern matching identifies a set of cases or patterns from a list that are used to test whether a set of data matches those cases; (iii) Exact and Contain-based matches on DBpedia allows ODMiner to perform the classic “exact match” and “contains” operations on the values (either literals or URIs) coming from a Linked Dataset (e.g. DBpedia), which is used as background knowledge.
Finally, we ran the experiment by configuring ODMiner to recognise data types by using only a small slice (i.e. 20%) of the whole dataset. The resources part of the slice were picked up randomly. Figure 2 shows the final results we recorded.
Although ODMiner requires a further and more detailed evaluation that should be performed on a larger set of OD datasets and a broader set of classes to recognise, the preliminary results of these tests are encouraging and show that ODMiner has a good accuracy in performing the linking task.
In this paper we present ODMiner, a novel tool to support the process of generating of Linked Open Data from Open Data format such as XML, JSON and CSV. It provides a HTTP REST interface and it is easy to extend by adding new algorithms and supported formats. The evaluation that we carried on demonstrates the effectiveness of our tool, though further investigation is needed. Possible future work includes the extension of ODMiner in order to 8 This JSON file is available on-line at http://eelst.cs.unibo.it/ld4ie/config.json. dbpedia:Person odm:TaxCode sioc:Site odm:Gender foaf:mbox support more formats like XSLX or OpenOffice spreadsheets and the study of solutions to mine meaningful information that are distributed across multiple fields. For example, the latter is the case of dates represented as multicolumn values in a CSV having a column for the day, another for the month and another for the year.