Many data mining problems can be solved better if they are augmented with additional background knowledge. This paper discusses a framework of adding background knowledge from Linked Open Data to a given data mining problem in a fully automatic, unsupervised manner. It introduces the FeGeLOD framework and its latest implementation, the RapidMiner Linked Open Data extension. We show the use of the approach in di erent problem domains and discuss current research directions.
Data Mining is the process of identifying novel, valid, and interesting patterns
in data [
In the recent years, Linked Open Data [
There are two principle strategies for using Linked Open Data for data
mining:
1. Developing specialized mining methods for Linked Open Data. Examples
include operators for rule learning algorithms, e.g., DL-Learner [
This paper introduces a method that follows the second principle, i.e., a preprocessing strategy. The rationale is that such a strategy allows for re-using many existing data mining algorithms and tools, and is thus considered more versatile.
The rest of this paper is structured as follows. Section 2 introduces our theoretical framework, for which the current implementation is discussed in section 3. Section 4 discusses di erent example applications which use the framework introduced in this paper, i.e., text classi cation and interpreting statistics. We conclude with a review of current challenges in section 5, and a short summary. 2
To augment a dataset with Linked Open Data, we propose a general pipeline
comprising three steps:
1. First, entities in Linked Open Data have to be recognized that correspond
entities in the original dataset. For example, in a dataset about cities,
DBpedia1 [
We have implemented the essential steps of our theoretical framework as an extension to RapidMiner 3, an open data mining platform4 (except for feature selection, which is already covered by RapidMiner and other extensions).
For the entity linking step, di erent strategies are currently supported by our implementation: { Creating links based on custom URI patterns, e.g., appending the string value of an attribute containing city names to a constant, such as http://dbpedia.org/resource/.
rapidminer-lod-extension/
ISBN City 3-2347-3427-1 Darmstadt
{ Discovering links by full text search with SPARQL statements.
{ Using the DBpedia lookup service5, optionally with type restrictions, and
di erent disambiguation strategies.
{ Using the DBpedia Spotlight service6 [
With respect to feature generation, di erent generators are supported by our implementation: { Adding datatype properties as features. This generator includes heuristic guessing of appropriate attribute types (e.g., recognizing numerics and dates). { Adding direct types as boolean features. { Adding boolean or numeric features for incoming and outgoing relations. { Adding boolean or numeric features for incoming and outgoing relations plus their type, i.e., using quali ed relations.
{ Adding features using custom SPARQL queries.
Details on those generators can be found in [
Background knowledge from Linked Open Data can be helpful in di erent tasks. In the following, we show examples from di erent domains, which have been built using either the RapidMiner Linked Open Data extension or one of its predecessors.
4
In [
In order to further enrich the dataset, we have assigned classes to the events automatically. Since some of the events already have classes (since they are harvested from pages with a topical structure), we had a training and testing set for English language events available.
As features for the classi cation, we have used direct types and categories of the resources involved in an event. The rationale is that for example, sports events can be identi ed by athletes and/or stadiums being involved, while politics events can be identi ed by politicians being involed. Using only such binary features, we were able to achieve an accuracy of 80% for a problem comprising more ten di erent categories and a training and test set of 1,000 instances.
Furthermore, since we did not use any textual features, but only structured
content from DBpedia, we were able to apply a model trained on the English
dataset on event datasets in other languages as well. We have shown that the
classi cation accuracy is still the same when applying the model trained on the
English dataset to a set of events in German language [
A similar use case is the classi cation of social media texts, such as Tweets,
for example for the use in emergency management applications [
In that experiment, we had gathered training data using Tweets from a particular city. However, when using the trained model on data from another city, the classi cation accuracy dropped to 85%. The reason is that an over tting e ect occurs, e.g., the names of major streets are used as indicators for identifying Tweets about car accidents.
To avoid the over tting, we used features from DBpedia, rst preprocessing the Tweets with DBpedia spotlight, and then adding additional types and categories for the identi ed concepts, just like in the event classi cation experiment above. Using those abstract concepts (e.g., dbpedia-owl:Road instead of the name of a particular road) remedies the over tting e ect and keeps the accuracy up on the same level when applying a model learned on data from one city to data from a di erent one. 4.2
In most data mining scenarios, there is already some data available. However, there are also cases where the amount of data is scarce. One typical example are statistics, where usually only one or a few target variables are produced, e.g., conducting a survey on the quality of living in di erent cities, or gathering data on drug abuse in di erent countries.
In most cases, people working with these statistics, such as journalists, are
interested in nding reasons for the e ects reported in those statistics. In [
Having enriched the statistics at hand with background information, we
analyze the generated features for correlation with the target variables, as well as
perform rule learning to nd more complex patterns. Fig. 3 depicts a screenshot
of the tool. Further details and examples can be found in [
The examples above have shown how Linked open Data provides additional value in di erent data mining problems. However, there are also some challenges to be addressed. 5.1
Despite Linked Open Data being built on well-de ned standards, there are different ways to provide Linked Open Data. In its current version, the RapidMiner Linked Open Data extension exploits SPARQL endpoints. However, there are datasets which do not have SPARQL endpoints, but which could provide interesting background knowledge in many cases, e.g., Freebase7 or OpenCyc8. For such datasets, di erent implementation of the generation algorithms are required.
Furthermore, there are non-standard SPARQL constructs, such as COUNT
or the asterisk operator for computing the transitive closure [
certain features in a more performant way. However, it is di cult to determine
the features supported by a SPARQL endpoint automatically, in particular since
vocabularies such as VoID do not provide means to describe technical details of
SPARQL endpoints [
Besides technical variations, there are also di erent ways to represent data. For example, the current generator versions rely on data values being direct properties of the entities at hand. However, there are di erent cases, as this example from CORDIS9 shows: EU18931 a Funding .
EU18931 has-grant-value [ has-amount 1300000 .
has-unit-of-measure EUR .
]
A similar challenge exists for data using the data cube vocabulary [
While our approach is unsupervised in that the user does not need to know many details about the datasets to use, this does not hold for selecting the
datasets themselves. Since there are no universal means to nd datasets that hold information on certain entities, this is di cult to circumvent, nevertheless, it would be a desirable feature of a fully automatic approach.
A possible way to deal with this problem could be to start with one dataset,
e.g., DBpedia, for which entity recognition can be performed in high quality.
From those datasets, links to other datasets (directly via owl:sameAs, or using
link repositories such as sameas.org 10) can be followed to successively add further
datasets. In that case, SPARQL endpoints should be added automatically and
dynamically, which, given only a URI, has been shown to be a di cult issue [
Entity linking is the rst step in our pipeline. It should therefore work with high quality, since errors made at this step are carried over to later steps (e.g., extracting features from entities that do not correspond the entity meant in the original dataset, such as population gures for a di erent city).
While linking is not di cult for entities such as countries or major cities, it
can become more di cult for other classes, such as universities or animals, as
discussed in [
Most datasets in Linked Open Data come with at least light-weight explicit semantics, i.e., they use a vocabulary that contains semantic information in the form of ontology statements. These can provide valuable information to an approach for automatically enriching a dataset.
Consider the case where direct types of entities are added as boolean
features. Using the schema information, the features form a hierarchy, e.g., African
Island Island. This hierarchy can be exploited, e.g., for improving feature
selection, similar to the approach described in [
The approach discussed above proposes a pipeline of three strictly sequential steps. In particular, it rst generates the whole set of possible features, which are then ltered in a subsequent step. 10 http://www.sameas.org/
This kind of approach has certain limitations with respect to the amount
of features that can be generated. In particular, we have not included a
generator which generates features for all individuals linked to a resource for
reasons of scalability. However, in a later step, it would turn out that most of
those features are not useful, and they would be removed again in the next
step. An extreme case is the generator for quali ed relations, which, when
combined with a deep class hierarchy such as YAGO [
An even more complex strategy for feature generation we have not pursued so far is the construction of new features from those already generated, e.g., the number of cinemas per inhabitants, where both number of cinemas and population are features from the Linked Open Data set.
To remedy those problems and arrive at more scalable approaches, it would be necessary to develop algorithms that combine feature selection and creation in a joint process. A straight forward approach could be to create features for a sample of the data rst, determine the relevant features, and then create only those features for the rest of the dataset. Exploiting semantics, as discussed above, could lead to more sophisticated approaches. 5.6
Linked Open Data may be incomplete by design, i.e., following the open world
assumption. In many cases, we ignore the open world assumption for creating
data mining features. For example, a generator adding types to an instance adds
the value false if the type is not present { which is not in line with the semantics
of RDF using the open world assumption [
Some of these problems may be remedied by trying to detect and ll in
missing information in a preprocessing step. For example, in [
A more subtle problem is the presence of biases in datasets. In the statistics use case discussed above, we have frequently observed explanations such as The quality of living is high in cities where many music recordings were produced, in particular when using DBpedia as a source of background knowledge. This is mainly an e ect of a skewed distribution of data in DBpedia (and Wikipedia), which contains more information on popular culture in the western world, so the bottom line of this explanation is that cities in the western world have a high quality of living. However, such biases are di cult to detect automatically. 6
In this paper, we have introduced both a theoretical framework as well as an implementation for using Linked Open Data as background knowledge in data mining. The implementation uses the data mining toolkit RapidMiner, which allows for combining the Linked Open Data speci c operators with various other operators for data processing and mining.
We have shown di erent use cases where background knowledge from Linked Open Data can improve the results and provide new insights that could not have been gained from the mere data without background knowledge.
While Linked Open Data provides a lot of opportunities, there are also a number of challenges to address, ranging from coping with the large variety of data representations within Linked Open Data to dealing with complex features in a scalable way. With this paper, we have sketched a research agenda by enumerating some of the most prevalent challenges. We are con dent that this research agenda will lead to a set of approaches and tools that allow for novel data mining tools which have access to a large amount of knowledge and provide deeper insights into data.
In the past two years, several people have contributed to the conception and/or implementation of our framework and the RapidMiner Linked Open Data extension. The author would like to thank Johannes Furnkranz, Raad Bahmani, Alexander Gabriel, and Simon Holthausen for their valuable contributions. The current team developing the extension includes Christian Bizer, Petar Ristoski, and Evgeny Mitichkin.
The author would further like to thank all the people developing the example applications discussed in this paper, including Daniel Hienert and Dennis Wegener at GESIS, and Axel Schulz at SAP Research.