Assigning a topic or a domain to a vocabulary in a catalog is not always a trivial task. Fortunately, ontology experts can use their previous experience to easily achieve this task. In the case of Linked Open Vocabularies (LOV), a few number of curators (only 4 people) and the high number of submissions lead to nd automatic solutions to suggest to curators a domain in which to attach a newly submitted vocabulary. This paper proposes a machine learning approach to automatically classify new submitted vocabularies into LOV using statistical models which take any texts description found in a vocabulary. The results show that the Support Vector Machine (SVM) model gives the best micro F1-score of 0:36. An evaluation with twelve vocabularies used for testing the classi er shades light for a possible integration of the results to assist curators in assigning domains to vocabularies in the future.
Linked Open Data (LOD) refers to the ecosystem of all the open source
structured data which follows the standard web technologies such as RDF, URIs and
HTTP. As the number of available data grows with time, new datasets following
these principles appear. Linked Open Vocabulary (LOV) 1 is an initiative which
aims to reference all the available vocabularies published on the Web
following best practices guided by the FAIR (Findable - Accessible - Interoperable
Reproducible) principles. Each vocabulary can be seen as a knowledge graph,
describing the properties and the purpose of the vocabulary, and which can be
connected to other vocabularies by di erent types of links. Therefore, LOV can
be seen as a knowledge graph of interlinked vocabularies [
When a new ontology is submitted for integration into LOV, a curator needs to assign at least one tag representing a domain or a category to the vocabulary Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
among existing 43 categories, such as \Environment", \Music" or \W3C REC". A category aims at grouping ontologies according to a domain. For example, the tag \W3C REC" represents ontologies recommended by the W3 Consortium, such as rdf or owl. As the number of domains increases and some vocabularies2 can be relatively small, the tagging process can be biased. Figure. 1 depicts the list of the tags available in LOV as the time of writing this paper, while Figure 2 depicts their distribution. One of the bene t of assigning a tag to a vocabulary is to index it according to a domain and make it easy to access from the interface. For example, to access to vocabularies in the IOT domain, the direct URL in LOV is https://lov.linkeddata.es/dataset/lov/vocabs? tag=IoT. Additionally, any newly added vocabulary should belong to at least one domain.
We propose a machine learning approach to automatically classify newly submitted vocabularies with statistical models which take texts describing the subjects of the vocabularies as input. Indeed, the majority of the graphs contains a lot of text describing the subjects and the properties of the vocabularies, in the form of string literals. For example, the URI in a given ontology (Class or Property) is often described by the predicate rdfs:comment with a text mentioning the comment of a given resource. Other predicates are often linked to texts containing information, such as rdfs:label or dct:description. We used all this text information to train several machine learning models in the purpose of classifying the vocabularies into di erent categories. This paper is structured as follows: Section 2 describes related work in graph classi cation, followed by the 2 In this paper, the terms ontology and vocabulary are interchangeable machine learning approach to build the classi er in Section 3. Section 5 provides an evaluation of our approach and a brief conclusion in Section 6 2
Graph classi cation is a problem well studied in the literature. Several
strategies have been developed to tackle this problem such as kernel methods or more
recently graph neural networks [
Classifying datasets created using semantic technologies has been applied
in the literature. The most closest work in the literature is described in [
While the mentioned approach uses supervised learning, we apply two more steps in preparing the corpus for input of the classi er, using Bag-of-Word and a Truncated SVD transformation. Additionally, we have a very small amount of corpus inherent to the size of vocabularies compared to the entire LOD datasets, and a higher number of available tags (43 in LOV compared to 8 for the LOD cloud). 3 3.1
Our approach has been to use the texts contained in the vocabularies to classify them into categories. Indeed, usually the subject of a RDF graph and the purpose of its entities are described in string literals following some speci c predicates. We rst extract this relevant textual information (string or literal) inside each graph (a dump representing the latest version of the vocabulary in N3), and concatenate it into one paragraph describing their subjects. To this end, we rst download each recent version of the vocabulary from LOV SPARQL endpoint (taking the most recent version tracked by LOV) and import them into graph objects with RDFLib3. Listing 1.1 depicts the SPARQL query used to retrieve the latest version of each vocabulary, alongside with their domains and unique pre x.
SELECT DISTINCT ? v o c a b P r e f i x ? domain ? v e r s i o n U r i f GRAPH <h t t p s : / / l o v . l i n k e d d a t a . e s / d a t a s e t / lov >f ? vocab a v o a f : Vocabulary . ? vocab vann : p r e f e r r e d N a m e s p a c e P r e f i x ? v o c a b P r e f i x .
? vocab dcterms : m o d i f i e d ? m o d i f i e d . ? vocab dcat : keyword ? domain . ? vocab dcat : d i s t r i b u t i o n ? v e r s i o n U r i .
BIND ( STRAFTER(STR( ? v e r s i o n U r i ) , "/ v e r s i o n s /" ) as ?v ) BIND(STRBEFORE(STR( ? v ) , " . " ) as ? v1 ) BIND (STR( ? m o d i f i e d ) as ? date )
FILTER ( ? date = ? v1 ) gg GROUP BY ? v o c a b P r e f i x ? domain ? v e r s i o n U r i ORDER BY ? v o c a b P r e f i x ? domain ? v e r s i o n U r i
Listing 1.1: SPARQL query to retrieve the latest versions of vocabularies stored in LOV
We then concatenate all the strings followed by the predicates having one of these su xes : comment, description, label and definition. The predicate rdfs:label is often used to give a name of an URI in natural language, while the su xes comment, description and definition are used to give insight on the meaning and purpose of a given ontology or entity. The result of this step has been the generation of a paragraph for each vocabulary. As the texts describe the
RDF properties of the graphs, they often contain the su xes of these properties formed of several words not separated by spaces, in camel case format. For example, if an extracted text mentions the property \UnitPriceSpeci cation", this expression will remains as a single unit in the nal text. However, it can imply a bias on the statistical model to be applied on this data. Consequently, we separate all these types of expression with spaces, when a uppercase occurs in the middle of a word. Therefore, by using this method, the expression \UnitPriceSpeci cation" will be transformed to "Unit Price Speci cation" in the nal text. After this transformation, the whole corpus' vocabulary is formed of 21; 435 different words. The mean word count for the paragraphs is 1168:5, the maximum is 86208 and the minimum 0. Two paragraphs were empty and 25 of them have less than 20 words. The text describing the rooms vocabulary 4 obtained with the pre-processing step described in this section is presented in Listing 1.2. This ontology describes the rooms one can nd in a building and has the following assigned tags in LOV: Geography and Environment.
Floor Section. Contains. Desk. Building. Floor. A space inside a structure, typically separated from the outside by exterior walls and from other rooms in the same structure by internal walls. A humanmade structure used for sheltering or continuous occupancy. Site. A simple vocabulary for describing the rooms in a building. An agent that generally occupies the physical area of the subject resource. Having this property implies being a spatial object. Being the object of this property implies being an agent. Intended for use with buildings, rooms, desks, etc. Room. The object resource is physically and spatially contained in the subject resource. Being the subject or object of this property implies being a spatial object. Intended for use in the context of buildings, rooms, etc. A table used in a work or o ce setting, typically for reading, writing, or computer use. A named part of a oor of a building. Typically used to denote several rooms that are grouped together based on spatial arrangement or use. A level part of a building that has a permanent roof. A storey of a building. Occupant. An area of land with a designated purpose, such as a university Campus, a housing estate, or a building site.
Listing 1.2: Paragraph describing the rooms vocabulary, obtained with the preprocessing pipeline described in Section 3.
3.2
As we cannot feed directly text paragraphs to the machine learning models, we
applied a processing pipeline for transforming the texts into xed-size vectors
of attributes. For this purpose, we used several techniques described in [
Fig. 3: Schematic view of the processing pipeline. From left to right, the diagram depicts the di erent steps: 1-Text extraction from Vocabulary dump; 2-BoW Transformation; 3-Normalization with TF-IDF; 4-Vector dimension reduction and nally the classi ers.
We then separated the data in two subsets composing of a training set (80% of the vocabularies) and a test set (the remaining 20%). In this paper, the dataset version of LOV used for the experiment is the snapshot as of May 7th, 2019 5, containing 666 vocabularies. We claim that the approach described in this paper can be replicated to any type of machine learning multi-label task with a knowledge graph as input.
As each vocabulary can have one to many tags, we tackle the problem as
a multi-label classi cation task. A machine learning model is trained on the
training set, trying to nd relation between the attributes describing the graphs
and their labels. The trained model is then applied to the test set. The predicted
labels are nally compared to the ones tagged by human curators, and the micro
precision, recall and f1-measure are computed, which are current supervised
learning metrics [
However, we had to apply a One-vs-Rest strategy for the SVM [
The results of the classi cation for the 4 machine learning models, using k = 50; 150; 300 for the truncated SVD are presented in Table 1. The MLP and the SVM give the best micro F1-score respectively of 0.34 and 0.36, with n = 150. In this section, we describe the evaluation of the classi er on newly submitted ontologies in LOV, and we discuss the results obtained comparing with manual assignment by two curators. 6 The ReLU is the most used activation function in neural network. f(z) is zero when z is less than zero and f(z) is equal to z when z is above or equal to zero.
5.1
For evaluating our model, we took a list of 12 vocabularies in the back-end of LOV and asked two curators to assign domains to each of the vocabulary. Then, we passed the same vocabularies to the SVM classi er. The classi er's results is then compared with the human assignment tags as presented in Table 2.
As the main goal of the system is to suggest recommendation to a curator, we compute a soft accuracy metric, corresponding to the number of graph with at least one match between one of the curator tags and the classi er suggestions, divided by the total number of tested vocabularies.
For a vocabulary i, its associated tags yi = fyi1; yi2; :::; yilg and the prediction of the classi er yipred = nyip1red; yip2red; :::; yim predo, we say that the classi er is softly accurate for the vocabulary i if 9yipkred 2 yipred such that yipkred 2 yi. The soft accuracy is then computed by the ratio of the number p of outputs softly accurate on the total number n of inputs. We get a result of 0:33 for this evaluation. 5.2
The results seem average regarding the precision in the detection from the classi er, compared to the curator. Their could be several explanations, like the disparity between the tags in the dataset (13 labels are used in less than 10 vocabularies), or the di erence of subjects in vocabularies tagged by the same label. For example, the "geography" tag is used for the rooms and the Postcode 8 ontologies, whereas they both describe completely di erent things, thus we can expect di erent words usage and very di erent feature vectors.
Furthermore, multi-label classi cation for tagging recommendation is a hard
task, especially when the number of possible tag is high (43) and the number of
examples is low (666) [
This paper addresses one main issue: build and evaluate a classi er based on the content of LOV catalog using machine learning technique. The nal goal of this work is to help the human curator of vocabularies to have a list of recommendations for a new ontology submitted in the back-end. The classi er implemented gives a micro F1-score of 36%. Although this score seems low, the system will not be used without a human that validates or not the suggested tag. We do not intend to compare the system with the human curator. Instead, we want to have a system that reduce possible risk of bias when assigning domains to vocabularies and suggest tags to the curator. Future work includes ingesting the feedback from the curators into the classi er to learn from newly added vocabularies for a continuous learning work ow, and test deep learning models with a transfer learning strategy to overcome the low-frequency of training examples.
Indeed, deep learning approach can perform well on multi-label classi cation,
but it needs a lot of training examples [