Ontology-based data access (OBDA) has gained attention in recent years for providing access to large volumes of data by using ontologies as a conceptual layer and exploring their ability to describe domains and deal with data incompleteness. This is done through mappings that connect the data in the database to the vocabulary of the ontology. The first OBDA studies were about data stored in relational databases. Recent studies have begun to extend the use of OBDA to NoSQL databases. In this paper, we present a novel approach for OBDA with document-oriented NoSQL databases. Differently from related works, our approach uses an access interface with an intermediate conceptual layer that is extensible and flexible, capable of providing access to different types of database management systems. To validate the approach, we have implemented a prototype for MongoDB, using a real-world application domain as a case study.
There are scenarios where the amount of available data grows exponentially,
the variety of data formats is large, and there’s a need to store and analyze
semistructured or even unstructured data, and relational databases are not the best
solution for storage and querying in these scenarios
There are four main groups of NoSQL Database Management System
(DBMS): document-oriented, column-oriented, graph-oriented and key-value stores.
As better explained in
With the increasing popularity of NoSQL DBMSs, there is growing interest in applying OBDA to these types of systems. But unlike relational DBMSs, there are neither standard query languages nor data models for NoSQL DBMSs. In this work, we present an approach for the use of OBDA with NoSQL Systems, so that it is possible to apply ODBA in domains with large volumes of data. We also aim to simplify the mapping and the process of converting SPARQL queries into queries for different types of NoSQL systems. To do this, we propose an intermediate conceptual layer that represents the models saved in the database and an extensible method of translating the queries to the target DBMSs. To evaluate its viability, we apply this approach in an ODBA that uses a document-oriented NoSQL database in MongoDB.
Botoeva et al.
In another interesting work, Michel et al.
query. As a result, they’ve shown that it is always possible to rewrite a query that produces correct results.
In this section, we talk about our approach to the problem of using ODBA to access NoSQL databases.
We propose the utilization of an OBDA model with an Access Interface comprised of an ontology, a mapping and an intermediate conceptual layer capable of providing data access to a NoSQL DBMS. Our approach aims to simplify the construction of a mapping to make it more flexible and generic, allowing its reuse independently of the DBMS selected for the persistence layer. In order to do that, we define an intermediate conceptual layer using classes in an object-oriented programming (OOP) language to represent the structure (schema) of the data persisted on the database.
On the one hand, this approach simplifies the construction of a mapping between ontology and OOP classes, since an object-oriented representation can be very close to the ontological domain representation. Knowledge representation in an ontology can be defined via classes, relations, hierarchy and inheritance, ideas very similar to the concepts behind an object-oriented language implementation. On the other hand, the task of accessing the data persisted in databases and mapping in objects can be delegated to ORM (Object-relational Mapping) or ODM (Object Document Mapper) libraries, very frequently used in industry. These libraries can access several types of data stored in different DBMS, such as relational databases or document-oriented databases, and turn them into object collections.
In the mapping proposed here, each class (or attribute) of the ontology must be associated with its respective class (or attribute) in the OOP model. In turn, the OOP model is associated with the database schema by means of an ORM or ODM. Once this mapping is defined, the Access Interface is able to receive a SPARQL query and perform the data recovery. From the SPARQL query, a graph describing the data which will be accessed by the query in the database is generated. The nodes in the graph represent the classes, while the edges denote the relations between the classes. Each node contains the list of attributes of the correspondent class required to answer the SPARQL query.
The description of all data that must be recovered from the database in order to answer a SPARQL query is codified in the mapping and in the graph. In this way, the data can be recovered by means of data access operations available in the ORM/ODM library or by internal mechanisms which generate queries for the employed DBMS.
A first implementation of this method – a project called onto-mongo – targeted document-oriented NoSQL DBMS; it is described in Section 3.2.
The onto-mongo2 project is a working prototype that applies the query translation method detailed in Section 3.1, generating a set of RDF triples as a result that is then used to populate an OWL ontology. As illustrated in Figure 1, the onto-mongo project has the following main components: an OWL Ontology, a NoSQL database, an Access Interface, an Ontology to Database collections mapping, a SPARQL to NoSQL query translator, and a RDF export.
As an example, an ontology called basic-lattes (Figure 2), about researchers and their publications, was created with the ontological classes Researcher and Publication. The ontology is a client to our application, and it is used to answer some competency questions related to the domain, which can be expressed as SPARQL queries.
The onto-mongo project uses a document-oriented NoSQL database as data
storage. As an example of application, we have used data about researchers’ scientific
production, extracted from the Lattes Platform3 using a tool called scriptLattes
have stored about a hundred of these documents into a document-oriented NoSQL database maintained in the MongoDB DBMS.
The Access Interface is a module capable of transforming a message containing a SPARQL query and answering the message with a set of RDF triples extracted from the database. This interface was developed using ruby4, an object-oriented language, while the ruby on rails5 web framework was used to create a service application to receive the messages.
The Access Interface has classes written in ruby which represent the structure of the database-stored collections. We call them model classes. Our example has two main model classes: Researcher and Publication. The former models data from researchers, and the latter models data from scientific publications.
It is possible to define all of the instance attributes which are persisted in the database by adding annotations to the model classes (i.e.: name or country). It’s also possible to define the relations between classes, indicating for example that an instance of the model Researcher might have a list of his or her publications that is also saved in the database. And, on the other side, we can indicate that a Publication must be related to an instance of Researcher.
Model classes are used to manipulate persistent objects. They interact with the database by means of CRUD (create, read, update, delete) methods. When a NoSQL database is used to persist the objects, the model classes can be seen as a conceptual schema for the database. For this reason, the model classes facilitate the ontology mapping phase.
3The Lattes Platform is a system maintained by the Brazilian Government to manage information on science, technology, and innovation related to researchers working in Brazil.
3.2.4. Mapping The mapping links the ontology classes to the Access Interface classes so that it is possible to query information related to these entities in the database. Using this approach, an ontology class is mapped to a ruby class that represents similar concepts. The relationships between classes of the ontology are also mapped to relations between classes of the object-oriented language.
The mapping is done through the creation of methods that indicates the equivalences that exist between the ontology and the model classes. Thus, it became possible to map the structure present in the ontology to the structure of the data collections present in the database, passing through the classes of the Access Interface. The mapping can be defined in a script placed in the initialize folder of the rails project, which will be executed when the program start up. In this script, the user can define the mapping for each class of the Ontology, telling which class model represents the class, which attributes of this model are equivalent to the attributes of the ontology class, and which relations between the collections in the document-oriented database are equivalent to the relations between the ontology classes. In the mapping, the user can also indicate the path of the OWL file which contains the definition of the ontology.
As an example, consider the following mapping for the ontology class Researcher:
Listing 1. Mapping of Researcher class to Researcher model 1 OntoMap . mapping ’ P e s q u i s a d o r ’ do 2 model R e s e a r c h e r 3 maps from : ’ nome ’ , to : : name 4 maps from : ’ p a i s ’ , to : : country 5 maps from : ’ n o m e e m c i t a c o e s ’ , to : : n a m e i n c i t a t i o n s 6 maps r e l a t i o n : ’ p u b l i c o u ’ , to : : p u b l i c a t i o n s 7 end
In line 1, we call the method mapping of the OntoMap module, passing as parameter the class Researcher present in the ontology, and a block with some definitions. In line 2, we say that the model Researcher present in the interface is equivalent to the Researcher class. In lines 3 to 5, we map some attributes of the model to attributes of the class in the ontology. The maps method receives two parameters: from, which refers to some relation or property belonging to an ontology class and to, that indicates which attribute of the class Researcher or relation with another class is equivalent. In line 3, we define a mapping between the property name and the attribute name. In line 6, the relation published that exists between the classes Researcher and Publication is mapped to the relation publications of the model, which represents the set of publications of a given researcher. It is important to note that an instance of the class Researcher may have one or more publications. Similarly, the mapping between the class Publication and the model Publication can be made as follows:
Listing 2. Mapping publication class to Publication model 1 OntoMap . mapping ’ P u b l i c a c a o ’ do
The presented mapping allowed the translation of SPARQL queries into MongoDB queries without the need for a direct mapping between the ontology structure and the database document collections. In other related works that addressed OBDA for relational DBMS, this type of mapping is done directly between SQL queries and ontology classes. Our approach thus has a conceptual layer serving as a link between ontology and data collections, and this conceptual layer (model) is described in an object-oriented language, following a software architectural pattern widely used. One of the main advantages of this approach is the possibility of reusing the conceptual layer for other types of DBMS. With no efforts, we can use this layer to persist and access data in relational databases.
The translator uses the mapping to transform SPARQL queries into queries to the DBMS. The SPARQL query is first transformed into a SPARQL Syntax Expression (S-Expression) through the sxp library 6, as this facilitates translation. The query is then transformed into a graph representing the RDF triples, filters, groupings, and variables that must be returned for this query. The translator has two different strategies: the first one translates the query using the data recovery methods from the model classes, and the other generates a MongoDB query in JSON, with aggregations and filters, which is executed by the DBMS itself. The first strategy is simpler because it only transforms the query into calls to methods of the model classes along with Mongoid filtering methods, such as the WHERE method, which receives a set of filters that must be applied to a list of objects. The retrieved data objects can also be processed with the filtering and map/reduce functions of the ruby collection libraries. The second strategy is the most appropriate for complex queries on large databases, which have several types of aggregations, or filtering for internal or related collections. With this strategy, these operations are executed by the database management system, which provides a better performance.
Currently, the translator only supports SPARQL queries in which the predicate of the RDF triples in the WHERE clauses are relations or attributes defined in the ontology and equality filters. The query in Listing 3 is an example of query supported by the translator.
The sets of objects extracted from the database are transformed into RDF triples and sent back to the initial caller as the answer. We are also able to insert these triples into the ontology as a way to populate it. Afterward, this information can be queried or analyzed directly without any prior knowledge about the structure of the database collections or the type of NoSQL system that is being used.
A case study was created to evaluate the feasibility of our approach, and we will
discuss it as an example to the reader. By starting with a query very similar to
the one described by Botoeva et al.
The NoSQL database contains about a hundred CVs extracted from the Lattes Platform. We also added some information about the researcher Kristen Nygaard and two of his publications that have the same year so that our example would be easier to illustrate. This was added to the researchers collection: • ”COOL (comprehensive object-oriented learning)”, 2002. • ”Class and Subclass Declarations”, 2002. • ”Classification of actions and inheritance also for methods”, 1987.
Using the same ontology discussed at section 3.2.1, we need to answer the following question: which researchers published two different publications in the same year? We can ask onto-mongo this question expressed in the following SPARQL query:
Listing 3. Example SPARQL query PREFIX : <http : / / onto−mongo/ b a s i c −l a t t e s /#> SELECT
? nomePesquisador ? t i t u l o P u b l i c a c a o 1 ? t i t u l o P u b l i c a c a o 2 ? ano WHERE { } ? p e s q u i s a d o r : nome ? nomePesquisador . ? p e s q u i s a d o r : p u b l i c o u ? p u b l i c a t i o n 1 . ? p e s q u i s a d o r : p u b l i c o u ? p u b l i c a t i o n 2 . ? p u b l i c a c a o 1 : a n o p u b l i c a c a o ? ano . ? p u b l i c a c a o 2 : a n o p u b l i c a c a o ? ano . ? p u b l i c a c a o 1 : t i t u l o ? t i t u l o P u b l i c a c a o 1 . ? p u b l i c a c a o 2 : t i t u l o ? t i t u l o P u b l i c a c a o 2 .
FILTER
(? p u b l i c a c a o 1 != ? p u b l i c a c a o 2 )
First, we transform this query into an equivalent SPARQL Syntax Expression, using the sxp library:
Listing 4. SPARQL query transformed into a S-Expression 1 ( p r o j e c t
The expression is divided into its components so that they can be translated to a database query. The algorithm divides the query into the following components: output variables (outputs), filters that must be applied (f ilters), the query body (body) and its attribute triples (attributes) and relations (relations). We will discuss these components in the next section by their order of evaluation.
From the SPARQL query, we construct the graph of Figure 4. In line 6 of this query example, the first triple is read and the first node of the graph of Figure 4 is generated containing the attribute :nome of class Pesquisador stating that the name of the researcher must be retrieved in the search that will be executed in MongoDB. From the mapping, we know that the field : nome is related to the field name present in Researcher, that is, it is just an attribute. In MongoDB, we say that this field should be projected (projection). Project is a command that returns JSON fields in the resulting document. It is similar to columns returned in a SELECT clause in SQL. Then, in lines 7 and 8 we create two nodes that are adjacent to the first node, referring to class Publica¸c˜ao, because there is a relation publicou. We know from the mapping that publicou is a relation between researcher and publication and publication is a list. For this reason, we generate an unwind command of publicacao1 and another of publicacao2. The unwind command output a document for each element of a list field. Because MongoDB does not have the JOIN operation, it is necessary to transform a list or array present within the document into a set of other objects for each element of that list using unwind. In the case of our example, the combined effect of the two unwind is equivalent to a SQL cross join between publicacao1 and publicacao2.
Since publication is a document that is inside the document researcher, we add the prefix so its attributes year and title can be found. Therefore, we make the projection of these attributes with its prefix(e.g.: ?publicacao1.year). Once the projections and unwinds are made, we then analyze the filters.
In lines 3 and 4 of the query in Listing 4, we can see the filters that should be applied to the query result. In this case, ?publicacao1 must be different from ?publicacao2. But we also take into account the information in lines 9 and 10: the query requires that the year of the publications be the same, since variable ?ano appears in ?publicacao1 and ?publicacao2. Hence, the filter should ensure that each of the publication pairs has different values for all their fields except by the year, which should be the same. In this case, the query should bring in all the distinct publications that were published in the same year.
The output variables can be seen on line 2. When the output projection is generated, it’s important to add the prefixes that are needed and then we are able to link the variables and the attributes present in the document by using the mapping. As an example, we can see that ?tituloP ublicacao1 gets its value from the field $publicacao1.title that has been projected.
The resulting query that is executed is the following:
Listing 5. Resulting query { : $ p r o j e c t => { : name => true , ’ pu bl ic ac ao 1 ’ => ’ $ p u b l i c a t i o n s ’ , ’ pu bl ic ac ao 2 ’ => ’ $ p u b l i c a t i o n s ’ } } , { : $unwind => ’ $publicacao1 ’ } , { : $unwind => ’ $publicacao2 ’ } , { : $ p r o j e c t => { : name => true , : p u b l i c a c a o 1 => true , : p u b l i c a c a o 2 => true , ’ f i l t e r 1 ’ => { ’ $and ’ => [ { : $eq => [ ’ $ p u b l i c a c a o 1 . year ’ , ’ $ p u b l i c a c a o 2 . year ’ ] } , { : $ne => [ ’ $ p u b l i c a c a o 1 . t i t l e ’ , ’ $ p u b l i c a c a o 2 . t i t l e ’ ]
,→ } ] } } } , { : $match => { ’ f i l t e r 1 ’ => t r u e } } , { : $ p r o j e c t => { ’ nomePesquisador ’ => ’ $name ’ , ’ t i t u l o P u b l i c a c a o 1 ’ => ’ $ p u b l i c a c a o 1 . t i t l e ’ , ’ t i t u l o P u b l i c a c a o 2 ’ => ’ $ p u b l i c a c a o 2 . t i t l e ’ , ’ ano ’ => ’ $ p u b l i c a c a o 1 . year ’ } }
By querying the sample data, we get as a result a JSON document containing unique publications that were published in the same year and that have the same author, which answers the initial question:
Listing 6. JSON documents that answer the query [ { ’ i d ’ => BSON : : O b j e c t I d ( ’ 5 8 3 5 e f f c c 0 4 8 b d 0 0 0 1 c d c 2 3 9 ’ ) , ’ researcherName ’ => ’ K r i s t e n Nygaard ’ , ’ p u b l i c a t i o n T i t l e 1 ’ => ’COOL ( Comprehensive Object−o r i e n t e d ,→ Learning ) ’ , ’ p u b l i c a t i o n T i t l e 2 ’ => ’ C l a s s and S u b c l a s s D e c l a r a t i o n s ’ , ’ year ’ => 2002 } ]
These documents are transformed into triples and then inserted on a RDF graph, which is returned as an answer to the initial query. It’s also possible to insert this new knowledge into the ontology and run some other queries or apply any form of logical inference.
The approach presented in this research offers advantages for the scientific community, mainly because it provides a different way of constructing mappings, making this process easier to OBDA Systems.
By avoiding a direct mapping to the database query language, as done by
Placing next to the mapping a conceptual layer that operates on classes in an OOP language makes our approach a more generic and flexible form of data access.
In the future, we intend to analyze the differences between our approach and existing approaches, evaluating advantages and disadvantages and investigating the amount of effort required to apply this solution to other types of NoSQL DBMS.
A future extension may allow onto-mongo to access other types of NoSQL DBMSs, such as Cassandra. We also need to make the translator more generic and flexible, as well as provide support to other types of queries like joins, filters, and groupings. A possible solution is to develop an abstract interface describing the methods that need to be implemented for a specific query translator, and then an intermediate user (the user who configured the mapping) will be able to implement these methods for a different kind of DBMS the user needs.
Also, we intend to simplify the mapping method so that the user can create it with as little knowledge of the Ruby language as possible.
One application of our approach currently being developed is a project to handle data extraction from the Lattes Platform in order to find co-authorship networks via Link Prediction and Machine Learning algorithms. For that, onto-mongo is being used to extract semantic information from a NoSQL database through SPARQL queries.
Another application in development by our research group will use this approach to access data from a MongoDB database on legal decisions.