The Handle Software manages references to resources of information. However, it does not support a search functionality. A prior implementation with Elasticsearch could not e ciently capture the complex structure of our dataset, especially the relationships between handles. In this paper, we apply a graph database together with Elasticsearch to provide more search capabilities to users. In addition, the graph can e ciently store meta-data provided during handle creation. Further use cases for this graph include redundancy detection (two or more handles pointing to the same URL), or bibliographic network analysis.
Nowadays, people often locate digital objects using Uniform Resource Locators
(URLs). However, URLs tend to be broken over time [
According to FAIR (Findable, Accessible, Interoperable, Reusable)
principles, data with PIDs and their meta-data are supposed to be ndable [
The paper is organized as follows. Section 2 discusses the system design. The system is evaluated in Section 3 and the conclusion is presented in Sections 4.
Copyright c 2019 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
Graph databases { a special category of NoSQL databases [
:Handle - handle :Handle_value :Handle_value - handle_value 4 https://neo4j.com 5 https://www.handle.net 6 http://dtr.pidconsortium.eu the handle node 10.123/456 to the handle node 10.123/789. Only when the handle node 10.123/789 is not found, the system will create that node rst, then add a relationship between those two handle nodes. Lastly, there are many cases where a handle value does not have an atomic value but a JavaScript Object Notation (JSON). In a naive approach, the system will create a new node and put the whole JSON inside. However, doing so leads to disadvantages. First, because it is just a string, it is very hard to distinguish between the key and the value to search on. Second, all the structures inside the JSON as well as the connections with other nodes are lost. Hence our system must parses each JSON object and creates an appropriate graph from it. The graph in Figure 2 shows what our system generates from the example data in Table 1. As can be noticed from the gure, there are two empty nodes in the graph. These empty nodes are the results of the JSON parsing process. The purpose of these nodes is to group related data together.
To improve the performance, each node will have one more property called nodeId. This property is unique among nodes and used as the key of a node. When a node is created, its nodeId is calculated by hashing the value of that node. This process is applied for non-handle nodes. Because the handle string is already unique, the nodeId of a handle node is the handle string itself. The nodeId property is indexed with a unique constraint. While indexing enhances the performance of the READ operation, other operations (CREATE, UPDATE, DELETE) are slowed down due to the updating of index table. Indeed, our graph database is under a heavy load of CREATE and DELETE operations. :Email :Email :URL
:Name ‐ Name: Triet Doan
:Handle ‐ Handle:10.123/456 :isPreviousVersionOf :Name :INST
:INST ‐ INST: GWDG :address
:city ‐ city: Göttingen :city
:country :address :country ‐ country: Germany
: rst_name :first_name ‐ first_name: Triet :last_name :last_name ‐ last_name: Doan :URL ‐ URL: http://www.google.com ‐ Handle:10.123/789 :Creator :Creator
:Handle However, because of the uniqueness of every node in our graph database, one single CREATE or DELETE involves many READ operations which greatly bene t from the index. We hence observed that indexing leads to a huge boost in the performance of the system (see Section 3). 3
The execution time was measured when the system was running under a heavy load scenario. During this time, the system had to retrieve data from two data sources and build a graph with around 1 million nodes and 2.5 million relationships. Figure 3 shows the number of handle values processed by the system per minute. The lower green line shows the execution time when data was collected without hashing and indexing. As can be seen from the chart, the system runs quite fast at the beginning with around 1000 handle values processed per minute. However, it quickly becomes slow over time. The reason for this performance loss is that whenever a node is created, the system must make sure that the node is unique. Therefore, the more nodes it has, the longer the checking time. After around 107 hours, which is about 4.5 days, the system became too slow. It processed only 130 handle values per minutes. This test was stopped after 119 hours (almost 5 days). If continued, it would have taken around 7 days to nish. For the second approach, with hashing and indexing, the performance was greatly improved as shown by the upper blue line in Figure 3. It can be seen that it runs quite stable with the number of processed handle values uctuating between 2000 to more than 3000 per minute. By exploiting the indexing feature, the performance is increased by factor 7. 4
Our rst achievement is the appropriate graph schema for the handle data. That graph schema is able to deal with the exibility in the creation of handle values The comparison of performance between indexing and non-indexing approach With indexing Without indexing 0 20 40
60 Hours 80 100 120 as well as maintaining a good performance of the system. A search engine for handles is the second achievement. It o ers a variety of search options from Elasticsearch and the ability to manage relationships between handles from Neo4j. Basic usages can be done through the Graphical User Interface (GUI), while a web-based tool is ready for more advanced purposes, such as some analyses which are performed to discover hidden knowledge inside the graph. A topic of future work to consider is the interoperability of the system: the graph database can be enriched by importing data from other platforms, such as DOI, ARK, ISBN, or ORCID.