XML has been the de facto standard for digital editions for years, but its serious limitations include an inability to represent overlapping markup and the encoding of multiple annotation hierarchies. With emerging graph database technologies we have the opportunity to develop new approaches. In this paper the advantages and modelling principles of graph-based digital editions will be discussed. XML in combination with TEI has become the standard format for digital editions. Most digital research environments, such as TextGrid or Ediarum, use TEI-XML for encoding sources and research data. But XML has some limitations which restrict researchers working on digital editions in certain ways. In this paper these limitations will be discussed and proposals for a graph-based digital edition will be presented.
The use of XML for digital editions is widespread and has become a standard over the last years.
XML has some inherent limitations, however, such as its inability to represent overlapping
markup as well as to encode multiple annotation hierarchies [
To solve this problem several approaches have been developed. Daniel Jettka presents
some of these problems in his paper [
What all these approaches have in common is that the complexity of the XML increases to a point where it becomes very hard to handle from an editor’s or application’s perspective. In a second approach, Stand-off markup can be used to separate the source from the related information. This means that the source’s XML content is indexed on a character basis and the related information is stored in another file with pointers to the index-numbers. With this approach, the simultaneous display of multiple annotation hierarchies is possible. A disadvantage of this solution is the rigidity of the index. If characters in the source file are changed after indexing and enriching the material, all indexes are worthless as long as the whole document is not re-indexed. There is currently no editing tool available for solving this problem that also takes basic aspects of usability into account. 3 3.1
My paper at http://mittelalter.hypotheses.org [
The second step is to append annotation information. Figure 2 shows overlapping markup of a reference to another database record, represented by the yellow node, and another annotation, represented by the green node. As mentioned initially, TEI-XML represents a standard for encoding sources in digital editions. If graph-based digital editions become a realistic alternative for digital editions, the graph database must be able to handle TEI-XML data. For this purpose a GitHub project for importing an TEI-XML document into Neo4j was started.
For a first feasibility study a TEI-XML document containing a Latin letter from the 17th century with a German abstract of the text has been used. The file includes additional information about the persons, places and other entities in the letter. In an elaborated virtual research environment this information would of course be separated from the text of the letter but for demonstrational purposes of the study this information was also added to the example XML file.
Listing 1 presents some parts of the identifying information about persons, places, and entities are listed. ...
Listing 1: Persons, places and items in the xml file < p a r t i c D e s c > < l i s t P e r s o n > < p e r s o n xml : id =" P006 " > < idno type =" gnd " > http :// d - nb . info / gnd / 1 1 9 3 5 7 1 0 0 < / idno > < p e r s N a m e type =" reg " > < surname > L u b i e n i e c k i </ surname > < forename > S t a n i s Å Ć a w </ forename > </ persName > </ person > < p l a c e xml : id =" DE - HAM " >
< p l a c e N a m e type =" reg " > Hamburg </ p l a c e N am e >
<idno type =" gnd "> http ://d-nb. info / gnd /4023118 -5 </ idno > </ place > <textClass > <keywords > <list > <item xml :id =" C_1664_W1 "> <idno type =" iau ">C /1664 W1 </ idno > <label >C /1664 W1 </ label > <rs type =" objectType " key =" astro_comet "/ > </item > <item xml :id =" Venus "> <idno type =" gnd "> http ://d-nb. info / gnd /4062527 -8 </ idno > <label > Venus </ label > <rs type =" objectType " key =" astro_planet "/ > </item >
</item > <item xml :id =" hypothesis "> <idno type =" hyp "> hypothesis </ idno > <label > hypothese about comets </ label > <rs type =" objectType " key =" hypothesis "/ > </item > </list > </ keywords >
Listing 2 shows the German abstract of the Latin letter. Within the abstract, persons, places and entities are identified and linked to the other entities shown in the first listing.
<abstract > <p>< name ref ="# P007 "> Langius </ name > entschuldigt sich , dass er die Anfrage <name ref ="# P006 "> Lubienietzki </ name > zu dem Kometen des Jahres 1664 versp ä tet beantwortet : Den <rs ref ="# C_1664_W1 "> Kometen </rs > habe er im ausgehenden Dezember 1664 und Anfang Januar 1665 im <name key ="# Hya "> Sternbild Hydra </ name > beobachtet , und ein ä hnliches Phä nomen sei im ausgehenden März bzw . Anfang April in <name ref ="# Peg "> Pegasus </ name > und <name ref ="# And "> Andromeda </ name > erschienen . In Anlehnung an <name ref ="# P001 "> Demokrit </ name > äu ß ert <name ref ="# P007 "> Langius </ name > die <rs ref =" hyp "> Hypothese , dass Kometen aus Atompartikeln gebildet werden : Sie entst ü nden im Kegelschatten der <name ref ="# Terra ">Erde </ name > und wü rden erst dann sichtbar , wenn sie aus diesem hervor - und in das Licht der <name ref ="# Sol "> Sonne </ name > hineintr äten </rs >. Seine Hypothese veranschaulicht <name ref ="# P007 "> Langius </ name > ...
If we select an XML-like approach to the graph the "xml-root" of the document is represented by the pink node (figure 3). All other informations can be reached from that node but they are not ordered in an hierarchical way as it would be in an XML-Document. Four nodes are directly connected to the root node. Those will be discussed below. 3.5
The two empty nodes linked to the root node are hyperedges.1 One hyperedge represents the sending information of the letter with a combined link to the author of the letter, and the 1For the concept of a hyper-edge: http://neo4j.com/docs/stable/cypher-cookbook-hyperedges.html. place from which it was sent. The other hyperedge represents the recipient information with a combined link to the person who received the letter and the place where it was received. This construct facilitates the traversal of the graph if there are many letters in the database. 3.6
Beside the root node there are two other linked nodes next to the hyperedge nodes. One leads to the beginning of the abstract, the other leads to the Latin text of the letter. Following the HAS_ABSTRACT-edge the abstract node can be reached. From this node, following the NEXT-edge we can get the abstract word by word. You can explore the abstract by doubleclicking on the word nodes and see the connected nodes for the annotation of persons and places. All word-nodes of the abstract are directly connected to a p-node too, which represents their belonging to the paragraph represented by the p-node, while the order of the words is represented by the NEXT edges. 3.7
Besides the possibility of exploring the abstract in the graph, a Cypher query can be used to get the text of the abstract as a result. Listing 3 shows the query and figure 4 shows the results in the Neo4j frontend.
// Get Abstract Text MATCH p = ( t : Tag { name : ’ abstract ’}) -[: NEXT *] - >( e ) WHERE NOT ( e ) -[: NEXT ] - >() RETURN REDUCE ( s ="" , x in tail ( nodes ( p )) | s +" " + x . text )
Let us now examine the cypher query. With the MATCH-p-statement we start a traversal query starting from a tag-node with the name "abstract" connected to the next node with NEXT-edges and to the following node with NEXT-edges, until we come to a node that has no departing NEXT-edge. The ’reduce’ function reduces the result to one single line of text only. The rest of the query converts the resulting chain of nodes to the represented words.
Querying for the tag-node the Neo4j frontend shows the structure of the imported TEI-XML as shown in figure 6. Starting from the pink root-node you can follow the abstract-node to the abstract or the body-node to the body of the TEI-XML file. All paragraphs in the body are connected to the body-node and their own hierarchy is represented by NEXT_TAG edges. One IS_CHILD_OF-edge leads to the head-node with all the entity information connected to it.
Beside the planets and comets a scientific hypothesis is mentioned in the letter and encoded in
the XML document as an item.2 In the following example, the statement of the hypothesis will
be connected with the parts of the text in the abstract in which it is explained, as well as with
the planets and persons mentioned. Modelling Digital Editions in Graph-Databases changes the
way pieces of information are explored. My paper [
// C r e a t e E d g e s from text n o d e s to the h y p t h e s i s node M A T C H ( from { id : ’ h y p o t h e s i s ’}) , ( to { text : ’ H y p ot h e s e , ’})
C R E A T E from -[: S T A R T _ H Y P O T H E S I S ] - > to ; M A T C H ( from { id : ’ h y p o t h e s i s ’}) , ( to { text : ’ h i n e i n t r ä ten ’})
C R E A T E from -[: E N D _ H Y P O T H E S I S ] - > to ; // C r e a t e e d g e s to the p r e s e n t e r of the h y p o t h e s i s M A T C H ( from { id : ’ h y p o t h e s i s ’}) , ( to { s u r n a m e : ’ Lange ’})
C R E A T E from -[: P R E S E N T E R _ O F _ H Y P O T H E S I S ] - > to ; // T h i n g s c o v e r e d by the h y p o t h e s i s M A T C H ( from { id : ’ h y p o t h e s i s ’}) , ( to { l a b e l : ’ Sol ’})
C R E A T E from -[: I S _ A B O U T ] - > to ; M A T C H ( from { id : ’ h y p o t h e s i s ’}) , ( to { l a b e l : ’ Terra ’})
C R E A T E from -[: I S _ A B O U T ] - > to ; M A T C H ( from { id : ’ h y p o t h e s i s ’}) , ( to { l a b e l : ’ C / 1 6 6 4 W1 ’})
C R E A T E from -[: I S _ A B O U T ] - > to ; In this paper a first glimpse of the great potential of graph-based digital editions was demonstrated even if we must concede that we lack of powerful tools und user-interfaces. The conceptual development is still in its early stages, but by using graph databases we could: seemlessly include multiple annotation hierachies within one dataset handle a scenario where several researchers are working together on the same source with a transparent separation of their different markup needs query the various annotational layers within one query language and changes to the source text will neither destroy the word-order nor the meaning of existing annotations.
Many of the discussed issues could certainly also be solved with specific markup strategies or with Stand-off markup, but this solution usually involves great effort and often produces XML that is very hard to handle and not human-readable at all. "Ease of use" has always been a major reason cited for the choice of XML as a (standard) format. Due to the complexity of future digital editions, there is no reason we should not take the step to graph-based digital editions. The contents of a graph database – its graphs – are human-readable and can represent very complex annotational hierarchies, while also removing the worry about boundary demarcation. Moreover everything can be exported to XML or graphml files for long-term archival purposes.
This becomes possible in a model which brings the way things are structured in the human mind and the database structure closer together. Graph-based digital editions are the next important step for the Digital Humanities, as the editing disciplines acquire a very flexible, easy-to-use but powerful tool. To achieve this goal it is important that we find standardized ways for modelling events, actions, political aims, etc., in order to explore and compare different sources from different contexts. The second important task is the programming of user interfaces which can be easily used and adjusted to the researchers needs and can explore more than one datasource on the internet. Developing this will be a big challenge. 5
The Neo4j-TEI-Importer was developed by Stefan Armbruster from Neo Technology ( stefan.armbruster@neotechnology.com) and I want to thank him for his help. He published the importer on GitHub (https://github.com/sarmbruster/neo4j-tei-importer). Thanks also to Sascha Kaufmann from the Digital Humanities Department of the University of Bern for fruitful discussions on how to model XML data in a graph database environment.