The growing impact of Semantic Wikis deduces the importance of nding a strategy to store textual articles, semantic metadata and management data. Due to their di erent characteristics, each data type requires a specialized storing system, as inappropriate storing reduces performance, robustness, exibility and scalability. Hence, it is important to identify a sophisticated strategy for storing and synchronizing di erent types of data structures in a way they provide the best mix of the previously mentioned properties. In this paper we compare fully structured, semi-structured and unstructured data and present their typical appliance. Moreover, we discuss how all data structures can be combined and stored for one application and consider three synchronization design alternatives to keep the distributed data storages consistent. Furthermore, we present the semantic wiki KiWi, which uses an RDF triplestore in combination with a relational database as basis for the persistence of data, and discuss its concrete implementation and design decisions.
Although the promise of e ective knowledge management has had the industry abuzz for well over a decade, the reality of available systems fails to meet the expectations. The EU-funded project KiWi - Knowledge in a Wiki project sets out to combine the wiki method of collaborative content creation with the technologies of the Semantic Web to bring knowledge management to the next level. Combining a Wiki with Semantic Web technologies results in three types of content:
about KiWi `s contents.
exibility and spreading of the data
In this paper we explain the di erences of data storage for these data types. We describe our choice of design and illustrate its usefulness for Semantic Social Software Applications. Furthermore, we explain how these three di erent approaches can be integrated in a single application, which is build with the Java Enterprise Edition (Java EE)1 platform.
We present and discuss three di erent kinds of data: structured, unstructured and semi-structured. We discuss the weaknesses and strengths of each of them and describe that for a semantic social software application a combination of them brings advantages in form of an improved exibility and performance. Furthermore, we describe several ways and designs how an application can implement the di erent ways of persisting data. The main challenge by developing an application which uses di erent data storages is to de ne a common interface for the access of data and to guarantee the synchronization of the di erent data storages.
Chapter 2 discusses the bene ts, techniques and di erences of structured, unstructured and semi-structured data. For this discussion examples for each paradigm are compared: An Apache Lucene full-text index for unstructured data, a relational database for fully structured data and an RDF triplestore for semi-structured data.
Chapter 3 describes several design patterns, which were used within KiWi to combine the three di erent approaches, focusing on the combination of relational databases and RDF triplestores. This chapter tries to answer the question which data should be stored where and discusses the decisions taken by the KiWi project.
Chapter 4 gives an overview of related work and chapter 5 summarizes the practical relevance of this approach. 2
Structured, Unstructured and Semi-Structured Data In the semantic wiki KiWi we need all three kinds of data: structured, unstructured and semi-structured. This chapter presents and compares the di erent forms of data and gives examples and state-of-the-art techniques. Finally, a tabular overview of the di erent kinds of data structures is given. 2.1
According to[
Storing data in an unstructured form without any de ned data schema is a common way of ling information. An example for unstructured data is a document that is archived in a le folder. Other examples are videos and images.
The advantage of unstructured data is, that no additional e ort on its classi cation is necessary. A limitation of this kind of data is, that no controlled navigation within unstructured content is possible.
A common technology to search in unstructured text documents is full-text search. The advantage of full-text search is, that it is completely decoupled from the data. This makes it very exible, because it can be used on every kind of textual data, even if no schema or structure is de ned. One limitation of full-text search is that it cannot be used to search for pictures or videos.
Full-Text Search can be optimized by generating a full-text index, that increases the performance of a full-text search query. A famous full-text search engine library is Apache Lucene2 . Other examples are MySql3 and Postgres indixes4. 2.2
Fully structured data follows a prede ned schema. "An instance of such a
schema is some data that conforms to this speci cation,"[
"The well-de ned schema of fully structured data enables e cient data processing and an improved storage and navigation of content,"[2, page 122]. The 2 http://lucene.apache.org 3 http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html 4 http://www.postgresql.org/docs/7.4/static/indexes.html cost for high performance and navigation is exibility and scalability. It is di cult to subsequently extend a previously de ned database schema that already contains content. For example, it is not possible to extend a single table row with a new attribute without creating another table column. This is unpro table for tables that contain thousands of other rows that do not need another attribute.
An advantage of relational database applications are the existing tools and web frameworks, which support the development of database-focused applications. For instance, Hibernate5 and Oracle TopLink 6 are Object/Relational (O/R) Mapping frameworks, which map classes and objects to relational database tables and rows. Moreover, there exist several practical tools for maintenance, management and administration of relational database systems. 2.3
Fig. 2: Sample RDF Graph
Semi-structured data is often explained as "...schemaless or self-describing,
terms that indicate that there is no separate description of the type or
structure of the data"[2, page 11]. Semi-structured data does not require a schema
de nition. This does not mean that the de nition of a schema is not possible,
it is rather optional. The instances do also exist in the case that the schema
changes. Furthermore, a schema can also be de ned according to already
existing instances (posteriori). The types of semi-structured data instances may be
de ned for a part of the data and it is also possible that a data instance has
more than one type[
One of the strengths of semi-structured data is "... the ability to
accommodate variations in structure"[2, page 12]. This means that data may be created
according to a speci cation or close to a type. For instance, elds can be
duplicated, data can be lacking or there may exist minor changes[
A typical example of semi-structured data is XML, which is a language for
data representation and exchange on the web. In XML data can be directly
encoded and a Document Type De nition (DTD) or XML Schema (XMLS) may
de ne the structure of the XML document[
In the research elds of the Semantic Web, knowledge is encoded in Resource
Description Framework (RDF) triples[
In KiWi, data sometimes needs to be transformed from one structure into another. For instance, fully structured data is converted into unstructured data when a user generates a PDF out of a wiki article and its management data like author, creation date and so forth. It is also possible to convert data from a database into semi-structured data, like an RDF graph. Several modern web applications use RSS feeds , which are generated by reading data of a relational database and provide it in RDF format.
On the contrary, it is more complex to transform unstructured information into semi- or fully structured information. KiWi structures textual content with techniques of information extraction and natural language processing. Tags, which describe the content of a text, are automatically extracted out of a wiki article. In this way the unstructured data can be converted into semi-structured data. 2.5
It can be summarized, that the high degree of typing enables a better performance and less exibility.
Serge Abiteboul, Peter Buneman and Dan Suciu de ne several reasons why
de ning a structure is good for[
Table 1 gives an overview over the strengths and weaknesses of the di erent storing structures in technology elds that may be important in practice. 2.6
To know how to combine a relational database and a triplestore we have to consider what data is stored where. Therefore, we review the strengths and weaknesses of di erent data structures and discuss the demand of structure characteristics for speci c data sets. A relational database stores fully structured data, which necessarily have a prede ned schema. Relational databases provide the application with a high query-performance and fast joins. Vulnerabilities are rare since more than 30 years of research, development and improvement eliminated most of them and increased the robustness.
Semi-structured data like RDF data does not have to prede ne a schema and is very scalable and exible. Furthermore, RDF and OWL7 allow the de nition of logical rules and many applications implement an inference layer that infers new triples by reasoning over the existing data set.
Thus, data that has a prede ned schema, that is sensitive and that is often queried should be stored in a relational database. Data that is added to the application lately (e.g. data for extensions or plug-ins) and data that might be important for reasoning should be stored in the triplestore. Figure 5 provides a quick overview over the division into relational database data and triplestore data. As one can see, the data sets are partially overlapping.
Relational Database
Sensitive Management
Data Data with a predefined schema Versioned Data
Core Component Management Data
Triplestore Non-sensitive Data
Data that can be access from other applications or agents
Automatically or manually generated data during runtime Plug-In & Extension Data Combining structured and unstructured data is an often applied strategy in web applications to achieve the advantages of both persistence types. The employment of all three alternatives, however, is uncommon.
KiWi is a platform for Semantic Social Software applications, implemented with Java EE technologies. We decided to store data in a semi-structured form, because we wanted to attain a better exibility and scalability than provided by the structured form. We also wanted to store data in a robust database with good query and join performance. We have to control a big amount of textual content, which needs to be queried for keywords.
Hence, we decided to combine unstructured, structured and semi-structured data storage and segmented the data into long textual content (unstructured ), core component data (fully structured ) and exible data (semi-structured ). For a better clarity, Table 2 visualizes the segmentation. The sets of fully structured and semi-structured data are overlapping, because we represent the non-sensitive core data additionally in the triplestore to get a complete data set that can be provided to other Semantic Web Applications (e.g. Linked Data8). Applications that store data in a triplestore as well as in a relational database have to implement a synchronization mechanism to keep information consistent. Such a synchronization mechanism can be implemented on di erent layers of an application.
Content Type Unstructured
Textual Content Fully Structured Sensitive Content &
System Maintenance Data Core Component Data
Example Wiki Articles, Blog Pages ContentItem, User data Semi-Structured Non-sensitive Core ContentItem-extending Component Data, Flexible Data, Use Case Data
Content & Individual Data
Database Layer Synchronization on the database layer is implemented by forcing a data storage (e.g. database) to update another data storage (e.g. triplestore) when a data item changed. For instance, every time an application writes on a database, the according operation could be executed on the triplestore, which might be hold in the database. This could be implemented using database triggers or Java EE persistence interceptors. Another possibility is that the triplestore is generated automatically from the entries within the database. Hence, the triplestore could be updated regularly. In both variants the database is de ned as master and the triplestore is de ned as slave. This design is illustrated in Figure 6.
This design bene ts from high performance and good integration of relational databases into existing software technology stacks (e.g. Java EE). Furthermore, functions provided by a triplestore, like reasoning, are possible, because the data also exists in a semi-structured form. The disadvantage is that this design does not o er the exibility of semi-structured data, and that the application has read only access to one data storage.
An alternative design is a bi-directional trigger synchronisation between relational database and triplestore. The triplestore, as well as the relational database can update each other with database triggers. The advantage of this design is that it allows writing access to both data storages. This design is illustrated in Figure 7. The limitation is, that some updates on the triplestore cannot be processed on the database and must be forbidden to keep consistency. Therefore, this design does not support the full exibility of semi-structured data, too. O/R Mapping Tool O/R mapping tools provide another layer for synchronisation. This design is illustrated in Figure 8. For instance, the Java Persistence API (JPA)9 could be extended to persist Java objects in the database as well as in the triplestore. This encloses the translation of JpaQL (JavaPersistenceApiQueryLanguage)10 queries into triplestore queries. This approach decouples the
10 http://java.sun.com/javaee/5/docs/tutorial/doc/bnbtg.html
Fig. 6: Database de ned as master and triplestore de ned as slave persistence layer from the application layer, and, therefore, provides the exibility of semi-structured data. Thus, additional attributes of an object may be de ned during the runtime of an application and persisted in a triplestore. This may be realized using Aspect Oriented Programming (AOP)11 techniques or dynamic languages like Groovy12. With this approach, distributed queries over several datasources could be realized. 11 http://www.eclipse.org/aspectj/ 12 http://groovy.codehaus.org/
the synchronisation of data is to implement it in the middleware or application layer. This layer could use normal JpaQL queries for the database as well as SPARQL commands to query the triplestore. This design is illustrated in Figure 9. A di erent alternative is to provide a general purpose query language for both data stores. In this way, distributed reasoning over the triplestore, as well as over the data in the relational database system could be enabled. This design is illustrated in Figure 10.
We decided to choose the Application Layer for synchronization, because it grants us exibility to improve weaknesses and to enforce the strengths of each data structure type. First, we will give you an overview over the triplestore position inside of KiWi.
Figure 11 illustrates the overall structure of KiWi. The combination of
triplestore and relational database can be found in the Persistence and Data Model
layers. As an RDF triplestore KiWi currently uses Sesame2 13. The relational
database connection is enabled through Hibernate with JPA. Storage con
gurations for relational database and triplestore can be applied with Java
annotations.
13 http://www.openrdf.org/
Transactional synchronization As Table 1 illustrated, transaction
management for unstructured and semi-structured data is not very common or
matured. Though, storing data in those federated, heterogeneous databases needs
to be controlled to avoid states of inconsistency. A global transaction
management is the easiest way to administer all three data structure types in terms of
their transactions. JBoss Seam[
Programmatic transaction processing requires the de nition of a start and
end time for the transaction. It allows exible pre- and post-treatment of the
application when the transaction ends. Declarative transaction processing, on the
other hand, is simpler than programmatic transaction management, because the
transaction start and end time is managed by the container[
A more detailed description of the transaction models in Java Enterprise
Applications, the concurrency problems that triplestores must consider and the
database synchronization is given in [
rel DB update fails
Tx start r e a d f r o m r e l . d a t a b a s e o r / a n d t r i p l e s t o r e m a k e l o c a l c h a n g e s t o t h e d a t a i t e m s r e a c h b e f o r e c o m p l e t i o n p h a s e s t o r e c h a n g e s in triplestore rel DB update s u c c e e d s commit Tx
Tx end Data Versioning Versioning of unstructured, semi-structured and fully structured data is an important core functionality of KiWi. RDF triple versioning is uncommon and few well-established RDF repositories allow versioning. Sesame2 puts RDF triples internally under version control, but it does not enable undo or redo functions.
With the chosen transaction strategy we can easily implement version-control
of unstructured, semi-structured and fully structured data. At the end of a
transaction, updates for all kinds of data are creates and stored as revisioning and
update tables in the relational database. This design was chosen to collect all
versioning data in a robust database, to enable easy querying, and, consequently,
to allow fast undo and redo functionality for all kinds of data. Versioning data
has a pre-de ned schema that will not be changed in the future.
Query & Reasoning With the chosen level of synchronization it is possible to
create a query language for all kinds of data. KiWi enables this global querying
that interprets to SQL and SPARQL15 queries. Furthermore, reasoning is not
limited to the RDF repository anymore. The interested reader is referred to [
Related Work
In the following we provide an overview over implementations of semi-structured
data into existing application stacks. Elmo[
Conclusion The main advantage of fully structured data is the strong typing which enables high performance and e ciency. On the other hand, unstructured and semistructured data allow a higher degree of exibility. In this paper we compared unstructured, semi-structured and fully structured information and discussed an application design which combines all three types of data, based on a relational database system combined with an RDF triplestore. We illustrated this design on the concrete implementation of the semantic wiki KiWi. We saw that a challenge for such an application is to avoid states of inconsistency and present three di erent layers where a synchronisation of data within an application could be implemented: 1 On a low level database layer, 2 On the he O/R mapping layer, and 3 On the application layer.
In KiWi the synchronisation of data is implemented on the application layer because it o ers database independence and enables the implementation of a common query language for all di erent data stores. 15 http://www.w3.org/TR/rdf-sparql-query/