We introduce ADOnIS, an information system which coherently integrates two important, yet mostly disparate data sources, namely structured, tabular data, and unstructured data in terms of publications. The integration is achieved by providing the underlying background knowledge of the domains involved in terms of adequately tailored ontologies. Once the two basic data sources are semantically linked, entirely novel opportunities for cross-source information retrieval arise which we will highlight in this paper.
Two mutually separated \data cultures" have emerged over the years and still
persist in the eld of information systems. On the one hand, the database
community focuses on the structured representation of slices of the reality, typically
in terms of relations and tables. On the other hand, the information retrieval
community deals with, from a computational view, unstructured data, namely
documents as streams of characters (and other media types, such as visual data)
and tries to computationally interpret (and thus restructure) the meaning
encoded in these textual data carriers. Both worlds rest on solid mathematical
foundations and stable technical implementations on the basis of which huge
amounts of structured and unstructured data can be managed and searched on
an industrial scale. Yet, with the exception of activities aiming at the Semantic
Web (for a survey, cf. [
This lack of integration hampers the usability of data at all levels. Consider,
as a concrete example, an interdisciplinary research community such as the one
established in the collaborative research center (CRC) AquaDiva, our research
environment [
to the BExIS 2 data portal, the central information system hub of the project to obtain data that have been collected already. Typically, the student will start with one query and then try to navigate results and nd related entries.
Her success will strongly depend on her familiarity with the special mix of domains, skills of interacting with search engines and data repositories (including SQL/SPARQL-style query languages), her knowledge of linguistic variants and the taxonomic structures of the relevant sublanguages. For instance, queries for \virus transport subsurface", \virus transport soil", and \phages transport soil" typically return only partially overlapping result sets in PubMed or standard data management systems. This is due to simplistic string matching criteria, the incapability to account for linguistic variations of the same content (in ection variants, phrasal paraphrases, or synonyms) and the general lack of conceptual background knowledge (e.g., the taxonomic or partonomic structure of the domains' terminologies).
In our work, we aim to account for these de ciencies in a systematic way. The solution we propose is implemented in ADOnIS, the AquaDiva Ontology-based Information System that provides integrated and seamless access to structured data and unstructured publications by making use of a variety of semantic technologies such as ontologies and natural language processing (NLP) tools. With this, we hope to reduce the cognitive burden put on searchers while, at the same time, we intend to increase the coverage and quality of search results. In this paper, we brie y describe the methodologies underlying ADOnIS and the way users can interact with the system. 2
Data in general and scienti c data speci cally can be roughly categorized into
structured and unstructured data. Unstructured data has no prede ned data
model and is typically text-heavy. Due to its unstructured nature, it is a
challenging task to extract speci c and useful information [
On the other hand, structured data is data that is organized according to a
prede ned (but not necessarily explicitly known) data model, such as a table in a
relational database (known data model), a document in RDF format ((partially)
known data model) or a spreadsheet (unknown implicit data model). This
prede ned data model (if known2) enables search based on structured queries (e.g.
SQL or SPARQL queries) with a well-known semantics. Although these kind
2 In cases where the underlying data model is implicit (e.g. in spreadsheets), it needs to
be provided by the data creator or has to be automatically extracted using
machinelearning techniques. The latter can be particularly challenging, since in contrast to
of query interfaces make it easy to e ectively identify and discover a piece of
information and access it in concise way, they are rather complex and thus less
suited to users with a non computer science background. Recent approaches have
therefore started to combine and integrate keyword-based search approaches for
unstructured data and concept-based approaches for structured data [
K-search is one of the earliest works on hybrid search that supports the
retrieval of documents and knowledge [
An semantic search architecture speci cally designed for biodiversity data
is suggested in [
text-based documents, e.g. data tables, often reveal only scarce information that might give a hint to its meaning.
We have implemented ADOnIS as an extension to the BExIS 2 data
management platform [
Scienti c data stored in BExIS 2 typically refer to eld observations and measurements and are organized in tables. Each table and its corresponding meta information is referred to as a dataset. In addition to the data table containing the data values, each dataset comprises the table schema (name, datatype and unit of measurement for each data column) and metadata such as information about the data provider. Both, the actual data values and the table schema, are stored in a relational database.
To make the semantics of datasets explicit, we annotate each data table with
conceptual knowledge encoded in ADOn , a domain-speci c ontology expressed
in Owl 2.4 The ontology is tailored to the needs of the description of
observational data from the life sciences domain. It only includes relevant classes
and properties of these as TBox statements. Assertions about data values and
data annotations, i.e. ABox statements, are not materialized in the ontology.
Instead, we use the ontology-based data access system Ontop [
ADOn Ontology & Semantic Annotation. As core ontology, we use a
modi ed version of the Extensible Observation Ontology (Oboe) [
Collections, which resemble the concept of a dataset in BExIS 2. Each data row in a BExIS 2 data table is modeled as one or more oboe:Observations. An observation refers to an oboe:Entity, e.g. a Tree, and a set of oboe:Measurements related to that entity. A measurement refers to an oboe:Characteristic, uses an oboe:Standard and results in a value. For instance, for a certain Tree entity, its Circumference (characteristic) might have been measured in meters (standard) and the measured value is 0:8. Oboe allows to indicate contextual relationships between observations, e.g. a tree might have been observed within a certain forest and this forest is located in a certain area. Modeling observations in this way enables logical inferences about entities and the relationships between them, as well as about measured characteristics of entities. In the life sciences domain, both observed entities and their characteristics are particularly important when trying to explain phenomena and thus play a key role when searching for datasets.
To cover domain-speci c characteristics and entities, we reuse concepts from
domain ontologies such as Obi (biomedical investigations),6 Envo
(environmental features),7 Ncit (biomedical concepts)8 and ChEBI (chemical
entities).9 These were selected using the Joyce tool for ontology selection and
integrated into our ontology applying strict methodological criteria to
guarantee non-redundancy, minimality, and optimal coverage [
Each BExIS 2 data value/data column was (manually10) annotated with an ontology class corresponding to the entity it refers to, an ontology class modeling the characteristic that was measured and a class referring to the measurement standard that was used. Moreover, for each dataset, we indicated contextual relationships between the observed entities. The semantic annotations are stored in a relational database.
Ontop Mappings In order to enable Sparql queries over the conceptual view given by the ontology, we de ned mappings that relate BExIS 2 datasets, the entities and characteristics they refer to, the measured values and the dataset annotations residing in the relational database to class and property symbols in the ontology. These mappings are xed for a given ontology and database. The subsequent mapping for example, creates a (virtual) instance for each characteristic measured in some annotated BExIS 2 dataset. It indicates the type of this instance (some subclass of oboe:Characteristic) as given by the semantic annotation stored in the database table annotation (cf. mapping below), and relates it to dataset instances that refer to this characteristic (not depicted). mappingId CHARACTERISTIC-TYPE target :crct_{crct_id} a <{crct}> . source SELECT DISTINCT crct, chrct_id FROM annotation Query Generation Using this approach, we can pose Sparql queries about observational data stored in BExIS 2 on the schema level as well as on the level of individual data values. At the moment, we do not use the full potential of this solution, but rather restrict ourselves to the retrieval of BExIS 2 datasets based on keyword queries. For that purpose, we translate the search terms into a set of Sparql queries. For each keyword that can be mapped to the label (via string comparison) of an ontology class C that is a subclass of oboe:Characteristic, we create the following SPARQL query (pre xes omitted) that returns all datasets that measure C.
SELECT DISTINCT ?dset WHERE { 10 We are currently working on a data upload wizard which analyzes new datasets to (semi-)automatically identify semantically annotated data attributes (the type of measurement referred to in a dataset column, its datatype and unit of measurement) that are already known to and maintained by ADOnIS . Such a mechanism will enable semantic annotation with little user interaction.
?dset ad:refersToCharacteristic ?char.
?char a <URI of C> }
For each keyword that can be mapped to the label of an ontology class E that is a subclass of oboe:Entity, this is done in a similar way, which also accounts for contextual relationships between entities. We create a Sparql query that asks for all datasets referring to entities of type E or to some entity that appears in the context of an entity of type E.
SELECT DISTINCT ?dset WHERE { ?dset ad:refersToEntity ?ent. { ?ent a <URI of D> } UNION { ?ent ad:hasEntityContext ?entC.
?entC a <URI of D> } }
If the label of a characteristic was entered directly before the label of an entity in the search box, we interpret this as a search for the given characteristic measured for the given entity. In case a keyword neither matches the label of an oboe:Characteristic nor the label of an oboe:Entity, we search for datasets containing data values matching the keyword. Finally, we return the union of the resulting datasets. The required information about the type of each provided keyword is delivered by an autocomplete function that provides suggestions while the user is typing words in the BExIS 2 search box. The suggestions are generated based on an index of entity and characteristic class labels de ned in the underlying ontology. The keywords provided by the user as well as the keyword-related information are passed to the structured search module, which has been implemented as web service with a REST-API. 3.2
Unstructured data are handled by the SeMedico system which receives feeds from two sources, viz. more than 26 million life science abstracts from Medline/PubMed1112 and more than 1.5 million life science full texts from PubMed Central from the open access subset. They are stored in a PostgreSQL database.13 Ontologies & Semantic Annotation. Terminological and ontological resources for the indexing of all documents come from various sources. Most notable among them is the NCBI Gene database.14 SeMedico's gene recognition and normalization engine maps gene mentions in the documents to unique NCBI 11 https://www.ncbi.nlm.nih.gov/pubmed 12 https://www.nlm.nih.gov/databases/download/pubmed_medline.html 13 https://www.postgresql.org/ 14 https://www.ncbi.nlm.nih.gov/gene Gene database entries to handle gene name synonymy and ambiguity. Additionally, SeMedico integrates the Gene Ontology (Go)15 and the Gene Regulation Ontology (Gro)16 for the semantic description of di erent types of gene events.
All resources are stored in a Neo4j17 graph database for direct access to their hierarchical structure. All terminologies, ontologies and databases are converted into a common JSON format. This format is then imported into Neo4j using a custom Neo4j server plugin.
Natural Language Processing. Before Medline and PubMed Central documents are added to SeMedico's index, they undergo an extensive linguistic analysis. The goal is to identify textual units referring to gene/protein mentions, ontology concepts, gene interaction events and factuality markers for them as expressed in the documents. To be able to recognize such higher-level semantic concepts, it is necessary to do basic linguistic analysis rst like sentence and token segmentation, part-of-speech tagging and chunking.
Semantic analysis includes species tagging by the Linnaeus tagger [
All documents undergo linguistic processing employing the Uima18
component repository JCoRe [
"Here we show that recombinant Pnc1 stimulates Sir2 HDAC activity." were semantic connections between genes, proteins or, in this case, enzymes are described. Such relations have a high information value for researchers who look for interaction data on speci c entities of interest. Modern relation extraction engines such as BioSem are far superior to simpler approaches which identify co-occurrences of entity within formal text units (e.g., sentences). 15 http://www.geneontology.org/ 16 https://bioportal.bioontology.org/ontologies/GRO 17 https://neo4j.com/ 18 https://uima.apache.org/
However, mere interaction extraction does not take into account the con dence level the authors of a publication assign to these observational data. Consider the following sentence: "These results may suggest that mTOR-mediated autophagy inhibition may result in mesangial cell proliferation in IgAN." While the sentence expresses some interaction between mTOR and igAN, the authors carefully use speculative words like may and suggest. Such information should be integrated into a scienti c data portal to serve as an indicator how trustworthy an information item really is. We store all these annotations together with the original, raw documents in the document database.
In a last step, the analysis results required for semantic search are sent to an ElasticSearch cluster for indexing. We use a custom ElasticSearch plugin to have ElasticSearch accept a term format that allows to exactly specify index terms within the ElasticSearch index.
We model the publication search module as a web service disclosing a RESTlike API. The API accepts parameters for a query string, a sorting criterion and the range of result documents that should be returned. The server then returns a JSON encoded response, including document text and bibliographic information. 4
In this section, we introduce the GUI provided to the end user to facilitate the search process as well as preliminary evaluation results to demonstrate the e ectiveness of the proposed method. To this end, we set up a running instance of the BExIS 2 system with the ADOnIS module that stores 55 real world datasets from the AquaDiva project19. The datasets comprise 880 data columns and 539; 774 data rows in total. This results in 2; 420; 012 single data values. For the unstructured data search results Semedico stores more than 26M MEDLINE citations and approximately 1:5M PubMed Central full texts from the open access subset in its index.
ADOnIS comes with a graphical user interface for the semantic search (Fig. 2). It is divided into three parts: the search box (top), where the user can enter keyword queries (one or more keywords), a section displaying publications (unstructured data) relevant to the query (left) and the list of retrieved BExIS 2 datasets (structured data) (right). An exemplary search using the keywords groundwater, concentration of and nitrate is shown in Fig. 2. The search delivers datasets that refer to the entity groundwater or entities that have been observed in the context of groundwater and datasets where the concentration of (characteristic) nitrate (entity) was measured. On the left-hand side, relevant publications are listed.
To demonstrate the e ectiveness of the search functionality of ADOnIS we compared its results to those of the original keyword-based search provided by 19 Currently, a subset of 15 datasets including 146 data attributes has been semantically annotated. BExIS 2 , which is powered by Apache Lucene20 indexing both datasets and its accompanying metadata. As a preliminary evaluation, we've run the system with keyword queries relevant within the AquaDiva project. We varied the query complexity by using one or more keywords. Exemplary results are reported in Table 1. In its current version, ADOnIS returns the union of both, the results returned by the semantic search and the results retrieved by the BExIS 2 standard search. This is to avoid an empty result set in cases where the semantic search does not retrieve any (exactly tting) datasets. As a consequence, ADOnIS can just return additional datasets that have not been found by the original BExIS 2 search.
For a single keyword, ADOnIS and BExIS 2 typically return the same results, since those keywords are often explicitly mentioned either in the datasets itself or in the metadata. However, if we consider more complex queries, ADOnIS delivers relevant results that BExIS 2 does not discover. As a next step, we will extend this preliminary evaluation. In particular, we plan to invite formal feedback from the AquaDiva researchers. This will cover both, an assessment of the relevance of the delivered search results21 as well as an evaluation of the user interface. In addition, we will evaluate how well the search scales with an increasing number of datasets. 5
We introduced ADOnIS, an information system which coherently integrates two important, yet mostly disparate data sources, namely structured data from databases (or spreadsheets), on the one hand, and unstructured data in terms 20 https://lucene.apache.org/ 21 Note that, even if datasets are annotated correctly, the search might deliver results that the user did not expect, since ADOnIS interprets the user's keywords in a certain way (cf. Sect. 3.1) that does not necessarily comply with the searcher's query intend. Such a mismatch would be discovered by a user study with the AquaDiva researchers. of publications, on the other hand. The integration is achieved by providing the underlying background knowledge of the domains involved in terms of adequately tailored ontologies. Once the two basic data sources are semantically linked, entirely novel opportunities for cross-source information retrieval arise. 6
This work has been mostly funded by the Deutsche Forschungsgemeinschaft (DFG) as part of the CRC 1076 AquaDiva.